Max level shown:
1 starting November 28, 2024 at 04:50:12 AM UTC
2 Creating 3 downstairs regions
3 Creating region directory at: /var/tmp/dsc/region
4 Creating dsc directory at: /tmp/dsc
5 Downstairs region 0 created at /var/tmp/dsc/region/8810 in 0.26808262
6 Downstairs region 1 created at /var/tmp/dsc/region/8820 in 0.020281237
7 Downstairs region 2 created at /var/tmp/dsc/region/8830 in 0.02076304
8 Created 3 regions
9 Starting 3 downstairs
10 Using existing output directory "/tmp/dsc"
11 Update our region info with: Some(RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 })
12 start ds: 8810
13 start ds: 8820
14 start ds: 8830
15 start access at:127.0.0.1:9998
16 Starting downstairs at port 8810
17 Starting downstairs at port 8820
18 Starting downstairs at port 8830
19 Make output file at "/tmp/dsc/downstairs-8820.txt"
20 Make output file at "/tmp/dsc/downstairs-8830.txt"
21 Make output file at "/tmp/dsc/downstairs-8810.txt"
22 [8810][0] initial start wait reports Starting
23 [8820][1] initial start wait reports Starting
24 [8830][2] initial start wait reports Starting
25 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1192)
26 [8820][1] initial start wait reports Running
27 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1193)
28 [8830][2] initial start wait reports Running
29 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1194)
30 [8810][0] initial start wait reports Running
31 All downstairs are running
32 Nov 28 04:50:13.123 INFO listening, local_addr: 127.0.0.1:9998
33 Control access at:127.0.0.1:9998
34 Running initial fill
35 Nov 28 04:50:18.217 INFO accepted connection, remote_addr: 127.0.0.1:58618, local_addr: 127.0.0.1:9998
36 Nov 28 04:50:18.220 INFO request completed, latency_us: 436, response_code: 200, uri: /regioninfo, method: GET, req_id: 89c45705-9524-4d9d-bfb8-e6f8583fa46e, remote_addr: 127.0.0.1:58618, local_addr: 127.0.0.1:9998
37 Nov 28 04:50:18.222 INFO request completed, latency_us: 102, response_code: 200, uri: /regioncount, method: GET, req_id: 0c676027-f7b5-4dde-9b16-5fb73647989b, remote_addr: 127.0.0.1:58618, local_addr: 127.0.0.1:9998
382024-11-28T04:50:18.222ZINFOcrucible: Use this region info from dsc: RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 } task = crutest
392024-11-28T04:50:18.222ZINFOcrucible: dsc has 3 regions. This means 1 sub_volumes task = crutest
40 Nov 28 04:50:18.223 INFO request completed, latency_us: 170, response_code: 200, uri: /port/cid/0, method: GET, req_id: 29374670-12bd-4527-ac31-2d8d8a241574, remote_addr: 127.0.0.1:58618, local_addr: 127.0.0.1:9998
41 Nov 28 04:50:18.223 INFO request completed, latency_us: 92, response_code: 200, uri: /port/cid/1, method: GET, req_id: 25918df2-f0a3-4271-a108-e2b29f8c5b94, remote_addr: 127.0.0.1:58618, local_addr: 127.0.0.1:9998
42 Nov 28 04:50:18.224 INFO request completed, latency_us: 91, response_code: 200, uri: /port/cid/2, method: GET, req_id: a13f8f8b-9ea9-4b2f-9212-19fd61864e1d, remote_addr: 127.0.0.1:58618, local_addr: 127.0.0.1:9998
432024-11-28T04:50:18.224ZINFOcrucible: SV 0 has targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830] task = crutest
442024-11-28T04:50:18.225ZINFOcrucible: Upstairs starts
452024-11-28T04:50:18.226ZINFOcrucible: Crucible Version: BuildInfo { version: "0.0.1", git_sha: "ccd99baabd70fa07cf893ca85ae393437a35a59a", git_commit_timestamp: "2024-11-28T04:33:57.000000000Z", git_branch: "renovate/opentelemetry-0.x", rustc_semver: "1.82.0", rustc_channel: "stable", rustc_host_triple: "x86_64-unknown-illumos", rustc_commit_sha: "f6e511eec7342f59a25f7c0534f1dbea00d01b14", cargo_triple: "x86_64-unknown-illumos", debug: true, opt_level: 0, }
462024-11-28T04:50:18.226ZINFOcrucible: Upstairs <-> Downstairs Message Version: 12
472024-11-28T04:50:18.226ZINFOcrucible: Using region definition RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 00000000-0000-0000-0000-000000000000, encrypted: false, database_read_version: 1, database_write_version: 1 }
482024-11-28T04:50:18.227ZINFOcrucible: Crucible 6e5f89f7-8ba3-4745-a027-8238620bce6c has session id: 4f594b6d-1325-4ca3-83ed-962688005b27 session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
492024-11-28T04:50:18.227ZINFOcrucible: Upstairs opts: Upstairs UUID: 6e5f89f7-8ba3-4745-a027-8238620bce6c, Targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830], lossy: false, flush_timeout: None, key populated: false, cert_pem populated: false, key_pem populated: false, root_cert_pem populated: false, Control: None, read_only: false session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
502024-11-28T04:50:18.227ZINFOcrucible: Crucible stats registered with UUID: 6e5f89f7-8ba3-4745-a027-8238620bce6c session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
512024-11-28T04:50:18.262ZINFOcrucible: Downstairs in volume = 3 task = crutest
522024-11-28T04:50:18.262ZINFOcrucible: The guest has requested activation with gen:1
532024-11-28T04:50:18.262ZINFOcrucible: 6e5f89f7-8ba3-4745-a027-8238620bce6c active request set session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
542024-11-28T04:50:18.262ZINFOcrucible: sending connect oneshot to client = downstairs client = 0 session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
552024-11-28T04:50:18.263ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 0 session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
562024-11-28T04:50:18.263ZINFOcrucible: sending connect oneshot to client = downstairs client = 1 session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
572024-11-28T04:50:18.263ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 1 session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
582024-11-28T04:50:18.263ZINFOcrucible: sending connect oneshot to client = downstairs client = 2 session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
592024-11-28T04:50:18.263ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 2 session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
602024-11-28T04:50:18.263ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
612024-11-28T04:50:18.263ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
622024-11-28T04:50:18.263ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
632024-11-28T04:50:18.263ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:44279) = io task client = 2 session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
642024-11-28T04:50:18.263ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:57655) = io task client = 1 session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
652024-11-28T04:50:18.263ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:46086) = io task client = 0 session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
662024-11-28T04:50:18.265ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 1 session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
672024-11-28T04:50:18.265ZINFOcrucible: [1] Transition from New to WaitActive = downstairs client = 1 session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
682024-11-28T04:50:18.265ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 2 session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
692024-11-28T04:50:18.265ZINFOcrucible: [2] Transition from New to WaitActive = downstairs client = 2 session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
702024-11-28T04:50:18.265ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 0 session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
712024-11-28T04:50:18.265ZINFOcrucible: [0] Transition from New to WaitActive = downstairs client = 0 session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
722024-11-28T04:50:18.266ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
732024-11-28T04:50:18.266ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
742024-11-28T04:50:18.266ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
752024-11-28T04:50:18.267ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 1 session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
762024-11-28T04:50:18.267ZINFOcrucible: [1] Transition from WaitActive to WaitQuorum = downstairs client = 1 session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
772024-11-28T04:50:18.267ZINFOcrucible: Waiting for more clients to be ready session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
782024-11-28T04:50:18.267ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 2 session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
792024-11-28T04:50:18.267ZINFOcrucible: [2] Transition from WaitActive to WaitQuorum = downstairs client = 2 session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
802024-11-28T04:50:18.267ZINFOcrucible: Waiting for more clients to be ready session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
812024-11-28T04:50:18.267ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 0 session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
822024-11-28T04:50:18.267ZINFOcrucible: [0] Transition from WaitActive to WaitQuorum = downstairs client = 0 session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
832024-11-28T04:50:18.268ZINFOcrucible: [0]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
842024-11-28T04:50:18.268ZINFOcrucible: [0]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
852024-11-28T04:50:18.268ZINFOcrucible: [0]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
862024-11-28T04:50:18.268ZINFOcrucible: [1]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
872024-11-28T04:50:18.268ZINFOcrucible: [1]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
882024-11-28T04:50:18.268ZINFOcrucible: [1]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
892024-11-28T04:50:18.268ZINFOcrucible: [2]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
902024-11-28T04:50:18.268ZINFOcrucible: [2]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
912024-11-28T04:50:18.268ZINFOcrucible: [2]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
922024-11-28T04:50:18.268ZINFOcrucible: Max found gen is 1 = downstairs session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
932024-11-28T04:50:18.268ZINFOcrucible: Generation requested: 1 >= found:1 = downstairs session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
942024-11-28T04:50:18.268ZINFOcrucible: Next flush: 1 = downstairs session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
952024-11-28T04:50:18.268ZINFOcrucible: All extents match = downstairs session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
962024-11-28T04:50:18.268ZINFOcrucible: No downstairs reconciliation required session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
972024-11-28T04:50:18.268ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 0 session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
982024-11-28T04:50:18.268ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 1 session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
992024-11-28T04:50:18.269ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 2 session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
1002024-11-28T04:50:18.269ZINFOcrucible: All required reconciliation work is completed session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
1012024-11-28T04:50:18.269ZINFOcrucible: Set Downstairs and Upstairs active after reconciliation session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
1022024-11-28T04:50:18.269ZINFOcrucible: 6e5f89f7-8ba3-4745-a027-8238620bce6c is now active with session: 4f594b6d-1325-4ca3-83ed-962688005b27 session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
1032024-11-28T04:50:18.269ZINFOcrucible: Set Active after no reconciliation session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
1042024-11-28T04:50:18.269ZINFOcrucible: The guest has finished waiting for activation with:1
105 Activate successful
106 Wait for a query_work_queue command to finish before sending IO
107 Disk: sv:1 bs:4096 ts:20480000 tb:5000 max_io:256 or 1048576
108 Fill test
109 Read and Verify all blocks (0..5000 range:false)
110 Wrote out file "/tmp/test_replay_verify.log"
111 CLIENT: Tests done. All submitted work has been ACK'd
112 ----------------------------------------------------------------
113 Crucible gen:1 GIO:true work queues: Upstairs:0 downstairs:20
114 States: Active Active Active
115 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
116 1021 Acked Read 256 Done Done Done false
117 1022 Acked Read 256 Done Done Done false
118 1023 Acked Read 256 Done Done Done false
119 1024 Acked Read 256 Done Done Done false
120 1025 Acked Read 256 Done Done Done false
121 1026 Acked Read 256 Done Done Done false
122 1027 Acked Read 256 Done Done Done false
123 1028 Acked Read 256 Done Done Done false
124 1029 Acked Read 256 Done Done Done false
125 1030 Acked Read 256 Done Done Done false
126 1031 Acked Read 256 Done Done Done false
127 1032 Acked Read 256 Done Done Done false
128 1033 Acked Read 256 Done Done Done false
129 1034 Acked Read 256 Done Done Done false
130 1035 Acked Read 256 Done Done Done false
131 1036 Acked Read 256 Done Done Done false
132 1037 Acked Read 256 Done Done Done false
133 1038 Acked Read 256 Done Done Done false
134 1039 Acked Read 136 Done Done Done false
135 1040 Acked Read 256 Done Done Done false
136 STATES DS:0 DS:1 DS:2 TOTAL
137 Sent 0 0 0 0
138 Done 20 20 20 60
139 Skipped 0 0 0 0
140 Error 0 0 0 0
141 Last Flush: 1020 1020 1020
142 Downstairs last five completed: 1020 1019 1018 1017 1016
143 Upstairs last five completed: 1040 1039 1038 1037 1036
144 CLIENT: Up:0 ds:20 act:3
145 ----------------------------------------------------------------
146 Crucible gen:1 GIO:true work queues: Upstairs:0 downstairs:0
147 Downstairs last five completed: 1041 1040 1039 1038 1037
148 Upstairs last five completed: 1041 1040 1039 1038 1037
149 CLIENT: Up:0 ds:0 act:3
150 CLIENT: All crucible jobs finished, exiting program
1512024-11-28T04:50:22.862ZWARNcrucible: Guest handle has been dropped
1522024-11-28T04:50:22.862ZWARNcrucible: client request queue closed unexpectedly; is the program exiting? = io task client = 0 session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
1532024-11-28T04:50:22.862ZWARNcrucible: client task is sending Done(QueueClosed) = io task client = 0 session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
1542024-11-28T04:50:22.862ZWARNcrucible: client task could not reply to main task; shutting down? = io task client = 0 session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
1552024-11-28T04:50:22.862ZINFOcrucible: client task is exiting = io task client = 0 session_id = 4f594b6d-1325-4ca3-83ed-962688005b27
156 Replay loop starts now November 28, 2024 at 04:50:22 AM UTC
157 Nov 28 04:50:22.935 INFO accepted connection, remote_addr: 127.0.0.1:54489, local_addr: 127.0.0.1:9998
158 Nov 28 04:50:22.935 INFO request completed, latency_us: 160, response_code: 200, uri: /regioninfo, method: GET, req_id: c9b93146-9684-44e9-982e-97b49bef0e9a, remote_addr: 127.0.0.1:54489, local_addr: 127.0.0.1:9998
1592024-11-28T04:50:22.936ZINFOcrucible: Use this region info from dsc: RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 } task = crutest
160 Nov 28 04:50:22.936 INFO request completed, latency_us: 110, response_code: 200, uri: /regioncount, method: GET, req_id: 057bb146-c802-4615-9f98-9d6744e21d28, remote_addr: 127.0.0.1:54489, local_addr: 127.0.0.1:9998
1612024-11-28T04:50:22.936ZINFOcrucible: dsc has 3 regions. This means 1 sub_volumes task = crutest
162 Nov 28 04:50:22.937 INFO request completed, latency_us: 120, response_code: 200, uri: /port/cid/0, method: GET, req_id: a0755133-9ca0-4d85-968d-c70627ead7c6, remote_addr: 127.0.0.1:54489, local_addr: 127.0.0.1:9998
163 Nov 28 04:50:22.937 INFO request completed, latency_us: 94, response_code: 200, uri: /port/cid/1, method: GET, req_id: 86d7d15d-e7f5-4650-8e4f-7d9337834863, remote_addr: 127.0.0.1:54489, local_addr: 127.0.0.1:9998
164 Nov 28 04:50:22.938 INFO request completed, latency_us: 94, response_code: 200, uri: /port/cid/2, method: GET, req_id: c04ade2e-adc8-42e4-890c-5a689c39ba83, remote_addr: 127.0.0.1:54489, local_addr: 127.0.0.1:9998
1652024-11-28T04:50:22.938ZINFOcrucible: SV 0 has targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830] task = crutest
1662024-11-28T04:50:22.938ZINFOcrucible: Upstairs starts
1672024-11-28T04:50:22.938ZINFOcrucible: Crucible Version: BuildInfo { version: "0.0.1", git_sha: "ccd99baabd70fa07cf893ca85ae393437a35a59a", git_commit_timestamp: "2024-11-28T04:33:57.000000000Z", git_branch: "renovate/opentelemetry-0.x", rustc_semver: "1.82.0", rustc_channel: "stable", rustc_host_triple: "x86_64-unknown-illumos", rustc_commit_sha: "f6e511eec7342f59a25f7c0534f1dbea00d01b14", cargo_triple: "x86_64-unknown-illumos", debug: true, opt_level: 0, }
1682024-11-28T04:50:22.938ZINFOcrucible: Upstairs <-> Downstairs Message Version: 12
1692024-11-28T04:50:22.938ZINFOcrucible: Using region definition RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 00000000-0000-0000-0000-000000000000, encrypted: false, database_read_version: 1, database_write_version: 1 }
1702024-11-28T04:50:22.939ZINFOcrucible: Crucible a27578cd-91bf-41ee-a091-ff1d3b239089 has session id: 4a2415b1-6594-436f-82e5-b8cf9a74eeaf session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
1712024-11-28T04:50:22.939ZINFOcrucible: Upstairs opts: Upstairs UUID: a27578cd-91bf-41ee-a091-ff1d3b239089, Targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830], lossy: false, flush_timeout: None, key populated: false, cert_pem populated: false, key_pem populated: false, root_cert_pem populated: false, Control: None, read_only: false session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
1722024-11-28T04:50:22.939ZINFOcrucible: Crucible stats registered with UUID: a27578cd-91bf-41ee-a091-ff1d3b239089 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
1732024-11-28T04:50:22.972ZINFOcrucible: Downstairs in volume = 3 task = crutest
1742024-11-28T04:50:22.972ZINFOcrucible: The guest has requested activation with gen:2
1752024-11-28T04:50:22.972ZINFOcrucible: a27578cd-91bf-41ee-a091-ff1d3b239089 active request set session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
1762024-11-28T04:50:22.972ZINFOcrucible: sending connect oneshot to client = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
1772024-11-28T04:50:22.973ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
1782024-11-28T04:50:22.973ZINFOcrucible: sending connect oneshot to client = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
1792024-11-28T04:50:22.973ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
1802024-11-28T04:50:22.973ZINFOcrucible: sending connect oneshot to client = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
1812024-11-28T04:50:22.973ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
1822024-11-28T04:50:22.973ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
1832024-11-28T04:50:22.973ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
1842024-11-28T04:50:22.973ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
1852024-11-28T04:50:22.973ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:50310) = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
1862024-11-28T04:50:22.973ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:50652) = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
1872024-11-28T04:50:22.973ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:34948) = io task client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
1882024-11-28T04:50:22.973ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
1892024-11-28T04:50:22.973ZINFOcrucible: [1] Transition from New to WaitActive = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
1902024-11-28T04:50:22.973ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
1912024-11-28T04:50:22.973ZINFOcrucible: [2] Transition from New to WaitActive = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
1922024-11-28T04:50:22.974ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
1932024-11-28T04:50:22.974ZINFOcrucible: [0] Transition from New to WaitActive = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
1942024-11-28T04:50:22.974ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
1952024-11-28T04:50:22.974ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
1962024-11-28T04:50:22.974ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
1972024-11-28T04:50:22.975ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
1982024-11-28T04:50:22.975ZINFOcrucible: [2] Transition from WaitActive to WaitQuorum = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
1992024-11-28T04:50:22.975ZINFOcrucible: Waiting for more clients to be ready session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
2002024-11-28T04:50:22.975ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
2012024-11-28T04:50:22.975ZINFOcrucible: [1] Transition from WaitActive to WaitQuorum = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
2022024-11-28T04:50:22.975ZINFOcrucible: Waiting for more clients to be ready session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
2032024-11-28T04:50:22.975ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
2042024-11-28T04:50:22.975ZINFOcrucible: [0] Transition from WaitActive to WaitQuorum = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
2052024-11-28T04:50:22.975ZINFOcrucible: [0]R flush_numbers[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
2062024-11-28T04:50:22.975ZINFOcrucible: [0]R generation[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
2072024-11-28T04:50:22.975ZINFOcrucible: [0]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
2082024-11-28T04:50:22.975ZINFOcrucible: [1]R flush_numbers[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
2092024-11-28T04:50:22.975ZINFOcrucible: [1]R generation[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
2102024-11-28T04:50:22.975ZINFOcrucible: [1]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
2112024-11-28T04:50:22.975ZINFOcrucible: [2]R flush_numbers[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
2122024-11-28T04:50:22.975ZINFOcrucible: [2]R generation[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
2132024-11-28T04:50:22.975ZINFOcrucible: [2]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
2142024-11-28T04:50:22.975ZINFOcrucible: Max found gen is 2 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
215 {"msg":"Generation requested: 2 >= found:2","v":0,"name":"crucible","level":30,"time":"2024-11-28T04:50:22.97572845Z","hostname":"w-01JDRHKEEFJ1228TFPSBEGXWHY","pid":1202,"":"Activate successful
216 Wait for a query_work_queue command to finish before sending IO
217 downstairs","":"downstairs","session_id":"4a2415b1-6594-436f-82e5-b8cf9a74eeaf"}
2182024-11-28T04:50:22.975ZINFOcrucible: Next flush: 2 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
2192024-11-28T04:50:22.975ZINFOcrucible: All extents match = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
2202024-11-28T04:50:22.975ZINFOcrucible: No downstairs reconciliation required session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
2212024-11-28T04:50:22.975ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
2222024-11-28T04:50:22.975ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
2232024-11-28T04:50:22.975ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
2242024-11-28T04:50:22.975ZINFOcrucible: All required reconciliation work is completed session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
2252024-11-28T04:50:22.975ZINFOcrucible: Set Downstairs and Upstairs active after reconciliation session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
2262024-11-28T04:50:22.975ZINFOcrucible: a27578cd-91bf-41ee-a091-ff1d3b239089 is now active with session: 4a2415b1-6594-436f-82e5-b8cf9a74eeaf session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
2272024-11-28T04:50:22.976ZINFOcrucible: Set Active after no reconciliation session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
2282024-11-28T04:50:22.976ZINFOcrucible: The guest has finished waiting for activation with:2
229 Disk: sv:1 bs:4096 ts:20480000 tb:5000 max_io:256 or 1048576
230 Loading write count information from file "/tmp/test_replay_verify.log"
231 Nov 28 04:50:23.010 INFO accepted connection, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
232 Main task has work to do, go find it
233 got dsc Stop(2)
234 stop 2
235 Nov 28 04:50:23.011 INFO request completed, latency_us: 173, response_code: 204, uri: /stop/cid/2, method: POST, req_id: c31aaabc-6d08-42b1-b6e0-d3f071cf5dab, remote_addr: 127.[8830] Got stop action so:false kr:false
236 0.0.1:60418, local_addr: 127.0.0.1:9998
2372024-11-28T04:50:23.011ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
2382024-11-28T04:50:23.011ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
2392024-11-28T04:50:23.011ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
2402024-11-28T04:50:23.012ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
241 Nov 28 04:50:23.012 INFO request completed, latency_us: 132, response_code: 200, uri: /state/cid/2{"msg":"[2] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-28T04:50:23.012081516Z","hostname":"w-01JDRHKEEFJ1228TFPSBEGXWHY","pid":1202,"client":"2","":"downstairs","session_id":"4a2415b1-6594-436f-82e5-b8cf9a74eeaf"}
242 , method: GET, req_id: ba6c0e1c-1fa3-483c-9647-76b7e9d232da, remote_addr: 127.0.0.1:60418{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2024-11-28T04:50:23.01212799Z","hostname":"w-01JDRHKEEFJ1228TFPSBEGXWHY","pid":1202,"":"io task","client":"2","":"downstairs","session_id":"4a2415b1-6594-436f-82e5-b8cf9a74eeaf"}
243 , local_addr: 127.0.0.1:9998
244 [8830] Exited with: ExitStatus(unix_wait_status(9))
245 [8830][2] reports Exit
246 Nov 28 04:50:27.014 INFO request completed, latency_us: 225, response_code: 200, uri: /state/cid/2, method: GET, req_id: f07d8179-4f1d-4f92-afb3-4e09a2cd06db, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
247 001/300 Read block 1105 len 24576
248 002/300 Read block 207 len 24576
249 003/300 Write block 2378 len 24576 data: 2 2 2 2 2 2
250 004/300 Write block 1786 len 4096 data: 2
251 005/300 Write block 2063 len 32768 data: 2 2 2 2 2 2 2 2
252 006/300 Write block 4149 len 36864 data: 2 2 2 2 2 2 2 2 2
253 007/300 Read block 2525 len 28672
254 008/300 Read block 547 len 32768
255 009/300 Read block 708 len 28672
256 010/300 Read block 1096 len 4096
257 011/300 Read block 4841 len 16384
258 012/300 Write block 3158 len 8192 data: 2 2
259 013/300 Write block 531 len 40960 data: 2 2 2 2 2 2 2 2 2 2
260 014/300 Write block 3260 len 36864 data: 2 2 2 2 2 2 2 2 2
261 015/300 Read block 299 len 16384
262 016/300 Read block 4427 len 12288
263 017/300 Write block 1794 len 12288 data: 2 2 2
264 018/300 Read block 2823 len 36864
265 019/300 Read block 1368 len 24576
266 020/300 Write block 3649 len 24576 data: 2 2 2 2 2 2
267 021/300 Read block 4662 len 12288
268 022/300 Write block 635 len 20480 data: 2 2 2 2 2
269 023/300 Read block 1176 len 8192
270 024/300 Read block 2846 len 24576
271 025/300 Write block 2617 len 36864 data: 2 2 2 2 2 2 2 2 2
272 026/300 Read block 2194 len 12288
273 027/300 Write block 2580 len 8192 data: 2 2
274 028/300 Write block 1382 len 16384 data: 2 2 2 2
275 029/300 Read block 2603 len 28672
276 030/300 Read block 700 len 32768
277 031/300 Write block 1969 len 28672 data: 2 2 2 2 2 2 2
278 032/300 Read block 4953 len 40960
279 033/300 Write block 4954 len 24576 data: 2 2 2 2 2 2
280 034/300 Write block 3490 len 24576 data: 2 2 2 2 2 2
281 035/300 Flush
282 036/300 Write block 3159 len 4096 data: 3
283 037/300 Flush
284 038/300 Write block 2491 len 12288 data: 2 2 2
285 039/300 Read block 3797 len 16384
286 040/300 Write block 1964 len 32768 data: 2 2 2 2 2 3 3 3
287 041/300 Write block 812 len 20480 data: 2 2 2 2 2
288 042/300 Write block 2065 len 32768 data: 3 3 3 3 3 3 2 2
289 043/300 Read block 4050 len 40960
290 044/300 Write block 858 len 8192 data: 2 2
291 045/300 Write block 4054 len 28672 data: 2 2 2 2 2 2 2
292 046/300 Read block 4899 len 20480
293 047/300 Write block 3527 len 36864 data: 2 2 2 2 2 2 2 2 2
294 048/300 Read block 3342 len 24576
295 049/300 Read block 3319 len 32768
296 050/300 Flush
297 051/300 Read block 3860 len 4096
298 052/300 Write block 828 len 40960 data: 2 2 2 2 2 2 2 2 2 2
299 053/300 Read block 2817 len 20480
300 054/300 Flush
301 055/300 Write block 3236 len 40960 data: 2 2 2 2 2 2 2 2 2 2
302 056/300 Write block 4860 len 20480 data: 2 2 2 2 2
303 057/300 Read block 1786 len 36864
304 058/300 Write block 656 len 28672 data: 2 2 2 2 2 2 2
305 059/300 Read block 1190 len 24576
306 060/300 Read block 2949 len 28672
307 061/300 Write block 2946 len 28672 data: 2 2 2 2 2 2 2
308 062/300 Write block 4983 len 36864 data: 2 2 2 2 2 2 2 2 2
309 063/300 Write block 4094 len 24576 data: 2 2 2 2 2 2
310 064/300 Write block 1785 len 32768 data: 2 3 2 2 2 2 2 2
311 065/300 Read block 932 len 4096
312 066/300 Read block 110 len 16384
313 067/300 Write block 3580 len 36864 data: 2 2 2 2 2 2 2 2 2
314 068/300 Write block 1698 len 40960 data: 2 2 2 2 2 2 2 2 2 2
315 069/300 Read block 2607 len 32768
316 070/300 Write block 3150 len 32768 data: 2 2 2 2 2 2 2 2
317 071/300 Read block 2613 len 16384
318 072/300 Read block 4490 len 8192
319 073/300 Read block 2675 len 8192
320 074/300 Write block 1156 len 8192 data: 2 2
321 075/300 Read block 3612 len 24576
322 076/300 Read block 4902 len 8192
323 077/300 Read block 2403 len 20480
324 078/300 Read block 1583 len 4096
325 079/300 Read block 3651 len 4096
326 080/300 Read block 2928 len 20480
327 081/300 Read block 1254 len 4096
328 082/300 Write block 1936 len 4096 data: 2
329 083/300 Read block 4052 len 24576
330 084/300 Read block 4486 len 16384
331 085/300 Write block 3058 len 32768 data: 2 2 2 2 2 2 2 2
332 086/300 Read block 2897 len 8192
333 087/300 Read block 4171 len 40960
334 088/300 Read block 1940 len 4096
335 089/300 Write block 3073 len 20480 data: 2 2 2 2 2
336 090/300 Write block 3812 len 32768 data: 2 2 2 2 2 2 2 2
337 091/300 Write block 1849 len 40960 data: 2 2 2 2 2 2 2 2 2 2
338 092/300 Read block 387 len 20480
339 093/300 Read block 96 len 24576
340 094/300 Read block 2712 len 28672
341 095/300 Write block 3925 len 36864 data: 2 2 2 2 2 2 2 2 2
342 096/300 Read block 3869 len 24576
343 097/300 Write block 2696 len 12288 data: 2 2 2
344 098/300 Read block 191 len 16384
345 099/300 Read block 106 len 4096
346 100/300 Write block 438 len 12288 data: 2 2 2
347 101/300 Write block 4237 len 4096 data: 2
348 102/300 Write block 2746 len 32768 data: 2 2 2 2 2 2 2 2
349 103/300 Write block 4529 len 16384 data: 2 2 2 2
350 104/300 Read block 977 len 8192
351 105/300 Write block 526 len 8192 data: 2 2
352 106/300 Write block 4152 len 20480 data: 3 3 3 3 3
353 107/300 Read block 2824 len 4096
354 108/300 Read block 4395 len 28672
355 109/300 Read block 759 len 28672
356 110/300 Flush
357 111/300 Read block 1835 len 24576
358 112/300 Read block 826 len 28672
359 113/300 Read block 1601 len 8192
360 114/300 Write block 2261 len 8192 data: 2 2
361 115/300 Read block 2079 len 36864
362 116/300 Write block 2557 len 40960 data: 2 2 2 2 2 2 2 2 2 2
363 117/300 Read block 2407 len 36864
364 118/300 Read block 1253 len 8192
365 119/300 Read block 2349 len 28672
366 120/300 Read block 3406 len 24576
367 121/300 Read block 179 len 40960
368 122/300 Write block 1575 len 28672 data: 2 2 2 2 2 2 2
369 123/300 Read block 4787 len 12288
370 124/300 Read block 3649 len 12288
371 125/300 Write block 705 len 12288 data: 2 2 2
372 126/300 Flush
373 127/300 Read block 1776 len 40960
374 128/300 Read block 4469 len 20480
375 129/300 Flush
376 130/300 Write block 2525 len 32768 data: 2 2 2 2 2 2 2 2
377 131/300 Write block 3426 len 36864 data: 2 2 2 2 2 2 2 2 2
378 132/300 Read block 4804 len 16384
379 133/300 Read block 2303 len 40960
380 134/300 Read block 995 len 28672
381 135/300 Flush
382 136/300 Read block 4082 len 8192
383 137/300 Read block 3615 len 20480
384 138/300 Write block 1175 len 40960 data: 2 2 2 2 2 2 2 2 2 2
385 139/300 Flush
386 140/300 Write block 4299 len 4096 data: 2
387 141/300 Read block 4299 len 16384
388 142/300 Read block 167 len 32768
389 143/300 Flush
390 144/300 Flush
391 145/300 Read block 4576 len 4096
392 146/300 Write block 965 len 40960 data: 2 2 2 2 2 2 2 2 2 2
393 147/300 Read block 3149 len 8192
394 148/300 Read block 3154 len 24576
395 149/300 Write block 4516 len 40960 data: 2 2 2 2 2 2 2 2 2 2
396 150/300 Write block 1047 len 28672 data: 2 2 2 2 2 2 2
397 151/300 Write block 4470 len 20480 data: 2 2 2 2 2
398 152/300 Write block 4582 len 12288 data: 2 2 2
399 153/300 Write block 3505 len 24576 data: 2 2 2 2 2 2
400 154/300 Read block 1810 len 24576
401 155/300 Write block 2942 len 8192 data: 2 2
402 156/300 Write block 4423 len 12288 data: 2 2 2
403 157/300 Write block 1529 len 4096 data: 2
404 158/300 Read block 987 len 8192
405 159/300 Write block 4914 len 24576 data: 2 2 2 2 2 2
406 160/300 Write block 161 len 36864 data: 2 2 2 2 2 2 2 2 2
407 161/300 Write block 4891 len 24576 data: 2 2 2 2 2 2
408 162/300 Read block 2744 len 16384
409 163/300 Write block 3040 len 24576 data: 2 2 2 2 2 2
410 164/300 Read block 3556 len 24576
411 165/300 Read block 4164 len 24576
412 166/300 Read block 4768 len 32768
413 167/300 Write block 2919 len 32768 data: 2 2 2 2 2 2 2 2
414 168/300 Write block 457 len 4096 data: 2
415 169/300 Write block 2157 len 28672 data: 2 2 2 2 2 2 2
416 170/300 Read block 297 len 4096
417 171/300 Read block 3031 len 8192
418 172/300 Read block 3063 len 20480
419 173/300 Write block 2314 len 32768 data: 2 2 2 2 2 2 2 2
420 174/300 Write block 858 len 16384 data: 3 3 2 2
421 175/300 Read block 3543 len 24576
422 176/300 Write block 2207 len 32768 data: 2 2 2 2 2 2 2 2
423 177/300 Read block 849 len 28672
424 178/300 Read block 4801 len 20480
425 179/300 Flush
426 180/300 Read block 3896 len 40960
427 181/300 Flush
428 182/300 Write block 226 len 8192 data: 2 2
429 183/300 Read block 819 len 20480
430 184/300 Read block 3763 len 28672
431 185/300 Write block 1982 len 20480 data: 2 2 2 2 2
432 186/300 Write block 1281 len 20480 data: 2 2 2 2 2
433 187/300 Write block 563 len 16384 data: 2 2 2 2
434 188/300 Write block 876 len 24576 data: 2 2 2 2 2 2
435 189/300 Flush
436 190/300 Write block 626 len 4096 data: 2
437 191/300 Write block 2706 len 20480 data: 2 2 2 2 2
438 192/300 Write block 2395 len 20480 data: 2 2 2 2 2
439 193/300 Read block 4829 len 16384
440 194/300 Read block 2984 len 40960
441 195/300 Write block 2272 len 4096 data: 2
442 196/300 Flush
443 197/300 Flush
444 198/300 Write block 3879 len 20480 data: 2 2 2 2 2
445 199/300 Read block 4097 len 20480
446 200/300 Read block 1004 len 8192
447 201/300 Read block 4923 len 28672
448 202/300 Write block 3078 len 20480 data: 2 2 2 2 2
449 203/300 Flush
450 204/300 Read block 2321 len 20480
451 205/300 Write block 366 len 20480 data: 2 2 2 2 2
452 206/300 Read block 2897 len 4096
453 207/300 Read block 4206 len 20480
454 208/300 Write block 99 len 20480 data: 2 2 2 2 2
455 209/300 Write block 1147 len 20480 data: 2 2 2 2 2
456 210/300 Read block 4823 len 32768
457 211/300 Read block 3093 len 36864
458 212/300 Read block 3464 len 8192
459 213/300 Read block 4101 len 36864
460 214/300 Write block 4090 len 40960 data: 2 2 2 2 3 3 3 3 3 3
461 215/300 Read block 2787 len 16384
462 216/300 Read block 2572 len 12288
463 217/300 Read block 2545 len 4096
464 218/300 Write block 530 len 28672 data: 2 3 3 3 3 3 3
465 219/300 Read block 3978 len 24576
466 220/300 Write block 4901 len 12288 data: 2 2 2
467 221/300 Write block 135 len 40960 data: 2 2 2 2 2 2 2 2 2 2
468 222/300 Write block 171 len 16384 data: 2 2 2 2
469 223/300 Read block 2470 len 40960
470 224/300 Read block 4889 len 4096
471 225/300 Write block 553 len 20480 data: 2 2 2 2 2
472 226/300 Flush
473 227/300 Write block 1240 len 32768 data: 2 2 2 2 2 2 2 2
474 228/300 Read block 3661 len 24576
475 229/300 Flush
476 230/300 Write block 4958 len 4096 data: 3
477 231/300 Read block 53 len 24576
478 232/300 Read block 864 len 12288
479 233/300 Read block 156 len 20480
480 234/300 Read block 31 len 4096
481 235/300 Write block 1709 len 32768 data: 2 2 2 2 2 2 2 2
482 236/300 Read block 3578 len 16384
483 237/300 Write block 3050 len 24576 data: 2 2 2 2 2 2
484 238/300 Read block 68 len 8192
485 239/300 Write block 287 len 32768 data: 2 2 2 2 2 2 2 2
486 240/300 Read block 4837 len 8192
487 241/300 Read block 688 len 28672
488 242/300 Flush
489 243/300 Write block 2733 len 4096 data: 2
490 244/300 Read block 2860 len 20480
491 245/300 Write block 2517 len 8192 data: 2 2
492 246/300 Read block 2309 len 20480
493 247/300 Read block 1431 len 4096
494 248/300 Read block 2381 len 28672
495 249/300 Write block 4007 len 16384 data: 2 2 2 2
496 250/300 Read block 2616 len 32768
497 251/300 Write block 2784 len 28672 data: 2 2 2 2 2 2 2
498 252/300 Write block 1405 len 8192 data: 2 2
499 253/300 Read block 2840 len 20480
500 254/300 Read block 4933 len 24576
501 255/300 Flush
502 256/300 Write block 3053 len 36864 data: 3 3 3 2 2 3 3 3 3
503 257/300 Read block 1348 len 20480
504 258/300 Flush
505 259/300 Read block 1184 len 4096
506 260/300 Write block 2284 len 4096 data: 2
507 261/300 Flush
508 262/300 Write block 2105 len 24576 data: 2 2 2 2 2 2
509 263/300 Write block 4344 len 24576 data: 2 2 2 2 2 2
510 264/300 Write block 501 len 28672 data: 2 2 2 2 2 2 2
511 265/300 Write block 4801 len 36864 data: 2 2 2 2 2 2 2 2 2
512 266/300 Flush
513 267/300 Read block 4374 len 4096
514 268/300 Write block 227 len 36864 data: 3 2 2 2 2 2 2 2 2
515 269/300 Read block 4898 len 28672
516 270/300 Read block 1789 len 36864
517 271/300 Read block 1219 len 20480
518 272/300 Write block 47 len 12288 data: 2 2 2
519 273/300 Write block 3033 len 40960 data: 2 2 2 2 2 2 2 3 3 3
520 274/300 Read block 4580 len 28672
521 275/300 Write block 4469 len 20480 data: 2 3 3 3 3
522 276/300 Read block 973 len 24576
523 277/300 Write block 4345 len 24576 data: 3 3 3 3 3 2
524 278/300 Write block 3502 len 32768 data: 2 2 2 3 3 3 3 3
525 279/300 Flush
526 280/300 Read block 4096 len 40960
527 281/300 Write block 1668 len 20480 data: 2 2 2 2 2
528 282/300 Read block 245 len 4096
529 283/300 Read block 3959 len 36864
530 284/300 Read block 2585 len 20480
531 285/300 Flush
532 286/300 Read block 4723 len 4096
533 287/300 Read block 3018 len 36864
534 288/300 Write block 3986 len 4096 data: 2
535 289/300 Write block 1044 len 40960 data: 2 2 2 3 3 3 3 3 3 3
536 290/300 Write block 371 len 40960 data: 2 2 2 2 2 2 2 2 2 2
537 291/300 Write block 535 len 40960 data: 4 4 3 3 3 3 2 2 2 2
538 292/300 Read block 2456 len 28672
539 293/300 Read block 4 len 24576
540 294/300 Read block 4696 len 4096
541 295/300 Read block 4900 len 4096
542 296/300 Write block 825 len 24576 data: 2 2 2 3 3 3
543 297/300 Write block 46 len 16384 data: 2 3 3 3
544 298/300 Read block 4011 len 8192
545 299/300 Read block 3654 len 36864
546 300/300 Read block 3163 len 8192
547 301/300 Read block 1270 len 28672
548 Main task has work to do, go find it
549 got dsc Start(2)
550 start 2
551 Nov 28 04:50:27.245 INFO request completed, latency_us: 194, response_code: 204, uri: /start/cid/2, method: POST, req_id: 473b2bf9-d857-414f-a70b-8969d65a3154, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
552 Replay: started 2, returned:Ok(())
553 ----------------------------------------------------------------
554 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
555 States: Active Active Offline
556 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
557 1000 Acked Read 6 Done Done Sent false
558 1001 Acked Read 6 Done Done Sent false
559 1002 Acked Write 6 Done Done Sent false
560 1003 Acked Write 1 Done Done Sent false
561 1004 Acked Write 8 Done Done Sent false
562 1005 Acked Write 9 Done Done Sent false
563 1006 Acked Read 7 Done Done Sent false
564 1007 Acked Read 8 Done Done Sent false
565 1008 Acked Read 7 Done Done Sent false
566 1009 Acked Read 1 Done Done Sent false
567 1010 Acked Read 4 Done Done Sent false
568 1011 Acked Write 2 Done Done Sent false
569 1012 Acked Write 10 Done Done Sent false
570 1013 Acked Write 9 Done Done Sent false
571 1014 Acked Read 4 Done Done Sent false
572 1015 Acked Read 3 Done Done Sent false
573 1016 Acked Write 3 Done Done Sent false
574 1017 Acked Read 9 Done Done Sent false
575 1018 Acked Read 6 Done Done Sent false
576 1019 Acked Write 6 Done Done Sent false
577 1020 Acked Read 3 Done Done Sent false
578 1021 Acked Write 5 Done Done Sent false
579 1022 Acked Read 2 Done Done Sent false
580 1023 Acked Read 6 Done Done Sent false
581 1024 Acked Write 9 Done Done Sent false
582 1025 Acked Read 3 Done Done Sent false
583 1026 Acked Write 2 Done Done Sent false
584 1027 Acked Write 4 Done Done Sent false
585 1028 Acked Read 7 Done Done Sent false
586 1029 Acked Read 8 Done Done Sent false
587 1030 Acked Write 7 Done Done Sent false
588 1031 Acked Read 10 Done Done Sent false
589 1032 Acked Write 6 Done Done Sent false
590 1033 Acked Write 6 Done Done Sent false
591 1034 Acked Flush 0 Done Done Sent false
592 1035 Acked Write 1 Done Done Sent false
593 1036 Acked Flush 0 Done Done Sent false
594 1037 Acked Write 3 Done Done Sent false
595 1038 Acked Read 4 Done Done Sent false
596 1039 Acked Write 8 Done Done Sent false
597 1040 Acked Write 5 Done Done Sent false
598 1041 Acked Write 8 Done Done Sent false
599 1042 Acked Read 10 Done Done Sent false
600 1043 Acked Write 2 Done Done Sent false
601 1044 Acked Write 7 Done Done Sent false
602 1045 Acked Read 5 Done Done Sent false
603 1046 Acked Write 9 Done Done Sent false
604 1047 Acked Read 6 Done Done Sent false
605 1048 Acked Read 8 Done Done Sent false
606 1049 Acked Flush 0 Done Done Sent false
607 1050 Acked Read 1 Done Done Sent false
608 1051 Acked Write 10 Done Done Sent false
609 1052 Acked Read 5 Done Done Sent false
610 1053 Acked Flush 0 Done Done Sent false
611 1054 Acked Write 10 Done Done Sent false
612 1055 Acked Write 5 Done Done Sent false
613 1056 Acked Read 9 Done Done Sent false
614 1057 Acked Write 7 Done Done Sent false
615 1058 Acked Read 6 Done Done Sent false
616 1059 Acked Read 7 Done Done Sent false
617 1060 Acked Write 7 Done Done Sent false
618 1061 Acked Write 9 Done Done Sent false
619 1062 Acked Write 6 Done Done Sent false
620 1063 Acked Write 8 Done Done Sent false
621 1064 Acked Read 1 Done Done Sent false
622 1065 Acked Read 4 Done Done Sent false
623 1066 Acked Write 9 Done Done Sent false
624 1067 Acked Write 10 Done Done Sent false
625 1068 Acked Read 8 Done Done Sent false
626 1069 Acked Write 8 Done Done Sent false
627 1070 Acked Read 4 Done Done Sent false
628 1071 Acked Read 2 Done Done Sent false
629 1072 Acked Read 2 Done Done Sent false
630 1073 Acked Write 2 Done Done Sent false
631 1074 Acked Read 6 Done Done Sent false
632 1075 Acked Read 2 Done Done Sent false
633 1076 Acked Read 5 Done Done Sent false
634 1077 Acked Read 1 Done Done Sent false
635 1078 Acked Read 1 Done Done Sent false
636 1079 Acked Read 5 Done Done Sent false
637 1080 Acked Read 1 Done Done Sent false
638 1081 Acked Write 1 Done Done Sent false
639 1082 Acked Read 6 Done Done Sent false
640 1083 Acked Read 4 Done Done Sent false
641 1084 Acked Write 8 Done Done Sent false
642 1085 Acked Read 2 Done Done Sent false
643 1086 Acked Read 10 Done Done Sent false
644 1087 Acked Read 1 Done Done Sent false
645 1088 Acked Write 5 Done Done Sent false
646 1089 Acked Write 8 Done Done Sent false
647 1090 Acked Write 10 Done Done Sent false
648 1091 Acked Read 5 Done Done Sent false
649 1092 Acked Read 6 Done Done Sent false
650 1093 Acked Read 7 Done Done Sent false
651 1094 Acked Write 9 Done Done Sent false
652 1095 Acked Read 6 Done Done Sent false
653 1096 Acked Write 3 Done Done Sent false
654 1097 Acked Read 4 Done Done Sent false
655 1098 Acked Read 1 Done Done Sent false
656 1099 Acked Write 3 Done Done Sent false
657 1100 Acked Write 1 Done Done Sent false
658 1101 Acked Write 8 Done Done Sent false
659 1102 Acked Write 4 Done Done Sent false
660 1103 Acked Read 2 Done Done Sent false
661 1104 Acked Write 2 Done Done Sent false
662 1105 Acked Write 5 Done Done Sent false
663 1106 Acked Read 1 Done Done Sent false
664 1107 Acked Read 7 Done Done Sent false
665 1108 Acked Read 7 Done Done Sent false
666 1109 Acked Flush 0 Done Done Sent false
667 1110 Acked Read 6 Done Done Sent false
668 1111 Acked Read 7 Done Done Sent false
669 1112 Acked Read 2 Done Done Sent false
670 1113 Acked Write 2 Done Done Sent false
671 1114 Acked Read 9 Done Done Sent false
672 1115 Acked Write 10 Done Done Sent false
673 1116 Acked Read 9 Done Done Sent false
674 1117 Acked Read 2 Done Done Sent false
675 1118 Acked Read 7 Done Done Sent false
676 1119 Acked Read 6 Done Done Sent false
677 1120 Acked Read 10 Done Done Sent false
678 1121 Acked Write 7 Done Done Sent false
679 1122 Acked Read 3 Done Done Sent false
680 1123 Acked Read 3 Done Done Sent false
681 1124 Acked Write 3 Done Done Sent false
682 1125 Acked Flush 0 Done Done Sent false
683 1126 Acked Read 10 Done Done Sent false
684 1127 Acked Read 5 Done Done Sent false
685 1128 Acked Flush 0 Done Done Sent false
686 1129 Acked Write 8 Done Done Sent false
687 1130 Acked Write 9 Done Done Sent false
688 1131 Acked Read 4 Done Done Sent false
689 1132 Acked Read 10 Done Done Sent false
690 1133 Acked Read 7 Done Done Sent false
691 1134 Acked Flush 0 Done Done Sent false
692 1135 Acked Read 2 Done Done Sent false
693 1136 Acked Read 5 Done Done Sent false
694 1137 Acked Write 10 Done Done Sent false
695 1138 Acked Flush 0 Done Done Sent false
696 1139 Acked Write 1 Done Done Sent false
697 1140 Acked Read 4 Done Done Sent false
698 1141 Acked Read 8 Done Done Sent false
699 1142 Acked Flush 0 Done Done Sent false
700 1143 Acked Flush 0 Done Done Sent false
701 1144 Acked Read 1 Done Done Sent false
702 1145 Acked Write 10 Done Done Sent false
703 1146 Acked Read 2 Done Done Sent false
704 1147 Acked Read 6 Done Done Sent false
705 1148 Acked Write 10 Done Done Sent false
706 1149 Acked Write 7 Done Done Sent false
707 1150 Acked Write 5 Done Done Sent false
708 1151 Acked Write 3 Done Done Sent false
709 1152 Acked Write 6 Done Done Sent false
710 1153 Acked Read 6 Done Done Sent false
711 1154 Acked Write 2 Done Done Sent false
712 1155 Acked Write 3 Done Done Sent false
713 1156 Acked Write 1 Done Done Sent false
714 1157 Acked Read 2 Done Done Sent false
715 1158 Acked Write 6 Done Done Sent false
716 1159 Acked Write 9 Done Done Sent false
717 1160 Acked Write 6 Done Done Sent false
718 1161 Acked Read 4 Done Done Sent false
719 1162 Acked Write 6 Done Done Sent false
720 1163 Acked Read 6 Done Done Sent false
721 1164 Acked Read 6 Done Done Sent false
722 1165 Acked Read 8 Done Done Sent false
723 1166 Acked Write 8 Done Done Sent false
724 1167 Acked Write 1 Done Done Sent false
725 1168 Acked Write 7 Done Done Sent false
726 1169 Acked Read 1 Done Done Sent false
727 1170 Acked Read 2 Done Done Sent false
728 1171 Acked Read 5 Done Done Sent false
729 1172 Acked Write 8 Done Done Sent false
730 1173 Acked Write 4 Done Done Sent false
731 1174 Acked Read 6 Done Done Sent false
732 1175 Acked Write 8 Done Done Sent false
733 1176 Acked Read 7 Done Done Sent false
734 1177 Acked Read 5 Done Done Sent false
735 1178 Acked Flush 0 Done Done Sent false
736 1179 Acked Read 10 Done Done Sent false
737 1180 Acked Flush 0 Done Done Sent false
738 1181 Acked Write 2 Done Done Sent false
739 1182 Acked Read 5 Done Done Sent false
740 1183 Acked Read 7 Done Done Sent false
741 1184 Acked Write 5 Done Done Sent false
742 1185 Acked Write 5 Done Done Sent false
743 1186 Acked Write 4 Done Done Sent false
744 1187 Acked Write 6 Done Done Sent false
745 1188 Acked Flush 0 Done Done Sent false
746 1189 Acked Write 1 Done Done Sent false
747 1190 Acked Write 5 Done Done Sent false
748 1191 Acked Write 5 Done Done Sent false
749 1192 Acked Read 4 Done Done Sent false
750 1193 Acked Read 10 Done Done Sent false
751 1194 Acked Write 1 Done Done Sent false
752 1195 Acked Flush 0 Done Done Sent false
753 1196 Acked Flush 0 Done Done Sent false
754 1197 Acked Write 5 Done Done Sent false
755 1198 Acked Read 5 Done Done Sent false
756 1199 Acked Read 2 Done Done Sent false
757 1200 Acked Read 7 Done Done Sent false
758 1201 Acked Write 5 Done Done Sent false
759 1202 Acked Flush 0 Done Done Sent false
760 1203 Acked Read 5 Done Done Sent false
761 1204 Acked Write 5 Done Done Sent false
762 1205 Acked Read 1 Done Done Sent false
763 1206 Acked Read 5 Done Done Sent false
764 1207 Acked Write 5 Done Done Sent false
765 1208 Acked Write 5 Done Done Sent false
766 1209 Acked Read 8 Done Done Sent false
767 1210 Acked Read 9 Done Done Sent false
768 1211 Acked Read 2 Done Done Sent false
769 1212 Acked Read 9 Done Done Sent false
770 1213 Acked Write 10 Done Done Sent false
771 1214 Acked Read 4 Done Done Sent false
772 1215 Acked Read 3 Done Done Sent false
773 1216 Acked Read 1 Done Done Sent false
774 1217 Acked Write 7 Done Done Sent false
775 1218 Acked Read 6 Done Done Sent false
776 1219 Acked Write 3 Done Done Sent false
777 1220 Acked Write 10 Done Done Sent false
778 1221 Acked Write 4 Done Done Sent false
779 1222 Acked Read 10 Done Done Sent false
780 1223 Acked Read 1 Done Done Sent false
781 1224 Acked Write 5 Done Done Sent false
782 1225 Acked Flush 0 Done Done Sent false
783 1226 Acked Write 8 Done Done Sent false
784 1227 Acked Read 6 Done Done Sent false
785 1228 Acked Flush 0 Done Done Sent false
786 1229 Acked Write 1 Done Done Sent false
787 1230 Acked Read 6 Done Done Sent false
788 1231 Acked Read 3 Done Done Sent false
789 1232 Acked Read 5 Done Done Sent false
790 1233 Acked Read 1 Done Done Sent false
791 1234 Acked Write 8 Done Done Sent false
792 1235 Acked Read 4 Done Done Sent false
793 1236 Acked Write 6 Done Done Sent false
794 1237 Acked Read 2 Done Done Sent false
795 1238 Acked Write 8 Done Done Sent false
796 1239 Acked Read 2 Done Done Sent false
797 1240 Acked Read 7 Done Done Sent false
798 1241 Acked Flush 0 Done Done Sent false
799 1242 Acked Write 1 Done Done Sent false
800 1243 Acked Read 5 Done Done Sent false
801 1244 Acked Write 2 Done Done Sent false
802 1245 Acked Read 5 Done Done Sent false
803 1246 Acked Read 1 Done Done Sent false
804 1247 Acked Read 7 Done Done Sent false
805 1248 Acked Write 4 Done Done Sent false
806 1249 Acked Read 8 Done Done Sent false
807 1250 Acked Write 7 Done Done Sent false
808 1251 Acked Write 2 Done Done Sent false
809 1252 Acked Read 5 Done Done Sent false
810 1253 Acked Read 6 Done Done Sent false
811 1254 Acked Flush 0 Done Done Sent false
812 1255 Acked Write 9 Done Done Sent false
813 1256 Acked Read 5 Done Done Sent false
814 1257 Acked Flush 0 Done Done Sent false
815 1258 Acked Read 1 Done Done Sent false
816 1259 Acked Write 1 Done Done Sent false
817 1260 Acked Flush 0 Done Done Sent false
818 1261 Acked Write 6 Done Done Sent false
819 1262 Acked Write 6 Done Done Sent false
820 1263 Acked Write 7 Done Done Sent false
821 1264 Acked Write 9 Done Done Sent false
822 1265 Acked Flush 0 Done Done Sent false
823 1266 Acked Read 1 Done Done Sent false
824 1267 Acked Write 9 Done Done Sent false
825 1268 Acked Read 7 Done Done Sent false
826 1269 Acked Read 9 Done Done Sent false
827 1270 Acked Read 5 Done Done Sent false
828 1271 Acked Write 3 Done Done Sent false
829 1272 Acked Write 10 Done Done Sent false
830 1273 Acked Read 7 Done Done Sent false
831 1274 Acked Write 5 Done Done Sent false
832 1275 Acked Read 6 Done Done Sent false
833 1276 Acked Write 6 Done Done Sent false
834 1277 Acked Write 8 Done Done Sent false
835 1278 Acked Flush 0 Done Done Sent false
836 1279 Acked Read 10 Done Done Sent false
837 1280 Acked Write 5 Done Done Sent false
838 1281 Acked Read 1 Done Done Sent false
839 1282 Acked Read 9 Done Done Sent false
840 1283 Acked Read 5 Done Done Sent false
841 1284 Acked Flush 0 Done Done Sent false
842 1285 Acked Read 1 Done Done Sent false
843 1286 Acked Read 9 Done Done Sent false
844 1287 Acked Write 1 Done Done Sent false
845 1288 Acked Write 10 Done Done Sent false
846 1289 Acked Write 10 Done Done Sent false
847 1290 Acked Write 10 Done Done Sent false
848 1291 Acked Read 7 Done Done Sent false
849 1292 Acked Read 6 Done Done Sent false
850 1293 Acked Read 1 Done Done Sent false
851 1294 Acked Read 1 Done Done Sent false
852 1295 Acked Write 6 Done Done Sent false
853 1296 Acked Write 4 Done Done Sent false
854 1297 Acked Read 2 Done Done Sent false
855 1298 Acked Read 9 Done Done Sent false
856 1299 Acked Read 2 Done Done Sent false
857 1300 Acked Read 7 Done Done Sent false
858 STATES DS:0 DS:1 DS:2 TOTAL
859 Sent 0 0 301 301
860 Done 301 301 0 602
861 Skipped 0 0 0 0
862 Error 0 0 0 0
863 Last Flush: 1284 1284 0
864 Downstairs last five completed:
865 Upstairs last five completed: 1300 1299 1298 1297 1296
866 CLIENT: Up:0 ds:301 act:2
867 [8830] got start action
868 [8830] I am going to restart
869 Starting downstairs at port 8830
870 Make output file at "/tmp/dsc/downstairs-8830.txt"
871 [8830][2] reports Starting
872 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1203)
873 [8830][2] reports Running
874 ----------------------------------------------------------------
875 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
876 States: Active Active Offline
877 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
878 1000 Acked Read 6 Done Done Sent false
879 1001 Acked Read 6 Done Done Sent false
880 1002 Acked Write 6 Done Done Sent false
881 1003 Acked Write 1 Done Done Sent false
882 1004 Acked Write 8 Done Done Sent false
883 1005 Acked Write 9 Done Done Sent false
884 1006 Acked Read 7 Done Done Sent false
885 1007 Acked Read 8 Done Done Sent false
886 1008 Acked Read 7 Done Done Sent false
887 1009 Acked Read 1 Done Done Sent false
888 1010 Acked Read 4 Done Done Sent false
889 1011 Acked Write 2 Done Done Sent false
890 1012 Acked Write 10 Done Done Sent false
891 1013 Acked Write 9 Done Done Sent false
892 1014 Acked Read 4 Done Done Sent false
893 1015 Acked Read 3 Done Done Sent false
894 1016 Acked Write 3 Done Done Sent false
895 1017 Acked Read 9 Done Done Sent false
896 1018 Acked Read 6 Done Done Sent false
897 1019 Acked Write 6 Done Done Sent false
898 1020 Acked Read 3 Done Done Sent false
899 1021 Acked Write 5 Done Done Sent false
900 1022 Acked Read 2 Done Done Sent false
901 1023 Acked Read 6 Done Done Sent false
902 1024 Acked Write 9 Done Done Sent false
903 1025 Acked Read 3 Done Done Sent false
904 1026 Acked Write 2 Done Done Sent false
905 1027 Acked Write 4 Done Done Sent false
906 1028 Acked Read 7 Done Done Sent false
907 1029 Acked Read 8 Done Done Sent false
908 1030 Acked Write 7 Done Done Sent false
909 1031 Acked Read 10 Done Done Sent false
910 1032 Acked Write 6 Done Done Sent false
911 1033 Acked Write 6 Done Done Sent false
912 1034 Acked Flush 0 Done Done Sent false
913 1035 Acked Write 1 Done Done Sent false
914 1036 Acked Flush 0 Done Done Sent false
915 1037 Acked Write 3 Done Done Sent false
916 1038 Acked Read 4 Done Done Sent false
917 1039 Acked Write 8 Done Done Sent false
918 1040 Acked Write 5 Done Done Sent false
919 1041 Acked Write 8 Done Done Sent false
920 1042 Acked Read 10 Done Done Sent false
921 1043 Acked Write 2 Done Done Sent false
922 1044 Acked Write 7 Done Done Sent false
923 1045 Acked Read 5 Done Done Sent false
924 1046 Acked Write 9 Done Done Sent false
925 1047 Acked Read 6 Done Done Sent false
926 1048 Acked Read 8 Done Done Sent false
927 1049 Acked Flush 0 Done Done Sent false
928 1050 Acked Read 1 Done Done Sent false
929 1051 Acked Write 10 Done Done Sent false
930 1052 Acked Read 5 Done Done Sent false
931 1053 Acked Flush 0 Done Done Sent false
932 1054 Acked Write 10 Done Done Sent false
933 1055 Acked Write 5 Done Done Sent false
934 1056 Acked Read 9 Done Done Sent false
935 1057 Acked Write 7 Done Done Sent false
936 1058 Acked Read 6 Done Done Sent false
937 1059 Acked Read 7 Done Done Sent false
938 1060 Acked Write 7 Done Done Sent false
939 1061 Acked Write 9 Done Done Sent false
940 1062 Acked Write 6 Done Done Sent false
941 1063 Acked Write 8 Done Done Sent false
942 1064 Acked Read 1 Done Done Sent false
943 1065 Acked Read 4 Done Done Sent false
944 1066 Acked Write 9 Done Done Sent false
945 1067 Acked Write 10 Done Done Sent false
946 1068 Acked Read 8 Done Done Sent false
947 1069 Acked Write 8 Done Done Sent false
948 1070 Acked Read 4 Done Done Sent false
949 1071 Acked Read 2 Done Done Sent false
950 1072 Acked Read 2 Done Done Sent false
951 1073 Acked Write 2 Done Done Sent false
952 1074 Acked Read 6 Done Done Sent false
953 1075 Acked Read 2 Done Done Sent false
954 1076 Acked Read 5 Done Done Sent false
955 1077 Acked Read 1 Done Done Sent false
956 1078 Acked Read 1 Done Done Sent false
957 1079 Acked Read 5 Done Done Sent false
958 1080 Acked Read 1 Done Done Sent false
959 1081 Acked Write 1 Done Done Sent false
960 1082 Acked Read 6 Done Done Sent false
961 1083 Acked Read 4 Done Done Sent false
962 1084 Acked Write 8 Done Done Sent false
963 1085 Acked Read 2 Done Done Sent false
964 1086 Acked Read 10 Done Done Sent false
965 1087 Acked Read 1 Done Done Sent false
966 1088 Acked Write 5 Done Done Sent false
967 1089 Acked Write 8 Done Done Sent false
968 1090 Acked Write 10 Done Done Sent false
969 1091 Acked Read 5 Done Done Sent false
970 1092 Acked Read 6 Done Done Sent false
971 1093 Acked Read 7 Done Done Sent false
972 1094 Acked Write 9 Done Done Sent false
973 1095 Acked Read 6 Done Done Sent false
974 1096 Acked Write 3 Done Done Sent false
975 1097 Acked Read 4 Done Done Sent false
976 1098 Acked Read 1 Done Done Sent false
977 1099 Acked Write 3 Done Done Sent false
978 1100 Acked Write 1 Done Done Sent false
979 1101 Acked Write 8 Done Done Sent false
980 1102 Acked Write 4 Done Done Sent false
981 1103 Acked Read 2 Done Done Sent false
982 1104 Acked Write 2 Done Done Sent false
983 1105 Acked Write 5 Done Done Sent false
984 1106 Acked Read 1 Done Done Sent false
985 1107 Acked Read 7 Done Done Sent false
986 1108 Acked Read 7 Done Done Sent false
987 1109 Acked Flush 0 Done Done Sent false
988 1110 Acked Read 6 Done Done Sent false
989 1111 Acked Read 7 Done Done Sent false
990 1112 Acked Read 2 Done Done Sent false
991 1113 Acked Write 2 Done Done Sent false
992 1114 Acked Read 9 Done Done Sent false
993 1115 Acked Write 10 Done Done Sent false
994 1116 Acked Read 9 Done Done Sent false
995 1117 Acked Read 2 Done Done Sent false
996 1118 Acked Read 7 Done Done Sent false
997 1119 Acked Read 6 Done Done Sent false
998 1120 Acked Read 10 Done Done Sent false
999 1121 Acked Write 7 Done Done Sent false
1000 1122 Acked Read 3 Done Done Sent false
1001 1123 Acked Read 3 Done Done Sent false
1002 1124 Acked Write 3 Done Done Sent false
1003 1125 Acked Flush 0 Done Done Sent false
1004 1126 Acked Read 10 Done Done Sent false
1005 1127 Acked Read 5 Done Done Sent false
1006 1128 Acked Flush 0 Done Done Sent false
1007 1129 Acked Write 8 Done Done Sent false
1008 1130 Acked Write 9 Done Done Sent false
1009 1131 Acked Read 4 Done Done Sent false
1010 1132 Acked Read 10 Done Done Sent false
1011 1133 Acked Read 7 Done Done Sent false
1012 1134 Acked Flush 0 Done Done Sent false
1013 1135 Acked Read 2 Done Done Sent false
1014 1136 Acked Read 5 Done Done Sent false
1015 1137 Acked Write 10 Done Done Sent false
1016 1138 Acked Flush 0 Done Done Sent false
1017 1139 Acked Write 1 Done Done Sent false
1018 1140 Acked Read 4 Done Done Sent false
1019 1141 Acked Read 8 Done Done Sent false
1020 1142 Acked Flush 0 Done Done Sent false
1021 1143 Acked Flush 0 Done Done Sent false
1022 1144 Acked Read 1 Done Done Sent false
1023 1145 Acked Write 10 Done Done Sent false
1024 1146 Acked Read 2 Done Done Sent false
1025 1147 Acked Read 6 Done Done Sent false
1026 1148 Acked Write 10 Done Done Sent false
1027 1149 Acked Write 7 Done Done Sent false
1028 1150 Acked Write 5 Done Done Sent false
1029 1151 Acked Write 3 Done Done Sent false
1030 1152 Acked Write 6 Done Done Sent false
1031 1153 Acked Read 6 Done Done Sent false
1032 1154 Acked Write 2 Done Done Sent false
1033 1155 Acked Write 3 Done Done Sent false
1034 1156 Acked Write 1 Done Done Sent false
1035 1157 Acked Read 2 Done Done Sent false
1036 1158 Acked Write 6 Done Done Sent false
1037 1159 Acked Write 9 Done Done Sent false
1038 1160 Acked Write 6 Done Done Sent false
1039 1161 Acked Read 4 Done Done Sent false
1040 1162 Acked Write 6 Done Done Sent false
1041 1163 Acked Read 6 Done Done Sent false
1042 1164 Acked Read 6 Done Done Sent false
1043 1165 Acked Read 8 Done Done Sent false
1044 1166 Acked Write 8 Done Done Sent false
1045 1167 Acked Write 1 Done Done Sent false
1046 1168 Acked Write 7 Done Done Sent false
1047 1169 Acked Read 1 Done Done Sent false
1048 1170 Acked Read 2 Done Done Sent false
1049 1171 Acked Read 5 Done Done Sent false
1050 1172 Acked Write 8 Done Done Sent false
1051 1173 Acked Write 4 Done Done Sent false
1052 1174 Acked Read 6 Done Done Sent false
1053 1175 Acked Write 8 Done Done Sent false
1054 1176 Acked Read 7 Done Done Sent false
1055 1177 Acked Read 5 Done Done Sent false
1056 1178 Acked Flush 0 Done Done Sent false
1057 1179 Acked Read 10 Done Done Sent false
1058 1180 Acked Flush 0 Done Done Sent false
1059 1181 Acked Write 2 Done Done Sent false
1060 1182 Acked Read 5 Done Done Sent false
1061 1183 Acked Read 7 Done Done Sent false
1062 1184 Acked Write 5 Done Done Sent false
1063 1185 Acked Write 5 Done Done Sent false
1064 1186 Acked Write 4 Done Done Sent false
1065 1187 Acked Write 6 Done Done Sent false
1066 1188 Acked Flush 0 Done Done Sent false
1067 1189 Acked Write 1 Done Done Sent false
1068 1190 Acked Write 5 Done Done Sent false
1069 1191 Acked Write 5 Done Done Sent false
1070 1192 Acked Read 4 Done Done Sent false
1071 1193 Acked Read 10 Done Done Sent false
1072 1194 Acked Write 1 Done Done Sent false
1073 1195 Acked Flush 0 Done Done Sent false
1074 1196 Acked Flush 0 Done Done Sent false
1075 1197 Acked Write 5 Done Done Sent false
1076 1198 Acked Read 5 Done Done Sent false
1077 1199 Acked Read 2 Done Done Sent false
1078 1200 Acked Read 7 Done Done Sent false
1079 1201 Acked Write 5 Done Done Sent false
1080 1202 Acked Flush 0 Done Done Sent false
1081 1203 Acked Read 5 Done Done Sent false
1082 1204 Acked Write 5 Done Done Sent false
1083 1205 Acked Read 1 Done Done Sent false
1084 1206 Acked Read 5 Done Done Sent false
1085 1207 Acked Write 5 Done Done Sent false
1086 1208 Acked Write 5 Done Done Sent false
1087 1209 Acked Read 8 Done Done Sent false
1088 1210 Acked Read 9 Done Done Sent false
1089 1211 Acked Read 2 Done Done Sent false
1090 1212 Acked Read 9 Done Done Sent false
1091 1213 Acked Write 10 Done Done Sent false
1092 1214 Acked Read 4 Done Done Sent false
1093 1215 Acked Read 3 Done Done Sent false
1094 1216 Acked Read 1 Done Done Sent false
1095 1217 Acked Write 7 Done Done Sent false
1096 1218 Acked Read 6 Done Done Sent false
1097 1219 Acked Write 3 Done Done Sent false
1098 1220 Acked Write 10 Done Done Sent false
1099 1221 Acked Write 4 Done Done Sent false
1100 1222 Acked Read 10 Done Done Sent false
1101 1223 Acked Read 1 Done Done Sent false
1102 1224 Acked Write 5 Done Done Sent false
1103 1225 Acked Flush 0 Done Done Sent false
1104 1226 Acked Write 8 Done Done Sent false
1105 1227 Acked Read 6 Done Done Sent false
1106 1228 Acked Flush 0 Done Done Sent false
1107 1229 Acked Write 1 Done Done Sent false
1108 1230 Acked Read 6 Done Done Sent false
1109 1231 Acked Read 3 Done Done Sent false
1110 1232 Acked Read 5 Done Done Sent false
1111 1233 Acked Read 1 Done Done Sent false
1112 1234 Acked Write 8 Done Done Sent false
1113 1235 Acked Read 4 Done Done Sent false
1114 1236 Acked Write 6 Done Done Sent false
1115 1237 Acked Read 2 Done Done Sent false
1116 1238 Acked Write 8 Done Done Sent false
1117 1239 Acked Read 2 Done Done Sent false
1118 1240 Acked Read 7 Done Done Sent false
1119 1241 Acked Flush 0 Done Done Sent false
1120 1242 Acked Write 1 Done Done Sent false
1121 1243 Acked Read 5 Done Done Sent false
1122 1244 Acked Write 2 Done Done Sent false
1123 1245 Acked Read 5 Done Done Sent false
1124 1246 Acked Read 1 Done Done Sent false
1125 1247 Acked Read 7 Done Done Sent false
1126 1248 Acked Write 4 Done Done Sent false
1127 1249 Acked Read 8 Done Done Sent false
1128 1250 Acked Write 7 Done Done Sent false
1129 1251 Acked Write 2 Done Done Sent false
1130 1252 Acked Read 5 Done Done Sent false
1131 1253 Acked Read 6 Done Done Sent false
1132 1254 Acked Flush 0 Done Done Sent false
1133 1255 Acked Write 9 Done Done Sent false
1134 1256 Acked Read 5 Done Done Sent false
1135 1257 Acked Flush 0 Done Done Sent false
1136 1258 Acked Read 1 Done Done Sent false
1137 1259 Acked Write 1 Done Done Sent false
1138 1260 Acked Flush 0 Done Done Sent false
1139 1261 Acked Write 6 Done Done Sent false
1140 1262 Acked Write 6 Done Done Sent false
1141 1263 Acked Write 7 Done Done Sent false
1142 1264 Acked Write 9 Done Done Sent false
1143 1265 Acked Flush 0 Done Done Sent false
1144 1266 Acked Read 1 Done Done Sent false
1145 1267 Acked Write 9 Done Done Sent false
1146 1268 Acked Read 7 Done Done Sent false
1147 1269 Acked Read 9 Done Done Sent false
1148 1270 Acked Read 5 Done Done Sent false
1149 1271 Acked Write 3 Done Done Sent false
1150 1272 Acked Write 10 Done Done Sent false
1151 1273 Acked Read 7 Done Done Sent false
1152 1274 Acked Write 5 Done Done Sent false
1153 1275 Acked Read 6 Done Done Sent false
1154 1276 Acked Write 6 Done Done Sent false
1155 1277 Acked Write 8 Done Done Sent false
1156 1278 Acked Flush 0 Done Done Sent false
1157 1279 Acked Read 10 Done Done Sent false
1158 1280 Acked Write 5 Done Done Sent false
1159 1281 Acked Read 1 Done Done Sent false
1160 1282 Acked Read 9 Done Done Sent false
1161 1283 Acked Read 5 Done Done Sent false
1162 1284 Acked Flush 0 Done Done Sent false
1163 1285 Acked Read 1 Done Done Sent false
1164 1286 Acked Read 9 Done Done Sent false
1165 1287 Acked Write 1 Done Done Sent false
1166 1288 Acked Write 10 Done Done Sent false
1167 1289 Acked Write 10 Done Done Sent false
1168 1290 Acked Write 10 Done Done Sent false
1169 1291 Acked Read 7 Done Done Sent false
1170 1292 Acked Read 6 Done Done Sent false
1171 1293 Acked Read 1 Done Done Sent false
1172 1294 Acked Read 1 Done Done Sent false
1173 1295 Acked Write 6 Done Done Sent false
1174 1296 Acked Write 4 Done Done Sent false
1175 1297 Acked Read 2 Done Done Sent false
1176 1298 Acked Read 9 Done Done Sent false
1177 1299 Acked Read 2 Done Done Sent false
1178 1300 Acked Read 7 Done Done Sent false
1179 1301 Acked Flush 0 Done Done Sent false
1180 STATES DS:0 DS:1 DS:2 TOTAL
1181 Sent 0 0 302 302
1182 Done 302 302 0 604
1183 Skipped 0 0 0 0
1184 Error 0 0 0 0
1185 Last Flush: 1301 1301 0
1186 Downstairs last five completed:
1187 Upstairs last five completed: 1301 1300 1299 1298 1297
1188 CLIENT: Up:0 ds:302 act:2
11892024-11-28T04:50:33.013ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
11902024-11-28T04:50:33.013ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:39941) = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
11912024-11-28T04:50:33.014ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
11922024-11-28T04:50:33.014ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
11932024-11-28T04:50:33.014ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
11942024-11-28T04:50:33.014ZINFOcrucible: send last flush ID to this DS: 0 = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
11952024-11-28T04:50:33.015ZINFOcrucible: Replied this last flush ID: 0 = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
11962024-11-28T04:50:33.015ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
11972024-11-28T04:50:33.015ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
11982024-11-28T04:50:33.015ZINFOcrucible: [2] client re-new 302 jobs since flush 0 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
11992024-11-28T04:50:33.015ZINFOcrucible: [2] Replayed 302 jobs since flush: 0 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
1200 ----------------------------------------------------------------
1201 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
1202 Downstairs last five completed: 1301 1300 1299 1298 1297
1203 Upstairs last five completed: 1301 1300 1299 1298 1297
1204 CLIENT: Up:0 ds:0 act:3
1205 Replay: All jobs finished
1206 Main task has work to do, go find it
1207 got dsc Stop(0)
1208 stop 0
1209 Nov 28 04:50:35.251 INFO request completed, latency_us: 315, response_code: 204, uri: /stop/cid/0, method: POST[8810] Got stop action so:false kr:false
1210 , req_id: e4659655-cdcd-43b5-8b93-f68f6485bf37, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
1211 Nov 28 04:50:35.251 INFO request completed, latency_us: 114, response_code: 200, uri: /state/cid/0, method: GET, req_id: 936f2583-1163-4480-b663-f03977a3a6c2, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
12122024-11-28T04:50:35.251ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
12132024-11-28T04:50:35.252ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
12142024-11-28T04:50:35.252ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
12152024-11-28T04:50:35.252ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
12162024-11-28T04:50:35.252ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
12172024-11-28T04:50:35.252ZINFOcrucible: client task is exiting = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
1218 [8810] Exited with: ExitStatus(unix_wait_status(9))
1219 [8810][0] reports Exit
1220 Nov 28 04:50:39.254 INFO request completed, latency_us: 202, response_code: 200, uri: /state/cid/0, method: GET, req_id: e48b72ee-b919-48da-a3a4-590a4718765a, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
1221 001/300 Write block 8 len 16384 data: 2 2 2 2
1222 002/300 Write block 2643 len 4096 data: 2
1223 003/300 Read block 334 len 4096
1224 004/300 Read block 876 len 12288
1225 005/300 Write block 1033 len 8192 data: 2 2
1226 006/300 Read block 3204 len 4096
1227 007/300 Read block 2993 len 4096
1228 008/300 Write block 1051 len 20480 data: 4 4 4 2 2
1229 009/300 Write block 851 len 8192 data: 2 2
1230 010/300 Write block 156 len 16384 data: 2 2 2 2
1231 011/300 Read block 2916 len 12288
1232 012/300 Write block 2325 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1233 013/300 Write block 3837 len 20480 data: 2 2 2 2 2
1234 014/300 Read block 1145 len 20480
1235 015/300 Write block 2792 len 4096 data: 2
1236 016/300 Read block 1579 len 40960
1237 017/300 Write block 4122 len 36864 data: 2 2 2 2 2 2 2 2 2
1238 018/300 Read block 1133 len 32768
1239 019/300 Read block 3282 len 40960
1240 020/300 Write block 4087 len 28672 data: 2 2 2 3 3 3 3
1241 021/300 Read block 3594 len 32768
1242 022/300 Read block 814 len 24576
1243 023/300 Write block 1368 len 4096 data: 2
1244 024/300 Read block 2497 len 36864
1245 025/300 Write block 4031 len 24576 data: 2 2 2 2 2 2
1246 026/300 Write block 2721 len 4096 data: 2
1247 027/300 Write block 1602 len 8192 data: 2 2
1248 028/300 Write block 1401 len 8192 data: 2 2
1249 029/300 Write block 1705 len 24576 data: 3 3 3 2 3 3
1250 030/300 Write block 3496 len 12288 data: 2 2 2
1251 031/300 Write block 189 len 32768 data: 2 2 2 2 2 2 2 2
1252 032/300 Write block 49 len 16384 data: 4 2 2 2
1253 033/300 Read block 2337 len 36864
1254 034/300 Read block 2752 len 20480
1255 035/300 Read block 63 len 4096
1256 036/300 Read block 3952 len 24576
1257 037/300 Read block 1902 len 28672
1258 038/300 Write block 4079 len 36864 data: 2 2 2 2 2 2 2 2 3
1259 039/300 Write block 2947 len 4096 data: 3
1260 040/300 Write block 4881 len 12288 data: 2 2 2
1261 041/300 Read block 2094 len 4096
1262 042/300 Flush
1263 043/300 Read block 550 len 16384
1264 044/300 Read block 3129 len 32768
1265 045/300 Flush
1266 046/300 Read block 3865 len 40960
1267 047/300 Read block 3686 len 8192
1268 048/300 Read block 1868 len 24576
1269 049/300 Write block 629 len 32768 data: 2 2 2 2 2 2 3 3
1270 050/300 Write block 1452 len 16384 data: 2 2 2 2
1271 051/300 Flush
1272 052/300 Write block 1427 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1273 053/300 Write block 2259 len 32768 data: 2 2 3 3 2 2 2 2
1274 054/300 Write block 3986 len 36864 data: 3 2 2 2 2 2 2 2 2
1275 055/300 Read block 4375 len 8192
1276 056/300 Read block 2839 len 4096
1277 057/300 Flush
1278 058/300 Read block 4589 len 36864
1279 059/300 Write block 16 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1280 060/300 Read block 889 len 36864
1281 061/300 Write block 4452 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1282 062/300 Read block 1512 len 20480
1283 063/300 Flush
1284 064/300 Read block 4920 len 4096
1285 065/300 Read block 4472 len 4096
1286 066/300 Write block 667 len 24576 data: 2 2 2 2 2 2
1287 067/300 Read block 4835 len 32768
1288 068/300 Write block 2034 len 16384 data: 2 2 2 2
1289 069/300 Write block 2713 len 36864 data: 2 2 2 2 2 2 2 2 3
1290 070/300 Flush
1291 071/300 Read block 2369 len 12288
1292 072/300 Read block 3195 len 24576
1293 073/300 Read block 2711 len 32768
1294 074/300 Read block 1540 len 32768
1295 075/300 Write block 850 len 4096 data: 2
1296 076/300 Write block 657 len 28672 data: 3 3 3 3 3 3 2
1297 077/300 Read block 1527 len 20480
1298 078/300 Write block 4264 len 28672 data: 2 2 2 2 2 2 2
1299 079/300 Write block 4065 len 20480 data: 2 2 2 2 2
1300 080/300 Flush
1301 081/300 Read block 2184 len 20480
1302 082/300 Flush
1303 083/300 Write block 1538 len 8192 data: 2 2
1304 084/300 Write block 1390 len 20480 data: 2 2 2 2 2
1305 085/300 Flush
1306 086/300 Write block 3084 len 8192 data: 2 2
1307 087/300 Write block 1827 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1308 088/300 Read block 4175 len 40960
1309 089/300 Read block 3374 len 8192
1310 090/300 Write block 2570 len 12288 data: 2 2 2
1311 091/300 Read block 3481 len 32768
1312 092/300 Flush
1313 093/300 Read block 2577 len 8192
1314 094/300 Write block 4676 len 28672 data: 2 2 2 2 2 2 2
1315 095/300 Read block 3661 len 28672
1316 096/300 Write block 4352 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1317 097/300 Write block 1422 len 36864 data: 2 2 2 2 2 3 3 3 3
1318 098/300 Write block 4115 len 4096 data: 2
1319 099/300 Flush
1320 100/300 Write block 4 len 32768 data: 2 2 2 2 3 3 3 3
1321 101/300 Write block 3668 len 12288 data: 2 2 2
1322 102/300 Read block 1071 len 40960
1323 103/300 Write block 3822 len 4096 data: 2
1324 104/300 Read block 3306 len 40960
1325 105/300 Read block 3597 len 40960
1326 106/300 Write block 589 len 4096 data: 2
1327 107/300 Flush
1328 108/300 Write block 1124 len 16384 data: 2 2 2 2
1329 109/300 Read block 110 len 16384
1330 110/300 Read block 1617 len 8192
1331 111/300 Read block 308 len 12288
1332 112/300 Read block 1693 len 20480
1333 113/300 Read block 4878 len 24576
1334 114/300 Write block 2440 len 8192 data: 2 2
1335 115/300 Write block 4497 len 4096 data: 2
1336 116/300 Read block 1231 len 16384
1337 117/300 Write block 4948 len 32768 data: 2 2 2 2 2 2 3 3
1338 118/300 Write block 2463 len 4096 data: 2
1339 119/300 Write block 3669 len 32768 data: 3 3 2 2 2 2 2 2
1340 120/300 Write block 2962 len 8192 data: 2 2
1341 121/300 Flush
1342 122/300 Write block 3310 len 36864 data: 2 2 2 2 2 2 2 2 2
1343 123/300 Read block 4242 len 8192
1344 124/300 Write block 2926 len 32768 data: 3 2 2 2 2 2 2 2
1345 125/300 Write block 2307 len 40960 data: 2 2 2 2 2 2 2 3 3 3
1346 126/300 Read block 4080 len 32768
1347 127/300 Read block 4004 len 36864
1348 128/300 Flush
1349 129/300 Read block 3624 len 28672
1350 130/300 Flush
1351 131/300 Read block 489 len 36864
1352 132/300 Read block 335 len 4096
1353 133/300 Flush
1354 134/300 Flush
1355 135/300 Write block 938 len 8192 data: 2 2
1356 136/300 Read block 976 len 16384
1357 137/300 Write block 2157 len 28672 data: 3 3 3 3 3 3 3
1358 138/300 Read block 2440 len 8192
1359 139/300 Write block 3674 len 16384 data: 3 3 3 2
1360 140/300 Read block 1375 len 4096
1361 141/300 Read block 1266 len 24576
1362 142/300 Read block 3188 len 36864
1363 143/300 Read block 2185 len 32768
1364 144/300 Read block 4668 len 16384
1365 145/300 Read block 135 len 16384
1366 146/300 Write block 4059 len 36864 data: 3 3 2 2 2 2 3 3 3
1367 147/300 Write block 1083 len 32768 data: 2 2 2 2 2 2 2 2
1368 148/300 Read block 4006 len 40960
1369 149/300 Read block 4007 len 32768
1370 150/300 Write block 4964 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1371 151/300 Read block 2060 len 12288
1372 152/300 Read block 4830 len 28672
1373 153/300 Flush
1374 154/300 Read block 4907 len 32768
1375 155/300 Read block 865 len 32768
1376 156/300 Write block 3658 len 4096 data: 2
1377 157/300 Flush
1378 158/300 Write block 365 len 36864 data: 2 3 3 3 3 3 3 3 3
1379 159/300 Write block 2104 len 8192 data: 2 3
1380 160/300 Read block 4803 len 20480
1381 161/300 Read block 2976 len 24576
1382 162/300 Read block 2860 len 28672
1383 163/300 Read block 4419 len 16384
1384 164/300 Read block 2791 len 8192
1385 165/300 Write block 3209 len 12288 data: 2 2 2
1386 166/300 Read block 2207 len 28672
1387 167/300 Read block 2740 len 36864
1388 168/300 Read block 3254 len 20480
1389 169/300 Read block 1413 len 24576
1390 170/300 Read block 3579 len 16384
1391 171/300 Read block 1671 len 12288
1392 172/300 Write block 1186 len 12288 data: 2 2 2
1393 173/300 Write block 4216 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1394 174/300 Read block 2732 len 4096
1395 175/300 Write block 3564 len 24576 data: 2 2 2 2 2 2
1396 176/300 Write block 1601 len 32768 data: 2 3 3 2 2 2 2 2
1397 177/300 Write block 2368 len 36864 data: 2 2 2 2 2 2 2 2 2
1398 178/300 Read block 3469 len 28672
1399 179/300 Read block 2083 len 28672
1400 180/300 Read block 1325 len 8192
1401 181/300 Write block 3000 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1402 182/300 Read block 4893 len 8192
1403 183/300 Read block 1790 len 12288
1404 184/300 Read block 4801 len 16384
1405 185/300 Flush
1406 186/300 Write block 1390 len 20480 data: 3 3 3 3 3
1407 187/300 Write block 1789 len 8192 data: 3 3
1408 188/300 Read block 689 len 36864
1409 189/300 Read block 1159 len 16384
1410 190/300 Write block 3746 len 8192 data: 2 2
1411 191/300 Read block 4470 len 12288
1412 192/300 Write block 826 len 20480 data: 3 3 4 4 4
1413 193/300 Read block 2324 len 40960
1414 194/300 Read block 4655 len 32768
1415 195/300 Flush
1416 196/300 Read block 2735 len 8192
1417 197/300 Flush
1418 198/300 Write block 963 len 24576 data: 2 2 3 3 3 3
1419 199/300 Read block 992 len 40960
1420 200/300 Read block 257 len 28672
1421 201/300 Write block 975 len 28672 data: 2 2 2 2 2 2 2
1422 202/300 Read block 706 len 36864
1423 203/300 Flush
1424 204/300 Flush
1425 205/300 Flush
1426 206/300 Flush
1427 207/300 Write block 1937 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1428 208/300 Read block 2854 len 32768
1429 209/300 Flush
1430 210/300 Read block 2255 len 16384
1431 211/300 Read block 1914 len 32768
1432 212/300 Read block 4178 len 4096
1433 213/300 Read block 2155 len 40960
1434 214/300 Write block 1256 len 4096 data: 2
1435 215/300 Read block 2078 len 36864
1436 216/300 Write block 840 len 28672 data: 2 2 2 2 2 2 2
1437 217/300 Read block 2581 len 28672
1438 218/300 Read block 4718 len 8192
1439 219/300 Flush
1440 220/300 Flush
1441 221/300 Flush
1442 222/300 Write block 1781 len 28672 data: 2 2 2 2 3 4 3
1443 223/300 Read block 3665 len 8192
1444 224/300 Write block 1260 len 20480 data: 2 2 2 2 2
1445 225/300 Read block 3383 len 28672
1446 226/300 Read block 219 len 20480
1447 227/300 Read block 3528 len 8192
1448 228/300 Write block 45 len 32768 data: 2 3 4 4 5 3 3 3
1449 229/300 Read block 219 len 40960
1450 230/300 Read block 1668 len 32768
1451 231/300 Read block 3299 len 28672
1452 232/300 Read block 4981 len 24576
1453 233/300 Read block 4545 len 4096
1454 234/300 Read block 15 len 36864
1455 235/300 Write block 2714 len 24576 data: 3 3 3 3 3 3
1456 236/300 Write block 4454 len 24576 data: 3 3 3 3 3 3
1457 237/300 Write block 4136 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1458 238/300 Write block 2112 len 32768 data: 2 2 2 2 2 2 2 2
1459 239/300 Write block 1414 len 36864 data: 2 2 2 2 2 2 2 2 3
1460 240/300 Read block 3733 len 28672
1461 241/300 Read block 1261 len 24576
1462 242/300 Read block 1687 len 4096
1463 243/300 Write block 501 len 32768 data: 3 3 3 3 3 3 3 2
1464 244/300 Read block 3193 len 16384
1465 245/300 Write block 4059 len 28672 data: 4 4 3 3 3 3 4
1466 246/300 Write block 1488 len 32768 data: 2 2 2 2 2 2 2 2
1467 247/300 Read block 2936 len 32768
1468 248/300 Write block 2773 len 36864 data: 2 2 2 2 2 2 2 2 2
1469 249/300 Write block 60 len 20480 data: 2 2 2 2 2
1470 250/300 Read block 3726 len 16384
1471 251/300 Flush
1472 252/300 Read block 1491 len 40960
1473 253/300 Write block 2920 len 32768 data: 3 3 3 3 3 3 4 3
1474 254/300 Write block 3177 len 8192 data: 2 2
1475 255/300 Write block 2298 len 4096 data: 2
1476 256/300 Flush
1477 257/300 Read block 1374 len 32768
1478 258/300 Read block 4849 len 28672
1479 259/300 Write block 4213 len 4096 data: 2
1480 260/300 Write block 3370 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1481 261/300 Read block 1352 len 40960
1482 262/300 Write block 1461 len 12288 data: 2 2 2
1483 263/300 Flush
1484 264/300 Read block 3647 len 20480
1485 265/300 Write block 3533 len 20480 data: 3 3 3 2 2
1486 266/300 Write block 2430 len 28672 data: 2 2 2 2 2 2 2
1487 267/300 Flush
1488 268/300 Read block 271 len 20480
1489 269/300 Write block 3746 len 12288 data: 3 3 2
1490 270/300 Read block 1330 len 40960
1491 271/300 Read block 403 len 8192
1492 272/300 Read block 1531 len 20480
1493 273/300 Write block 3567 len 24576 data: 3 3 3 2 2 2
1494 274/300 Write block 3328 len 16384 data: 2 2 2 2
1495 275/300 Read block 4518 len 32768
1496 276/300 Read block 3768 len 20480
1497 277/300 Read block 3562 len 36864
1498 278/300 Read block 3068 len 40960
1499 279/300 Write block 3011 len 20480 data: 2 2 2 2 2
1500 280/300 Write block 2685 len 20480 data: 2 2 2 2 2
1501 281/300 Read block 386 len 40960
1502 282/300 Read block 3440 len 36864
1503 283/300 Write block 3508 len 4096 data: 4
1504 284/300 Read block 102 len 28672
1505 285/300 Read block 1435 len 32768
1506 286/300 Flush
1507 287/300 Write block 2050 len 16384 data: 2 2 2 2
1508 288/300 Read block 4119 len 16384
1509 289/300 Read block 1052 len 12288
1510 290/300 Write block 1339 len 24576 data: 2 2 2 2 2 2
1511 291/300 Read block 1855 len 24576
1512 292/300 Read block 216 len 40960
1513 293/300 Read block 4428 len 40960
1514 294/300 Write block 2468 len 24576 data: 2 2 2 2 2 2
1515 295/300 Read block 2096 len 20480
1516 296/300 Read block 1041 len 24576
1517 297/300 Write block 814 len 4096 data: 3
1518 298/300 Write block 733 len 16384 data: 2 2 2 2
1519 299/300 Write block 2386 len 16384 data: 2 2 2 2
1520 300/300 Read block 2264 len 20480
1521 301/300 Read block 4038 len 4096
1522 Main task has work to do, go find it
1523 got dsc Start(0)
1524 start 0
1525 Nov 28 04:50:39.488 INFO request completed, latency_us: 217, response_code: 204, uri: /start/cid/0, method: POST, req_id: 4d8a9a2a-da8a-4e0e-bd81-ef65e7e7840d, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
1526 Replay: started 0, returned:Ok(())
1527 ----------------------------------------------------------------
1528 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
1529 States: Offline Active Active
1530 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
1531 1302 Acked Write 4 Sent Done Done false
1532 1303 Acked Write 1 Sent Done Done false
1533 1304 Acked Read 1 Sent Done Done false
1534 1305 Acked Read 3 Sent Done Done false
1535 1306 Acked Write 2 Sent Done Done false
1536 1307 Acked Read 1 Sent Done Done false
1537 1308 Acked Read 1 Sent Done Done false
1538 1309 Acked Write 5 Sent Done Done false
1539 1310 Acked Write 2 Sent Done Done false
1540 1311 Acked Write 4 Sent Done Done false
1541 1312 Acked Read 3 Sent Done Done false
1542 1313 Acked Write 10 Sent Done Done false
1543 1314 Acked Write 5 Sent Done Done false
1544 1315 Acked Read 5 Sent Done Done false
1545 1316 Acked Write 1 Sent Done Done false
1546 1317 Acked Read 10 Sent Done Done false
1547 1318 Acked Write 9 Sent Done Done false
1548 1319 Acked Read 8 Sent Done Done false
1549 1320 Acked Read 10 Sent Done Done false
1550 1321 Acked Write 7 Sent Done Done false
1551 1322 Acked Read 8 Sent Done Done false
1552 1323 Acked Read 6 Sent Done Done false
1553 1324 Acked Write 1 Sent Done Done false
1554 1325 Acked Read 9 Sent Done Done false
1555 1326 Acked Write 6 Sent Done Done false
1556 1327 Acked Write 1 Sent Done Done false
1557 1328 Acked Write 2 Sent Done Done false
1558 1329 Acked Write 2 Sent Done Done false
1559 1330 Acked Write 6 Sent Done Done false
1560 1331 Acked Write 3 Sent Done Done false
1561 1332 Acked Write 8 Sent Done Done false
1562 1333 Acked Write 4 Sent Done Done false
1563 1334 Acked Read 9 Sent Done Done false
1564 1335 Acked Read 5 Sent Done Done false
1565 1336 Acked Read 1 Sent Done Done false
1566 1337 Acked Read 6 Sent Done Done false
1567 1338 Acked Read 7 Sent Done Done false
1568 1339 Acked Write 9 Sent Done Done false
1569 1340 Acked Write 1 Sent Done Done false
1570 1341 Acked Write 3 Sent Done Done false
1571 1342 Acked Read 1 Sent Done Done false
1572 1343 Acked Flush 0 Sent Done Done false
1573 1344 Acked Read 4 Sent Done Done false
1574 1345 Acked Read 8 Sent Done Done false
1575 1346 Acked Flush 0 Sent Done Done false
1576 1347 Acked Read 10 Sent Done Done false
1577 1348 Acked Read 2 Sent Done Done false
1578 1349 Acked Read 6 Sent Done Done false
1579 1350 Acked Write 8 Sent Done Done false
1580 1351 Acked Write 4 Sent Done Done false
1581 1352 Acked Flush 0 Sent Done Done false
1582 1353 Acked Write 10 Sent Done Done false
1583 1354 Acked Write 8 Sent Done Done false
1584 1355 Acked Write 9 Sent Done Done false
1585 1356 Acked Read 2 Sent Done Done false
1586 1357 Acked Read 1 Sent Done Done false
1587 1358 Acked Flush 0 Sent Done Done false
1588 1359 Acked Read 9 Sent Done Done false
1589 1360 Acked Write 10 Sent Done Done false
1590 1361 Acked Read 9 Sent Done Done false
1591 1362 Acked Write 10 Sent Done Done false
1592 1363 Acked Read 5 Sent Done Done false
1593 1364 Acked Flush 0 Sent Done Done false
1594 1365 Acked Read 1 Sent Done Done false
1595 1366 Acked Read 1 Sent Done Done false
1596 1367 Acked Write 6 Sent Done Done false
1597 1368 Acked Read 8 Sent Done Done false
1598 1369 Acked Write 4 Sent Done Done false
1599 1370 Acked Write 9 Sent Done Done false
1600 1371 Acked Flush 0 Sent Done Done false
1601 1372 Acked Read 3 Sent Done Done false
1602 1373 Acked Read 6 Sent Done Done false
1603 1374 Acked Read 8 Sent Done Done false
1604 1375 Acked Read 8 Sent Done Done false
1605 1376 Acked Write 1 Sent Done Done false
1606 1377 Acked Write 7 Sent Done Done false
1607 1378 Acked Read 5 Sent Done Done false
1608 1379 Acked Write 7 Sent Done Done false
1609 1380 Acked Write 5 Sent Done Done false
1610 1381 Acked Flush 0 Sent Done Done false
1611 1382 Acked Read 5 Sent Done Done false
1612 1383 Acked Flush 0 Sent Done Done false
1613 1384 Acked Write 2 Sent Done Done false
1614 1385 Acked Write 5 Sent Done Done false
1615 1386 Acked Flush 0 Sent Done Done false
1616 1387 Acked Write 2 Sent Done Done false
1617 1388 Acked Write 10 Sent Done Done false
1618 1389 Acked Read 10 Sent Done Done false
1619 1390 Acked Read 2 Sent Done Done false
1620 1391 Acked Write 3 Sent Done Done false
1621 1392 Acked Read 8 Sent Done Done false
1622 1393 Acked Flush 0 Sent Done Done false
1623 1394 Acked Read 2 Sent Done Done false
1624 1395 Acked Write 7 Sent Done Done false
1625 1396 Acked Read 7 Sent Done Done false
1626 1397 Acked Write 10 Sent Done Done false
1627 1398 Acked Write 9 Sent Done Done false
1628 1399 Acked Write 1 Sent Done Done false
1629 1400 Acked Flush 0 Sent Done Done false
1630 1401 Acked Write 8 Sent Done Done false
1631 1402 Acked Write 3 Sent Done Done false
1632 1403 Acked Read 10 Sent Done Done false
1633 1404 Acked Write 1 Sent Done Done false
1634 1405 Acked Read 10 Sent Done Done false
1635 1406 Acked Read 10 Sent Done Done false
1636 1407 Acked Write 1 Sent Done Done false
1637 1408 Acked Flush 0 Sent Done Done false
1638 1409 Acked Write 4 Sent Done Done false
1639 1410 Acked Read 4 Sent Done Done false
1640 1411 Acked Read 2 Sent Done Done false
1641 1412 Acked Read 3 Sent Done Done false
1642 1413 Acked Read 5 Sent Done Done false
1643 1414 Acked Read 6 Sent Done Done false
1644 1415 Acked Write 2 Sent Done Done false
1645 1416 Acked Write 1 Sent Done Done false
1646 1417 Acked Read 4 Sent Done Done false
1647 1418 Acked Write 8 Sent Done Done false
1648 1419 Acked Write 1 Sent Done Done false
1649 1420 Acked Write 8 Sent Done Done false
1650 1421 Acked Write 2 Sent Done Done false
1651 1422 Acked Flush 0 Sent Done Done false
1652 1423 Acked Write 9 Sent Done Done false
1653 1424 Acked Read 2 Sent Done Done false
1654 1425 Acked Write 8 Sent Done Done false
1655 1426 Acked Write 10 Sent Done Done false
1656 1427 Acked Read 8 Sent Done Done false
1657 1428 Acked Read 9 Sent Done Done false
1658 1429 Acked Flush 0 Sent Done Done false
1659 1430 Acked Read 7 Sent Done Done false
1660 1431 Acked Flush 0 Sent Done Done false
1661 1432 Acked Read 9 Sent Done Done false
1662 1433 Acked Read 1 Sent Done Done false
1663 1434 Acked Flush 0 Sent Done Done false
1664 1435 Acked Flush 0 Sent Done Done false
1665 1436 Acked Write 2 Sent Done Done false
1666 1437 Acked Read 4 Sent Done Done false
1667 1438 Acked Write 7 Sent Done Done false
1668 1439 Acked Read 2 Sent Done Done false
1669 1440 Acked Write 4 Sent Done Done false
1670 1441 Acked Read 1 Sent Done Done false
1671 1442 Acked Read 6 Sent Done Done false
1672 1443 Acked Read 9 Sent Done Done false
1673 1444 Acked Read 8 Sent Done Done false
1674 1445 Acked Read 4 Sent Done Done false
1675 1446 Acked Read 4 Sent Done Done false
1676 1447 Acked Write 9 Sent Done Done false
1677 1448 Acked Write 8 Sent Done Done false
1678 1449 Acked Read 10 Sent Done Done false
1679 1450 Acked Read 8 Sent Done Done false
1680 1451 Acked Write 10 Sent Done Done false
1681 1452 Acked Read 3 Sent Done Done false
1682 1453 Acked Read 7 Sent Done Done false
1683 1454 Acked Flush 0 Sent Done Done false
1684 1455 Acked Read 8 Sent Done Done false
1685 1456 Acked Read 8 Sent Done Done false
1686 1457 Acked Write 1 Sent Done Done false
1687 1458 Acked Flush 0 Sent Done Done false
1688 1459 Acked Write 9 Sent Done Done false
1689 1460 Acked Write 2 Sent Done Done false
1690 1461 Acked Read 5 Sent Done Done false
1691 1462 Acked Read 6 Sent Done Done false
1692 1463 Acked Read 7 Sent Done Done false
1693 1464 Acked Read 4 Sent Done Done false
1694 1465 Acked Read 2 Sent Done Done false
1695 1466 Acked Write 3 Sent Done Done false
1696 1467 Acked Read 7 Sent Done Done false
1697 1468 Acked Read 9 Sent Done Done false
1698 1469 Acked Read 5 Sent Done Done false
1699 1470 Acked Read 6 Sent Done Done false
1700 1471 Acked Read 4 Sent Done Done false
1701 1472 Acked Read 3 Sent Done Done false
1702 1473 Acked Write 3 Sent Done Done false
1703 1474 Acked Write 10 Sent Done Done false
1704 1475 Acked Read 1 Sent Done Done false
1705 1476 Acked Write 6 Sent Done Done false
1706 1477 Acked Write 8 Sent Done Done false
1707 1478 Acked Write 9 Sent Done Done false
1708 1479 Acked Read 7 Sent Done Done false
1709 1480 Acked Read 7 Sent Done Done false
1710 1481 Acked Read 2 Sent Done Done false
1711 1482 Acked Write 10 Sent Done Done false
1712 1483 Acked Read 2 Sent Done Done false
1713 1484 Acked Read 3 Sent Done Done false
1714 1485 Acked Read 4 Sent Done Done false
1715 1486 Acked Flush 0 Sent Done Done false
1716 1487 Acked Write 5 Sent Done Done false
1717 1488 Acked Write 2 Sent Done Done false
1718 1489 Acked Read 9 Sent Done Done false
1719 1490 Acked Read 4 Sent Done Done false
1720 1491 Acked Write 2 Sent Done Done false
1721 1492 Acked Read 3 Sent Done Done false
1722 1493 Acked Write 5 Sent Done Done false
1723 1494 Acked Read 10 Sent Done Done false
1724 1495 Acked Read 8 Sent Done Done false
1725 1496 Acked Flush 0 Sent Done Done false
1726 1497 Acked Read 2 Sent Done Done false
1727 1498 Acked Flush 0 Sent Done Done false
1728 1499 Acked Write 6 Sent Done Done false
1729 1500 Acked Read 10 Sent Done Done false
1730 1501 Acked Read 7 Sent Done Done false
1731 1502 Acked Write 7 Sent Done Done false
1732 1503 Acked Read 9 Sent Done Done false
1733 1504 Acked Flush 0 Sent Done Done false
1734 1505 Acked Flush 0 Sent Done Done false
1735 1506 Acked Flush 0 Sent Done Done false
1736 1507 Acked Flush 0 Sent Done Done false
1737 1508 Acked Write 10 Sent Done Done false
1738 1509 Acked Read 8 Sent Done Done false
1739 1510 Acked Flush 0 Sent Done Done false
1740 1511 Acked Read 4 Sent Done Done false
1741 1512 Acked Read 8 Sent Done Done false
1742 1513 Acked Read 1 Sent Done Done false
1743 1514 Acked Read 10 Sent Done Done false
1744 1515 Acked Write 1 Sent Done Done false
1745 1516 Acked Read 9 Sent Done Done false
1746 1517 Acked Write 7 Sent Done Done false
1747 1518 Acked Read 7 Sent Done Done false
1748 1519 Acked Read 2 Sent Done Done false
1749 1520 Acked Flush 0 Sent Done Done false
1750 1521 Acked Flush 0 Sent Done Done false
1751 1522 Acked Flush 0 Sent Done Done false
1752 1523 Acked Write 7 Sent Done Done false
1753 1524 Acked Read 2 Sent Done Done false
1754 1525 Acked Write 5 Sent Done Done false
1755 1526 Acked Read 7 Sent Done Done false
1756 1527 Acked Read 5 Sent Done Done false
1757 1528 Acked Read 2 Sent Done Done false
1758 1529 Acked Write 8 Sent Done Done false
1759 1530 Acked Read 10 Sent Done Done false
1760 1531 Acked Read 8 Sent Done Done false
1761 1532 Acked Read 7 Sent Done Done false
1762 1533 Acked Read 6 Sent Done Done false
1763 1534 Acked Read 1 Sent Done Done false
1764 1535 Acked Read 9 Sent Done Done false
1765 1536 Acked Write 6 Sent Done Done false
1766 1537 Acked Write 6 Sent Done Done false
1767 1538 Acked Write 10 Sent Done Done false
1768 1539 Acked Write 8 Sent Done Done false
1769 1540 Acked Write 9 Sent Done Done false
1770 1541 Acked Read 7 Sent Done Done false
1771 1542 Acked Read 6 Sent Done Done false
1772 1543 Acked Read 1 Sent Done Done false
1773 1544 Acked Write 8 Sent Done Done false
1774 1545 Acked Read 4 Sent Done Done false
1775 1546 Acked Write 7 Sent Done Done false
1776 1547 Acked Write 8 Sent Done Done false
1777 1548 Acked Read 8 Sent Done Done false
1778 1549 Acked Write 9 Sent Done Done false
1779 1550 Acked Write 5 Sent Done Done false
1780 1551 Acked Read 4 Sent Done Done false
1781 1552 Acked Flush 0 Sent Done Done false
1782 1553 Acked Read 10 Sent Done Done false
1783 1554 Acked Write 8 Sent Done Done false
1784 1555 Acked Write 2 Sent Done Done false
1785 1556 Acked Write 1 Sent Done Done false
1786 1557 Acked Flush 0 Sent Done Done false
1787 1558 Acked Read 8 Sent Done Done false
1788 1559 Acked Read 7 Sent Done Done false
1789 1560 Acked Write 1 Sent Done Done false
1790 1561 Acked Write 10 Sent Done Done false
1791 1562 Acked Read 10 Sent Done Done false
1792 1563 Acked Write 3 Sent Done Done false
1793 1564 Acked Flush 0 Sent Done Done false
1794 1565 Acked Read 5 Sent Done Done false
1795 1566 Acked Write 5 Sent Done Done false
1796 1567 Acked Write 7 Sent Done Done false
1797 1568 Acked Flush 0 Sent Done Done false
1798 1569 Acked Read 5 Sent Done Done false
1799 1570 Acked Write 3 Sent Done Done false
1800 1571 Acked Read 10 Sent Done Done false
1801 1572 Acked Read 2 Sent Done Done false
1802 1573 Acked Read 5 Sent Done Done false
1803 1574 Acked Write 6 Sent Done Done false
1804 1575 Acked Write 4 Sent Done Done false
1805 1576 Acked Read 8 Sent Done Done false
1806 1577 Acked Read 5 Sent Done Done false
1807 1578 Acked Read 9 Sent Done Done false
1808 1579 Acked Read 10 Sent Done Done false
1809 1580 Acked Write 5 Sent Done Done false
1810 1581 Acked Write 5 Sent Done Done false
1811 1582 Acked Read 10 Sent Done Done false
1812 1583 Acked Read 9 Sent Done Done false
1813 1584 Acked Write 1 Sent Done Done false
1814 1585 Acked Read 7 Sent Done Done false
1815 1586 Acked Read 8 Sent Done Done false
1816 1587 Acked Flush 0 Sent Done Done false
1817 1588 Acked Write 4 Sent Done Done false
1818 1589 Acked Read 4 Sent Done Done false
1819 1590 Acked Read 3 Sent Done Done false
1820 1591 Acked Write 6 Sent Done Done false
1821 1592 Acked Read 6 Sent Done Done false
1822 1593 Acked Read 10 Sent Done Done false
1823 1594 Acked Read 10 Sent Done Done false
1824 1595 Acked Write 6 Sent Done Done false
1825 1596 Acked Read 5 Sent Done Done false
1826 1597 Acked Read 6 Sent Done Done false
1827 1598 Acked Write 1 Sent Done Done false
1828 1599 Acked Write 4 Sent Done Done false
1829 1600 Acked Write 4 Sent Done Done false
1830 1601 Acked Read 5 Sent Done Done false
1831 1602 Acked Read 1 Sent Done Done false
1832 STATES DS:0 DS:1 DS:2 TOTAL
1833 Sent 301 0 0 301
1834 Done 0 301 301 602
1835 Skipped 0 0 0 0
1836 Error 0 0 0 0
1837 Last Flush: 1301 1587 1587
1838 Downstairs last five completed: 1301 1300 1299 1298 1297
1839 Upstairs last five completed: 1602 1601 1600 1599 1598
1840 CLIENT: Up:0 ds:301 act:2
1841 [8810] got start action
1842 [8810] I am going to restart
1843 Starting downstairs at port 8810
1844 Make output file at "/tmp/dsc/downstairs-8810.txt"
1845 [8810][0] reports Starting
1846 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1204)
1847 [8810][0] reports Running
1848 ----------------------------------------------------------------
1849 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
1850 States: Offline Active Active
1851 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
1852 1302 Acked Write 4 Sent Done Done false
1853 1303 Acked Write 1 Sent Done Done false
1854 1304 Acked Read 1 Sent Done Done false
1855 1305 Acked Read 3 Sent Done Done false
1856 1306 Acked Write 2 Sent Done Done false
1857 1307 Acked Read 1 Sent Done Done false
1858 1308 Acked Read 1 Sent Done Done false
1859 1309 Acked Write 5 Sent Done Done false
1860 1310 Acked Write 2 Sent Done Done false
1861 1311 Acked Write 4 Sent Done Done false
1862 1312 Acked Read 3 Sent Done Done false
1863 1313 Acked Write 10 Sent Done Done false
1864 1314 Acked Write 5 Sent Done Done false
1865 1315 Acked Read 5 Sent Done Done false
1866 1316 Acked Write 1 Sent Done Done false
1867 1317 Acked Read 10 Sent Done Done false
1868 1318 Acked Write 9 Sent Done Done false
1869 1319 Acked Read 8 Sent Done Done false
1870 1320 Acked Read 10 Sent Done Done false
1871 1321 Acked Write 7 Sent Done Done false
1872 1322 Acked Read 8 Sent Done Done false
1873 1323 Acked Read 6 Sent Done Done false
1874 1324 Acked Write 1 Sent Done Done false
1875 1325 Acked Read 9 Sent Done Done false
1876 1326 Acked Write 6 Sent Done Done false
1877 1327 Acked Write 1 Sent Done Done false
1878 1328 Acked Write 2 Sent Done Done false
1879 1329 Acked Write 2 Sent Done Done false
1880 1330 Acked Write 6 Sent Done Done false
1881 1331 Acked Write 3 Sent Done Done false
1882 1332 Acked Write 8 Sent Done Done false
1883 1333 Acked Write 4 Sent Done Done false
1884 1334 Acked Read 9 Sent Done Done false
1885 1335 Acked Read 5 Sent Done Done false
1886 1336 Acked Read 1 Sent Done Done false
1887 1337 Acked Read 6 Sent Done Done false
1888 1338 Acked Read 7 Sent Done Done false
1889 1339 Acked Write 9 Sent Done Done false
1890 1340 Acked Write 1 Sent Done Done false
1891 1341 Acked Write 3 Sent Done Done false
1892 1342 Acked Read 1 Sent Done Done false
1893 1343 Acked Flush 0 Sent Done Done false
1894 1344 Acked Read 4 Sent Done Done false
1895 1345 Acked Read 8 Sent Done Done false
1896 1346 Acked Flush 0 Sent Done Done false
1897 1347 Acked Read 10 Sent Done Done false
1898 1348 Acked Read 2 Sent Done Done false
1899 1349 Acked Read 6 Sent Done Done false
1900 1350 Acked Write 8 Sent Done Done false
1901 1351 Acked Write 4 Sent Done Done false
1902 1352 Acked Flush 0 Sent Done Done false
1903 1353 Acked Write 10 Sent Done Done false
1904 1354 Acked Write 8 Sent Done Done false
1905 1355 Acked Write 9 Sent Done Done false
1906 1356 Acked Read 2 Sent Done Done false
1907 1357 Acked Read 1 Sent Done Done false
1908 1358 Acked Flush 0 Sent Done Done false
1909 1359 Acked Read 9 Sent Done Done false
1910 1360 Acked Write 10 Sent Done Done false
1911 1361 Acked Read 9 Sent Done Done false
1912 1362 Acked Write 10 Sent Done Done false
1913 1363 Acked Read 5 Sent Done Done false
1914 1364 Acked Flush 0 Sent Done Done false
1915 1365 Acked Read 1 Sent Done Done false
1916 1366 Acked Read 1 Sent Done Done false
1917 1367 Acked Write 6 Sent Done Done false
1918 1368 Acked Read 8 Sent Done Done false
1919 1369 Acked Write 4 Sent Done Done false
1920 1370 Acked Write 9 Sent Done Done false
1921 1371 Acked Flush 0 Sent Done Done false
1922 1372 Acked Read 3 Sent Done Done false
1923 1373 Acked Read 6 Sent Done Done false
1924 1374 Acked Read 8 Sent Done Done false
1925 1375 Acked Read 8 Sent Done Done false
1926 1376 Acked Write 1 Sent Done Done false
1927 1377 Acked Write 7 Sent Done Done false
1928 1378 Acked Read 5 Sent Done Done false
1929 1379 Acked Write 7 Sent Done Done false
1930 1380 Acked Write 5 Sent Done Done false
1931 1381 Acked Flush 0 Sent Done Done false
1932 1382 Acked Read 5 Sent Done Done false
1933 1383 Acked Flush 0 Sent Done Done false
1934 1384 Acked Write 2 Sent Done Done false
1935 1385 Acked Write 5 Sent Done Done false
1936 1386 Acked Flush 0 Sent Done Done false
1937 1387 Acked Write 2 Sent Done Done false
1938 1388 Acked Write 10 Sent Done Done false
1939 1389 Acked Read 10 Sent Done Done false
1940 1390 Acked Read 2 Sent Done Done false
1941 1391 Acked Write 3 Sent Done Done false
1942 1392 Acked Read 8 Sent Done Done false
1943 1393 Acked Flush 0 Sent Done Done false
1944 1394 Acked Read 2 Sent Done Done false
1945 1395 Acked Write 7 Sent Done Done false
1946 1396 Acked Read 7 Sent Done Done false
1947 1397 Acked Write 10 Sent Done Done false
1948 1398 Acked Write 9 Sent Done Done false
1949 1399 Acked Write 1 Sent Done Done false
1950 1400 Acked Flush 0 Sent Done Done false
1951 1401 Acked Write 8 Sent Done Done false
1952 1402 Acked Write 3 Sent Done Done false
1953 1403 Acked Read 10 Sent Done Done false
1954 1404 Acked Write 1 Sent Done Done false
1955 1405 Acked Read 10 Sent Done Done false
1956 1406 Acked Read 10 Sent Done Done false
1957 1407 Acked Write 1 Sent Done Done false
1958 1408 Acked Flush 0 Sent Done Done false
1959 1409 Acked Write 4 Sent Done Done false
1960 1410 Acked Read 4 Sent Done Done false
1961 1411 Acked Read 2 Sent Done Done false
1962 1412 Acked Read 3 Sent Done Done false
1963 1413 Acked Read 5 Sent Done Done false
1964 1414 Acked Read 6 Sent Done Done false
1965 1415 Acked Write 2 Sent Done Done false
1966 1416 Acked Write 1 Sent Done Done false
1967 1417 Acked Read 4 Sent Done Done false
1968 1418 Acked Write 8 Sent Done Done false
1969 1419 Acked Write 1 Sent Done Done false
1970 1420 Acked Write 8 Sent Done Done false
1971 1421 Acked Write 2 Sent Done Done false
1972 1422 Acked Flush 0 Sent Done Done false
1973 1423 Acked Write 9 Sent Done Done false
1974 1424 Acked Read 2 Sent Done Done false
1975 1425 Acked Write 8 Sent Done Done false
1976 1426 Acked Write 10 Sent Done Done false
1977 1427 Acked Read 8 Sent Done Done false
1978 1428 Acked Read 9 Sent Done Done false
1979 1429 Acked Flush 0 Sent Done Done false
1980 1430 Acked Read 7 Sent Done Done false
1981 1431 Acked Flush 0 Sent Done Done false
1982 1432 Acked Read 9 Sent Done Done false
1983 1433 Acked Read 1 Sent Done Done false
1984 1434 Acked Flush 0 Sent Done Done false
1985 1435 Acked Flush 0 Sent Done Done false
1986 1436 Acked Write 2 Sent Done Done false
1987 1437 Acked Read 4 Sent Done Done false
1988 1438 Acked Write 7 Sent Done Done false
1989 1439 Acked Read 2 Sent Done Done false
1990 1440 Acked Write 4 Sent Done Done false
1991 1441 Acked Read 1 Sent Done Done false
1992 1442 Acked Read 6 Sent Done Done false
1993 1443 Acked Read 9 Sent Done Done false
1994 1444 Acked Read 8 Sent Done Done false
1995 1445 Acked Read 4 Sent Done Done false
1996 1446 Acked Read 4 Sent Done Done false
1997 1447 Acked Write 9 Sent Done Done false
1998 1448 Acked Write 8 Sent Done Done false
1999 1449 Acked Read 10 Sent Done Done false
2000 1450 Acked Read 8 Sent Done Done false
2001 1451 Acked Write 10 Sent Done Done false
2002 1452 Acked Read 3 Sent Done Done false
2003 1453 Acked Read 7 Sent Done Done false
2004 1454 Acked Flush 0 Sent Done Done false
2005 1455 Acked Read 8 Sent Done Done false
2006 1456 Acked Read 8 Sent Done Done false
2007 1457 Acked Write 1 Sent Done Done false
2008 1458 Acked Flush 0 Sent Done Done false
2009 1459 Acked Write 9 Sent Done Done false
2010 1460 Acked Write 2 Sent Done Done false
2011 1461 Acked Read 5 Sent Done Done false
2012 1462 Acked Read 6 Sent Done Done false
2013 1463 Acked Read 7 Sent Done Done false
2014 1464 Acked Read 4 Sent Done Done false
2015 1465 Acked Read 2 Sent Done Done false
2016 1466 Acked Write 3 Sent Done Done false
2017 1467 Acked Read 7 Sent Done Done false
2018 1468 Acked Read 9 Sent Done Done false
2019 1469 Acked Read 5 Sent Done Done false
2020 1470 Acked Read 6 Sent Done Done false
2021 1471 Acked Read 4 Sent Done Done false
2022 1472 Acked Read 3 Sent Done Done false
2023 1473 Acked Write 3 Sent Done Done false
2024 1474 Acked Write 10 Sent Done Done false
2025 1475 Acked Read 1 Sent Done Done false
2026 1476 Acked Write 6 Sent Done Done false
2027 1477 Acked Write 8 Sent Done Done false
2028 1478 Acked Write 9 Sent Done Done false
2029 1479 Acked Read 7 Sent Done Done false
2030 1480 Acked Read 7 Sent Done Done false
2031 1481 Acked Read 2 Sent Done Done false
2032 1482 Acked Write 10 Sent Done Done false
2033 1483 Acked Read 2 Sent Done Done false
2034 1484 Acked Read 3 Sent Done Done false
2035 1485 Acked Read 4 Sent Done Done false
2036 1486 Acked Flush 0 Sent Done Done false
2037 1487 Acked Write 5 Sent Done Done false
2038 1488 Acked Write 2 Sent Done Done false
2039 1489 Acked Read 9 Sent Done Done false
2040 1490 Acked Read 4 Sent Done Done false
2041 1491 Acked Write 2 Sent Done Done false
2042 1492 Acked Read 3 Sent Done Done false
2043 1493 Acked Write 5 Sent Done Done false
2044 1494 Acked Read 10 Sent Done Done false
2045 1495 Acked Read 8 Sent Done Done false
2046 1496 Acked Flush 0 Sent Done Done false
2047 1497 Acked Read 2 Sent Done Done false
2048 1498 Acked Flush 0 Sent Done Done false
2049 1499 Acked Write 6 Sent Done Done false
2050 1500 Acked Read 10 Sent Done Done false
2051 1501 Acked Read 7 Sent Done Done false
2052 1502 Acked Write 7 Sent Done Done false
2053 1503 Acked Read 9 Sent Done Done false
2054 1504 Acked Flush 0 Sent Done Done false
2055 1505 Acked Flush 0 Sent Done Done false
2056 1506 Acked Flush 0 Sent Done Done false
2057 1507 Acked Flush 0 Sent Done Done false
2058 1508 Acked Write 10 Sent Done Done false
2059 1509 Acked Read 8 Sent Done Done false
2060 1510 Acked Flush 0 Sent Done Done false
2061 1511 Acked Read 4 Sent Done Done false
2062 1512 Acked Read 8 Sent Done Done false
2063 1513 Acked Read 1 Sent Done Done false
2064 1514 Acked Read 10 Sent Done Done false
2065 1515 Acked Write 1 Sent Done Done false
2066 1516 Acked Read 9 Sent Done Done false
2067 1517 Acked Write 7 Sent Done Done false
2068 1518 Acked Read 7 Sent Done Done false
2069 1519 Acked Read 2 Sent Done Done false
2070 1520 Acked Flush 0 Sent Done Done false
2071 1521 Acked Flush 0 Sent Done Done false
2072 1522 Acked Flush 0 Sent Done Done false
2073 1523 Acked Write 7 Sent Done Done false
2074 1524 Acked Read 2 Sent Done Done false
2075 1525 Acked Write 5 Sent Done Done false
2076 1526 Acked Read 7 Sent Done Done false
2077 1527 Acked Read 5 Sent Done Done false
2078 1528 Acked Read 2 Sent Done Done false
2079 1529 Acked Write 8 Sent Done Done false
2080 1530 Acked Read 10 Sent Done Done false
2081 1531 Acked Read 8 Sent Done Done false
2082 1532 Acked Read 7 Sent Done Done false
2083 1533 Acked Read 6 Sent Done Done false
2084 1534 Acked Read 1 Sent Done Done false
2085 1535 Acked Read 9 Sent Done Done false
2086 1536 Acked Write 6 Sent Done Done false
2087 1537 Acked Write 6 Sent Done Done false
2088 1538 Acked Write 10 Sent Done Done false
2089 1539 Acked Write 8 Sent Done Done false
2090 1540 Acked Write 9 Sent Done Done false
2091 1541 Acked Read 7 Sent Done Done false
2092 1542 Acked Read 6 Sent Done Done false
2093 1543 Acked Read 1 Sent Done Done false
2094 1544 Acked Write 8 Sent Done Done false
2095 1545 Acked Read 4 Sent Done Done false
2096 1546 Acked Write 7 Sent Done Done false
2097 1547 Acked Write 8 Sent Done Done false
2098 1548 Acked Read 8 Sent Done Done false
2099 1549 Acked Write 9 Sent Done Done false
2100 1550 Acked Write 5 Sent Done Done false
2101 1551 Acked Read 4 Sent Done Done false
2102 1552 Acked Flush 0 Sent Done Done false
2103 1553 Acked Read 10 Sent Done Done false
2104 1554 Acked Write 8 Sent Done Done false
2105 1555 Acked Write 2 Sent Done Done false
2106 1556 Acked Write 1 Sent Done Done false
2107 1557 Acked Flush 0 Sent Done Done false
2108 1558 Acked Read 8 Sent Done Done false
2109 1559 Acked Read 7 Sent Done Done false
2110 1560 Acked Write 1 Sent Done Done false
2111 1561 Acked Write 10 Sent Done Done false
2112 1562 Acked Read 10 Sent Done Done false
2113 1563 Acked Write 3 Sent Done Done false
2114 1564 Acked Flush 0 Sent Done Done false
2115 1565 Acked Read 5 Sent Done Done false
2116 1566 Acked Write 5 Sent Done Done false
2117 1567 Acked Write 7 Sent Done Done false
2118 1568 Acked Flush 0 Sent Done Done false
2119 1569 Acked Read 5 Sent Done Done false
2120 1570 Acked Write 3 Sent Done Done false
2121 1571 Acked Read 10 Sent Done Done false
2122 1572 Acked Read 2 Sent Done Done false
2123 1573 Acked Read 5 Sent Done Done false
2124 1574 Acked Write 6 Sent Done Done false
2125 1575 Acked Write 4 Sent Done Done false
2126 1576 Acked Read 8 Sent Done Done false
2127 1577 Acked Read 5 Sent Done Done false
2128 1578 Acked Read 9 Sent Done Done false
2129 1579 Acked Read 10 Sent Done Done false
2130 1580 Acked Write 5 Sent Done Done false
2131 1581 Acked Write 5 Sent Done Done false
2132 1582 Acked Read 10 Sent Done Done false
2133 1583 Acked Read 9 Sent Done Done false
2134 1584 Acked Write 1 Sent Done Done false
2135 1585 Acked Read 7 Sent Done Done false
2136 1586 Acked Read 8 Sent Done Done false
2137 1587 Acked Flush 0 Sent Done Done false
2138 1588 Acked Write 4 Sent Done Done false
2139 1589 Acked Read 4 Sent Done Done false
2140 1590 Acked Read 3 Sent Done Done false
2141 1591 Acked Write 6 Sent Done Done false
2142 1592 Acked Read 6 Sent Done Done false
2143 1593 Acked Read 10 Sent Done Done false
2144 1594 Acked Read 10 Sent Done Done false
2145 1595 Acked Write 6 Sent Done Done false
2146 1596 Acked Read 5 Sent Done Done false
2147 1597 Acked Read 6 Sent Done Done false
2148 1598 Acked Write 1 Sent Done Done false
2149 1599 Acked Write 4 Sent Done Done false
2150 1600 Acked Write 4 Sent Done Done false
2151 1601 Acked Read 5 Sent Done Done false
2152 1602 Acked Read 1 Sent Done Done false
2153 1603 Acked Flush 0 Sent Done Done false
2154 STATES DS:0 DS:1 DS:2 TOTAL
2155 Sent 302 0 0 302
2156 Done 0 302 302 604
2157 Skipped 0 0 0 0
2158 Error 0 0 0 0
2159 Last Flush: 1301 1603 1603
2160 Downstairs last five completed: 1301 1300 1299 1298 1297
2161 Upstairs last five completed: 1603 1602 1601 1600 1599
2162 CLIENT: Up:0 ds:302 act:2
21632024-11-28T04:50:45.252ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
21642024-11-28T04:50:45.252ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:42218) = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
21652024-11-28T04:50:45.253ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
21662024-11-28T04:50:45.253ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
21672024-11-28T04:50:45.254ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
21682024-11-28T04:50:45.254ZINFOcrucible: send last flush ID to this DS: 1301 = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
21692024-11-28T04:50:45.254ZINFOcrucible: Replied this last flush ID: 1301 = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
21702024-11-28T04:50:45.254ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
21712024-11-28T04:50:45.254ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
21722024-11-28T04:50:45.254ZINFOcrucible: [0] client re-new 302 jobs since flush 1301 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
21732024-11-28T04:50:45.254ZINFOcrucible: [0] Replayed 302 jobs since flush: 1301 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
2174 ----------------------------------------------------------------
2175 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
2176 Downstairs last five completed: 1603 1602 1601 1600 1599
2177 Upstairs last five completed: 1603 1602 1601 1600 1599
2178 CLIENT: Up:0 ds:0 act:3
2179 Replay: All jobs finished
2180 Main task has work to do, go find it
2181 got dsc Stop(0)
2182 stop 0
2183 Nov 28 04:50:47.494 INFO request completed, latency_us: 332, response_code[8810] Got stop action so:false kr:false
2184 : 204, uri: /stop/cid/0, method: POST, req_id: bc3b2399-c1c4-4bad-9389-d4e9d0e465bd, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
21852024-11-28T04:50:47.494ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
21862024-11-28T04:50:47.494ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
21872024-11-28T04:50:47.494ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
21882024-11-28T04:50:47.494ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
21892024-11-28T04:50:47.494ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
2190 Nov 28 04:50:47.494 INFO request completed, latency_us: 183, response_code: 200, uri: /state/cid/0, method: GET, req_id: a76817e4-922f-4e85-bf77-ca79e81066fb, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
21912024-11-28T04:50:47.495ZINFOcrucible: client task is exiting = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
2192 [8810] Exited with: ExitStatus(unix_wait_status(9))
2193 [8810][0] reports Exit
2194 Nov 28 04:50:51.496 INFO request completed, latency_us: 208, response_code: 200, uri: /state/cid/0, method: GET, req_id: 2dc6d678-eda3-4c32-88a9-d1a961d02a9a, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
2195 001/300 Read block 1349 len 36864
2196 002/300 Flush
2197 003/300 Read block 4275 len 20480
2198 004/300 Write block 4118 len 20480 data: 2 2 2 2 3
2199 005/300 Write block 2374 len 36864 data: 3 3 3 2 3 3 3 3 3
2200 006/300 Write block 3604 len 24576 data: 2 2 2 2 2 2
2201 007/300 Read block 2 len 28672
2202 008/300 Write block 2786 len 20480 data: 3 3 3 3 3
2203 009/300 Read block 2001 len 16384
2204 010/300 Write block 2227 len 8192 data: 2 2
2205 011/300 Read block 4411 len 16384
2206 012/300 Read block 1599 len 12288
2207 013/300 Write block 4259 len 8192 data: 2 2
2208 014/300 Write block 1124 len 8192 data: 3 3
2209 015/300 Flush
2210 016/300 Flush
2211 017/300 Read block 1646 len 8192
2212 018/300 Write block 2923 len 8192 data: 4 4
2213 019/300 Read block 3195 len 32768
2214 020/300 Write block 2083 len 36864 data: 2 2 2 2 2 2 2 2 2
2215 021/300 Read block 4345 len 28672
2216 022/300 Read block 3387 len 36864
2217 023/300 Flush
2218 024/300 Write block 878 len 40960 data: 3 3 3 3 2 2 2 2 2 2
2219 025/300 Write block 2255 len 12288 data: 2 2 2
2220 026/300 Write block 4569 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2221 027/300 Read block 4493 len 16384
2222 028/300 Read block 1668 len 20480
2223 029/300 Write block 4549 len 4096 data: 2
2224 030/300 Write block 4880 len 40960 data: 2 3 3 3 2 2 2 2 2 2
2225 031/300 Write block 2100 len 16384 data: 2 2 2 2
2226 032/300 Read block 1175 len 40960
2227 033/300 Read block 2390 len 32768
2228 034/300 Read block 3967 len 4096
2229 035/300 Read block 2009 len 20480
2230 036/300 Write block 2032 len 32768 data: 2 2 3 3 3 3 2 2
2231 037/300 Write block 1623 len 32768 data: 2 2 2 2 2 2 2 2
2232 038/300 Write block 4266 len 4096 data: 3
2233 039/300 Read block 1486 len 40960
2234 040/300 Write block 3536 len 20480 data: 3 3 2 2 2
2235 041/300 Read block 2363 len 40960
2236 042/300 Write block 4743 len 12288 data: 2 2 2
2237 043/300 Read block 1886 len 40960
2238 044/300 Write block 2058 len 12288 data: 2 2 2
2239 045/300 Write block 4884 len 32768 data: 3 3 3 3 3 3 2 3
2240 046/300 Read block 2507 len 40960
2241 047/300 Read block 4453 len 32768
2242 048/300 Write block 3049 len 24576 data: 2 3 3 3 4 4
2243 049/300 Read block 386 len 8192
2244 050/300 Write block 4768 len 32768 data: 2 2 2 2 2 2 2 2
2245 051/300 Flush
2246 052/300 Flush
2247 053/300 Read block 939 len 8192
2248 054/300 Read block 4094 len 36864
2249 055/300 Write block 2514 len 8192 data: 2 2
2250 056/300 Read block 4188 len 20480
2251 057/300 Write block 4077 len 12288 data: 2 2 3
2252 058/300 Read block 436 len 32768
2253 059/300 Read block 3093 len 8192
2254 060/300 Write block 3116 len 24576 data: 2 2 2 2 2 2
2255 061/300 Write block 1878 len 28672 data: 2 2 2 2 2 2 2
2256 062/300 Read block 987 len 40960
2257 063/300 Read block 2295 len 8192
2258 064/300 Read block 3470 len 36864
2259 065/300 Read block 3542 len 12288
2260 066/300 Read block 3039 len 28672
2261 067/300 Read block 3281 len 32768
2262 068/300 Read block 2425 len 32768
2263 069/300 Write block 4363 len 28672 data: 2 2 2 2 2 2 2
2264 070/300 Read block 2020 len 32768
2265 071/300 Read block 3577 len 28672
2266 072/300 Write block 4596 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2267 073/300 Write block 304 len 28672 data: 2 2 2 2 2 2 2
2268 074/300 Read block 2965 len 12288
2269 075/300 Write block 3650 len 16384 data: 3 3 3 3
2270 076/300 Write block 1040 len 28672 data: 2 2 2 2 3 3 3
2271 077/300 Read block 2341 len 32768
2272 078/300 Write block 4711 len 8192 data: 2 2
2273 079/300 Write block 1250 len 16384 data: 2 2 2 2
2274 080/300 Read block 1905 len 16384
2275 081/300 Write block 676 len 16384 data: 2 2 2 2
2276 082/300 Write block 1966 len 36864 data: 3 3 3 4 4 4 3 3 3
2277 083/300 Write block 4122 len 4096 data: 4
2278 084/300 Read block 2803 len 24576
2279 085/300 Read block 676 len 4096
2280 086/300 Read block 2955 len 28672
2281 087/300 Read block 1303 len 24576
2282 088/300 Read block 3822 len 8192
2283 089/300 Flush
2284 090/300 Write block 469 len 12288 data: 2 2 2
2285 091/300 Write block 685 len 8192 data: 2 2
2286 092/300 Write block 3548 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2287 093/300 Read block 4314 len 32768
2288 094/300 Write block 810 len 24576 data: 2 2 3 3 4 3
2289 095/300 Read block 2496 len 8192
2290 096/300 Flush
2291 097/300 Write block 566 len 4096 data: 3
2292 098/300 Write block 900 len 36864 data: 2 2 2 2 2 2 2 2 2
2293 099/300 Read block 212 len 28672
2294 100/300 Read block 508 len 8192
2295 101/300 Read block 4421 len 20480
2296 102/300 Read block 3828 len 20480
2297 103/300 Write block 2141 len 28672 data: 2 2 2 2 2 2 2
2298 104/300 Write block 4245 len 36864 data: 2 2 2 2 2 2 2 2 2
2299 105/300 Write block 126 len 36864 data: 2 2 2 2 2 2 2 2 2
2300 106/300 Read block 2625 len 4096
2301 107/300 Read block 1298 len 8192
2302 108/300 Write block 862 len 24576 data: 2 2 2 2 2 2
2303 109/300 Read block 3107 len 12288
2304 110/300 Read block 2648 len 32768
2305 111/300 Read block 962 len 12288
2306 112/300 Write block 2519 len 36864 data: 2 2 2 2 2 2 3 3 3
2307 113/300 Read block 2265 len 40960
2308 114/300 Read block 1991 len 40960
2309 115/300 Read block 671 len 28672
2310 116/300 Write block 781 len 32768 data: 2 2 2 2 2 2 2 2
2311 117/300 Read block 4444 len 20480
2312 118/300 Read block 2023 len 28672
2313 119/300 Read block 1069 len 36864
2314 120/300 Read block 22 len 20480
2315 121/300 Write block 4416 len 16384 data: 2 2 2 2
2316 122/300 Read block 4767 len 8192
2317 123/300 Read block 80 len 24576
2318 124/300 Read block 3561 len 20480
2319 125/300 Read block 3880 len 36864
2320 126/300 Read block 240 len 32768
2321 127/300 Read block 1845 len 40960
2322 128/300 Read block 3608 len 20480
2323 129/300 Read block 389 len 32768
2324 130/300 Write block 1830 len 40960 data: 3 3 3 3 3 3 3 2 2 2
2325 131/300 Read block 763 len 28672
2326 132/300 Flush
2327 133/300 Write block 3950 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2328 134/300 Read block 377 len 36864
2329 135/300 Read block 2896 len 32768
2330 136/300 Flush
2331 137/300 Read block 3958 len 4096
2332 138/300 Write block 1210 len 32768 data: 2 2 2 2 2 2 2 2
2333 139/300 Read block 2725 len 32768
2334 140/300 Read block 647 len 8192
2335 141/300 Read block 3970 len 16384
2336 142/300 Read block 1156 len 32768
2337 143/300 Write block 52 len 24576 data: 4 2 2 2 2 2
2338 144/300 Read block 1803 len 28672
2339 145/300 Write block 3239 len 8192 data: 3 3
2340 146/300 Write block 2582 len 24576 data: 2 2 2 2 2 2
2341 147/300 Read block 3077 len 4096
2342 148/300 Write block 1333 len 8192 data: 2 2
2343 149/300 Flush
2344 150/300 Read block 1655 len 24576
2345 151/300 Write block 4634 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2346 152/300 Write block 2209 len 40960 data: 3 3 3 3 3 3 2 2 2 2
2347 153/300 Read block 422 len 8192
2348 154/300 Read block 2472 len 4096
2349 155/300 Read block 3342 len 8192
2350 156/300 Read block 3126 len 40960
2351 157/300 Write block 4923 len 12288 data: 2 2 2
2352 158/300 Read block 4421 len 36864
2353 159/300 Read block 412 len 24576
2354 160/300 Read block 2952 len 8192
2355 161/300 Read block 3236 len 28672
2356 162/300 Read block 3593 len 4096
2357 163/300 Write block 2331 len 32768 data: 3 3 3 3 2 2 2 2
2358 164/300 Write block 3140 len 8192 data: 2 2
2359 165/300 Read block 2893 len 4096
2360 166/300 Read block 2657 len 28672
2361 167/300 Flush
2362 168/300 Write block 3540 len 12288 data: 3 2 2
2363 169/300 Flush
2364 170/300 Write block 554 len 24576 data: 3 3 3 3 2 2
2365 171/300 Write block 1724 len 8192 data: 2 2
2366 172/300 Read block 1655 len 20480
2367 173/300 Read block 4112 len 12288
2368 174/300 Read block 4721 len 16384
2369 175/300 Read block 2385 len 8192
2370 176/300 Write block 3213 len 36864 data: 2 2 2 2 2 2 2 2 2
2371 177/300 Write block 814 len 32768 data: 5 4 3 2 2 2 2 2
2372 178/300 Write block 2109 len 36864 data: 3 3 2 3 3 3 3 3 3
2373 179/300 Flush
2374 180/300 Read block 1903 len 16384
2375 181/300 Write block 1493 len 28672 data: 3 3 3 2 2 2 2
2376 182/300 Read block 4064 len 20480
2377 183/300 Flush
2378 184/300 Write block 4524 len 12288 data: 3 3 2
2379 185/300 Read block 3008 len 4096
2380 186/300 Write block 3436 len 24576 data: 2 2 2 2 2 2
2381 187/300 Flush
2382 188/300 Write block 2408 len 12288 data: 2 2 2
2383 189/300 Read block 3783 len 20480
2384 190/300 Write block 196 len 12288 data: 3 2 2
2385 191/300 Read block 384 len 24576
2386 192/300 Read block 3471 len 28672
2387 193/300 Write block 293 len 24576 data: 3 3 2 2 2 2
2388 194/300 Read block 1822 len 20480
2389 195/300 Read block 1922 len 20480
2390 196/300 Read block 2031 len 8192
2391 197/300 Read block 2262 len 20480
2392 198/300 Write block 3797 len 32768 data: 2 2 2 2 2 2 2 2
2393 199/300 Write block 1243 len 12288 data: 3 3 3
2394 200/300 Read block 4382 len 32768
2395 201/300 Read block 2009 len 12288
2396 202/300 Write block 4871 len 12288 data: 2 2 2
2397 203/300 Read block 3 len 40960
2398 204/300 Write block 1838 len 40960 data: 3 3 2 2 2 2 2 2 2 2
2399 205/300 Write block 4720 len 28672 data: 2 2 2 2 2 2 2
2400 206/300 Flush
2401 207/300 Flush
2402 208/300 Write block 2647 len 28672 data: 2 2 2 2 2 2 2
2403 209/300 Read block 855 len 24576
2404 210/300 Read block 4291 len 28672
2405 211/300 Write block 2618 len 4096 data: 3
2406 212/300 Read block 3445 len 20480
2407 213/300 Read block 4988 len 8192
2408 214/300 Write block 4430 len 16384 data: 2 2 2 2
2409 215/300 Write block 3187 len 36864 data: 2 2 2 2 2 2 2 2 2
2410 216/300 Write block 1889 len 20480 data: 2 2 2 2 2
2411 217/300 Read block 1456 len 20480
2412 218/300 Read block 482 len 32768
2413 219/300 Write block 3499 len 20480 data: 2 2 2 3 3
2414 220/300 Read block 2247 len 16384
2415 221/300 Read block 3773 len 40960
2416 222/300 Read block 1038 len 20480
2417 223/300 Read block 2851 len 20480
2418 224/300 Read block 3846 len 28672
2419 225/300 Write block 88 len 16384 data: 2 2 2 2
2420 226/300 Flush
2421 227/300 Write block 4771 len 4096 data: 3
2422 228/300 Read block 2198 len 36864
2423 229/300 Read block 4818 len 4096
2424 230/300 Flush
2425 231/300 Flush
2426 232/300 Write block 4492 len 8192 data: 2 2
2427 233/300 Read block 1453 len 8192
2428 234/300 Read block 212 len 40960
2429 235/300 Write block 3043 len 36864 data: 3 3 3 2 2 2 3 4 4
2430 236/300 Read block 3310 len 24576
2431 237/300 Read block 3474 len 28672
2432 238/300 Read block 1837 len 8192
2433 239/300 Write block 3637 len 36864 data: 2 2 2 2 2 2 2 2 2
2434 240/300 Read block 4761 len 8192
2435 241/300 Write block 4096 len 8192 data: 4 4
2436 242/300 Read block 9 len 8192
2437 243/300 Read block 1593 len 12288
2438 244/300 Flush
2439 245/300 Read block 4482 len 28672
2440 246/300 Read block 4701 len 32768
2441 247/300 Read block 394 len 24576
2442 248/300 Read block 731 len 8192
2443 249/300 Read block 1612 len 16384
2444 250/300 Read block 4664 len 24576
2445 251/300 Read block 3188 len 32768
2446 252/300 Read block 1350 len 16384
2447 253/300 Write block 1847 len 12288 data: 3 2 3
2448 254/300 Read block 860 len 4096
2449 255/300 Read block 657 len 8192
2450 256/300 Write block 3944 len 40960 data: 2 2 2 2 2 2 3 3 3 3
2451 257/300 Write block 2959 len 36864 data: 2 2 2 3 3 2 2 2 2
2452 258/300 Flush
2453 259/300 Read block 4544 len 12288
2454 260/300 Read block 3421 len 32768
2455 261/300 Write block 591 len 12288 data: 2 2 2
2456 262/300 Read block 1380 len 24576
2457 263/300 Read block 1329 len 40960
2458 264/300 Flush
2459 265/300 Write block 2110 len 28672 data: 4 3 4 4 4 4 4
2460 266/300 Read block 3652 len 32768
2461 267/300 Read block 2698 len 24576
2462 268/300 Write block 1467 len 24576 data: 2 2 2 2 2 2
2463 269/300 Read block 4549 len 36864
2464 270/300 Read block 3562 len 16384
2465 271/300 Read block 1252 len 8192
2466 272/300 Read block 801 len 12288
2467 273/300 Write block 1179 len 20480 data: 3 3 3 3 3
2468 274/300 Write block 4043 len 8192 data: 2 2
2469 275/300 Read block 4456 len 36864
2470 276/300 Flush
2471 277/300 Read block 1683 len 36864
2472 278/300 Read block 842 len 20480
2473 279/300 Read block 4981 len 4096
2474 280/300 Read block 2765 len 4096
2475 281/300 Write block 140 len 16384 data: 3 3 3 3
2476 282/300 Read block 2817 len 20480
2477 283/300 Read block 3348 len 36864
2478 284/300 Flush
2479 285/300 Read block 4652 len 12288
2480 286/300 Write block 3875 len 24576 data: 2 2 2 2 3 3
2481 287/300 Read block 1018 len 4096
2482 288/300 Read block 777 len 28672
2483 289/300 Read block 62 len 12288
2484 290/300 Write block 4138 len 16384 data: 3 3 3 3
2485 291/300 Write block 3173 len 24576 data: 2 2 2 2 3 3
2486 292/300 Write block 3922 len 16384 data: 2 2 2 3
2487 293/300 Write block 4824 len 12288 data: 2 2 2
2488 294/300 Read block 1351 len 8192
2489 295/300 Read block 4441 len 32768
2490 296/300 Flush
2491 297/300 Read block 3132 len 32768
2492 298/300 Write block 4594 len 36864 data: 2 2 3 3 3 3 3 3 3
2493 299/300 Read block 489 len 20480
2494 300/300 Flush
2495 301/300 Read block 3530 len 32768
2496 Main task has work to do, go find it
2497 got dsc Start(0)
2498 start 0
2499 Nov 28 04:50:51.738 INFO request completed, latency_us: 219, response_code: 204, uri: /start/cid/0, method: POST, req_id: 8fd9774f-161a-4648-a594-65dec163933e, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
2500 Replay: started 0, returned:Ok(())
2501 ----------------------------------------------------------------
2502 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
2503 States: Offline Active Active
2504 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
2505 1604 Acked Read 9 Sent Done Done false
2506 1605 Acked Flush 0 Sent Done Done false
2507 1606 Acked Read 5 Sent Done Done false
2508 1607 Acked Write 5 Sent Done Done false
2509 1608 Acked Write 9 Sent Done Done false
2510 1609 Acked Write 6 Sent Done Done false
2511 1610 Acked Read 7 Sent Done Done false
2512 1611 Acked Write 5 Sent Done Done false
2513 1612 Acked Read 4 Sent Done Done false
2514 1613 Acked Write 2 Sent Done Done false
2515 1614 Acked Read 4 Sent Done Done false
2516 1615 Acked Read 3 Sent Done Done false
2517 1616 Acked Write 2 Sent Done Done false
2518 1617 Acked Write 2 Sent Done Done false
2519 1618 Acked Flush 0 Sent Done Done false
2520 1619 Acked Flush 0 Sent Done Done false
2521 1620 Acked Read 2 Sent Done Done false
2522 1621 Acked Write 2 Sent Done Done false
2523 1622 Acked Read 8 Sent Done Done false
2524 1623 Acked Write 9 Sent Done Done false
2525 1624 Acked Read 7 Sent Done Done false
2526 1625 Acked Read 9 Sent Done Done false
2527 1626 Acked Flush 0 Sent Done Done false
2528 1627 Acked Write 10 Sent Done Done false
2529 1628 Acked Write 3 Sent Done Done false
2530 1629 Acked Write 10 Sent Done Done false
2531 1630 Acked Read 4 Sent Done Done false
2532 1631 Acked Read 5 Sent Done Done false
2533 1632 Acked Write 1 Sent Done Done false
2534 1633 Acked Write 10 Sent Done Done false
2535 1634 Acked Write 4 Sent Done Done false
2536 1635 Acked Read 10 Sent Done Done false
2537 1636 Acked Read 8 Sent Done Done false
2538 1637 Acked Read 1 Sent Done Done false
2539 1638 Acked Read 5 Sent Done Done false
2540 1639 Acked Write 8 Sent Done Done false
2541 1640 Acked Write 8 Sent Done Done false
2542 1641 Acked Write 1 Sent Done Done false
2543 1642 Acked Read 10 Sent Done Done false
2544 1643 Acked Write 5 Sent Done Done false
2545 1644 Acked Read 10 Sent Done Done false
2546 1645 Acked Write 3 Sent Done Done false
2547 1646 Acked Read 10 Sent Done Done false
2548 1647 Acked Write 3 Sent Done Done false
2549 1648 Acked Write 8 Sent Done Done false
2550 1649 Acked Read 10 Sent Done Done false
2551 1650 Acked Read 8 Sent Done Done false
2552 1651 Acked Write 6 Sent Done Done false
2553 1652 Acked Read 2 Sent Done Done false
2554 1653 Acked Write 8 Sent Done Done false
2555 1654 Acked Flush 0 Sent Done Done false
2556 1655 Acked Flush 0 Sent Done Done false
2557 1656 Acked Read 2 Sent Done Done false
2558 1657 Acked Read 9 Sent Done Done false
2559 1658 Acked Write 2 Sent Done Done false
2560 1659 Acked Read 5 Sent Done Done false
2561 1660 Acked Write 3 Sent Done Done false
2562 1661 Acked Read 8 Sent Done Done false
2563 1662 Acked Read 2 Sent Done Done false
2564 1663 Acked Write 6 Sent Done Done false
2565 1664 Acked Write 7 Sent Done Done false
2566 1665 Acked Read 10 Sent Done Done false
2567 1666 Acked Read 2 Sent Done Done false
2568 1667 Acked Read 9 Sent Done Done false
2569 1668 Acked Read 3 Sent Done Done false
2570 1669 Acked Read 7 Sent Done Done false
2571 1670 Acked Read 8 Sent Done Done false
2572 1671 Acked Read 8 Sent Done Done false
2573 1672 Acked Write 7 Sent Done Done false
2574 1673 Acked Read 8 Sent Done Done false
2575 1674 Acked Read 7 Sent Done Done false
2576 1675 Acked Write 10 Sent Done Done false
2577 1676 Acked Write 7 Sent Done Done false
2578 1677 Acked Read 3 Sent Done Done false
2579 1678 Acked Write 4 Sent Done Done false
2580 1679 Acked Write 7 Sent Done Done false
2581 1680 Acked Read 8 Sent Done Done false
2582 1681 Acked Write 2 Sent Done Done false
2583 1682 Acked Write 4 Sent Done Done false
2584 1683 Acked Read 4 Sent Done Done false
2585 1684 Acked Write 4 Sent Done Done false
2586 1685 Acked Write 9 Sent Done Done false
2587 1686 Acked Write 1 Sent Done Done false
2588 1687 Acked Read 6 Sent Done Done false
2589 1688 Acked Read 1 Sent Done Done false
2590 1689 Acked Read 7 Sent Done Done false
2591 1690 Acked Read 6 Sent Done Done false
2592 1691 Acked Read 2 Sent Done Done false
2593 1692 Acked Flush 0 Sent Done Done false
2594 1693 Acked Write 3 Sent Done Done false
2595 1694 Acked Write 2 Sent Done Done false
2596 1695 Acked Write 10 Sent Done Done false
2597 1696 Acked Read 8 Sent Done Done false
2598 1697 Acked Write 6 Sent Done Done false
2599 1698 Acked Read 2 Sent Done Done false
2600 1699 Acked Flush 0 Sent Done Done false
2601 1700 Acked Write 1 Sent Done Done false
2602 1701 Acked Write 9 Sent Done Done false
2603 1702 Acked Read 7 Sent Done Done false
2604 1703 Acked Read 2 Sent Done Done false
2605 1704 Acked Read 5 Sent Done Done false
2606 1705 Acked Read 5 Sent Done Done false
2607 1706 Acked Write 7 Sent Done Done false
2608 1707 Acked Write 9 Sent Done Done false
2609 1708 Acked Write 9 Sent Done Done false
2610 1709 Acked Read 1 Sent Done Done false
2611 1710 Acked Read 2 Sent Done Done false
2612 1711 Acked Write 6 Sent Done Done false
2613 1712 Acked Read 3 Sent Done Done false
2614 1713 Acked Read 8 Sent Done Done false
2615 1714 Acked Read 3 Sent Done Done false
2616 1715 Acked Write 9 Sent Done Done false
2617 1716 Acked Read 10 Sent Done Done false
2618 1717 Acked Read 10 Sent Done Done false
2619 1718 Acked Read 7 Sent Done Done false
2620 1719 Acked Write 8 Sent Done Done false
2621 1720 Acked Read 5 Sent Done Done false
2622 1721 Acked Read 7 Sent Done Done false
2623 1722 Acked Read 9 Sent Done Done false
2624 1723 Acked Read 5 Sent Done Done false
2625 1724 Acked Write 4 Sent Done Done false
2626 1725 Acked Read 2 Sent Done Done false
2627 1726 Acked Read 6 Sent Done Done false
2628 1727 Acked Read 5 Sent Done Done false
2629 1728 Acked Read 9 Sent Done Done false
2630 1729 Acked Read 8 Sent Done Done false
2631 1730 Acked Read 10 Sent Done Done false
2632 1731 Acked Read 5 Sent Done Done false
2633 1732 Acked Read 8 Sent Done Done false
2634 1733 Acked Write 10 Sent Done Done false
2635 1734 Acked Read 7 Sent Done Done false
2636 1735 Acked Flush 0 Sent Done Done false
2637 1736 Acked Write 10 Sent Done Done false
2638 1737 Acked Read 9 Sent Done Done false
2639 1738 Acked Read 8 Sent Done Done false
2640 1739 Acked Flush 0 Sent Done Done false
2641 1740 Acked Read 1 Sent Done Done false
2642 1741 Acked Write 8 Sent Done Done false
2643 1742 Acked Read 8 Sent Done Done false
2644 1743 Acked Read 2 Sent Done Done false
2645 1744 Acked Read 4 Sent Done Done false
2646 1745 Acked Read 8 Sent Done Done false
2647 1746 Acked Write 6 Sent Done Done false
2648 1747 Acked Read 7 Sent Done Done false
2649 1748 Acked Write 2 Sent Done Done false
2650 1749 Acked Write 6 Sent Done Done false
2651 1750 Acked Read 1 Sent Done Done false
2652 1751 Acked Write 2 Sent Done Done false
2653 1752 Acked Flush 0 Sent Done Done false
2654 1753 Acked Read 6 Sent Done Done false
2655 1754 Acked Write 10 Sent Done Done false
2656 1755 Acked Write 10 Sent Done Done false
2657 1756 Acked Read 2 Sent Done Done false
2658 1757 Acked Read 1 Sent Done Done false
2659 1758 Acked Read 2 Sent Done Done false
2660 1759 Acked Read 10 Sent Done Done false
2661 1760 Acked Write 3 Sent Done Done false
2662 1761 Acked Read 9 Sent Done Done false
2663 1762 Acked Read 6 Sent Done Done false
2664 1763 Acked Read 2 Sent Done Done false
2665 1764 Acked Read 7 Sent Done Done false
2666 1765 Acked Read 1 Sent Done Done false
2667 1766 Acked Write 8 Sent Done Done false
2668 1767 Acked Write 2 Sent Done Done false
2669 1768 Acked Read 1 Sent Done Done false
2670 1769 Acked Read 7 Sent Done Done false
2671 1770 Acked Flush 0 Sent Done Done false
2672 1771 Acked Write 3 Sent Done Done false
2673 1772 Acked Flush 0 Sent Done Done false
2674 1773 Acked Write 6 Sent Done Done false
2675 1774 Acked Write 2 Sent Done Done false
2676 1775 Acked Read 5 Sent Done Done false
2677 1776 Acked Read 3 Sent Done Done false
2678 1777 Acked Read 4 Sent Done Done false
2679 1778 Acked Read 2 Sent Done Done false
2680 1779 Acked Write 9 Sent Done Done false
2681 1780 Acked Write 8 Sent Done Done false
2682 1781 Acked Write 9 Sent Done Done false
2683 1782 Acked Flush 0 Sent Done Done false
2684 1783 Acked Read 4 Sent Done Done false
2685 1784 Acked Write 7 Sent Done Done false
2686 1785 Acked Read 5 Sent Done Done false
2687 1786 Acked Flush 0 Sent Done Done false
2688 1787 Acked Write 3 Sent Done Done false
2689 1788 Acked Read 1 Sent Done Done false
2690 1789 Acked Write 6 Sent Done Done false
2691 1790 Acked Flush 0 Sent Done Done false
2692 1791 Acked Write 3 Sent Done Done false
2693 1792 Acked Read 5 Sent Done Done false
2694 1793 Acked Write 3 Sent Done Done false
2695 1794 Acked Read 6 Sent Done Done false
2696 1795 Acked Read 7 Sent Done Done false
2697 1796 Acked Write 6 Sent Done Done false
2698 1797 Acked Read 5 Sent Done Done false
2699 1798 Acked Read 5 Sent Done Done false
2700 1799 Acked Read 2 Sent Done Done false
2701 1800 Acked Read 5 Sent Done Done false
2702 1801 Acked Write 8 Sent Done Done false
2703 1802 Acked Write 3 Sent Done Done false
2704 1803 Acked Read 8 Sent Done Done false
2705 1804 Acked Read 3 Sent Done Done false
2706 1805 Acked Write 3 Sent Done Done false
2707 1806 Acked Read 10 Sent Done Done false
2708 1807 Acked Write 10 Sent Done Done false
2709 1808 Acked Write 7 Sent Done Done false
2710 1809 Acked Flush 0 Sent Done Done false
2711 1810 Acked Flush 0 Sent Done Done false
2712 1811 Acked Write 7 Sent Done Done false
2713 1812 Acked Read 6 Sent Done Done false
2714 1813 Acked Read 7 Sent Done Done false
2715 1814 Acked Write 1 Sent Done Done false
2716 1815 Acked Read 5 Sent Done Done false
2717 1816 Acked Read 2 Sent Done Done false
2718 1817 Acked Write 4 Sent Done Done false
2719 1818 Acked Write 9 Sent Done Done false
2720 1819 Acked Write 5 Sent Done Done false
2721 1820 Acked Read 5 Sent Done Done false
2722 1821 Acked Read 8 Sent Done Done false
2723 1822 Acked Write 5 Sent Done Done false
2724 1823 Acked Read 4 Sent Done Done false
2725 1824 Acked Read 10 Sent Done Done false
2726 1825 Acked Read 5 Sent Done Done false
2727 1826 Acked Read 5 Sent Done Done false
2728 1827 Acked Read 7 Sent Done Done false
2729 1828 Acked Write 4 Sent Done Done false
2730 1829 Acked Flush 0 Sent Done Done false
2731 1830 Acked Write 1 Sent Done Done false
2732 1831 Acked Read 9 Sent Done Done false
2733 1832 Acked Read 1 Sent Done Done false
2734 1833 Acked Flush 0 Sent Done Done false
2735 1834 Acked Flush 0 Sent Done Done false
2736 1835 Acked Write 2 Sent Done Done false
2737 1836 Acked Read 2 Sent Done Done false
2738 1837 Acked Read 10 Sent Done Done false
2739 1838 Acked Write 9 Sent Done Done false
2740 1839 Acked Read 6 Sent Done Done false
2741 1840 Acked Read 7 Sent Done Done false
2742 1841 Acked Read 2 Sent Done Done false
2743 1842 Acked Write 9 Sent Done Done false
2744 1843 Acked Read 2 Sent Done Done false
2745 1844 Acked Write 2 Sent Done Done false
2746 1845 Acked Read 2 Sent Done Done false
2747 1846 Acked Read 3 Sent Done Done false
2748 1847 Acked Flush 0 Sent Done Done false
2749 1848 Acked Read 7 Sent Done Done false
2750 1849 Acked Read 8 Sent Done Done false
2751 1850 Acked Read 6 Sent Done Done false
2752 1851 Acked Read 2 Sent Done Done false
2753 1852 Acked Read 4 Sent Done Done false
2754 1853 Acked Read 6 Sent Done Done false
2755 1854 Acked Read 8 Sent Done Done false
2756 1855 Acked Read 4 Sent Done Done false
2757 1856 Acked Write 3 Sent Done Done false
2758 1857 Acked Read 1 Sent Done Done false
2759 1858 Acked Read 2 Sent Done Done false
2760 1859 Acked Write 10 Sent Done Done false
2761 1860 Acked Write 9 Sent Done Done false
2762 1861 Acked Flush 0 Sent Done Done false
2763 1862 Acked Read 3 Sent Done Done false
2764 1863 Acked Read 8 Sent Done Done false
2765 1864 Acked Write 3 Sent Done Done false
2766 1865 Acked Read 6 Sent Done Done false
2767 1866 Acked Read 10 Sent Done Done false
2768 1867 Acked Flush 0 Sent Done Done false
2769 1868 Acked Write 7 Sent Done Done false
2770 1869 Acked Read 8 Sent Done Done false
2771 1870 Acked Read 6 Sent Done Done false
2772 1871 Acked Write 6 Sent Done Done false
2773 1872 Acked Read 9 Sent Done Done false
2774 1873 Acked Read 4 Sent Done Done false
2775 1874 Acked Read 2 Sent Done Done false
2776 1875 Acked Read 3 Sent Done Done false
2777 1876 Acked Write 5 Sent Done Done false
2778 1877 Acked Write 2 Sent Done Done false
2779 1878 Acked Read 9 Sent Done Done false
2780 1879 Acked Flush 0 Sent Done Done false
2781 1880 Acked Read 9 Sent Done Done false
2782 1881 Acked Read 5 Sent Done Done false
2783 1882 Acked Read 1 Sent Done Done false
2784 1883 Acked Read 1 Sent Done Done false
2785 1884 Acked Write 4 Sent Done Done false
2786 1885 Acked Read 5 Sent Done Done false
2787 1886 Acked Read 9 Sent Done Done false
2788 1887 Acked Flush 0 Sent Done Done false
2789 1888 Acked Read 3 Sent Done Done false
2790 1889 Acked Write 6 Sent Done Done false
2791 1890 Acked Read 1 Sent Done Done false
2792 1891 Acked Read 7 Sent Done Done false
2793 1892 Acked Read 3 Sent Done Done false
2794 1893 Acked Write 4 Sent Done Done false
2795 1894 Acked Write 6 Sent Done Done false
2796 1895 Acked Write 4 Sent Done Done false
2797 1896 Acked Write 3 Sent Done Done false
2798 1897 Acked Read 2 Sent Done Done false
2799 1898 Acked Read 8 Sent Done Done false
2800 1899 Acked Flush 0 Sent Done Done false
2801 1900 Acked Read 8 Sent Done Done false
2802 1901 Acked Write 9 Sent Done Done false
2803 1902 Acked Read 5 Sent Done Done false
2804 1903 Acked Flush 0 Sent Done Done false
2805 1904 Acked Read 8 Sent Done Done false
2806 STATES DS:0 DS:1 DS:2 TOTAL
2807 Sent 301 0 0 301
2808 Done 0 301 301 602
2809 Skipped 0 0 0 0
2810 Error 0 0 0 0
2811 Last Flush: 1603 1903 1903
2812 Downstairs last five completed: 1603 1602 1601 1600 1599
2813 Upstairs last five completed: 1904 1903 1902 1901 1900
2814 CLIENT: Up:0 ds:301 act:2
2815 [8810] got start action
2816 [8810] I am going to restart
2817 Starting downstairs at port 8810
2818 Make output file at "/tmp/dsc/downstairs-8810.txt"
2819 [8810][0] reports Starting
2820 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1205)
2821 [8810][0] reports Running
2822 ----------------------------------------------------------------
2823 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
2824 States: Offline Active Active
2825 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
2826 1604 Acked Read 9 Sent Done Done false
2827 1605 Acked Flush 0 Sent Done Done false
2828 1606 Acked Read 5 Sent Done Done false
2829 1607 Acked Write 5 Sent Done Done false
2830 1608 Acked Write 9 Sent Done Done false
2831 1609 Acked Write 6 Sent Done Done false
2832 1610 Acked Read 7 Sent Done Done false
2833 1611 Acked Write 5 Sent Done Done false
2834 1612 Acked Read 4 Sent Done Done false
2835 1613 Acked Write 2 Sent Done Done false
2836 1614 Acked Read 4 Sent Done Done false
2837 1615 Acked Read 3 Sent Done Done false
2838 1616 Acked Write 2 Sent Done Done false
2839 1617 Acked Write 2 Sent Done Done false
2840 1618 Acked Flush 0 Sent Done Done false
2841 1619 Acked Flush 0 Sent Done Done false
2842 1620 Acked Read 2 Sent Done Done false
2843 1621 Acked Write 2 Sent Done Done false
2844 1622 Acked Read 8 Sent Done Done false
2845 1623 Acked Write 9 Sent Done Done false
2846 1624 Acked Read 7 Sent Done Done false
2847 1625 Acked Read 9 Sent Done Done false
2848 1626 Acked Flush 0 Sent Done Done false
2849 1627 Acked Write 10 Sent Done Done false
2850 1628 Acked Write 3 Sent Done Done false
2851 1629 Acked Write 10 Sent Done Done false
2852 1630 Acked Read 4 Sent Done Done false
2853 1631 Acked Read 5 Sent Done Done false
2854 1632 Acked Write 1 Sent Done Done false
2855 1633 Acked Write 10 Sent Done Done false
2856 1634 Acked Write 4 Sent Done Done false
2857 1635 Acked Read 10 Sent Done Done false
2858 1636 Acked Read 8 Sent Done Done false
2859 1637 Acked Read 1 Sent Done Done false
2860 1638 Acked Read 5 Sent Done Done false
2861 1639 Acked Write 8 Sent Done Done false
2862 1640 Acked Write 8 Sent Done Done false
2863 1641 Acked Write 1 Sent Done Done false
2864 1642 Acked Read 10 Sent Done Done false
2865 1643 Acked Write 5 Sent Done Done false
2866 1644 Acked Read 10 Sent Done Done false
2867 1645 Acked Write 3 Sent Done Done false
2868 1646 Acked Read 10 Sent Done Done false
2869 1647 Acked Write 3 Sent Done Done false
2870 1648 Acked Write 8 Sent Done Done false
2871 1649 Acked Read 10 Sent Done Done false
2872 1650 Acked Read 8 Sent Done Done false
2873 1651 Acked Write 6 Sent Done Done false
2874 1652 Acked Read 2 Sent Done Done false
2875 1653 Acked Write 8 Sent Done Done false
2876 1654 Acked Flush 0 Sent Done Done false
2877 1655 Acked Flush 0 Sent Done Done false
2878 1656 Acked Read 2 Sent Done Done false
2879 1657 Acked Read 9 Sent Done Done false
2880 1658 Acked Write 2 Sent Done Done false
2881 1659 Acked Read 5 Sent Done Done false
2882 1660 Acked Write 3 Sent Done Done false
2883 1661 Acked Read 8 Sent Done Done false
2884 1662 Acked Read 2 Sent Done Done false
2885 1663 Acked Write 6 Sent Done Done false
2886 1664 Acked Write 7 Sent Done Done false
2887 1665 Acked Read 10 Sent Done Done false
2888 1666 Acked Read 2 Sent Done Done false
2889 1667 Acked Read 9 Sent Done Done false
2890 1668 Acked Read 3 Sent Done Done false
2891 1669 Acked Read 7 Sent Done Done false
2892 1670 Acked Read 8 Sent Done Done false
2893 1671 Acked Read 8 Sent Done Done false
2894 1672 Acked Write 7 Sent Done Done false
2895 1673 Acked Read 8 Sent Done Done false
2896 1674 Acked Read 7 Sent Done Done false
2897 1675 Acked Write 10 Sent Done Done false
2898 1676 Acked Write 7 Sent Done Done false
2899 1677 Acked Read 3 Sent Done Done false
2900 1678 Acked Write 4 Sent Done Done false
2901 1679 Acked Write 7 Sent Done Done false
2902 1680 Acked Read 8 Sent Done Done false
2903 1681 Acked Write 2 Sent Done Done false
2904 1682 Acked Write 4 Sent Done Done false
2905 1683 Acked Read 4 Sent Done Done false
2906 1684 Acked Write 4 Sent Done Done false
2907 1685 Acked Write 9 Sent Done Done false
2908 1686 Acked Write 1 Sent Done Done false
2909 1687 Acked Read 6 Sent Done Done false
2910 1688 Acked Read 1 Sent Done Done false
2911 1689 Acked Read 7 Sent Done Done false
2912 1690 Acked Read 6 Sent Done Done false
2913 1691 Acked Read 2 Sent Done Done false
2914 1692 Acked Flush 0 Sent Done Done false
2915 1693 Acked Write 3 Sent Done Done false
2916 1694 Acked Write 2 Sent Done Done false
2917 1695 Acked Write 10 Sent Done Done false
2918 1696 Acked Read 8 Sent Done Done false
2919 1697 Acked Write 6 Sent Done Done false
2920 1698 Acked Read 2 Sent Done Done false
2921 1699 Acked Flush 0 Sent Done Done false
2922 1700 Acked Write 1 Sent Done Done false
2923 1701 Acked Write 9 Sent Done Done false
2924 1702 Acked Read 7 Sent Done Done false
2925 1703 Acked Read 2 Sent Done Done false
2926 1704 Acked Read 5 Sent Done Done false
2927 1705 Acked Read 5 Sent Done Done false
2928 1706 Acked Write 7 Sent Done Done false
2929 1707 Acked Write 9 Sent Done Done false
2930 1708 Acked Write 9 Sent Done Done false
2931 1709 Acked Read 1 Sent Done Done false
2932 1710 Acked Read 2 Sent Done Done false
2933 1711 Acked Write 6 Sent Done Done false
2934 1712 Acked Read 3 Sent Done Done false
2935 1713 Acked Read 8 Sent Done Done false
2936 1714 Acked Read 3 Sent Done Done false
2937 1715 Acked Write 9 Sent Done Done false
2938 1716 Acked Read 10 Sent Done Done false
2939 1717 Acked Read 10 Sent Done Done false
2940 1718 Acked Read 7 Sent Done Done false
2941 1719 Acked Write 8 Sent Done Done false
2942 1720 Acked Read 5 Sent Done Done false
2943 1721 Acked Read 7 Sent Done Done false
2944 1722 Acked Read 9 Sent Done Done false
2945 1723 Acked Read 5 Sent Done Done false
2946 1724 Acked Write 4 Sent Done Done false
2947 1725 Acked Read 2 Sent Done Done false
2948 1726 Acked Read 6 Sent Done Done false
2949 1727 Acked Read 5 Sent Done Done false
2950 1728 Acked Read 9 Sent Done Done false
2951 1729 Acked Read 8 Sent Done Done false
2952 1730 Acked Read 10 Sent Done Done false
2953 1731 Acked Read 5 Sent Done Done false
2954 1732 Acked Read 8 Sent Done Done false
2955 1733 Acked Write 10 Sent Done Done false
2956 1734 Acked Read 7 Sent Done Done false
2957 1735 Acked Flush 0 Sent Done Done false
2958 1736 Acked Write 10 Sent Done Done false
2959 1737 Acked Read 9 Sent Done Done false
2960 1738 Acked Read 8 Sent Done Done false
2961 1739 Acked Flush 0 Sent Done Done false
2962 1740 Acked Read 1 Sent Done Done false
2963 1741 Acked Write 8 Sent Done Done false
2964 1742 Acked Read 8 Sent Done Done false
2965 1743 Acked Read 2 Sent Done Done false
2966 1744 Acked Read 4 Sent Done Done false
2967 1745 Acked Read 8 Sent Done Done false
2968 1746 Acked Write 6 Sent Done Done false
2969 1747 Acked Read 7 Sent Done Done false
2970 1748 Acked Write 2 Sent Done Done false
2971 1749 Acked Write 6 Sent Done Done false
2972 1750 Acked Read 1 Sent Done Done false
2973 1751 Acked Write 2 Sent Done Done false
2974 1752 Acked Flush 0 Sent Done Done false
2975 1753 Acked Read 6 Sent Done Done false
2976 1754 Acked Write 10 Sent Done Done false
2977 1755 Acked Write 10 Sent Done Done false
2978 1756 Acked Read 2 Sent Done Done false
2979 1757 Acked Read 1 Sent Done Done false
2980 1758 Acked Read 2 Sent Done Done false
2981 1759 Acked Read 10 Sent Done Done false
2982 1760 Acked Write 3 Sent Done Done false
2983 1761 Acked Read 9 Sent Done Done false
2984 1762 Acked Read 6 Sent Done Done false
2985 1763 Acked Read 2 Sent Done Done false
2986 1764 Acked Read 7 Sent Done Done false
2987 1765 Acked Read 1 Sent Done Done false
2988 1766 Acked Write 8 Sent Done Done false
2989 1767 Acked Write 2 Sent Done Done false
2990 1768 Acked Read 1 Sent Done Done false
2991 1769 Acked Read 7 Sent Done Done false
2992 1770 Acked Flush 0 Sent Done Done false
2993 1771 Acked Write 3 Sent Done Done false
2994 1772 Acked Flush 0 Sent Done Done false
2995 1773 Acked Write 6 Sent Done Done false
2996 1774 Acked Write 2 Sent Done Done false
2997 1775 Acked Read 5 Sent Done Done false
2998 1776 Acked Read 3 Sent Done Done false
2999 1777 Acked Read 4 Sent Done Done false
3000 1778 Acked Read 2 Sent Done Done false
3001 1779 Acked Write 9 Sent Done Done false
3002 1780 Acked Write 8 Sent Done Done false
3003 1781 Acked Write 9 Sent Done Done false
3004 1782 Acked Flush 0 Sent Done Done false
3005 1783 Acked Read 4 Sent Done Done false
3006 1784 Acked Write 7 Sent Done Done false
3007 1785 Acked Read 5 Sent Done Done false
3008 1786 Acked Flush 0 Sent Done Done false
3009 1787 Acked Write 3 Sent Done Done false
3010 1788 Acked Read 1 Sent Done Done false
3011 1789 Acked Write 6 Sent Done Done false
3012 1790 Acked Flush 0 Sent Done Done false
3013 1791 Acked Write 3 Sent Done Done false
3014 1792 Acked Read 5 Sent Done Done false
3015 1793 Acked Write 3 Sent Done Done false
3016 1794 Acked Read 6 Sent Done Done false
3017 1795 Acked Read 7 Sent Done Done false
3018 1796 Acked Write 6 Sent Done Done false
3019 1797 Acked Read 5 Sent Done Done false
3020 1798 Acked Read 5 Sent Done Done false
3021 1799 Acked Read 2 Sent Done Done false
3022 1800 Acked Read 5 Sent Done Done false
3023 1801 Acked Write 8 Sent Done Done false
3024 1802 Acked Write 3 Sent Done Done false
3025 1803 Acked Read 8 Sent Done Done false
3026 1804 Acked Read 3 Sent Done Done false
3027 1805 Acked Write 3 Sent Done Done false
3028 1806 Acked Read 10 Sent Done Done false
3029 1807 Acked Write 10 Sent Done Done false
3030 1808 Acked Write 7 Sent Done Done false
3031 1809 Acked Flush 0 Sent Done Done false
3032 1810 Acked Flush 0 Sent Done Done false
3033 1811 Acked Write 7 Sent Done Done false
3034 1812 Acked Read 6 Sent Done Done false
3035 1813 Acked Read 7 Sent Done Done false
3036 1814 Acked Write 1 Sent Done Done false
3037 1815 Acked Read 5 Sent Done Done false
3038 1816 Acked Read 2 Sent Done Done false
3039 1817 Acked Write 4 Sent Done Done false
3040 1818 Acked Write 9 Sent Done Done false
3041 1819 Acked Write 5 Sent Done Done false
3042 1820 Acked Read 5 Sent Done Done false
3043 1821 Acked Read 8 Sent Done Done false
3044 1822 Acked Write 5 Sent Done Done false
3045 1823 Acked Read 4 Sent Done Done false
3046 1824 Acked Read 10 Sent Done Done false
3047 1825 Acked Read 5 Sent Done Done false
3048 1826 Acked Read 5 Sent Done Done false
3049 1827 Acked Read 7 Sent Done Done false
3050 1828 Acked Write 4 Sent Done Done false
3051 1829 Acked Flush 0 Sent Done Done false
3052 1830 Acked Write 1 Sent Done Done false
3053 1831 Acked Read 9 Sent Done Done false
3054 1832 Acked Read 1 Sent Done Done false
3055 1833 Acked Flush 0 Sent Done Done false
3056 1834 Acked Flush 0 Sent Done Done false
3057 1835 Acked Write 2 Sent Done Done false
3058 1836 Acked Read 2 Sent Done Done false
3059 1837 Acked Read 10 Sent Done Done false
3060 1838 Acked Write 9 Sent Done Done false
3061 1839 Acked Read 6 Sent Done Done false
3062 1840 Acked Read 7 Sent Done Done false
3063 1841 Acked Read 2 Sent Done Done false
3064 1842 Acked Write 9 Sent Done Done false
3065 1843 Acked Read 2 Sent Done Done false
3066 1844 Acked Write 2 Sent Done Done false
3067 1845 Acked Read 2 Sent Done Done false
3068 1846 Acked Read 3 Sent Done Done false
3069 1847 Acked Flush 0 Sent Done Done false
3070 1848 Acked Read 7 Sent Done Done false
3071 1849 Acked Read 8 Sent Done Done false
3072 1850 Acked Read 6 Sent Done Done false
3073 1851 Acked Read 2 Sent Done Done false
3074 1852 Acked Read 4 Sent Done Done false
3075 1853 Acked Read 6 Sent Done Done false
3076 1854 Acked Read 8 Sent Done Done false
3077 1855 Acked Read 4 Sent Done Done false
3078 1856 Acked Write 3 Sent Done Done false
3079 1857 Acked Read 1 Sent Done Done false
3080 1858 Acked Read 2 Sent Done Done false
3081 1859 Acked Write 10 Sent Done Done false
3082 1860 Acked Write 9 Sent Done Done false
3083 1861 Acked Flush 0 Sent Done Done false
3084 1862 Acked Read 3 Sent Done Done false
3085 1863 Acked Read 8 Sent Done Done false
3086 1864 Acked Write 3 Sent Done Done false
3087 1865 Acked Read 6 Sent Done Done false
3088 1866 Acked Read 10 Sent Done Done false
3089 1867 Acked Flush 0 Sent Done Done false
3090 1868 Acked Write 7 Sent Done Done false
3091 1869 Acked Read 8 Sent Done Done false
3092 1870 Acked Read 6 Sent Done Done false
3093 1871 Acked Write 6 Sent Done Done false
3094 1872 Acked Read 9 Sent Done Done false
3095 1873 Acked Read 4 Sent Done Done false
3096 1874 Acked Read 2 Sent Done Done false
3097 1875 Acked Read 3 Sent Done Done false
3098 1876 Acked Write 5 Sent Done Done false
3099 1877 Acked Write 2 Sent Done Done false
3100 1878 Acked Read 9 Sent Done Done false
3101 1879 Acked Flush 0 Sent Done Done false
3102 1880 Acked Read 9 Sent Done Done false
3103 1881 Acked Read 5 Sent Done Done false
3104 1882 Acked Read 1 Sent Done Done false
3105 1883 Acked Read 1 Sent Done Done false
3106 1884 Acked Write 4 Sent Done Done false
3107 1885 Acked Read 5 Sent Done Done false
3108 1886 Acked Read 9 Sent Done Done false
3109 1887 Acked Flush 0 Sent Done Done false
3110 1888 Acked Read 3 Sent Done Done false
3111 1889 Acked Write 6 Sent Done Done false
3112 1890 Acked Read 1 Sent Done Done false
3113 1891 Acked Read 7 Sent Done Done false
3114 1892 Acked Read 3 Sent Done Done false
3115 1893 Acked Write 4 Sent Done Done false
3116 1894 Acked Write 6 Sent Done Done false
3117 1895 Acked Write 4 Sent Done Done false
3118 1896 Acked Write 3 Sent Done Done false
3119 1897 Acked Read 2 Sent Done Done false
3120 1898 Acked Read 8 Sent Done Done false
3121 1899 Acked Flush 0 Sent Done Done false
3122 1900 Acked Read 8 Sent Done Done false
3123 1901 Acked Write 9 Sent Done Done false
3124 1902 Acked Read 5 Sent Done Done false
3125 1903 Acked Flush 0 Sent Done Done false
3126 1904 Acked Read 8 Sent Done Done false
3127 1905 Acked Flush 0 Sent Done Done false
3128 STATES DS:0 DS:1 DS:2 TOTAL
3129 Sent 302 0 0 302
3130 Done 0 302 302 604
3131 Skipped 0 0 0 0
3132 Error 0 0 0 0
3133 Last Flush: 1603 1905 1905
3134 Downstairs last five completed: 1603 1602 1601 1600 1599
3135 Upstairs last five completed: 1905 1904 1903 1902 1901
3136 CLIENT: Up:0 ds:302 act:2
31372024-11-28T04:50:57.495ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
31382024-11-28T04:50:57.495ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:45551) = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
31392024-11-28T04:50:57.496ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
31402024-11-28T04:50:57.497ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
31412024-11-28T04:50:57.497ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
31422024-11-28T04:50:57.497ZINFOcrucible: send last flush ID to this DS: 1603 = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
31432024-11-28T04:50:57.497ZINFOcrucible: Replied this last flush ID: 1603 = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
31442024-11-28T04:50:57.497ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
31452024-11-28T04:50:57.497ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
31462024-11-28T04:50:57.497ZINFOcrucible: [0] client re-new 302 jobs since flush 1603 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
31472024-11-28T04:50:57.497ZINFOcrucible: [0] Replayed 302 jobs since flush: 1603 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
3148 ----------------------------------------------------------------
3149 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
3150 Downstairs last five completed: 1905 1904 1903 1902 1901
3151 Upstairs last five completed: 1905 1904 1903 1902 1901
3152 CLIENT: Up:0 ds:0 act:3
3153 Replay: All jobs finished
3154 Main task has work to do, go find it
3155 got dsc Stop(0)
3156 stop 0
3157 Nov 28 04:50:59.743 INFO request completed, latency_us: 355, response_code: 204, uri: /stop/cid/0, method: POST[8810] Got stop action so:false kr:false
3158 , req_id: cdd070c8-556d-4f54-a105-f712c8ab6c43, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
3159 Nov 28 04:50:59.744 INFO request completed, latency_us: 116, response_code: 200, uri: /state/cid/0, method: GET, req_id: a8f06773-6036-4a78-817a-7c2971ad3e46, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
31602024-11-28T04:50:59.744ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
31612024-11-28T04:50:59.744ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
31622024-11-28T04:50:59.744ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
31632024-11-28T04:50:59.744ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
31642024-11-28T04:50:59.744ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
31652024-11-28T04:50:59.744ZINFOcrucible: client task is exiting = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
3166 [8810] Exited with: ExitStatus(unix_wait_status(9))
3167 [8810][0] reports Exit
3168 Nov 28 04:51:03.746 INFO request completed, latency_us: 201, response_code: 200, uri: /state/cid/0, method: GET, req_id: 10870d69-dfe6-41ae-b164-68d265331ccc, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
3169 001/300 Read block 3612 len 12288
3170 002/300 Read block 412 len 28672
3171 003/300 Flush
3172 004/300 Write block 181 len 4096 data: 2
3173 005/300 Read block 4324 len 36864
3174 006/300 Read block 15 len 28672
3175 007/300 Read block 4669 len 4096
3176 008/300 Write block 3696 len 16384 data: 2 2 2 2
3177 009/300 Read block 581 len 28672
3178 010/300 Read block 4487 len 28672
3179 011/300 Write block 3732 len 20480 data: 2 2 2 2 2
3180 012/300 Read block 4823 len 16384
3181 013/300 Write block 2431 len 8192 data: 3 3
3182 014/300 Read block 3589 len 4096
3183 015/300 Read block 3695 len 8192
3184 016/300 Write block 712 len 20480 data: 2 2 2 2 2
3185 017/300 Flush
3186 018/300 Read block 96 len 24576
3187 019/300 Write block 2363 len 12288 data: 2 2 2
3188 020/300 Read block 4803 len 20480
3189 021/300 Read block 4959 len 40960
3190 022/300 Write block 1226 len 8192 data: 2 2
3191 023/300 Read block 1219 len 36864
3192 024/300 Write block 1367 len 4096 data: 2
3193 025/300 Read block 1422 len 28672
3194 026/300 Read block 370 len 16384
3195 027/300 Read block 512 len 4096
3196 028/300 Write block 4989 len 16384 data: 3 3 3 2
3197 029/300 Read block 4736 len 16384
3198 030/300 Read block 1301 len 36864
3199 031/300 Read block 4072 len 36864
3200 032/300 Write block 711 len 40960 data: 2 3 3 3 3 3 2 2 2 2
3201 033/300 Read block 128 len 4096
3202 034/300 Flush
3203 035/300 Read block 3448 len 40960
3204 036/300 Flush
3205 037/300 Write block 1265 len 12288 data: 2 2 2
3206 038/300 Write block 1618 len 4096 data: 2
3207 039/300 Write block 3249 len 40960 data: 2 2 2 2 2 2 2 2 2 2
3208 040/300 Write block 845 len 8192 data: 3 3
3209 041/300 Read block 783 len 24576
3210 042/300 Read block 273 len 36864
3211 043/300 Write block 4992 len 12288 data: 3 2 2
3212 044/300 Write block 2564 len 28672 data: 3 3 3 2 2 2 3
3213 045/300 Flush
3214 046/300 Write block 619 len 32768 data: 2 2 2 2 2 2 2 3
3215 047/300 Read block 1212 len 16384
3216 048/300 Read block 4682 len 8192
3217 049/300 Write block 3205 len 20480 data: 2 2 2 2 3
3218 050/300 Read block 962 len 12288
3219 051/300 Write block 2974 len 32768 data: 2 2 2 2 2 2 2 2
3220 052/300 Read block 158 len 20480
3221 053/300 Read block 1285 len 28672
3222 054/300 Write block 2459 len 28672 data: 2 2 2 2 3 2 2
3223 055/300 Write block 3332 len 24576 data: 2 2 2 2 2 2
3224 056/300 Read block 2184 len 8192
3225 057/300 Read block 3740 len 28672
3226 058/300 Read block 3820 len 24576
3227 059/300 Write block 2467 len 40960 data: 2 3 3 3 3 3 3 2 2 2
3228 060/300 Read block 1158 len 20480
3229 061/300 Read block 4866 len 28672
3230 062/300 Write block 1075 len 20480 data: 2 2 2 2 2
3231 063/300 Write block 2474 len 12288 data: 3 3 3
3232 064/300 Read block 4401 len 24576
3233 065/300 Write block 3886 len 32768 data: 2 2 2 2 2 2 2 2
3234 066/300 Read block 2915 len 36864
3235 067/300 Read block 2798 len 12288
3236 068/300 Read block 4301 len 32768
3237 069/300 Flush
3238 070/300 Write block 2960 len 12288 data: 3 3 4
3239 071/300 Read block 1799 len 8192
3240 072/300 Write block 2637 len 20480 data: 2 2 2 2 2
3241 073/300 Read block 626 len 28672
3242 074/300 Read block 3694 len 32768
3243 075/300 Write block 4648 len 16384 data: 2 2 2 2
3244 076/300 Flush
3245 077/300 Write block 629 len 36864 data: 3 3 3 3 3 3 4 4 3
3246 078/300 Read block 4670 len 28672
3247 079/300 Read block 3594 len 24576
3248 080/300 Write block 1980 len 12288 data: 2 2 3
3249 081/300 Read block 1380 len 16384
3250 082/300 Write block 335 len 12288 data: 2 2 2
3251 083/300 Read block 211 len 40960
3252 084/300 Read block 1535 len 28672
3253 085/300 Read block 4197 len 20480
3254 086/300 Read block 1281 len 24576
3255 087/300 Read block 3659 len 24576
3256 088/300 Write block 3171 len 28672 data: 2 2 3 3 3 3 4
3257 089/300 Write block 1725 len 4096 data: 3
3258 090/300 Write block 1096 len 12288 data: 2 2 2
3259 091/300 Write block 187 len 28672 data: 2 2 3 3 3 3 3
3260 092/300 Read block 4723 len 12288
3261 093/300 Write block 2972 len 24576 data: 2 2 3 3 3 3
3262 094/300 Read block 4423 len 32768
3263 095/300 Write block 1273 len 40960 data: 2 2 2 2 2 2 2 2 3 3
3264 096/300 Read block 1808 len 8192
3265 097/300 Write block 4646 len 32768 data: 2 2 3 3 3 3 2 2
3266 098/300 Read block 1889 len 16384
3267 099/300 Write block 3340 len 12288 data: 2 2 2
3268 100/300 Read block 1263 len 24576
3269 101/300 Write block 3791 len 12288 data: 2 2 2
3270 102/300 Read block 2439 len 12288
3271 103/300 Write block 193 len 40960 data: 4 3 3 4 3 3 2 2 2 2
3272 104/300 Read block 1146 len 4096
3273 105/300 Read block 1598 len 20480
3274 106/300 Write block 1932 len 16384 data: 2 2 2 2
3275 107/300 Read block 4111 len 20480
3276 108/300 Write block 3220 len 8192 data: 3 3
3277 109/300 Read block 223 len 24576
3278 110/300 Read block 3518 len 36864
3279 111/300 Read block 158 len 4096
3280 112/300 Read block 3305 len 20480
3281 113/300 Read block 4990 len 32768
3282 114/300 Write block 3185 len 36864 data: 2 2 3 3 3 3 3 3 3
3283 115/300 Flush
3284 116/300 Read block 1644 len 16384
3285 117/300 Read block 2750 len 8192
3286 118/300 Read block 2559 len 40960
3287 119/300 Write block 256 len 20480 data: 2 2 2 2 2
3288 120/300 Write block 2917 len 24576 data: 2 2 3 4 4 4
3289 121/300 Read block 4845 len 36864
3290 122/300 Read block 3708 len 4096
3291 123/300 Write block 2522 len 32768 data: 3 3 3 4 4 4 3 3
3292 124/300 Write block 347 len 40960 data: 2 2 2 2 2 2 2 2 2 2
3293 125/300 Read block 2180 len 24576
3294 126/300 Write block 2330 len 12288 data: 3 4 4
3295 127/300 Write block 2349 len 20480 data: 2 2 2 2 2
3296 128/300 Read block 3314 len 32768
3297 129/300 Read block 1787 len 32768
3298 130/300 Write block 1280 len 12288 data: 3 4 4
3299 131/300 Read block 1319 len 36864
3300 132/300 Write block 2092 len 12288 data: 2 2 2
3301 133/300 Read block 3823 len 28672
3302 134/300 Flush
3303 135/300 Read block 2251 len 28672
3304 136/300 Read block 1810 len 4096
3305 137/300 Read block 3178 len 4096
3306 138/300 Write block 594 len 24576 data: 2 2 2 2 2 2
3307 139/300 Write block 1282 len 32768 data: 5 3 3 3 2 2 2 2
3308 140/300 Read block 4408 len 32768
3309 141/300 Write block 3280 len 28672 data: 2 2 2 2 2 2 2
3310 142/300 Read block 620 len 12288
3311 143/300 Write block 4033 len 24576 data: 3 3 3 3 2 2
3312 144/300 Read block 2738 len 32768
3313 145/300 Read block 2556 len 8192
3314 146/300 Flush
3315 147/300 Read block 4926 len 8192
3316 148/300 Read block 1909 len 28672
3317 149/300 Write block 2112 len 20480 data: 5 5 5 5 5
3318 150/300 Write block 4773 len 8192 data: 3 3
3319 151/300 Write block 2890 len 24576 data: 2 2 2 2 2 2
3320 152/300 Write block 4758 len 20480 data: 2 2 2 2 2
3321 153/300 Write block 4565 len 28672 data: 2 2 2 2 3 3 3
3322 154/300 Read block 3884 len 20480
3323 155/300 Write block 4805 len 8192 data: 3 3
3324 156/300 Read block 2673 len 4096
3325 157/300 Write block 1524 len 24576 data: 2 2 2 2 2 3
3326 158/300 Write block 729 len 24576 data: 2 2 2 2 3 3
3327 159/300 Flush
3328 160/300 Read block 462 len 4096
3329 161/300 Flush
3330 162/300 Read block 2598 len 32768
3331 163/300 Read block 3191 len 32768
3332 164/300 Write block 4611 len 16384 data: 2 2 2 2
3333 165/300 Read block 3805 len 32768
3334 166/300 Read block 4711 len 12288
3335 167/300 Write block 1708 len 8192 data: 3 4
3336 168/300 Read block 4842 len 24576
3337 169/300 Write block 1024 len 12288 data: 2 2 2
3338 170/300 Flush
3339 171/300 Read block 2828 len 4096
3340 172/300 Read block 2372 len 36864
3341 173/300 Read block 4429 len 32768
3342 174/300 Read block 1610 len 36864
3343 175/300 Read block 3972 len 28672
3344 176/300 Write block 2935 len 4096 data: 2
3345 177/300 Read block 2283 len 20480
3346 178/300 Read block 70 len 16384
3347 179/300 Write block 322 len 40960 data: 2 2 2 2 2 2 2 2 2 2
3348 180/300 Read block 845 len 36864
3349 181/300 Flush
3350 182/300 Read block 3978 len 20480
3351 183/300 Write block 246 len 28672 data: 2 2 2 2 2 2 2
3352 184/300 Flush
3353 185/300 Read block 821 len 28672
3354 186/300 Read block 1456 len 20480
3355 187/300 Read block 1190 len 32768
3356 188/300 Write block 1880 len 28672 data: 3 3 3 3 3 2 2
3357 189/300 Flush
3358 190/300 Read block 2745 len 16384
3359 191/300 Write block 4638 len 8192 data: 3 3
3360 192/300 Read block 1664 len 40960
3361 193/300 Read block 400 len 40960
3362 194/300 Write block 1731 len 28672 data: 2 2 2 2 2 2 2
3363 195/300 Write block 2373 len 32768 data: 3 4 4 4 3 4 4 4
3364 196/300 Write block 190 len 12288 data: 4 4 4
3365 197/300 Write block 1997 len 36864 data: 2 2 2 2 2 2 2 2 2
3366 198/300 Write block 3506 len 36864 data: 4 4 5 4 3 2 2 2 2
3367 199/300 Write block 846 len 12288 data: 4 2 2
3368 200/300 Flush
3369 201/300 Read block 103 len 28672
3370 202/300 Read block 3202 len 12288
3371 203/300 Read block 774 len 8192
3372 204/300 Read block 3202 len 16384
3373 205/300 Read block 749 len 12288
3374 206/300 Read block 1392 len 20480
3375 207/300 Read block 939 len 8192
3376 208/300 Flush
3377 209/300 Flush
3378 210/300 Read block 1168 len 32768
3379 211/300 Write block 830 len 16384 data: 5 3 3 3
3380 212/300 Read block 924 len 16384
3381 213/300 Read block 1462 len 24576
3382 214/300 Read block 270 len 20480
3383 215/300 Read block 3681 len 20480
3384 216/300 Write block 4641 len 12288 data: 3 3 3
3385 217/300 Flush
3386 218/300 Read block 2909 len 12288
3387 219/300 Write block 814 len 20480 data: 6 5 4 3 3
3388 220/300 Flush
3389 221/300 Write block 2749 len 16384 data: 3 3 3 3
3390 222/300 Write block 2829 len 24576 data: 2 2 2 2 2 2
3391 223/300 Read block 4954 len 36864
3392 224/300 Read block 4088 len 8192
3393 225/300 Read block 4965 len 8192
3394 226/300 Read block 4795 len 16384
3395 227/300 Write block 2571 len 32768 data: 3 3 2 2 2 2 2 2
3396 228/300 Write block 9 len 24576 data: 4 4 4 2 2 2
3397 229/300 Flush
3398 230/300 Write block 4215 len 28672 data: 2 3 3 3 3 3 3
3399 231/300 Write block 3163 len 16384 data: 2 2 2 2
3400 232/300 Write block 862 len 24576 data: 3 3 3 3 3 3
3401 233/300 Read block 79 len 28672
3402 234/300 Write block 4150 len 24576 data: 3 3 4 4 4 4
3403 235/300 Flush
3404 236/300 Read block 4392 len 36864
3405 237/300 Read block 1805 len 36864
3406 238/300 Read block 3425 len 32768
3407 239/300 Read block 2982 len 4096
3408 240/300 Read block 2131 len 16384
3409 241/300 Read block 3872 len 36864
3410 242/300 Flush
3411 243/300 Read block 1518 len 28672
3412 244/300 Read block 4062 len 20480
3413 245/300 Write block 3614 len 4096 data: 2
3414 246/300 Read block 1686 len 24576
3415 247/300 Write block 109 len 4096 data: 2
3416 248/300 Flush
3417 249/300 Flush
3418 250/300 Flush
3419 251/300 Read block 4777 len 20480
3420 252/300 Write block 4967 len 4096 data: 3
3421 253/300 Read block 392 len 20480
3422 254/300 Flush
3423 255/300 Read block 467 len 20480
3424 256/300 Write block 4392 len 4096 data: 2
3425 257/300 Write block 1749 len 32768 data: 2 2 2 2 2 2 2 2
3426 258/300 Read block 3832 len 8192
3427 259/300 Read block 2653 len 12288
3428 260/300 Write block 1134 len 20480 data: 2 2 2 2 2
3429 261/300 Write block 3133 len 12288 data: 2 2 2
3430 262/300 Read block 237 len 24576
3431 263/300 Read block 3585 len 20480
3432 264/300 Flush
3433 265/300 Write block 1712 len 8192 data: 3 3
3434 266/300 Write block 3642 len 36864 data: 3 3 3 3 2 2 2 3 4
3435 267/300 Read block 1395 len 4096
3436 268/300 Flush
3437 269/300 Write block 2768 len 4096 data: 2
3438 270/300 Write block 4545 len 32768 data: 2 2 2 2 3 2 2 2
3439 271/300 Write block 1021 len 4096 data: 2
3440 272/300 Write block 2535 len 32768 data: 2 2 2 2 2 2 2 2
3441 273/300 Write block 754 len 36864 data: 2 2 2 2 2 2 2 2 2
3442 274/300 Read block 2912 len 20480
3443 275/300 Write block 4102 len 16384 data: 2 2 2 2
3444 276/300 Write block 151 len 28672 data: 2 2 2 2 2 3 3
3445 277/300 Read block 533 len 24576
3446 278/300 Flush
3447 279/300 Flush
3448 280/300 Read block 4339 len 12288
3449 281/300 Flush
3450 282/300 Read block 508 len 20480
3451 283/300 Write block 1607 len 8192 data: 3 3
3452 284/300 Read block 3099 len 28672
3453 285/300 Read block 4209 len 8192
3454 286/300 Write block 1966 len 8192 data: 4 4
3455 287/300 Read block 102 len 32768
3456 288/300 Write block 962 len 24576 data: 2 3 3 4 4 4
3457 289/300 Write block 618 len 4096 data: 2
3458 290/300 Write block 1122 len 20480 data: 2 2 4 4 3
3459 291/300 Read block 168 len 32768
3460 292/300 Write block 826 len 24576 data: 4 4 5 5 6 4
3461 293/300 Write block 3883 len 36864 data: 3 2 2 3 3 3 3 3 3
3462 294/300 Flush
3463 295/300 Flush
3464 296/300 Read block 3802 len 4096
3465 297/300 Read block 4254 len 36864
3466 298/300 Read block 2315 len 8192
3467 299/300 Write block 2718 len 12288 data: 4 4 3
3468 300/300 Read block 791 len 24576
3469 301/300 Read block 1439 len 4096
3470 Main task has work to do, go find it
3471 got dsc Start(0)
3472 start 0
3473 Nov 28 04:51:03.978 INFO request completed, latency_us: 207, response_code: 204, uri: /start/cid/0, method: POST, req_id: 6b603711-b6f0-43c9-add1-d00acdb10c77, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
3474 Replay: started 0, returned:Ok(())
3475 ----------------------------------------------------------------
3476 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
3477 States: Offline Active Active
3478 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
3479 1906 Acked Read 3 Sent Done Done false
3480 1907 Acked Read 7 Sent Done Done false
3481 1908 Acked Flush 0 Sent Done Done false
3482 1909 Acked Write 1 Sent Done Done false
3483 1910 Acked Read 9 Sent Done Done false
3484 1911 Acked Read 7 Sent Done Done false
3485 1912 Acked Read 1 Sent Done Done false
3486 1913 Acked Write 4 Sent Done Done false
3487 1914 Acked Read 7 Sent Done Done false
3488 1915 Acked Read 7 Sent Done Done false
3489 1916 Acked Write 5 Sent Done Done false
3490 1917 Acked Read 4 Sent Done Done false
3491 1918 Acked Write 2 Sent Done Done false
3492 1919 Acked Read 1 Sent Done Done false
3493 1920 Acked Read 2 Sent Done Done false
3494 1921 Acked Write 5 Sent Done Done false
3495 1922 Acked Flush 0 Sent Done Done false
3496 1923 Acked Read 6 Sent Done Done false
3497 1924 Acked Write 3 Sent Done Done false
3498 1925 Acked Read 5 Sent Done Done false
3499 1926 Acked Read 10 Sent Done Done false
3500 1927 Acked Write 2 Sent Done Done false
3501 1928 Acked Read 9 Sent Done Done false
3502 1929 Acked Write 1 Sent Done Done false
3503 1930 Acked Read 7 Sent Done Done false
3504 1931 Acked Read 4 Sent Done Done false
3505 1932 Acked Read 1 Sent Done Done false
3506 1933 Acked Write 4 Sent Done Done false
3507 1934 Acked Read 4 Sent Done Done false
3508 1935 Acked Read 9 Sent Done Done false
3509 1936 Acked Read 9 Sent Done Done false
3510 1937 Acked Write 10 Sent Done Done false
3511 1938 Acked Read 1 Sent Done Done false
3512 1939 Acked Flush 0 Sent Done Done false
3513 1940 Acked Read 10 Sent Done Done false
3514 1941 Acked Flush 0 Sent Done Done false
3515 1942 Acked Write 3 Sent Done Done false
3516 1943 Acked Write 1 Sent Done Done false
3517 1944 Acked Write 10 Sent Done Done false
3518 1945 Acked Write 2 Sent Done Done false
3519 1946 Acked Read 6 Sent Done Done false
3520 1947 Acked Read 9 Sent Done Done false
3521 1948 Acked Write 3 Sent Done Done false
3522 1949 Acked Write 7 Sent Done Done false
3523 1950 Acked Flush 0 Sent Done Done false
3524 1951 Acked Write 8 Sent Done Done false
3525 1952 Acked Read 4 Sent Done Done false
3526 1953 Acked Read 2 Sent Done Done false
3527 1954 Acked Write 5 Sent Done Done false
3528 1955 Acked Read 3 Sent Done Done false
3529 1956 Acked Write 8 Sent Done Done false
3530 1957 Acked Read 5 Sent Done Done false
3531 1958 Acked Read 7 Sent Done Done false
3532 1959 Acked Write 7 Sent Done Done false
3533 1960 Acked Write 6 Sent Done Done false
3534 1961 Acked Read 2 Sent Done Done false
3535 1962 Acked Read 7 Sent Done Done false
3536 1963 Acked Read 6 Sent Done Done false
3537 1964 Acked Write 10 Sent Done Done false
3538 1965 Acked Read 5 Sent Done Done false
3539 1966 Acked Read 7 Sent Done Done false
3540 1967 Acked Write 5 Sent Done Done false
3541 1968 Acked Write 3 Sent Done Done false
3542 1969 Acked Read 6 Sent Done Done false
3543 1970 Acked Write 8 Sent Done Done false
3544 1971 Acked Read 9 Sent Done Done false
3545 1972 Acked Read 3 Sent Done Done false
3546 1973 Acked Read 8 Sent Done Done false
3547 1974 Acked Flush 0 Sent Done Done false
3548 1975 Acked Write 3 Sent Done Done false
3549 1976 Acked Read 2 Sent Done Done false
3550 1977 Acked Write 5 Sent Done Done false
3551 1978 Acked Read 7 Sent Done Done false
3552 1979 Acked Read 8 Sent Done Done false
3553 1980 Acked Write 4 Sent Done Done false
3554 1981 Acked Flush 0 Sent Done Done false
3555 1982 Acked Write 9 Sent Done Done false
3556 1983 Acked Read 7 Sent Done Done false
3557 1984 Acked Read 6 Sent Done Done false
3558 1985 Acked Write 3 Sent Done Done false
3559 1986 Acked Read 4 Sent Done Done false
3560 1987 Acked Write 3 Sent Done Done false
3561 1988 Acked Read 10 Sent Done Done false
3562 1989 Acked Read 7 Sent Done Done false
3563 1990 Acked Read 5 Sent Done Done false
3564 1991 Acked Read 6 Sent Done Done false
3565 1992 Acked Read 6 Sent Done Done false
3566 1993 Acked Write 7 Sent Done Done false
3567 1994 Acked Write 1 Sent Done Done false
3568 1995 Acked Write 3 Sent Done Done false
3569 1996 Acked Write 7 Sent Done Done false
3570 1997 Acked Read 3 Sent Done Done false
3571 1998 Acked Write 6 Sent Done Done false
3572 1999 Acked Read 8 Sent Done Done false
3573 2000 Acked Write 10 Sent Done Done false
3574 2001 Acked Read 2 Sent Done Done false
3575 2002 Acked Write 8 Sent Done Done false
3576 2003 Acked Read 4 Sent Done Done false
3577 2004 Acked Write 3 Sent Done Done false
3578 2005 Acked Read 6 Sent Done Done false
3579 2006 Acked Write 3 Sent Done Done false
3580 2007 Acked Read 3 Sent Done Done false
3581 2008 Acked Write 10 Sent Done Done false
3582 2009 Acked Read 1 Sent Done Done false
3583 2010 Acked Read 5 Sent Done Done false
3584 2011 Acked Write 4 Sent Done Done false
3585 2012 Acked Read 5 Sent Done Done false
3586 2013 Acked Write 2 Sent Done Done false
3587 2014 Acked Read 6 Sent Done Done false
3588 2015 Acked Read 9 Sent Done Done false
3589 2016 Acked Read 1 Sent Done Done false
3590 2017 Acked Read 5 Sent Done Done false
3591 2018 Acked Read 8 Sent Done Done false
3592 2019 Acked Write 9 Sent Done Done false
3593 2020 Acked Flush 0 Sent Done Done false
3594 2021 Acked Read 4 Sent Done Done false
3595 2022 Acked Read 2 Sent Done Done false
3596 2023 Acked Read 10 Sent Done Done false
3597 2024 Acked Write 5 Sent Done Done false
3598 2025 Acked Write 6 Sent Done Done false
3599 2026 Acked Read 9 Sent Done Done false
3600 2027 Acked Read 1 Sent Done Done false
3601 2028 Acked Write 8 Sent Done Done false
3602 2029 Acked Write 10 Sent Done Done false
3603 2030 Acked Read 6 Sent Done Done false
3604 2031 Acked Write 3 Sent Done Done false
3605 2032 Acked Write 5 Sent Done Done false
3606 2033 Acked Read 8 Sent Done Done false
3607 2034 Acked Read 8 Sent Done Done false
3608 2035 Acked Write 3 Sent Done Done false
3609 2036 Acked Read 9 Sent Done Done false
3610 2037 Acked Write 3 Sent Done Done false
3611 2038 Acked Read 7 Sent Done Done false
3612 2039 Acked Flush 0 Sent Done Done false
3613 2040 Acked Read 7 Sent Done Done false
3614 2041 Acked Read 1 Sent Done Done false
3615 2042 Acked Read 1 Sent Done Done false
3616 2043 Acked Write 6 Sent Done Done false
3617 2044 Acked Write 8 Sent Done Done false
3618 2045 Acked Read 8 Sent Done Done false
3619 2046 Acked Write 7 Sent Done Done false
3620 2047 Acked Read 3 Sent Done Done false
3621 2048 Acked Write 6 Sent Done Done false
3622 2049 Acked Read 8 Sent Done Done false
3623 2050 Acked Read 2 Sent Done Done false
3624 2051 Acked Flush 0 Sent Done Done false
3625 2052 Acked Read 2 Sent Done Done false
3626 2053 Acked Read 7 Sent Done Done false
3627 2054 Acked Write 5 Sent Done Done false
3628 2055 Acked Write 2 Sent Done Done false
3629 2056 Acked Write 6 Sent Done Done false
3630 2057 Acked Write 5 Sent Done Done false
3631 2058 Acked Write 7 Sent Done Done false
3632 2059 Acked Read 5 Sent Done Done false
3633 2060 Acked Write 2 Sent Done Done false
3634 2061 Acked Read 1 Sent Done Done false
3635 2062 Acked Write 6 Sent Done Done false
3636 2063 Acked Write 6 Sent Done Done false
3637 2064 Acked Flush 0 Sent Done Done false
3638 2065 Acked Read 1 Sent Done Done false
3639 2066 Acked Flush 0 Sent Done Done false
3640 2067 Acked Read 8 Sent Done Done false
3641 2068 Acked Read 8 Sent Done Done false
3642 2069 Acked Write 4 Sent Done Done false
3643 2070 Acked Read 8 Sent Done Done false
3644 2071 Acked Read 3 Sent Done Done false
3645 2072 Acked Write 2 Sent Done Done false
3646 2073 Acked Read 6 Sent Done Done false
3647 2074 Acked Write 3 Sent Done Done false
3648 2075 Acked Flush 0 Sent Done Done false
3649 2076 Acked Read 1 Sent Done Done false
3650 2077 Acked Read 9 Sent Done Done false
3651 2078 Acked Read 8 Sent Done Done false
3652 2079 Acked Read 9 Sent Done Done false
3653 2080 Acked Read 7 Sent Done Done false
3654 2081 Acked Write 1 Sent Done Done false
3655 2082 Acked Read 5 Sent Done Done false
3656 2083 Acked Read 4 Sent Done Done false
3657 2084 Acked Write 10 Sent Done Done false
3658 2085 Acked Read 9 Sent Done Done false
3659 2086 Acked Flush 0 Sent Done Done false
3660 2087 Acked Read 5 Sent Done Done false
3661 2088 Acked Write 7 Sent Done Done false
3662 2089 Acked Flush 0 Sent Done Done false
3663 2090 Acked Read 7 Sent Done Done false
3664 2091 Acked Read 5 Sent Done Done false
3665 2092 Acked Read 8 Sent Done Done false
3666 2093 Acked Write 7 Sent Done Done false
3667 2094 Acked Flush 0 Sent Done Done false
3668 2095 Acked Read 4 Sent Done Done false
3669 2096 Acked Write 2 Sent Done Done false
3670 2097 Acked Read 10 Sent Done Done false
3671 2098 Acked Read 10 Sent Done Done false
3672 2099 Acked Write 7 Sent Done Done false
3673 2100 Acked Write 8 Sent Done Done false
3674 2101 Acked Write 3 Sent Done Done false
3675 2102 Acked Write 9 Sent Done Done false
3676 2103 Acked Write 9 Sent Done Done false
3677 2104 Acked Write 3 Sent Done Done false
3678 2105 Acked Flush 0 Sent Done Done false
3679 2106 Acked Read 7 Sent Done Done false
3680 2107 Acked Read 3 Sent Done Done false
3681 2108 Acked Read 2 Sent Done Done false
3682 2109 Acked Read 4 Sent Done Done false
3683 2110 Acked Read 3 Sent Done Done false
3684 2111 Acked Read 5 Sent Done Done false
3685 2112 Acked Read 2 Sent Done Done false
3686 2113 Acked Flush 0 Sent Done Done false
3687 2114 Acked Flush 0 Sent Done Done false
3688 2115 Acked Read 8 Sent Done Done false
3689 2116 Acked Write 4 Sent Done Done false
3690 2117 Acked Read 4 Sent Done Done false
3691 2118 Acked Read 6 Sent Done Done false
3692 2119 Acked Read 5 Sent Done Done false
3693 2120 Acked Read 5 Sent Done Done false
3694 2121 Acked Write 3 Sent Done Done false
3695 2122 Acked Flush 0 Sent Done Done false
3696 2123 Acked Read 3 Sent Done Done false
3697 2124 Acked Write 5 Sent Done Done false
3698 2125 Acked Flush 0 Sent Done Done false
3699 2126 Acked Write 4 Sent Done Done false
3700 2127 Acked Write 6 Sent Done Done false
3701 2128 Acked Read 9 Sent Done Done false
3702 2129 Acked Read 2 Sent Done Done false
3703 2130 Acked Read 2 Sent Done Done false
3704 2131 Acked Read 4 Sent Done Done false
3705 2132 Acked Write 8 Sent Done Done false
3706 2133 Acked Write 6 Sent Done Done false
3707 2134 Acked Flush 0 Sent Done Done false
3708 2135 Acked Write 7 Sent Done Done false
3709 2136 Acked Write 4 Sent Done Done false
3710 2137 Acked Write 6 Sent Done Done false
3711 2138 Acked Read 7 Sent Done Done false
3712 2139 Acked Write 6 Sent Done Done false
3713 2140 Acked Flush 0 Sent Done Done false
3714 2141 Acked Read 9 Sent Done Done false
3715 2142 Acked Read 9 Sent Done Done false
3716 2143 Acked Read 8 Sent Done Done false
3717 2144 Acked Read 1 Sent Done Done false
3718 2145 Acked Read 4 Sent Done Done false
3719 2146 Acked Read 9 Sent Done Done false
3720 2147 Acked Flush 0 Sent Done Done false
3721 2148 Acked Read 7 Sent Done Done false
3722 2149 Acked Read 5 Sent Done Done false
3723 2150 Acked Write 1 Sent Done Done false
3724 2151 Acked Read 6 Sent Done Done false
3725 2152 Acked Write 1 Sent Done Done false
3726 2153 Acked Flush 0 Sent Done Done false
3727 2154 Acked Flush 0 Sent Done Done false
3728 2155 Acked Flush 0 Sent Done Done false
3729 2156 Acked Read 5 Sent Done Done false
3730 2157 Acked Write 1 Sent Done Done false
3731 2158 Acked Read 5 Sent Done Done false
3732 2159 Acked Flush 0 Sent Done Done false
3733 2160 Acked Read 5 Sent Done Done false
3734 2161 Acked Write 1 Sent Done Done false
3735 2162 Acked Write 8 Sent Done Done false
3736 2163 Acked Read 2 Sent Done Done false
3737 2164 Acked Read 3 Sent Done Done false
3738 2165 Acked Write 5 Sent Done Done false
3739 2166 Acked Write 3 Sent Done Done false
3740 2167 Acked Read 6 Sent Done Done false
3741 2168 Acked Read 5 Sent Done Done false
3742 2169 Acked Flush 0 Sent Done Done false
3743 2170 Acked Write 2 Sent Done Done false
3744 2171 Acked Write 9 Sent Done Done false
3745 2172 Acked Read 1 Sent Done Done false
3746 2173 Acked Flush 0 Sent Done Done false
3747 2174 Acked Write 1 Sent Done Done false
3748 2175 Acked Write 8 Sent Done Done false
3749 2176 Acked Write 1 Sent Done Done false
3750 2177 Acked Write 8 Sent Done Done false
3751 2178 Acked Write 9 Sent Done Done false
3752 2179 Acked Read 5 Sent Done Done false
3753 2180 Acked Write 4 Sent Done Done false
3754 2181 Acked Write 7 Sent Done Done false
3755 2182 Acked Read 6 Sent Done Done false
3756 2183 Acked Flush 0 Sent Done Done false
3757 2184 Acked Flush 0 Sent Done Done false
3758 2185 Acked Read 3 Sent Done Done false
3759 2186 Acked Flush 0 Sent Done Done false
3760 2187 Acked Read 5 Sent Done Done false
3761 2188 Acked Write 2 Sent Done Done false
3762 2189 Acked Read 7 Sent Done Done false
3763 2190 Acked Read 2 Sent Done Done false
3764 2191 Acked Write 2 Sent Done Done false
3765 2192 Acked Read 8 Sent Done Done false
3766 2193 Acked Write 6 Sent Done Done false
3767 2194 Acked Write 1 Sent Done Done false
3768 2195 Acked Write 5 Sent Done Done false
3769 2196 Acked Read 8 Sent Done Done false
3770 2197 Acked Write 6 Sent Done Done false
3771 2198 Acked Write 9 Sent Done Done false
3772 2199 Acked Flush 0 Sent Done Done false
3773 2200 Acked Flush 0 Sent Done Done false
3774 2201 Acked Read 1 Sent Done Done false
3775 2202 Acked Read 9 Sent Done Done false
3776 2203 Acked Read 2 Sent Done Done false
3777 2204 Acked Write 3 Sent Done Done false
3778 2205 Acked Read 6 Sent Done Done false
3779 2206 Acked Read 1 Sent Done Done false
3780 STATES DS:0 DS:1 DS:2 TOTAL
3781 Sent 301 0 0 301
3782 Done 0 301 301 602
3783 Skipped 0 0 0 0
3784 Error 0 0 0 0
3785 Last Flush: 1905 2200 2200
3786 Downstairs last five completed: 1905 1904 1903 1902 1901
3787 Upstairs last five completed: 2206 2205 2204 2203 2202
3788 CLIENT: Up:0 ds:301 act:2
3789 [8810] got start action
3790 [8810] I am going to restart
3791 Starting downstairs at port 8810
3792 Make output file at "/tmp/dsc/downstairs-8810.txt"
3793 [8810][0] reports Starting
3794 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1206)
3795 [8810][0] reports Running
3796 ----------------------------------------------------------------
3797 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
3798 States: Offline Active Active
3799 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
3800 1906 Acked Read 3 Sent Done Done false
3801 1907 Acked Read 7 Sent Done Done false
3802 1908 Acked Flush 0 Sent Done Done false
3803 1909 Acked Write 1 Sent Done Done false
3804 1910 Acked Read 9 Sent Done Done false
3805 1911 Acked Read 7 Sent Done Done false
3806 1912 Acked Read 1 Sent Done Done false
3807 1913 Acked Write 4 Sent Done Done false
3808 1914 Acked Read 7 Sent Done Done false
3809 1915 Acked Read 7 Sent Done Done false
3810 1916 Acked Write 5 Sent Done Done false
3811 1917 Acked Read 4 Sent Done Done false
3812 1918 Acked Write 2 Sent Done Done false
3813 1919 Acked Read 1 Sent Done Done false
3814 1920 Acked Read 2 Sent Done Done false
3815 1921 Acked Write 5 Sent Done Done false
3816 1922 Acked Flush 0 Sent Done Done false
3817 1923 Acked Read 6 Sent Done Done false
3818 1924 Acked Write 3 Sent Done Done false
3819 1925 Acked Read 5 Sent Done Done false
3820 1926 Acked Read 10 Sent Done Done false
3821 1927 Acked Write 2 Sent Done Done false
3822 1928 Acked Read 9 Sent Done Done false
3823 1929 Acked Write 1 Sent Done Done false
3824 1930 Acked Read 7 Sent Done Done false
3825 1931 Acked Read 4 Sent Done Done false
3826 1932 Acked Read 1 Sent Done Done false
3827 1933 Acked Write 4 Sent Done Done false
3828 1934 Acked Read 4 Sent Done Done false
3829 1935 Acked Read 9 Sent Done Done false
3830 1936 Acked Read 9 Sent Done Done false
3831 1937 Acked Write 10 Sent Done Done false
3832 1938 Acked Read 1 Sent Done Done false
3833 1939 Acked Flush 0 Sent Done Done false
3834 1940 Acked Read 10 Sent Done Done false
3835 1941 Acked Flush 0 Sent Done Done false
3836 1942 Acked Write 3 Sent Done Done false
3837 1943 Acked Write 1 Sent Done Done false
3838 1944 Acked Write 10 Sent Done Done false
3839 1945 Acked Write 2 Sent Done Done false
3840 1946 Acked Read 6 Sent Done Done false
3841 1947 Acked Read 9 Sent Done Done false
3842 1948 Acked Write 3 Sent Done Done false
3843 1949 Acked Write 7 Sent Done Done false
3844 1950 Acked Flush 0 Sent Done Done false
3845 1951 Acked Write 8 Sent Done Done false
3846 1952 Acked Read 4 Sent Done Done false
3847 1953 Acked Read 2 Sent Done Done false
3848 1954 Acked Write 5 Sent Done Done false
3849 1955 Acked Read 3 Sent Done Done false
3850 1956 Acked Write 8 Sent Done Done false
3851 1957 Acked Read 5 Sent Done Done false
3852 1958 Acked Read 7 Sent Done Done false
3853 1959 Acked Write 7 Sent Done Done false
3854 1960 Acked Write 6 Sent Done Done false
3855 1961 Acked Read 2 Sent Done Done false
3856 1962 Acked Read 7 Sent Done Done false
3857 1963 Acked Read 6 Sent Done Done false
3858 1964 Acked Write 10 Sent Done Done false
3859 1965 Acked Read 5 Sent Done Done false
3860 1966 Acked Read 7 Sent Done Done false
3861 1967 Acked Write 5 Sent Done Done false
3862 1968 Acked Write 3 Sent Done Done false
3863 1969 Acked Read 6 Sent Done Done false
3864 1970 Acked Write 8 Sent Done Done false
3865 1971 Acked Read 9 Sent Done Done false
3866 1972 Acked Read 3 Sent Done Done false
3867 1973 Acked Read 8 Sent Done Done false
3868 1974 Acked Flush 0 Sent Done Done false
3869 1975 Acked Write 3 Sent Done Done false
3870 1976 Acked Read 2 Sent Done Done false
3871 1977 Acked Write 5 Sent Done Done false
3872 1978 Acked Read 7 Sent Done Done false
3873 1979 Acked Read 8 Sent Done Done false
3874 1980 Acked Write 4 Sent Done Done false
3875 1981 Acked Flush 0 Sent Done Done false
3876 1982 Acked Write 9 Sent Done Done false
3877 1983 Acked Read 7 Sent Done Done false
3878 1984 Acked Read 6 Sent Done Done false
3879 1985 Acked Write 3 Sent Done Done false
3880 1986 Acked Read 4 Sent Done Done false
3881 1987 Acked Write 3 Sent Done Done false
3882 1988 Acked Read 10 Sent Done Done false
3883 1989 Acked Read 7 Sent Done Done false
3884 1990 Acked Read 5 Sent Done Done false
3885 1991 Acked Read 6 Sent Done Done false
3886 1992 Acked Read 6 Sent Done Done false
3887 1993 Acked Write 7 Sent Done Done false
3888 1994 Acked Write 1 Sent Done Done false
3889 1995 Acked Write 3 Sent Done Done false
3890 1996 Acked Write 7 Sent Done Done false
3891 1997 Acked Read 3 Sent Done Done false
3892 1998 Acked Write 6 Sent Done Done false
3893 1999 Acked Read 8 Sent Done Done false
3894 2000 Acked Write 10 Sent Done Done false
3895 2001 Acked Read 2 Sent Done Done false
3896 2002 Acked Write 8 Sent Done Done false
3897 2003 Acked Read 4 Sent Done Done false
3898 2004 Acked Write 3 Sent Done Done false
3899 2005 Acked Read 6 Sent Done Done false
3900 2006 Acked Write 3 Sent Done Done false
3901 2007 Acked Read 3 Sent Done Done false
3902 2008 Acked Write 10 Sent Done Done false
3903 2009 Acked Read 1 Sent Done Done false
3904 2010 Acked Read 5 Sent Done Done false
3905 2011 Acked Write 4 Sent Done Done false
3906 2012 Acked Read 5 Sent Done Done false
3907 2013 Acked Write 2 Sent Done Done false
3908 2014 Acked Read 6 Sent Done Done false
3909 2015 Acked Read 9 Sent Done Done false
3910 2016 Acked Read 1 Sent Done Done false
3911 2017 Acked Read 5 Sent Done Done false
3912 2018 Acked Read 8 Sent Done Done false
3913 2019 Acked Write 9 Sent Done Done false
3914 2020 Acked Flush 0 Sent Done Done false
3915 2021 Acked Read 4 Sent Done Done false
3916 2022 Acked Read 2 Sent Done Done false
3917 2023 Acked Read 10 Sent Done Done false
3918 2024 Acked Write 5 Sent Done Done false
3919 2025 Acked Write 6 Sent Done Done false
3920 2026 Acked Read 9 Sent Done Done false
3921 2027 Acked Read 1 Sent Done Done false
3922 2028 Acked Write 8 Sent Done Done false
3923 2029 Acked Write 10 Sent Done Done false
3924 2030 Acked Read 6 Sent Done Done false
3925 2031 Acked Write 3 Sent Done Done false
3926 2032 Acked Write 5 Sent Done Done false
3927 2033 Acked Read 8 Sent Done Done false
3928 2034 Acked Read 8 Sent Done Done false
3929 2035 Acked Write 3 Sent Done Done false
3930 2036 Acked Read 9 Sent Done Done false
3931 2037 Acked Write 3 Sent Done Done false
3932 2038 Acked Read 7 Sent Done Done false
3933 2039 Acked Flush 0 Sent Done Done false
3934 2040 Acked Read 7 Sent Done Done false
3935 2041 Acked Read 1 Sent Done Done false
3936 2042 Acked Read 1 Sent Done Done false
3937 2043 Acked Write 6 Sent Done Done false
3938 2044 Acked Write 8 Sent Done Done false
3939 2045 Acked Read 8 Sent Done Done false
3940 2046 Acked Write 7 Sent Done Done false
3941 2047 Acked Read 3 Sent Done Done false
3942 2048 Acked Write 6 Sent Done Done false
3943 2049 Acked Read 8 Sent Done Done false
3944 2050 Acked Read 2 Sent Done Done false
3945 2051 Acked Flush 0 Sent Done Done false
3946 2052 Acked Read 2 Sent Done Done false
3947 2053 Acked Read 7 Sent Done Done false
3948 2054 Acked Write 5 Sent Done Done false
3949 2055 Acked Write 2 Sent Done Done false
3950 2056 Acked Write 6 Sent Done Done false
3951 2057 Acked Write 5 Sent Done Done false
3952 2058 Acked Write 7 Sent Done Done false
3953 2059 Acked Read 5 Sent Done Done false
3954 2060 Acked Write 2 Sent Done Done false
3955 2061 Acked Read 1 Sent Done Done false
3956 2062 Acked Write 6 Sent Done Done false
3957 2063 Acked Write 6 Sent Done Done false
3958 2064 Acked Flush 0 Sent Done Done false
3959 2065 Acked Read 1 Sent Done Done false
3960 2066 Acked Flush 0 Sent Done Done false
3961 2067 Acked Read 8 Sent Done Done false
3962 2068 Acked Read 8 Sent Done Done false
3963 2069 Acked Write 4 Sent Done Done false
3964 2070 Acked Read 8 Sent Done Done false
3965 2071 Acked Read 3 Sent Done Done false
3966 2072 Acked Write 2 Sent Done Done false
3967 2073 Acked Read 6 Sent Done Done false
3968 2074 Acked Write 3 Sent Done Done false
3969 2075 Acked Flush 0 Sent Done Done false
3970 2076 Acked Read 1 Sent Done Done false
3971 2077 Acked Read 9 Sent Done Done false
3972 2078 Acked Read 8 Sent Done Done false
3973 2079 Acked Read 9 Sent Done Done false
3974 2080 Acked Read 7 Sent Done Done false
3975 2081 Acked Write 1 Sent Done Done false
3976 2082 Acked Read 5 Sent Done Done false
3977 2083 Acked Read 4 Sent Done Done false
3978 2084 Acked Write 10 Sent Done Done false
3979 2085 Acked Read 9 Sent Done Done false
3980 2086 Acked Flush 0 Sent Done Done false
3981 2087 Acked Read 5 Sent Done Done false
3982 2088 Acked Write 7 Sent Done Done false
3983 2089 Acked Flush 0 Sent Done Done false
3984 2090 Acked Read 7 Sent Done Done false
3985 2091 Acked Read 5 Sent Done Done false
3986 2092 Acked Read 8 Sent Done Done false
3987 2093 Acked Write 7 Sent Done Done false
3988 2094 Acked Flush 0 Sent Done Done false
3989 2095 Acked Read 4 Sent Done Done false
3990 2096 Acked Write 2 Sent Done Done false
3991 2097 Acked Read 10 Sent Done Done false
3992 2098 Acked Read 10 Sent Done Done false
3993 2099 Acked Write 7 Sent Done Done false
3994 2100 Acked Write 8 Sent Done Done false
3995 2101 Acked Write 3 Sent Done Done false
3996 2102 Acked Write 9 Sent Done Done false
3997 2103 Acked Write 9 Sent Done Done false
3998 2104 Acked Write 3 Sent Done Done false
3999 2105 Acked Flush 0 Sent Done Done false
4000 2106 Acked Read 7 Sent Done Done false
4001 2107 Acked Read 3 Sent Done Done false
4002 2108 Acked Read 2 Sent Done Done false
4003 2109 Acked Read 4 Sent Done Done false
4004 2110 Acked Read 3 Sent Done Done false
4005 2111 Acked Read 5 Sent Done Done false
4006 2112 Acked Read 2 Sent Done Done false
4007 2113 Acked Flush 0 Sent Done Done false
4008 2114 Acked Flush 0 Sent Done Done false
4009 2115 Acked Read 8 Sent Done Done false
4010 2116 Acked Write 4 Sent Done Done false
4011 2117 Acked Read 4 Sent Done Done false
4012 2118 Acked Read 6 Sent Done Done false
4013 2119 Acked Read 5 Sent Done Done false
4014 2120 Acked Read 5 Sent Done Done false
4015 2121 Acked Write 3 Sent Done Done false
4016 2122 Acked Flush 0 Sent Done Done false
4017 2123 Acked Read 3 Sent Done Done false
4018 2124 Acked Write 5 Sent Done Done false
4019 2125 Acked Flush 0 Sent Done Done false
4020 2126 Acked Write 4 Sent Done Done false
4021 2127 Acked Write 6 Sent Done Done false
4022 2128 Acked Read 9 Sent Done Done false
4023 2129 Acked Read 2 Sent Done Done false
4024 2130 Acked Read 2 Sent Done Done false
4025 2131 Acked Read 4 Sent Done Done false
4026 2132 Acked Write 8 Sent Done Done false
4027 2133 Acked Write 6 Sent Done Done false
4028 2134 Acked Flush 0 Sent Done Done false
4029 2135 Acked Write 7 Sent Done Done false
4030 2136 Acked Write 4 Sent Done Done false
4031 2137 Acked Write 6 Sent Done Done false
4032 2138 Acked Read 7 Sent Done Done false
4033 2139 Acked Write 6 Sent Done Done false
4034 2140 Acked Flush 0 Sent Done Done false
4035 2141 Acked Read 9 Sent Done Done false
4036 2142 Acked Read 9 Sent Done Done false
4037 2143 Acked Read 8 Sent Done Done false
4038 2144 Acked Read 1 Sent Done Done false
4039 2145 Acked Read 4 Sent Done Done false
4040 2146 Acked Read 9 Sent Done Done false
4041 2147 Acked Flush 0 Sent Done Done false
4042 2148 Acked Read 7 Sent Done Done false
4043 2149 Acked Read 5 Sent Done Done false
4044 2150 Acked Write 1 Sent Done Done false
4045 2151 Acked Read 6 Sent Done Done false
4046 2152 Acked Write 1 Sent Done Done false
4047 2153 Acked Flush 0 Sent Done Done false
4048 2154 Acked Flush 0 Sent Done Done false
4049 2155 Acked Flush 0 Sent Done Done false
4050 2156 Acked Read 5 Sent Done Done false
4051 2157 Acked Write 1 Sent Done Done false
4052 2158 Acked Read 5 Sent Done Done false
4053 2159 Acked Flush 0 Sent Done Done false
4054 2160 Acked Read 5 Sent Done Done false
4055 2161 Acked Write 1 Sent Done Done false
4056 2162 Acked Write 8 Sent Done Done false
4057 2163 Acked Read 2 Sent Done Done false
4058 2164 Acked Read 3 Sent Done Done false
4059 2165 Acked Write 5 Sent Done Done false
4060 2166 Acked Write 3 Sent Done Done false
4061 2167 Acked Read 6 Sent Done Done false
4062 2168 Acked Read 5 Sent Done Done false
4063 2169 Acked Flush 0 Sent Done Done false
4064 2170 Acked Write 2 Sent Done Done false
4065 2171 Acked Write 9 Sent Done Done false
4066 2172 Acked Read 1 Sent Done Done false
4067 2173 Acked Flush 0 Sent Done Done false
4068 2174 Acked Write 1 Sent Done Done false
4069 2175 Acked Write 8 Sent Done Done false
4070 2176 Acked Write 1 Sent Done Done false
4071 2177 Acked Write 8 Sent Done Done false
4072 2178 Acked Write 9 Sent Done Done false
4073 2179 Acked Read 5 Sent Done Done false
4074 2180 Acked Write 4 Sent Done Done false
4075 2181 Acked Write 7 Sent Done Done false
4076 2182 Acked Read 6 Sent Done Done false
4077 2183 Acked Flush 0 Sent Done Done false
4078 2184 Acked Flush 0 Sent Done Done false
4079 2185 Acked Read 3 Sent Done Done false
4080 2186 Acked Flush 0 Sent Done Done false
4081 2187 Acked Read 5 Sent Done Done false
4082 2188 Acked Write 2 Sent Done Done false
4083 2189 Acked Read 7 Sent Done Done false
4084 2190 Acked Read 2 Sent Done Done false
4085 2191 Acked Write 2 Sent Done Done false
4086 2192 Acked Read 8 Sent Done Done false
4087 2193 Acked Write 6 Sent Done Done false
4088 2194 Acked Write 1 Sent Done Done false
4089 2195 Acked Write 5 Sent Done Done false
4090 2196 Acked Read 8 Sent Done Done false
4091 2197 Acked Write 6 Sent Done Done false
4092 2198 Acked Write 9 Sent Done Done false
4093 2199 Acked Flush 0 Sent Done Done false
4094 2200 Acked Flush 0 Sent Done Done false
4095 2201 Acked Read 1 Sent Done Done false
4096 2202 Acked Read 9 Sent Done Done false
4097 2203 Acked Read 2 Sent Done Done false
4098 2204 Acked Write 3 Sent Done Done false
4099 2205 Acked Read 6 Sent Done Done false
4100 2206 Acked Read 1 Sent Done Done false
4101 2207 Acked Flush 0 Sent Done Done false
4102 STATES DS:0 DS:1 DS:2 TOTAL
4103 Sent 302 0 0 302
4104 Done 0 302 302 604
4105 Skipped 0 0 0 0
4106 Error 0 0 0 0
4107 Last Flush: 1905 2207 2207
4108 Downstairs last five completed: 1905 1904 1903 1902 1901
4109 Upstairs last five completed: 2207 2206 2205 2204 2203
4110 CLIENT: Up:0 ds:302 act:2
41112024-11-28T04:51:09.746ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
41122024-11-28T04:51:09.746ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:52516) = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
41132024-11-28T04:51:09.747ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
41142024-11-28T04:51:09.747ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
41152024-11-28T04:51:09.747ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
41162024-11-28T04:51:09.747ZINFOcrucible: send last flush ID to this DS: 1905 = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
41172024-11-28T04:51:09.748ZINFOcrucible: Replied this last flush ID: 1905 = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
41182024-11-28T04:51:09.748ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
41192024-11-28T04:51:09.748ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
41202024-11-28T04:51:09.748ZINFOcrucible: [0] client re-new 302 jobs since flush 1905 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
41212024-11-28T04:51:09.748ZINFOcrucible: [0] Replayed 302 jobs since flush: 1905 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
4122 ----------------------------------------------------------------
4123 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
4124 Downstairs last five completed: 2207 2206 2205 2204 2203
4125 Upstairs last five completed: 2207 2206 2205 2204 2203
4126 CLIENT: Up:0 ds:0 act:3
4127 Replay: All jobs finished
4128 Main task has work to do, go find it
4129 got dsc Stop(0)
4130 stop 0
4131 Nov 28 04:51:11.982 INFO request completed, latency_us[8810] Got stop action so:false kr:false
4132 : 365, response_code: 204, uri: /stop/cid/0, method: POST, req_id: a5e31103-7b96-4e16-b4d5-0f7e881c289f, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
41332024-11-28T04:51:11.983ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
41342024-11-28T04:51:11.983ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
41352024-11-28T04:51:11.983ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
41362024-11-28T04:51:11.983ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
4137 Nov 28 04:51:11.983 INFO request completed, latency_us:{"msg":"[0] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-28T04:51:11.983702255Z","hostname":"w-01JDRHKEEFJ1228TFPSBEGXWHY","pid":1202,"client":"0","":"downstairs","session_id":"4a2415b1-6594-436f-82e5-b8cf9a74eeaf"}
4138 131, response_code: 200, uri: /state/cid/0, method: GET, req_id: 680f1311-d9b0-4f84-a5d7-1006439569e7, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
41392024-11-28T04:51:11.983ZINFOcrucible: client task is exiting = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
4140 [8810] Exited with: ExitStatus(unix_wait_status(9))
4141 [8810][0] reports Exit
4142 Nov 28 04:51:15.985 INFO request completed, latency_us: 329, response_code: 200, uri: /state/cid/0, method: GET, req_id: b5b97ccd-334c-4a08-9120-d788427e7ac2, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
4143 001/300 Write block 3088 len 8192 data: 2 2
4144 002/300 Write block 3205 len 12288 data: 3 3 3
4145 003/300 Write block 3753 len 4096 data: 2
4146 004/300 Read block 2416 len 28672
4147 005/300 Read block 1917 len 20480
4148 006/300 Write block 4396 len 20480 data: 2 2 2 2 2
4149 007/300 Write block 3284 len 20480 data: 3 3 3 2 2
4150 008/300 Write block 473 len 12288 data: 2 2 2
4151 009/300 Write block 3582 len 4096 data: 3
4152 010/300 Read block 4221 len 24576
4153 011/300 Write block 2530 len 20480 data: 3 3 3 2 2
4154 012/300 Read block 2932 len 24576
4155 013/300 Read block 4033 len 32768
4156 014/300 Write block 793 len 16384 data: 2 2 2 2
4157 015/300 Write block 2168 len 32768 data: 2 2 2 2 2 2 2 2
4158 016/300 Read block 1290 len 8192
4159 017/300 Flush
4160 018/300 Read block 2145 len 36864
4161 019/300 Read block 2188 len 40960
4162 020/300 Read block 2786 len 24576
4163 021/300 Read block 4102 len 8192
4164 022/300 Read block 1694 len 28672
4165 023/300 Write block 1531 len 32768 data: 2 2 2 2 2 2 2 3
4166 024/300 Read block 1353 len 28672
4167 025/300 Read block 2803 len 24576
4168 026/300 Read block 2632 len 16384
4169 027/300 Read block 3145 len 28672
4170 028/300 Write block 4949 len 16384 data: 3 3 3 3
4171 029/300 Read block 169 len 32768
4172 030/300 Read block 4930 len 8192
4173 031/300 Read block 558 len 12288
4174 032/300 Read block 4948 len 32768
4175 033/300 Write block 1381 len 28672 data: 2 3 3 3 3 2 2
4176 034/300 Read block 3173 len 32768
4177 035/300 Write block 4625 len 8192 data: 2 2
4178 036/300 Read block 2956 len 36864
4179 037/300 Write block 1533 len 20480 data: 3 3 3 3 3
4180 038/300 Write block 202 len 36864 data: 3 2 2 2 2 2 2 2 2
4181 039/300 Read block 454 len 32768
4182 040/300 Read block 412 len 32768
4183 041/300 Write block 2675 len 40960 data: 2 2 2 2 2 2 2 2 2 2
4184 042/300 Write block 149 len 12288 data: 2 2 3
4185 043/300 Write block 3409 len 4096 data: 2
4186 044/300 Write block 898 len 36864 data: 2 2 3 3 3 3 3 3 3
4187 045/300 Flush
4188 046/300 Write block 1710 len 36864 data: 4 3 4 4 3 3 3 2 2
4189 047/300 Flush
4190 048/300 Read block 4305 len 4096
4191 049/300 Write block 3946 len 32768 data: 3 3 3 3 4 4 4 4
4192 050/300 Read block 1074 len 4096
4193 051/300 Write block 4463 len 24576 data: 2 2 2 2 2 2
4194 052/300 Read block 4182 len 36864
4195 053/300 Write block 3510 len 40960 data: 4 3 3 3 3 2 2 2 2 2
4196 054/300 Read block 2136 len 4096
4197 055/300 Read block 3241 len 32768
4198 056/300 Write block 2845 len 36864 data: 2 2 2 2 2 2 2 2 2
4199 057/300 Read block 3529 len 28672
4200 058/300 Write block 398 len 28672 data: 2 2 2 2 2 2 2
4201 059/300 Write block 2102 len 16384 data: 3 3 3 4
4202 060/300 Read block 1195 len 4096
4203 061/300 Read block 859 len 36864
4204 062/300 Write block 4059 len 16384 data: 5 5 4 4
4205 063/300 Write block 3797 len 8192 data: 3 3
4206 064/300 Write block 1515 len 40960 data: 2 2 2 2 2 2 2 2 2 3
4207 065/300 Write block 2309 len 20480 data: 3 3 3 3 3
4208 066/300 Write block 4282 len 8192 data: 2 2
4209 067/300 Write block 1075 len 28672 data: 3 3 3 3 3 2 2
4210 068/300 Read block 1663 len 40960
4211 069/300 Read block 4377 len 36864
4212 070/300 Read block 661 len 20480
4213 071/300 Read block 4869 len 20480
4214 072/300 Read block 3903 len 12288
4215 073/300 Read block 3103 len 12288
4216 074/300 Write block 1482 len 8192 data: 2 2
4217 075/300 Write block 536 len 36864 data: 5 4 4 4 4 3 3 3 3
4218 076/300 Read block 163 len 16384
4219 077/300 Read block 3755 len 40960
4220 078/300 Read block 4055 len 8192
4221 079/300 Read block 4913 len 24576
4222 080/300 Read block 3482 len 4096
4223 081/300 Write block 103 len 28672 data: 3 2 2 2 2 2 3
4224 082/300 Read block 2412 len 40960
4225 083/300 Write block 1880 len 28672 data: 4 4 4 4 4 3 3
4226 084/300 Read block 3576 len 32768
4227 085/300 Flush
4228 086/300 Write block 2183 len 20480 data: 2 2 2 2 2
4229 087/300 Write block 3519 len 40960 data: 3 2 2 2 2 2 2 2 3 3
4230 088/300 Read block 1930 len 40960
4231 089/300 Write block 505 len 28672 data: 4 4 4 3 2 2 2
4232 090/300 Read block 4003 len 28672
4233 091/300 Write block 4455 len 20480 data: 4 4 4 4 4
4234 092/300 Write block 659 len 40960 data: 4 4 4 4 3 2 2 2 3 3
4235 093/300 Write block 4562 len 20480 data: 2 2 2 3 3
4236 094/300 Write block 1820 len 24576 data: 2 2 2 2 2 2
4237 095/300 Read block 1334 len 24576
4238 096/300 Read block 3339 len 24576
4239 097/300 Read block 3231 len 24576
4240 098/300 Write block 3827 len 20480 data: 2 2 2 2 2
4241 099/300 Read block 3366 len 36864
4242 100/300 Read block 3107 len 28672
4243 101/300 Flush
4244 102/300 Write block 459 len 16384 data: 2 2 2 2
4245 103/300 Read block 3062 len 36864
4246 104/300 Flush
4247 105/300 Read block 920 len 8192
4248 106/300 Write block 319 len 28672 data: 2 2 2 3 3 3 3
4249 107/300 Read block 4796 len 16384
4250 108/300 Write block 387 len 32768 data: 2 2 2 2 2 2 2 2
4251 109/300 Read block 4035 len 24576
4252 110/300 Flush
4253 111/300 Write block 4387 len 20480 data: 2 2 2 2 2
4254 112/300 Write block 3427 len 8192 data: 3 3
4255 113/300 Flush
4256 114/300 Read block 4464 len 20480
4257 115/300 Write block 4849 len 28672 data: 2 2 2 2 2 2 2
4258 116/300 Write block 2459 len 36864 data: 3 3 3 3 4 3 3 2 3
4259 117/300 Read block 2355 len 20480
4260 118/300 Read block 696 len 16384
4261 119/300 Read block 3927 len 4096
4262 120/300 Read block 3780 len 8192
4263 121/300 Write block 3789 len 12288 data: 2 2 3
4264 122/300 Read block 1835 len 36864
4265 123/300 Read block 1432 len 8192
4266 124/300 Read block 2791 len 32768
4267 125/300 Read block 3124 len 8192
4268 126/300 Write block 2685 len 24576 data: 3 3 3 3 3 2
4269 127/300 Flush
4270 128/300 Write block 2533 len 24576 data: 3 3 3 3 3 3
4271 129/300 Read block 2564 len 20480
4272 130/300 Write block 4845 len 32768 data: 2 2 2 2 3 3 3 3
4273 131/300 Write block 3323 len 28672 data: 2 2 2 2 2 3 3
4274 132/300 Write block 1549 len 4096 data: 2
4275 133/300 Write block 3941 len 12288 data: 2 2 2
4276 134/300 Read block 4132 len 16384
4277 135/300 Write block 759 len 20480 data: 3 3 3 3 2
4278 136/300 Read block 2261 len 40960
4279 137/300 Write block 4104 len 4096 data: 3
4280 138/300 Flush
4281 139/300 Read block 1416 len 40960
4282 140/300 Write block 2224 len 12288 data: 2 2 2
4283 141/300 Flush
4284 142/300 Write block 2228 len 40960 data: 3 2 2 2 2 2 2 2 2 2
4285 143/300 Read block 770 len 40960
4286 144/300 Write block 3792 len 16384 data: 3 3 2 2
4287 145/300 Read block 3774 len 28672
4288 146/300 Write block 3896 len 16384 data: 2 2 2 2
4289 147/300 Write block 4523 len 20480 data: 3 4 4 3 2
4290 148/300 Read block 2835 len 32768
4291 149/300 Read block 4634 len 16384
4292 150/300 Read block 2592 len 32768
4293 151/300 Read block 3566 len 40960
4294 152/300 Flush
4295 153/300 Flush
4296 154/300 Write block 881 len 20480 data: 4 3 3 3 3
4297 155/300 Write block 188 len 40960 data: 3 4 5 5 5 5 4 4 5 4
4298 156/300 Write block 1742 len 8192 data: 2 2
4299 157/300 Read block 4273 len 40960
4300 158/300 Write block 3080 len 20480 data: 3 3 3 2 3
4301 159/300 Flush
4302 160/300 Read block 842 len 24576
4303 161/300 Write block 2433 len 28672 data: 3 3 3 3 2 2 2
4304 162/300 Read block 1124 len 28672
4305 163/300 Read block 526 len 16384
4306 164/300 Flush
4307 165/300 Write block 446 len 8192 data: 2 2
4308 166/300 Flush
4309 167/300 Read block 3885 len 32768
4310 168/300 Write block 1779 len 12288 data: 2 2 3
4311 169/300 Write block 2172 len 40960 data: 3 3 3 3 2 2 2 2 2 2
4312 170/300 Read block 3206 len 20480
4313 171/300 Read block 1564 len 16384
4314 172/300 Flush
4315 173/300 Read block 3900 len 12288
4316 174/300 Write block 2358 len 28672 data: 2 2 2 2 2 3 3
4317 175/300 Read block 2008 len 16384
4318 176/300 Write block 321 len 4096 data: 3
4319 177/300 Read block 1570 len 24576
4320 178/300 Write block 4857 len 24576 data: 2 2 2 3 3 3
4321 179/300 Read block 4665 len 4096
4322 180/300 Flush
4323 181/300 Read block 3974 len 16384
4324 182/300 Read block 4922 len 20480
4325 183/300 Read block 3091 len 8192
4326 184/300 Read block 2351 len 28672
4327 185/300 Write block 2388 len 40960 data: 3 3 2 2 2 2 2 3 3 3
4328 186/300 Write block 1491 len 8192 data: 3 3
4329 187/300 Read block 1710 len 16384
4330 188/300 Read block 876 len 36864
4331 189/300 Write block 1459 len 40960 data: 2 2 3 3 3 2 2 2 3 3
4332 190/300 Read block 2445 len 36864
4333 191/300 Write block 4442 len 20480 data: 2 2 2 2 2
4334 192/300 Flush
4335 193/300 Flush
4336 194/300 Read block 2211 len 8192
4337 195/300 Read block 4929 len 28672
4338 196/300 Read block 4861 len 36864
4339 197/300 Write block 3481 len 32768 data: 2 2 2 2 2 2 2 2
4340 198/300 Read block 2827 len 36864
4341 199/300 Read block 3354 len 36864
4342 200/300 Write block 2447 len 4096 data: 2
4343 201/300 Read block 1096 len 4096
4344 202/300 Read block 2913 len 4096
4345 203/300 Write block 659 len 20480 data: 5 5 5 5 4
4346 204/300 Write block 2520 len 36864 data: 3 3 4 4 4 5 5 5 4
4347 205/300 Write block 631 len 8192 data: 4 4
4348 206/300 Read block 3685 len 40960
4349 207/300 Read block 4985 len 4096
4350 208/300 Write block 2929 len 8192 data: 3 3
4351 209/300 Read block 1020 len 36864
4352 210/300 Write block 4257 len 4096 data: 2
4353 211/300 Read block 3694 len 16384
4354 212/300 Write block 3842 len 8192 data: 2 2
4355 213/300 Write block 3285 len 28672 data: 4 4 3 3 2 2 2
4356 214/300 Write block 4515 len 4096 data: 2
4357 215/300 Write block 1540 len 8192 data: 2 2
4358 216/300 Read block 3506 len 8192
4359 217/300 Write block 4474 len 28672 data: 3 2 2 2 2 2 2
4360 218/300 Read block 4506 len 32768
4361 219/300 Read block 1304 len 36864
4362 220/300 Read block 1893 len 8192
4363 221/300 Read block 1036 len 8192
4364 222/300 Read block 4747 len 24576
4365 223/300 Flush
4366 224/300 Write block 4552 len 16384 data: 3 2 2 2
4367 225/300 Write block 2193 len 32768 data: 2 2 2 2 2 2 2 2
4368 226/300 Read block 1243 len 40960
4369 227/300 Write block 163 len 28672 data: 3 3 3 3 3 3 3
4370 228/300 Write block 1537 len 36864 data: 4 4 3 3 3 2 2 2 2
4371 229/300 Read block 2418 len 8192
4372 230/300 Read block 3745 len 32768
4373 231/300 Read block 3653 len 28672
4374 232/300 Read block 460 len 24576
4375 233/300 Read block 2999 len 20480
4376 234/300 Read block 4608 len 36864
4377 235/300 Write block 464 len 12288 data: 2 2 2
4378 236/300 Write block 3403 len 20480 data: 2 2 2 2 2
4379 237/300 Flush
4380 238/300 Read block 4635 len 4096
4381 239/300 Read block 2167 len 32768
4382 240/300 Read block 1781 len 4096
4383 241/300 Write block 3150 len 4096 data: 3
4384 242/300 Read block 3578 len 16384
4385 243/300 Write block 3154 len 40960 data: 3 3 3 3 3 4 2 2 2 3
4386 244/300 Write block 930 len 28672 data: 2 2 2 2 2 2 2
4387 245/300 Read block 4825 len 8192
4388 246/300 Flush
4389 247/300 Write block 4966 len 12288 data: 3 4 3
4390 248/300 Read block 4018 len 8192
4391 249/300 Write block 4920 len 4096 data: 2
4392 250/300 Read block 410 len 16384
4393 251/300 Read block 3848 len 4096
4394 252/300 Read block 4854 len 28672
4395 253/300 Read block 1034 len 20480
4396 254/300 Read block 4529 len 16384
4397 255/300 Read block 4019 len 12288
4398 256/300 Write block 2804 len 8192 data: 2 2
4399 257/300 Read block 2185 len 20480
4400 258/300 Write block 821 len 40960 data: 3 2 2 2 3 5 5 6 6 7
4401 259/300 Read block 1998 len 16384
4402 260/300 Read block 2167 len 8192
4403 261/300 Write block 4227 len 8192 data: 2 2
4404 262/300 Read block 3875 len 24576
4405 263/300 Write block 439 len 24576 data: 3 3 2 2 2 2
4406 264/300 Write block 1708 len 8192 data: 4 5
4407 265/300 Flush
4408 266/300 Write block 2268 len 32768 data: 2 2 2 2 3 2 2 2
4409 267/300 Write block 258 len 4096 data: 3
4410 268/300 Read block 2894 len 40960
4411 269/300 Write block 3851 len 24576 data: 2 2 2 2 2 2
4412 270/300 Write block 1707 len 32768 data: 4 5 6 5 4 5 5 4
4413 271/300 Read block 4068 len 24576
4414 272/300 Read block 3781 len 4096
4415 273/300 Read block 4113 len 24576
4416 274/300 Write block 659 len 40960 data: 6 6 6 6 5 3 3 3 4 4
4417 275/300 Write block 3449 len 20480 data: 2 2 2 2 2
4418 276/300 Flush
4419 277/300 Write block 1560 len 12288 data: 2 2 2
4420 278/300 Write block 4899 len 24576 data: 2 2 3 3 3 2
4421 279/300 Read block 282 len 40960
4422 280/300 Read block 4578 len 8192
4423 281/300 Read block 976 len 20480
4424 282/300 Read block 1015 len 20480
4425 283/300 Read block 3810 len 16384
4426 284/300 Write block 4208 len 24576 data: 2 2 2 2 2 3
4427 285/300 Write block 649 len 40960 data: 2 2 2 2 2 2 2 3 4 4
4428 286/300 Write block 3999 len 8192 data: 2 2
4429 287/300 Write block 3330 len 8192 data: 3 3
4430 288/300 Read block 4790 len 20480
4431 289/300 Write block 3628 len 24576 data: 2 2 2 2 2 2
4432 290/300 Read block 3264 len 20480
4433 291/300 Read block 100 len 32768
4434 292/300 Read block 607 len 4096
4435 293/300 Write block 3316 len 12288 data: 3 3 3
4436 294/300 Write block 4140 len 12288 data: 4 4 3
4437 295/300 Write block 3933 len 20480 data: 3 2 2 2 2
4438 296/300 Read block 1697 len 36864
4439 297/300 Write block 2753 len 28672 data: 3 2 2 2 2 2 2
4440 298/300 Write block 1609 len 36864 data: 2 2 2 2 2 2 2 2 2
4441 299/300 Write block 1341 len 24576 data: 3 3 3 3 2 2
4442 300/300 Write block 306 len 16384 data: 3 3 3 3
4443 301/300 Read block 2505 len 40960
4444 Main task has work to do, go find it
4445 got dsc Start(0)
4446 start 0
4447 Nov 28 04:51:16.218 INFO request completed, latency_us: 206, response_code: 204, uri: /start/cid/0, method: POST, req_id: 8a0b1b7e-cca2-4f47-9dbd-5ca8911fa177, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
4448 Replay: started 0, returned:Ok(())
4449 ----------------------------------------------------------------
4450 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
4451 States: Offline Active Active
4452 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
4453 2208 Acked Write 2 Sent Done Done false
4454 2209 Acked Write 3 Sent Done Done false
4455 2210 Acked Write 1 Sent Done Done false
4456 2211 Acked Read 7 Sent Done Done false
4457 2212 Acked Read 5 Sent Done Done false
4458 2213 Acked Write 5 Sent Done Done false
4459 2214 Acked Write 5 Sent Done Done false
4460 2215 Acked Write 3 Sent Done Done false
4461 2216 Acked Write 1 Sent Done Done false
4462 2217 Acked Read 6 Sent Done Done false
4463 2218 Acked Write 5 Sent Done Done false
4464 2219 Acked Read 6 Sent Done Done false
4465 2220 Acked Read 8 Sent Done Done false
4466 2221 Acked Write 4 Sent Done Done false
4467 2222 Acked Write 8 Sent Done Done false
4468 2223 Acked Read 2 Sent Done Done false
4469 2224 Acked Flush 0 Sent Done Done false
4470 2225 Acked Read 9 Sent Done Done false
4471 2226 Acked Read 10 Sent Done Done false
4472 2227 Acked Read 6 Sent Done Done false
4473 2228 Acked Read 2 Sent Done Done false
4474 2229 Acked Read 7 Sent Done Done false
4475 2230 Acked Write 8 Sent Done Done false
4476 2231 Acked Read 7 Sent Done Done false
4477 2232 Acked Read 6 Sent Done Done false
4478 2233 Acked Read 4 Sent Done Done false
4479 2234 Acked Read 7 Sent Done Done false
4480 2235 Acked Write 4 Sent Done Done false
4481 2236 Acked Read 8 Sent Done Done false
4482 2237 Acked Read 2 Sent Done Done false
4483 2238 Acked Read 3 Sent Done Done false
4484 2239 Acked Read 8 Sent Done Done false
4485 2240 Acked Write 7 Sent Done Done false
4486 2241 Acked Read 8 Sent Done Done false
4487 2242 Acked Write 2 Sent Done Done false
4488 2243 Acked Read 9 Sent Done Done false
4489 2244 Acked Write 5 Sent Done Done false
4490 2245 Acked Write 9 Sent Done Done false
4491 2246 Acked Read 8 Sent Done Done false
4492 2247 Acked Read 8 Sent Done Done false
4493 2248 Acked Write 10 Sent Done Done false
4494 2249 Acked Write 3 Sent Done Done false
4495 2250 Acked Write 1 Sent Done Done false
4496 2251 Acked Write 9 Sent Done Done false
4497 2252 Acked Flush 0 Sent Done Done false
4498 2253 Acked Write 9 Sent Done Done false
4499 2254 Acked Flush 0 Sent Done Done false
4500 2255 Acked Read 1 Sent Done Done false
4501 2256 Acked Write 8 Sent Done Done false
4502 2257 Acked Read 1 Sent Done Done false
4503 2258 Acked Write 6 Sent Done Done false
4504 2259 Acked Read 9 Sent Done Done false
4505 2260 Acked Write 10 Sent Done Done false
4506 2261 Acked Read 1 Sent Done Done false
4507 2262 Acked Read 8 Sent Done Done false
4508 2263 Acked Write 9 Sent Done Done false
4509 2264 Acked Read 7 Sent Done Done false
4510 2265 Acked Write 7 Sent Done Done false
4511 2266 Acked Write 4 Sent Done Done false
4512 2267 Acked Read 1 Sent Done Done false
4513 2268 Acked Read 9 Sent Done Done false
4514 2269 Acked Write 4 Sent Done Done false
4515 2270 Acked Write 2 Sent Done Done false
4516 2271 Acked Write 10 Sent Done Done false
4517 2272 Acked Write 5 Sent Done Done false
4518 2273 Acked Write 2 Sent Done Done false
4519 2274 Acked Write 7 Sent Done Done false
4520 2275 Acked Read 10 Sent Done Done false
4521 2276 Acked Read 9 Sent Done Done false
4522 2277 Acked Read 5 Sent Done Done false
4523 2278 Acked Read 5 Sent Done Done false
4524 2279 Acked Read 3 Sent Done Done false
4525 2280 Acked Read 3 Sent Done Done false
4526 2281 Acked Write 2 Sent Done Done false
4527 2282 Acked Write 9 Sent Done Done false
4528 2283 Acked Read 4 Sent Done Done false
4529 2284 Acked Read 10 Sent Done Done false
4530 2285 Acked Read 2 Sent Done Done false
4531 2286 Acked Read 6 Sent Done Done false
4532 2287 Acked Read 1 Sent Done Done false
4533 2288 Acked Write 7 Sent Done Done false
4534 2289 Acked Read 10 Sent Done Done false
4535 2290 Acked Write 7 Sent Done Done false
4536 2291 Acked Read 8 Sent Done Done false
4537 2292 Acked Flush 0 Sent Done Done false
4538 2293 Acked Write 5 Sent Done Done false
4539 2294 Acked Write 10 Sent Done Done false
4540 2295 Acked Read 10 Sent Done Done false
4541 2296 Acked Write 7 Sent Done Done false
4542 2297 Acked Read 7 Sent Done Done false
4543 2298 Acked Write 5 Sent Done Done false
4544 2299 Acked Write 10 Sent Done Done false
4545 2300 Acked Write 5 Sent Done Done false
4546 2301 Acked Write 6 Sent Done Done false
4547 2302 Acked Read 6 Sent Done Done false
4548 2303 Acked Read 6 Sent Done Done false
4549 2304 Acked Read 6 Sent Done Done false
4550 2305 Acked Write 5 Sent Done Done false
4551 2306 Acked Read 9 Sent Done Done false
4552 2307 Acked Read 7 Sent Done Done false
4553 2308 Acked Flush 0 Sent Done Done false
4554 2309 Acked Write 4 Sent Done Done false
4555 2310 Acked Read 9 Sent Done Done false
4556 2311 Acked Flush 0 Sent Done Done false
4557 2312 Acked Read 2 Sent Done Done false
4558 2313 Acked Write 7 Sent Done Done false
4559 2314 Acked Read 4 Sent Done Done false
4560 2315 Acked Write 8 Sent Done Done false
4561 2316 Acked Read 6 Sent Done Done false
4562 2317 Acked Flush 0 Sent Done Done false
4563 2318 Acked Write 5 Sent Done Done false
4564 2319 Acked Write 2 Sent Done Done false
4565 2320 Acked Flush 0 Sent Done Done false
4566 2321 Acked Read 5 Sent Done Done false
4567 2322 Acked Write 7 Sent Done Done false
4568 2323 Acked Write 9 Sent Done Done false
4569 2324 Acked Read 5 Sent Done Done false
4570 2325 Acked Read 4 Sent Done Done false
4571 2326 Acked Read 1 Sent Done Done false
4572 2327 Acked Read 2 Sent Done Done false
4573 2328 Acked Write 3 Sent Done Done false
4574 2329 Acked Read 9 Sent Done Done false
4575 2330 Acked Read 2 Sent Done Done false
4576 2331 Acked Read 8 Sent Done Done false
4577 2332 Acked Read 2 Sent Done Done false
4578 2333 Acked Write 6 Sent Done Done false
4579 2334 Acked Flush 0 Sent Done Done false
4580 2335 Acked Write 6 Sent Done Done false
4581 2336 Acked Read 5 Sent Done Done false
4582 2337 Acked Write 8 Sent Done Done false
4583 2338 Acked Write 7 Sent Done Done false
4584 2339 Acked Write 1 Sent Done Done false
4585 2340 Acked Write 3 Sent Done Done false
4586 2341 Acked Read 4 Sent Done Done false
4587 2342 Acked Write 5 Sent Done Done false
4588 2343 Acked Read 10 Sent Done Done false
4589 2344 Acked Write 1 Sent Done Done false
4590 2345 Acked Flush 0 Sent Done Done false
4591 2346 Acked Read 10 Sent Done Done false
4592 2347 Acked Write 3 Sent Done Done false
4593 2348 Acked Flush 0 Sent Done Done false
4594 2349 Acked Write 10 Sent Done Done false
4595 2350 Acked Read 10 Sent Done Done false
4596 2351 Acked Write 4 Sent Done Done false
4597 2352 Acked Read 7 Sent Done Done false
4598 2353 Acked Write 4 Sent Done Done false
4599 2354 Acked Write 5 Sent Done Done false
4600 2355 Acked Read 8 Sent Done Done false
4601 2356 Acked Read 4 Sent Done Done false
4602 2357 Acked Read 8 Sent Done Done false
4603 2358 Acked Read 10 Sent Done Done false
4604 2359 Acked Flush 0 Sent Done Done false
4605 2360 Acked Flush 0 Sent Done Done false
4606 2361 Acked Write 5 Sent Done Done false
4607 2362 Acked Write 10 Sent Done Done false
4608 2363 Acked Write 2 Sent Done Done false
4609 2364 Acked Read 10 Sent Done Done false
4610 2365 Acked Write 5 Sent Done Done false
4611 2366 Acked Flush 0 Sent Done Done false
4612 2367 Acked Read 6 Sent Done Done false
4613 2368 Acked Write 7 Sent Done Done false
4614 2369 Acked Read 7 Sent Done Done false
4615 2370 Acked Read 4 Sent Done Done false
4616 2371 Acked Flush 0 Sent Done Done false
4617 2372 Acked Write 2 Sent Done Done false
4618 2373 Acked Flush 0 Sent Done Done false
4619 2374 Acked Read 8 Sent Done Done false
4620 2375 Acked Write 3 Sent Done Done false
4621 2376 Acked Write 10 Sent Done Done false
4622 2377 Acked Read 5 Sent Done Done false
4623 2378 Acked Read 4 Sent Done Done false
4624 2379 Acked Flush 0 Sent Done Done false
4625 2380 Acked Read 3 Sent Done Done false
4626 2381 Acked Write 7 Sent Done Done false
4627 2382 Acked Read 4 Sent Done Done false
4628 2383 Acked Write 1 Sent Done Done false
4629 2384 Acked Read 6 Sent Done Done false
4630 2385 Acked Write 6 Sent Done Done false
4631 2386 Acked Read 1 Sent Done Done false
4632 2387 Acked Flush 0 Sent Done Done false
4633 2388 Acked Read 4 Sent Done Done false
4634 2389 Acked Read 5 Sent Done Done false
4635 2390 Acked Read 2 Sent Done Done false
4636 2391 Acked Read 7 Sent Done Done false
4637 2392 Acked Write 10 Sent Done Done false
4638 2393 Acked Write 2 Sent Done Done false
4639 2394 Acked Read 4 Sent Done Done false
4640 2395 Acked Read 9 Sent Done Done false
4641 2396 Acked Write 10 Sent Done Done false
4642 2397 Acked Read 9 Sent Done Done false
4643 2398 Acked Write 5 Sent Done Done false
4644 2399 Acked Flush 0 Sent Done Done false
4645 2400 Acked Flush 0 Sent Done Done false
4646 2401 Acked Read 2 Sent Done Done false
4647 2402 Acked Read 7 Sent Done Done false
4648 2403 Acked Read 9 Sent Done Done false
4649 2404 Acked Write 8 Sent Done Done false
4650 2405 Acked Read 9 Sent Done Done false
4651 2406 Acked Read 9 Sent Done Done false
4652 2407 Acked Write 1 Sent Done Done false
4653 2408 Acked Read 1 Sent Done Done false
4654 2409 Acked Read 1 Sent Done Done false
4655 2410 Acked Write 5 Sent Done Done false
4656 2411 Acked Write 9 Sent Done Done false
4657 2412 Acked Write 2 Sent Done Done false
4658 2413 Acked Read 10 Sent Done Done false
4659 2414 Acked Read 1 Sent Done Done false
4660 2415 Acked Write 2 Sent Done Done false
4661 2416 Acked Read 9 Sent Done Done false
4662 2417 Acked Write 1 Sent Done Done false
4663 2418 Acked Read 4 Sent Done Done false
4664 2419 Acked Write 2 Sent Done Done false
4665 2420 Acked Write 7 Sent Done Done false
4666 2421 Acked Write 1 Sent Done Done false
4667 2422 Acked Write 2 Sent Done Done false
4668 2423 Acked Read 2 Sent Done Done false
4669 2424 Acked Write 7 Sent Done Done false
4670 2425 Acked Read 8 Sent Done Done false
4671 2426 Acked Read 9 Sent Done Done false
4672 2427 Acked Read 2 Sent Done Done false
4673 2428 Acked Read 2 Sent Done Done false
4674 2429 Acked Read 6 Sent Done Done false
4675 2430 Acked Flush 0 Sent Done Done false
4676 2431 Acked Write 4 Sent Done Done false
4677 2432 Acked Write 8 Sent Done Done false
4678 2433 Acked Read 10 Sent Done Done false
4679 2434 Acked Write 7 Sent Done Done false
4680 2435 Acked Write 9 Sent Done Done false
4681 2436 Acked Read 2 Sent Done Done false
4682 2437 Acked Read 8 Sent Done Done false
4683 2438 Acked Read 7 Sent Done Done false
4684 2439 Acked Read 6 Sent Done Done false
4685 2440 Acked Read 5 Sent Done Done false
4686 2441 Acked Read 9 Sent Done Done false
4687 2442 Acked Write 3 Sent Done Done false
4688 2443 Acked Write 5 Sent Done Done false
4689 2444 Acked Flush 0 Sent Done Done false
4690 2445 Acked Read 1 Sent Done Done false
4691 2446 Acked Read 8 Sent Done Done false
4692 2447 Acked Read 1 Sent Done Done false
4693 2448 Acked Write 1 Sent Done Done false
4694 2449 Acked Read 4 Sent Done Done false
4695 2450 Acked Write 10 Sent Done Done false
4696 2451 Acked Write 7 Sent Done Done false
4697 2452 Acked Read 2 Sent Done Done false
4698 2453 Acked Flush 0 Sent Done Done false
4699 2454 Acked Write 3 Sent Done Done false
4700 2455 Acked Read 2 Sent Done Done false
4701 2456 Acked Write 1 Sent Done Done false
4702 2457 Acked Read 4 Sent Done Done false
4703 2458 Acked Read 1 Sent Done Done false
4704 2459 Acked Read 7 Sent Done Done false
4705 2460 Acked Read 5 Sent Done Done false
4706 2461 Acked Read 4 Sent Done Done false
4707 2462 Acked Read 3 Sent Done Done false
4708 2463 Acked Write 2 Sent Done Done false
4709 2464 Acked Read 5 Sent Done Done false
4710 2465 Acked Write 10 Sent Done Done false
4711 2466 Acked Read 4 Sent Done Done false
4712 2467 Acked Read 2 Sent Done Done false
4713 2468 Acked Write 2 Sent Done Done false
4714 2469 Acked Read 6 Sent Done Done false
4715 2470 Acked Write 6 Sent Done Done false
4716 2471 Acked Write 2 Sent Done Done false
4717 2472 Acked Flush 0 Sent Done Done false
4718 2473 Acked Write 8 Sent Done Done false
4719 2474 Acked Write 1 Sent Done Done false
4720 2475 Acked Read 10 Sent Done Done false
4721 2476 Acked Write 6 Sent Done Done false
4722 2477 Acked Write 8 Sent Done Done false
4723 2478 Acked Read 6 Sent Done Done false
4724 2479 Acked Read 1 Sent Done Done false
4725 2480 Acked Read 6 Sent Done Done false
4726 2481 Acked Write 10 Sent Done Done false
4727 2482 Acked Write 5 Sent Done Done false
4728 2483 Acked Flush 0 Sent Done Done false
4729 2484 Acked Write 3 Sent Done Done false
4730 2485 Acked Write 6 Sent Done Done false
4731 2486 Acked Read 10 Sent Done Done false
4732 2487 Acked Read 2 Sent Done Done false
4733 2488 Acked Read 5 Sent Done Done false
4734 2489 Acked Read 5 Sent Done Done false
4735 2490 Acked Read 4 Sent Done Done false
4736 2491 Acked Write 6 Sent Done Done false
4737 2492 Acked Write 10 Sent Done Done false
4738 2493 Acked Write 2 Sent Done Done false
4739 2494 Acked Write 2 Sent Done Done false
4740 2495 Acked Read 5 Sent Done Done false
4741 2496 Acked Write 6 Sent Done Done false
4742 2497 Acked Read 5 Sent Done Done false
4743 2498 Acked Read 8 Sent Done Done false
4744 2499 Acked Read 1 Sent Done Done false
4745 2500 Acked Write 3 Sent Done Done false
4746 2501 Acked Write 3 Sent Done Done false
4747 2502 Acked Write 5 Sent Done Done false
4748 2503 Acked Read 9 Sent Done Done false
4749 2504 Acked Write 7 Sent Done Done false
4750 2505 Acked Write 9 Sent Done Done false
4751 2506 Acked Write 6 Sent Done Done false
4752 2507 Acked Write 4 Sent Done Done false
4753 2508 Acked Read 10 Sent Done Done false
4754 STATES DS:0 DS:1 DS:2 TOTAL
4755 Sent 301 0 0 301
4756 Done 0 301 301 602
4757 Skipped 0 0 0 0
4758 Error 0 0 0 0
4759 Last Flush: 2207 2483 2483
4760 Downstairs last five completed: 2207 2206 2205 2204 2203
4761 Upstairs last five completed: 2508 2507 2506 2505 2504
4762 CLIENT: Up:0 ds:301 act:2
4763 [8810] got start action
4764 [8810] I am going to restart
4765 Starting downstairs at port 8810
4766 Make output file at "/tmp/dsc/downstairs-8810.txt"
4767 [8810][0] reports Starting
4768 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1207)
4769 [8810][0] reports Running
4770 ----------------------------------------------------------------
4771 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
4772 States: Offline Active Active
4773 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
4774 2208 Acked Write 2 Sent Done Done false
4775 2209 Acked Write 3 Sent Done Done false
4776 2210 Acked Write 1 Sent Done Done false
4777 2211 Acked Read 7 Sent Done Done false
4778 2212 Acked Read 5 Sent Done Done false
4779 2213 Acked Write 5 Sent Done Done false
4780 2214 Acked Write 5 Sent Done Done false
4781 2215 Acked Write 3 Sent Done Done false
4782 2216 Acked Write 1 Sent Done Done false
4783 2217 Acked Read 6 Sent Done Done false
4784 2218 Acked Write 5 Sent Done Done false
4785 2219 Acked Read 6 Sent Done Done false
4786 2220 Acked Read 8 Sent Done Done false
4787 2221 Acked Write 4 Sent Done Done false
4788 2222 Acked Write 8 Sent Done Done false
4789 2223 Acked Read 2 Sent Done Done false
4790 2224 Acked Flush 0 Sent Done Done false
4791 2225 Acked Read 9 Sent Done Done false
4792 2226 Acked Read 10 Sent Done Done false
4793 2227 Acked Read 6 Sent Done Done false
4794 2228 Acked Read 2 Sent Done Done false
4795 2229 Acked Read 7 Sent Done Done false
4796 2230 Acked Write 8 Sent Done Done false
4797 2231 Acked Read 7 Sent Done Done false
4798 2232 Acked Read 6 Sent Done Done false
4799 2233 Acked Read 4 Sent Done Done false
4800 2234 Acked Read 7 Sent Done Done false
4801 2235 Acked Write 4 Sent Done Done false
4802 2236 Acked Read 8 Sent Done Done false
4803 2237 Acked Read 2 Sent Done Done false
4804 2238 Acked Read 3 Sent Done Done false
4805 2239 Acked Read 8 Sent Done Done false
4806 2240 Acked Write 7 Sent Done Done false
4807 2241 Acked Read 8 Sent Done Done false
4808 2242 Acked Write 2 Sent Done Done false
4809 2243 Acked Read 9 Sent Done Done false
4810 2244 Acked Write 5 Sent Done Done false
4811 2245 Acked Write 9 Sent Done Done false
4812 2246 Acked Read 8 Sent Done Done false
4813 2247 Acked Read 8 Sent Done Done false
4814 2248 Acked Write 10 Sent Done Done false
4815 2249 Acked Write 3 Sent Done Done false
4816 2250 Acked Write 1 Sent Done Done false
4817 2251 Acked Write 9 Sent Done Done false
4818 2252 Acked Flush 0 Sent Done Done false
4819 2253 Acked Write 9 Sent Done Done false
4820 2254 Acked Flush 0 Sent Done Done false
4821 2255 Acked Read 1 Sent Done Done false
4822 2256 Acked Write 8 Sent Done Done false
4823 2257 Acked Read 1 Sent Done Done false
4824 2258 Acked Write 6 Sent Done Done false
4825 2259 Acked Read 9 Sent Done Done false
4826 2260 Acked Write 10 Sent Done Done false
4827 2261 Acked Read 1 Sent Done Done false
4828 2262 Acked Read 8 Sent Done Done false
4829 2263 Acked Write 9 Sent Done Done false
4830 2264 Acked Read 7 Sent Done Done false
4831 2265 Acked Write 7 Sent Done Done false
4832 2266 Acked Write 4 Sent Done Done false
4833 2267 Acked Read 1 Sent Done Done false
4834 2268 Acked Read 9 Sent Done Done false
4835 2269 Acked Write 4 Sent Done Done false
4836 2270 Acked Write 2 Sent Done Done false
4837 2271 Acked Write 10 Sent Done Done false
4838 2272 Acked Write 5 Sent Done Done false
4839 2273 Acked Write 2 Sent Done Done false
4840 2274 Acked Write 7 Sent Done Done false
4841 2275 Acked Read 10 Sent Done Done false
4842 2276 Acked Read 9 Sent Done Done false
4843 2277 Acked Read 5 Sent Done Done false
4844 2278 Acked Read 5 Sent Done Done false
4845 2279 Acked Read 3 Sent Done Done false
4846 2280 Acked Read 3 Sent Done Done false
4847 2281 Acked Write 2 Sent Done Done false
4848 2282 Acked Write 9 Sent Done Done false
4849 2283 Acked Read 4 Sent Done Done false
4850 2284 Acked Read 10 Sent Done Done false
4851 2285 Acked Read 2 Sent Done Done false
4852 2286 Acked Read 6 Sent Done Done false
4853 2287 Acked Read 1 Sent Done Done false
4854 2288 Acked Write 7 Sent Done Done false
4855 2289 Acked Read 10 Sent Done Done false
4856 2290 Acked Write 7 Sent Done Done false
4857 2291 Acked Read 8 Sent Done Done false
4858 2292 Acked Flush 0 Sent Done Done false
4859 2293 Acked Write 5 Sent Done Done false
4860 2294 Acked Write 10 Sent Done Done false
4861 2295 Acked Read 10 Sent Done Done false
4862 2296 Acked Write 7 Sent Done Done false
4863 2297 Acked Read 7 Sent Done Done false
4864 2298 Acked Write 5 Sent Done Done false
4865 2299 Acked Write 10 Sent Done Done false
4866 2300 Acked Write 5 Sent Done Done false
4867 2301 Acked Write 6 Sent Done Done false
4868 2302 Acked Read 6 Sent Done Done false
4869 2303 Acked Read 6 Sent Done Done false
4870 2304 Acked Read 6 Sent Done Done false
4871 2305 Acked Write 5 Sent Done Done false
4872 2306 Acked Read 9 Sent Done Done false
4873 2307 Acked Read 7 Sent Done Done false
4874 2308 Acked Flush 0 Sent Done Done false
4875 2309 Acked Write 4 Sent Done Done false
4876 2310 Acked Read 9 Sent Done Done false
4877 2311 Acked Flush 0 Sent Done Done false
4878 2312 Acked Read 2 Sent Done Done false
4879 2313 Acked Write 7 Sent Done Done false
4880 2314 Acked Read 4 Sent Done Done false
4881 2315 Acked Write 8 Sent Done Done false
4882 2316 Acked Read 6 Sent Done Done false
4883 2317 Acked Flush 0 Sent Done Done false
4884 2318 Acked Write 5 Sent Done Done false
4885 2319 Acked Write 2 Sent Done Done false
4886 2320 Acked Flush 0 Sent Done Done false
4887 2321 Acked Read 5 Sent Done Done false
4888 2322 Acked Write 7 Sent Done Done false
4889 2323 Acked Write 9 Sent Done Done false
4890 2324 Acked Read 5 Sent Done Done false
4891 2325 Acked Read 4 Sent Done Done false
4892 2326 Acked Read 1 Sent Done Done false
4893 2327 Acked Read 2 Sent Done Done false
4894 2328 Acked Write 3 Sent Done Done false
4895 2329 Acked Read 9 Sent Done Done false
4896 2330 Acked Read 2 Sent Done Done false
4897 2331 Acked Read 8 Sent Done Done false
4898 2332 Acked Read 2 Sent Done Done false
4899 2333 Acked Write 6 Sent Done Done false
4900 2334 Acked Flush 0 Sent Done Done false
4901 2335 Acked Write 6 Sent Done Done false
4902 2336 Acked Read 5 Sent Done Done false
4903 2337 Acked Write 8 Sent Done Done false
4904 2338 Acked Write 7 Sent Done Done false
4905 2339 Acked Write 1 Sent Done Done false
4906 2340 Acked Write 3 Sent Done Done false
4907 2341 Acked Read 4 Sent Done Done false
4908 2342 Acked Write 5 Sent Done Done false
4909 2343 Acked Read 10 Sent Done Done false
4910 2344 Acked Write 1 Sent Done Done false
4911 2345 Acked Flush 0 Sent Done Done false
4912 2346 Acked Read 10 Sent Done Done false
4913 2347 Acked Write 3 Sent Done Done false
4914 2348 Acked Flush 0 Sent Done Done false
4915 2349 Acked Write 10 Sent Done Done false
4916 2350 Acked Read 10 Sent Done Done false
4917 2351 Acked Write 4 Sent Done Done false
4918 2352 Acked Read 7 Sent Done Done false
4919 2353 Acked Write 4 Sent Done Done false
4920 2354 Acked Write 5 Sent Done Done false
4921 2355 Acked Read 8 Sent Done Done false
4922 2356 Acked Read 4 Sent Done Done false
4923 2357 Acked Read 8 Sent Done Done false
4924 2358 Acked Read 10 Sent Done Done false
4925 2359 Acked Flush 0 Sent Done Done false
4926 2360 Acked Flush 0 Sent Done Done false
4927 2361 Acked Write 5 Sent Done Done false
4928 2362 Acked Write 10 Sent Done Done false
4929 2363 Acked Write 2 Sent Done Done false
4930 2364 Acked Read 10 Sent Done Done false
4931 2365 Acked Write 5 Sent Done Done false
4932 2366 Acked Flush 0 Sent Done Done false
4933 2367 Acked Read 6 Sent Done Done false
4934 2368 Acked Write 7 Sent Done Done false
4935 2369 Acked Read 7 Sent Done Done false
4936 2370 Acked Read 4 Sent Done Done false
4937 2371 Acked Flush 0 Sent Done Done false
4938 2372 Acked Write 2 Sent Done Done false
4939 2373 Acked Flush 0 Sent Done Done false
4940 2374 Acked Read 8 Sent Done Done false
4941 2375 Acked Write 3 Sent Done Done false
4942 2376 Acked Write 10 Sent Done Done false
4943 2377 Acked Read 5 Sent Done Done false
4944 2378 Acked Read 4 Sent Done Done false
4945 2379 Acked Flush 0 Sent Done Done false
4946 2380 Acked Read 3 Sent Done Done false
4947 2381 Acked Write 7 Sent Done Done false
4948 2382 Acked Read 4 Sent Done Done false
4949 2383 Acked Write 1 Sent Done Done false
4950 2384 Acked Read 6 Sent Done Done false
4951 2385 Acked Write 6 Sent Done Done false
4952 2386 Acked Read 1 Sent Done Done false
4953 2387 Acked Flush 0 Sent Done Done false
4954 2388 Acked Read 4 Sent Done Done false
4955 2389 Acked Read 5 Sent Done Done false
4956 2390 Acked Read 2 Sent Done Done false
4957 2391 Acked Read 7 Sent Done Done false
4958 2392 Acked Write 10 Sent Done Done false
4959 2393 Acked Write 2 Sent Done Done false
4960 2394 Acked Read 4 Sent Done Done false
4961 2395 Acked Read 9 Sent Done Done false
4962 2396 Acked Write 10 Sent Done Done false
4963 2397 Acked Read 9 Sent Done Done false
4964 2398 Acked Write 5 Sent Done Done false
4965 2399 Acked Flush 0 Sent Done Done false
4966 2400 Acked Flush 0 Sent Done Done false
4967 2401 Acked Read 2 Sent Done Done false
4968 2402 Acked Read 7 Sent Done Done false
4969 2403 Acked Read 9 Sent Done Done false
4970 2404 Acked Write 8 Sent Done Done false
4971 2405 Acked Read 9 Sent Done Done false
4972 2406 Acked Read 9 Sent Done Done false
4973 2407 Acked Write 1 Sent Done Done false
4974 2408 Acked Read 1 Sent Done Done false
4975 2409 Acked Read 1 Sent Done Done false
4976 2410 Acked Write 5 Sent Done Done false
4977 2411 Acked Write 9 Sent Done Done false
4978 2412 Acked Write 2 Sent Done Done false
4979 2413 Acked Read 10 Sent Done Done false
4980 2414 Acked Read 1 Sent Done Done false
4981 2415 Acked Write 2 Sent Done Done false
4982 2416 Acked Read 9 Sent Done Done false
4983 2417 Acked Write 1 Sent Done Done false
4984 2418 Acked Read 4 Sent Done Done false
4985 2419 Acked Write 2 Sent Done Done false
4986 2420 Acked Write 7 Sent Done Done false
4987 2421 Acked Write 1 Sent Done Done false
4988 2422 Acked Write 2 Sent Done Done false
4989 2423 Acked Read 2 Sent Done Done false
4990 2424 Acked Write 7 Sent Done Done false
4991 2425 Acked Read 8 Sent Done Done false
4992 2426 Acked Read 9 Sent Done Done false
4993 2427 Acked Read 2 Sent Done Done false
4994 2428 Acked Read 2 Sent Done Done false
4995 2429 Acked Read 6 Sent Done Done false
4996 2430 Acked Flush 0 Sent Done Done false
4997 2431 Acked Write 4 Sent Done Done false
4998 2432 Acked Write 8 Sent Done Done false
4999 2433 Acked Read 10 Sent Done Done false
5000 2434 Acked Write 7 Sent Done Done false
5001 2435 Acked Write 9 Sent Done Done false
5002 2436 Acked Read 2 Sent Done Done false
5003 2437 Acked Read 8 Sent Done Done false
5004 2438 Acked Read 7 Sent Done Done false
5005 2439 Acked Read 6 Sent Done Done false
5006 2440 Acked Read 5 Sent Done Done false
5007 2441 Acked Read 9 Sent Done Done false
5008 2442 Acked Write 3 Sent Done Done false
5009 2443 Acked Write 5 Sent Done Done false
5010 2444 Acked Flush 0 Sent Done Done false
5011 2445 Acked Read 1 Sent Done Done false
5012 2446 Acked Read 8 Sent Done Done false
5013 2447 Acked Read 1 Sent Done Done false
5014 2448 Acked Write 1 Sent Done Done false
5015 2449 Acked Read 4 Sent Done Done false
5016 2450 Acked Write 10 Sent Done Done false
5017 2451 Acked Write 7 Sent Done Done false
5018 2452 Acked Read 2 Sent Done Done false
5019 2453 Acked Flush 0 Sent Done Done false
5020 2454 Acked Write 3 Sent Done Done false
5021 2455 Acked Read 2 Sent Done Done false
5022 2456 Acked Write 1 Sent Done Done false
5023 2457 Acked Read 4 Sent Done Done false
5024 2458 Acked Read 1 Sent Done Done false
5025 2459 Acked Read 7 Sent Done Done false
5026 2460 Acked Read 5 Sent Done Done false
5027 2461 Acked Read 4 Sent Done Done false
5028 2462 Acked Read 3 Sent Done Done false
5029 2463 Acked Write 2 Sent Done Done false
5030 2464 Acked Read 5 Sent Done Done false
5031 2465 Acked Write 10 Sent Done Done false
5032 2466 Acked Read 4 Sent Done Done false
5033 2467 Acked Read 2 Sent Done Done false
5034 2468 Acked Write 2 Sent Done Done false
5035 2469 Acked Read 6 Sent Done Done false
5036 2470 Acked Write 6 Sent Done Done false
5037 2471 Acked Write 2 Sent Done Done false
5038 2472 Acked Flush 0 Sent Done Done false
5039 2473 Acked Write 8 Sent Done Done false
5040 2474 Acked Write 1 Sent Done Done false
5041 2475 Acked Read 10 Sent Done Done false
5042 2476 Acked Write 6 Sent Done Done false
5043 2477 Acked Write 8 Sent Done Done false
5044 2478 Acked Read 6 Sent Done Done false
5045 2479 Acked Read 1 Sent Done Done false
5046 2480 Acked Read 6 Sent Done Done false
5047 2481 Acked Write 10 Sent Done Done false
5048 2482 Acked Write 5 Sent Done Done false
5049 2483 Acked Flush 0 Sent Done Done false
5050 2484 Acked Write 3 Sent Done Done false
5051 2485 Acked Write 6 Sent Done Done false
5052 2486 Acked Read 10 Sent Done Done false
5053 2487 Acked Read 2 Sent Done Done false
5054 2488 Acked Read 5 Sent Done Done false
5055 2489 Acked Read 5 Sent Done Done false
5056 2490 Acked Read 4 Sent Done Done false
5057 2491 Acked Write 6 Sent Done Done false
5058 2492 Acked Write 10 Sent Done Done false
5059 2493 Acked Write 2 Sent Done Done false
5060 2494 Acked Write 2 Sent Done Done false
5061 2495 Acked Read 5 Sent Done Done false
5062 2496 Acked Write 6 Sent Done Done false
5063 2497 Acked Read 5 Sent Done Done false
5064 2498 Acked Read 8 Sent Done Done false
5065 2499 Acked Read 1 Sent Done Done false
5066 2500 Acked Write 3 Sent Done Done false
5067 2501 Acked Write 3 Sent Done Done false
5068 2502 Acked Write 5 Sent Done Done false
5069 2503 Acked Read 9 Sent Done Done false
5070 2504 Acked Write 7 Sent Done Done false
5071 2505 Acked Write 9 Sent Done Done false
5072 2506 Acked Write 6 Sent Done Done false
5073 2507 Acked Write 4 Sent Done Done false
5074 2508 Acked Read 10 Sent Done Done false
5075 2509 Acked Flush 0 Sent Done Done false
5076 STATES DS:0 DS:1 DS:2 TOTAL
5077 Sent 302 0 0 302
5078 Done 0 302 302 604
5079 Skipped 0 0 0 0
5080 Error 0 0 0 0
5081 Last Flush: 2207 2509 2509
5082 Downstairs last five completed: 2207 2206 2205 2204 2203
5083 Upstairs last five completed: 2509 2508 2507 2506 2505
5084 CLIENT: Up:0 ds:302 act:2
50852024-11-28T04:51:21.985ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
50862024-11-28T04:51:21.985ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:49422) = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
50872024-11-28T04:51:21.986ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
50882024-11-28T04:51:21.987ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
50892024-11-28T04:51:21.987ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
50902024-11-28T04:51:21.987ZINFOcrucible: send last flush ID to this DS: 2207 = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
50912024-11-28T04:51:21.987ZINFOcrucible: Replied this last flush ID: 2207 = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
50922024-11-28T04:51:21.987ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
50932024-11-28T04:51:21.987ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
50942024-11-28T04:51:21.987ZINFOcrucible: [0] client re-new 302 jobs since flush 2207 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
50952024-11-28T04:51:21.987ZINFOcrucible: [0] Replayed 302 jobs since flush: 2207 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
5096 ----------------------------------------------------------------
5097 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
5098 Downstairs last five completed: 2509 2508 2507 2506 2505
5099 Upstairs last five completed: 2509 2508 2507 2506 2505
5100 CLIENT: Up:0 ds:0 act:3
5101 Replay: All jobs finished
5102 Main task has work to do, go find it
5103 got dsc Stop(0)
5104 stop 0
5105 Nov 28 04:51:24.222 INFO request completed, latency_us: 314, response_code: 204, uri: /stop/cid/0[8810] Got stop action so:false kr:false
5106 , method: POST, req_id: da3a10d8-7c2d-4f18-ac12-2767be05c91f, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
51072024-11-28T04:51:24.223ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
51082024-11-28T04:51:24.223ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
51092024-11-28T04:51:24.223ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
51102024-11-28T04:51:24.223ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
51112024-11-28T04:51:24.223ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
5112 Nov 28 04:51:24.223 INFO request completed, latency_us: 201, response_code: 200, uri: /state/cid/0, method: GET, req_id: 0ed35f55-b48a-4ad4-8079-df8338ad4b23, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
51132024-11-28T04:51:24.223ZINFOcrucible: client task is exiting = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
5114 [8810] Exited with: ExitStatus(unix_wait_status(9))
5115 [8810][0] reports Exit
5116 Nov 28 04:51:28.225 INFO request completed, latency_us: 211, response_code: 200, uri: /state/cid/0, method: GET, req_id: 23b660d5-0dc3-46b8-b41f-79aebef3ed19, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
5117 001/300 Read block 226 len 8192
5118 002/300 Read block 4610 len 16384
5119 003/300 Read block 3228 len 32768
5120 004/300 Read block 422 len 28672
5121 005/300 Write block 1851 len 4096 data: 3
5122 006/300 Write block 4582 len 20480 data: 3 3 3 2 2
5123 007/300 Write block 1737 len 16384 data: 3 2 2 2
5124 008/300 Write block 2394 len 16384 data: 3 4 4 4
5125 009/300 Write block 4078 len 20480 data: 3 4 3 3 3
5126 010/300 Write block 4125 len 40960 data: 3 3 3 3 3 3 2 2 2 2
5127 011/300 Write block 2236 len 4096 data: 3
5128 012/300 Read block 3301 len 24576
5129 013/300 Read block 2902 len 36864
5130 014/300 Read block 1620 len 12288
5131 015/300 Write block 537 len 4096 data: 5
5132 016/300 Write block 310 len 20480 data: 3 2 2 2 2
5133 017/300 Read block 2774 len 40960
5134 018/300 Write block 4678 len 40960 data: 3 3 3 3 3 2 2 2 2 2
5135 019/300 Flush
5136 020/300 Read block 437 len 28672
5137 021/300 Read block 330 len 4096
5138 022/300 Write block 4181 len 12288 data: 2 2 2
5139 023/300 Flush
5140 024/300 Read block 1468 len 12288
5141 025/300 Flush
5142 026/300 Read block 227 len 8192
5143 027/300 Read block 1130 len 28672
5144 028/300 Read block 199 len 16384
5145 029/300 Read block 194 len 20480
5146 030/300 Write block 4844 len 32768 data: 2 3 3 3 3 4 4 4
5147 031/300 Read block 3051 len 12288
5148 032/300 Read block 3992 len 20480
5149 033/300 Read block 538 len 16384
5150 034/300 Read block 3169 len 24576
5151 035/300 Write block 4997 len 8192 data: 2 2
5152 036/300 Write block 545 len 20480 data: 2 2 2 2 2
5153 037/300 Read block 247 len 32768
5154 038/300 Read block 2874 len 40960
5155 039/300 Write block 3351 len 16384 data: 2 2 2 2
5156 040/300 Read block 4654 len 16384
5157 041/300 Read block 1410 len 8192
5158 042/300 Write block 2873 len 40960 data: 2 2 2 2 2 2 2 2 2 2
5159 043/300 Read block 77 len 4096
5160 044/300 Flush
5161 045/300 Read block 490 len 24576
5162 046/300 Write block 3556 len 28672 data: 3 3 2 2 2 2 2
5163 047/300 Read block 2182 len 40960
5164 048/300 Write block 4737 len 20480 data: 2 2 2 2 2
5165 049/300 Write block 4327 len 12288 data: 2 2 2
5166 050/300 Write block 1554 len 28672 data: 2 2 2 2 2 2 3
5167 051/300 Read block 2883 len 40960
5168 052/300 Read block 3229 len 4096
5169 053/300 Write block 684 len 36864 data: 2 3 3 2 2 2 2 2 2
5170 054/300 Write block 2571 len 20480 data: 4 4 3 3 3
5171 055/300 Write block 4361 len 8192 data: 3 2
5172 056/300 Read block 582 len 12288
5173 057/300 Read block 3584 len 8192
5174 058/300 Read block 578 len 8192
5175 059/300 Read block 3807 len 32768
5176 060/300 Read block 1799 len 36864
5177 061/300 Read block 840 len 36864
5178 062/300 Read block 865 len 16384
5179 063/300 Flush
5180 064/300 Write block 245 len 32768 data: 2 3 3 3 3 3 3 3
5181 065/300 Read block 164 len 20480
5182 066/300 Write block 1705 len 32768 data: 4 4 5 6 7 6 5 6
5183 067/300 Write block 958 len 16384 data: 2 2 2 2
5184 068/300 Write block 4444 len 28672 data: 3 3 3 2 2 2 2
5185 069/300 Write block 3534 len 12288 data: 4 4 4
5186 070/300 Write block 3965 len 12288 data: 2 2 2
5187 071/300 Read block 1050 len 28672
5188 072/300 Write block 3549 len 36864 data: 3 3 3 3 3 3 3 4 4
5189 073/300 Read block 2248 len 12288
5190 074/300 Flush
5191 075/300 Read block 2823 len 24576
5192 076/300 Write block 3450 len 20480 data: 3 3 3 3 2
5193 077/300 Read block 3985 len 32768
5194 078/300 Write block 3225 len 8192 data: 2 2
5195 079/300 Read block 4529 len 8192
5196 080/300 Flush
5197 081/300 Write block 527 len 20480 data: 3 2 2 3 4
5198 082/300 Write block 1150 len 16384 data: 3 3 2 2
5199 083/300 Write block 1089 len 40960 data: 3 3 2 2 2 2 2 3 3 3
5200 084/300 Read block 2963 len 4096
5201 085/300 Write block 3776 len 4096 data: 2
5202 086/300 Read block 3109 len 12288
5203 087/300 Write block 1079 len 24576 data: 4 3 3 2 3 3
5204 088/300 Write block 597 len 28672 data: 3 3 3 2 2 2 2
5205 089/300 Read block 1692 len 40960
5206 090/300 Read block 4465 len 12288
5207 091/300 Read block 3411 len 16384
5208 092/300 Read block 3623 len 28672
5209 093/300 Read block 703 len 36864
5210 094/300 Read block 3266 len 28672
5211 095/300 Read block 1684 len 28672
5212 096/300 Read block 1532 len 8192
5213 097/300 Read block 2317 len 8192
5214 098/300 Write block 3024 len 16384 data: 2 2 2 2
5215 099/300 Read block 1665 len 36864
5216 100/300 Write block 473 len 12288 data: 3 3 3
5217 101/300 Write block 107 len 16384 data: 3 3 4 2
5218 102/300 Read block 41 len 20480
5219 103/300 Read block 3122 len 12288
5220 104/300 Read block 3277 len 8192
5221 105/300 Write block 621 len 32768 data: 3 3 3 3 3 4 2 2
5222 106/300 Read block 1128 len 4096
5223 107/300 Read block 3460 len 32768
5224 108/300 Read block 2173 len 40960
5225 109/300 Read block 4025 len 12288
5226 110/300 Read block 2358 len 24576
5227 111/300 Read block 844 len 24576
5228 112/300 Read block 2491 len 16384
5229 113/300 Read block 3106 len 12288
5230 114/300 Flush
5231 115/300 Read block 3463 len 16384
5232 116/300 Read block 4688 len 32768
5233 117/300 Flush
5234 118/300 Write block 847 len 32768 data: 3 3 2 3 3 3 2 2
5235 119/300 Write block 141 len 16384 data: 4 4 4 3
5236 120/300 Read block 723 len 24576
5237 121/300 Read block 2020 len 36864
5238 122/300 Read block 2633 len 4096
5239 123/300 Read block 2895 len 24576
5240 124/300 Flush
5241 125/300 Read block 4320 len 32768
5242 126/300 Write block 4228 len 8192 data: 3 2
5243 127/300 Read block 1691 len 12288
5244 128/300 Read block 4675 len 4096
5245 129/300 Read block 600 len 36864
5246 130/300 Read block 5 len 24576
5247 131/300 Read block 3068 len 8192
5248 132/300 Read block 1277 len 40960
5249 133/300 Write block 1852 len 40960 data: 3 3 3 3 3 3 3 2 2 2
5250 134/300 Flush
5251 135/300 Write block 803 len 32768 data: 2 2 2 2 2 2 2 3
5252 136/300 Write block 3119 len 4096 data: 3
5253 137/300 Read block 945 len 24576
5254 138/300 Read block 2153 len 12288
5255 139/300 Read block 1753 len 12288
5256 140/300 Write block 2002 len 24576 data: 3 3 3 3 2 2
5257 141/300 Flush
5258 142/300 Write block 3411 len 24576 data: 2 2 2 2 2 2
5259 143/300 Read block 1390 len 28672
5260 144/300 Flush
5261 145/300 Flush
5262 146/300 Write block 1580 len 36864 data: 3 3 2 2 2 2 2 2 2
5263 147/300 Read block 932 len 8192
5264 148/300 Read block 4885 len 28672
5265 149/300 Read block 3379 len 8192
5266 150/300 Write block 3173 len 32768 data: 4 4 4 4 5 4 2 2
5267 151/300 Read block 2497 len 4096
5268 152/300 Read block 3622 len 24576
5269 153/300 Write block 241 len 28672 data: 2 2 2 2 3 4 4
5270 154/300 Flush
5271 155/300 Read block 733 len 8192
5272 156/300 Write block 3423 len 32768 data: 2 2 2 3 4 4 3 3
5273 157/300 Write block 595 len 32768 data: 3 3 4 4 4 3 3 3
5274 158/300 Flush
5275 159/300 Read block 213 len 4096
5276 160/300 Write block 2693 len 20480 data: 2 2 2 3 3
5277 161/300 Read block 4062 len 4096
5278 162/300 Read block 2158 len 28672
5279 163/300 Read block 2252 len 24576
5280 164/300 Read block 3979 len 32768
5281 165/300 Flush
5282 166/300 Read block 1101 len 12288
5283 167/300 Read block 1960 len 4096
5284 168/300 Flush
5285 169/300 Write block 3875 len 8192 data: 3 3
5286 170/300 Read block 2648 len 36864
5287 171/300 Flush
5288 172/300 Write block 2670 len 4096 data: 2
5289 173/300 Read block 3317 len 24576
5290 174/300 Read block 4444 len 24576
5291 175/300 Read block 3245 len 16384
5292 176/300 Write block 2734 len 20480 data: 2 2 2 2 2
5293 177/300 Write block 3303 len 28672 data: 2 2 2 2 2 2 2
5294 178/300 Read block 3486 len 16384
5295 179/300 Read block 429 len 16384
5296 180/300 Write block 4671 len 32768 data: 2 2 2 2 2 3 3 4
5297 181/300 Read block 3856 len 16384
5298 182/300 Write block 2701 len 24576 data: 2 2 2 2 2 3
5299 183/300 Write block 2623 len 24576 data: 3 3 3 2 2 2
5300 184/300 Read block 97 len 24576
5301 185/300 Flush
5302 186/300 Write block 2241 len 20480 data: 2 2 2 2 2
5303 187/300 Write block 316 len 32768 data: 2 2 2 3 3 4 4 4
5304 188/300 Read block 1322 len 40960
5305 189/300 Write block 3709 len 12288 data: 2 2 2
5306 190/300 Write block 335 len 12288 data: 3 3 3
5307 191/300 Flush
5308 192/300 Read block 1362 len 32768
5309 193/300 Write block 562 len 20480 data: 2 3 3 3 4
5310 194/300 Read block 403 len 4096
5311 195/300 Write block 1100 len 20480 data: 2 2 2 2 2
5312 196/300 Read block 3196 len 16384
5313 197/300 Write block 4630 len 20480 data: 2 2 2 2 3
5314 198/300 Read block 1117 len 32768
5315 199/300 Read block 941 len 28672
5316 200/300 Write block 1742 len 40960 data: 3 3 2 2 2 2 2 3 3 3
5317 201/300 Write block 375 len 40960 data: 3 3 3 3 3 3 2 2 2 2
5318 202/300 Write block 488 len 20480 data: 2 2 2 2 2
5319 203/300 Read block 3888 len 32768
5320 204/300 Read block 333 len 32768
5321 205/300 Write block 1357 len 40960 data: 2 2 2 2 2 2 2 2 2 2
5322 206/300 Write block 4560 len 40960 data: 2 2 3 3 3 4 4 3 3 4
5323 207/300 Write block 3536 len 28672 data: 5 4 3 3 4 3 3
5324 208/300 Read block 1139 len 20480
5325 209/300 Read block 1347 len 16384
5326 210/300 Read block 306 len 8192
5327 211/300 Write block 4918 len 16384 data: 3 3 3 2
5328 212/300 Flush
5329 213/300 Read block 2453 len 32768
5330 214/300 Read block 3052 len 24576
5331 215/300 Write block 4883 len 4096 data: 4
5332 216/300 Read block 2829 len 16384
5333 217/300 Write block 3018 len 32768 data: 2 2 2 2 2 2 3 3
5334 218/300 Read block 4375 len 40960
5335 219/300 Flush
5336 220/300 Read block 4656 len 28672
5337 221/300 Flush
5338 222/300 Read block 3380 len 36864
5339 223/300 Flush
5340 224/300 Read block 3185 len 32768
5341 225/300 Flush
5342 226/300 Write block 1961 len 20480 data: 2 2 2 3 3
5343 227/300 Read block 376 len 4096
5344 228/300 Write block 2962 len 16384 data: 5 4 3 3
5345 229/300 Read block 2994 len 40960
5346 230/300 Write block 4844 len 24576 data: 3 4 4 4 4 5
5347 231/300 Write block 1782 len 12288 data: 3 3 3
5348 232/300 Read block 4074 len 32768
5349 233/300 Read block 1589 len 8192
5350 234/300 Flush
5351 235/300 Read block 2983 len 12288
5352 236/300 Write block 963 len 40960 data: 4 4 5 5 5 4 3 3 3 3
5353 237/300 Write block 1395 len 8192 data: 2 2
5354 238/300 Read block 4700 len 40960
5355 239/300 Read block 4179 len 36864
5356 240/300 Write block 4281 len 12288 data: 2 3 3
5357 241/300 Write block 1010 len 16384 data: 2 2 2 2
5358 242/300 Read block 1891 len 12288
5359 243/300 Read block 2787 len 8192
5360 244/300 Write block 338 len 40960 data: 2 2 2 2 2 2 2 2 2 3
5361 245/300 Flush
5362 246/300 Read block 4037 len 20480
5363 247/300 Flush
5364 248/300 Read block 631 len 28672
5365 249/300 Write block 511 len 12288 data: 3 2 2
5366 250/300 Write block 4704 len 32768 data: 2 2 2 2 2 2 2 3
5367 251/300 Write block 1755 len 16384 data: 3 3 2 2
5368 252/300 Read block 4632 len 40960
5369 253/300 Flush
5370 254/300 Flush
5371 255/300 Flush
5372 256/300 Read block 1774 len 4096
5373 257/300 Write block 1196 len 12288 data: 2 2 2
5374 258/300 Read block 740 len 40960
5375 259/300 Write block 2944 len 36864 data: 2 2 3 4 3 3 3 3 3
5376 260/300 Read block 2261 len 24576
5377 261/300 Read block 2113 len 12288
5378 262/300 Flush
5379 263/300 Read block 3298 len 16384
5380 264/300 Write block 2992 len 4096 data: 2
5381 265/300 Read block 3231 len 36864
5382 266/300 Read block 4378 len 8192
5383 267/300 Flush
5384 268/300 Read block 4869 len 16384
5385 269/300 Write block 1776 len 4096 data: 2
5386 270/300 Write block 2431 len 40960 data: 4 4 4 4 4 4 3 3 3 3
5387 271/300 Read block 1642 len 20480
5388 272/300 Read block 571 len 8192
5389 273/300 Read block 115 len 4096
5390 274/300 Read block 733 len 32768
5391 275/300 Write block 2324 len 12288 data: 2 3 3
5392 276/300 Write block 395 len 12288 data: 2 2 2
5393 277/300 Read block 4163 len 4096
5394 278/300 Read block 1065 len 16384
5395 279/300 Write block 3761 len 28672 data: 2 2 2 2 2 2 2
5396 280/300 Write block 4460 len 32768 data: 3 3 2 3 3 3 3 3
5397 281/300 Read block 134 len 4096
5398 282/300 Flush
5399 283/300 Read block 4442 len 8192
5400 284/300 Read block 3903 len 20480
5401 285/300 Flush
5402 286/300 Write block 764 len 32768 data: 2 2 2 2 2 2 2 2
5403 287/300 Read block 4212 len 8192
5404 288/300 Flush
5405 289/300 Read block 51 len 16384
5406 290/300 Write block 2703 len 16384 data: 3 3 3 4
5407 291/300 Read block 4021 len 24576
5408 292/300 Write block 3200 len 8192 data: 2 2
5409 293/300 Write block 3761 len 4096 data: 3
5410 294/300 Write block 873 len 36864 data: 2 2 2 3 3 4 4 4 5
5411 295/300 Flush
5412 296/300 Flush
5413 297/300 Write block 3830 len 16384 data: 3 3 2 2
5414 298/300 Write block 4309 len 32768 data: 2 2 2 2 2 2 2 2
5415 299/300 Read block 938 len 28672
5416 300/300 Flush
5417 301/300 Write block 2440 len 16384 data: 4 3 2 2
5418 Main task has work to do, go find it
5419 got dsc Start(0)
5420 start 0
5421 Nov 28 04:51:28.452 INFO request completed, latency_us: 208, response_code: 204, uri: /start/cid/0, method: POST, req_id: 185653e9-8f1c-4f4b-b22c-1517ea99b9e2, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
5422 Replay: started 0, returned:Ok(())
5423 ----------------------------------------------------------------
5424 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
5425 States: Offline Active Active
5426 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
5427 2510 Acked Read 2 Sent Done Done false
5428 2511 Acked Read 4 Sent Done Done false
5429 2512 Acked Read 8 Sent Done Done false
5430 2513 Acked Read 7 Sent Done Done false
5431 2514 Acked Write 1 Sent Done Done false
5432 2515 Acked Write 5 Sent Done Done false
5433 2516 Acked Write 4 Sent Done Done false
5434 2517 Acked Write 4 Sent Done Done false
5435 2518 Acked Write 5 Sent Done Done false
5436 2519 Acked Write 10 Sent Done Done false
5437 2520 Acked Write 1 Sent Done Done false
5438 2521 Acked Read 6 Sent Done Done false
5439 2522 Acked Read 9 Sent Done Done false
5440 2523 Acked Read 3 Sent Done Done false
5441 2524 Acked Write 1 Sent Done Done false
5442 2525 Acked Write 5 Sent Done Done false
5443 2526 Acked Read 10 Sent Done Done false
5444 2527 Acked Write 10 Sent Done Done false
5445 2528 Acked Flush 0 Sent Done Done false
5446 2529 Acked Read 7 Sent Done Done false
5447 2530 Acked Read 1 Sent Done Done false
5448 2531 Acked Write 3 Sent Done Done false
5449 2532 Acked Flush 0 Sent Done Done false
5450 2533 Acked Read 3 Sent Done Done false
5451 2534 Acked Flush 0 Sent Done Done false
5452 2535 Acked Read 2 Sent Done Done false
5453 2536 Acked Read 7 Sent Done Done false
5454 2537 Acked Read 4 Sent Done Done false
5455 2538 Acked Read 5 Sent Done Done false
5456 2539 Acked Write 8 Sent Done Done false
5457 2540 Acked Read 3 Sent Done Done false
5458 2541 Acked Read 5 Sent Done Done false
5459 2542 Acked Read 4 Sent Done Done false
5460 2543 Acked Read 6 Sent Done Done false
5461 2544 Acked Write 2 Sent Done Done false
5462 2545 Acked Write 5 Sent Done Done false
5463 2546 Acked Read 8 Sent Done Done false
5464 2547 Acked Read 10 Sent Done Done false
5465 2548 Acked Write 4 Sent Done Done false
5466 2549 Acked Read 4 Sent Done Done false
5467 2550 Acked Read 2 Sent Done Done false
5468 2551 Acked Write 10 Sent Done Done false
5469 2552 Acked Read 1 Sent Done Done false
5470 2553 Acked Flush 0 Sent Done Done false
5471 2554 Acked Read 6 Sent Done Done false
5472 2555 Acked Write 7 Sent Done Done false
5473 2556 Acked Read 10 Sent Done Done false
5474 2557 Acked Write 5 Sent Done Done false
5475 2558 Acked Write 3 Sent Done Done false
5476 2559 Acked Write 7 Sent Done Done false
5477 2560 Acked Read 10 Sent Done Done false
5478 2561 Acked Read 1 Sent Done Done false
5479 2562 Acked Write 9 Sent Done Done false
5480 2563 Acked Write 5 Sent Done Done false
5481 2564 Acked Write 2 Sent Done Done false
5482 2565 Acked Read 3 Sent Done Done false
5483 2566 Acked Read 2 Sent Done Done false
5484 2567 Acked Read 2 Sent Done Done false
5485 2568 Acked Read 8 Sent Done Done false
5486 2569 Acked Read 9 Sent Done Done false
5487 2570 Acked Read 9 Sent Done Done false
5488 2571 Acked Read 4 Sent Done Done false
5489 2572 Acked Flush 0 Sent Done Done false
5490 2573 Acked Write 8 Sent Done Done false
5491 2574 Acked Read 5 Sent Done Done false
5492 2575 Acked Write 8 Sent Done Done false
5493 2576 Acked Write 4 Sent Done Done false
5494 2577 Acked Write 7 Sent Done Done false
5495 2578 Acked Write 3 Sent Done Done false
5496 2579 Acked Write 3 Sent Done Done false
5497 2580 Acked Read 7 Sent Done Done false
5498 2581 Acked Write 9 Sent Done Done false
5499 2582 Acked Read 3 Sent Done Done false
5500 2583 Acked Flush 0 Sent Done Done false
5501 2584 Acked Read 6 Sent Done Done false
5502 2585 Acked Write 5 Sent Done Done false
5503 2586 Acked Read 8 Sent Done Done false
5504 2587 Acked Write 2 Sent Done Done false
5505 2588 Acked Read 2 Sent Done Done false
5506 2589 Acked Flush 0 Sent Done Done false
5507 2590 Acked Write 5 Sent Done Done false
5508 2591 Acked Write 4 Sent Done Done false
5509 2592 Acked Write 10 Sent Done Done false
5510 2593 Acked Read 1 Sent Done Done false
5511 2594 Acked Write 1 Sent Done Done false
5512 2595 Acked Read 3 Sent Done Done false
5513 2596 Acked Write 6 Sent Done Done false
5514 2597 Acked Write 7 Sent Done Done false
5515 2598 Acked Read 10 Sent Done Done false
5516 2599 Acked Read 3 Sent Done Done false
5517 2600 Acked Read 4 Sent Done Done false
5518 2601 Acked Read 7 Sent Done Done false
5519 2602 Acked Read 9 Sent Done Done false
5520 2603 Acked Read 7 Sent Done Done false
5521 2604 Acked Read 7 Sent Done Done false
5522 2605 Acked Read 2 Sent Done Done false
5523 2606 Acked Read 2 Sent Done Done false
5524 2607 Acked Write 4 Sent Done Done false
5525 2608 Acked Read 9 Sent Done Done false
5526 2609 Acked Write 3 Sent Done Done false
5527 2610 Acked Write 4 Sent Done Done false
5528 2611 Acked Read 5 Sent Done Done false
5529 2612 Acked Read 3 Sent Done Done false
5530 2613 Acked Read 2 Sent Done Done false
5531 2614 Acked Write 8 Sent Done Done false
5532 2615 Acked Read 1 Sent Done Done false
5533 2616 Acked Read 8 Sent Done Done false
5534 2617 Acked Read 10 Sent Done Done false
5535 2618 Acked Read 3 Sent Done Done false
5536 2619 Acked Read 6 Sent Done Done false
5537 2620 Acked Read 6 Sent Done Done false
5538 2621 Acked Read 4 Sent Done Done false
5539 2622 Acked Read 3 Sent Done Done false
5540 2623 Acked Flush 0 Sent Done Done false
5541 2624 Acked Read 4 Sent Done Done false
5542 2625 Acked Read 8 Sent Done Done false
5543 2626 Acked Flush 0 Sent Done Done false
5544 2627 Acked Write 8 Sent Done Done false
5545 2628 Acked Write 4 Sent Done Done false
5546 2629 Acked Read 6 Sent Done Done false
5547 2630 Acked Read 9 Sent Done Done false
5548 2631 Acked Read 1 Sent Done Done false
5549 2632 Acked Read 6 Sent Done Done false
5550 2633 Acked Flush 0 Sent Done Done false
5551 2634 Acked Read 8 Sent Done Done false
5552 2635 Acked Write 2 Sent Done Done false
5553 2636 Acked Read 3 Sent Done Done false
5554 2637 Acked Read 1 Sent Done Done false
5555 2638 Acked Read 9 Sent Done Done false
5556 2639 Acked Read 6 Sent Done Done false
5557 2640 Acked Read 2 Sent Done Done false
5558 2641 Acked Read 10 Sent Done Done false
5559 2642 Acked Write 10 Sent Done Done false
5560 2643 Acked Flush 0 Sent Done Done false
5561 2644 Acked Write 8 Sent Done Done false
5562 2645 Acked Write 1 Sent Done Done false
5563 2646 Acked Read 6 Sent Done Done false
5564 2647 Acked Read 3 Sent Done Done false
5565 2648 Acked Read 3 Sent Done Done false
5566 2649 Acked Write 6 Sent Done Done false
5567 2650 Acked Flush 0 Sent Done Done false
5568 2651 Acked Write 6 Sent Done Done false
5569 2652 Acked Read 7 Sent Done Done false
5570 2653 Acked Flush 0 Sent Done Done false
5571 2654 Acked Flush 0 Sent Done Done false
5572 2655 Acked Write 9 Sent Done Done false
5573 2656 Acked Read 2 Sent Done Done false
5574 2657 Acked Read 7 Sent Done Done false
5575 2658 Acked Read 2 Sent Done Done false
5576 2659 Acked Write 8 Sent Done Done false
5577 2660 Acked Read 1 Sent Done Done false
5578 2661 Acked Read 6 Sent Done Done false
5579 2662 Acked Write 7 Sent Done Done false
5580 2663 Acked Flush 0 Sent Done Done false
5581 2664 Acked Read 2 Sent Done Done false
5582 2665 Acked Write 8 Sent Done Done false
5583 2666 Acked Write 8 Sent Done Done false
5584 2667 Acked Flush 0 Sent Done Done false
5585 2668 Acked Read 1 Sent Done Done false
5586 2669 Acked Write 5 Sent Done Done false
5587 2670 Acked Read 1 Sent Done Done false
5588 2671 Acked Read 7 Sent Done Done false
5589 2672 Acked Read 6 Sent Done Done false
5590 2673 Acked Read 8 Sent Done Done false
5591 2674 Acked Flush 0 Sent Done Done false
5592 2675 Acked Read 3 Sent Done Done false
5593 2676 Acked Read 1 Sent Done Done false
5594 2677 Acked Flush 0 Sent Done Done false
5595 2678 Acked Write 2 Sent Done Done false
5596 2679 Acked Read 9 Sent Done Done false
5597 2680 Acked Flush 0 Sent Done Done false
5598 2681 Acked Write 1 Sent Done Done false
5599 2682 Acked Read 6 Sent Done Done false
5600 2683 Acked Read 6 Sent Done Done false
5601 2684 Acked Read 4 Sent Done Done false
5602 2685 Acked Write 5 Sent Done Done false
5603 2686 Acked Write 7 Sent Done Done false
5604 2687 Acked Read 4 Sent Done Done false
5605 2688 Acked Read 4 Sent Done Done false
5606 2689 Acked Write 8 Sent Done Done false
5607 2690 Acked Read 4 Sent Done Done false
5608 2691 Acked Write 6 Sent Done Done false
5609 2692 Acked Write 6 Sent Done Done false
5610 2693 Acked Read 6 Sent Done Done false
5611 2694 Acked Flush 0 Sent Done Done false
5612 2695 Acked Write 5 Sent Done Done false
5613 2696 Acked Write 8 Sent Done Done false
5614 2697 Acked Read 10 Sent Done Done false
5615 2698 Acked Write 3 Sent Done Done false
5616 2699 Acked Write 3 Sent Done Done false
5617 2700 Acked Flush 0 Sent Done Done false
5618 2701 Acked Read 8 Sent Done Done false
5619 2702 Acked Write 5 Sent Done Done false
5620 2703 Acked Read 1 Sent Done Done false
5621 2704 Acked Write 5 Sent Done Done false
5622 2705 Acked Read 4 Sent Done Done false
5623 2706 Acked Write 5 Sent Done Done false
5624 2707 Acked Read 8 Sent Done Done false
5625 2708 Acked Read 7 Sent Done Done false
5626 2709 Acked Write 10 Sent Done Done false
5627 2710 Acked Write 10 Sent Done Done false
5628 2711 Acked Write 5 Sent Done Done false
5629 2712 Acked Read 8 Sent Done Done false
5630 2713 Acked Read 8 Sent Done Done false
5631 2714 Acked Write 10 Sent Done Done false
5632 2715 Acked Write 10 Sent Done Done false
5633 2716 Acked Write 7 Sent Done Done false
5634 2717 Acked Read 5 Sent Done Done false
5635 2718 Acked Read 4 Sent Done Done false
5636 2719 Acked Read 2 Sent Done Done false
5637 2720 Acked Write 4 Sent Done Done false
5638 2721 Acked Flush 0 Sent Done Done false
5639 2722 Acked Read 8 Sent Done Done false
5640 2723 Acked Read 6 Sent Done Done false
5641 2724 Acked Write 1 Sent Done Done false
5642 2725 Acked Read 4 Sent Done Done false
5643 2726 Acked Write 8 Sent Done Done false
5644 2727 Acked Read 10 Sent Done Done false
5645 2728 Acked Flush 0 Sent Done Done false
5646 2729 Acked Read 7 Sent Done Done false
5647 2730 Acked Flush 0 Sent Done Done false
5648 2731 Acked Read 9 Sent Done Done false
5649 2732 Acked Flush 0 Sent Done Done false
5650 2733 Acked Read 8 Sent Done Done false
5651 2734 Acked Flush 0 Sent Done Done false
5652 2735 Acked Write 5 Sent Done Done false
5653 2736 Acked Read 1 Sent Done Done false
5654 2737 Acked Write 4 Sent Done Done false
5655 2738 Acked Read 10 Sent Done Done false
5656 2739 Acked Write 6 Sent Done Done false
5657 2740 Acked Write 3 Sent Done Done false
5658 2741 Acked Read 8 Sent Done Done false
5659 2742 Acked Read 2 Sent Done Done false
5660 2743 Acked Flush 0 Sent Done Done false
5661 2744 Acked Read 3 Sent Done Done false
5662 2745 Acked Write 10 Sent Done Done false
5663 2746 Acked Write 2 Sent Done Done false
5664 2747 Acked Read 10 Sent Done Done false
5665 2748 Acked Read 9 Sent Done Done false
5666 2749 Acked Write 3 Sent Done Done false
5667 2750 Acked Write 4 Sent Done Done false
5668 2751 Acked Read 3 Sent Done Done false
5669 2752 Acked Read 2 Sent Done Done false
5670 2753 Acked Write 10 Sent Done Done false
5671 2754 Acked Flush 0 Sent Done Done false
5672 2755 Acked Read 5 Sent Done Done false
5673 2756 Acked Flush 0 Sent Done Done false
5674 2757 Acked Read 7 Sent Done Done false
5675 2758 Acked Write 3 Sent Done Done false
5676 2759 Acked Write 8 Sent Done Done false
5677 2760 Acked Write 4 Sent Done Done false
5678 2761 Acked Read 10 Sent Done Done false
5679 2762 Acked Flush 0 Sent Done Done false
5680 2763 Acked Flush 0 Sent Done Done false
5681 2764 Acked Flush 0 Sent Done Done false
5682 2765 Acked Read 1 Sent Done Done false
5683 2766 Acked Write 3 Sent Done Done false
5684 2767 Acked Read 10 Sent Done Done false
5685 2768 Acked Write 9 Sent Done Done false
5686 2769 Acked Read 6 Sent Done Done false
5687 2770 Acked Read 3 Sent Done Done false
5688 2771 Acked Flush 0 Sent Done Done false
5689 2772 Acked Read 4 Sent Done Done false
5690 2773 Acked Write 1 Sent Done Done false
5691 2774 Acked Read 9 Sent Done Done false
5692 2775 Acked Read 2 Sent Done Done false
5693 2776 Acked Flush 0 Sent Done Done false
5694 2777 Acked Read 4 Sent Done Done false
5695 2778 Acked Write 1 Sent Done Done false
5696 2779 Acked Write 10 Sent Done Done false
5697 2780 Acked Read 5 Sent Done Done false
5698 2781 Acked Read 2 Sent Done Done false
5699 2782 Acked Read 1 Sent Done Done false
5700 2783 Acked Read 8 Sent Done Done false
5701 2784 Acked Write 3 Sent Done Done false
5702 2785 Acked Write 3 Sent Done Done false
5703 2786 Acked Read 1 Sent Done Done false
5704 2787 Acked Read 4 Sent Done Done false
5705 2788 Acked Write 7 Sent Done Done false
5706 2789 Acked Write 8 Sent Done Done false
5707 2790 Acked Read 1 Sent Done Done false
5708 2791 Acked Flush 0 Sent Done Done false
5709 2792 Acked Read 2 Sent Done Done false
5710 2793 Acked Read 5 Sent Done Done false
5711 2794 Acked Flush 0 Sent Done Done false
5712 2795 Acked Write 8 Sent Done Done false
5713 2796 Acked Read 2 Sent Done Done false
5714 2797 Acked Flush 0 Sent Done Done false
5715 2798 Acked Read 4 Sent Done Done false
5716 2799 Acked Write 4 Sent Done Done false
5717 2800 Acked Read 6 Sent Done Done false
5718 2801 Acked Write 2 Sent Done Done false
5719 2802 Acked Write 1 Sent Done Done false
5720 2803 Acked Write 9 Sent Done Done false
5721 2804 Acked Flush 0 Sent Done Done false
5722 2805 Acked Flush 0 Sent Done Done false
5723 2806 Acked Write 4 Sent Done Done false
5724 2807 Acked Write 8 Sent Done Done false
5725 2808 Acked Read 7 Sent Done Done false
5726 2809 Acked Flush 0 Sent Done Done false
5727 2810 Acked Write 4 Sent Done Done false
5728 STATES DS:0 DS:1 DS:2 TOTAL
5729 Sent 301 0 0 301
5730 Done 0 301 301 602
5731 Skipped 0 0 0 0
5732 Error 0 0 0 0
5733 Last Flush: 2509 2809 2809
5734 Downstairs last five completed: 2509 2508 2507 2506 2505
5735 Upstairs last five completed: 2810 2809 2808 2807 2806
5736 CLIENT: Up:0 ds:301 act:2
5737 [8810] got start action
5738 [8810] I am going to restart
5739 Starting downstairs at port 8810
5740 Make output file at "/tmp/dsc/downstairs-8810.txt"
5741 [8810][0] reports Starting
5742 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1208)
5743 [8810][0] reports Running
5744 ----------------------------------------------------------------
5745 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
5746 States: Offline Active Active
5747 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
5748 2510 Acked Read 2 Sent Done Done false
5749 2511 Acked Read 4 Sent Done Done false
5750 2512 Acked Read 8 Sent Done Done false
5751 2513 Acked Read 7 Sent Done Done false
5752 2514 Acked Write 1 Sent Done Done false
5753 2515 Acked Write 5 Sent Done Done false
5754 2516 Acked Write 4 Sent Done Done false
5755 2517 Acked Write 4 Sent Done Done false
5756 2518 Acked Write 5 Sent Done Done false
5757 2519 Acked Write 10 Sent Done Done false
5758 2520 Acked Write 1 Sent Done Done false
5759 2521 Acked Read 6 Sent Done Done false
5760 2522 Acked Read 9 Sent Done Done false
5761 2523 Acked Read 3 Sent Done Done false
5762 2524 Acked Write 1 Sent Done Done false
5763 2525 Acked Write 5 Sent Done Done false
5764 2526 Acked Read 10 Sent Done Done false
5765 2527 Acked Write 10 Sent Done Done false
5766 2528 Acked Flush 0 Sent Done Done false
5767 2529 Acked Read 7 Sent Done Done false
5768 2530 Acked Read 1 Sent Done Done false
5769 2531 Acked Write 3 Sent Done Done false
5770 2532 Acked Flush 0 Sent Done Done false
5771 2533 Acked Read 3 Sent Done Done false
5772 2534 Acked Flush 0 Sent Done Done false
5773 2535 Acked Read 2 Sent Done Done false
5774 2536 Acked Read 7 Sent Done Done false
5775 2537 Acked Read 4 Sent Done Done false
5776 2538 Acked Read 5 Sent Done Done false
5777 2539 Acked Write 8 Sent Done Done false
5778 2540 Acked Read 3 Sent Done Done false
5779 2541 Acked Read 5 Sent Done Done false
5780 2542 Acked Read 4 Sent Done Done false
5781 2543 Acked Read 6 Sent Done Done false
5782 2544 Acked Write 2 Sent Done Done false
5783 2545 Acked Write 5 Sent Done Done false
5784 2546 Acked Read 8 Sent Done Done false
5785 2547 Acked Read 10 Sent Done Done false
5786 2548 Acked Write 4 Sent Done Done false
5787 2549 Acked Read 4 Sent Done Done false
5788 2550 Acked Read 2 Sent Done Done false
5789 2551 Acked Write 10 Sent Done Done false
5790 2552 Acked Read 1 Sent Done Done false
5791 2553 Acked Flush 0 Sent Done Done false
5792 2554 Acked Read 6 Sent Done Done false
5793 2555 Acked Write 7 Sent Done Done false
5794 2556 Acked Read 10 Sent Done Done false
5795 2557 Acked Write 5 Sent Done Done false
5796 2558 Acked Write 3 Sent Done Done false
5797 2559 Acked Write 7 Sent Done Done false
5798 2560 Acked Read 10 Sent Done Done false
5799 2561 Acked Read 1 Sent Done Done false
5800 2562 Acked Write 9 Sent Done Done false
5801 2563 Acked Write 5 Sent Done Done false
5802 2564 Acked Write 2 Sent Done Done false
5803 2565 Acked Read 3 Sent Done Done false
5804 2566 Acked Read 2 Sent Done Done false
5805 2567 Acked Read 2 Sent Done Done false
5806 2568 Acked Read 8 Sent Done Done false
5807 2569 Acked Read 9 Sent Done Done false
5808 2570 Acked Read 9 Sent Done Done false
5809 2571 Acked Read 4 Sent Done Done false
5810 2572 Acked Flush 0 Sent Done Done false
5811 2573 Acked Write 8 Sent Done Done false
5812 2574 Acked Read 5 Sent Done Done false
5813 2575 Acked Write 8 Sent Done Done false
5814 2576 Acked Write 4 Sent Done Done false
5815 2577 Acked Write 7 Sent Done Done false
5816 2578 Acked Write 3 Sent Done Done false
5817 2579 Acked Write 3 Sent Done Done false
5818 2580 Acked Read 7 Sent Done Done false
5819 2581 Acked Write 9 Sent Done Done false
5820 2582 Acked Read 3 Sent Done Done false
5821 2583 Acked Flush 0 Sent Done Done false
5822 2584 Acked Read 6 Sent Done Done false
5823 2585 Acked Write 5 Sent Done Done false
5824 2586 Acked Read 8 Sent Done Done false
5825 2587 Acked Write 2 Sent Done Done false
5826 2588 Acked Read 2 Sent Done Done false
5827 2589 Acked Flush 0 Sent Done Done false
5828 2590 Acked Write 5 Sent Done Done false
5829 2591 Acked Write 4 Sent Done Done false
5830 2592 Acked Write 10 Sent Done Done false
5831 2593 Acked Read 1 Sent Done Done false
5832 2594 Acked Write 1 Sent Done Done false
5833 2595 Acked Read 3 Sent Done Done false
5834 2596 Acked Write 6 Sent Done Done false
5835 2597 Acked Write 7 Sent Done Done false
5836 2598 Acked Read 10 Sent Done Done false
5837 2599 Acked Read 3 Sent Done Done false
5838 2600 Acked Read 4 Sent Done Done false
5839 2601 Acked Read 7 Sent Done Done false
5840 2602 Acked Read 9 Sent Done Done false
5841 2603 Acked Read 7 Sent Done Done false
5842 2604 Acked Read 7 Sent Done Done false
5843 2605 Acked Read 2 Sent Done Done false
5844 2606 Acked Read 2 Sent Done Done false
5845 2607 Acked Write 4 Sent Done Done false
5846 2608 Acked Read 9 Sent Done Done false
5847 2609 Acked Write 3 Sent Done Done false
5848 2610 Acked Write 4 Sent Done Done false
5849 2611 Acked Read 5 Sent Done Done false
5850 2612 Acked Read 3 Sent Done Done false
5851 2613 Acked Read 2 Sent Done Done false
5852 2614 Acked Write 8 Sent Done Done false
5853 2615 Acked Read 1 Sent Done Done false
5854 2616 Acked Read 8 Sent Done Done false
5855 2617 Acked Read 10 Sent Done Done false
5856 2618 Acked Read 3 Sent Done Done false
5857 2619 Acked Read 6 Sent Done Done false
5858 2620 Acked Read 6 Sent Done Done false
5859 2621 Acked Read 4 Sent Done Done false
5860 2622 Acked Read 3 Sent Done Done false
5861 2623 Acked Flush 0 Sent Done Done false
5862 2624 Acked Read 4 Sent Done Done false
5863 2625 Acked Read 8 Sent Done Done false
5864 2626 Acked Flush 0 Sent Done Done false
5865 2627 Acked Write 8 Sent Done Done false
5866 2628 Acked Write 4 Sent Done Done false
5867 2629 Acked Read 6 Sent Done Done false
5868 2630 Acked Read 9 Sent Done Done false
5869 2631 Acked Read 1 Sent Done Done false
5870 2632 Acked Read 6 Sent Done Done false
5871 2633 Acked Flush 0 Sent Done Done false
5872 2634 Acked Read 8 Sent Done Done false
5873 2635 Acked Write 2 Sent Done Done false
5874 2636 Acked Read 3 Sent Done Done false
5875 2637 Acked Read 1 Sent Done Done false
5876 2638 Acked Read 9 Sent Done Done false
5877 2639 Acked Read 6 Sent Done Done false
5878 2640 Acked Read 2 Sent Done Done false
5879 2641 Acked Read 10 Sent Done Done false
5880 2642 Acked Write 10 Sent Done Done false
5881 2643 Acked Flush 0 Sent Done Done false
5882 2644 Acked Write 8 Sent Done Done false
5883 2645 Acked Write 1 Sent Done Done false
5884 2646 Acked Read 6 Sent Done Done false
5885 2647 Acked Read 3 Sent Done Done false
5886 2648 Acked Read 3 Sent Done Done false
5887 2649 Acked Write 6 Sent Done Done false
5888 2650 Acked Flush 0 Sent Done Done false
5889 2651 Acked Write 6 Sent Done Done false
5890 2652 Acked Read 7 Sent Done Done false
5891 2653 Acked Flush 0 Sent Done Done false
5892 2654 Acked Flush 0 Sent Done Done false
5893 2655 Acked Write 9 Sent Done Done false
5894 2656 Acked Read 2 Sent Done Done false
5895 2657 Acked Read 7 Sent Done Done false
5896 2658 Acked Read 2 Sent Done Done false
5897 2659 Acked Write 8 Sent Done Done false
5898 2660 Acked Read 1 Sent Done Done false
5899 2661 Acked Read 6 Sent Done Done false
5900 2662 Acked Write 7 Sent Done Done false
5901 2663 Acked Flush 0 Sent Done Done false
5902 2664 Acked Read 2 Sent Done Done false
5903 2665 Acked Write 8 Sent Done Done false
5904 2666 Acked Write 8 Sent Done Done false
5905 2667 Acked Flush 0 Sent Done Done false
5906 2668 Acked Read 1 Sent Done Done false
5907 2669 Acked Write 5 Sent Done Done false
5908 2670 Acked Read 1 Sent Done Done false
5909 2671 Acked Read 7 Sent Done Done false
5910 2672 Acked Read 6 Sent Done Done false
5911 2673 Acked Read 8 Sent Done Done false
5912 2674 Acked Flush 0 Sent Done Done false
5913 2675 Acked Read 3 Sent Done Done false
5914 2676 Acked Read 1 Sent Done Done false
5915 2677 Acked Flush 0 Sent Done Done false
5916 2678 Acked Write 2 Sent Done Done false
5917 2679 Acked Read 9 Sent Done Done false
5918 2680 Acked Flush 0 Sent Done Done false
5919 2681 Acked Write 1 Sent Done Done false
5920 2682 Acked Read 6 Sent Done Done false
5921 2683 Acked Read 6 Sent Done Done false
5922 2684 Acked Read 4 Sent Done Done false
5923 2685 Acked Write 5 Sent Done Done false
5924 2686 Acked Write 7 Sent Done Done false
5925 2687 Acked Read 4 Sent Done Done false
5926 2688 Acked Read 4 Sent Done Done false
5927 2689 Acked Write 8 Sent Done Done false
5928 2690 Acked Read 4 Sent Done Done false
5929 2691 Acked Write 6 Sent Done Done false
5930 2692 Acked Write 6 Sent Done Done false
5931 2693 Acked Read 6 Sent Done Done false
5932 2694 Acked Flush 0 Sent Done Done false
5933 2695 Acked Write 5 Sent Done Done false
5934 2696 Acked Write 8 Sent Done Done false
5935 2697 Acked Read 10 Sent Done Done false
5936 2698 Acked Write 3 Sent Done Done false
5937 2699 Acked Write 3 Sent Done Done false
5938 2700 Acked Flush 0 Sent Done Done false
5939 2701 Acked Read 8 Sent Done Done false
5940 2702 Acked Write 5 Sent Done Done false
5941 2703 Acked Read 1 Sent Done Done false
5942 2704 Acked Write 5 Sent Done Done false
5943 2705 Acked Read 4 Sent Done Done false
5944 2706 Acked Write 5 Sent Done Done false
5945 2707 Acked Read 8 Sent Done Done false
5946 2708 Acked Read 7 Sent Done Done false
5947 2709 Acked Write 10 Sent Done Done false
5948 2710 Acked Write 10 Sent Done Done false
5949 2711 Acked Write 5 Sent Done Done false
5950 2712 Acked Read 8 Sent Done Done false
5951 2713 Acked Read 8 Sent Done Done false
5952 2714 Acked Write 10 Sent Done Done false
5953 2715 Acked Write 10 Sent Done Done false
5954 2716 Acked Write 7 Sent Done Done false
5955 2717 Acked Read 5 Sent Done Done false
5956 2718 Acked Read 4 Sent Done Done false
5957 2719 Acked Read 2 Sent Done Done false
5958 2720 Acked Write 4 Sent Done Done false
5959 2721 Acked Flush 0 Sent Done Done false
5960 2722 Acked Read 8 Sent Done Done false
5961 2723 Acked Read 6 Sent Done Done false
5962 2724 Acked Write 1 Sent Done Done false
5963 2725 Acked Read 4 Sent Done Done false
5964 2726 Acked Write 8 Sent Done Done false
5965 2727 Acked Read 10 Sent Done Done false
5966 2728 Acked Flush 0 Sent Done Done false
5967 2729 Acked Read 7 Sent Done Done false
5968 2730 Acked Flush 0 Sent Done Done false
5969 2731 Acked Read 9 Sent Done Done false
5970 2732 Acked Flush 0 Sent Done Done false
5971 2733 Acked Read 8 Sent Done Done false
5972 2734 Acked Flush 0 Sent Done Done false
5973 2735 Acked Write 5 Sent Done Done false
5974 2736 Acked Read 1 Sent Done Done false
5975 2737 Acked Write 4 Sent Done Done false
5976 2738 Acked Read 10 Sent Done Done false
5977 2739 Acked Write 6 Sent Done Done false
5978 2740 Acked Write 3 Sent Done Done false
5979 2741 Acked Read 8 Sent Done Done false
5980 2742 Acked Read 2 Sent Done Done false
5981 2743 Acked Flush 0 Sent Done Done false
5982 2744 Acked Read 3 Sent Done Done false
5983 2745 Acked Write 10 Sent Done Done false
5984 2746 Acked Write 2 Sent Done Done false
5985 2747 Acked Read 10 Sent Done Done false
5986 2748 Acked Read 9 Sent Done Done false
5987 2749 Acked Write 3 Sent Done Done false
5988 2750 Acked Write 4 Sent Done Done false
5989 2751 Acked Read 3 Sent Done Done false
5990 2752 Acked Read 2 Sent Done Done false
5991 2753 Acked Write 10 Sent Done Done false
5992 2754 Acked Flush 0 Sent Done Done false
5993 2755 Acked Read 5 Sent Done Done false
5994 2756 Acked Flush 0 Sent Done Done false
5995 2757 Acked Read 7 Sent Done Done false
5996 2758 Acked Write 3 Sent Done Done false
5997 2759 Acked Write 8 Sent Done Done false
5998 2760 Acked Write 4 Sent Done Done false
5999 2761 Acked Read 10 Sent Done Done false
6000 2762 Acked Flush 0 Sent Done Done false
6001 2763 Acked Flush 0 Sent Done Done false
6002 2764 Acked Flush 0 Sent Done Done false
6003 2765 Acked Read 1 Sent Done Done false
6004 2766 Acked Write 3 Sent Done Done false
6005 2767 Acked Read 10 Sent Done Done false
6006 2768 Acked Write 9 Sent Done Done false
6007 2769 Acked Read 6 Sent Done Done false
6008 2770 Acked Read 3 Sent Done Done false
6009 2771 Acked Flush 0 Sent Done Done false
6010 2772 Acked Read 4 Sent Done Done false
6011 2773 Acked Write 1 Sent Done Done false
6012 2774 Acked Read 9 Sent Done Done false
6013 2775 Acked Read 2 Sent Done Done false
6014 2776 Acked Flush 0 Sent Done Done false
6015 2777 Acked Read 4 Sent Done Done false
6016 2778 Acked Write 1 Sent Done Done false
6017 2779 Acked Write 10 Sent Done Done false
6018 2780 Acked Read 5 Sent Done Done false
6019 2781 Acked Read 2 Sent Done Done false
6020 2782 Acked Read 1 Sent Done Done false
6021 2783 Acked Read 8 Sent Done Done false
6022 2784 Acked Write 3 Sent Done Done false
6023 2785 Acked Write 3 Sent Done Done false
6024 2786 Acked Read 1 Sent Done Done false
6025 2787 Acked Read 4 Sent Done Done false
6026 2788 Acked Write 7 Sent Done Done false
6027 2789 Acked Write 8 Sent Done Done false
6028 2790 Acked Read 1 Sent Done Done false
6029 2791 Acked Flush 0 Sent Done Done false
6030 2792 Acked Read 2 Sent Done Done false
6031 2793 Acked Read 5 Sent Done Done false
6032 2794 Acked Flush 0 Sent Done Done false
6033 2795 Acked Write 8 Sent Done Done false
6034 2796 Acked Read 2 Sent Done Done false
6035 2797 Acked Flush 0 Sent Done Done false
6036 2798 Acked Read 4 Sent Done Done false
6037 2799 Acked Write 4 Sent Done Done false
6038 2800 Acked Read 6 Sent Done Done false
6039 2801 Acked Write 2 Sent Done Done false
6040 2802 Acked Write 1 Sent Done Done false
6041 2803 Acked Write 9 Sent Done Done false
6042 2804 Acked Flush 0 Sent Done Done false
6043 2805 Acked Flush 0 Sent Done Done false
6044 2806 Acked Write 4 Sent Done Done false
6045 2807 Acked Write 8 Sent Done Done false
6046 2808 Acked Read 7 Sent Done Done false
6047 2809 Acked Flush 0 Sent Done Done false
6048 2810 Acked Write 4 Sent Done Done false
6049 2811 Acked Flush 0 Sent Done Done false
6050 STATES DS:0 DS:1 DS:2 TOTAL
6051 Sent 302 0 0 302
6052 Done 0 302 302 604
6053 Skipped 0 0 0 0
6054 Error 0 0 0 0
6055 Last Flush: 2509 2811 2811
6056 Downstairs last five completed: 2509 2508 2507 2506 2505
6057 Upstairs last five completed: 2811 2810 2809 2808 2807
6058 CLIENT: Up:0 ds:302 act:2
60592024-11-28T04:51:34.224ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
60602024-11-28T04:51:34.224ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:64505) = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
60612024-11-28T04:51:34.225ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
60622024-11-28T04:51:34.226ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
60632024-11-28T04:51:34.226ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
60642024-11-28T04:51:34.226ZINFOcrucible: send last flush ID to this DS: 2509 = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
60652024-11-28T04:51:34.226ZINFOcrucible: Replied this last flush ID: 2509 = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
60662024-11-28T04:51:34.226ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
60672024-11-28T04:51:34.226ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
60682024-11-28T04:51:34.226ZINFOcrucible: [0] client re-new 302 jobs since flush 2509 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
60692024-11-28T04:51:34.226ZINFOcrucible: [0] Replayed 302 jobs since flush: 2509 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
6070 ----------------------------------------------------------------
6071 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
6072 Downstairs last five completed: 2811 2810 2809 2808 2807
6073 Upstairs last five completed: 2811 2810 2809 2808 2807
6074 CLIENT: Up:0 ds:0 act:3
6075 Replay: All jobs finished
6076 Main task has work to do, go find it
6077 got dsc Stop(0)
6078 stop 0
6079 Nov 28 04:51:36.456 INFO request completed, latency_us: 313, response_code:[8810] Got stop action so:false kr:false
6080 204, uri: /stop/cid/0, method: POST, req_id: 8f64e160-f9b3-4036-96d9-128e7d937933, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
6081 Nov 28 04:51:36.457 INFO request completed, latency_us: 128, response_code: 200, uri: /state/cid/0, method: GET, req_id: 6692e46c-f0a9-456a-b067-c9a3c1167d2a, remote_addr: 127.0.0.1:60418{"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2024-11-28T04:51:36.457151581Z","hostname":"w-01JDRHKEEFJ1228TFPSBEGXWHY","pid":1202,"":"io task","client":"0","":"downstairs","session_id":"4a2415b1-6594-436f-82e5-b8cf9a74eeaf"}
6082 , local_addr: 127.0.0.1:9998
60832024-11-28T04:51:36.457ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
60842024-11-28T04:51:36.457ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
60852024-11-28T04:51:36.457ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
60862024-11-28T04:51:36.457ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
60872024-11-28T04:51:36.457ZINFOcrucible: client task is exiting = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
6088 [8810] Exited with: ExitStatus(unix_wait_status(9))
6089 [8810][0] reports Exit
6090 Nov 28 04:51:40.459 INFO request completed, latency_us: 231, response_code: 200, uri: /state/cid/0, method: GET, req_id: 2e800b9a-17d7-4166-af72-f421b6469ff1, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
6091 001/300 Write block 245 len 36864 data: 4 5 5 4 4 4 4 4 2
6092 002/300 Write block 1609 len 28672 data: 3 3 3 3 3 3 3
6093 003/300 Read block 4838 len 36864
6094 004/300 Flush
6095 005/300 Read block 3346 len 20480
6096 006/300 Read block 4383 len 4096
6097 007/300 Write block 53 len 36864 data: 3 3 3 3 3 2 2 3 3
6098 008/300 Read block 1540 len 16384
6099 009/300 Write block 3633 len 32768 data: 3 2 2 2 3 3 3 3
6100 010/300 Read block 3933 len 32768
6101 011/300 Write block 3613 len 16384 data: 2 3 2 2
6102 012/300 Write block 2788 len 20480 data: 4 4 4 2 3
6103 013/300 Read block 4645 len 8192
6104 014/300 Read block 2550 len 40960
6105 015/300 Read block 3291 len 40960
6106 016/300 Write block 2517 len 8192 data: 3 3
6107 017/300 Read block 3765 len 40960
6108 018/300 Write block 4159 len 40960 data: 2 2 2 2 2 2 2 2 2 2
6109 019/300 Write block 4768 len 12288 data: 3 3 3
6110 020/300 Read block 4498 len 28672
6111 021/300 Write block 4288 len 24576 data: 2 2 2 2 2 2
6112 022/300 Write block 3472 len 36864 data: 2 2 2 2 2 2 2 2 2
6113 023/300 Read block 2997 len 36864
6114 024/300 Write block 1062 len 12288 data: 2 2 2
6115 025/300 Read block 2881 len 28672
6116 026/300 Read block 3587 len 20480
6117 027/300 Read block 601 len 32768
6118 028/300 Read block 1527 len 16384
6119 029/300 Write block 3005 len 16384 data: 3 3 3 3
6120 030/300 Read block 860 len 4096
6121 031/300 Read block 4457 len 32768
6122 032/300 Write block 85 len 20480 data: 2 2 2 3 3
6123 033/300 Write block 3902 len 16384 data: 2 2 2 2
6124 034/300 Read block 1119 len 20480
6125 035/300 Write block 4445 len 12288 data: 4 4 3
6126 036/300 Read block 1804 len 8192
6127 037/300 Read block 4559 len 8192
6128 038/300 Read block 4777 len 32768
6129 039/300 Write block 354 len 16384 data: 3 3 3 2
6130 040/300 Read block 874 len 36864
6131 041/300 Write block 4968 len 28672 data: 4 3 3 3 3 3 2
6132 042/300 Flush
6133 043/300 Read block 4634 len 16384
6134 044/300 Read block 1977 len 24576
6135 045/300 Read block 1345 len 16384
6136 046/300 Write block 1329 len 12288 data: 2 2 2
6137 047/300 Write block 547 len 24576 data: 3 3 3 2 2 2
6138 048/300 Read block 1744 len 16384
6139 049/300 Read block 2034 len 24576
6140 050/300 Write block 2850 len 28672 data: 3 3 3 3 2 2 2
6141 051/300 Read block 682 len 20480
6142 052/300 Read block 3337 len 32768
6143 053/300 Write block 2975 len 24576 data: 4 4 4 3 3 3
6144 054/300 Read block 1114 len 4096
6145 055/300 Write block 2965 len 20480 data: 4 3 3 2 2
6146 056/300 Read block 3842 len 32768
6147 057/300 Read block 4464 len 20480
6148 058/300 Write block 2028 len 36864 data: 2 2 2 2 3 3 4 4 4
6149 059/300 Read block 1258 len 28672
6150 060/300 Read block 2736 len 20480
6151 061/300 Write block 2653 len 40960 data: 3 2 2 2 2 2 2 2 2 2
6152 062/300 Write block 208 len 24576 data: 3 3 3 2 2 2
6153 063/300 Write block 4302 len 20480 data: 2 2 2 2 2
6154 064/300 Write block 1098 len 36864 data: 4 2 3 3 3 3 3 2 2
6155 065/300 Flush
6156 066/300 Flush
6157 067/300 Write block 3417 len 20480 data: 2 2 2 2 2
6158 068/300 Write block 1589 len 8192 data: 2 2
6159 069/300 Write block 553 len 32768 data: 3 4 4 4 4 3 3 2
6160 070/300 Write block 2506 len 8192 data: 2 2
6161 071/300 Read block 3701 len 32768
6162 072/300 Read block 1900 len 40960
6163 073/300 Write block 590 len 40960 data: 2 3 3 3 3 4 4 5 5 5
6164 074/300 Read block 2406 len 24576
6165 075/300 Read block 3678 len 16384
6166 076/300 Read block 1708 len 32768
6167 077/300 Write block 2944 len 36864 data: 3 3 4 5 4 4 4 4 4
6168 078/300 Write block 1751 len 20480 data: 4 3 3 3 4
6169 079/300 Write block 4834 len 8192 data: 2 2
6170 080/300 Write block 4842 len 8192 data: 2 2
6171 081/300 Write block 1762 len 40960 data: 2 2 2 2 2 2 2 2 2 2
6172 082/300 Read block 4806 len 24576
6173 083/300 Write block 927 len 4096 data: 2
6174 084/300 Write block 4070 len 12288 data: 2 2 2
6175 085/300 Read block 2552 len 4096
6176 086/300 Write block 1372 len 40960 data: 2 2 2 2 2 2 2 2 2 3
6177 087/300 Write block 3772 len 36864 data: 2 2 2 2 3 2 2 2 2
6178 088/300 Read block 2296 len 8192
6179 089/300 Write block 4084 len 40960 data: 3 3 3 4 3 3 4 4 4 4
6180 090/300 Read block 2320 len 8192
6181 091/300 Write block 984 len 36864 data: 2 2 2 2 2 2 2 2 2
6182 092/300 Read block 1938 len 32768
6183 093/300 Read block 3445 len 4096
6184 094/300 Write block 4862 len 28672 data: 4 3 3 2 2 2 2
6185 095/300 Write block 3632 len 32768 data: 3 4 3 3 3 4 4 4
6186 096/300 Read block 2106 len 28672
6187 097/300 Read block 2794 len 28672
6188 098/300 Flush
6189 099/300 Write block 1399 len 4096 data: 2
6190 100/300 Write block 4661 len 20480 data: 2 2 2 2 2
6191 101/300 Read block 1206 len 4096
6192 102/300 Read block 3896 len 36864
6193 103/300 Write block 472 len 4096 data: 2
6194 104/300 Write block 472 len 24576 data: 3 4 4 4 2 2
6195 105/300 Write block 4655 len 32768 data: 2 2 2 2 2 2 3 3
6196 106/300 Write block 3032 len 16384 data: 2 3 3 3
6197 107/300 Write block 2784 len 8192 data: 3 3
6198 108/300 Write block 4187 len 32768 data: 2 2 2 2 2 2 2 2
6199 109/300 Write block 691 len 28672 data: 3 3 2 2 2 2 2
6200 110/300 Read block 2891 len 16384
6201 111/300 Read block 3684 len 24576
6202 112/300 Read block 155 len 4096
6203 113/300 Read block 2525 len 4096
6204 114/300 Read block 1400 len 12288
6205 115/300 Write block 98 len 8192 data: 2 3
6206 116/300 Read block 732 len 16384
6207 117/300 Read block 3072 len 8192
6208 118/300 Read block 1565 len 20480
6209 119/300 Write block 1073 len 20480 data: 2 2 4 4 4
6210 120/300 Write block 2705 len 40960 data: 4 5 3 3 3 3 2 2 3 4
6211 121/300 Write block 3900 len 40960 data: 2 2 3 3 3 3 2 2 2 2
6212 122/300 Write block 3819 len 12288 data: 3 2 2
6213 123/300 Read block 128 len 12288
6214 124/300 Write block 405 len 40960 data: 2 2 2 2 2 2 2 2 2 2
6215 125/300 Read block 3521 len 32768
6216 126/300 Write block 1767 len 24576 data: 3 3 3 3 3 2
6217 127/300 Write block 651 len 36864 data: 3 3 3 3 3 4 5 5 7
6218 128/300 Write block 542 len 24576 data: 4 4 4 3 3 4
6219 129/300 Read block 2992 len 24576
6220 130/300 Read block 3441 len 28672
6221 131/300 Write block 3185 len 16384 data: 3 3 4 4
6222 132/300 Flush
6223 133/300 Read block 3167 len 12288
6224 134/300 Flush
6225 135/300 Read block 2971 len 4096
6226 136/300 Flush
6227 137/300 Read block 2113 len 24576
6228 138/300 Write block 494 len 28672 data: 2 2 2 2 2 2 2
6229 139/300 Read block 2880 len 24576
6230 140/300 Read block 1536 len 20480
6231 141/300 Write block 2439 len 28672 data: 4 5 4 3 3 2 2
6232 142/300 Write block 4534 len 12288 data: 2 2 2
6233 143/300 Read block 596 len 28672
6234 144/300 Write block 2769 len 20480 data: 2 2 2 2 3
6235 145/300 Read block 1410 len 12288
6236 146/300 Read block 2358 len 32768
6237 147/300 Write block 4094 len 20480 data: 4 4 5 5 4
6238 148/300 Read block 4951 len 16384
6239 149/300 Write block 2061 len 28672 data: 2 2 3 3 4 4 4
6240 150/300 Flush
6241 151/300 Write block 2093 len 24576 data: 3 3 2 2 2 2
6242 152/300 Read block 981 len 4096
6243 153/300 Read block 1826 len 8192
6244 154/300 Write block 3902 len 16384 data: 4 4 4 4
6245 155/300 Flush
6246 156/300 Read block 4518 len 8192
6247 157/300 Flush
6248 158/300 Write block 3798 len 24576 data: 4 3 3 3 3 3
6249 159/300 Write block 633 len 28672 data: 4 4 5 5 4 3 3
6250 160/300 Write block 2612 len 12288 data: 2 2 2
6251 161/300 Write block 4624 len 28672 data: 2 3 3 2 2 2 3
6252 162/300 Write block 2563 len 36864 data: 3 4 4 4 3 3 3 4 5
6253 163/300 Write block 1890 len 40960 data: 3 3 3 3 2 2 2 2 2 2
6254 164/300 Read block 4771 len 4096
6255 165/300 Write block 374 len 40960 data: 3 4 4 4 4 4 4 3 3 3
6256 166/300 Write block 4679 len 4096 data: 4
6257 167/300 Read block 3614 len 24576
6258 168/300 Read block 4172 len 28672
6259 169/300 Write block 1149 len 24576 data: 3 4 4 3 3 2
6260 170/300 Flush
6261 171/300 Read block 3470 len 4096
6262 172/300 Write block 4474 len 40960 data: 4 3 3 3 3 3 3 2 2 2
6263 173/300 Read block 104 len 16384
6264 174/300 Read block 426 len 12288
6265 175/300 Write block 3954 len 36864 data: 3 3 3 3 3 3 2 2 2
6266 176/300 Read block 569 len 4096
6267 177/300 Read block 3034 len 4096
6268 178/300 Read block 732 len 24576
6269 179/300 Read block 3622 len 12288
6270 180/300 Read block 4463 len 8192
6271 181/300 Read block 3545 len 12288
6272 182/300 Write block 3404 len 20480 data: 3 3 3 3 2
6273 183/300 Read block 982 len 16384
6274 184/300 Write block 48 len 36864 data: 5 6 4 4 5 4 4 4 4
6275 185/300 Read block 4842 len 16384
6276 186/300 Flush
6277 187/300 Read block 3495 len 24576
6278 188/300 Read block 959 len 8192
6279 189/300 Read block 3277 len 20480
6280 190/300 Write block 2476 len 40960 data: 4 2 2 2 2 2 2 2 2 2
6281 191/300 Read block 4986 len 28672
6282 192/300 Read block 1812 len 16384
6283 193/300 Read block 3973 len 40960
6284 194/300 Read block 4543 len 8192
6285 195/300 Flush
6286 196/300 Write block 3287 len 32768 data: 4 4 3 3 3 2 2 2
6287 197/300 Read block 815 len 40960
6288 198/300 Read block 3221 len 20480
6289 199/300 Read block 2795 len 28672
6290 200/300 Write block 2757 len 40960 data: 3 3 3 2 2 2 2 2 2 2
6291 201/300 Read block 3814 len 12288
6292 202/300 Read block 4781 len 8192
6293 203/300 Write block 217 len 20480 data: 2 2 2 2 2
6294 204/300 Read block 1710 len 12288
6295 205/300 Write block 4158 len 16384 data: 2 3 3 3
6296 206/300 Read block 124 len 40960
6297 207/300 Write block 3332 len 8192 data: 3 3
6298 208/300 Read block 988 len 12288
6299 209/300 Read block 2656 len 28672
6300 210/300 Read block 3467 len 20480
6301 211/300 Read block 3043 len 12288
6302 212/300 Read block 506 len 12288
6303 213/300 Read block 2309 len 12288
6304 214/300 Read block 1677 len 32768
6305 215/300 Read block 4638 len 8192
6306 216/300 Read block 3719 len 28672
6307 217/300 Read block 970 len 12288
6308 218/300 Write block 632 len 40960 data: 5 5 5 6 6 5 4 4 2 2
6309 219/300 Write block 1059 len 4096 data: 2
6310 220/300 Read block 3486 len 4096
6311 221/300 Write block 3370 len 8192 data: 3 3
6312 222/300 Write block 243 len 12288 data: 3 3 5
6313 223/300 Read block 833 len 40960
6314 224/300 Write block 4266 len 32768 data: 4 3 3 3 3 2 2 2
6315 225/300 Read block 4163 len 28672
6316 226/300 Read block 1246 len 12288
6317 227/300 Read block 4199 len 4096
6318 228/300 Flush
6319 229/300 Read block 1663 len 16384
6320 230/300 Write block 120 len 12288 data: 2 2 2
6321 231/300 Read block 992 len 20480
6322 232/300 Read block 1417 len 40960
6323 233/300 Write block 4851 len 20480 data: 5 4 3 3 3
6324 234/300 Write block 2775 len 8192 data: 3 3
6325 235/300 Read block 3205 len 28672
6326 236/300 Read block 4304 len 8192
6327 237/300 Read block 21 len 36864
6328 238/300 Flush
6329 239/300 Read block 2534 len 36864
6330 240/300 Read block 4084 len 4096
6331 241/300 Read block 1364 len 4096
6332 242/300 Read block 1346 len 16384
6333 243/300 Write block 4199 len 28672 data: 2 2 2 2 2 2 2
6334 244/300 Flush
6335 245/300 Read block 4970 len 20480
6336 246/300 Read block 326 len 36864
6337 247/300 Write block 3263 len 8192 data: 3 3
6338 248/300 Write block 1901 len 12288 data: 2 2 2
6339 249/300 Read block 742 len 36864
6340 250/300 Write block 690 len 20480 data: 3 4 4 3 3
6341 251/300 Write block 275 len 36864 data: 2 2 2 2 2 2 2 2 2
6342 252/300 Write block 4444 len 28672 data: 4 5 5 4 3 3 3
6343 253/300 Write block 1019 len 40960 data: 2 2 3 2 2 3 3 3 2 2
6344 254/300 Read block 4684 len 20480
6345 255/300 Read block 3772 len 32768
6346 256/300 Flush
6347 257/300 Write block 984 len 8192 data: 3 3
6348 258/300 Read block 579 len 24576
6349 259/300 Read block 2584 len 36864
6350 260/300 Write block 1814 len 28672 data: 2 2 2 2 2 2 3
6351 261/300 Read block 4793 len 16384
6352 262/300 Read block 4199 len 16384
6353 263/300 Write block 407 len 36864 data: 3 3 3 3 3 3 3 3 2
6354 264/300 Read block 2104 len 28672
6355 265/300 Write block 996 len 24576 data: 2 2 2 2 2 2
6356 266/300 Write block 3315 len 36864 data: 3 4 4 4 2 2 2 2 3
6357 267/300 Read block 3438 len 28672
6358 268/300 Read block 1626 len 12288
6359 269/300 Read block 1820 len 32768
6360 270/300 Read block 3192 len 8192
6361 271/300 Read block 2288 len 32768
6362 272/300 Write block 2587 len 8192 data: 3 2
6363 273/300 Read block 4342 len 12288
6364 274/300 Write block 4990 len 40960 data: 4 4 4 3 3 2 2 3 3 2
6365 275/300 Write block 4406 len 20480 data: 2 2 2 2 2
6366 276/300 Read block 2317 len 4096
6367 277/300 Read block 2518 len 40960
6368 278/300 Read block 2627 len 8192
6369 279/300 Read block 3361 len 28672
6370 280/300 Read block 2600 len 40960
6371 281/300 Flush
6372 282/300 Write block 2750 len 20480 data: 4 4 4 4 3
6373 283/300 Write block 4384 len 4096 data: 2
6374 284/300 Flush
6375 285/300 Read block 2830 len 40960
6376 286/300 Write block 25 len 36864 data: 3 2 2 2 2 2 2 2 2
6377 287/300 Write block 3447 len 20480 data: 2 2 3 4 4
6378 288/300 Write block 3476 len 20480 data: 3 3 3 3 3
6379 289/300 Read block 3184 len 28672
6380 290/300 Flush
6381 291/300 Read block 2523 len 36864
6382 292/300 Write block 3516 len 4096 data: 3
6383 293/300 Read block 2635 len 8192
6384 294/300 Read block 452 len 32768
6385 295/300 Flush
6386 296/300 Read block 1090 len 16384
6387 297/300 Flush
6388 298/300 Read block 731 len 20480
6389 299/300 Read block 466 len 40960
6390 300/300 Write block 160 len 20480 data: 2 3 3 4 4
6391 301/300 Write block 3416 len 16384 data: 3 3 3 3
6392 Main task has work to do, go find it
6393 got dsc Start(0)
6394 start 0
6395 Nov 28 04:51:40.692 INFO request completed, latency_us: 206, response_code: 204, uri: /start/cid/0, method: POST, req_id: 1ffe7c9b-af58-4a67-8744-c97af62b59e2, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
6396 Replay: started 0, returned:Ok(())
6397 ----------------------------------------------------------------
6398 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
6399 States: Offline Active Active
6400 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
6401 2812 Acked Write 9 Sent Done Done false
6402 2813 Acked Write 7 Sent Done Done false
6403 2814 Acked Read 9 Sent Done Done false
6404 2815 Acked Flush 0 Sent Done Done false
6405 2816 Acked Read 5 Sent Done Done false
6406 2817 Acked Read 1 Sent Done Done false
6407 2818 Acked Write 9 Sent Done Done false
6408 2819 Acked Read 4 Sent Done Done false
6409 2820 Acked Write 8 Sent Done Done false
6410 2821 Acked Read 8 Sent Done Done false
6411 2822 Acked Write 4 Sent Done Done false
6412 2823 Acked Write 5 Sent Done Done false
6413 2824 Acked Read 2 Sent Done Done false
6414 2825 Acked Read 10 Sent Done Done false
6415 2826 Acked Read 10 Sent Done Done false
6416 2827 Acked Write 2 Sent Done Done false
6417 2828 Acked Read 10 Sent Done Done false
6418 2829 Acked Write 10 Sent Done Done false
6419 2830 Acked Write 3 Sent Done Done false
6420 2831 Acked Read 7 Sent Done Done false
6421 2832 Acked Write 6 Sent Done Done false
6422 2833 Acked Write 9 Sent Done Done false
6423 2834 Acked Read 9 Sent Done Done false
6424 2835 Acked Write 3 Sent Done Done false
6425 2836 Acked Read 7 Sent Done Done false
6426 2837 Acked Read 5 Sent Done Done false
6427 2838 Acked Read 8 Sent Done Done false
6428 2839 Acked Read 4 Sent Done Done false
6429 2840 Acked Write 4 Sent Done Done false
6430 2841 Acked Read 1 Sent Done Done false
6431 2842 Acked Read 8 Sent Done Done false
6432 2843 Acked Write 5 Sent Done Done false
6433 2844 Acked Write 4 Sent Done Done false
6434 2845 Acked Read 5 Sent Done Done false
6435 2846 Acked Write 3 Sent Done Done false
6436 2847 Acked Read 2 Sent Done Done false
6437 2848 Acked Read 2 Sent Done Done false
6438 2849 Acked Read 8 Sent Done Done false
6439 2850 Acked Write 4 Sent Done Done false
6440 2851 Acked Read 9 Sent Done Done false
6441 2852 Acked Write 7 Sent Done Done false
6442 2853 Acked Flush 0 Sent Done Done false
6443 2854 Acked Read 4 Sent Done Done false
6444 2855 Acked Read 6 Sent Done Done false
6445 2856 Acked Read 4 Sent Done Done false
6446 2857 Acked Write 3 Sent Done Done false
6447 2858 Acked Write 6 Sent Done Done false
6448 2859 Acked Read 4 Sent Done Done false
6449 2860 Acked Read 6 Sent Done Done false
6450 2861 Acked Write 7 Sent Done Done false
6451 2862 Acked Read 5 Sent Done Done false
6452 2863 Acked Read 8 Sent Done Done false
6453 2864 Acked Write 6 Sent Done Done false
6454 2865 Acked Read 1 Sent Done Done false
6455 2866 Acked Write 5 Sent Done Done false
6456 2867 Acked Read 8 Sent Done Done false
6457 2868 Acked Read 5 Sent Done Done false
6458 2869 Acked Write 9 Sent Done Done false
6459 2870 Acked Read 7 Sent Done Done false
6460 2871 Acked Read 5 Sent Done Done false
6461 2872 Acked Write 10 Sent Done Done false
6462 2873 Acked Write 6 Sent Done Done false
6463 2874 Acked Write 5 Sent Done Done false
6464 2875 Acked Write 9 Sent Done Done false
6465 2876 Acked Flush 0 Sent Done Done false
6466 2877 Acked Flush 0 Sent Done Done false
6467 2878 Acked Write 5 Sent Done Done false
6468 2879 Acked Write 2 Sent Done Done false
6469 2880 Acked Write 8 Sent Done Done false
6470 2881 Acked Write 2 Sent Done Done false
6471 2882 Acked Read 8 Sent Done Done false
6472 2883 Acked Read 10 Sent Done Done false
6473 2884 Acked Write 10 Sent Done Done false
6474 2885 Acked Read 6 Sent Done Done false
6475 2886 Acked Read 4 Sent Done Done false
6476 2887 Acked Read 8 Sent Done Done false
6477 2888 Acked Write 9 Sent Done Done false
6478 2889 Acked Write 5 Sent Done Done false
6479 2890 Acked Write 2 Sent Done Done false
6480 2891 Acked Write 2 Sent Done Done false
6481 2892 Acked Write 10 Sent Done Done false
6482 2893 Acked Read 6 Sent Done Done false
6483 2894 Acked Write 1 Sent Done Done false
6484 2895 Acked Write 3 Sent Done Done false
6485 2896 Acked Read 1 Sent Done Done false
6486 2897 Acked Write 10 Sent Done Done false
6487 2898 Acked Write 9 Sent Done Done false
6488 2899 Acked Read 2 Sent Done Done false
6489 2900 Acked Write 10 Sent Done Done false
6490 2901 Acked Read 2 Sent Done Done false
6491 2902 Acked Write 9 Sent Done Done false
6492 2903 Acked Read 8 Sent Done Done false
6493 2904 Acked Read 1 Sent Done Done false
6494 2905 Acked Write 7 Sent Done Done false
6495 2906 Acked Write 8 Sent Done Done false
6496 2907 Acked Read 7 Sent Done Done false
6497 2908 Acked Read 7 Sent Done Done false
6498 2909 Acked Flush 0 Sent Done Done false
6499 2910 Acked Write 1 Sent Done Done false
6500 2911 Acked Write 5 Sent Done Done false
6501 2912 Acked Read 1 Sent Done Done false
6502 2913 Acked Read 9 Sent Done Done false
6503 2914 Acked Write 1 Sent Done Done false
6504 2915 Acked Write 6 Sent Done Done false
6505 2916 Acked Write 8 Sent Done Done false
6506 2917 Acked Write 4 Sent Done Done false
6507 2918 Acked Write 2 Sent Done Done false
6508 2919 Acked Write 8 Sent Done Done false
6509 2920 Acked Write 7 Sent Done Done false
6510 2921 Acked Read 4 Sent Done Done false
6511 2922 Acked Read 6 Sent Done Done false
6512 2923 Acked Read 1 Sent Done Done false
6513 2924 Acked Read 1 Sent Done Done false
6514 2925 Acked Read 3 Sent Done Done false
6515 2926 Acked Write 2 Sent Done Done false
6516 2927 Acked Read 4 Sent Done Done false
6517 2928 Acked Read 2 Sent Done Done false
6518 2929 Acked Read 5 Sent Done Done false
6519 2930 Acked Write 5 Sent Done Done false
6520 2931 Acked Write 10 Sent Done Done false
6521 2932 Acked Write 10 Sent Done Done false
6522 2933 Acked Write 3 Sent Done Done false
6523 2934 Acked Read 3 Sent Done Done false
6524 2935 Acked Write 10 Sent Done Done false
6525 2936 Acked Read 8 Sent Done Done false
6526 2937 Acked Write 6 Sent Done Done false
6527 2938 Acked Write 9 Sent Done Done false
6528 2939 Acked Write 6 Sent Done Done false
6529 2940 Acked Read 6 Sent Done Done false
6530 2941 Acked Read 7 Sent Done Done false
6531 2942 Acked Write 4 Sent Done Done false
6532 2943 Acked Flush 0 Sent Done Done false
6533 2944 Acked Read 3 Sent Done Done false
6534 2945 Acked Flush 0 Sent Done Done false
6535 2946 Acked Read 1 Sent Done Done false
6536 2947 Acked Flush 0 Sent Done Done false
6537 2948 Acked Read 6 Sent Done Done false
6538 2949 Acked Write 7 Sent Done Done false
6539 2950 Acked Read 6 Sent Done Done false
6540 2951 Acked Read 5 Sent Done Done false
6541 2952 Acked Write 7 Sent Done Done false
6542 2953 Acked Write 3 Sent Done Done false
6543 2954 Acked Read 7 Sent Done Done false
6544 2955 Acked Write 5 Sent Done Done false
6545 2956 Acked Read 3 Sent Done Done false
6546 2957 Acked Read 8 Sent Done Done false
6547 2958 Acked Write 5 Sent Done Done false
6548 2959 Acked Read 4 Sent Done Done false
6549 2960 Acked Write 7 Sent Done Done false
6550 2961 Acked Flush 0 Sent Done Done false
6551 2962 Acked Write 6 Sent Done Done false
6552 2963 Acked Read 1 Sent Done Done false
6553 2964 Acked Read 2 Sent Done Done false
6554 2965 Acked Write 4 Sent Done Done false
6555 2966 Acked Flush 0 Sent Done Done false
6556 2967 Acked Read 2 Sent Done Done false
6557 2968 Acked Flush 0 Sent Done Done false
6558 2969 Acked Write 6 Sent Done Done false
6559 2970 Acked Write 7 Sent Done Done false
6560 2971 Acked Write 3 Sent Done Done false
6561 2972 Acked Write 7 Sent Done Done false
6562 2973 Acked Write 9 Sent Done Done false
6563 2974 Acked Write 10 Sent Done Done false
6564 2975 Acked Read 1 Sent Done Done false
6565 2976 Acked Write 10 Sent Done Done false
6566 2977 Acked Write 1 Sent Done Done false
6567 2978 Acked Read 6 Sent Done Done false
6568 2979 Acked Read 7 Sent Done Done false
6569 2980 Acked Write 6 Sent Done Done false
6570 2981 Acked Flush 0 Sent Done Done false
6571 2982 Acked Read 1 Sent Done Done false
6572 2983 Acked Write 10 Sent Done Done false
6573 2984 Acked Read 4 Sent Done Done false
6574 2985 Acked Read 3 Sent Done Done false
6575 2986 Acked Write 9 Sent Done Done false
6576 2987 Acked Read 1 Sent Done Done false
6577 2988 Acked Read 1 Sent Done Done false
6578 2989 Acked Read 6 Sent Done Done false
6579 2990 Acked Read 3 Sent Done Done false
6580 2991 Acked Read 2 Sent Done Done false
6581 2992 Acked Read 3 Sent Done Done false
6582 2993 Acked Write 5 Sent Done Done false
6583 2994 Acked Read 4 Sent Done Done false
6584 2995 Acked Write 9 Sent Done Done false
6585 2996 Acked Read 4 Sent Done Done false
6586 2997 Acked Flush 0 Sent Done Done false
6587 2998 Acked Read 6 Sent Done Done false
6588 2999 Acked Read 2 Sent Done Done false
6589 3000 Acked Read 5 Sent Done Done false
6590 3001 Acked Write 10 Sent Done Done false
6591 3002 Acked Read 7 Sent Done Done false
6592 3003 Acked Read 4 Sent Done Done false
6593 3004 Acked Read 10 Sent Done Done false
6594 3005 Acked Read 2 Sent Done Done false
6595 3006 Acked Flush 0 Sent Done Done false
6596 3007 Acked Write 8 Sent Done Done false
6597 3008 Acked Read 10 Sent Done Done false
6598 3009 Acked Read 5 Sent Done Done false
6599 3010 Acked Read 7 Sent Done Done false
6600 3011 Acked Write 10 Sent Done Done false
6601 3012 Acked Read 3 Sent Done Done false
6602 3013 Acked Read 2 Sent Done Done false
6603 3014 Acked Write 5 Sent Done Done false
6604 3015 Acked Read 3 Sent Done Done false
6605 3016 Acked Write 4 Sent Done Done false
6606 3017 Acked Read 10 Sent Done Done false
6607 3018 Acked Write 2 Sent Done Done false
6608 3019 Acked Read 3 Sent Done Done false
6609 3020 Acked Read 7 Sent Done Done false
6610 3021 Acked Read 5 Sent Done Done false
6611 3022 Acked Read 3 Sent Done Done false
6612 3023 Acked Read 3 Sent Done Done false
6613 3024 Acked Read 3 Sent Done Done false
6614 3025 Acked Read 8 Sent Done Done false
6615 3026 Acked Read 2 Sent Done Done false
6616 3027 Acked Read 7 Sent Done Done false
6617 3028 Acked Read 3 Sent Done Done false
6618 3029 Acked Write 10 Sent Done Done false
6619 3030 Acked Write 1 Sent Done Done false
6620 3031 Acked Read 1 Sent Done Done false
6621 3032 Acked Write 2 Sent Done Done false
6622 3033 Acked Write 3 Sent Done Done false
6623 3034 Acked Read 10 Sent Done Done false
6624 3035 Acked Write 8 Sent Done Done false
6625 3036 Acked Read 7 Sent Done Done false
6626 3037 Acked Read 3 Sent Done Done false
6627 3038 Acked Read 1 Sent Done Done false
6628 3039 Acked Flush 0 Sent Done Done false
6629 3040 Acked Read 4 Sent Done Done false
6630 3041 Acked Write 3 Sent Done Done false
6631 3042 Acked Read 5 Sent Done Done false
6632 3043 Acked Read 10 Sent Done Done false
6633 3044 Acked Write 5 Sent Done Done false
6634 3045 Acked Write 2 Sent Done Done false
6635 3046 Acked Read 7 Sent Done Done false
6636 3047 Acked Read 2 Sent Done Done false
6637 3048 Acked Read 9 Sent Done Done false
6638 3049 Acked Flush 0 Sent Done Done false
6639 3050 Acked Read 9 Sent Done Done false
6640 3051 Acked Read 1 Sent Done Done false
6641 3052 Acked Read 1 Sent Done Done false
6642 3053 Acked Read 4 Sent Done Done false
6643 3054 Acked Write 7 Sent Done Done false
6644 3055 Acked Flush 0 Sent Done Done false
6645 3056 Acked Read 5 Sent Done Done false
6646 3057 Acked Read 9 Sent Done Done false
6647 3058 Acked Write 2 Sent Done Done false
6648 3059 Acked Write 3 Sent Done Done false
6649 3060 Acked Read 9 Sent Done Done false
6650 3061 Acked Write 5 Sent Done Done false
6651 3062 Acked Write 9 Sent Done Done false
6652 3063 Acked Write 7 Sent Done Done false
6653 3064 Acked Write 10 Sent Done Done false
6654 3065 Acked Read 5 Sent Done Done false
6655 3066 Acked Read 8 Sent Done Done false
6656 3067 Acked Flush 0 Sent Done Done false
6657 3068 Acked Write 2 Sent Done Done false
6658 3069 Acked Read 6 Sent Done Done false
6659 3070 Acked Read 9 Sent Done Done false
6660 3071 Acked Write 7 Sent Done Done false
6661 3072 Acked Read 4 Sent Done Done false
6662 3073 Acked Read 4 Sent Done Done false
6663 3074 Acked Write 9 Sent Done Done false
6664 3075 Acked Read 7 Sent Done Done false
6665 3076 Acked Write 6 Sent Done Done false
6666 3077 Acked Write 9 Sent Done Done false
6667 3078 Acked Read 7 Sent Done Done false
6668 3079 Acked Read 3 Sent Done Done false
6669 3080 Acked Read 8 Sent Done Done false
6670 3081 Acked Read 2 Sent Done Done false
6671 3082 Acked Read 8 Sent Done Done false
6672 3083 Acked Write 2 Sent Done Done false
6673 3084 Acked Read 3 Sent Done Done false
6674 3085 Acked Write 10 Sent Done Done false
6675 3086 Acked Write 5 Sent Done Done false
6676 3087 Acked Read 1 Sent Done Done false
6677 3088 Acked Read 10 Sent Done Done false
6678 3089 Acked Read 2 Sent Done Done false
6679 3090 Acked Read 7 Sent Done Done false
6680 3091 Acked Read 10 Sent Done Done false
6681 3092 Acked Flush 0 Sent Done Done false
6682 3093 Acked Write 5 Sent Done Done false
6683 3094 Acked Write 1 Sent Done Done false
6684 3095 Acked Flush 0 Sent Done Done false
6685 3096 Acked Read 10 Sent Done Done false
6686 3097 Acked Write 9 Sent Done Done false
6687 3098 Acked Write 5 Sent Done Done false
6688 3099 Acked Write 5 Sent Done Done false
6689 3100 Acked Read 7 Sent Done Done false
6690 3101 Acked Flush 0 Sent Done Done false
6691 3102 Acked Read 9 Sent Done Done false
6692 3103 Acked Write 1 Sent Done Done false
6693 3104 Acked Read 2 Sent Done Done false
6694 3105 Acked Read 8 Sent Done Done false
6695 3106 Acked Flush 0 Sent Done Done false
6696 3107 Acked Read 4 Sent Done Done false
6697 3108 Acked Flush 0 Sent Done Done false
6698 3109 Acked Read 5 Sent Done Done false
6699 3110 Acked Read 10 Sent Done Done false
6700 3111 Acked Write 5 Sent Done Done false
6701 3112 Acked Write 4 Sent Done Done false
6702 STATES DS:0 DS:1 DS:2 TOTAL
6703 Sent 301 0 0 301
6704 Done 0 301 301 602
6705 Skipped 0 0 0 0
6706 Error 0 0 0 0
6707 Last Flush: 2811 3108 3108
6708 Downstairs last five completed: 2811 2810 2809 2808 2807
6709 Upstairs last five completed: 3112 3111 3110 3109 3108
6710 CLIENT: Up:0 ds:301 act:2
6711 [8810] got start action
6712 [8810] I am going to restart
6713 Starting downstairs at port 8810
6714 Make output file at "/tmp/dsc/downstairs-8810.txt"
6715 [8810][0] reports Starting
6716 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1209)
6717 [8810][0] reports Running
6718 ----------------------------------------------------------------
6719 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
6720 States: Offline Active Active
6721 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
6722 2812 Acked Write 9 Sent Done Done false
6723 2813 Acked Write 7 Sent Done Done false
6724 2814 Acked Read 9 Sent Done Done false
6725 2815 Acked Flush 0 Sent Done Done false
6726 2816 Acked Read 5 Sent Done Done false
6727 2817 Acked Read 1 Sent Done Done false
6728 2818 Acked Write 9 Sent Done Done false
6729 2819 Acked Read 4 Sent Done Done false
6730 2820 Acked Write 8 Sent Done Done false
6731 2821 Acked Read 8 Sent Done Done false
6732 2822 Acked Write 4 Sent Done Done false
6733 2823 Acked Write 5 Sent Done Done false
6734 2824 Acked Read 2 Sent Done Done false
6735 2825 Acked Read 10 Sent Done Done false
6736 2826 Acked Read 10 Sent Done Done false
6737 2827 Acked Write 2 Sent Done Done false
6738 2828 Acked Read 10 Sent Done Done false
6739 2829 Acked Write 10 Sent Done Done false
6740 2830 Acked Write 3 Sent Done Done false
6741 2831 Acked Read 7 Sent Done Done false
6742 2832 Acked Write 6 Sent Done Done false
6743 2833 Acked Write 9 Sent Done Done false
6744 2834 Acked Read 9 Sent Done Done false
6745 2835 Acked Write 3 Sent Done Done false
6746 2836 Acked Read 7 Sent Done Done false
6747 2837 Acked Read 5 Sent Done Done false
6748 2838 Acked Read 8 Sent Done Done false
6749 2839 Acked Read 4 Sent Done Done false
6750 2840 Acked Write 4 Sent Done Done false
6751 2841 Acked Read 1 Sent Done Done false
6752 2842 Acked Read 8 Sent Done Done false
6753 2843 Acked Write 5 Sent Done Done false
6754 2844 Acked Write 4 Sent Done Done false
6755 2845 Acked Read 5 Sent Done Done false
6756 2846 Acked Write 3 Sent Done Done false
6757 2847 Acked Read 2 Sent Done Done false
6758 2848 Acked Read 2 Sent Done Done false
6759 2849 Acked Read 8 Sent Done Done false
6760 2850 Acked Write 4 Sent Done Done false
6761 2851 Acked Read 9 Sent Done Done false
6762 2852 Acked Write 7 Sent Done Done false
6763 2853 Acked Flush 0 Sent Done Done false
6764 2854 Acked Read 4 Sent Done Done false
6765 2855 Acked Read 6 Sent Done Done false
6766 2856 Acked Read 4 Sent Done Done false
6767 2857 Acked Write 3 Sent Done Done false
6768 2858 Acked Write 6 Sent Done Done false
6769 2859 Acked Read 4 Sent Done Done false
6770 2860 Acked Read 6 Sent Done Done false
6771 2861 Acked Write 7 Sent Done Done false
6772 2862 Acked Read 5 Sent Done Done false
6773 2863 Acked Read 8 Sent Done Done false
6774 2864 Acked Write 6 Sent Done Done false
6775 2865 Acked Read 1 Sent Done Done false
6776 2866 Acked Write 5 Sent Done Done false
6777 2867 Acked Read 8 Sent Done Done false
6778 2868 Acked Read 5 Sent Done Done false
6779 2869 Acked Write 9 Sent Done Done false
6780 2870 Acked Read 7 Sent Done Done false
6781 2871 Acked Read 5 Sent Done Done false
6782 2872 Acked Write 10 Sent Done Done false
6783 2873 Acked Write 6 Sent Done Done false
6784 2874 Acked Write 5 Sent Done Done false
6785 2875 Acked Write 9 Sent Done Done false
6786 2876 Acked Flush 0 Sent Done Done false
6787 2877 Acked Flush 0 Sent Done Done false
6788 2878 Acked Write 5 Sent Done Done false
6789 2879 Acked Write 2 Sent Done Done false
6790 2880 Acked Write 8 Sent Done Done false
6791 2881 Acked Write 2 Sent Done Done false
6792 2882 Acked Read 8 Sent Done Done false
6793 2883 Acked Read 10 Sent Done Done false
6794 2884 Acked Write 10 Sent Done Done false
6795 2885 Acked Read 6 Sent Done Done false
6796 2886 Acked Read 4 Sent Done Done false
6797 2887 Acked Read 8 Sent Done Done false
6798 2888 Acked Write 9 Sent Done Done false
6799 2889 Acked Write 5 Sent Done Done false
6800 2890 Acked Write 2 Sent Done Done false
6801 2891 Acked Write 2 Sent Done Done false
6802 2892 Acked Write 10 Sent Done Done false
6803 2893 Acked Read 6 Sent Done Done false
6804 2894 Acked Write 1 Sent Done Done false
6805 2895 Acked Write 3 Sent Done Done false
6806 2896 Acked Read 1 Sent Done Done false
6807 2897 Acked Write 10 Sent Done Done false
6808 2898 Acked Write 9 Sent Done Done false
6809 2899 Acked Read 2 Sent Done Done false
6810 2900 Acked Write 10 Sent Done Done false
6811 2901 Acked Read 2 Sent Done Done false
6812 2902 Acked Write 9 Sent Done Done false
6813 2903 Acked Read 8 Sent Done Done false
6814 2904 Acked Read 1 Sent Done Done false
6815 2905 Acked Write 7 Sent Done Done false
6816 2906 Acked Write 8 Sent Done Done false
6817 2907 Acked Read 7 Sent Done Done false
6818 2908 Acked Read 7 Sent Done Done false
6819 2909 Acked Flush 0 Sent Done Done false
6820 2910 Acked Write 1 Sent Done Done false
6821 2911 Acked Write 5 Sent Done Done false
6822 2912 Acked Read 1 Sent Done Done false
6823 2913 Acked Read 9 Sent Done Done false
6824 2914 Acked Write 1 Sent Done Done false
6825 2915 Acked Write 6 Sent Done Done false
6826 2916 Acked Write 8 Sent Done Done false
6827 2917 Acked Write 4 Sent Done Done false
6828 2918 Acked Write 2 Sent Done Done false
6829 2919 Acked Write 8 Sent Done Done false
6830 2920 Acked Write 7 Sent Done Done false
6831 2921 Acked Read 4 Sent Done Done false
6832 2922 Acked Read 6 Sent Done Done false
6833 2923 Acked Read 1 Sent Done Done false
6834 2924 Acked Read 1 Sent Done Done false
6835 2925 Acked Read 3 Sent Done Done false
6836 2926 Acked Write 2 Sent Done Done false
6837 2927 Acked Read 4 Sent Done Done false
6838 2928 Acked Read 2 Sent Done Done false
6839 2929 Acked Read 5 Sent Done Done false
6840 2930 Acked Write 5 Sent Done Done false
6841 2931 Acked Write 10 Sent Done Done false
6842 2932 Acked Write 10 Sent Done Done false
6843 2933 Acked Write 3 Sent Done Done false
6844 2934 Acked Read 3 Sent Done Done false
6845 2935 Acked Write 10 Sent Done Done false
6846 2936 Acked Read 8 Sent Done Done false
6847 2937 Acked Write 6 Sent Done Done false
6848 2938 Acked Write 9 Sent Done Done false
6849 2939 Acked Write 6 Sent Done Done false
6850 2940 Acked Read 6 Sent Done Done false
6851 2941 Acked Read 7 Sent Done Done false
6852 2942 Acked Write 4 Sent Done Done false
6853 2943 Acked Flush 0 Sent Done Done false
6854 2944 Acked Read 3 Sent Done Done false
6855 2945 Acked Flush 0 Sent Done Done false
6856 2946 Acked Read 1 Sent Done Done false
6857 2947 Acked Flush 0 Sent Done Done false
6858 2948 Acked Read 6 Sent Done Done false
6859 2949 Acked Write 7 Sent Done Done false
6860 2950 Acked Read 6 Sent Done Done false
6861 2951 Acked Read 5 Sent Done Done false
6862 2952 Acked Write 7 Sent Done Done false
6863 2953 Acked Write 3 Sent Done Done false
6864 2954 Acked Read 7 Sent Done Done false
6865 2955 Acked Write 5 Sent Done Done false
6866 2956 Acked Read 3 Sent Done Done false
6867 2957 Acked Read 8 Sent Done Done false
6868 2958 Acked Write 5 Sent Done Done false
6869 2959 Acked Read 4 Sent Done Done false
6870 2960 Acked Write 7 Sent Done Done false
6871 2961 Acked Flush 0 Sent Done Done false
6872 2962 Acked Write 6 Sent Done Done false
6873 2963 Acked Read 1 Sent Done Done false
6874 2964 Acked Read 2 Sent Done Done false
6875 2965 Acked Write 4 Sent Done Done false
6876 2966 Acked Flush 0 Sent Done Done false
6877 2967 Acked Read 2 Sent Done Done false
6878 2968 Acked Flush 0 Sent Done Done false
6879 2969 Acked Write 6 Sent Done Done false
6880 2970 Acked Write 7 Sent Done Done false
6881 2971 Acked Write 3 Sent Done Done false
6882 2972 Acked Write 7 Sent Done Done false
6883 2973 Acked Write 9 Sent Done Done false
6884 2974 Acked Write 10 Sent Done Done false
6885 2975 Acked Read 1 Sent Done Done false
6886 2976 Acked Write 10 Sent Done Done false
6887 2977 Acked Write 1 Sent Done Done false
6888 2978 Acked Read 6 Sent Done Done false
6889 2979 Acked Read 7 Sent Done Done false
6890 2980 Acked Write 6 Sent Done Done false
6891 2981 Acked Flush 0 Sent Done Done false
6892 2982 Acked Read 1 Sent Done Done false
6893 2983 Acked Write 10 Sent Done Done false
6894 2984 Acked Read 4 Sent Done Done false
6895 2985 Acked Read 3 Sent Done Done false
6896 2986 Acked Write 9 Sent Done Done false
6897 2987 Acked Read 1 Sent Done Done false
6898 2988 Acked Read 1 Sent Done Done false
6899 2989 Acked Read 6 Sent Done Done false
6900 2990 Acked Read 3 Sent Done Done false
6901 2991 Acked Read 2 Sent Done Done false
6902 2992 Acked Read 3 Sent Done Done false
6903 2993 Acked Write 5 Sent Done Done false
6904 2994 Acked Read 4 Sent Done Done false
6905 2995 Acked Write 9 Sent Done Done false
6906 2996 Acked Read 4 Sent Done Done false
6907 2997 Acked Flush 0 Sent Done Done false
6908 2998 Acked Read 6 Sent Done Done false
6909 2999 Acked Read 2 Sent Done Done false
6910 3000 Acked Read 5 Sent Done Done false
6911 3001 Acked Write 10 Sent Done Done false
6912 3002 Acked Read 7 Sent Done Done false
6913 3003 Acked Read 4 Sent Done Done false
6914 3004 Acked Read 10 Sent Done Done false
6915 3005 Acked Read 2 Sent Done Done false
6916 3006 Acked Flush 0 Sent Done Done false
6917 3007 Acked Write 8 Sent Done Done false
6918 3008 Acked Read 10 Sent Done Done false
6919 3009 Acked Read 5 Sent Done Done false
6920 3010 Acked Read 7 Sent Done Done false
6921 3011 Acked Write 10 Sent Done Done false
6922 3012 Acked Read 3 Sent Done Done false
6923 3013 Acked Read 2 Sent Done Done false
6924 3014 Acked Write 5 Sent Done Done false
6925 3015 Acked Read 3 Sent Done Done false
6926 3016 Acked Write 4 Sent Done Done false
6927 3017 Acked Read 10 Sent Done Done false
6928 3018 Acked Write 2 Sent Done Done false
6929 3019 Acked Read 3 Sent Done Done false
6930 3020 Acked Read 7 Sent Done Done false
6931 3021 Acked Read 5 Sent Done Done false
6932 3022 Acked Read 3 Sent Done Done false
6933 3023 Acked Read 3 Sent Done Done false
6934 3024 Acked Read 3 Sent Done Done false
6935 3025 Acked Read 8 Sent Done Done false
6936 3026 Acked Read 2 Sent Done Done false
6937 3027 Acked Read 7 Sent Done Done false
6938 3028 Acked Read 3 Sent Done Done false
6939 3029 Acked Write 10 Sent Done Done false
6940 3030 Acked Write 1 Sent Done Done false
6941 3031 Acked Read 1 Sent Done Done false
6942 3032 Acked Write 2 Sent Done Done false
6943 3033 Acked Write 3 Sent Done Done false
6944 3034 Acked Read 10 Sent Done Done false
6945 3035 Acked Write 8 Sent Done Done false
6946 3036 Acked Read 7 Sent Done Done false
6947 3037 Acked Read 3 Sent Done Done false
6948 3038 Acked Read 1 Sent Done Done false
6949 3039 Acked Flush 0 Sent Done Done false
6950 3040 Acked Read 4 Sent Done Done false
6951 3041 Acked Write 3 Sent Done Done false
6952 3042 Acked Read 5 Sent Done Done false
6953 3043 Acked Read 10 Sent Done Done false
6954 3044 Acked Write 5 Sent Done Done false
6955 3045 Acked Write 2 Sent Done Done false
6956 3046 Acked Read 7 Sent Done Done false
6957 3047 Acked Read 2 Sent Done Done false
6958 3048 Acked Read 9 Sent Done Done false
6959 3049 Acked Flush 0 Sent Done Done false
6960 3050 Acked Read 9 Sent Done Done false
6961 3051 Acked Read 1 Sent Done Done false
6962 3052 Acked Read 1 Sent Done Done false
6963 3053 Acked Read 4 Sent Done Done false
6964 3054 Acked Write 7 Sent Done Done false
6965 3055 Acked Flush 0 Sent Done Done false
6966 3056 Acked Read 5 Sent Done Done false
6967 3057 Acked Read 9 Sent Done Done false
6968 3058 Acked Write 2 Sent Done Done false
6969 3059 Acked Write 3 Sent Done Done false
6970 3060 Acked Read 9 Sent Done Done false
6971 3061 Acked Write 5 Sent Done Done false
6972 3062 Acked Write 9 Sent Done Done false
6973 3063 Acked Write 7 Sent Done Done false
6974 3064 Acked Write 10 Sent Done Done false
6975 3065 Acked Read 5 Sent Done Done false
6976 3066 Acked Read 8 Sent Done Done false
6977 3067 Acked Flush 0 Sent Done Done false
6978 3068 Acked Write 2 Sent Done Done false
6979 3069 Acked Read 6 Sent Done Done false
6980 3070 Acked Read 9 Sent Done Done false
6981 3071 Acked Write 7 Sent Done Done false
6982 3072 Acked Read 4 Sent Done Done false
6983 3073 Acked Read 4 Sent Done Done false
6984 3074 Acked Write 9 Sent Done Done false
6985 3075 Acked Read 7 Sent Done Done false
6986 3076 Acked Write 6 Sent Done Done false
6987 3077 Acked Write 9 Sent Done Done false
6988 3078 Acked Read 7 Sent Done Done false
6989 3079 Acked Read 3 Sent Done Done false
6990 3080 Acked Read 8 Sent Done Done false
6991 3081 Acked Read 2 Sent Done Done false
6992 3082 Acked Read 8 Sent Done Done false
6993 3083 Acked Write 2 Sent Done Done false
6994 3084 Acked Read 3 Sent Done Done false
6995 3085 Acked Write 10 Sent Done Done false
6996 3086 Acked Write 5 Sent Done Done false
6997 3087 Acked Read 1 Sent Done Done false
6998 3088 Acked Read 10 Sent Done Done false
6999 3089 Acked Read 2 Sent Done Done false
7000 3090 Acked Read 7 Sent Done Done false
7001 3091 Acked Read 10 Sent Done Done false
7002 3092 Acked Flush 0 Sent Done Done false
7003 3093 Acked Write 5 Sent Done Done false
7004 3094 Acked Write 1 Sent Done Done false
7005 3095 Acked Flush 0 Sent Done Done false
7006 3096 Acked Read 10 Sent Done Done false
7007 3097 Acked Write 9 Sent Done Done false
7008 3098 Acked Write 5 Sent Done Done false
7009 3099 Acked Write 5 Sent Done Done false
7010 3100 Acked Read 7 Sent Done Done false
7011 3101 Acked Flush 0 Sent Done Done false
7012 3102 Acked Read 9 Sent Done Done false
7013 3103 Acked Write 1 Sent Done Done false
7014 3104 Acked Read 2 Sent Done Done false
7015 3105 Acked Read 8 Sent Done Done false
7016 3106 Acked Flush 0 Sent Done Done false
7017 3107 Acked Read 4 Sent Done Done false
7018 3108 Acked Flush 0 Sent Done Done false
7019 3109 Acked Read 5 Sent Done Done false
7020 3110 Acked Read 10 Sent Done Done false
7021 3111 Acked Write 5 Sent Done Done false
7022 3112 Acked Write 4 Sent Done Done false
7023 3113 Acked Flush 0 Sent Done Done false
7024 STATES DS:0 DS:1 DS:2 TOTAL
7025 Sent 302 0 0 302
7026 Done 0 302 302 604
7027 Skipped 0 0 0 0
7028 Error 0 0 0 0
7029 Last Flush: 2811 3113 3113
7030 Downstairs last five completed: 2811 2810 2809 2808 2807
7031 Upstairs last five completed: 3113 3112 3111 3110 3109
7032 CLIENT: Up:0 ds:302 act:2
70332024-11-28T04:51:46.459ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
70342024-11-28T04:51:46.459ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:45847) = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
70352024-11-28T04:51:46.460ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
70362024-11-28T04:51:46.460ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
70372024-11-28T04:51:46.461ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
70382024-11-28T04:51:46.461ZINFOcrucible: send last flush ID to this DS: 2811 = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
70392024-11-28T04:51:46.461ZINFOcrucible: Replied this last flush ID: 2811 = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
70402024-11-28T04:51:46.461ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
70412024-11-28T04:51:46.461ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
70422024-11-28T04:51:46.461ZINFOcrucible: [0] client re-new 302 jobs since flush 2811 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
70432024-11-28T04:51:46.461ZINFOcrucible: [0] Replayed 302 jobs since flush: 2811 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
7044 ----------------------------------------------------------------
7045 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
7046 Downstairs last five completed: 3113 3112 3111 3110 3109
7047 Upstairs last five completed: 3113 3112 3111 3110 3109
7048 CLIENT: Up:0 ds:0 act:3
7049 Replay: All jobs finished
7050 Main task has work to do, go find it
7051 got dsc Stop(2)
7052 stop 2
7053 Nov 28 04:51:48.696 INFO request completed, [8830] Got stop action so:false kr:false
7054 latency_us: 342, response_code: 204, uri: /stop/cid/2, method: POST, req_id: f7869cc9-1d99-4f10-a07f-5bc9e6612600, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
70552024-11-28T04:51:48.697ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
70562024-11-28T04:51:48.697ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
7057 Nov 28 04:51:48.697 INFO request completed, {"msg":"downstairs task for 2 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2024-11-28T04:51:48.697389487Z","hostname":"w-01JDRHKEEFJ1228TFPSBEGXWHY","pid":1202,"session_id":"4a2415b1-6594-436f-82e5-b8cf9a74eeaf"}
7058 latency_us: 111{"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-28T04:51:48.697417723Z","hostname":"w-01JDRHKEEFJ1228TFPSBEGXWHY","pid":1202,"client":"2","":"downstairs","session_id":"4a2415b1-6594-436f-82e5-b8cf9a74eeaf"}
7059 , response_code: 200, uri: /state/cid/2, method: GET, req_id: 2a408550-0f44-406b-8274-c273b67fb3c4, remote_addr: 127.{"msg":"[2] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-28T04:51:48.69743432Z","hostname":"w-01JDRHKEEFJ1228TFPSBEGXWHY","pid":1202,"client":"2","":"downstairs","session_id":"4a2415b1-6594-436f-82e5-b8cf9a74eeaf"}
7060 0.0.1:60418, local_addr: 127.0.0.1:9998
70612024-11-28T04:51:48.697ZINFOcrucible: client task is exiting = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
7062 [8830] Exited with: ExitStatus(unix_wait_status(9))
7063 [8830][2] reports Exit
7064 Nov 28 04:51:52.699 INFO request completed, latency_us: 198, response_code: 200, uri: /state/cid/2, method: GET, req_id: 6fa59cd6-9859-4fa4-885f-e94231ee7941, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
7065 001/300 Write block 1342 len 12288 data: 4 4 4
7066 002/300 Flush
7067 003/300 Write block 2028 len 28672 data: 3 3 3 3 4 4 5
7068 004/300 Read block 1480 len 40960
7069 005/300 Write block 1398 len 40960 data: 2 3 2 3 3 2 2 3 3 2
7070 006/300 Flush
7071 007/300 Write block 2787 len 36864 data: 4 5 5 5 3 4 2 2 2
7072 008/300 Write block 2615 len 4096 data: 2
7073 009/300 Read block 2873 len 24576
7074 010/300 Read block 3266 len 36864
7075 011/300 Write block 1706 len 32768 data: 5 6 7 8 7 6 7 6
7076 012/300 Write block 9 len 40960 data: 5 5 5 3 3 3 2 3 3 3
7077 013/300 Read block 2167 len 32768
7078 014/300 Flush
7079 015/300 Read block 2414 len 32768
7080 016/300 Write block 1079 len 4096 data: 5
7081 017/300 Read block 3359 len 20480
7082 018/300 Flush
7083 019/300 Write block 757 len 36864 data: 3 3 4 4 4 4 3 3 3
7084 020/300 Read block 2409 len 4096
7085 021/300 Flush
7086 022/300 Write block 3218 len 40960 data: 3 3 4 4 2 2 2 3 3 2
7087 023/300 Read block 459 len 16384
7088 024/300 Read block 2668 len 16384
7089 025/300 Read block 915 len 24576
7090 026/300 Write block 1504 len 12288 data: 2 2 2
7091 027/300 Flush
7092 028/300 Read block 2539 len 16384
7093 029/300 Write block 1549 len 36864 data: 3 2 2 2 2 3 3 3 3
7094 030/300 Write block 625 len 12288 data: 4 5 3
7095 031/300 Write block 664 len 4096 data: 4
7096 032/300 Read block 1262 len 24576
7097 033/300 Write block 2987 len 24576 data: 2 2 2 2 2 3
7098 034/300 Write block 816 len 16384 data: 5 4 4 3
7099 035/300 Read block 4755 len 28672
7100 036/300 Write block 672 len 4096 data: 3
7101 037/300 Read block 741 len 24576
7102 038/300 Read block 2702 len 8192
7103 039/300 Read block 339 len 24576
7104 040/300 Read block 2307 len 12288
7105 041/300 Write block 261 len 24576 data: 2 2 2 2 2 2
7106 042/300 Read block 1580 len 24576
7107 043/300 Flush
7108 044/300 Write block 3485 len 20480 data: 3 3 3 3 2
7109 045/300 Read block 2778 len 4096
7110 046/300 Read block 2750 len 32768
7111 047/300 Write block 3212 len 32768 data: 2 3 3 3 3 3 4 4
7112 048/300 Read block 2540 len 36864
7113 049/300 Read block 1231 len 40960
7114 050/300 Write block 2201 len 32768 data: 2 2 2 2 2 2 3 3
7115 051/300 Write block 4592 len 4096 data: 2
7116 052/300 Read block 2857 len 4096
7117 053/300 Read block 1564 len 8192
7118 054/300 Read block 4482 len 12288
7119 055/300 Read block 3573 len 8192
7120 056/300 Read block 4883 len 12288
7121 057/300 Write block 2788 len 12288 data: 6 6 6
7122 058/300 Read block 1565 len 16384
7123 059/300 Write block 3424 len 40960 data: 3 3 4 5 5 4 4 3 3 3
7124 060/300 Flush
7125 061/300 Flush
7126 062/300 Read block 3019 len 36864
7127 063/300 Write block 4078 len 16384 data: 4 5 4 4
7128 064/300 Read block 3118 len 40960
7129 065/300 Write block 261 len 8192 data: 3 3
7130 066/300 Flush
7131 067/300 Read block 3746 len 28672
7132 068/300 Write block 180 len 4096 data: 2
7133 069/300 Flush
7134 070/300 Read block 3416 len 28672
7135 071/300 Read block 4371 len 12288
7136 072/300 Read block 4726 len 16384
7137 073/300 Read block 2597 len 40960
7138 074/300 Read block 1886 len 16384
7139 075/300 Read block 1295 len 8192
7140 076/300 Read block 1636 len 8192
7141 077/300 Write block 913 len 40960 data: 2 2 2 2 2 2 2 2 2 2
7142 078/300 Read block 2676 len 32768
7143 079/300 Write block 3276 len 32768 data: 2 2 2 2 3 3 3 3
7144 080/300 Read block 1872 len 24576
7145 081/300 Flush
7146 082/300 Write block 3819 len 28672 data: 4 3 3 3 2 2 2
7147 083/300 Read block 785 len 12288
7148 084/300 Write block 2110 len 36864 data: 5 4 6 6 6 6 6 4 3
7149 085/300 Write block 2712 len 24576 data: 3 4 5 4 4 4
7150 086/300 Write block 3558 len 12288 data: 3 3 3
7151 087/300 Read block 245 len 36864
7152 088/300 Read block 1656 len 8192
7153 089/300 Read block 2944 len 32768
7154 090/300 Write block 601 len 40960 data: 4 4 3 2 2 2 2 2 2 2
7155 091/300 Write block 317 len 36864 data: 3 3 4 4 5 5 5 4 4
7156 092/300 Write block 3329 len 12288 data: 4 4 4
7157 093/300 Write block 4131 len 32768 data: 3 3 3 3 2 3 3 4
7158 094/300 Flush
7159 095/300 Read block 3750 len 20480
7160 096/300 Read block 320 len 36864
7161 097/300 Read block 3646 len 40960
7162 098/300 Write block 2539 len 16384 data: 3 3 3 3
7163 099/300 Read block 2290 len 24576
7164 100/300 Write block 4797 len 36864 data: 2 2 2 2 3 3 3 3 4
7165 101/300 Read block 3592 len 36864
7166 102/300 Write block 1348 len 32768 data: 2 2 2 2 2 2 2 2
7167 103/300 Write block 3834 len 8192 data: 2 2
7168 104/300 Flush
7169 105/300 Read block 2067 len 12288
7170 106/300 Read block 2388 len 32768
7171 107/300 Write block 736 len 12288 data: 3 2 2
7172 108/300 Read block 1450 len 28672
7173 109/300 Read block 2766 len 12288
7174 110/300 Read block 4965 len 4096
7175 111/300 Write block 2788 len 16384 data: 7 7 7 4
7176 112/300 Read block 2485 len 20480
7177 113/300 Flush
7178 114/300 Read block 884 len 20480
7179 115/300 Write block 3693 len 28672 data: 2 2 2 3 3 3 3
7180 116/300 Read block 1016 len 24576
7181 117/300 Flush
7182 118/300 Read block 1586 len 12288
7183 119/300 Write block 4950 len 40960 data: 4 4 4 3 4 4 3 3 4 3
7184 120/300 Write block 1485 len 12288 data: 2 2 2
7185 121/300 Read block 2104 len 16384
7186 122/300 Read block 1432 len 4096
7187 123/300 Read block 3999 len 28672
7188 124/300 Read block 2447 len 12288
7189 125/300 Write block 3790 len 12288 data: 3 4 4
7190 126/300 Write block 2246 len 40960 data: 2 2 2 2 2 2 2 2 2 3
7191 127/300 Read block 1098 len 28672
7192 128/300 Read block 2985 len 12288
7193 129/300 Read block 474 len 28672
7194 130/300 Write block 4626 len 24576 data: 4 3 3 3 4 3
7195 131/300 Write block 4172 len 12288 data: 2 2 2
7196 132/300 Write block 2694 len 40960 data: 3 3 4 4 3 2 2 3 3 4
7197 133/300 Read block 4531 len 24576
7198 134/300 Write block 3271 len 20480 data: 2 2 2 2 2
7199 135/300 Read block 3314 len 32768
7200 136/300 Read block 2174 len 8192
7201 137/300 Write block 2357 len 32768 data: 2 3 3 3 3 3 4 4
7202 138/300 Write block 1835 len 4096 data: 4
7203 139/300 Read block 2713 len 32768
7204 140/300 Read block 4385 len 16384
7205 141/300 Write block 2595 len 28672 data: 2 2 2 2 2 2 2
7206 142/300 Read block 2107 len 32768
7207 143/300 Write block 2999 len 32768 data: 2 3 3 3 3 3 4 4
7208 144/300 Read block 3665 len 28672
7209 145/300 Write block 506 len 12288 data: 5 5 4
7210 146/300 Write block 2206 len 36864 data: 3 4 4 4 4 4 4 4 4
7211 147/300 Write block 1844 len 4096 data: 3
7212 148/300 Write block 3052 len 8192 data: 4 5
7213 149/300 Read block 876 len 12288
7214 150/300 Read block 3708 len 36864
7215 151/300 Flush
7216 152/300 Read block 244 len 36864
7217 153/300 Read block 3333 len 12288
7218 154/300 Write block 1271 len 40960 data: 2 2 3 3 3 3 3 3 3 4
7219 155/300 Flush
7220 156/300 Read block 2850 len 24576
7221 157/300 Write block 2203 len 32768 data: 3 3 3 4 5 5 5 5
7222 158/300 Read block 1785 len 36864
7223 159/300 Read block 4734 len 16384
7224 160/300 Read block 2532 len 4096
7225 161/300 Flush
7226 162/300 Write block 3579 len 28672 data: 2 3 3 4 3 3 3
7227 163/300 Read block 2247 len 40960
7228 164/300 Read block 439 len 32768
7229 165/300 Read block 1494 len 36864
7230 166/300 Read block 2947 len 12288
7231 167/300 Read block 4281 len 36864
7232 168/300 Flush
7233 169/300 Write block 14 len 16384 data: 4 3 4 4
7234 170/300 Write block 978 len 4096 data: 3
7235 171/300 Read block 4319 len 4096
7236 172/300 Read block 2527 len 40960
7237 173/300 Flush
7238 174/300 Read block 4778 len 20480
7239 175/300 Flush
7240 176/300 Write block 3810 len 28672 data: 2 2 3 3 3 3 3
7241 177/300 Flush
7242 178/300 Write block 616 len 16384 data: 2 2 3 3
7243 179/300 Flush
7244 180/300 Write block 188 len 4096 data: 4
7245 181/300 Read block 3278 len 16384
7246 182/300 Write block 334 len 40960 data: 2 4 4 4 3 3 3 3 3 3
7247 183/300 Read block 3499 len 24576
7248 184/300 Read block 1512 len 12288
7249 185/300 Read block 4610 len 36864
7250 186/300 Write block 3032 len 32768 data: 3 4 4 4 3 3 3 3
7251 187/300 Read block 4964 len 32768
7252 188/300 Read block 1014 len 28672
7253 189/300 Write block 2168 len 12288 data: 3 3 3
7254 190/300 Read block 1482 len 8192
7255 191/300 Read block 1629 len 36864
7256 192/300 Read block 18 len 28672
7257 193/300 Write block 2419 len 16384 data: 2 2 2 2
7258 194/300 Read block 1647 len 40960
7259 195/300 Read block 840 len 24576
7260 196/300 Read block 732 len 12288
7261 197/300 Write block 1888 len 32768 data: 2 3 4 4 4 4 3 3
7262 198/300 Read block 817 len 40960
7263 199/300 Write block 4862 len 4096 data: 5
7264 200/300 Write block 1500 len 8192 data: 2 2
7265 201/300 Read block 4166 len 20480
7266 202/300 Read block 1605 len 8192
7267 203/300 Write block 2192 len 16384 data: 2 3 3 3
7268 204/300 Write block 1837 len 32768 data: 3 4 4 3 3 3 3 4
7269 205/300 Read block 4110 len 36864
7270 206/300 Read block 3200 len 28672
7271 207/300 Write block 743 len 20480 data: 2 2 2 2 2
7272 208/300 Write block 2279 len 36864 data: 2 2 2 2 2 3 2 2 2
7273 209/300 Write block 229 len 8192 data: 3 3
7274 210/300 Write block 3150 len 20480 data: 4 3 3 3 4
7275 211/300 Write block 1978 len 8192 data: 2 2
7276 212/300 Flush
7277 213/300 Read block 1324 len 16384
7278 214/300 Read block 1681 len 24576
7279 215/300 Read block 1054 len 20480
7280 216/300 Read block 202 len 4096
7281 217/300 Read block 2263 len 24576
7282 218/300 Write block 1703 len 40960 data: 3 3 5 6 7 8 9 8 7 8
7283 219/300 Write block 88 len 16384 data: 4 4 3 3
7284 220/300 Write block 3975 len 32768 data: 2 2 2 2 2 2 2 2
7285 221/300 Read block 2823 len 40960
7286 222/300 Flush
7287 223/300 Write block 1075 len 12288 data: 5 5 5
7288 224/300 Read block 3975 len 20480
7289 225/300 Read block 4066 len 28672
7290 226/300 Read block 4337 len 40960
7291 227/300 Write block 862 len 8192 data: 4 4
7292 228/300 Read block 1465 len 36864
7293 229/300 Write block 287 len 16384 data: 3 3 3 3
7294 230/300 Read block 4314 len 8192
7295 231/300 Read block 1258 len 12288
7296 232/300 Read block 298 len 40960
7297 233/300 Write block 2779 len 24576 data: 3 3 3 2 2 4
7298 234/300 Write block 3687 len 8192 data: 2 2
7299 235/300 Write block 4407 len 4096 data: 3
7300 236/300 Write block 3165 len 20480 data: 3 3 2 2 2
7301 237/300 Flush
7302 238/300 Read block 3286 len 16384
7303 239/300 Write block 3662 len 20480 data: 2 2 2 2 2
7304 240/300 Write block 135 len 24576 data: 3 3 3 3 3 4
7305 241/300 Write block 1476 len 24576 data: 2 2 2 2 2 2
7306 242/300 Read block 3582 len 8192
7307 243/300 Read block 2835 len 20480
7308 244/300 Write block 1203 len 32768 data: 2 2 2 2 2 2 2 3
7309 245/300 Write block 4925 len 12288 data: 3 2 2
7310 246/300 Write block 373 len 12288 data: 4 4 5
7311 247/300 Flush
7312 248/300 Read block 4399 len 20480
7313 249/300 Write block 3138 len 16384 data: 2 2 3 3
7314 250/300 Flush
7315 251/300 Write block 4242 len 28672 data: 2 2 2 3 3 3 3
7316 252/300 Read block 2695 len 36864
7317 253/300 Read block 4274 len 20480
7318 254/300 Flush
7319 255/300 Read block 987 len 40960
7320 256/300 Read block 1077 len 24576
7321 257/300 Read block 4180 len 12288
7322 258/300 Write block 1702 len 16384 data: 3 4 4 6
7323 259/300 Flush
7324 260/300 Write block 865 len 40960 data: 4 4 4 2 2 2 2 2 3 3
7325 261/300 Read block 1487 len 8192
7326 262/300 Read block 3302 len 28672
7327 263/300 Write block 1253 len 24576 data: 3 2 2 3 2 2
7328 264/300 Read block 2194 len 36864
7329 265/300 Write block 3806 len 8192 data: 2 2
7330 266/300 Flush
7331 267/300 Write block 1743 len 28672 data: 4 3 3 3 3 3 4
7332 268/300 Read block 707 len 8192
7333 269/300 Write block 2021 len 28672 data: 2 2 2 2 2 2 2
7334 270/300 Write block 3882 len 12288 data: 3 4 3
7335 271/300 Read block 3805 len 40960
7336 272/300 Write block 1251 len 4096 data: 3
7337 273/300 Write block 1778 len 4096 data: 2
7338 274/300 Read block 4421 len 16384
7339 275/300 Read block 4109 len 36864
7340 276/300 Write block 1493 len 4096 data: 4
7341 277/300 Flush
7342 278/300 Read block 1711 len 40960
7343 279/300 Read block 2670 len 24576
7344 280/300 Flush
7345 281/300 Read block 713 len 12288
7346 282/300 Read block 3759 len 32768
7347 283/300 Write block 3738 len 24576 data: 2 2 2 2 2 2
7348 284/300 Read block 3249 len 40960
7349 285/300 Read block 4527 len 20480
7350 286/300 Read block 4017 len 4096
7351 287/300 Read block 3368 len 40960
7352 288/300 Read block 4086 len 16384
7353 289/300 Write block 9 len 32768 data: 6 6 6 4 4 5 4 5
7354 290/300 Flush
7355 291/300 Read block 242 len 8192
7356 292/300 Flush
7357 293/300 Read block 2569 len 32768
7358 294/300 Read block 2 len 24576
7359 295/300 Read block 4982 len 32768
7360 296/300 Read block 1565 len 4096
7361 297/300 Read block 2001 len 12288
7362 298/300 Read block 4597 len 36864
7363 299/300 Write block 727 len 20480 data: 2 2 3 3 3
7364 300/300 Write block 2282 len 28672 data: 3 3 4 3 3 3 2
7365 301/300 Read block 4799 len 4096
7366 Main task has work to do, go find it
7367 got dsc Start(2)
7368 start 2
7369 Nov 28 04:51:52.936 INFO request completed, latency_us: 195, response_code: 204, uri: /start/cid/2, method: POST, req_id: 1b56672c-c160-44f9-ad07-8f03209afd3a, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
7370 Replay: started 2, returned:Ok(())
7371 ----------------------------------------------------------------
7372 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
7373 States: Active Active Offline
7374 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
7375 3114 Acked Write 3 Done Done Sent false
7376 3115 Acked Flush 0 Done Done Sent false
7377 3116 Acked Write 7 Done Done Sent false
7378 3117 Acked Read 10 Done Done Sent false
7379 3118 Acked Write 10 Done Done Sent false
7380 3119 Acked Flush 0 Done Done Sent false
7381 3120 Acked Write 9 Done Done Sent false
7382 3121 Acked Write 1 Done Done Sent false
7383 3122 Acked Read 6 Done Done Sent false
7384 3123 Acked Read 9 Done Done Sent false
7385 3124 Acked Write 8 Done Done Sent false
7386 3125 Acked Write 10 Done Done Sent false
7387 3126 Acked Read 8 Done Done Sent false
7388 3127 Acked Flush 0 Done Done Sent false
7389 3128 Acked Read 8 Done Done Sent false
7390 3129 Acked Write 1 Done Done Sent false
7391 3130 Acked Read 5 Done Done Sent false
7392 3131 Acked Flush 0 Done Done Sent false
7393 3132 Acked Write 9 Done Done Sent false
7394 3133 Acked Read 1 Done Done Sent false
7395 3134 Acked Flush 0 Done Done Sent false
7396 3135 Acked Write 10 Done Done Sent false
7397 3136 Acked Read 4 Done Done Sent false
7398 3137 Acked Read 4 Done Done Sent false
7399 3138 Acked Read 6 Done Done Sent false
7400 3139 Acked Write 3 Done Done Sent false
7401 3140 Acked Flush 0 Done Done Sent false
7402 3141 Acked Read 4 Done Done Sent false
7403 3142 Acked Write 9 Done Done Sent false
7404 3143 Acked Write 3 Done Done Sent false
7405 3144 Acked Write 1 Done Done Sent false
7406 3145 Acked Read 6 Done Done Sent false
7407 3146 Acked Write 6 Done Done Sent false
7408 3147 Acked Write 4 Done Done Sent false
7409 3148 Acked Read 7 Done Done Sent false
7410 3149 Acked Write 1 Done Done Sent false
7411 3150 Acked Read 6 Done Done Sent false
7412 3151 Acked Read 2 Done Done Sent false
7413 3152 Acked Read 6 Done Done Sent false
7414 3153 Acked Read 3 Done Done Sent false
7415 3154 Acked Write 6 Done Done Sent false
7416 3155 Acked Read 6 Done Done Sent false
7417 3156 Acked Flush 0 Done Done Sent false
7418 3157 Acked Write 5 Done Done Sent false
7419 3158 Acked Read 1 Done Done Sent false
7420 3159 Acked Read 8 Done Done Sent false
7421 3160 Acked Write 8 Done Done Sent false
7422 3161 Acked Read 9 Done Done Sent false
7423 3162 Acked Read 10 Done Done Sent false
7424 3163 Acked Write 8 Done Done Sent false
7425 3164 Acked Write 1 Done Done Sent false
7426 3165 Acked Read 1 Done Done Sent false
7427 3166 Acked Read 2 Done Done Sent false
7428 3167 Acked Read 3 Done Done Sent false
7429 3168 Acked Read 2 Done Done Sent false
7430 3169 Acked Read 3 Done Done Sent false
7431 3170 Acked Write 3 Done Done Sent false
7432 3171 Acked Read 4 Done Done Sent false
7433 3172 Acked Write 10 Done Done Sent false
7434 3173 Acked Flush 0 Done Done Sent false
7435 3174 Acked Flush 0 Done Done Sent false
7436 3175 Acked Read 9 Done Done Sent false
7437 3176 Acked Write 4 Done Done Sent false
7438 3177 Acked Read 10 Done Done Sent false
7439 3178 Acked Write 2 Done Done Sent false
7440 3179 Acked Flush 0 Done Done Sent false
7441 3180 Acked Read 7 Done Done Sent false
7442 3181 Acked Write 1 Done Done Sent false
7443 3182 Acked Flush 0 Done Done Sent false
7444 3183 Acked Read 7 Done Done Sent false
7445 3184 Acked Read 3 Done Done Sent false
7446 3185 Acked Read 4 Done Done Sent false
7447 3186 Acked Read 10 Done Done Sent false
7448 3187 Acked Read 4 Done Done Sent false
7449 3188 Acked Read 2 Done Done Sent false
7450 3189 Acked Read 2 Done Done Sent false
7451 3190 Acked Write 10 Done Done Sent false
7452 3191 Acked Read 8 Done Done Sent false
7453 3192 Acked Write 8 Done Done Sent false
7454 3193 Acked Read 6 Done Done Sent false
7455 3194 Acked Flush 0 Done Done Sent false
7456 3195 Acked Write 7 Done Done Sent false
7457 3196 Acked Read 3 Done Done Sent false
7458 3197 Acked Write 9 Done Done Sent false
7459 3198 Acked Write 6 Done Done Sent false
7460 3199 Acked Write 3 Done Done Sent false
7461 3200 Acked Read 9 Done Done Sent false
7462 3201 Acked Read 2 Done Done Sent false
7463 3202 Acked Read 8 Done Done Sent false
7464 3203 Acked Write 10 Done Done Sent false
7465 3204 Acked Write 9 Done Done Sent false
7466 3205 Acked Write 3 Done Done Sent false
7467 3206 Acked Write 8 Done Done Sent false
7468 3207 Acked Flush 0 Done Done Sent false
7469 3208 Acked Read 5 Done Done Sent false
7470 3209 Acked Read 9 Done Done Sent false
7471 3210 Acked Read 10 Done Done Sent false
7472 3211 Acked Write 4 Done Done Sent false
7473 3212 Acked Read 6 Done Done Sent false
7474 3213 Acked Write 9 Done Done Sent false
7475 3214 Acked Read 9 Done Done Sent false
7476 3215 Acked Write 8 Done Done Sent false
7477 3216 Acked Write 2 Done Done Sent false
7478 3217 Acked Flush 0 Done Done Sent false
7479 3218 Acked Read 3 Done Done Sent false
7480 3219 Acked Read 8 Done Done Sent false
7481 3220 Acked Write 3 Done Done Sent false
7482 3221 Acked Read 7 Done Done Sent false
7483 3222 Acked Read 3 Done Done Sent false
7484 3223 Acked Read 1 Done Done Sent false
7485 3224 Acked Write 4 Done Done Sent false
7486 3225 Acked Read 5 Done Done Sent false
7487 3226 Acked Flush 0 Done Done Sent false
7488 3227 Acked Read 5 Done Done Sent false
7489 3228 Acked Write 7 Done Done Sent false
7490 3229 Acked Read 6 Done Done Sent false
7491 3230 Acked Flush 0 Done Done Sent false
7492 3231 Acked Read 3 Done Done Sent false
7493 3232 Acked Write 10 Done Done Sent false
7494 3233 Acked Write 3 Done Done Sent false
7495 3234 Acked Read 4 Done Done Sent false
7496 3235 Acked Read 1 Done Done Sent false
7497 3236 Acked Read 7 Done Done Sent false
7498 3237 Acked Read 3 Done Done Sent false
7499 3238 Acked Write 3 Done Done Sent false
7500 3239 Acked Write 10 Done Done Sent false
7501 3240 Acked Read 7 Done Done Sent false
7502 3241 Acked Read 3 Done Done Sent false
7503 3242 Acked Read 7 Done Done Sent false
7504 3243 Acked Write 6 Done Done Sent false
7505 3244 Acked Write 3 Done Done Sent false
7506 3245 Acked Write 10 Done Done Sent false
7507 3246 Acked Read 6 Done Done Sent false
7508 3247 Acked Write 5 Done Done Sent false
7509 3248 Acked Read 8 Done Done Sent false
7510 3249 Acked Read 2 Done Done Sent false
7511 3250 Acked Write 8 Done Done Sent false
7512 3251 Acked Write 1 Done Done Sent false
7513 3252 Acked Read 8 Done Done Sent false
7514 3253 Acked Read 4 Done Done Sent false
7515 3254 Acked Write 7 Done Done Sent false
7516 3255 Acked Read 8 Done Done Sent false
7517 3256 Acked Write 8 Done Done Sent false
7518 3257 Acked Read 7 Done Done Sent false
7519 3258 Acked Write 3 Done Done Sent false
7520 3259 Acked Write 9 Done Done Sent false
7521 3260 Acked Write 1 Done Done Sent false
7522 3261 Acked Write 2 Done Done Sent false
7523 3262 Acked Read 3 Done Done Sent false
7524 3263 Acked Read 9 Done Done Sent false
7525 3264 Acked Flush 0 Done Done Sent false
7526 3265 Acked Read 9 Done Done Sent false
7527 3266 Acked Read 3 Done Done Sent false
7528 3267 Acked Write 10 Done Done Sent false
7529 3268 Acked Flush 0 Done Done Sent false
7530 3269 Acked Read 6 Done Done Sent false
7531 3270 Acked Write 8 Done Done Sent false
7532 3271 Acked Read 9 Done Done Sent false
7533 3272 Acked Read 4 Done Done Sent false
7534 3273 Acked Read 1 Done Done Sent false
7535 3274 Acked Flush 0 Done Done Sent false
7536 3275 Acked Write 7 Done Done Sent false
7537 3276 Acked Read 10 Done Done Sent false
7538 3277 Acked Read 8 Done Done Sent false
7539 3278 Acked Read 9 Done Done Sent false
7540 3279 Acked Read 3 Done Done Sent false
7541 3280 Acked Read 9 Done Done Sent false
7542 3281 Acked Flush 0 Done Done Sent false
7543 3282 Acked Write 4 Done Done Sent false
7544 3283 Acked Write 1 Done Done Sent false
7545 3284 Acked Read 1 Done Done Sent false
7546 3285 Acked Read 10 Done Done Sent false
7547 3286 Acked Flush 0 Done Done Sent false
7548 3287 Acked Read 5 Done Done Sent false
7549 3288 Acked Flush 0 Done Done Sent false
7550 3289 Acked Write 7 Done Done Sent false
7551 3290 Acked Flush 0 Done Done Sent false
7552 3291 Acked Write 4 Done Done Sent false
7553 3292 Acked Flush 0 Done Done Sent false
7554 3293 Acked Write 1 Done Done Sent false
7555 3294 Acked Read 4 Done Done Sent false
7556 3295 Acked Write 10 Done Done Sent false
7557 3296 Acked Read 6 Done Done Sent false
7558 3297 Acked Read 3 Done Done Sent false
7559 3298 Acked Read 9 Done Done Sent false
7560 3299 Acked Write 8 Done Done Sent false
7561 3300 Acked Read 8 Done Done Sent false
7562 3301 Acked Read 7 Done Done Sent false
7563 3302 Acked Write 3 Done Done Sent false
7564 3303 Acked Read 2 Done Done Sent false
7565 3304 Acked Read 9 Done Done Sent false
7566 3305 Acked Read 7 Done Done Sent false
7567 3306 Acked Write 4 Done Done Sent false
7568 3307 Acked Read 10 Done Done Sent false
7569 3308 Acked Read 6 Done Done Sent false
7570 3309 Acked Read 3 Done Done Sent false
7571 3310 Acked Write 8 Done Done Sent false
7572 3311 Acked Read 10 Done Done Sent false
7573 3312 Acked Write 1 Done Done Sent false
7574 3313 Acked Write 2 Done Done Sent false
7575 3314 Acked Read 5 Done Done Sent false
7576 3315 Acked Read 2 Done Done Sent false
7577 3316 Acked Write 4 Done Done Sent false
7578 3317 Acked Write 8 Done Done Sent false
7579 3318 Acked Read 9 Done Done Sent false
7580 3319 Acked Read 7 Done Done Sent false
7581 3320 Acked Write 5 Done Done Sent false
7582 3321 Acked Write 9 Done Done Sent false
7583 3322 Acked Write 2 Done Done Sent false
7584 3323 Acked Write 5 Done Done Sent false
7585 3324 Acked Write 2 Done Done Sent false
7586 3325 Acked Flush 0 Done Done Sent false
7587 3326 Acked Read 4 Done Done Sent false
7588 3327 Acked Read 6 Done Done Sent false
7589 3328 Acked Read 5 Done Done Sent false
7590 3329 Acked Read 1 Done Done Sent false
7591 3330 Acked Read 6 Done Done Sent false
7592 3331 Acked Write 10 Done Done Sent false
7593 3332 Acked Write 4 Done Done Sent false
7594 3333 Acked Write 8 Done Done Sent false
7595 3334 Acked Read 10 Done Done Sent false
7596 3335 Acked Flush 0 Done Done Sent false
7597 3336 Acked Write 3 Done Done Sent false
7598 3337 Acked Read 5 Done Done Sent false
7599 3338 Acked Read 7 Done Done Sent false
7600 3339 Acked Read 10 Done Done Sent false
7601 3340 Acked Write 2 Done Done Sent false
7602 3341 Acked Read 9 Done Done Sent false
7603 3342 Acked Write 4 Done Done Sent false
7604 3343 Acked Read 2 Done Done Sent false
7605 3344 Acked Read 3 Done Done Sent false
7606 3345 Acked Read 10 Done Done Sent false
7607 3346 Acked Write 6 Done Done Sent false
7608 3347 Acked Write 2 Done Done Sent false
7609 3348 Acked Write 1 Done Done Sent false
7610 3349 Acked Write 5 Done Done Sent false
7611 3350 Acked Flush 0 Done Done Sent false
7612 3351 Acked Read 4 Done Done Sent false
7613 3352 Acked Write 5 Done Done Sent false
7614 3353 Acked Write 6 Done Done Sent false
7615 3354 Acked Write 6 Done Done Sent false
7616 3355 Acked Read 2 Done Done Sent false
7617 3356 Acked Read 5 Done Done Sent false
7618 3357 Acked Write 8 Done Done Sent false
7619 3358 Acked Write 3 Done Done Sent false
7620 3359 Acked Write 3 Done Done Sent false
7621 3360 Acked Flush 0 Done Done Sent false
7622 3361 Acked Read 5 Done Done Sent false
7623 3362 Acked Write 4 Done Done Sent false
7624 3363 Acked Flush 0 Done Done Sent false
7625 3364 Acked Write 7 Done Done Sent false
7626 3365 Acked Read 9 Done Done Sent false
7627 3366 Acked Read 5 Done Done Sent false
7628 3367 Acked Flush 0 Done Done Sent false
7629 3368 Acked Read 10 Done Done Sent false
7630 3369 Acked Read 6 Done Done Sent false
7631 3370 Acked Read 3 Done Done Sent false
7632 3371 Acked Write 4 Done Done Sent false
7633 3372 Acked Flush 0 Done Done Sent false
7634 3373 Acked Write 10 Done Done Sent false
7635 3374 Acked Read 2 Done Done Sent false
7636 3375 Acked Read 7 Done Done Sent false
7637 3376 Acked Write 6 Done Done Sent false
7638 3377 Acked Read 9 Done Done Sent false
7639 3378 Acked Write 2 Done Done Sent false
7640 3379 Acked Flush 0 Done Done Sent false
7641 3380 Acked Write 7 Done Done Sent false
7642 3381 Acked Read 2 Done Done Sent false
7643 3382 Acked Write 7 Done Done Sent false
7644 3383 Acked Write 3 Done Done Sent false
7645 3384 Acked Read 10 Done Done Sent false
7646 3385 Acked Write 1 Done Done Sent false
7647 3386 Acked Write 1 Done Done Sent false
7648 3387 Acked Read 4 Done Done Sent false
7649 3388 Acked Read 9 Done Done Sent false
7650 3389 Acked Write 1 Done Done Sent false
7651 3390 Acked Flush 0 Done Done Sent false
7652 3391 Acked Read 10 Done Done Sent false
7653 3392 Acked Read 6 Done Done Sent false
7654 3393 Acked Flush 0 Done Done Sent false
7655 3394 Acked Read 3 Done Done Sent false
7656 3395 Acked Read 8 Done Done Sent false
7657 3396 Acked Write 6 Done Done Sent false
7658 3397 Acked Read 10 Done Done Sent false
7659 3398 Acked Read 5 Done Done Sent false
7660 3399 Acked Read 1 Done Done Sent false
7661 3400 Acked Read 10 Done Done Sent false
7662 3401 Acked Read 4 Done Done Sent false
7663 3402 Acked Write 8 Done Done Sent false
7664 3403 Acked Flush 0 Done Done Sent false
7665 3404 Acked Read 2 Done Done Sent false
7666 3405 Acked Flush 0 Done Done Sent false
7667 3406 Acked Read 8 Done Done Sent false
7668 3407 Acked Read 6 Done Done Sent false
7669 3408 Acked Read 8 Done Done Sent false
7670 3409 Acked Read 1 Done Done Sent false
7671 3410 Acked Read 3 Done Done Sent false
7672 3411 Acked Read 9 Done Done Sent false
7673 3412 Acked Write 5 Done Done Sent false
7674 3413 Acked Write 7 Done Done Sent false
7675 3414 Acked Read 1 Done Done Sent false
7676 STATES DS:0 DS:1 DS:2 TOTAL
7677 Sent 0 0 301 301
7678 Done 301 301 0 602
7679 Skipped 0 0 0 0
7680 Error 0 0 0 0
7681 Last Flush: 3405 3405 3113
7682 Downstairs last five completed: 3113 3112 3111 3110 3109
7683 Upstairs last five completed: 3414 3413 3412 3411 3410
7684 CLIENT: Up:0 ds:301 act:2
7685 ----------------------------------------------------------------
7686 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
7687 States: Active Active Offline
7688 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
7689 3114 Acked Write 3 Done Done Sent false
7690 3115 Acked Flush 0 Done Done Sent false
7691 3116 Acked Write 7 Done Done Sent false
7692 3117 Acked Read 10 Done Done Sent false
7693 3118 Acked Write 10 Done Done Sent false
7694 3119 Acked Flush 0 Done Done Sent false
7695 3120 Acked Write 9 Done Done Sent false
7696 3121 Acked Write 1 Done Done Sent false
7697 3122 Acked Read 6 Done Done Sent false
7698 3123 Acked Read 9 Done Done Sent false
7699 3124 Acked Write 8 Done Done Sent false
7700 3125 Acked Write 10 Done Done Sent false
7701 3126 Acked Read 8 Done Done Sent false
7702 3127 Acked Flush 0 Done Done Sent false
7703 3128 Acked Read 8 Done Done Sent false
7704 3129 Acked Write 1 Done Done Sent false
7705 3130 Acked Read 5 Done Done Sent false
7706 3131 Acked Flush 0 Done Done Sent false
7707 3132 Acked Write 9 Done Done Sent false
7708 3133 Acked Read 1 Done Done Sent false
7709 3134 Acked Flush 0 Done Done Sent false
7710 3135 Acked Write 10 Done Done Sent false
7711 3136 Acked Read 4 Done Done Sent false
7712 3137 Acked Read 4 Done Done Sent false
7713 3138 Acked Read 6 Done Done Sent false
7714 3139 Acked Write 3 Done Done Sent false
7715 3140 Acked Flush 0 Done Done Sent false
7716 3141 Acked Read 4 Done Done Sent false
7717 3142 Acked Write 9 Done Done Sent false
7718 3143 Acked Write 3 Done Done Sent false
7719 3144 Acked Write 1 Done Done Sent false
7720 3145 Acked Read 6 Done Done Sent false
7721 3146 Acked Write 6 Done Done Sent false
7722 3147 Acked Write 4 Done Done Sent false
7723 3148 Acked Read 7 Done Done Sent false
7724 3149 Acked Write 1 Done Done Sent false
7725 3150 Acked Read 6 Done Done Sent false
7726 3151 Acked Read 2 Done Done Sent false
7727 3152 Acked Read 6 Done Done Sent false
7728 3153 Acked Read 3 Done Done Sent false
7729 3154 Acked Write 6 Done Done Sent false
7730 3155 Acked Read 6 Done Done Sent false
7731 3156 Acked Flush 0 Done Done Sent false
7732 3157 Acked Write 5 Done Done Sent false
7733 3158 Acked Read 1 Done Done Sent false
7734 3159 Acked Read 8 Done Done Sent false
7735 3160 Acked Write 8 Done Done Sent false
7736 3161 Acked Read 9 Done Done Sent false
7737 3162 Acked Read 10 Done Done Sent false
7738 3163 Acked Write 8 Done Done Sent false
7739 3164 Acked Write 1 Done Done Sent false
7740 3165 Acked Read 1 Done Done Sent false
7741 3166 Acked Read 2 Done Done Sent false
7742 3167 Acked Read 3 Done Done Sent false
7743 3168 Acked Read 2 Done Done Sent false
7744 3169 Acked Read 3 Done Done Sent false
7745 3170 Acked Write 3 Done Done Sent false
7746 3171 Acked Read 4 Done Done Sent false
7747 3172 Acked Write 10 Done Done Sent false
7748 3173 Acked Flush 0 Done Done Sent false
7749 3174 Acked Flush 0 Done Done Sent false
7750 3175 Acked Read 9 Done Done Sent false
7751 3176 Acked Write 4 Done Done Sent false
7752 3177 Acked Read 10 Done Done Sent false
7753 3178 Acked Write 2 Done Done Sent false
7754 3179 Acked Flush 0 Done Done Sent false
7755 3180 Acked Read 7 Done Done Sent false
7756 3181 Acked Write 1 Done Done Sent false
7757 3182 Acked Flush 0 Done Done Sent false
7758 3183 Acked Read 7 Done Done Sent false
7759 3184 Acked Read 3 Done Done Sent false
7760 3185 Acked Read 4 Done Done Sent false
7761 3186 Acked Read 10 Done Done Sent false
7762 3187 Acked Read 4 Done Done Sent false
7763 3188 Acked Read 2 Done Done Sent false
7764 3189 Acked Read 2 Done Done Sent false
7765 3190 Acked Write 10 Done Done Sent false
7766 3191 Acked Read 8 Done Done Sent false
7767 3192 Acked Write 8 Done Done Sent false
7768 3193 Acked Read 6 Done Done Sent false
7769 3194 Acked Flush 0 Done Done Sent false
7770 3195 Acked Write 7 Done Done Sent false
7771 3196 Acked Read 3 Done Done Sent false
7772 3197 Acked Write 9 Done Done Sent false
7773 3198 Acked Write 6 Done Done Sent false
7774 3199 Acked Write 3 Done Done Sent false
7775 3200 Acked Read 9 Done Done Sent false
7776 3201 Acked Read 2 Done Done Sent false
7777 3202 Acked Read 8 Done Done Sent false
7778 3203 Acked Write 10 Done Done Sent false
7779 3204 Acked Write 9 Done Done Sent false
7780 3205 Acked Write 3 Done Done Sent false
7781 3206 Acked Write 8 Done Done Sent false
7782 3207 Acked Flush 0 Done Done Sent false
7783 3208 Acked Read 5 Done Done Sent false
7784 3209 Acked Read 9 Done Done Sent false
7785 3210 Acked Read 10 Done Done Sent false
7786 3211 Acked Write 4 Done Done Sent false
7787 3212 Acked Read 6 Done Done Sent false
7788 3213 Acked Write 9 Done Done Sent false
7789 3214 Acked Read 9 Done Done Sent false
7790 3215 Acked Write 8 Done Done Sent false
7791 3216 Acked Write 2 Done Done Sent false
7792 3217 Acked Flush 0 Done Done Sent false
7793 3218 Acked Read 3 Done Done Sent false
7794 3219 Acked Read 8 Done Done Sent false
7795 3220 Acked Write 3 Done Done Sent false
7796 3221 Acked Read 7 Done Done Sent false
7797 3222 Acked Read 3 Done Done Sent false
7798 3223 Acked Read 1 Done Done Sent false
7799 3224 Acked Write 4 Done Done Sent false
7800 3225 Acked Read 5 Done Done Sent false
7801 3226 Acked Flush 0 Done Done Sent false
7802 3227 Acked Read 5 Done Done Sent false
7803 3228 Acked Write 7 Done Done Sent false
7804 3229 Acked Read 6 Done Done Sent false
7805 3230 Acked Flush 0 Done Done Sent false
7806 3231 Acked Read 3 Done Done Sent false
7807 3232 Acked Write 10 Done Done Sent false
7808 3233 Acked Write 3 Done Done Sent false
7809 3234 Acked Read 4 Done Done Sent false
7810 3235 Acked Read 1 Done Done Sent false
7811 3236 Acked Read 7 Done Done Sent false
7812 3237 Acked Read 3 Done Done Sent false
7813 3238 Acked Write 3 Done Done Sent false
7814 3239 Acked Write 10 Done Done Sent false
7815 3240 Acked Read 7 Done Done Sent false
7816 3241 Acked Read 3 Done Done Sent false
7817 3242 Acked Read 7 Done Done Sent false
7818 3243 Acked Write 6 Done Done Sent false
7819 3244 Acked Write 3 Done Done Sent false
7820 3245 Acked Write 10 Done Done Sent false
7821 3246 Acked Read 6 Done Done Sent false
7822 3247 Acked Write 5 Done Done Sent false
7823 3248 Acked Read 8 Done Done Sent false
7824 3249 Acked Read 2 Done Done Sent false
7825 3250 Acked Write 8 Done Done Sent false
7826 3251 Acked Write 1 Done Done Sent false
7827 3252 Acked Read 8 Done Done Sent false
7828 3253 Acked Read 4 Done Done Sent false
7829 3254 Acked Write 7 Done Done Sent false
7830 3255 Acked Read 8 Done Done Sent false
7831 3256 Acked Write 8 Done Done Sent false
7832 3257 Acked Read 7 Done Done Sent false
7833 3258 Acked Write 3 Done Done Sent false
7834 3259 Acked Write 9 Done Done Sent false
7835 3260 Acked Write 1 Done Done Sent false
7836 3261 Acked Write 2 Done Done Sent false
7837 3262 Acked Read 3 Done Done Sent false
7838 3263 Acked Read 9 Done Done Sent false
7839 3264 Acked Flush 0 Done Done Sent false
7840 3265 Acked Read 9 Done Done Sent false
7841 3266 Acked Read 3 Done Done Sent false
7842 3267 Acked Write 10 Done Done Sent false
7843 3268 Acked Flush 0 Done Done Sent false
7844 3269 Acked Read 6 Done Done Sent false
7845 3270 Acked Write 8 Done Done Sent false
7846 3271 Acked Read 9 Done Done Sent false
7847 3272 Acked Read 4 Done Done Sent false
7848 3273 Acked Read 1 Done Done Sent false
7849 3274 Acked Flush 0 Done Done Sent false
7850 3275 Acked Write 7 Done Done Sent false
7851 3276 Acked Read 10 Done Done Sent false
7852 3277 Acked Read 8 Done Done Sent false
7853 3278 Acked Read 9 Done Done Sent false
7854 3279 Acked Read 3 Done Done Sent false
7855 3280 Acked Read 9 Done Done Sent false
7856 3281 Acked Flush 0 Done Done Sent false
7857 3282 Acked Write 4 Done Done Sent false
7858 3283 Acked Write 1 Done Done Sent false
7859 3284 Acked Read 1 Done Done Sent false
7860 3285 Acked Read 10 Done Done Sent false
7861 3286 Acked Flush 0 Done Done Sent false
7862 3287 Acked Read 5 Done Done Sent false
7863 3288 Acked Flush 0 Done Done Sent false
7864 3289 Acked Write 7 Done Done Sent false
7865 3290 Acked Flush 0 Done Done Sent false
7866 3291 Acked Write 4 Done Done Sent false
7867 3292 Acked Flush 0 Done Done Sent false
7868 3293 Acked Write 1 Done Done Sent false
7869 3294 Acked Read 4 Done Done Sent false
7870 3295 Acked Write 10 Done Done Sent false
7871 3296 Acked Read 6 Done Done Sent false
7872 3297 Acked Read 3 Done Done Sent false
7873 3298 Acked Read 9 Done Done Sent false
7874 3299 Acked Write 8 Done Done Sent false
7875 3300 Acked Read 8 Done Done Sent false
7876 3301 Acked Read 7 Done Done Sent false
7877 3302 Acked Write 3 Done Done Sent false
7878 3303 Acked Read 2 Done Done Sent false
7879 3304 Acked Read 9 Done Done Sent false
7880 3305 Acked Read 7 Done Done Sent false
7881 3306 Acked Write 4 Done Done Sent false
7882 3307 Acked Read 10 Done Done Sent false
7883 3308 Acked Read 6 Done Done Sent false
7884 3309 Acked Read 3 Done Done Sent false
7885 3310 Acked Write 8 Done Done Sent false
7886 3311 Acked Read 10 Done Done Sent false
7887 3312 Acked Write 1 Done Done Sent false
7888 3313 Acked Write 2 Done Done Sent false
7889 3314 Acked Read 5 Done Done Sent false
7890 3315 Acked Read 2 Done Done Sent false
7891 3316 Acked Write 4 Done Done Sent false
7892 3317 Acked Write 8 Done Done Sent false
7893 3318 Acked Read 9 Done Done Sent false
7894 3319 Acked Read 7 Done Done Sent false
7895 3320 Acked Write 5 Done Done Sent false
7896 3321 Acked Write 9 Done Done Sent false
7897 3322 Acked Write 2 Done Done Sent false
7898 3323 Acked Write 5 Done Done Sent false
7899 3324 Acked Write 2 Done Done Sent false
7900 3325 Acked Flush 0 Done Done Sent false
7901 3326 Acked Read 4 Done Done Sent false
7902 3327 Acked Read 6 Done Done Sent false
7903 3328 Acked Read 5 Done Done Sent false
7904 3329 Acked Read 1 Done Done Sent false
7905 3330 Acked Read 6 Done Done Sent false
7906 3331 Acked Write 10 Done Done Sent false
7907 3332 Acked Write 4 Done Done Sent false
7908 3333 Acked Write 8 Done Done Sent false
7909 3334 Acked Read 10 Done Done Sent false
7910 3335 Acked Flush 0 Done Done Sent false
7911 3336 Acked Write 3 Done Done Sent false
7912 3337 Acked Read 5 Done Done Sent false
7913 3338 Acked Read 7 Done Done Sent false
7914 3339 Acked Read 10 Done Done Sent false
7915 3340 Acked Write 2 Done Done Sent false
7916 3341 Acked Read 9 Done Done Sent false
7917 3342 Acked Write 4 Done Done Sent false
7918 3343 Acked Read 2 Done Done Sent false
7919 3344 Acked Read 3 Done Done Sent false
7920 3345 Acked Read 10 Done Done Sent false
7921 3346 Acked Write 6 Done Done Sent false
7922 3347 Acked Write 2 Done Done Sent false
7923 3348 Acked Write 1 Done Done Sent false
7924 3349 Acked Write 5 Done Done Sent false
7925 3350 Acked Flush 0 Done Done Sent false
7926 3351 Acked Read 4 Done Done Sent false
7927 3352 Acked Write 5 Done Done Sent false
7928 3353 Acked Write 6 Done Done Sent false
7929 3354 Acked Write 6 Done Done Sent false
7930 3355 Acked Read 2 Done Done Sent false
7931 3356 Acked Read 5 Done Done Sent false
7932 3357 Acked Write 8 Done Done Sent false
7933 3358 Acked Write 3 Done Done Sent false
7934 3359 Acked Write 3 Done Done Sent false
7935 3360 Acked Flush 0 Done Done Sent false
7936 3361 Acked Read 5 Done Done Sent false
7937 3362 Acked Write 4 Done Done Sent false
7938 3363 Acked Flush 0 Done Done Sent false
7939 3364 Acked Write 7 Done Done Sent false
7940 3365 Acked Read 9 Done Done Sent false
7941 3366 Acked Read 5 Done Done Sent false
7942 3367 Acked Flush 0 Done Done Sent false
7943 3368 Acked Read 10 Done Done Sent false
7944 3369 Acked Read 6 Done Done Sent false
7945 3370 Acked Read 3 Done Done Sent false
7946 3371 Acked Write 4 Done Done Sent false
7947 3372 Acked Flush 0 Done Done Sent false
7948 3373 Acked Write 10 Done Done Sent false
7949 3374 Acked Read 2 Done Done Sent false
7950 3375 Acked Read 7 Done Done Sent false
7951 3376 Acked Write 6 Done Done Sent false
7952 3377 Acked Read 9 Done Done Sent false
7953 3378 Acked Write 2 Done Done Sent false
7954 3379 Acked Flush 0 Done Done Sent false
7955 3380 Acked Write 7 Done Done Sent false
7956 3381 Acked Read 2 Done Done Sent false
7957 3382 Acked Write 7 Done Done Sent false
7958 3383 Acked Write 3 Done Done Sent false
7959 3384 Acked Read 10 Done Done Sent false
7960 3385 Acked Write 1 Done Done Sent false
7961 3386 Acked Write 1 Done Done Sent false
7962 3387 Acked Read 4 Done Done Sent false
7963 3388 Acked Read 9 Done Done Sent false
7964 3389 Acked Write 1 Done Done Sent false
7965 3390 Acked Flush 0 Done Done Sent false
7966 3391 Acked Read 10 Done Done Sent false
7967 3392 Acked Read 6 Done Done Sent false
7968 3393 Acked Flush 0 Done Done Sent false
7969 3394 Acked Read 3 Done Done Sent false
7970 3395 Acked Read 8 Done Done Sent false
7971 3396 Acked Write 6 Done Done Sent false
7972 3397 Acked Read 10 Done Done Sent false
7973 3398 Acked Read 5 Done Done Sent false
7974 3399 Acked Read 1 Done Done Sent false
7975 3400 Acked Read 10 Done Done Sent false
7976 3401 Acked Read 4 Done Done Sent false
7977 3402 Acked Write 8 Done Done Sent false
7978 3403 Acked Flush 0 Done Done Sent false
7979 3404 Acked Read 2 Done Done Sent false
7980 3405 Acked Flush 0 Done Done Sent false
7981 3406 Acked Read 8 Done Done Sent false
7982 3407 Acked Read 6 Done Done Sent false
7983 3408 Acked Read 8 Done Done Sent false
7984 3409 Acked Read 1 Done Done Sent false
7985 3410 Acked Read 3 Done Done Sent false
7986 3411 Acked Read 9 Done Done Sent false
7987 3412 Acked Write 5 Done Done Sent false
7988 3413 Acked Write 7 Done Done Sent false
7989 3414 Acked Read 1 Done Done Sent false
7990 3415 Acked Flush 0 Done Done Sent false
7991 STATES DS:0 DS:1 DS:2 TOTAL
7992 Sent 0 0 302 302
7993 Done 302 302 0 604
7994 Skipped 0 0 0 0
7995 Error 0 0 0 0
7996 Last Flush: 3415 3415 3113
7997 Downstairs last five completed: 3113 3112 3111 3110 3109
7998 Upstairs last five completed: 3415 3414 3413 3412 3411
7999 CLIENT: Up:0 ds:302 act:2
80002024-11-28T04:51:58.699ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
80012024-11-28T04:51:58.699ZWARNcrucible: ds_connection connect to 127.0.0.1:8830 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
80022024-11-28T04:51:58.699ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
80032024-11-28T04:51:58.699ZWARNcrucible: downstairs task for 2 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
80042024-11-28T04:51:58.699ZINFOcrucible: client task is exiting = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
8005 ----------------------------------------------------------------
8006 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
8007 States: Active Active Offline
8008 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
8009 3114 Acked Write 3 Done Done Sent false
8010 3115 Acked Flush 0 Done Done Sent false
8011 3116 Acked Write 7 Done Done Sent false
8012 3117 Acked Read 10 Done Done Sent false
8013 3118 Acked Write 10 Done Done Sent false
8014 3119 Acked Flush 0 Done Done Sent false
8015 3120 Acked Write 9 Done Done Sent false
8016 3121 Acked Write 1 Done Done Sent false
8017 3122 Acked Read 6 Done Done Sent false
8018 3123 Acked Read 9 Done Done Sent false
8019 3124 Acked Write 8 Done Done Sent false
8020 3125 Acked Write 10 Done Done Sent false
8021 3126 Acked Read 8 Done Done Sent false
8022 3127 Acked Flush 0 Done Done Sent false
8023 3128 Acked Read 8 Done Done Sent false
8024 3129 Acked Write 1 Done Done Sent false
8025 3130 Acked Read 5 Done Done Sent false
8026 3131 Acked Flush 0 Done Done Sent false
8027 3132 Acked Write 9 Done Done Sent false
8028 3133 Acked Read 1 Done Done Sent false
8029 3134 Acked Flush 0 Done Done Sent false
8030 3135 Acked Write 10 Done Done Sent false
8031 3136 Acked Read 4 Done Done Sent false
8032 3137 Acked Read 4 Done Done Sent false
8033 3138 Acked Read 6 Done Done Sent false
8034 3139 Acked Write 3 Done Done Sent false
8035 3140 Acked Flush 0 Done Done Sent false
8036 3141 Acked Read 4 Done Done Sent false
8037 3142 Acked Write 9 Done Done Sent false
8038 3143 Acked Write 3 Done Done Sent false
8039 3144 Acked Write 1 Done Done Sent false
8040 3145 Acked Read 6 Done Done Sent false
8041 3146 Acked Write 6 Done Done Sent false
8042 3147 Acked Write 4 Done Done Sent false
8043 3148 Acked Read 7 Done Done Sent false
8044 3149 Acked Write 1 Done Done Sent false
8045 3150 Acked Read 6 Done Done Sent false
8046 3151 Acked Read 2 Done Done Sent false
8047 3152 Acked Read 6 Done Done Sent false
8048 3153 Acked Read 3 Done Done Sent false
8049 3154 Acked Write 6 Done Done Sent false
8050 3155 Acked Read 6 Done Done Sent false
8051 3156 Acked Flush 0 Done Done Sent false
8052 3157 Acked Write 5 Done Done Sent false
8053 3158 Acked Read 1 Done Done Sent false
8054 3159 Acked Read 8 Done Done Sent false
8055 3160 Acked Write 8 Done Done Sent false
8056 3161 Acked Read 9 Done Done Sent false
8057 3162 Acked Read 10 Done Done Sent false
8058 3163 Acked Write 8 Done Done Sent false
8059 3164 Acked Write 1 Done Done Sent false
8060 3165 Acked Read 1 Done Done Sent false
8061 3166 Acked Read 2 Done Done Sent false
8062 3167 Acked Read 3 Done Done Sent false
8063 3168 Acked Read 2 Done Done Sent false
8064 3169 Acked Read 3 Done Done Sent false
8065 3170 Acked Write 3 Done Done Sent false
8066 3171 Acked Read 4 Done Done Sent false
8067 3172 Acked Write 10 Done Done Sent false
8068 3173 Acked Flush 0 Done Done Sent false
8069 3174 Acked Flush 0 Done Done Sent false
8070 3175 Acked Read 9 Done Done Sent false
8071 3176 Acked Write 4 Done Done Sent false
8072 3177 Acked Read 10 Done Done Sent false
8073 3178 Acked Write 2 Done Done Sent false
8074 3179 Acked Flush 0 Done Done Sent false
8075 3180 Acked Read 7 Done Done Sent false
8076 3181 Acked Write 1 Done Done Sent false
8077 3182 Acked Flush 0 Done Done Sent false
8078 3183 Acked Read 7 Done Done Sent false
8079 3184 Acked Read 3 Done Done Sent false
8080 3185 Acked Read 4 Done Done Sent false
8081 3186 Acked Read 10 Done Done Sent false
8082 3187 Acked Read 4 Done Done Sent false
8083 3188 Acked Read 2 Done Done Sent false
8084 3189 Acked Read 2 Done Done Sent false
8085 3190 Acked Write 10 Done Done Sent false
8086 3191 Acked Read 8 Done Done Sent false
8087 3192 Acked Write 8 Done Done Sent false
8088 3193 Acked Read 6 Done Done Sent false
8089 3194 Acked Flush 0 Done Done Sent false
8090 3195 Acked Write 7 Done Done Sent false
8091 3196 Acked Read 3 Done Done Sent false
8092 3197 Acked Write 9 Done Done Sent false
8093 3198 Acked Write 6 Done Done Sent false
8094 3199 Acked Write 3 Done Done Sent false
8095 3200 Acked Read 9 Done Done Sent false
8096 3201 Acked Read 2 Done Done Sent false
8097 3202 Acked Read 8 Done Done Sent false
8098 3203 Acked Write 10 Done Done Sent false
8099 3204 Acked Write 9 Done Done Sent false
8100 3205 Acked Write 3 Done Done Sent false
8101 3206 Acked Write 8 Done Done Sent false
8102 3207 Acked Flush 0 Done Done Sent false
8103 3208 Acked Read 5 Done Done Sent false
8104 3209 Acked Read 9 Done Done Sent false
8105 3210 Acked Read 10 Done Done Sent false
8106 3211 Acked Write 4 Done Done Sent false
8107 3212 Acked Read 6 Done Done Sent false
8108 3213 Acked Write 9 Done Done Sent false
8109 3214 Acked Read 9 Done Done Sent false
8110 3215 Acked Write 8 Done Done Sent false
8111 3216 Acked Write 2 Done Done Sent false
8112 3217 Acked Flush 0 Done Done Sent false
8113 3218 Acked Read 3 Done Done Sent false
8114 3219 Acked Read 8 Done Done Sent false
8115 3220 Acked Write 3 Done Done Sent false
8116 3221 Acked Read 7 Done Done Sent false
8117 3222 Acked Read 3 Done Done Sent false
8118 3223 Acked Read 1 Done Done Sent false
8119 3224 Acked Write 4 Done Done Sent false
8120 3225 Acked Read 5 Done Done Sent false
8121 3226 Acked Flush 0 Done Done Sent false
8122 3227 Acked Read 5 Done Done Sent false
8123 3228 Acked Write 7 Done Done Sent false
8124 3229 Acked Read 6 Done Done Sent false
8125 3230 Acked Flush 0 Done Done Sent false
8126 3231 Acked Read 3 Done Done Sent false
8127 3232 Acked Write 10 Done Done Sent false
8128 3233 Acked Write 3 Done Done Sent false
8129 3234 Acked Read 4 Done Done Sent false
8130 3235 Acked Read 1 Done Done Sent false
8131 3236 Acked Read 7 Done Done Sent false
8132 3237 Acked Read 3 Done Done Sent false
8133 3238 Acked Write 3 Done Done Sent false
8134 3239 Acked Write 10 Done Done Sent false
8135 3240 Acked Read 7 Done Done Sent false
8136 3241 Acked Read 3 Done Done Sent false
8137 3242 Acked Read 7 Done Done Sent false
8138 3243 Acked Write 6 Done Done Sent false
8139 3244 Acked Write 3 Done Done Sent false
8140 3245 Acked Write 10 Done Done Sent false
8141 3246 Acked Read 6 Done Done Sent false
8142 3247 Acked Write 5 Done Done Sent false
8143 3248 Acked Read 8 Done Done Sent false
8144 3249 Acked Read 2 Done Done Sent false
8145 3250 Acked Write 8 Done Done Sent false
8146 3251 Acked Write 1 Done Done Sent false
8147 3252 Acked Read 8 Done Done Sent false
8148 3253 Acked Read 4 Done Done Sent false
8149 3254 Acked Write 7 Done Done Sent false
8150 3255 Acked Read 8 Done Done Sent false
8151 3256 Acked Write 8 Done Done Sent false
8152 3257 Acked Read 7 Done Done Sent false
8153 3258 Acked Write 3 Done Done Sent false
8154 3259 Acked Write 9 Done Done Sent false
8155 3260 Acked Write 1 Done Done Sent false
8156 3261 Acked Write 2 Done Done Sent false
8157 3262 Acked Read 3 Done Done Sent false
8158 3263 Acked Read 9 Done Done Sent false
8159 3264 Acked Flush 0 Done Done Sent false
8160 3265 Acked Read 9 Done Done Sent false
8161 3266 Acked Read 3 Done Done Sent false
8162 3267 Acked Write 10 Done Done Sent false
8163 3268 Acked Flush 0 Done Done Sent false
8164 3269 Acked Read 6 Done Done Sent false
8165 3270 Acked Write 8 Done Done Sent false
8166 3271 Acked Read 9 Done Done Sent false
8167 3272 Acked Read 4 Done Done Sent false
8168 3273 Acked Read 1 Done Done Sent false
8169 3274 Acked Flush 0 Done Done Sent false
8170 3275 Acked Write 7 Done Done Sent false
8171 3276 Acked Read 10 Done Done Sent false
8172 3277 Acked Read 8 Done Done Sent false
8173 3278 Acked Read 9 Done Done Sent false
8174 3279 Acked Read 3 Done Done Sent false
8175 3280 Acked Read 9 Done Done Sent false
8176 3281 Acked Flush 0 Done Done Sent false
8177 3282 Acked Write 4 Done Done Sent false
8178 3283 Acked Write 1 Done Done Sent false
8179 3284 Acked Read 1 Done Done Sent false
8180 3285 Acked Read 10 Done Done Sent false
8181 3286 Acked Flush 0 Done Done Sent false
8182 3287 Acked Read 5 Done Done Sent false
8183 3288 Acked Flush 0 Done Done Sent false
8184 3289 Acked Write 7 Done Done Sent false
8185 3290 Acked Flush 0 Done Done Sent false
8186 3291 Acked Write 4 Done Done Sent false
8187 3292 Acked Flush 0 Done Done Sent false
8188 3293 Acked Write 1 Done Done Sent false
8189 3294 Acked Read 4 Done Done Sent false
8190 3295 Acked Write 10 Done Done Sent false
8191 3296 Acked Read 6 Done Done Sent false
8192 3297 Acked Read 3 Done Done Sent false
8193 3298 Acked Read 9 Done Done Sent false
8194 3299 Acked Write 8 Done Done Sent false
8195 3300 Acked Read 8 Done Done Sent false
8196 3301 Acked Read 7 Done Done Sent false
8197 3302 Acked Write 3 Done Done Sent false
8198 3303 Acked Read 2 Done Done Sent false
8199 3304 Acked Read 9 Done Done Sent false
8200 3305 Acked Read 7 Done Done Sent false
8201 3306 Acked Write 4 Done Done Sent false
8202 3307 Acked Read 10 Done Done Sent false
8203 3308 Acked Read 6 Done Done Sent false
8204 3309 Acked Read 3 Done Done Sent false
8205 3310 Acked Write 8 Done Done Sent false
8206 3311 Acked Read 10 Done Done Sent false
8207 3312 Acked Write 1 Done Done Sent false
8208 3313 Acked Write 2 Done Done Sent false
8209 3314 Acked Read 5 Done Done Sent false
8210 3315 Acked Read 2 Done Done Sent false
8211 3316 Acked Write 4 Done Done Sent false
8212 3317 Acked Write 8 Done Done Sent false
8213 3318 Acked Read 9 Done Done Sent false
8214 3319 Acked Read 7 Done Done Sent false
8215 3320 Acked Write 5 Done Done Sent false
8216 3321 Acked Write 9 Done Done Sent false
8217 3322 Acked Write 2 Done Done Sent false
8218 3323 Acked Write 5 Done Done Sent false
8219 3324 Acked Write 2 Done Done Sent false
8220 3325 Acked Flush 0 Done Done Sent false
8221 3326 Acked Read 4 Done Done Sent false
8222 3327 Acked Read 6 Done Done Sent false
8223 3328 Acked Read 5 Done Done Sent false
8224 3329 Acked Read 1 Done Done Sent false
8225 3330 Acked Read 6 Done Done Sent false
8226 3331 Acked Write 10 Done Done Sent false
8227 3332 Acked Write 4 Done Done Sent false
8228 3333 Acked Write 8 Done Done Sent false
8229 3334 Acked Read 10 Done Done Sent false
8230 3335 Acked Flush 0 Done Done Sent false
8231 3336 Acked Write 3 Done Done Sent false
8232 3337 Acked Read 5 Done Done Sent false
8233 3338 Acked Read 7 Done Done Sent false
8234 3339 Acked Read 10 Done Done Sent false
8235 3340 Acked Write 2 Done Done Sent false
8236 3341 Acked Read 9 Done Done Sent false
8237 3342 Acked Write 4 Done Done Sent false
8238 3343 Acked Read 2 Done Done Sent false
8239 3344 Acked Read 3 Done Done Sent false
8240 3345 Acked Read 10 Done Done Sent false
8241 3346 Acked Write 6 Done Done Sent false
8242 3347 Acked Write 2 Done Done Sent false
8243 3348 Acked Write 1 Done Done Sent false
8244 3349 Acked Write 5 Done Done Sent false
8245 3350 Acked Flush 0 Done Done Sent false
8246 3351 Acked Read 4 Done Done Sent false
8247 3352 Acked Write 5 Done Done Sent false
8248 3353 Acked Write 6 Done Done Sent false
8249 3354 Acked Write 6 Done Done Sent false
8250 3355 Acked Read 2 Done Done Sent false
8251 3356 Acked Read 5 Done Done Sent false
8252 3357 Acked Write 8 Done Done Sent false
8253 3358 Acked Write 3 Done Done Sent false
8254 3359 Acked Write 3 Done Done Sent false
8255 3360 Acked Flush 0 Done Done Sent false
8256 3361 Acked Read 5 Done Done Sent false
8257 3362 Acked Write 4 Done Done Sent false
8258 3363 Acked Flush 0 Done Done Sent false
8259 3364 Acked Write 7 Done Done Sent false
8260 3365 Acked Read 9 Done Done Sent false
8261 3366 Acked Read 5 Done Done Sent false
8262 3367 Acked Flush 0 Done Done Sent false
8263 3368 Acked Read 10 Done Done Sent false
8264 3369 Acked Read 6 Done Done Sent false
8265 3370 Acked Read 3 Done Done Sent false
8266 3371 Acked Write 4 Done Done Sent false
8267 3372 Acked Flush 0 Done Done Sent false
8268 3373 Acked Write 10 Done Done Sent false
8269 3374 Acked Read 2 Done Done Sent false
8270 3375 Acked Read 7 Done Done Sent false
8271 3376 Acked Write 6 Done Done Sent false
8272 3377 Acked Read 9 Done Done Sent false
8273 3378 Acked Write 2 Done Done Sent false
8274 3379 Acked Flush 0 Done Done Sent false
8275 3380 Acked Write 7 Done Done Sent false
8276 3381 Acked Read 2 Done Done Sent false
8277 3382 Acked Write 7 Done Done Sent false
8278 3383 Acked Write 3 Done Done Sent false
8279 3384 Acked Read 10 Done Done Sent false
8280 3385 Acked Write 1 Done Done Sent false
8281 3386 Acked Write 1 Done Done Sent false
8282 3387 Acked Read 4 Done Done Sent false
8283 3388 Acked Read 9 Done Done Sent false
8284 3389 Acked Write 1 Done Done Sent false
8285 3390 Acked Flush 0 Done Done Sent false
8286 3391 Acked Read 10 Done Done Sent false
8287 3392 Acked Read 6 Done Done Sent false
8288 3393 Acked Flush 0 Done Done Sent false
8289 3394 Acked Read 3 Done Done Sent false
8290 3395 Acked Read 8 Done Done Sent false
8291 3396 Acked Write 6 Done Done Sent false
8292 3397 Acked Read 10 Done Done Sent false
8293 3398 Acked Read 5 Done Done Sent false
8294 3399 Acked Read 1 Done Done Sent false
8295 3400 Acked Read 10 Done Done Sent false
8296 3401 Acked Read 4 Done Done Sent false
8297 3402 Acked Write 8 Done Done Sent false
8298 3403 Acked Flush 0 Done Done Sent false
8299 3404 Acked Read 2 Done Done Sent false
8300 3405 Acked Flush 0 Done Done Sent false
8301 3406 Acked Read 8 Done Done Sent false
8302 3407 Acked Read 6 Done Done Sent false
8303 3408 Acked Read 8 Done Done Sent false
8304 3409 Acked Read 1 Done Done Sent false
8305 3410 Acked Read 3 Done Done Sent false
8306 3411 Acked Read 9 Done Done Sent false
8307 3412 Acked Write 5 Done Done Sent false
8308 3413 Acked Write 7 Done Done Sent false
8309 3414 Acked Read 1 Done Done Sent false
8310 3415 Acked Flush 0 Done Done Sent false
8311 STATES DS:0 DS:1 DS:2 TOTAL
8312 Sent 0 0 302 302
8313 Done 302 302 0 604
8314 Skipped 0 0 0 0
8315 Error 0 0 0 0
8316 Last Flush: 3415 3415 3113
8317 Downstairs last five completed: 3113 3112 3111 3110 3109
8318 Upstairs last five completed: 3415 3414 3413 3412 3411
8319 CLIENT: Up:0 ds:302 act:2
8320 [8830] got start action
8321 [8830] I am going to restart
8322 Starting downstairs at port 8830
8323 Make output file at "/tmp/dsc/downstairs-8830.txt"
8324 [8830][2] reports Starting
8325 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1210)
8326 [8830][2] reports Running
8327 ----------------------------------------------------------------
8328 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
8329 States: Active Active Offline
8330 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
8331 3114 Acked Write 3 Done Done Sent false
8332 3115 Acked Flush 0 Done Done Sent false
8333 3116 Acked Write 7 Done Done Sent false
8334 3117 Acked Read 10 Done Done Sent false
8335 3118 Acked Write 10 Done Done Sent false
8336 3119 Acked Flush 0 Done Done Sent false
8337 3120 Acked Write 9 Done Done Sent false
8338 3121 Acked Write 1 Done Done Sent false
8339 3122 Acked Read 6 Done Done Sent false
8340 3123 Acked Read 9 Done Done Sent false
8341 3124 Acked Write 8 Done Done Sent false
8342 3125 Acked Write 10 Done Done Sent false
8343 3126 Acked Read 8 Done Done Sent false
8344 3127 Acked Flush 0 Done Done Sent false
8345 3128 Acked Read 8 Done Done Sent false
8346 3129 Acked Write 1 Done Done Sent false
8347 3130 Acked Read 5 Done Done Sent false
8348 3131 Acked Flush 0 Done Done Sent false
8349 3132 Acked Write 9 Done Done Sent false
8350 3133 Acked Read 1 Done Done Sent false
8351 3134 Acked Flush 0 Done Done Sent false
8352 3135 Acked Write 10 Done Done Sent false
8353 3136 Acked Read 4 Done Done Sent false
8354 3137 Acked Read 4 Done Done Sent false
8355 3138 Acked Read 6 Done Done Sent false
8356 3139 Acked Write 3 Done Done Sent false
8357 3140 Acked Flush 0 Done Done Sent false
8358 3141 Acked Read 4 Done Done Sent false
8359 3142 Acked Write 9 Done Done Sent false
8360 3143 Acked Write 3 Done Done Sent false
8361 3144 Acked Write 1 Done Done Sent false
8362 3145 Acked Read 6 Done Done Sent false
8363 3146 Acked Write 6 Done Done Sent false
8364 3147 Acked Write 4 Done Done Sent false
8365 3148 Acked Read 7 Done Done Sent false
8366 3149 Acked Write 1 Done Done Sent false
8367 3150 Acked Read 6 Done Done Sent false
8368 3151 Acked Read 2 Done Done Sent false
8369 3152 Acked Read 6 Done Done Sent false
8370 3153 Acked Read 3 Done Done Sent false
8371 3154 Acked Write 6 Done Done Sent false
8372 3155 Acked Read 6 Done Done Sent false
8373 3156 Acked Flush 0 Done Done Sent false
8374 3157 Acked Write 5 Done Done Sent false
8375 3158 Acked Read 1 Done Done Sent false
8376 3159 Acked Read 8 Done Done Sent false
8377 3160 Acked Write 8 Done Done Sent false
8378 3161 Acked Read 9 Done Done Sent false
8379 3162 Acked Read 10 Done Done Sent false
8380 3163 Acked Write 8 Done Done Sent false
8381 3164 Acked Write 1 Done Done Sent false
8382 3165 Acked Read 1 Done Done Sent false
8383 3166 Acked Read 2 Done Done Sent false
8384 3167 Acked Read 3 Done Done Sent false
8385 3168 Acked Read 2 Done Done Sent false
8386 3169 Acked Read 3 Done Done Sent false
8387 3170 Acked Write 3 Done Done Sent false
8388 3171 Acked Read 4 Done Done Sent false
8389 3172 Acked Write 10 Done Done Sent false
8390 3173 Acked Flush 0 Done Done Sent false
8391 3174 Acked Flush 0 Done Done Sent false
8392 3175 Acked Read 9 Done Done Sent false
8393 3176 Acked Write 4 Done Done Sent false
8394 3177 Acked Read 10 Done Done Sent false
8395 3178 Acked Write 2 Done Done Sent false
8396 3179 Acked Flush 0 Done Done Sent false
8397 3180 Acked Read 7 Done Done Sent false
8398 3181 Acked Write 1 Done Done Sent false
8399 3182 Acked Flush 0 Done Done Sent false
8400 3183 Acked Read 7 Done Done Sent false
8401 3184 Acked Read 3 Done Done Sent false
8402 3185 Acked Read 4 Done Done Sent false
8403 3186 Acked Read 10 Done Done Sent false
8404 3187 Acked Read 4 Done Done Sent false
8405 3188 Acked Read 2 Done Done Sent false
8406 3189 Acked Read 2 Done Done Sent false
8407 3190 Acked Write 10 Done Done Sent false
8408 3191 Acked Read 8 Done Done Sent false
8409 3192 Acked Write 8 Done Done Sent false
8410 3193 Acked Read 6 Done Done Sent false
8411 3194 Acked Flush 0 Done Done Sent false
8412 3195 Acked Write 7 Done Done Sent false
8413 3196 Acked Read 3 Done Done Sent false
8414 3197 Acked Write 9 Done Done Sent false
8415 3198 Acked Write 6 Done Done Sent false
8416 3199 Acked Write 3 Done Done Sent false
8417 3200 Acked Read 9 Done Done Sent false
8418 3201 Acked Read 2 Done Done Sent false
8419 3202 Acked Read 8 Done Done Sent false
8420 3203 Acked Write 10 Done Done Sent false
8421 3204 Acked Write 9 Done Done Sent false
8422 3205 Acked Write 3 Done Done Sent false
8423 3206 Acked Write 8 Done Done Sent false
8424 3207 Acked Flush 0 Done Done Sent false
8425 3208 Acked Read 5 Done Done Sent false
8426 3209 Acked Read 9 Done Done Sent false
8427 3210 Acked Read 10 Done Done Sent false
8428 3211 Acked Write 4 Done Done Sent false
8429 3212 Acked Read 6 Done Done Sent false
8430 3213 Acked Write 9 Done Done Sent false
8431 3214 Acked Read 9 Done Done Sent false
8432 3215 Acked Write 8 Done Done Sent false
8433 3216 Acked Write 2 Done Done Sent false
8434 3217 Acked Flush 0 Done Done Sent false
8435 3218 Acked Read 3 Done Done Sent false
8436 3219 Acked Read 8 Done Done Sent false
8437 3220 Acked Write 3 Done Done Sent false
8438 3221 Acked Read 7 Done Done Sent false
8439 3222 Acked Read 3 Done Done Sent false
8440 3223 Acked Read 1 Done Done Sent false
8441 3224 Acked Write 4 Done Done Sent false
8442 3225 Acked Read 5 Done Done Sent false
8443 3226 Acked Flush 0 Done Done Sent false
8444 3227 Acked Read 5 Done Done Sent false
8445 3228 Acked Write 7 Done Done Sent false
8446 3229 Acked Read 6 Done Done Sent false
8447 3230 Acked Flush 0 Done Done Sent false
8448 3231 Acked Read 3 Done Done Sent false
8449 3232 Acked Write 10 Done Done Sent false
8450 3233 Acked Write 3 Done Done Sent false
8451 3234 Acked Read 4 Done Done Sent false
8452 3235 Acked Read 1 Done Done Sent false
8453 3236 Acked Read 7 Done Done Sent false
8454 3237 Acked Read 3 Done Done Sent false
8455 3238 Acked Write 3 Done Done Sent false
8456 3239 Acked Write 10 Done Done Sent false
8457 3240 Acked Read 7 Done Done Sent false
8458 3241 Acked Read 3 Done Done Sent false
8459 3242 Acked Read 7 Done Done Sent false
8460 3243 Acked Write 6 Done Done Sent false
8461 3244 Acked Write 3 Done Done Sent false
8462 3245 Acked Write 10 Done Done Sent false
8463 3246 Acked Read 6 Done Done Sent false
8464 3247 Acked Write 5 Done Done Sent false
8465 3248 Acked Read 8 Done Done Sent false
8466 3249 Acked Read 2 Done Done Sent false
8467 3250 Acked Write 8 Done Done Sent false
8468 3251 Acked Write 1 Done Done Sent false
8469 3252 Acked Read 8 Done Done Sent false
8470 3253 Acked Read 4 Done Done Sent false
8471 3254 Acked Write 7 Done Done Sent false
8472 3255 Acked Read 8 Done Done Sent false
8473 3256 Acked Write 8 Done Done Sent false
8474 3257 Acked Read 7 Done Done Sent false
8475 3258 Acked Write 3 Done Done Sent false
8476 3259 Acked Write 9 Done Done Sent false
8477 3260 Acked Write 1 Done Done Sent false
8478 3261 Acked Write 2 Done Done Sent false
8479 3262 Acked Read 3 Done Done Sent false
8480 3263 Acked Read 9 Done Done Sent false
8481 3264 Acked Flush 0 Done Done Sent false
8482 3265 Acked Read 9 Done Done Sent false
8483 3266 Acked Read 3 Done Done Sent false
8484 3267 Acked Write 10 Done Done Sent false
8485 3268 Acked Flush 0 Done Done Sent false
8486 3269 Acked Read 6 Done Done Sent false
8487 3270 Acked Write 8 Done Done Sent false
8488 3271 Acked Read 9 Done Done Sent false
8489 3272 Acked Read 4 Done Done Sent false
8490 3273 Acked Read 1 Done Done Sent false
8491 3274 Acked Flush 0 Done Done Sent false
8492 3275 Acked Write 7 Done Done Sent false
8493 3276 Acked Read 10 Done Done Sent false
8494 3277 Acked Read 8 Done Done Sent false
8495 3278 Acked Read 9 Done Done Sent false
8496 3279 Acked Read 3 Done Done Sent false
8497 3280 Acked Read 9 Done Done Sent false
8498 3281 Acked Flush 0 Done Done Sent false
8499 3282 Acked Write 4 Done Done Sent false
8500 3283 Acked Write 1 Done Done Sent false
8501 3284 Acked Read 1 Done Done Sent false
8502 3285 Acked Read 10 Done Done Sent false
8503 3286 Acked Flush 0 Done Done Sent false
8504 3287 Acked Read 5 Done Done Sent false
8505 3288 Acked Flush 0 Done Done Sent false
8506 3289 Acked Write 7 Done Done Sent false
8507 3290 Acked Flush 0 Done Done Sent false
8508 3291 Acked Write 4 Done Done Sent false
8509 3292 Acked Flush 0 Done Done Sent false
8510 3293 Acked Write 1 Done Done Sent false
8511 3294 Acked Read 4 Done Done Sent false
8512 3295 Acked Write 10 Done Done Sent false
8513 3296 Acked Read 6 Done Done Sent false
8514 3297 Acked Read 3 Done Done Sent false
8515 3298 Acked Read 9 Done Done Sent false
8516 3299 Acked Write 8 Done Done Sent false
8517 3300 Acked Read 8 Done Done Sent false
8518 3301 Acked Read 7 Done Done Sent false
8519 3302 Acked Write 3 Done Done Sent false
8520 3303 Acked Read 2 Done Done Sent false
8521 3304 Acked Read 9 Done Done Sent false
8522 3305 Acked Read 7 Done Done Sent false
8523 3306 Acked Write 4 Done Done Sent false
8524 3307 Acked Read 10 Done Done Sent false
8525 3308 Acked Read 6 Done Done Sent false
8526 3309 Acked Read 3 Done Done Sent false
8527 3310 Acked Write 8 Done Done Sent false
8528 3311 Acked Read 10 Done Done Sent false
8529 3312 Acked Write 1 Done Done Sent false
8530 3313 Acked Write 2 Done Done Sent false
8531 3314 Acked Read 5 Done Done Sent false
8532 3315 Acked Read 2 Done Done Sent false
8533 3316 Acked Write 4 Done Done Sent false
8534 3317 Acked Write 8 Done Done Sent false
8535 3318 Acked Read 9 Done Done Sent false
8536 3319 Acked Read 7 Done Done Sent false
8537 3320 Acked Write 5 Done Done Sent false
8538 3321 Acked Write 9 Done Done Sent false
8539 3322 Acked Write 2 Done Done Sent false
8540 3323 Acked Write 5 Done Done Sent false
8541 3324 Acked Write 2 Done Done Sent false
8542 3325 Acked Flush 0 Done Done Sent false
8543 3326 Acked Read 4 Done Done Sent false
8544 3327 Acked Read 6 Done Done Sent false
8545 3328 Acked Read 5 Done Done Sent false
8546 3329 Acked Read 1 Done Done Sent false
8547 3330 Acked Read 6 Done Done Sent false
8548 3331 Acked Write 10 Done Done Sent false
8549 3332 Acked Write 4 Done Done Sent false
8550 3333 Acked Write 8 Done Done Sent false
8551 3334 Acked Read 10 Done Done Sent false
8552 3335 Acked Flush 0 Done Done Sent false
8553 3336 Acked Write 3 Done Done Sent false
8554 3337 Acked Read 5 Done Done Sent false
8555 3338 Acked Read 7 Done Done Sent false
8556 3339 Acked Read 10 Done Done Sent false
8557 3340 Acked Write 2 Done Done Sent false
8558 3341 Acked Read 9 Done Done Sent false
8559 3342 Acked Write 4 Done Done Sent false
8560 3343 Acked Read 2 Done Done Sent false
8561 3344 Acked Read 3 Done Done Sent false
8562 3345 Acked Read 10 Done Done Sent false
8563 3346 Acked Write 6 Done Done Sent false
8564 3347 Acked Write 2 Done Done Sent false
8565 3348 Acked Write 1 Done Done Sent false
8566 3349 Acked Write 5 Done Done Sent false
8567 3350 Acked Flush 0 Done Done Sent false
8568 3351 Acked Read 4 Done Done Sent false
8569 3352 Acked Write 5 Done Done Sent false
8570 3353 Acked Write 6 Done Done Sent false
8571 3354 Acked Write 6 Done Done Sent false
8572 3355 Acked Read 2 Done Done Sent false
8573 3356 Acked Read 5 Done Done Sent false
8574 3357 Acked Write 8 Done Done Sent false
8575 3358 Acked Write 3 Done Done Sent false
8576 3359 Acked Write 3 Done Done Sent false
8577 3360 Acked Flush 0 Done Done Sent false
8578 3361 Acked Read 5 Done Done Sent false
8579 3362 Acked Write 4 Done Done Sent false
8580 3363 Acked Flush 0 Done Done Sent false
8581 3364 Acked Write 7 Done Done Sent false
8582 3365 Acked Read 9 Done Done Sent false
8583 3366 Acked Read 5 Done Done Sent false
8584 3367 Acked Flush 0 Done Done Sent false
8585 3368 Acked Read 10 Done Done Sent false
8586 3369 Acked Read 6 Done Done Sent false
8587 3370 Acked Read 3 Done Done Sent false
8588 3371 Acked Write 4 Done Done Sent false
8589 3372 Acked Flush 0 Done Done Sent false
8590 3373 Acked Write 10 Done Done Sent false
8591 3374 Acked Read 2 Done Done Sent false
8592 3375 Acked Read 7 Done Done Sent false
8593 3376 Acked Write 6 Done Done Sent false
8594 3377 Acked Read 9 Done Done Sent false
8595 3378 Acked Write 2 Done Done Sent false
8596 3379 Acked Flush 0 Done Done Sent false
8597 3380 Acked Write 7 Done Done Sent false
8598 3381 Acked Read 2 Done Done Sent false
8599 3382 Acked Write 7 Done Done Sent false
8600 3383 Acked Write 3 Done Done Sent false
8601 3384 Acked Read 10 Done Done Sent false
8602 3385 Acked Write 1 Done Done Sent false
8603 3386 Acked Write 1 Done Done Sent false
8604 3387 Acked Read 4 Done Done Sent false
8605 3388 Acked Read 9 Done Done Sent false
8606 3389 Acked Write 1 Done Done Sent false
8607 3390 Acked Flush 0 Done Done Sent false
8608 3391 Acked Read 10 Done Done Sent false
8609 3392 Acked Read 6 Done Done Sent false
8610 3393 Acked Flush 0 Done Done Sent false
8611 3394 Acked Read 3 Done Done Sent false
8612 3395 Acked Read 8 Done Done Sent false
8613 3396 Acked Write 6 Done Done Sent false
8614 3397 Acked Read 10 Done Done Sent false
8615 3398 Acked Read 5 Done Done Sent false
8616 3399 Acked Read 1 Done Done Sent false
8617 3400 Acked Read 10 Done Done Sent false
8618 3401 Acked Read 4 Done Done Sent false
8619 3402 Acked Write 8 Done Done Sent false
8620 3403 Acked Flush 0 Done Done Sent false
8621 3404 Acked Read 2 Done Done Sent false
8622 3405 Acked Flush 0 Done Done Sent false
8623 3406 Acked Read 8 Done Done Sent false
8624 3407 Acked Read 6 Done Done Sent false
8625 3408 Acked Read 8 Done Done Sent false
8626 3409 Acked Read 1 Done Done Sent false
8627 3410 Acked Read 3 Done Done Sent false
8628 3411 Acked Read 9 Done Done Sent false
8629 3412 Acked Write 5 Done Done Sent false
8630 3413 Acked Write 7 Done Done Sent false
8631 3414 Acked Read 1 Done Done Sent false
8632 3415 Acked Flush 0 Done Done Sent false
8633 STATES DS:0 DS:1 DS:2 TOTAL
8634 Sent 0 0 302 302
8635 Done 302 302 0 604
8636 Skipped 0 0 0 0
8637 Error 0 0 0 0
8638 Last Flush: 3415 3415 3113
8639 Downstairs last five completed: 3113 3112 3111 3110 3109
8640 Upstairs last five completed: 3415 3414 3413 3412 3411
8641 CLIENT: Up:0 ds:302 act:2
86422024-11-28T04:52:08.700ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
86432024-11-28T04:52:08.700ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:56860) = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
86442024-11-28T04:52:08.701ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
86452024-11-28T04:52:08.702ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
86462024-11-28T04:52:08.702ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
86472024-11-28T04:52:08.702ZINFOcrucible: send last flush ID to this DS: 3113 = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
86482024-11-28T04:52:08.702ZINFOcrucible: Replied this last flush ID: 3113 = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
86492024-11-28T04:52:08.702ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
86502024-11-28T04:52:08.702ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
86512024-11-28T04:52:08.702ZINFOcrucible: [2] client re-new 302 jobs since flush 3113 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
86522024-11-28T04:52:08.703ZINFOcrucible: [2] Replayed 302 jobs since flush: 3113 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
8653 ----------------------------------------------------------------
8654 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
8655 Downstairs last five completed: 3415 3414 3413 3412 3411
8656 Upstairs last five completed: 3415 3414 3413 3412 3411
8657 CLIENT: Up:0 ds:0 act:3
8658 Replay: All jobs finished
8659 Main task has work to do, go find it
8660 got dsc Stop(0)
8661 stop 0
8662 Nov 28 04:52:08.945 INFO request completed, latency_us[8810] Got stop action so:false kr:false
8663 : 358, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 22515810-1570-40b0-a238-f653e8469785, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
8664 Nov 28 04:52:08.945 INFO accepted connection, remote_addr: 127.0.0.1:39672, local_addr: 127.0.0.1:9998
86652024-11-28T04:52:08.945ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
86662024-11-28T04:52:08.945ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
86672024-11-28T04:52:08.945ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
8668 Nov 28 04:52:08.945 INFO request completed, {"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-28T04:52:08.945913824Z","hostname":"w-01JDRHKEEFJ1228TFPSBEGXWHY","pid":1202,"client":"0","":"downstairs","session_id":"4a2415b1-6594-436f-82e5-b8cf9a74eeaf"}
8669 latency_us: 128, response_code: 200, uri: /state/cid/0, method: GET, req_id: 70e96953-9379-4c59-ae5d-39a02f8452b4, remote_addr: 127.0.0.1:60418, {"msg":"[0] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-28T04:52:08.945965577Z","hostname":"w-01JDRHKEEFJ1228TFPSBEGXWHY","pid":1202,"client":"0","":"downstairs","session_id":"4a2415b1-6594-436f-82e5-b8cf9a74eeaf"}
8670 local_addr: 127.0.0.1:9998
86712024-11-28T04:52:08.945ZINFOcrucible: client task is exiting = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
8672 [8810] Exited with: ExitStatus(unix_wait_status(9))
8673 [8810][0] reports Exit
8674 Nov 28 04:52:12.947 INFO request completed, latency_us: 220, response_code: 200, uri: /state/cid/0, method: GET, req_id: 3636a9bf-81c3-49ba-99fc-b4220c701e9c, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
8675 001/300 Write block 4247 len 4096 data: 4
8676 002/300 Read block 999 len 32768
8677 003/300 Write block 4546 len 20480 data: 3 3 3 4 3
8678 004/300 Flush
8679 005/300 Write block 1762 len 12288 data: 3 3 3
8680 006/300 Read block 2017 len 32768
8681 007/300 Write block 367 len 16384 data: 4 4 4 4
8682 008/300 Write block 2507 len 36864 data: 3 2 2 2 2 2 2 3 3
8683 009/300 Write block 3283 len 32768 data: 4 4 5 5 5 5 4 4
8684 010/300 Read block 4955 len 24576
8685 011/300 Read block 4249 len 16384
8686 012/300 Read block 2143 len 28672
8687 013/300 Read block 3082 len 24576
8688 014/300 Write block 1369 len 4096 data: 2
8689 015/300 Read block 4557 len 20480
8690 016/300 Read block 1688 len 4096
8691 017/300 Read block 1137 len 16384
8692 018/300 Write block 2364 len 40960 data: 5 3 2 2 3 3 3 3 3 4
8693 019/300 Read block 466 len 28672
8694 020/300 Read block 3009 len 4096
8695 021/300 Read block 3866 len 16384
8696 022/300 Read block 4784 len 32768
8697 023/300 Write block 116 len 20480 data: 2 2 2 2 3
8698 024/300 Write block 4153 len 40960 data: 5 5 5 4 3 3 4 4 4 3
8699 025/300 Read block 1521 len 32768
8700 026/300 Write block 2337 len 32768 data: 3 3 2 2 2 2 2 2
8701 027/300 Flush
8702 028/300 Write block 2663 len 40960 data: 2 2 2 2 2 2 2 3 2 2
8703 029/300 Write block 1639 len 24576 data: 2 2 2 2 2 2
8704 030/300 Read block 340 len 4096
8705 031/300 Read block 75 len 16384
8706 032/300 Write block 1270 len 24576 data: 2 3 3 4 4 4
8707 033/300 Read block 1794 len 4096
8708 034/300 Write block 2112 len 8192 data: 7 7
8709 035/300 Write block 3778 len 24576 data: 3 3 3 2 2 2
8710 036/300 Write block 2498 len 28672 data: 2 2 2 2 2 2 2
8711 037/300 Read block 3294 len 12288
8712 038/300 Read block 116 len 40960
8713 039/300 Write block 748 len 28672 data: 2 2 2 2 2 2 3
8714 040/300 Read block 3683 len 8192
8715 041/300 Read block 2147 len 36864
8716 042/300 Write block 1265 len 40960 data: 3 3 3 2 2 3 4 4 5 5
8717 043/300 Write block 1322 len 20480 data: 2 2 2 2 2
8718 044/300 Write block 650 len 4096 data: 3
8719 045/300 Read block 167 len 4096
8720 046/300 Write block 649 len 12288 data: 3 4 4
8721 047/300 Read block 2817 len 28672
8722 048/300 Write block 4356 len 40960 data: 3 3 3 3 3 4 3 3 3 3
8723 049/300 Read block 4955 len 4096
8724 050/300 Read block 3408 len 12288
8725 051/300 Read block 3102 len 32768
8726 052/300 Read block 1956 len 24576
8727 053/300 Read block 3742 len 20480
8728 054/300 Read block 1874 len 32768
8729 055/300 Write block 2319 len 32768 data: 3 3 3 2 2 3 4 4
8730 056/300 Flush
8731 057/300 Write block 1233 len 28672 data: 2 2 2 2 2 2 2
8732 058/300 Flush
8733 059/300 Read block 623 len 24576
8734 060/300 Write block 2771 len 32768 data: 3 3 4 3 4 4 3 3
8735 061/300 Read block 3886 len 36864
8736 062/300 Read block 2515 len 28672
8737 063/300 Write block 2407 len 40960 data: 2 3 3 3 2 2 2 2 2 2
8738 064/300 Read block 2230 len 32768
8739 065/300 Read block 1799 len 32768
8740 066/300 Read block 1466 len 12288
8741 067/300 Write block 1396 len 16384 data: 3 2 3 4
8742 068/300 Flush
8743 069/300 Write block 879 len 40960 data: 5 5 6 4 4 4 4 3 3 2
8744 070/300 Read block 3621 len 4096
8745 071/300 Read block 985 len 20480
8746 072/300 Read block 3657 len 32768
8747 073/300 Read block 2756 len 32768
8748 074/300 Write block 891 len 8192 data: 2 2
8749 075/300 Read block 2381 len 8192
8750 076/300 Read block 2820 len 12288
8751 077/300 Write block 1706 len 24576 data: 7 8 9 10 9 8
8752 078/300 Read block 4580 len 28672
8753 079/300 Read block 1074 len 28672
8754 080/300 Read block 1542 len 32768
8755 081/300 Read block 4271 len 28672
8756 082/300 Read block 4234 len 36864
8757 083/300 Write block 3608 len 12288 data: 3 3 2
8758 084/300 Flush
8759 085/300 Flush
8760 086/300 Read block 3260 len 8192
8761 087/300 Write block 2521 len 4096 data: 4
8762 088/300 Write block 3237 len 12288 data: 3 3 4
8763 089/300 Read block 2717 len 28672
8764 090/300 Write block 3454 len 40960 data: 3 2 2 2 2 2 2 2 2 2
8765 091/300 Write block 4960 len 8192 data: 2 2
8766 092/300 Write block 1768 len 40960 data: 4 4 4 4 3 2 2 2 3 2
8767 093/300 Write block 378 len 24576 data: 5 5 5 4 4 4
8768 094/300 Read block 3665 len 20480
8769 095/300 Flush
8770 096/300 Read block 3414 len 24576
8771 097/300 Write block 1497 len 24576 data: 3 3 3 3 3 2
8772 098/300 Read block 0 len 28672
8773 099/300 Read block 2928 len 24576
8774 100/300 Read block 1337 len 8192
8775 101/300 Read block 2125 len 24576
8776 102/300 Read block 3382 len 20480
8777 103/300 Write block 2047 len 40960 data: 2 2 2 3 3 3 3 2 2 2
8778 104/300 Read block 880 len 24576
8779 105/300 Write block 763 len 16384 data: 4 4 4 3
8780 106/300 Read block 3539 len 40960
8781 107/300 Read block 1708 len 8192
8782 108/300 Write block 4725 len 24576 data: 3 3 2 2 2 2
8783 109/300 Write block 2022 len 36864 data: 3 3 3 3 3 3 4 4 4
8784 110/300 Read block 4185 len 32768
8785 111/300 Read block 4756 len 40960
8786 112/300 Read block 4926 len 20480
8787 113/300 Flush
8788 114/300 Write block 1591 len 36864 data: 2 2 2 2 2 2 2 2 2
8789 115/300 Write block 4379 len 8192 data: 2 2
8790 116/300 Read block 1847 len 20480
8791 117/300 Read block 2829 len 24576
8792 118/300 Read block 1955 len 32768
8793 119/300 Read block 2244 len 16384
8794 120/300 Read block 4101 len 20480
8795 121/300 Write block 1684 len 32768 data: 2 2 2 2 2 2 2 2
8796 122/300 Read block 715 len 12288
8797 123/300 Write block 2123 len 32768 data: 2 2 2 2 2 2 2 2
8798 124/300 Read block 3793 len 36864
8799 125/300 Flush
8800 126/300 Write block 446 len 24576 data: 3 3 2 2 2 2
8801 127/300 Read block 3381 len 4096
8802 128/300 Read block 2018 len 36864
8803 129/300 Write block 4929 len 8192 data: 2 2
8804 130/300 Flush
8805 131/300 Read block 2452 len 16384
8806 132/300 Read block 1977 len 32768
8807 133/300 Read block 1667 len 12288
8808 134/300 Read block 4851 len 4096
8809 135/300 Write block 3465 len 16384 data: 2 2 2 2
8810 136/300 Write block 4328 len 20480 data: 3 3 2 2 2
8811 137/300 Write block 1975 len 32768 data: 3 2 2 3 3 3 3 4
8812 138/300 Read block 830 len 40960
8813 139/300 Flush
8814 140/300 Read block 877 len 12288
8815 141/300 Read block 2231 len 4096
8816 142/300 Write block 126 len 24576 data: 3 3 3 3 3 3
8817 143/300 Read block 2050 len 32768
8818 144/300 Write block 340 len 12288 data: 4 4 4
8819 145/300 Read block 3620 len 8192
8820 146/300 Write block 310 len 24576 data: 4 3 3 3 3 2
8821 147/300 Write block 4380 len 32768 data: 3 2 2 2 3 2 2 3
8822 148/300 Read block 4994 len 4096
8823 149/300 Read block 101 len 36864
8824 150/300 Read block 805 len 4096
8825 151/300 Flush
8826 152/300 Write block 4075 len 40960 data: 2 2 3 5 6 5 5 4 3 4
8827 153/300 Write block 4714 len 8192 data: 2 2
8828 154/300 Read block 1439 len 28672
8829 155/300 Write block 1272 len 12288 data: 5 6 6
8830 156/300 Read block 491 len 20480
8831 157/300 Flush
8832 158/300 Write block 4830 len 4096 data: 2
8833 159/300 Write block 4912 len 4096 data: 2
8834 160/300 Write block 1476 len 28672 data: 3 3 3 3 3 3 3
8835 161/300 Read block 282 len 8192
8836 162/300 Write block 4593 len 32768 data: 2 3 3 4 4 4 4 4
8837 163/300 Write block 1598 len 20480 data: 3 3 2 3 4
8838 164/300 Read block 2868 len 12288
8839 165/300 Write block 3967 len 16384 data: 3 2 2 2
8840 166/300 Write block 2599 len 12288 data: 3 3 3
8841 167/300 Read block 2614 len 16384
8842 168/300 Read block 506 len 32768
8843 169/300 Write block 3432 len 20480 data: 4 4 3 2 3
8844 170/300 Write block 3129 len 20480 data: 2 2 2 2 3
8845 171/300 Read block 1059 len 16384
8846 172/300 Flush
8847 173/300 Read block 3806 len 28672
8848 174/300 Read block 4497 len 16384
8849 175/300 Write block 3203 len 36864 data: 2 2 4 4 4 3 4 3 3
8850 176/300 Write block 1957 len 16384 data: 2 2 2 2
8851 177/300 Read block 1228 len 24576
8852 178/300 Read block 1579 len 28672
8853 179/300 Write block 1303 len 36864 data: 2 2 2 2 2 2 2 2 2
8854 180/300 Read block 1818 len 40960
8855 181/300 Write block 3620 len 40960 data: 2 2 2 2 2 2 2 2 3 3
8856 182/300 Write block 3854 len 24576 data: 3 3 3 2 2 2
8857 183/300 Write block 4118 len 20480 data: 3 3 3 3 5
8858 184/300 Write block 1497 len 4096 data: 4
8859 185/300 Read block 2018 len 24576
8860 186/300 Write block 3090 len 20480 data: 2 2 2 2 2
8861 187/300 Write block 2579 len 4096 data: 2
8862 188/300 Read block 4913 len 36864
8863 189/300 Flush
8864 190/300 Read block 2930 len 36864
8865 191/300 Write block 4243 len 32768 data: 3 3 4 4 5 4 3 3
8866 192/300 Write block 1242 len 20480 data: 3 4 4 4 3
8867 193/300 Read block 4229 len 36864
8868 194/300 Read block 3401 len 36864
8869 195/300 Read block 4334 len 36864
8870 196/300 Write block 4266 len 20480 data: 5 4 4 4 4
8871 197/300 Write block 3491 len 36864 data: 3 3 3 3 3 3 3 3 3
8872 198/300 Read block 304 len 16384
8873 199/300 Read block 2822 len 36864
8874 200/300 Write block 451 len 24576 data: 3 2 2 2 2 2
8875 201/300 Write block 2789 len 24576 data: 8 8 5 5 3 3
8876 202/300 Read block 3958 len 12288
8877 203/300 Read block 2507 len 4096
8878 204/300 Read block 4380 len 36864
8879 205/300 Write block 1806 len 8192 data: 2 2
8880 206/300 Write block 2078 len 36864 data: 2 2 2 2 2 3 3 3 3
8881 207/300 Flush
8882 208/300 Read block 3983 len 28672
8883 209/300 Write block 1220 len 20480 data: 2 2 2 2 2
8884 210/300 Write block 3253 len 24576 data: 3 3 3 3 3 3
8885 211/300 Flush
8886 212/300 Write block 2021 len 16384 data: 3 4 4 4
8887 213/300 Read block 4507 len 8192
8888 214/300 Write block 2877 len 32768 data: 3 3 3 3 3 3 2 2
8889 215/300 Write block 4300 len 8192 data: 2 2
8890 216/300 Read block 2892 len 36864
8891 217/300 Write block 30 len 32768 data: 3 3 3 3 2 2 2 2
8892 218/300 Read block 3265 len 36864
8893 219/300 Flush
8894 220/300 Read block 136 len 12288
8895 221/300 Read block 3069 len 32768
8896 222/300 Write block 3226 len 40960 data: 4 3 2 2 2 2 2 2 2 2
8897 223/300 Write block 4747 len 36864 data: 2 2 2 2 2 2 2 2 2
8898 224/300 Read block 4345 len 24576
8899 225/300 Read block 1562 len 40960
8900 226/300 Read block 3574 len 40960
8901 227/300 Read block 983 len 40960
8902 228/300 Read block 3779 len 24576
8903 229/300 Flush
8904 230/300 Flush
8905 231/300 Read block 1277 len 16384
8906 232/300 Write block 596 len 8192 data: 5 6
8907 233/300 Write block 3866 len 32768 data: 2 2 2 2 2 2 2 2
8908 234/300 Read block 2383 len 16384
8909 235/300 Read block 3664 len 4096
8910 236/300 Read block 382 len 36864
8911 237/300 Flush
8912 238/300 Read block 2284 len 36864
8913 239/300 Flush
8914 240/300 Read block 1452 len 36864
8915 241/300 Read block 826 len 28672
8916 242/300 Write block 4923 len 8192 data: 3 3
8917 243/300 Write block 1097 len 20480 data: 4 5 3 4 4
8918 244/300 Write block 4255 len 8192 data: 2 2
8919 245/300 Write block 2847 len 24576 data: 3 3 3 4 4 4
8920 246/300 Read block 2550 len 16384
8921 247/300 Read block 4492 len 28672
8922 248/300 Write block 151 len 40960 data: 4 3 3 3 3 4 4 3 3 3
8923 249/300 Read block 1231 len 12288
8924 250/300 Read block 1006 len 40960
8925 251/300 Write block 4848 len 40960 data: 5 6 5 6 5 4 4 4 2 3
8926 252/300 Write block 908 len 4096 data: 3
8927 253/300 Write block 2743 len 36864 data: 2 2 2 3 3 3 4 5 5
8928 254/300 Write block 3862 len 32768 data: 2 2 2 2 3 3 3 3
8929 255/300 Read block 4882 len 36864
8930 256/300 Write block 2177 len 12288 data: 3 3 3
8931 257/300 Write block 853 len 32768 data: 3 3 2 2 2 4 4 3
8932 258/300 Write block 512 len 28672 data: 3 3 2 2 2 2 2
8933 259/300 Read block 1264 len 12288
8934 260/300 Write block 3759 len 36864 data: 2 2 4 3 3 3 3 3 3
8935 261/300 Write block 446 len 4096 data: 4
8936 262/300 Write block 4358 len 36864 data: 4 4 4 5 4 4 4 4 3
8937 263/300 Read block 3195 len 24576
8938 264/300 Flush
8939 265/300 Read block 2478 len 12288
8940 266/300 Read block 262 len 28672
8941 267/300 Read block 4868 len 12288
8942 268/300 Read block 611 len 12288
8943 269/300 Write block 1605 len 4096 data: 3
8944 270/300 Write block 1694 len 12288 data: 2 2 2
8945 271/300 Read block 3801 len 40960
8946 272/300 Flush
8947 273/300 Write block 2126 len 8192 data: 3 3
8948 274/300 Read block 1482 len 4096
8949 275/300 Read block 3215 len 32768
8950 276/300 Flush
8951 277/300 Write block 379 len 24576 data: 6 6 5 5 5 3
8952 278/300 Read block 2849 len 24576
8953 279/300 Read block 355 len 8192
8954 280/300 Read block 1170 len 32768
8955 281/300 Read block 3445 len 12288
8956 282/300 Write block 3217 len 8192 data: 4 5
8957 283/300 Read block 336 len 20480
8958 284/300 Read block 2548 len 8192
8959 285/300 Read block 2727 len 28672
8960 286/300 Flush
8961 287/300 Read block 1228 len 12288
8962 288/300 Write block 4672 len 24576 data: 3 3 3 3 4 4
8963 289/300 Read block 3594 len 12288
8964 290/300 Read block 1513 len 24576
8965 291/300 Flush
8966 292/300 Flush
8967 293/300 Write block 3122 len 32768 data: 2 2 2 2 2 2 2 3
8968 294/300 Write block 2522 len 28672 data: 5 5 5 6 6 6 5
8969 295/300 Read block 1876 len 36864
8970 296/300 Write block 485 len 12288 data: 2 2 2
8971 297/300 Read block 193 len 4096
8972 298/300 Read block 4667 len 20480
8973 299/300 Flush
8974 300/300 Write block 3925 len 4096 data: 4
8975 301/300 Flush
8976 Main task has work to do, go find it
8977 got dsc Start(0)
8978 start 0
8979 Nov 28 04:52:13.185 INFO request completed, latency_us: 213, response_code: 204, uri: /start/cid/0, method: POST, req_id: bc6b5370-18e8-48ec-b082-3b0a3dbee87f, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
8980 Replay: started 0, returned:Ok(())
8981 ----------------------------------------------------------------
8982 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
8983 States: Offline Active Active
8984 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
8985 3416 Acked Write 1 Sent Done Done false
8986 3417 Acked Read 8 Sent Done Done false
8987 3418 Acked Write 5 Sent Done Done false
8988 3419 Acked Flush 0 Sent Done Done false
8989 3420 Acked Write 3 Sent Done Done false
8990 3421 Acked Read 8 Sent Done Done false
8991 3422 Acked Write 4 Sent Done Done false
8992 3423 Acked Write 9 Sent Done Done false
8993 3424 Acked Write 8 Sent Done Done false
8994 3425 Acked Read 6 Sent Done Done false
8995 3426 Acked Read 4 Sent Done Done false
8996 3427 Acked Read 7 Sent Done Done false
8997 3428 Acked Read 6 Sent Done Done false
8998 3429 Acked Write 1 Sent Done Done false
8999 3430 Acked Read 5 Sent Done Done false
9000 3431 Acked Read 1 Sent Done Done false
9001 3432 Acked Read 4 Sent Done Done false
9002 3433 Acked Write 10 Sent Done Done false
9003 3434 Acked Read 7 Sent Done Done false
9004 3435 Acked Read 1 Sent Done Done false
9005 3436 Acked Read 4 Sent Done Done false
9006 3437 Acked Read 8 Sent Done Done false
9007 3438 Acked Write 5 Sent Done Done false
9008 3439 Acked Write 10 Sent Done Done false
9009 3440 Acked Read 8 Sent Done Done false
9010 3441 Acked Write 8 Sent Done Done false
9011 3442 Acked Flush 0 Sent Done Done false
9012 3443 Acked Write 10 Sent Done Done false
9013 3444 Acked Write 6 Sent Done Done false
9014 3445 Acked Read 1 Sent Done Done false
9015 3446 Acked Read 4 Sent Done Done false
9016 3447 Acked Write 6 Sent Done Done false
9017 3448 Acked Read 1 Sent Done Done false
9018 3449 Acked Write 2 Sent Done Done false
9019 3450 Acked Write 6 Sent Done Done false
9020 3451 Acked Write 7 Sent Done Done false
9021 3452 Acked Read 3 Sent Done Done false
9022 3453 Acked Read 10 Sent Done Done false
9023 3454 Acked Write 7 Sent Done Done false
9024 3455 Acked Read 2 Sent Done Done false
9025 3456 Acked Read 9 Sent Done Done false
9026 3457 Acked Write 10 Sent Done Done false
9027 3458 Acked Write 5 Sent Done Done false
9028 3459 Acked Write 1 Sent Done Done false
9029 3460 Acked Read 1 Sent Done Done false
9030 3461 Acked Write 3 Sent Done Done false
9031 3462 Acked Read 7 Sent Done Done false
9032 3463 Acked Write 10 Sent Done Done false
9033 3464 Acked Read 1 Sent Done Done false
9034 3465 Acked Read 3 Sent Done Done false
9035 3466 Acked Read 8 Sent Done Done false
9036 3467 Acked Read 6 Sent Done Done false
9037 3468 Acked Read 5 Sent Done Done false
9038 3469 Acked Read 8 Sent Done Done false
9039 3470 Acked Write 8 Sent Done Done false
9040 3471 Acked Flush 0 Sent Done Done false
9041 3472 Acked Write 7 Sent Done Done false
9042 3473 Acked Flush 0 Sent Done Done false
9043 3474 Acked Read 6 Sent Done Done false
9044 3475 Acked Write 8 Sent Done Done false
9045 3476 Acked Read 9 Sent Done Done false
9046 3477 Acked Read 7 Sent Done Done false
9047 3478 Acked Write 10 Sent Done Done false
9048 3479 Acked Read 8 Sent Done Done false
9049 3480 Acked Read 8 Sent Done Done false
9050 3481 Acked Read 3 Sent Done Done false
9051 3482 Acked Write 4 Sent Done Done false
9052 3483 Acked Flush 0 Sent Done Done false
9053 3484 Acked Write 10 Sent Done Done false
9054 3485 Acked Read 1 Sent Done Done false
9055 3486 Acked Read 5 Sent Done Done false
9056 3487 Acked Read 8 Sent Done Done false
9057 3488 Acked Read 8 Sent Done Done false
9058 3489 Acked Write 2 Sent Done Done false
9059 3490 Acked Read 2 Sent Done Done false
9060 3491 Acked Read 3 Sent Done Done false
9061 3492 Acked Write 6 Sent Done Done false
9062 3493 Acked Read 7 Sent Done Done false
9063 3494 Acked Read 7 Sent Done Done false
9064 3495 Acked Read 8 Sent Done Done false
9065 3496 Acked Read 7 Sent Done Done false
9066 3497 Acked Read 9 Sent Done Done false
9067 3498 Acked Write 3 Sent Done Done false
9068 3499 Acked Flush 0 Sent Done Done false
9069 3500 Acked Flush 0 Sent Done Done false
9070 3501 Acked Read 2 Sent Done Done false
9071 3502 Acked Write 1 Sent Done Done false
9072 3503 Acked Write 3 Sent Done Done false
9073 3504 Acked Read 7 Sent Done Done false
9074 3505 Acked Write 10 Sent Done Done false
9075 3506 Acked Write 2 Sent Done Done false
9076 3507 Acked Write 10 Sent Done Done false
9077 3508 Acked Write 6 Sent Done Done false
9078 3509 Acked Read 5 Sent Done Done false
9079 3510 Acked Flush 0 Sent Done Done false
9080 3511 Acked Read 6 Sent Done Done false
9081 3512 Acked Write 6 Sent Done Done false
9082 3513 Acked Read 7 Sent Done Done false
9083 3514 Acked Read 6 Sent Done Done false
9084 3515 Acked Read 2 Sent Done Done false
9085 3516 Acked Read 6 Sent Done Done false
9086 3517 Acked Read 5 Sent Done Done false
9087 3518 Acked Write 10 Sent Done Done false
9088 3519 Acked Read 6 Sent Done Done false
9089 3520 Acked Write 4 Sent Done Done false
9090 3521 Acked Read 10 Sent Done Done false
9091 3522 Acked Read 2 Sent Done Done false
9092 3523 Acked Write 6 Sent Done Done false
9093 3524 Acked Write 9 Sent Done Done false
9094 3525 Acked Read 8 Sent Done Done false
9095 3526 Acked Read 10 Sent Done Done false
9096 3527 Acked Read 5 Sent Done Done false
9097 3528 Acked Flush 0 Sent Done Done false
9098 3529 Acked Write 9 Sent Done Done false
9099 3530 Acked Write 2 Sent Done Done false
9100 3531 Acked Read 5 Sent Done Done false
9101 3532 Acked Read 6 Sent Done Done false
9102 3533 Acked Read 8 Sent Done Done false
9103 3534 Acked Read 4 Sent Done Done false
9104 3535 Acked Read 5 Sent Done Done false
9105 3536 Acked Write 8 Sent Done Done false
9106 3537 Acked Read 3 Sent Done Done false
9107 3538 Acked Write 8 Sent Done Done false
9108 3539 Acked Read 9 Sent Done Done false
9109 3540 Acked Flush 0 Sent Done Done false
9110 3541 Acked Write 6 Sent Done Done false
9111 3542 Acked Read 1 Sent Done Done false
9112 3543 Acked Read 9 Sent Done Done false
9113 3544 Acked Write 2 Sent Done Done false
9114 3545 Acked Flush 0 Sent Done Done false
9115 3546 Acked Read 4 Sent Done Done false
9116 3547 Acked Read 8 Sent Done Done false
9117 3548 Acked Read 3 Sent Done Done false
9118 3549 Acked Read 1 Sent Done Done false
9119 3550 Acked Write 4 Sent Done Done false
9120 3551 Acked Write 5 Sent Done Done false
9121 3552 Acked Write 8 Sent Done Done false
9122 3553 Acked Read 10 Sent Done Done false
9123 3554 Acked Flush 0 Sent Done Done false
9124 3555 Acked Read 3 Sent Done Done false
9125 3556 Acked Read 1 Sent Done Done false
9126 3557 Acked Write 6 Sent Done Done false
9127 3558 Acked Read 8 Sent Done Done false
9128 3559 Acked Write 3 Sent Done Done false
9129 3560 Acked Read 2 Sent Done Done false
9130 3561 Acked Write 6 Sent Done Done false
9131 3562 Acked Write 8 Sent Done Done false
9132 3563 Acked Read 1 Sent Done Done false
9133 3564 Acked Read 9 Sent Done Done false
9134 3565 Acked Read 1 Sent Done Done false
9135 3566 Acked Flush 0 Sent Done Done false
9136 3567 Acked Write 10 Sent Done Done false
9137 3568 Acked Write 2 Sent Done Done false
9138 3569 Acked Read 7 Sent Done Done false
9139 3570 Acked Write 3 Sent Done Done false
9140 3571 Acked Read 5 Sent Done Done false
9141 3572 Acked Flush 0 Sent Done Done false
9142 3573 Acked Write 1 Sent Done Done false
9143 3574 Acked Write 1 Sent Done Done false
9144 3575 Acked Write 7 Sent Done Done false
9145 3576 Acked Read 2 Sent Done Done false
9146 3577 Acked Write 8 Sent Done Done false
9147 3578 Acked Write 5 Sent Done Done false
9148 3579 Acked Read 3 Sent Done Done false
9149 3580 Acked Write 4 Sent Done Done false
9150 3581 Acked Write 3 Sent Done Done false
9151 3582 Acked Read 4 Sent Done Done false
9152 3583 Acked Read 8 Sent Done Done false
9153 3584 Acked Write 5 Sent Done Done false
9154 3585 Acked Write 5 Sent Done Done false
9155 3586 Acked Read 4 Sent Done Done false
9156 3587 Acked Flush 0 Sent Done Done false
9157 3588 Acked Read 7 Sent Done Done false
9158 3589 Acked Read 4 Sent Done Done false
9159 3590 Acked Write 9 Sent Done Done false
9160 3591 Acked Write 4 Sent Done Done false
9161 3592 Acked Read 6 Sent Done Done false
9162 3593 Acked Read 7 Sent Done Done false
9163 3594 Acked Write 9 Sent Done Done false
9164 3595 Acked Read 10 Sent Done Done false
9165 3596 Acked Write 10 Sent Done Done false
9166 3597 Acked Write 6 Sent Done Done false
9167 3598 Acked Write 5 Sent Done Done false
9168 3599 Acked Write 1 Sent Done Done false
9169 3600 Acked Read 6 Sent Done Done false
9170 3601 Acked Write 5 Sent Done Done false
9171 3602 Acked Write 1 Sent Done Done false
9172 3603 Acked Read 9 Sent Done Done false
9173 3604 Acked Flush 0 Sent Done Done false
9174 3605 Acked Read 9 Sent Done Done false
9175 3606 Acked Write 8 Sent Done Done false
9176 3607 Acked Write 5 Sent Done Done false
9177 3608 Acked Read 9 Sent Done Done false
9178 3609 Acked Read 9 Sent Done Done false
9179 3610 Acked Read 9 Sent Done Done false
9180 3611 Acked Write 5 Sent Done Done false
9181 3612 Acked Write 9 Sent Done Done false
9182 3613 Acked Read 4 Sent Done Done false
9183 3614 Acked Read 9 Sent Done Done false
9184 3615 Acked Write 6 Sent Done Done false
9185 3616 Acked Write 6 Sent Done Done false
9186 3617 Acked Read 3 Sent Done Done false
9187 3618 Acked Read 1 Sent Done Done false
9188 3619 Acked Read 9 Sent Done Done false
9189 3620 Acked Write 2 Sent Done Done false
9190 3621 Acked Write 9 Sent Done Done false
9191 3622 Acked Flush 0 Sent Done Done false
9192 3623 Acked Read 7 Sent Done Done false
9193 3624 Acked Write 5 Sent Done Done false
9194 3625 Acked Write 6 Sent Done Done false
9195 3626 Acked Flush 0 Sent Done Done false
9196 3627 Acked Write 4 Sent Done Done false
9197 3628 Acked Read 2 Sent Done Done false
9198 3629 Acked Write 8 Sent Done Done false
9199 3630 Acked Write 2 Sent Done Done false
9200 3631 Acked Read 9 Sent Done Done false
9201 3632 Acked Write 8 Sent Done Done false
9202 3633 Acked Read 9 Sent Done Done false
9203 3634 Acked Flush 0 Sent Done Done false
9204 3635 Acked Read 3 Sent Done Done false
9205 3636 Acked Read 8 Sent Done Done false
9206 3637 Acked Write 10 Sent Done Done false
9207 3638 Acked Write 9 Sent Done Done false
9208 3639 Acked Read 6 Sent Done Done false
9209 3640 Acked Read 10 Sent Done Done false
9210 3641 Acked Read 10 Sent Done Done false
9211 3642 Acked Read 10 Sent Done Done false
9212 3643 Acked Read 6 Sent Done Done false
9213 3644 Acked Flush 0 Sent Done Done false
9214 3645 Acked Flush 0 Sent Done Done false
9215 3646 Acked Read 4 Sent Done Done false
9216 3647 Acked Write 2 Sent Done Done false
9217 3648 Acked Write 8 Sent Done Done false
9218 3649 Acked Read 4 Sent Done Done false
9219 3650 Acked Read 1 Sent Done Done false
9220 3651 Acked Read 9 Sent Done Done false
9221 3652 Acked Flush 0 Sent Done Done false
9222 3653 Acked Read 9 Sent Done Done false
9223 3654 Acked Flush 0 Sent Done Done false
9224 3655 Acked Read 9 Sent Done Done false
9225 3656 Acked Read 7 Sent Done Done false
9226 3657 Acked Write 2 Sent Done Done false
9227 3658 Acked Write 5 Sent Done Done false
9228 3659 Acked Write 2 Sent Done Done false
9229 3660 Acked Write 6 Sent Done Done false
9230 3661 Acked Read 4 Sent Done Done false
9231 3662 Acked Read 7 Sent Done Done false
9232 3663 Acked Write 10 Sent Done Done false
9233 3664 Acked Read 3 Sent Done Done false
9234 3665 Acked Read 10 Sent Done Done false
9235 3666 Acked Write 10 Sent Done Done false
9236 3667 Acked Write 1 Sent Done Done false
9237 3668 Acked Write 9 Sent Done Done false
9238 3669 Acked Write 8 Sent Done Done false
9239 3670 Acked Read 9 Sent Done Done false
9240 3671 Acked Write 3 Sent Done Done false
9241 3672 Acked Write 8 Sent Done Done false
9242 3673 Acked Write 7 Sent Done Done false
9243 3674 Acked Read 3 Sent Done Done false
9244 3675 Acked Write 9 Sent Done Done false
9245 3676 Acked Write 1 Sent Done Done false
9246 3677 Acked Write 9 Sent Done Done false
9247 3678 Acked Read 6 Sent Done Done false
9248 3679 Acked Flush 0 Sent Done Done false
9249 3680 Acked Read 3 Sent Done Done false
9250 3681 Acked Read 7 Sent Done Done false
9251 3682 Acked Read 3 Sent Done Done false
9252 3683 Acked Read 3 Sent Done Done false
9253 3684 Acked Write 1 Sent Done Done false
9254 3685 Acked Write 3 Sent Done Done false
9255 3686 Acked Read 10 Sent Done Done false
9256 3687 Acked Flush 0 Sent Done Done false
9257 3688 Acked Write 2 Sent Done Done false
9258 3689 Acked Read 1 Sent Done Done false
9259 3690 Acked Read 8 Sent Done Done false
9260 3691 Acked Flush 0 Sent Done Done false
9261 3692 Acked Write 6 Sent Done Done false
9262 3693 Acked Read 6 Sent Done Done false
9263 3694 Acked Read 2 Sent Done Done false
9264 3695 Acked Read 8 Sent Done Done false
9265 3696 Acked Read 3 Sent Done Done false
9266 3697 Acked Write 2 Sent Done Done false
9267 3698 Acked Read 5 Sent Done Done false
9268 3699 Acked Read 2 Sent Done Done false
9269 3700 Acked Read 7 Sent Done Done false
9270 3701 Acked Flush 0 Sent Done Done false
9271 3702 Acked Read 3 Sent Done Done false
9272 3703 Acked Write 6 Sent Done Done false
9273 3704 Acked Read 3 Sent Done Done false
9274 3705 Acked Read 6 Sent Done Done false
9275 3706 Acked Flush 0 Sent Done Done false
9276 3707 Acked Flush 0 Sent Done Done false
9277 3708 Acked Write 8 Sent Done Done false
9278 3709 Acked Write 7 Sent Done Done false
9279 3710 Acked Read 9 Sent Done Done false
9280 3711 Acked Write 3 Sent Done Done false
9281 3712 Acked Read 1 Sent Done Done false
9282 3713 Acked Read 5 Sent Done Done false
9283 3714 Acked Flush 0 Sent Done Done false
9284 3715 Acked Write 1 Sent Done Done false
9285 3716 Acked Flush 0 Sent Done Done false
9286 STATES DS:0 DS:1 DS:2 TOTAL
9287 Sent 301 0 0 301
9288 Done 0 301 301 602
9289 Skipped 0 0 0 0
9290 Error 0 0 0 0
9291 Last Flush: 3415 3716 3716
9292 Downstairs last five completed: 3415 3414 3413 3412 3411
9293 Upstairs last five completed: 3716 3715 3714 3713 3712
9294 CLIENT: Up:0 ds:301 act:2
9295 [8810] got start action
9296 [8810] I am going to restart
9297 Starting downstairs at port 8810
9298 Make output file at "/tmp/dsc/downstairs-8810.txt"
9299 [8810][0] reports Starting
9300 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1211)
9301 [8810][0] reports Running
9302 ----------------------------------------------------------------
9303 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
9304 States: Offline Active Active
9305 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
9306 3416 Acked Write 1 Sent Done Done false
9307 3417 Acked Read 8 Sent Done Done false
9308 3418 Acked Write 5 Sent Done Done false
9309 3419 Acked Flush 0 Sent Done Done false
9310 3420 Acked Write 3 Sent Done Done false
9311 3421 Acked Read 8 Sent Done Done false
9312 3422 Acked Write 4 Sent Done Done false
9313 3423 Acked Write 9 Sent Done Done false
9314 3424 Acked Write 8 Sent Done Done false
9315 3425 Acked Read 6 Sent Done Done false
9316 3426 Acked Read 4 Sent Done Done false
9317 3427 Acked Read 7 Sent Done Done false
9318 3428 Acked Read 6 Sent Done Done false
9319 3429 Acked Write 1 Sent Done Done false
9320 3430 Acked Read 5 Sent Done Done false
9321 3431 Acked Read 1 Sent Done Done false
9322 3432 Acked Read 4 Sent Done Done false
9323 3433 Acked Write 10 Sent Done Done false
9324 3434 Acked Read 7 Sent Done Done false
9325 3435 Acked Read 1 Sent Done Done false
9326 3436 Acked Read 4 Sent Done Done false
9327 3437 Acked Read 8 Sent Done Done false
9328 3438 Acked Write 5 Sent Done Done false
9329 3439 Acked Write 10 Sent Done Done false
9330 3440 Acked Read 8 Sent Done Done false
9331 3441 Acked Write 8 Sent Done Done false
9332 3442 Acked Flush 0 Sent Done Done false
9333 3443 Acked Write 10 Sent Done Done false
9334 3444 Acked Write 6 Sent Done Done false
9335 3445 Acked Read 1 Sent Done Done false
9336 3446 Acked Read 4 Sent Done Done false
9337 3447 Acked Write 6 Sent Done Done false
9338 3448 Acked Read 1 Sent Done Done false
9339 3449 Acked Write 2 Sent Done Done false
9340 3450 Acked Write 6 Sent Done Done false
9341 3451 Acked Write 7 Sent Done Done false
9342 3452 Acked Read 3 Sent Done Done false
9343 3453 Acked Read 10 Sent Done Done false
9344 3454 Acked Write 7 Sent Done Done false
9345 3455 Acked Read 2 Sent Done Done false
9346 3456 Acked Read 9 Sent Done Done false
9347 3457 Acked Write 10 Sent Done Done false
9348 3458 Acked Write 5 Sent Done Done false
9349 3459 Acked Write 1 Sent Done Done false
9350 3460 Acked Read 1 Sent Done Done false
9351 3461 Acked Write 3 Sent Done Done false
9352 3462 Acked Read 7 Sent Done Done false
9353 3463 Acked Write 10 Sent Done Done false
9354 3464 Acked Read 1 Sent Done Done false
9355 3465 Acked Read 3 Sent Done Done false
9356 3466 Acked Read 8 Sent Done Done false
9357 3467 Acked Read 6 Sent Done Done false
9358 3468 Acked Read 5 Sent Done Done false
9359 3469 Acked Read 8 Sent Done Done false
9360 3470 Acked Write 8 Sent Done Done false
9361 3471 Acked Flush 0 Sent Done Done false
9362 3472 Acked Write 7 Sent Done Done false
9363 3473 Acked Flush 0 Sent Done Done false
9364 3474 Acked Read 6 Sent Done Done false
9365 3475 Acked Write 8 Sent Done Done false
9366 3476 Acked Read 9 Sent Done Done false
9367 3477 Acked Read 7 Sent Done Done false
9368 3478 Acked Write 10 Sent Done Done false
9369 3479 Acked Read 8 Sent Done Done false
9370 3480 Acked Read 8 Sent Done Done false
9371 3481 Acked Read 3 Sent Done Done false
9372 3482 Acked Write 4 Sent Done Done false
9373 3483 Acked Flush 0 Sent Done Done false
9374 3484 Acked Write 10 Sent Done Done false
9375 3485 Acked Read 1 Sent Done Done false
9376 3486 Acked Read 5 Sent Done Done false
9377 3487 Acked Read 8 Sent Done Done false
9378 3488 Acked Read 8 Sent Done Done false
9379 3489 Acked Write 2 Sent Done Done false
9380 3490 Acked Read 2 Sent Done Done false
9381 3491 Acked Read 3 Sent Done Done false
9382 3492 Acked Write 6 Sent Done Done false
9383 3493 Acked Read 7 Sent Done Done false
9384 3494 Acked Read 7 Sent Done Done false
9385 3495 Acked Read 8 Sent Done Done false
9386 3496 Acked Read 7 Sent Done Done false
9387 3497 Acked Read 9 Sent Done Done false
9388 3498 Acked Write 3 Sent Done Done false
9389 3499 Acked Flush 0 Sent Done Done false
9390 3500 Acked Flush 0 Sent Done Done false
9391 3501 Acked Read 2 Sent Done Done false
9392 3502 Acked Write 1 Sent Done Done false
9393 3503 Acked Write 3 Sent Done Done false
9394 3504 Acked Read 7 Sent Done Done false
9395 3505 Acked Write 10 Sent Done Done false
9396 3506 Acked Write 2 Sent Done Done false
9397 3507 Acked Write 10 Sent Done Done false
9398 3508 Acked Write 6 Sent Done Done false
9399 3509 Acked Read 5 Sent Done Done false
9400 3510 Acked Flush 0 Sent Done Done false
9401 3511 Acked Read 6 Sent Done Done false
9402 3512 Acked Write 6 Sent Done Done false
9403 3513 Acked Read 7 Sent Done Done false
9404 3514 Acked Read 6 Sent Done Done false
9405 3515 Acked Read 2 Sent Done Done false
9406 3516 Acked Read 6 Sent Done Done false
9407 3517 Acked Read 5 Sent Done Done false
9408 3518 Acked Write 10 Sent Done Done false
9409 3519 Acked Read 6 Sent Done Done false
9410 3520 Acked Write 4 Sent Done Done false
9411 3521 Acked Read 10 Sent Done Done false
9412 3522 Acked Read 2 Sent Done Done false
9413 3523 Acked Write 6 Sent Done Done false
9414 3524 Acked Write 9 Sent Done Done false
9415 3525 Acked Read 8 Sent Done Done false
9416 3526 Acked Read 10 Sent Done Done false
9417 3527 Acked Read 5 Sent Done Done false
9418 3528 Acked Flush 0 Sent Done Done false
9419 3529 Acked Write 9 Sent Done Done false
9420 3530 Acked Write 2 Sent Done Done false
9421 3531 Acked Read 5 Sent Done Done false
9422 3532 Acked Read 6 Sent Done Done false
9423 3533 Acked Read 8 Sent Done Done false
9424 3534 Acked Read 4 Sent Done Done false
9425 3535 Acked Read 5 Sent Done Done false
9426 3536 Acked Write 8 Sent Done Done false
9427 3537 Acked Read 3 Sent Done Done false
9428 3538 Acked Write 8 Sent Done Done false
9429 3539 Acked Read 9 Sent Done Done false
9430 3540 Acked Flush 0 Sent Done Done false
9431 3541 Acked Write 6 Sent Done Done false
9432 3542 Acked Read 1 Sent Done Done false
9433 3543 Acked Read 9 Sent Done Done false
9434 3544 Acked Write 2 Sent Done Done false
9435 3545 Acked Flush 0 Sent Done Done false
9436 3546 Acked Read 4 Sent Done Done false
9437 3547 Acked Read 8 Sent Done Done false
9438 3548 Acked Read 3 Sent Done Done false
9439 3549 Acked Read 1 Sent Done Done false
9440 3550 Acked Write 4 Sent Done Done false
9441 3551 Acked Write 5 Sent Done Done false
9442 3552 Acked Write 8 Sent Done Done false
9443 3553 Acked Read 10 Sent Done Done false
9444 3554 Acked Flush 0 Sent Done Done false
9445 3555 Acked Read 3 Sent Done Done false
9446 3556 Acked Read 1 Sent Done Done false
9447 3557 Acked Write 6 Sent Done Done false
9448 3558 Acked Read 8 Sent Done Done false
9449 3559 Acked Write 3 Sent Done Done false
9450 3560 Acked Read 2 Sent Done Done false
9451 3561 Acked Write 6 Sent Done Done false
9452 3562 Acked Write 8 Sent Done Done false
9453 3563 Acked Read 1 Sent Done Done false
9454 3564 Acked Read 9 Sent Done Done false
9455 3565 Acked Read 1 Sent Done Done false
9456 3566 Acked Flush 0 Sent Done Done false
9457 3567 Acked Write 10 Sent Done Done false
9458 3568 Acked Write 2 Sent Done Done false
9459 3569 Acked Read 7 Sent Done Done false
9460 3570 Acked Write 3 Sent Done Done false
9461 3571 Acked Read 5 Sent Done Done false
9462 3572 Acked Flush 0 Sent Done Done false
9463 3573 Acked Write 1 Sent Done Done false
9464 3574 Acked Write 1 Sent Done Done false
9465 3575 Acked Write 7 Sent Done Done false
9466 3576 Acked Read 2 Sent Done Done false
9467 3577 Acked Write 8 Sent Done Done false
9468 3578 Acked Write 5 Sent Done Done false
9469 3579 Acked Read 3 Sent Done Done false
9470 3580 Acked Write 4 Sent Done Done false
9471 3581 Acked Write 3 Sent Done Done false
9472 3582 Acked Read 4 Sent Done Done false
9473 3583 Acked Read 8 Sent Done Done false
9474 3584 Acked Write 5 Sent Done Done false
9475 3585 Acked Write 5 Sent Done Done false
9476 3586 Acked Read 4 Sent Done Done false
9477 3587 Acked Flush 0 Sent Done Done false
9478 3588 Acked Read 7 Sent Done Done false
9479 3589 Acked Read 4 Sent Done Done false
9480 3590 Acked Write 9 Sent Done Done false
9481 3591 Acked Write 4 Sent Done Done false
9482 3592 Acked Read 6 Sent Done Done false
9483 3593 Acked Read 7 Sent Done Done false
9484 3594 Acked Write 9 Sent Done Done false
9485 3595 Acked Read 10 Sent Done Done false
9486 3596 Acked Write 10 Sent Done Done false
9487 3597 Acked Write 6 Sent Done Done false
9488 3598 Acked Write 5 Sent Done Done false
9489 3599 Acked Write 1 Sent Done Done false
9490 3600 Acked Read 6 Sent Done Done false
9491 3601 Acked Write 5 Sent Done Done false
9492 3602 Acked Write 1 Sent Done Done false
9493 3603 Acked Read 9 Sent Done Done false
9494 3604 Acked Flush 0 Sent Done Done false
9495 3605 Acked Read 9 Sent Done Done false
9496 3606 Acked Write 8 Sent Done Done false
9497 3607 Acked Write 5 Sent Done Done false
9498 3608 Acked Read 9 Sent Done Done false
9499 3609 Acked Read 9 Sent Done Done false
9500 3610 Acked Read 9 Sent Done Done false
9501 3611 Acked Write 5 Sent Done Done false
9502 3612 Acked Write 9 Sent Done Done false
9503 3613 Acked Read 4 Sent Done Done false
9504 3614 Acked Read 9 Sent Done Done false
9505 3615 Acked Write 6 Sent Done Done false
9506 3616 Acked Write 6 Sent Done Done false
9507 3617 Acked Read 3 Sent Done Done false
9508 3618 Acked Read 1 Sent Done Done false
9509 3619 Acked Read 9 Sent Done Done false
9510 3620 Acked Write 2 Sent Done Done false
9511 3621 Acked Write 9 Sent Done Done false
9512 3622 Acked Flush 0 Sent Done Done false
9513 3623 Acked Read 7 Sent Done Done false
9514 3624 Acked Write 5 Sent Done Done false
9515 3625 Acked Write 6 Sent Done Done false
9516 3626 Acked Flush 0 Sent Done Done false
9517 3627 Acked Write 4 Sent Done Done false
9518 3628 Acked Read 2 Sent Done Done false
9519 3629 Acked Write 8 Sent Done Done false
9520 3630 Acked Write 2 Sent Done Done false
9521 3631 Acked Read 9 Sent Done Done false
9522 3632 Acked Write 8 Sent Done Done false
9523 3633 Acked Read 9 Sent Done Done false
9524 3634 Acked Flush 0 Sent Done Done false
9525 3635 Acked Read 3 Sent Done Done false
9526 3636 Acked Read 8 Sent Done Done false
9527 3637 Acked Write 10 Sent Done Done false
9528 3638 Acked Write 9 Sent Done Done false
9529 3639 Acked Read 6 Sent Done Done false
9530 3640 Acked Read 10 Sent Done Done false
9531 3641 Acked Read 10 Sent Done Done false
9532 3642 Acked Read 10 Sent Done Done false
9533 3643 Acked Read 6 Sent Done Done false
9534 3644 Acked Flush 0 Sent Done Done false
9535 3645 Acked Flush 0 Sent Done Done false
9536 3646 Acked Read 4 Sent Done Done false
9537 3647 Acked Write 2 Sent Done Done false
9538 3648 Acked Write 8 Sent Done Done false
9539 3649 Acked Read 4 Sent Done Done false
9540 3650 Acked Read 1 Sent Done Done false
9541 3651 Acked Read 9 Sent Done Done false
9542 3652 Acked Flush 0 Sent Done Done false
9543 3653 Acked Read 9 Sent Done Done false
9544 3654 Acked Flush 0 Sent Done Done false
9545 3655 Acked Read 9 Sent Done Done false
9546 3656 Acked Read 7 Sent Done Done false
9547 3657 Acked Write 2 Sent Done Done false
9548 3658 Acked Write 5 Sent Done Done false
9549 3659 Acked Write 2 Sent Done Done false
9550 3660 Acked Write 6 Sent Done Done false
9551 3661 Acked Read 4 Sent Done Done false
9552 3662 Acked Read 7 Sent Done Done false
9553 3663 Acked Write 10 Sent Done Done false
9554 3664 Acked Read 3 Sent Done Done false
9555 3665 Acked Read 10 Sent Done Done false
9556 3666 Acked Write 10 Sent Done Done false
9557 3667 Acked Write 1 Sent Done Done false
9558 3668 Acked Write 9 Sent Done Done false
9559 3669 Acked Write 8 Sent Done Done false
9560 3670 Acked Read 9 Sent Done Done false
9561 3671 Acked Write 3 Sent Done Done false
9562 3672 Acked Write 8 Sent Done Done false
9563 3673 Acked Write 7 Sent Done Done false
9564 3674 Acked Read 3 Sent Done Done false
9565 3675 Acked Write 9 Sent Done Done false
9566 3676 Acked Write 1 Sent Done Done false
9567 3677 Acked Write 9 Sent Done Done false
9568 3678 Acked Read 6 Sent Done Done false
9569 3679 Acked Flush 0 Sent Done Done false
9570 3680 Acked Read 3 Sent Done Done false
9571 3681 Acked Read 7 Sent Done Done false
9572 3682 Acked Read 3 Sent Done Done false
9573 3683 Acked Read 3 Sent Done Done false
9574 3684 Acked Write 1 Sent Done Done false
9575 3685 Acked Write 3 Sent Done Done false
9576 3686 Acked Read 10 Sent Done Done false
9577 3687 Acked Flush 0 Sent Done Done false
9578 3688 Acked Write 2 Sent Done Done false
9579 3689 Acked Read 1 Sent Done Done false
9580 3690 Acked Read 8 Sent Done Done false
9581 3691 Acked Flush 0 Sent Done Done false
9582 3692 Acked Write 6 Sent Done Done false
9583 3693 Acked Read 6 Sent Done Done false
9584 3694 Acked Read 2 Sent Done Done false
9585 3695 Acked Read 8 Sent Done Done false
9586 3696 Acked Read 3 Sent Done Done false
9587 3697 Acked Write 2 Sent Done Done false
9588 3698 Acked Read 5 Sent Done Done false
9589 3699 Acked Read 2 Sent Done Done false
9590 3700 Acked Read 7 Sent Done Done false
9591 3701 Acked Flush 0 Sent Done Done false
9592 3702 Acked Read 3 Sent Done Done false
9593 3703 Acked Write 6 Sent Done Done false
9594 3704 Acked Read 3 Sent Done Done false
9595 3705 Acked Read 6 Sent Done Done false
9596 3706 Acked Flush 0 Sent Done Done false
9597 3707 Acked Flush 0 Sent Done Done false
9598 3708 Acked Write 8 Sent Done Done false
9599 3709 Acked Write 7 Sent Done Done false
9600 3710 Acked Read 9 Sent Done Done false
9601 3711 Acked Write 3 Sent Done Done false
9602 3712 Acked Read 1 Sent Done Done false
9603 3713 Acked Read 5 Sent Done Done false
9604 3714 Acked Flush 0 Sent Done Done false
9605 3715 Acked Write 1 Sent Done Done false
9606 3716 Acked Flush 0 Sent Done Done false
9607 STATES DS:0 DS:1 DS:2 TOTAL
9608 Sent 301 0 0 301
9609 Done 0 301 301 602
9610 Skipped 0 0 0 0
9611 Error 0 0 0 0
9612 Last Flush: 3415 3716 3716
9613 Downstairs last five completed: 3415 3414 3413 3412 3411
9614 Upstairs last five completed: 3716 3715 3714 3713 3712
9615 CLIENT: Up:0 ds:301 act:2
96162024-11-28T04:52:18.946ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
96172024-11-28T04:52:18.946ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:59198) = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
96182024-11-28T04:52:18.947ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
96192024-11-28T04:52:18.948ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
96202024-11-28T04:52:18.948ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
96212024-11-28T04:52:18.948ZINFOcrucible: send last flush ID to this DS: 3415 = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
96222024-11-28T04:52:18.948ZINFOcrucible: Replied this last flush ID: 3415 = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
96232024-11-28T04:52:18.948ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
96242024-11-28T04:52:18.948ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
96252024-11-28T04:52:18.948ZINFOcrucible: [0] client re-new 301 jobs since flush 3415 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
96262024-11-28T04:52:18.948ZINFOcrucible: [0] Replayed 301 jobs since flush: 3415 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
9627 ----------------------------------------------------------------
9628 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
9629 Downstairs last five completed: 3716 3715 3714 3713 3712
9630 Upstairs last five completed: 3716 3715 3714 3713 3712
9631 CLIENT: Up:0 ds:0 act:3
9632 Replay: All jobs finished
9633 Main task has work to do, go find it
9634 got dsc Stop(1)
9635 stop 1
9636 Nov 28 04:52:21.191 INFO request completed, latency_us: 348, response_code: 204, uri:[8820] Got stop action so:false kr:false
9637 /stop/cid/1, method: POST, req_id: c09e62bf-cc9e-4164-a44f-35eb128fc024, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
96382024-11-28T04:52:21.192ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
96392024-11-28T04:52:21.192ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
96402024-11-28T04:52:21.192ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
96412024-11-28T04:52:21.192ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
9642 Nov 28 04:52:21.192 INFO request completed, latency_us: 142, response_code: 200{"msg":"[1] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-28T04:52:21.192331532Z","hostname":"w-01JDRHKEEFJ1228TFPSBEGXWHY","pid":1202,"client":"1","":"downstairs","session_id":"4a2415b1-6594-436f-82e5-b8cf9a74eeaf"}
9643 , uri: /state/cid/1, method: GET, req_id: 04c8b21a-8457-4175-9046-5ecd7a72007e, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
96442024-11-28T04:52:21.192ZINFOcrucible: client task is exiting = io task client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
9645 [8820] Exited with: ExitStatus(unix_wait_status(9))
9646 [8820][1] reports Exit
9647 Nov 28 04:52:25.195 INFO request completed, latency_us: 221, response_code: 200, uri: /state/cid/1, method: GET, req_id: ebfe8349-fe2f-49d0-b5a3-37bd0968ef82, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
9648 001/300 Read block 2123 len 36864
9649 002/300 Read block 2875 len 40960
9650 003/300 Write block 3842 len 36864 data: 3 3 2 2 2 2 2 2 2
9651 004/300 Flush
9652 005/300 Write block 1436 len 8192 data: 3 2
9653 006/300 Write block 666 len 28672 data: 4 5 5 3 3 3 4
9654 007/300 Write block 4287 len 12288 data: 2 3 3
9655 008/300 Write block 3304 len 32768 data: 3 3 3 3 3 3 3 3
9656 009/300 Write block 3125 len 24576 data: 3 3 3 3 4 3
9657 010/300 Write block 2308 len 36864 data: 3 4 4 4 4 4 4 4 4
9658 011/300 Read block 888 len 16384
9659 012/300 Write block 4370 len 8192 data: 2 2
9660 013/300 Flush
9661 014/300 Read block 3931 len 32768
9662 015/300 Read block 4632 len 12288
9663 016/300 Read block 2537 len 8192
9664 017/300 Write block 4976 len 28672 data: 2 2 2 2 2 2 2
9665 018/300 Write block 1605 len 40960 data: 4 3 4 4 4 4 4 4 4 4
9666 019/300 Write block 3788 len 36864 data: 2 3 4 5 5 4 3 3 2
9667 020/300 Write block 4446 len 28672 data: 6 5 4 4 4 2 3
9668 021/300 Write block 382 len 24576 data: 6 6 4 2 2 3
9669 022/300 Write block 908 len 20480 data: 4 2 2 2 2
9670 023/300 Flush
9671 024/300 Read block 3145 len 32768
9672 025/300 Write block 3396 len 36864 data: 2 2 2 2 2 2 2 3 4
9673 026/300 Read block 383 len 12288
9674 027/300 Write block 468 len 40960 data: 2 3 3 3 4 5 5 5 3 3
9675 028/300 Write block 3445 len 40960 data: 2 2 3 3 4 5 5 4 4 4
9676 029/300 Flush
9677 030/300 Read block 1033 len 4096
9678 031/300 Write block 2373 len 36864 data: 5 5 5 5 4 5 5 5 4
9679 032/300 Read block 3819 len 4096
9680 033/300 Read block 361 len 32768
9681 034/300 Write block 2876 len 24576 data: 3 4 4 4 4 4
9682 035/300 Read block 711 len 20480
9683 036/300 Read block 1259 len 28672
9684 037/300 Write block 4621 len 4096 data: 2
9685 038/300 Read block 847 len 20480
9686 039/300 Read block 501 len 4096
9687 040/300 Write block 3302 len 16384 data: 2 3 4 4
9688 041/300 Write block 3630 len 32768 data: 3 3 4 5 4 4 4 5
9689 042/300 Read block 4441 len 4096
9690 043/300 Write block 81 len 12288 data: 2 2 2
9691 044/300 Read block 733 len 32768
9692 045/300 Read block 779 len 16384
9693 046/300 Read block 3454 len 12288
9694 047/300 Read block 238 len 40960
9695 048/300 Read block 4722 len 32768
9696 049/300 Read block 4165 len 40960
9697 050/300 Write block 407 len 36864 data: 4 4 4 4 4 4 4 4 3
9698 051/300 Read block 1097 len 12288
9699 052/300 Write block 1030 len 16384 data: 2 2 2 3
9700 053/300 Write block 1067 len 36864 data: 2 2 2 2 2 2 3 3 6
9701 054/300 Write block 1978 len 32768 data: 4 4 4 4 5 3 3 3
9702 055/300 Read block 4196 len 28672
9703 056/300 Read block 880 len 16384
9704 057/300 Read block 3415 len 12288
9705 058/300 Read block 4331 len 16384
9706 059/300 Write block 4545 len 8192 data: 3 4
9707 060/300 Read block 1926 len 40960
9708 061/300 Read block 4288 len 36864
9709 062/300 Read block 4510 len 40960
9710 063/300 Write block 3820 len 32768 data: 4 4 4 3 3 3 2 3
9711 064/300 Read block 2362 len 36864
9712 065/300 Flush
9713 066/300 Write block 2419 len 36864 data: 3 3 3 3 2 2 2 2 2
9714 067/300 Write block 560 len 20480 data: 3 2 3 4 4
9715 068/300 Read block 2874 len 20480
9716 069/300 Read block 735 len 24576
9717 070/300 Write block 4658 len 20480 data: 3 3 3 4 4
9718 071/300 Flush
9719 072/300 Write block 2312 len 16384 data: 5 5 5 5
9720 073/300 Write block 4374 len 20480 data: 2 2 2 2 2
9721 074/300 Write block 4523 len 40960 data: 4 5 5 4 3 2 3 3 3 3
9722 075/300 Read block 2818 len 4096
9723 076/300 Write block 519 len 24576 data: 2 2 2 2 2 2
9724 077/300 Read block 246 len 4096
9725 078/300 Read block 655 len 24576
9726 079/300 Write block 1335 len 32768 data: 2 2 2 2 3 3 4 5
9727 080/300 Write block 4423 len 40960 data: 3 3 3 2 2 2 2 3 3 3
9728 081/300 Read block 1702 len 8192
9729 082/300 Read block 1607 len 20480
9730 083/300 Write block 2773 len 32768 data: 5 4 5 5 4 4 4 4
9731 084/300 Write block 3134 len 20480 data: 3 3 2 2 3
9732 085/300 Read block 4409 len 16384
9733 086/300 Write block 4701 len 4096 data: 2
9734 087/300 Flush
9735 088/300 Read block 375 len 24576
9736 089/300 Read block 2454 len 4096
9737 090/300 Read block 1709 len 32768
9738 091/300 Write block 1775 len 40960 data: 3 4 3 3 3 3 4 4 4 4
9739 092/300 Read block 632 len 28672
9740 093/300 Read block 852 len 12288
9741 094/300 Read block 3349 len 12288
9742 095/300 Read block 891 len 20480
9743 096/300 Read block 4325 len 28672
9744 097/300 Read block 4362 len 40960
9745 098/300 Read block 4780 len 32768
9746 099/300 Write block 2484 len 12288 data: 3 3 2
9747 100/300 Write block 3089 len 28672 data: 3 3 3 3 3 3 2
9748 101/300 Write block 4340 len 40960 data: 2 2 2 2 3 4 4 4 4 4
9749 102/300 Write block 3498 len 40960 data: 4 4 3 3 4 4 3 4 5 5
9750 103/300 Write block 341 len 12288 data: 5 5 4
9751 104/300 Write block 1361 len 20480 data: 3 3 3 3 3
9752 105/300 Write block 3911 len 8192 data: 2 2
9753 106/300 Read block 2232 len 12288
9754 107/300 Write block 4371 len 4096 data: 3
9755 108/300 Read block 542 len 32768
9756 109/300 Write block 3860 len 12288 data: 2 2 3
9757 110/300 Read block 3309 len 16384
9758 111/300 Write block 4503 len 24576 data: 2 2 2 2 2 2
9759 112/300 Read block 2381 len 24576
9760 113/300 Write block 1875 len 40960 data: 2 2 2 3 3 5 5 5 5 5
9761 114/300 Write block 4833 len 20480 data: 2 3 3 2 2
9762 115/300 Write block 1108 len 16384 data: 2 2 2 2
9763 116/300 Write block 2982 len 8192 data: 2 2
9764 117/300 Read block 1950 len 8192
9765 118/300 Read block 4484 len 12288
9766 119/300 Write block 1081 len 8192 data: 4 3
9767 120/300 Write block 2264 len 16384 data: 3 3 3 2
9768 121/300 Read block 1518 len 36864
9769 122/300 Read block 2634 len 36864
9770 123/300 Read block 1748 len 4096
9771 124/300 Write block 4296 len 40960 data: 2 2 2 3 3 3 3 3 3 3
9772 125/300 Write block 1907 len 20480 data: 2 2 2 2 2
9773 126/300 Write block 122 len 8192 data: 3 2
9774 127/300 Read block 1211 len 32768
9775 128/300 Read block 2238 len 20480
9776 129/300 Write block 638 len 32768 data: 5 5 3 3 2 2 2 2
9777 130/300 Write block 4077 len 4096 data: 4
9778 131/300 Read block 2216 len 8192
9779 132/300 Read block 2510 len 24576
9780 133/300 Read block 4531 len 4096
9781 134/300 Read block 4826 len 4096
9782 135/300 Read block 4534 len 36864
9783 136/300 Read block 411 len 20480
9784 137/300 Write block 2127 len 12288 data: 4 3 3
9785 138/300 Write block 1049 len 36864 data: 4 4 5 5 5 3 3 2 2
9786 139/300 Read block 3721 len 12288
9787 140/300 Read block 3133 len 28672
9788 141/300 Write block 1181 len 40960 data: 4 4 4 3 2 3 3 3 2 2
9789 142/300 Read block 1173 len 36864
9790 143/300 Read block 3823 len 24576
9791 144/300 Write block 4377 len 12288 data: 3 3 3
9792 145/300 Flush
9793 146/300 Read block 1344 len 16384
9794 147/300 Flush
9795 148/300 Flush
9796 149/300 Read block 2466 len 32768
9797 150/300 Read block 662 len 16384
9798 151/300 Read block 2058 len 20480
9799 152/300 Write block 4042 len 24576 data: 2 3 3 2 2 2
9800 153/300 Write block 4178 len 12288 data: 2 2 2
9801 154/300 Write block 3990 len 36864 data: 3 3 3 3 3 2 2 2 2
9802 155/300 Read block 304 len 20480
9803 156/300 Flush
9804 157/300 Write block 4641 len 40960 data: 4 4 4 2 2 3 3 4 4 4
9805 158/300 Write block 31 len 4096 data: 4
9806 159/300 Write block 1015 len 8192 data: 2 2
9807 160/300 Read block 345 len 24576
9808 161/300 Flush
9809 162/300 Read block 293 len 20480
9810 163/300 Read block 2909 len 8192
9811 164/300 Read block 2931 len 4096
9812 165/300 Write block 166 len 4096 data: 4
9813 166/300 Read block 1301 len 32768
9814 167/300 Write block 3665 len 32768 data: 3 3 2 3 4 4 3 3
9815 168/300 Read block 1983 len 16384
9816 169/300 Write block 1488 len 40960 data: 3 3 3 4 4 5 4 4 3 5
9817 170/300 Write block 2978 len 20480 data: 4 4 4 3 3
9818 171/300 Read block 4633 len 28672
9819 172/300 Flush
9820 173/300 Read block 3991 len 24576
9821 174/300 Read block 2543 len 12288
9822 175/300 Write block 4151 len 28672 data: 4 5 6 6 6 5 4
9823 176/300 Read block 2800 len 40960
9824 177/300 Read block 3020 len 40960
9825 178/300 Flush
9826 179/300 Write block 3057 len 40960 data: 3 4 4 4 4 3 3 3 3 2
9827 180/300 Read block 3594 len 40960
9828 181/300 Write block 1032 len 16384 data: 3 4 3 2
9829 182/300 Write block 2235 len 12288 data: 3 4 3
9830 183/300 Read block 3924 len 16384
9831 184/300 Read block 2111 len 24576
9832 185/300 Read block 1008 len 24576
9833 186/300 Read block 1169 len 40960
9834 187/300 Write block 3039 len 4096 data: 4
9835 188/300 Read block 3403 len 16384
9836 189/300 Flush
9837 190/300 Write block 81 len 20480 data: 3 3 3 2 3
9838 191/300 Write block 723 len 4096 data: 2
9839 192/300 Read block 1217 len 36864
9840 193/300 Write block 30 len 40960 data: 4 5 4 4 3 3 3 3 2 2
9841 194/300 Write block 2289 len 16384 data: 2 2 2 2
9842 195/300 Read block 4809 len 20480
9843 196/300 Write block 15 len 16384 data: 5 6 5 4
9844 197/300 Write block 1193 len 24576 data: 2 2 2 3 3 3
9845 198/300 Read block 1804 len 16384
9846 199/300 Read block 4726 len 28672
9847 200/300 Read block 916 len 36864
9848 201/300 Read block 4614 len 4096
9849 202/300 Read block 4151 len 36864
9850 203/300 Write block 506 len 40960 data: 6 6 5 3 3 4 4 4 3 3
9851 204/300 Write block 3960 len 28672 data: 3 3 3 2 2 3 3
9852 205/300 Write block 1794 len 24576 data: 3 3 3 2 2 2
9853 206/300 Read block 2017 len 4096
9854 207/300 Flush
9855 208/300 Read block 3092 len 24576
9856 209/300 Flush
9857 210/300 Read block 4132 len 8192
9858 211/300 Write block 334 len 8192 data: 3 5
9859 212/300 Write block 48 len 12288 data: 6 7 5
9860 213/300 Write block 4790 len 4096 data: 2
9861 214/300 Write block 2355 len 16384 data: 2 2 3 4
9862 215/300 Read block 3092 len 4096
9863 216/300 Read block 3119 len 4096
9864 217/300 Write block 708 len 40960 data: 2 2 2 3 4 4 4 4 4 3
9865 218/300 Flush
9866 219/300 Read block 1898 len 24576
9867 220/300 Write block 2054 len 36864 data: 3 3 3 2 3 3 3 3 3
9868 221/300 Read block 2849 len 20480
9869 222/300 Write block 2376 len 4096 data: 6
9870 223/300 Write block 359 len 32768 data: 2 2 2 2 2 2 3 4
9871 224/300 Read block 4049 len 24576
9872 225/300 Read block 3805 len 36864
9873 226/300 Write block 659 len 36864 data: 8 7 7 7 6 5 4 5 6
9874 227/300 Read block 1513 len 32768
9875 228/300 Write block 260 len 32768 data: 3 4 4 3 3 3 3 2
9876 229/300 Flush
9877 230/300 Write block 3229 len 16384 data: 3 3 3 3
9878 231/300 Read block 2334 len 28672
9879 232/300 Read block 2458 len 4096
9880 233/300 Read block 2739 len 4096
9881 234/300 Read block 3083 len 8192
9882 235/300 Write block 4963 len 36864 data: 2 3 3 4 5 5 4 4 4
9883 236/300 Write block 668 len 32768 data: 6 4 4 4 5 2 2 2
9884 237/300 Flush
9885 238/300 Flush
9886 239/300 Read block 424 len 36864
9887 240/300 Read block 207 len 28672
9888 241/300 Read block 2485 len 40960
9889 242/300 Read block 3792 len 28672
9890 243/300 Read block 411 len 40960
9891 244/300 Write block 2901 len 20480 data: 2 2 2 2 2
9892 245/300 Write block 4176 len 40960 data: 2 2 3 3 3 3 3 3 2 2
9893 246/300 Read block 139 len 36864
9894 247/300 Read block 2453 len 8192
9895 248/300 Read block 318 len 24576
9896 249/300 Read block 1892 len 12288
9897 250/300 Read block 2974 len 36864
9898 251/300 Read block 2676 len 8192
9899 252/300 Read block 4543 len 28672
9900 253/300 Read block 1487 len 40960
9901 254/300 Read block 1038 len 16384
9902 255/300 Write block 4929 len 20480 data: 3 3 2 2 2
9903 256/300 Flush
9904 257/300 Flush
9905 258/300 Read block 653 len 28672
9906 259/300 Write block 4018 len 4096 data: 2
9907 260/300 Flush
9908 261/300 Flush
9909 262/300 Read block 4257 len 32768
9910 263/300 Read block 3065 len 16384
9911 264/300 Write block 982 len 36864 data: 2 2 4 4 3 3 3 3 3
9912 265/300 Write block 41 len 40960 data: 2 2 2 2 3 4 5 7 8 6
9913 266/300 Read block 4000 len 8192
9914 267/300 Write block 3034 len 24576 data: 5 5 4 4 4 5
9915 268/300 Read block 3815 len 16384
9916 269/300 Write block 3893 len 12288 data: 3 2 2
9917 270/300 Flush
9918 271/300 Write block 4102 len 40960 data: 3 3 4 3 2 2 2 2 2 2
9919 272/300 Write block 612 len 28672 data: 2 2 2 2 3 3 4
9920 273/300 Read block 357 len 4096
9921 274/300 Read block 4611 len 4096
9922 275/300 Read block 535 len 40960
9923 276/300 Read block 2322 len 4096
9924 277/300 Read block 886 len 32768
9925 278/300 Write block 61 len 36864 data: 4 3 3 3 2 2 2 2 2
9926 279/300 Read block 3214 len 28672
9927 280/300 Read block 1446 len 16384
9928 281/300 Read block 4309 len 32768
9929 282/300 Read block 4689 len 40960
9930 283/300 Write block 2187 len 40960 data: 3 2 2 2 2 3 4 4 4 3
9931 284/300 Read block 3764 len 36864
9932 285/300 Read block 2784 len 32768
9933 286/300 Write block 655 len 4096 data: 4
9934 287/300 Read block 2672 len 24576
9935 288/300 Write block 2665 len 40960 data: 3 3 3 3 3 4 3 3 2 2
9936 289/300 Read block 2061 len 4096
9937 290/300 Write block 364 len 12288 data: 3 4 5
9938 291/300 Flush
9939 292/300 Read block 4022 len 40960
9940 293/300 Write block 4674 len 32768 data: 4 4 5 5 5 5 4 4
9941 294/300 Read block 4066 len 36864
9942 295/300 Read block 3367 len 8192
9943 296/300 Read block 2547 len 12288
9944 297/300 Read block 4930 len 4096
9945 298/300 Flush
9946 299/300 Read block 1906 len 20480
9947 300/300 Flush
9948 301/300 Read block 4718 len 4096
9949 Main task has work to do, go find it
9950 got dsc Start(1)
9951 start 1
9952 Nov 28 04:52:25.433 INFO request completed, latency_us: 202, response_code: 204, uri: /start/cid/1, method: POST, req_id: 6390192d-9573-4496-aaee-85e3d034fc46, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
9953 Replay: started 1, returned:Ok(())
9954 ----------------------------------------------------------------
9955 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
9956 States: Active Offline Active
9957 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
9958 3717 Acked Read 9 Done Sent Done false
9959 3718 Acked Read 10 Done Sent Done false
9960 3719 Acked Write 9 Done Sent Done false
9961 3720 Acked Flush 0 Done Sent Done false
9962 3721 Acked Write 2 Done Sent Done false
9963 3722 Acked Write 7 Done Sent Done false
9964 3723 Acked Write 3 Done Sent Done false
9965 3724 Acked Write 8 Done Sent Done false
9966 3725 Acked Write 6 Done Sent Done false
9967 3726 Acked Write 9 Done Sent Done false
9968 3727 Acked Read 4 Done Sent Done false
9969 3728 Acked Write 2 Done Sent Done false
9970 3729 Acked Flush 0 Done Sent Done false
9971 3730 Acked Read 8 Done Sent Done false
9972 3731 Acked Read 3 Done Sent Done false
9973 3732 Acked Read 2 Done Sent Done false
9974 3733 Acked Write 7 Done Sent Done false
9975 3734 Acked Write 10 Done Sent Done false
9976 3735 Acked Write 9 Done Sent Done false
9977 3736 Acked Write 7 Done Sent Done false
9978 3737 Acked Write 6 Done Sent Done false
9979 3738 Acked Write 5 Done Sent Done false
9980 3739 Acked Flush 0 Done Sent Done false
9981 3740 Acked Read 8 Done Sent Done false
9982 3741 Acked Write 9 Done Sent Done false
9983 3742 Acked Read 3 Done Sent Done false
9984 3743 Acked Write 10 Done Sent Done false
9985 3744 Acked Write 10 Done Sent Done false
9986 3745 Acked Flush 0 Done Sent Done false
9987 3746 Acked Read 1 Done Sent Done false
9988 3747 Acked Write 9 Done Sent Done false
9989 3748 Acked Read 1 Done Sent Done false
9990 3749 Acked Read 8 Done Sent Done false
9991 3750 Acked Write 6 Done Sent Done false
9992 3751 Acked Read 5 Done Sent Done false
9993 3752 Acked Read 7 Done Sent Done false
9994 3753 Acked Write 1 Done Sent Done false
9995 3754 Acked Read 5 Done Sent Done false
9996 3755 Acked Read 1 Done Sent Done false
9997 3756 Acked Write 4 Done Sent Done false
9998 3757 Acked Write 8 Done Sent Done false
9999 3758 Acked Read 1 Done Sent Done false
10000 3759 Acked Write 3 Done Sent Done false
10001 3760 Acked Read 8 Done Sent Done false
10002 3761 Acked Read 4 Done Sent Done false
10003 3762 Acked Read 3 Done Sent Done false
10004 3763 Acked Read 10 Done Sent Done false
10005 3764 Acked Read 8 Done Sent Done false
10006 3765 Acked Read 10 Done Sent Done false
10007 3766 Acked Write 9 Done Sent Done false
10008 3767 Acked Read 3 Done Sent Done false
10009 3768 Acked Write 4 Done Sent Done false
10010 3769 Acked Write 9 Done Sent Done false
10011 3770 Acked Write 8 Done Sent Done false
10012 3771 Acked Read 7 Done Sent Done false
10013 3772 Acked Read 4 Done Sent Done false
10014 3773 Acked Read 3 Done Sent Done false
10015 3774 Acked Read 4 Done Sent Done false
10016 3775 Acked Write 2 Done Sent Done false
10017 3776 Acked Read 10 Done Sent Done false
10018 3777 Acked Read 9 Done Sent Done false
10019 3778 Acked Read 10 Done Sent Done false
10020 3779 Acked Write 8 Done Sent Done false
10021 3780 Acked Read 9 Done Sent Done false
10022 3781 Acked Flush 0 Done Sent Done false
10023 3782 Acked Write 9 Done Sent Done false
10024 3783 Acked Write 5 Done Sent Done false
10025 3784 Acked Read 5 Done Sent Done false
10026 3785 Acked Read 6 Done Sent Done false
10027 3786 Acked Write 5 Done Sent Done false
10028 3787 Acked Flush 0 Done Sent Done false
10029 3788 Acked Write 4 Done Sent Done false
10030 3789 Acked Write 5 Done Sent Done false
10031 3790 Acked Write 10 Done Sent Done false
10032 3791 Acked Read 1 Done Sent Done false
10033 3792 Acked Write 6 Done Sent Done false
10034 3793 Acked Read 1 Done Sent Done false
10035 3794 Acked Read 6 Done Sent Done false
10036 3795 Acked Write 8 Done Sent Done false
10037 3796 Acked Write 10 Done Sent Done false
10038 3797 Acked Read 2 Done Sent Done false
10039 3798 Acked Read 5 Done Sent Done false
10040 3799 Acked Write 8 Done Sent Done false
10041 3800 Acked Write 5 Done Sent Done false
10042 3801 Acked Read 4 Done Sent Done false
10043 3802 Acked Write 1 Done Sent Done false
10044 3803 Acked Flush 0 Done Sent Done false
10045 3804 Acked Read 6 Done Sent Done false
10046 3805 Acked Read 1 Done Sent Done false
10047 3806 Acked Read 8 Done Sent Done false
10048 3807 Acked Write 10 Done Sent Done false
10049 3808 Acked Read 7 Done Sent Done false
10050 3809 Acked Read 3 Done Sent Done false
10051 3810 Acked Read 3 Done Sent Done false
10052 3811 Acked Read 5 Done Sent Done false
10053 3812 Acked Read 7 Done Sent Done false
10054 3813 Acked Read 10 Done Sent Done false
10055 3814 Acked Read 8 Done Sent Done false
10056 3815 Acked Write 3 Done Sent Done false
10057 3816 Acked Write 7 Done Sent Done false
10058 3817 Acked Write 10 Done Sent Done false
10059 3818 Acked Write 10 Done Sent Done false
10060 3819 Acked Write 3 Done Sent Done false
10061 3820 Acked Write 5 Done Sent Done false
10062 3821 Acked Write 2 Done Sent Done false
10063 3822 Acked Read 3 Done Sent Done false
10064 3823 Acked Write 1 Done Sent Done false
10065 3824 Acked Read 8 Done Sent Done false
10066 3825 Acked Write 3 Done Sent Done false
10067 3826 Acked Read 4 Done Sent Done false
10068 3827 Acked Write 6 Done Sent Done false
10069 3828 Acked Read 6 Done Sent Done false
10070 3829 Acked Write 10 Done Sent Done false
10071 3830 Acked Write 5 Done Sent Done false
10072 3831 Acked Write 4 Done Sent Done false
10073 3832 Acked Write 2 Done Sent Done false
10074 3833 Acked Read 2 Done Sent Done false
10075 3834 Acked Read 3 Done Sent Done false
10076 3835 Acked Write 2 Done Sent Done false
10077 3836 Acked Write 4 Done Sent Done false
10078 3837 Acked Read 9 Done Sent Done false
10079 3838 Acked Read 9 Done Sent Done false
10080 3839 Acked Read 1 Done Sent Done false
10081 3840 Acked Write 10 Done Sent Done false
10082 3841 Acked Write 5 Done Sent Done false
10083 3842 Acked Write 2 Done Sent Done false
10084 3843 Acked Read 8 Done Sent Done false
10085 3844 Acked Read 5 Done Sent Done false
10086 3845 Acked Write 8 Done Sent Done false
10087 3846 Acked Write 1 Done Sent Done false
10088 3847 Acked Read 2 Done Sent Done false
10089 3848 Acked Read 6 Done Sent Done false
10090 3849 Acked Read 1 Done Sent Done false
10091 3850 Acked Read 1 Done Sent Done false
10092 3851 Acked Read 9 Done Sent Done false
10093 3852 Acked Read 5 Done Sent Done false
10094 3853 Acked Write 3 Done Sent Done false
10095 3854 Acked Write 9 Done Sent Done false
10096 3855 Acked Read 3 Done Sent Done false
10097 3856 Acked Read 7 Done Sent Done false
10098 3857 Acked Write 10 Done Sent Done false
10099 3858 Acked Read 9 Done Sent Done false
10100 3859 Acked Read 6 Done Sent Done false
10101 3860 Acked Write 3 Done Sent Done false
10102 3861 Acked Flush 0 Done Sent Done false
10103 3862 Acked Read 4 Done Sent Done false
10104 3863 Acked Flush 0 Done Sent Done false
10105 3864 Acked Flush 0 Done Sent Done false
10106 3865 Acked Read 8 Done Sent Done false
10107 3866 Acked Read 4 Done Sent Done false
10108 3867 Acked Read 5 Done Sent Done false
10109 3868 Acked Write 6 Done Sent Done false
10110 3869 Acked Write 3 Done Sent Done false
10111 3870 Acked Write 9 Done Sent Done false
10112 3871 Acked Read 5 Done Sent Done false
10113 3872 Acked Flush 0 Done Sent Done false
10114 3873 Acked Write 10 Done Sent Done false
10115 3874 Acked Write 1 Done Sent Done false
10116 3875 Acked Write 2 Done Sent Done false
10117 3876 Acked Read 6 Done Sent Done false
10118 3877 Acked Flush 0 Done Sent Done false
10119 3878 Acked Read 5 Done Sent Done false
10120 3879 Acked Read 2 Done Sent Done false
10121 3880 Acked Read 1 Done Sent Done false
10122 3881 Acked Write 1 Done Sent Done false
10123 3882 Acked Read 8 Done Sent Done false
10124 3883 Acked Write 8 Done Sent Done false
10125 3884 Acked Read 4 Done Sent Done false
10126 3885 Acked Write 10 Done Sent Done false
10127 3886 Acked Write 5 Done Sent Done false
10128 3887 Acked Read 7 Done Sent Done false
10129 3888 Acked Flush 0 Done Sent Done false
10130 3889 Acked Read 6 Done Sent Done false
10131 3890 Acked Read 3 Done Sent Done false
10132 3891 Acked Write 7 Done Sent Done false
10133 3892 Acked Read 10 Done Sent Done false
10134 3893 Acked Read 10 Done Sent Done false
10135 3894 Acked Flush 0 Done Sent Done false
10136 3895 Acked Write 10 Done Sent Done false
10137 3896 Acked Read 10 Done Sent Done false
10138 3897 Acked Write 4 Done Sent Done false
10139 3898 Acked Write 3 Done Sent Done false
10140 3899 Acked Read 4 Done Sent Done false
10141 3900 Acked Read 6 Done Sent Done false
10142 3901 Acked Read 6 Done Sent Done false
10143 3902 Acked Read 10 Done Sent Done false
10144 3903 Acked Write 1 Done Sent Done false
10145 3904 Acked Read 4 Done Sent Done false
10146 3905 Acked Flush 0 Done Sent Done false
10147 3906 Acked Write 5 Done Sent Done false
10148 3907 Acked Write 1 Done Sent Done false
10149 3908 Acked Read 9 Done Sent Done false
10150 3909 Acked Write 10 Done Sent Done false
10151 3910 Acked Write 4 Done Sent Done false
10152 3911 Acked Read 5 Done Sent Done false
10153 3912 Acked Write 4 Done Sent Done false
10154 3913 Acked Write 6 Done Sent Done false
10155 3914 Acked Read 4 Done Sent Done false
10156 3915 Acked Read 7 Done Sent Done false
10157 3916 Acked Read 9 Done Sent Done false
10158 3917 Acked Read 1 Done Sent Done false
10159 3918 Acked Read 9 Done Sent Done false
10160 3919 Acked Write 10 Done Sent Done false
10161 3920 Acked Write 7 Done Sent Done false
10162 3921 Acked Write 6 Done Sent Done false
10163 3922 Acked Read 1 Done Sent Done false
10164 3923 Acked Flush 0 Done Sent Done false
10165 3924 Acked Read 6 Done Sent Done false
10166 3925 Acked Flush 0 Done Sent Done false
10167 3926 Acked Read 2 Done Sent Done false
10168 3927 Acked Write 2 Done Sent Done false
10169 3928 Acked Write 3 Done Sent Done false
10170 3929 Acked Write 1 Done Sent Done false
10171 3930 Acked Write 4 Done Sent Done false
10172 3931 Acked Read 1 Done Sent Done false
10173 3932 Acked Read 1 Done Sent Done false
10174 3933 Acked Write 10 Done Sent Done false
10175 3934 Acked Flush 0 Done Sent Done false
10176 3935 Acked Read 6 Done Sent Done false
10177 3936 Acked Write 9 Done Sent Done false
10178 3937 Acked Read 5 Done Sent Done false
10179 3938 Acked Write 1 Done Sent Done false
10180 3939 Acked Write 8 Done Sent Done false
10181 3940 Acked Read 6 Done Sent Done false
10182 3941 Acked Read 9 Done Sent Done false
10183 3942 Acked Write 9 Done Sent Done false
10184 3943 Acked Read 8 Done Sent Done false
10185 3944 Acked Write 8 Done Sent Done false
10186 3945 Acked Flush 0 Done Sent Done false
10187 3946 Acked Write 4 Done Sent Done false
10188 3947 Acked Read 7 Done Sent Done false
10189 3948 Acked Read 1 Done Sent Done false
10190 3949 Acked Read 1 Done Sent Done false
10191 3950 Acked Read 2 Done Sent Done false
10192 3951 Acked Write 9 Done Sent Done false
10193 3952 Acked Write 8 Done Sent Done false
10194 3953 Acked Flush 0 Done Sent Done false
10195 3954 Acked Flush 0 Done Sent Done false
10196 3955 Acked Read 9 Done Sent Done false
10197 3956 Acked Read 7 Done Sent Done false
10198 3957 Acked Read 10 Done Sent Done false
10199 3958 Acked Read 7 Done Sent Done false
10200 3959 Acked Read 10 Done Sent Done false
10201 3960 Acked Write 5 Done Sent Done false
10202 3961 Acked Write 10 Done Sent Done false
10203 3962 Acked Read 9 Done Sent Done false
10204 3963 Acked Read 2 Done Sent Done false
10205 3964 Acked Read 6 Done Sent Done false
10206 3965 Acked Read 3 Done Sent Done false
10207 3966 Acked Read 9 Done Sent Done false
10208 3967 Acked Read 2 Done Sent Done false
10209 3968 Acked Read 7 Done Sent Done false
10210 3969 Acked Read 10 Done Sent Done false
10211 3970 Acked Read 4 Done Sent Done false
10212 3971 Acked Write 5 Done Sent Done false
10213 3972 Acked Flush 0 Done Sent Done false
10214 3973 Acked Flush 0 Done Sent Done false
10215 3974 Acked Read 7 Done Sent Done false
10216 3975 Acked Write 1 Done Sent Done false
10217 3976 Acked Flush 0 Done Sent Done false
10218 3977 Acked Flush 0 Done Sent Done false
10219 3978 Acked Read 8 Done Sent Done false
10220 3979 Acked Read 4 Done Sent Done false
10221 3980 Acked Write 9 Done Sent Done false
10222 3981 Acked Write 10 Done Sent Done false
10223 3982 Acked Read 2 Done Sent Done false
10224 3983 Acked Write 6 Done Sent Done false
10225 3984 Acked Read 4 Done Sent Done false
10226 3985 Acked Write 3 Done Sent Done false
10227 3986 Acked Flush 0 Done Sent Done false
10228 3987 Acked Write 10 Done Sent Done false
10229 3988 Acked Write 7 Done Sent Done false
10230 3989 Acked Read 1 Done Sent Done false
10231 3990 Acked Read 1 Done Sent Done false
10232 3991 Acked Read 10 Done Sent Done false
10233 3992 Acked Read 1 Done Sent Done false
10234 3993 Acked Read 8 Done Sent Done false
10235 3994 Acked Write 9 Done Sent Done false
10236 3995 Acked Read 7 Done Sent Done false
10237 3996 Acked Read 4 Done Sent Done false
10238 3997 Acked Read 8 Done Sent Done false
10239 3998 Acked Read 10 Done Sent Done false
10240 3999 Acked Write 10 Done Sent Done false
10241 4000 Acked Read 9 Done Sent Done false
10242 4001 Acked Read 8 Done Sent Done false
10243 4002 Acked Write 1 Done Sent Done false
10244 4003 Acked Read 6 Done Sent Done false
10245 4004 Acked Write 10 Done Sent Done false
10246 4005 Acked Read 1 Done Sent Done false
10247 4006 Acked Write 3 Done Sent Done false
10248 4007 Acked Flush 0 Done Sent Done false
10249 4008 Acked Read 10 Done Sent Done false
10250 4009 Acked Write 8 Done Sent Done false
10251 4010 Acked Read 9 Done Sent Done false
10252 4011 Acked Read 2 Done Sent Done false
10253 4012 Acked Read 3 Done Sent Done false
10254 4013 Acked Read 1 Done Sent Done false
10255 4014 Acked Flush 0 Done Sent Done false
10256 4015 Acked Read 5 Done Sent Done false
10257 4016 Acked Flush 0 Done Sent Done false
10258 4017 Acked Read 1 Done Sent Done false
10259 STATES DS:0 DS:1 DS:2 TOTAL
10260 Sent 0 301 0 301
10261 Done 301 0 301 602
10262 Skipped 0 0 0 0
10263 Error 0 0 0 0
10264 Last Flush: 4016 3716 4016
10265 Downstairs last five completed: 3716 3715 3714 3713 3712
10266 Upstairs last five completed: 4017 4016 4015 4014 4013
10267 CLIENT: Up:0 ds:301 act:2
10268 [8820] got start action
10269 [8820] I am going to restart
10270 Starting downstairs at port 8820
10271 Make output file at "/tmp/dsc/downstairs-8820.txt"
10272 [8820][1] reports Starting
10273 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1212)
10274 [8820][1] reports Running
10275 ----------------------------------------------------------------
10276 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
10277 States: Active Offline Active
10278 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
10279 3717 Acked Read 9 Done Sent Done false
10280 3718 Acked Read 10 Done Sent Done false
10281 3719 Acked Write 9 Done Sent Done false
10282 3720 Acked Flush 0 Done Sent Done false
10283 3721 Acked Write 2 Done Sent Done false
10284 3722 Acked Write 7 Done Sent Done false
10285 3723 Acked Write 3 Done Sent Done false
10286 3724 Acked Write 8 Done Sent Done false
10287 3725 Acked Write 6 Done Sent Done false
10288 3726 Acked Write 9 Done Sent Done false
10289 3727 Acked Read 4 Done Sent Done false
10290 3728 Acked Write 2 Done Sent Done false
10291 3729 Acked Flush 0 Done Sent Done false
10292 3730 Acked Read 8 Done Sent Done false
10293 3731 Acked Read 3 Done Sent Done false
10294 3732 Acked Read 2 Done Sent Done false
10295 3733 Acked Write 7 Done Sent Done false
10296 3734 Acked Write 10 Done Sent Done false
10297 3735 Acked Write 9 Done Sent Done false
10298 3736 Acked Write 7 Done Sent Done false
10299 3737 Acked Write 6 Done Sent Done false
10300 3738 Acked Write 5 Done Sent Done false
10301 3739 Acked Flush 0 Done Sent Done false
10302 3740 Acked Read 8 Done Sent Done false
10303 3741 Acked Write 9 Done Sent Done false
10304 3742 Acked Read 3 Done Sent Done false
10305 3743 Acked Write 10 Done Sent Done false
10306 3744 Acked Write 10 Done Sent Done false
10307 3745 Acked Flush 0 Done Sent Done false
10308 3746 Acked Read 1 Done Sent Done false
10309 3747 Acked Write 9 Done Sent Done false
10310 3748 Acked Read 1 Done Sent Done false
10311 3749 Acked Read 8 Done Sent Done false
10312 3750 Acked Write 6 Done Sent Done false
10313 3751 Acked Read 5 Done Sent Done false
10314 3752 Acked Read 7 Done Sent Done false
10315 3753 Acked Write 1 Done Sent Done false
10316 3754 Acked Read 5 Done Sent Done false
10317 3755 Acked Read 1 Done Sent Done false
10318 3756 Acked Write 4 Done Sent Done false
10319 3757 Acked Write 8 Done Sent Done false
10320 3758 Acked Read 1 Done Sent Done false
10321 3759 Acked Write 3 Done Sent Done false
10322 3760 Acked Read 8 Done Sent Done false
10323 3761 Acked Read 4 Done Sent Done false
10324 3762 Acked Read 3 Done Sent Done false
10325 3763 Acked Read 10 Done Sent Done false
10326 3764 Acked Read 8 Done Sent Done false
10327 3765 Acked Read 10 Done Sent Done false
10328 3766 Acked Write 9 Done Sent Done false
10329 3767 Acked Read 3 Done Sent Done false
10330 3768 Acked Write 4 Done Sent Done false
10331 3769 Acked Write 9 Done Sent Done false
10332 3770 Acked Write 8 Done Sent Done false
10333 3771 Acked Read 7 Done Sent Done false
10334 3772 Acked Read 4 Done Sent Done false
10335 3773 Acked Read 3 Done Sent Done false
10336 3774 Acked Read 4 Done Sent Done false
10337 3775 Acked Write 2 Done Sent Done false
10338 3776 Acked Read 10 Done Sent Done false
10339 3777 Acked Read 9 Done Sent Done false
10340 3778 Acked Read 10 Done Sent Done false
10341 3779 Acked Write 8 Done Sent Done false
10342 3780 Acked Read 9 Done Sent Done false
10343 3781 Acked Flush 0 Done Sent Done false
10344 3782 Acked Write 9 Done Sent Done false
10345 3783 Acked Write 5 Done Sent Done false
10346 3784 Acked Read 5 Done Sent Done false
10347 3785 Acked Read 6 Done Sent Done false
10348 3786 Acked Write 5 Done Sent Done false
10349 3787 Acked Flush 0 Done Sent Done false
10350 3788 Acked Write 4 Done Sent Done false
10351 3789 Acked Write 5 Done Sent Done false
10352 3790 Acked Write 10 Done Sent Done false
10353 3791 Acked Read 1 Done Sent Done false
10354 3792 Acked Write 6 Done Sent Done false
10355 3793 Acked Read 1 Done Sent Done false
10356 3794 Acked Read 6 Done Sent Done false
10357 3795 Acked Write 8 Done Sent Done false
10358 3796 Acked Write 10 Done Sent Done false
10359 3797 Acked Read 2 Done Sent Done false
10360 3798 Acked Read 5 Done Sent Done false
10361 3799 Acked Write 8 Done Sent Done false
10362 3800 Acked Write 5 Done Sent Done false
10363 3801 Acked Read 4 Done Sent Done false
10364 3802 Acked Write 1 Done Sent Done false
10365 3803 Acked Flush 0 Done Sent Done false
10366 3804 Acked Read 6 Done Sent Done false
10367 3805 Acked Read 1 Done Sent Done false
10368 3806 Acked Read 8 Done Sent Done false
10369 3807 Acked Write 10 Done Sent Done false
10370 3808 Acked Read 7 Done Sent Done false
10371 3809 Acked Read 3 Done Sent Done false
10372 3810 Acked Read 3 Done Sent Done false
10373 3811 Acked Read 5 Done Sent Done false
10374 3812 Acked Read 7 Done Sent Done false
10375 3813 Acked Read 10 Done Sent Done false
10376 3814 Acked Read 8 Done Sent Done false
10377 3815 Acked Write 3 Done Sent Done false
10378 3816 Acked Write 7 Done Sent Done false
10379 3817 Acked Write 10 Done Sent Done false
10380 3818 Acked Write 10 Done Sent Done false
10381 3819 Acked Write 3 Done Sent Done false
10382 3820 Acked Write 5 Done Sent Done false
10383 3821 Acked Write 2 Done Sent Done false
10384 3822 Acked Read 3 Done Sent Done false
10385 3823 Acked Write 1 Done Sent Done false
10386 3824 Acked Read 8 Done Sent Done false
10387 3825 Acked Write 3 Done Sent Done false
10388 3826 Acked Read 4 Done Sent Done false
10389 3827 Acked Write 6 Done Sent Done false
10390 3828 Acked Read 6 Done Sent Done false
10391 3829 Acked Write 10 Done Sent Done false
10392 3830 Acked Write 5 Done Sent Done false
10393 3831 Acked Write 4 Done Sent Done false
10394 3832 Acked Write 2 Done Sent Done false
10395 3833 Acked Read 2 Done Sent Done false
10396 3834 Acked Read 3 Done Sent Done false
10397 3835 Acked Write 2 Done Sent Done false
10398 3836 Acked Write 4 Done Sent Done false
10399 3837 Acked Read 9 Done Sent Done false
10400 3838 Acked Read 9 Done Sent Done false
10401 3839 Acked Read 1 Done Sent Done false
10402 3840 Acked Write 10 Done Sent Done false
10403 3841 Acked Write 5 Done Sent Done false
10404 3842 Acked Write 2 Done Sent Done false
10405 3843 Acked Read 8 Done Sent Done false
10406 3844 Acked Read 5 Done Sent Done false
10407 3845 Acked Write 8 Done Sent Done false
10408 3846 Acked Write 1 Done Sent Done false
10409 3847 Acked Read 2 Done Sent Done false
10410 3848 Acked Read 6 Done Sent Done false
10411 3849 Acked Read 1 Done Sent Done false
10412 3850 Acked Read 1 Done Sent Done false
10413 3851 Acked Read 9 Done Sent Done false
10414 3852 Acked Read 5 Done Sent Done false
10415 3853 Acked Write 3 Done Sent Done false
10416 3854 Acked Write 9 Done Sent Done false
10417 3855 Acked Read 3 Done Sent Done false
10418 3856 Acked Read 7 Done Sent Done false
10419 3857 Acked Write 10 Done Sent Done false
10420 3858 Acked Read 9 Done Sent Done false
10421 3859 Acked Read 6 Done Sent Done false
10422 3860 Acked Write 3 Done Sent Done false
10423 3861 Acked Flush 0 Done Sent Done false
10424 3862 Acked Read 4 Done Sent Done false
10425 3863 Acked Flush 0 Done Sent Done false
10426 3864 Acked Flush 0 Done Sent Done false
10427 3865 Acked Read 8 Done Sent Done false
10428 3866 Acked Read 4 Done Sent Done false
10429 3867 Acked Read 5 Done Sent Done false
10430 3868 Acked Write 6 Done Sent Done false
10431 3869 Acked Write 3 Done Sent Done false
10432 3870 Acked Write 9 Done Sent Done false
10433 3871 Acked Read 5 Done Sent Done false
10434 3872 Acked Flush 0 Done Sent Done false
10435 3873 Acked Write 10 Done Sent Done false
10436 3874 Acked Write 1 Done Sent Done false
10437 3875 Acked Write 2 Done Sent Done false
10438 3876 Acked Read 6 Done Sent Done false
10439 3877 Acked Flush 0 Done Sent Done false
10440 3878 Acked Read 5 Done Sent Done false
10441 3879 Acked Read 2 Done Sent Done false
10442 3880 Acked Read 1 Done Sent Done false
10443 3881 Acked Write 1 Done Sent Done false
10444 3882 Acked Read 8 Done Sent Done false
10445 3883 Acked Write 8 Done Sent Done false
10446 3884 Acked Read 4 Done Sent Done false
10447 3885 Acked Write 10 Done Sent Done false
10448 3886 Acked Write 5 Done Sent Done false
10449 3887 Acked Read 7 Done Sent Done false
10450 3888 Acked Flush 0 Done Sent Done false
10451 3889 Acked Read 6 Done Sent Done false
10452 3890 Acked Read 3 Done Sent Done false
10453 3891 Acked Write 7 Done Sent Done false
10454 3892 Acked Read 10 Done Sent Done false
10455 3893 Acked Read 10 Done Sent Done false
10456 3894 Acked Flush 0 Done Sent Done false
10457 3895 Acked Write 10 Done Sent Done false
10458 3896 Acked Read 10 Done Sent Done false
10459 3897 Acked Write 4 Done Sent Done false
10460 3898 Acked Write 3 Done Sent Done false
10461 3899 Acked Read 4 Done Sent Done false
10462 3900 Acked Read 6 Done Sent Done false
10463 3901 Acked Read 6 Done Sent Done false
10464 3902 Acked Read 10 Done Sent Done false
10465 3903 Acked Write 1 Done Sent Done false
10466 3904 Acked Read 4 Done Sent Done false
10467 3905 Acked Flush 0 Done Sent Done false
10468 3906 Acked Write 5 Done Sent Done false
10469 3907 Acked Write 1 Done Sent Done false
10470 3908 Acked Read 9 Done Sent Done false
10471 3909 Acked Write 10 Done Sent Done false
10472 3910 Acked Write 4 Done Sent Done false
10473 3911 Acked Read 5 Done Sent Done false
10474 3912 Acked Write 4 Done Sent Done false
10475 3913 Acked Write 6 Done Sent Done false
10476 3914 Acked Read 4 Done Sent Done false
10477 3915 Acked Read 7 Done Sent Done false
10478 3916 Acked Read 9 Done Sent Done false
10479 3917 Acked Read 1 Done Sent Done false
10480 3918 Acked Read 9 Done Sent Done false
10481 3919 Acked Write 10 Done Sent Done false
10482 3920 Acked Write 7 Done Sent Done false
10483 3921 Acked Write 6 Done Sent Done false
10484 3922 Acked Read 1 Done Sent Done false
10485 3923 Acked Flush 0 Done Sent Done false
10486 3924 Acked Read 6 Done Sent Done false
10487 3925 Acked Flush 0 Done Sent Done false
10488 3926 Acked Read 2 Done Sent Done false
10489 3927 Acked Write 2 Done Sent Done false
10490 3928 Acked Write 3 Done Sent Done false
10491 3929 Acked Write 1 Done Sent Done false
10492 3930 Acked Write 4 Done Sent Done false
10493 3931 Acked Read 1 Done Sent Done false
10494 3932 Acked Read 1 Done Sent Done false
10495 3933 Acked Write 10 Done Sent Done false
10496 3934 Acked Flush 0 Done Sent Done false
10497 3935 Acked Read 6 Done Sent Done false
10498 3936 Acked Write 9 Done Sent Done false
10499 3937 Acked Read 5 Done Sent Done false
10500 3938 Acked Write 1 Done Sent Done false
10501 3939 Acked Write 8 Done Sent Done false
10502 3940 Acked Read 6 Done Sent Done false
10503 3941 Acked Read 9 Done Sent Done false
10504 3942 Acked Write 9 Done Sent Done false
10505 3943 Acked Read 8 Done Sent Done false
10506 3944 Acked Write 8 Done Sent Done false
10507 3945 Acked Flush 0 Done Sent Done false
10508 3946 Acked Write 4 Done Sent Done false
10509 3947 Acked Read 7 Done Sent Done false
10510 3948 Acked Read 1 Done Sent Done false
10511 3949 Acked Read 1 Done Sent Done false
10512 3950 Acked Read 2 Done Sent Done false
10513 3951 Acked Write 9 Done Sent Done false
10514 3952 Acked Write 8 Done Sent Done false
10515 3953 Acked Flush 0 Done Sent Done false
10516 3954 Acked Flush 0 Done Sent Done false
10517 3955 Acked Read 9 Done Sent Done false
10518 3956 Acked Read 7 Done Sent Done false
10519 3957 Acked Read 10 Done Sent Done false
10520 3958 Acked Read 7 Done Sent Done false
10521 3959 Acked Read 10 Done Sent Done false
10522 3960 Acked Write 5 Done Sent Done false
10523 3961 Acked Write 10 Done Sent Done false
10524 3962 Acked Read 9 Done Sent Done false
10525 3963 Acked Read 2 Done Sent Done false
10526 3964 Acked Read 6 Done Sent Done false
10527 3965 Acked Read 3 Done Sent Done false
10528 3966 Acked Read 9 Done Sent Done false
10529 3967 Acked Read 2 Done Sent Done false
10530 3968 Acked Read 7 Done Sent Done false
10531 3969 Acked Read 10 Done Sent Done false
10532 3970 Acked Read 4 Done Sent Done false
10533 3971 Acked Write 5 Done Sent Done false
10534 3972 Acked Flush 0 Done Sent Done false
10535 3973 Acked Flush 0 Done Sent Done false
10536 3974 Acked Read 7 Done Sent Done false
10537 3975 Acked Write 1 Done Sent Done false
10538 3976 Acked Flush 0 Done Sent Done false
10539 3977 Acked Flush 0 Done Sent Done false
10540 3978 Acked Read 8 Done Sent Done false
10541 3979 Acked Read 4 Done Sent Done false
10542 3980 Acked Write 9 Done Sent Done false
10543 3981 Acked Write 10 Done Sent Done false
10544 3982 Acked Read 2 Done Sent Done false
10545 3983 Acked Write 6 Done Sent Done false
10546 3984 Acked Read 4 Done Sent Done false
10547 3985 Acked Write 3 Done Sent Done false
10548 3986 Acked Flush 0 Done Sent Done false
10549 3987 Acked Write 10 Done Sent Done false
10550 3988 Acked Write 7 Done Sent Done false
10551 3989 Acked Read 1 Done Sent Done false
10552 3990 Acked Read 1 Done Sent Done false
10553 3991 Acked Read 10 Done Sent Done false
10554 3992 Acked Read 1 Done Sent Done false
10555 3993 Acked Read 8 Done Sent Done false
10556 3994 Acked Write 9 Done Sent Done false
10557 3995 Acked Read 7 Done Sent Done false
10558 3996 Acked Read 4 Done Sent Done false
10559 3997 Acked Read 8 Done Sent Done false
10560 3998 Acked Read 10 Done Sent Done false
10561 3999 Acked Write 10 Done Sent Done false
10562 4000 Acked Read 9 Done Sent Done false
10563 4001 Acked Read 8 Done Sent Done false
10564 4002 Acked Write 1 Done Sent Done false
10565 4003 Acked Read 6 Done Sent Done false
10566 4004 Acked Write 10 Done Sent Done false
10567 4005 Acked Read 1 Done Sent Done false
10568 4006 Acked Write 3 Done Sent Done false
10569 4007 Acked Flush 0 Done Sent Done false
10570 4008 Acked Read 10 Done Sent Done false
10571 4009 Acked Write 8 Done Sent Done false
10572 4010 Acked Read 9 Done Sent Done false
10573 4011 Acked Read 2 Done Sent Done false
10574 4012 Acked Read 3 Done Sent Done false
10575 4013 Acked Read 1 Done Sent Done false
10576 4014 Acked Flush 0 Done Sent Done false
10577 4015 Acked Read 5 Done Sent Done false
10578 4016 Acked Flush 0 Done Sent Done false
10579 4017 Acked Read 1 Done Sent Done false
10580 4018 Acked Flush 0 Done Sent Done false
10581 STATES DS:0 DS:1 DS:2 TOTAL
10582 Sent 0 302 0 302
10583 Done 302 0 302 604
10584 Skipped 0 0 0 0
10585 Error 0 0 0 0
10586 Last Flush: 4018 3716 4018
10587 Downstairs last five completed: 3716 3715 3714 3713 3712
10588 Upstairs last five completed: 4018 4017 4016 4015 4014
10589 CLIENT: Up:0 ds:302 act:2
105902024-11-28T04:52:31.193ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
105912024-11-28T04:52:31.193ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:40828) = io task client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
105922024-11-28T04:52:31.194ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
105932024-11-28T04:52:31.195ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
105942024-11-28T04:52:31.195ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
105952024-11-28T04:52:31.195ZINFOcrucible: send last flush ID to this DS: 3716 = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
105962024-11-28T04:52:31.195ZINFOcrucible: Replied this last flush ID: 3716 = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
105972024-11-28T04:52:31.195ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
105982024-11-28T04:52:31.195ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
105992024-11-28T04:52:31.195ZINFOcrucible: [1] client re-new 302 jobs since flush 3716 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
106002024-11-28T04:52:31.195ZINFOcrucible: [1] Replayed 302 jobs since flush: 3716 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
10601 ----------------------------------------------------------------
10602 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
10603 Downstairs last five completed: 4018 4017 4016 4015 4014
10604 Upstairs last five completed: 4018 4017 4016 4015 4014
10605 CLIENT: Up:0 ds:0 act:3
10606 Replay: All jobs finished
10607 Main task has work to do, go find it
10608 got dsc Stop(1)
10609 stop 1
10610 [8820] Got stop action so:false kr:false
10611 Nov 28 04:52:33.439 INFO request completed, latency_us: 322, response_code: 204, uri: /stop/cid/1, method: POST, req_id: eb4df1ef-765d-476d-a9fd-d2450838f8bc, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
10612 Nov 28 04:52:33.439 INFO {"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2024-11-28T04:52:33.439859916Z","hostname":"w-01JDRHKEEFJ1228TFPSBEGXWHY","pid":1202,"":"io task","client":"1","":"downstairs","session_id":"4a2415b1-6594-436f-82e5-b8cf9a74eeaf"}
10613 request completed, latency_us: 114, response_code: 200, uri: /state/cid/1, method: GET, req_id: f0997f40-aaed-40b9-928a-92fbf261d324, remote_addr: 127{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2024-11-28T04:52:33.439924017Z","hostname":"w-01JDRHKEEFJ1228TFPSBEGXWHY","pid":1202,"":"io task","client":"1","":"downstairs","session_id":"4a2415b1-6594-436f-82e5-b8cf9a74eeaf"}
10614 .0.0.1:60418, local_addr: 127.0{"msg":"downstairs task for 1 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2024-11-28T04:52:33.439978269Z","hostname":"w-01JDRHKEEFJ1228TFPSBEGXWHY","pid":1202,"session_id":"4a2415b1-6594-436f-82e5-b8cf9a74eeaf"}
10615 .{"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-28T04:52:33.440011175Z","hostname":"w-01JDRHKEEFJ1228TFPSBEGXWHY","pid":1202,"client":"1","":"downstairs","session_id":"4a2415b1-6594-436f-82e5-b8cf9a74eeaf"}
10616 0.1:9998
106172024-11-28T04:52:33.440ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
106182024-11-28T04:52:33.440ZINFOcrucible: client task is exiting = io task client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
10619 [8820] Exited with: ExitStatus(unix_wait_status(9))
10620 [8820][1] reports Exit
10621 Nov 28 04:52:37.441 INFO request completed, latency_us: 218, response_code: 200, uri: /state/cid/1, method: GET, req_id: e68ece58-7e0a-4be1-83aa-8493629d8a27, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
10622 001/300 Read block 1574 len 4096
10623 002/300 Read block 2387 len 12288
10624 003/300 Read block 3875 len 40960
10625 004/300 Write block 3325 len 20480 data: 3 3 3 4 5
10626 005/300 Read block 2173 len 36864
10627 006/300 Read block 1516 len 28672
10628 007/300 Read block 1333 len 8192
10629 008/300 Write block 954 len 16384 data: 2 2 2 2
10630 009/300 Write block 4420 len 4096 data: 2
10631 010/300 Read block 647 len 8192
10632 011/300 Read block 873 len 32768
10633 012/300 Write block 4358 len 36864 data: 5 5 5 6 5 5 5 5 4
10634 013/300 Read block 4288 len 4096
10635 014/300 Flush
10636 015/300 Write block 14 len 28672 data: 6 6 7 6 5 3 3
10637 016/300 Write block 3074 len 20480 data: 3 3 3 3 3
10638 017/300 Flush
10639 018/300 Read block 2509 len 32768
10640 019/300 Read block 1184 len 40960
10641 020/300 Read block 496 len 16384
10642 021/300 Read block 2515 len 4096
10643 022/300 Read block 860 len 40960
10644 023/300 Write block 4856 len 16384 data: 3 4 3 3
10645 024/300 Write block 3005 len 12288 data: 5 5 4
10646 025/300 Read block 203 len 12288
10647 026/300 Read block 3770 len 24576
10648 027/300 Write block 3222 len 16384 data: 3 3 3 4
10649 028/300 Write block 1988 len 8192 data: 2 2
10650 029/300 Write block 4980 len 32768 data: 3 3 3 3 3 3 3 3
10651 030/300 Write block 4803 len 16384 data: 4 4 5 4
10652 031/300 Read block 2642 len 4096
10653 032/300 Read block 1588 len 8192
10654 033/300 Flush
10655 034/300 Write block 3215 len 16384 data: 4 4 5 6
10656 035/300 Read block 3476 len 4096
10657 036/300 Write block 2044 len 16384 data: 2 2 2 3
10658 037/300 Read block 3213 len 32768
10659 038/300 Write block 1688 len 12288 data: 3 3 3
10660 039/300 Read block 3472 len 24576
10661 040/300 Write block 1658 len 12288 data: 2 2 2
10662 041/300 Write block 4012 len 40960 data: 2 2 2 2 2 2 3 2 2 2
10663 042/300 Read block 2843 len 20480
10664 043/300 Write block 4033 len 4096 data: 4
10665 044/300 Write block 1147 len 4096 data: 3
10666 045/300 Write block 4805 len 4096 data: 6
10667 046/300 Read block 531 len 40960
10668 047/300 Read block 2407 len 20480
10669 048/300 Write block 2958 len 36864 data: 2 3 4 4 6 5 4 5 4
10670 049/300 Read block 226 len 16384
10671 050/300 Write block 3286 len 16384 data: 6 6 6 5
10672 051/300 Write block 1703 len 28672 data: 5 5 7 8 9 10 11
10673 052/300 Write block 3686 len 40960 data: 2 3 3 2 2 2 2 3 3 3
10674 053/300 Read block 3557 len 8192
10675 054/300 Read block 3549 len 36864
10676 055/300 Read block 929 len 4096
10677 056/300 Write block 2194 len 16384 data: 5 5 4 3
10678 057/300 Read block 2542 len 36864
10679 058/300 Write block 3717 len 16384 data: 2 2 2 2
10680 059/300 Read block 2009 len 12288
10681 060/300 Read block 2530 len 32768
10682 061/300 Flush
10683 062/300 Write block 634 len 24576 data: 6 7 7 6 6 6
10684 063/300 Read block 3933 len 12288
10685 064/300 Write block 3041 len 32768 data: 4 4 4 4 4 3 3 3
10686 065/300 Write block 1219 len 40960 data: 2 3 3 3 3 3 2 3 3 2
10687 066/300 Read block 607 len 8192
10688 067/300 Write block 3115 len 24576 data: 2 3 3 3 4 3
10689 068/300 Write block 3008 len 20480 data: 4 3 2 3 3
10690 069/300 Read block 785 len 8192
10691 070/300 Write block 1606 len 4096 data: 4
10692 071/300 Read block 4210 len 32768
10693 072/300 Read block 114 len 32768
10694 073/300 Write block 2520 len 28672 data: 4 5 6 6 6 7 7
10695 074/300 Read block 1437 len 24576
10696 075/300 Read block 4445 len 40960
10697 076/300 Write block 1104 len 8192 data: 4 3
10698 077/300 Write block 436 len 40960 data: 2 2 3 4 4 3 3 3 3 2
10699 078/300 Write block 944 len 20480 data: 2 2 2 2 2
10700 079/300 Write block 862 len 16384 data: 5 5 4 5
10701 080/300 Write block 4791 len 36864 data: 2 2 2 2 2 2 3 3 3
10702 081/300 Write block 513 len 24576 data: 5 4 4 3 3 3
10703 082/300 Write block 4558 len 40960 data: 2 2 3 3 4 4 4 5 5 4
10704 083/300 Read block 162 len 12288
10705 084/300 Write block 4446 len 20480 data: 7 6 5 5 5
10706 085/300 Read block 1296 len 12288
10707 086/300 Write block 3858 len 12288 data: 3 3 3
10708 087/300 Write block 3013 len 8192 data: 3 3
10709 088/300 Write block 2226 len 24576 data: 3 3 4 3 3 3
10710 089/300 Read block 4116 len 32768
10711 090/300 Write block 1508 len 20480 data: 2 2 2 2 2
10712 091/300 Read block 4329 len 16384
10713 092/300 Read block 1565 len 24576
10714 093/300 Read block 4337 len 24576
10715 094/300 Write block 432 len 20480 data: 2 2 2 2 3
10716 095/300 Read block 1557 len 8192
10717 096/300 Write block 2741 len 4096 data: 2
10718 097/300 Read block 3881 len 12288
10719 098/300 Read block 4855 len 32768
10720 099/300 Write block 4116 len 28672 data: 2 2 4 4 4 4 6
10721 100/300 Read block 2412 len 8192
10722 101/300 Read block 2936 len 20480
10723 102/300 Flush
10724 103/300 Read block 927 len 20480
10725 104/300 Read block 2363 len 28672
10726 105/300 Write block 642 len 32768 data: 3 3 3 3 2 2 2 4
10727 106/300 Read block 1015 len 28672
10728 107/300 Write block 4575 len 40960 data: 3 3 3 3 2 2 2 4 4 4
10729 108/300 Read block 428 len 32768
10730 109/300 Read block 2483 len 40960
10731 110/300 Write block 3539 len 20480 data: 4 5 4 4 2
10732 111/300 Write block 3649 len 12288 data: 4 5 4
10733 112/300 Read block 4191 len 8192
10734 113/300 Write block 2899 len 16384 data: 2 2 3 3
10735 114/300 Write block 3465 len 4096 data: 3
10736 115/300 Read block 1812 len 16384
10737 116/300 Write block 3023 len 8192 data: 3 4
10738 117/300 Read block 2122 len 12288
10739 118/300 Read block 3230 len 8192
10740 119/300 Read block 2576 len 24576
10741 120/300 Write block 662 len 36864 data: 8 7 6 5 6 7 7 5 5
10742 121/300 Read block 4152 len 24576
10743 122/300 Read block 911 len 40960
10744 123/300 Read block 4994 len 4096
10745 124/300 Write block 2084 len 8192 data: 4 4
10746 125/300 Write block 4322 len 4096 data: 2
10747 126/300 Read block 357 len 8192
10748 127/300 Write block 3917 len 4096 data: 2
10749 128/300 Read block 3920 len 24576
10750 129/300 Read block 4785 len 24576
10751 130/300 Read block 1288 len 32768
10752 131/300 Write block 3859 len 12288 data: 4 4 3
10753 132/300 Read block 3209 len 40960
10754 133/300 Read block 1470 len 12288
10755 134/300 Write block 2076 len 24576 data: 2 2 3 3 3 3
10756 135/300 Read block 3479 len 32768
10757 136/300 Write block 4163 len 20480 data: 3 3 3 3 3
10758 137/300 Write block 2715 len 12288 data: 5 5 5
10759 138/300 Read block 2545 len 32768
10760 139/300 Read block 4480 len 36864
10761 140/300 Read block 827 len 8192
10762 141/300 Write block 2887 len 28672 data: 2 2 2 3 3 3 3
10763 142/300 Read block 1860 len 40960
10764 143/300 Read block 3924 len 24576
10765 144/300 Write block 4345 len 16384 data: 5 5 5 5
10766 145/300 Write block 2649 len 8192 data: 3 3
10767 146/300 Read block 2797 len 24576
10768 147/300 Write block 2297 len 4096 data: 2
10769 148/300 Write block 162 len 20480 data: 4 5 5 4 5
10770 149/300 Read block 1905 len 36864
10771 150/300 Write block 3134 len 32768 data: 4 4 3 3 4 3 4 4
10772 151/300 Read block 2798 len 12288
10773 152/300 Write block 162 len 12288 data: 5 6 6
10774 153/300 Flush
10775 154/300 Read block 1422 len 28672
10776 155/300 Write block 4774 len 36864 data: 4 3 2 2 2 2 2 2 2
10777 156/300 Write block 3823 len 12288 data: 4 4 4
10778 157/300 Read block 4514 len 16384
10779 158/300 Read block 1473 len 8192
10780 159/300 Read block 544 len 4096
10781 160/300 Read block 1931 len 16384
10782 161/300 Read block 1724 len 36864
10783 162/300 Write block 4230 len 40960 data: 2 2 2 2 2 2 2 3 2 2
10784 163/300 Write block 1914 len 36864 data: 2 2 2 2 2 2 2 2 2
10785 164/300 Read block 2526 len 28672
10786 165/300 Write block 1716 len 20480 data: 4 3 3 2 2
10787 166/300 Read block 4206 len 28672
10788 167/300 Write block 2630 len 12288 data: 2 2 2
10789 168/300 Write block 3255 len 28672 data: 4 4 4 4 2 3 3
10790 169/300 Write block 1364 len 12288 data: 4 4 3
10791 170/300 Read block 2557 len 28672
10792 171/300 Flush
10793 172/300 Read block 3078 len 4096
10794 173/300 Read block 3195 len 12288
10795 174/300 Write block 1026 len 36864 data: 4 3 3 2 3 3 4 5 4
10796 175/300 Read block 972 len 24576
10797 176/300 Write block 4216 len 24576 data: 4 4 4 4 4 4
10798 177/300 Read block 927 len 32768
10799 178/300 Write block 851 len 8192 data: 4 4
10800 179/300 Read block 933 len 36864
10801 180/300 Write block 2391 len 32768 data: 3 3 3 4 5 5 5 3
10802 181/300 Write block 3805 len 16384 data: 2 3 3 2
10803 182/300 Read block 2327 len 28672
10804 183/300 Write block 612 len 24576 data: 3 3 3 3 4 4
10805 184/300 Read block 1134 len 4096
10806 185/300 Read block 1306 len 16384
10807 186/300 Read block 2955 len 40960
10808 187/300 Write block 3410 len 28672 data: 2 3 3 3 3 3 4
10809 188/300 Write block 1789 len 28672 data: 4 4 3 3 2 4 4
10810 189/300 Write block 1796 len 8192 data: 4 3
10811 190/300 Read block 2719 len 28672
10812 191/300 Write block 3708 len 12288 data: 2 3 3
10813 192/300 Write block 1107 len 8192 data: 2 3
10814 193/300 Write block 3241 len 8192 data: 3 3
10815 194/300 Read block 3968 len 32768
10816 195/300 Write block 867 len 28672 data: 5 3 3 3 3 3 4
10817 196/300 Read block 3959 len 28672
10818 197/300 Write block 3995 len 12288 data: 3 3 3
10819 198/300 Read block 3824 len 8192
10820 199/300 Write block 4534 len 20480 data: 3 3 3 2 2
10821 200/300 Read block 3637 len 36864
10822 201/300 Read block 3586 len 40960
10823 202/300 Read block 4282 len 20480
10824 203/300 Write block 3079 len 32768 data: 3 4 4 4 3 4 3 2
10825 204/300 Read block 3318 len 32768
10826 205/300 Write block 3699 len 28672 data: 4 2 2 2 2 2 2
10827 206/300 Read block 2135 len 20480
10828 207/300 Read block 3027 len 8192
10829 208/300 Read block 3051 len 12288
10830 209/300 Write block 2241 len 36864 data: 3 3 3 3 3 3 3 3 3
10831 210/300 Read block 4210 len 24576
10832 211/300 Write block 1977 len 4096 data: 3
10833 212/300 Flush
10834 213/300 Read block 1646 len 36864
10835 214/300 Write block 4721 len 40960 data: 3 3 3 3 4 4 3 3 3 3
10836 215/300 Read block 748 len 20480
10837 216/300 Read block 3962 len 4096
10838 217/300 Write block 4119 len 12288 data: 5 5 5
10839 218/300 Read block 840 len 32768
10840 219/300 Write block 1108 len 40960 data: 4 3 3 3 2 2 2 2 2 2
10841 220/300 Read block 1703 len 16384
10842 221/300 Flush
10843 222/300 Write block 2662 len 20480 data: 3 3 3 4 4
10844 223/300 Write block 4642 len 32768 data: 5 5 3 3 4 4 5 5
10845 224/300 Write block 2283 len 12288 data: 4 5 4
10846 225/300 Write block 1087 len 24576 data: 3 3 4 4 3 3
10847 226/300 Write block 1543 len 28672 data: 3 3 3 2 2 2 4
10848 227/300 Read block 2426 len 28672
10849 228/300 Write block 175 len 8192 data: 2 2
10850 229/300 Flush
10851 230/300 Read block 4651 len 4096
10852 231/300 Flush
10853 232/300 Write block 3961 len 12288 data: 4 4 3
10854 233/300 Flush
10855 234/300 Write block 2148 len 20480 data: 2 2 2 2 2
10856 235/300 Read block 3017 len 40960
10857 236/300 Write block 1811 len 24576 data: 2 2 2 3 3 3
10858 237/300 Write block 322 len 40960 data: 6 6 5 5 3 3 3 3 3 3
10859 238/300 Flush
10860 239/300 Write block 2550 len 32768 data: 2 2 2 2 2 2 2 3
10861 240/300 Write block 4522 len 4096 data: 3
10862 241/300 Write block 3049 len 4096 data: 4
10863 242/300 Read block 4067 len 20480
10864 243/300 Write block 201 len 4096 data: 3
10865 244/300 Read block 3664 len 28672
10866 245/300 Write block 1845 len 36864 data: 3 3 4 3 4 3 4 4 4
10867 246/300 Write block 1629 len 24576 data: 3 3 2 2 2 2
10868 247/300 Read block 3122 len 36864
10869 248/300 Write block 4455 len 16384 data: 5 5 5 5
10870 249/300 Read block 3162 len 4096
10871 250/300 Read block 3529 len 8192
10872 251/300 Write block 3113 len 32768 data: 2 2 3 4 4 4 5 4
10873 252/300 Write block 210 len 16384 data: 4 3 3 3
10874 253/300 Read block 2056 len 28672
10875 254/300 Read block 3942 len 12288
10876 255/300 Write block 3164 len 36864 data: 3 4 4 3 3 3 2 3 3
10877 256/300 Write block 1218 len 32768 data: 2 3 4 4 4 4 4 3
10878 257/300 Write block 1158 len 28672 data: 2 2 2 2 2 2 2
10879 258/300 Write block 3429 len 32768 data: 5 5 4 5 5 4 3 4
10880 259/300 Flush
10881 260/300 Read block 512 len 40960
10882 261/300 Read block 1380 len 20480
10883 262/300 Write block 3198 len 8192 data: 2 2
10884 263/300 Read block 4216 len 28672
10885 264/300 Read block 4112 len 32768
10886 265/300 Flush
10887 266/300 Write block 1958 len 32768 data: 3 3 3 3 3 3 4 4
10888 267/300 Flush
10889 268/300 Read block 3463 len 20480
10890 269/300 Read block 539 len 16384
10891 270/300 Read block 2944 len 20480
10892 271/300 Write block 366 len 12288 data: 6 5 5
10893 272/300 Read block 2563 len 24576
10894 273/300 Write block 4869 len 16384 data: 2 2 3 3
10895 274/300 Read block 2181 len 8192
10896 275/300 Write block 3923 len 32768 data: 3 3 5 3 3 3 3 3
10897 276/300 Write block 211 len 40960 data: 4 4 4 2 2 2 3 3 3 3
10898 277/300 Read block 3442 len 24576
10899 278/300 Read block 3815 len 32768
10900 279/300 Write block 179 len 24576 data: 2 3 3 2 2 2
10901 280/300 Read block 3238 len 28672
10902 281/300 Read block 2060 len 8192
10903 282/300 Read block 2391 len 12288
10904 283/300 Read block 1906 len 20480
10905 284/300 Write block 967 len 32768 data: 6 5 4 4 4 4 3 3
10906 285/300 Read block 2465 len 28672
10907 286/300 Write block 1355 len 20480 data: 3 2 3 3 3
10908 287/300 Write block 2941 len 4096 data: 2
10909 288/300 Write block 1791 len 4096 data: 4
10910 289/300 Write block 3744 len 8192 data: 2 2
10911 290/300 Read block 3391 len 40960
10912 291/300 Read block 1386 len 20480
10913 292/300 Read block 3293 len 12288
10914 293/300 Read block 2158 len 28672
10915 294/300 Write block 2364 len 8192 data: 6 4
10916 295/300 Read block 2785 len 28672
10917 296/300 Read block 1105 len 4096
10918 297/300 Read block 951 len 40960
10919 298/300 Write block 3693 len 24576 data: 4 4 4 4 4 4
10920 299/300 Read block 2773 len 32768
10921 300/300 Read block 532 len 28672
10922 301/300 Write block 1921 len 4096 data: 3
10923 Main task has work to do, go find it
10924 got dsc Start(1)
10925 start 1
10926 Nov 28 04:52:37.674 INFO request completed, latency_us: 221, response_code: 204, uri: /start/cid/1, method: POST, req_id: baecf164-06e5-4dce-801d-b61386eebb4e, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
10927 Replay: started 1, returned:Ok(())
10928 ----------------------------------------------------------------
10929 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
10930 States: Active Offline Active
10931 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
10932 4019 Acked Read 1 Done Sent Done false
10933 4020 Acked Read 3 Done Sent Done false
10934 4021 Acked Read 10 Done Sent Done false
10935 4022 Acked Write 5 Done Sent Done false
10936 4023 Acked Read 9 Done Sent Done false
10937 4024 Acked Read 7 Done Sent Done false
10938 4025 Acked Read 2 Done Sent Done false
10939 4026 Acked Write 4 Done Sent Done false
10940 4027 Acked Write 1 Done Sent Done false
10941 4028 Acked Read 2 Done Sent Done false
10942 4029 Acked Read 8 Done Sent Done false
10943 4030 Acked Write 9 Done Sent Done false
10944 4031 Acked Read 1 Done Sent Done false
10945 4032 Acked Flush 0 Done Sent Done false
10946 4033 Acked Write 7 Done Sent Done false
10947 4034 Acked Write 5 Done Sent Done false
10948 4035 Acked Flush 0 Done Sent Done false
10949 4036 Acked Read 8 Done Sent Done false
10950 4037 Acked Read 10 Done Sent Done false
10951 4038 Acked Read 4 Done Sent Done false
10952 4039 Acked Read 1 Done Sent Done false
10953 4040 Acked Read 10 Done Sent Done false
10954 4041 Acked Write 4 Done Sent Done false
10955 4042 Acked Write 3 Done Sent Done false
10956 4043 Acked Read 3 Done Sent Done false
10957 4044 Acked Read 6 Done Sent Done false
10958 4045 Acked Write 4 Done Sent Done false
10959 4046 Acked Write 2 Done Sent Done false
10960 4047 Acked Write 8 Done Sent Done false
10961 4048 Acked Write 4 Done Sent Done false
10962 4049 Acked Read 1 Done Sent Done false
10963 4050 Acked Read 2 Done Sent Done false
10964 4051 Acked Flush 0 Done Sent Done false
10965 4052 Acked Write 4 Done Sent Done false
10966 4053 Acked Read 1 Done Sent Done false
10967 4054 Acked Write 4 Done Sent Done false
10968 4055 Acked Read 8 Done Sent Done false
10969 4056 Acked Write 3 Done Sent Done false
10970 4057 Acked Read 6 Done Sent Done false
10971 4058 Acked Write 3 Done Sent Done false
10972 4059 Acked Write 10 Done Sent Done false
10973 4060 Acked Read 5 Done Sent Done false
10974 4061 Acked Write 1 Done Sent Done false
10975 4062 Acked Write 1 Done Sent Done false
10976 4063 Acked Write 1 Done Sent Done false
10977 4064 Acked Read 10 Done Sent Done false
10978 4065 Acked Read 5 Done Sent Done false
10979 4066 Acked Write 9 Done Sent Done false
10980 4067 Acked Read 4 Done Sent Done false
10981 4068 Acked Write 4 Done Sent Done false
10982 4069 Acked Write 7 Done Sent Done false
10983 4070 Acked Write 10 Done Sent Done false
10984 4071 Acked Read 2 Done Sent Done false
10985 4072 Acked Read 9 Done Sent Done false
10986 4073 Acked Read 1 Done Sent Done false
10987 4074 Acked Write 4 Done Sent Done false
10988 4075 Acked Read 9 Done Sent Done false
10989 4076 Acked Write 4 Done Sent Done false
10990 4077 Acked Read 3 Done Sent Done false
10991 4078 Acked Read 8 Done Sent Done false
10992 4079 Acked Flush 0 Done Sent Done false
10993 4080 Acked Write 6 Done Sent Done false
10994 4081 Acked Read 3 Done Sent Done false
10995 4082 Acked Write 8 Done Sent Done false
10996 4083 Acked Write 10 Done Sent Done false
10997 4084 Acked Read 2 Done Sent Done false
10998 4085 Acked Write 6 Done Sent Done false
10999 4086 Acked Write 5 Done Sent Done false
11000 4087 Acked Read 2 Done Sent Done false
11001 4088 Acked Write 1 Done Sent Done false
11002 4089 Acked Read 8 Done Sent Done false
11003 4090 Acked Read 8 Done Sent Done false
11004 4091 Acked Write 7 Done Sent Done false
11005 4092 Acked Read 6 Done Sent Done false
11006 4093 Acked Read 10 Done Sent Done false
11007 4094 Acked Write 2 Done Sent Done false
11008 4095 Acked Write 10 Done Sent Done false
11009 4096 Acked Write 5 Done Sent Done false
11010 4097 Acked Write 4 Done Sent Done false
11011 4098 Acked Write 9 Done Sent Done false
11012 4099 Acked Write 6 Done Sent Done false
11013 4100 Acked Write 10 Done Sent Done false
11014 4101 Acked Read 3 Done Sent Done false
11015 4102 Acked Write 5 Done Sent Done false
11016 4103 Acked Read 3 Done Sent Done false
11017 4104 Acked Write 3 Done Sent Done false
11018 4105 Acked Write 2 Done Sent Done false
11019 4106 Acked Write 6 Done Sent Done false
11020 4107 Acked Read 8 Done Sent Done false
11021 4108 Acked Write 5 Done Sent Done false
11022 4109 Acked Read 4 Done Sent Done false
11023 4110 Acked Read 6 Done Sent Done false
11024 4111 Acked Read 6 Done Sent Done false
11025 4112 Acked Write 5 Done Sent Done false
11026 4113 Acked Read 2 Done Sent Done false
11027 4114 Acked Write 1 Done Sent Done false
11028 4115 Acked Read 3 Done Sent Done false
11029 4116 Acked Read 8 Done Sent Done false
11030 4117 Acked Write 7 Done Sent Done false
11031 4118 Acked Read 2 Done Sent Done false
11032 4119 Acked Read 5 Done Sent Done false
11033 4120 Acked Flush 0 Done Sent Done false
11034 4121 Acked Read 5 Done Sent Done false
11035 4122 Acked Read 7 Done Sent Done false
11036 4123 Acked Write 8 Done Sent Done false
11037 4124 Acked Read 7 Done Sent Done false
11038 4125 Acked Write 10 Done Sent Done false
11039 4126 Acked Read 8 Done Sent Done false
11040 4127 Acked Read 10 Done Sent Done false
11041 4128 Acked Write 5 Done Sent Done false
11042 4129 Acked Write 3 Done Sent Done false
11043 4130 Acked Read 2 Done Sent Done false
11044 4131 Acked Write 4 Done Sent Done false
11045 4132 Acked Write 1 Done Sent Done false
11046 4133 Acked Read 4 Done Sent Done false
11047 4134 Acked Write 2 Done Sent Done false
11048 4135 Acked Read 3 Done Sent Done false
11049 4136 Acked Read 2 Done Sent Done false
11050 4137 Acked Read 6 Done Sent Done false
11051 4138 Acked Write 9 Done Sent Done false
11052 4139 Acked Read 6 Done Sent Done false
11053 4140 Acked Read 10 Done Sent Done false
11054 4141 Acked Read 1 Done Sent Done false
11055 4142 Acked Write 2 Done Sent Done false
11056 4143 Acked Write 1 Done Sent Done false
11057 4144 Acked Read 2 Done Sent Done false
11058 4145 Acked Write 1 Done Sent Done false
11059 4146 Acked Read 6 Done Sent Done false
11060 4147 Acked Read 6 Done Sent Done false
11061 4148 Acked Read 8 Done Sent Done false
11062 4149 Acked Write 3 Done Sent Done false
11063 4150 Acked Read 10 Done Sent Done false
11064 4151 Acked Read 3 Done Sent Done false
11065 4152 Acked Write 6 Done Sent Done false
11066 4153 Acked Read 8 Done Sent Done false
11067 4154 Acked Write 5 Done Sent Done false
11068 4155 Acked Write 3 Done Sent Done false
11069 4156 Acked Read 8 Done Sent Done false
11070 4157 Acked Read 9 Done Sent Done false
11071 4158 Acked Read 2 Done Sent Done false
11072 4159 Acked Write 7 Done Sent Done false
11073 4160 Acked Read 10 Done Sent Done false
11074 4161 Acked Read 6 Done Sent Done false
11075 4162 Acked Write 4 Done Sent Done false
11076 4163 Acked Write 2 Done Sent Done false
11077 4164 Acked Read 6 Done Sent Done false
11078 4165 Acked Write 1 Done Sent Done false
11079 4166 Acked Write 5 Done Sent Done false
11080 4167 Acked Read 9 Done Sent Done false
11081 4168 Acked Write 8 Done Sent Done false
11082 4169 Acked Read 3 Done Sent Done false
11083 4170 Acked Write 3 Done Sent Done false
11084 4171 Acked Flush 0 Done Sent Done false
11085 4172 Acked Read 7 Done Sent Done false
11086 4173 Acked Write 9 Done Sent Done false
11087 4174 Acked Write 3 Done Sent Done false
11088 4175 Acked Read 4 Done Sent Done false
11089 4176 Acked Read 2 Done Sent Done false
11090 4177 Acked Read 1 Done Sent Done false
11091 4178 Acked Read 4 Done Sent Done false
11092 4179 Acked Read 9 Done Sent Done false
11093 4180 Acked Write 10 Done Sent Done false
11094 4181 Acked Write 9 Done Sent Done false
11095 4182 Acked Read 7 Done Sent Done false
11096 4183 Acked Write 5 Done Sent Done false
11097 4184 Acked Read 7 Done Sent Done false
11098 4185 Acked Write 3 Done Sent Done false
11099 4186 Acked Write 7 Done Sent Done false
11100 4187 Acked Write 3 Done Sent Done false
11101 4188 Acked Read 7 Done Sent Done false
11102 4189 Acked Flush 0 Done Sent Done false
11103 4190 Acked Read 1 Done Sent Done false
11104 4191 Acked Read 3 Done Sent Done false
11105 4192 Acked Write 9 Done Sent Done false
11106 4193 Acked Read 6 Done Sent Done false
11107 4194 Acked Write 6 Done Sent Done false
11108 4195 Acked Read 8 Done Sent Done false
11109 4196 Acked Write 2 Done Sent Done false
11110 4197 Acked Read 9 Done Sent Done false
11111 4198 Acked Write 8 Done Sent Done false
11112 4199 Acked Write 4 Done Sent Done false
11113 4200 Acked Read 7 Done Sent Done false
11114 4201 Acked Write 6 Done Sent Done false
11115 4202 Acked Read 1 Done Sent Done false
11116 4203 Acked Read 4 Done Sent Done false
11117 4204 Acked Read 10 Done Sent Done false
11118 4205 Acked Write 7 Done Sent Done false
11119 4206 Acked Write 7 Done Sent Done false
11120 4207 Acked Write 2 Done Sent Done false
11121 4208 Acked Read 7 Done Sent Done false
11122 4209 Acked Write 3 Done Sent Done false
11123 4210 Acked Write 2 Done Sent Done false
11124 4211 Acked Write 2 Done Sent Done false
11125 4212 Acked Read 8 Done Sent Done false
11126 4213 Acked Write 7 Done Sent Done false
11127 4214 Acked Read 7 Done Sent Done false
11128 4215 Acked Write 3 Done Sent Done false
11129 4216 Acked Read 2 Done Sent Done false
11130 4217 Acked Write 5 Done Sent Done false
11131 4218 Acked Read 9 Done Sent Done false
11132 4219 Acked Read 10 Done Sent Done false
11133 4220 Acked Read 5 Done Sent Done false
11134 4221 Acked Write 8 Done Sent Done false
11135 4222 Acked Read 8 Done Sent Done false
11136 4223 Acked Write 7 Done Sent Done false
11137 4224 Acked Read 5 Done Sent Done false
11138 4225 Acked Read 2 Done Sent Done false
11139 4226 Acked Read 3 Done Sent Done false
11140 4227 Acked Write 9 Done Sent Done false
11141 4228 Acked Read 6 Done Sent Done false
11142 4229 Acked Write 1 Done Sent Done false
11143 4230 Acked Flush 0 Done Sent Done false
11144 4231 Acked Read 9 Done Sent Done false
11145 4232 Acked Write 10 Done Sent Done false
11146 4233 Acked Read 5 Done Sent Done false
11147 4234 Acked Read 1 Done Sent Done false
11148 4235 Acked Write 3 Done Sent Done false
11149 4236 Acked Read 8 Done Sent Done false
11150 4237 Acked Write 10 Done Sent Done false
11151 4238 Acked Read 4 Done Sent Done false
11152 4239 Acked Flush 0 Done Sent Done false
11153 4240 Acked Write 5 Done Sent Done false
11154 4241 Acked Write 8 Done Sent Done false
11155 4242 Acked Write 3 Done Sent Done false
11156 4243 Acked Write 6 Done Sent Done false
11157 4244 Acked Write 7 Done Sent Done false
11158 4245 Acked Read 7 Done Sent Done false
11159 4246 Acked Write 2 Done Sent Done false
11160 4247 Acked Flush 0 Done Sent Done false
11161 4248 Acked Read 1 Done Sent Done false
11162 4249 Acked Flush 0 Done Sent Done false
11163 4250 Acked Write 3 Done Sent Done false
11164 4251 Acked Flush 0 Done Sent Done false
11165 4252 Acked Write 5 Done Sent Done false
11166 4253 Acked Read 10 Done Sent Done false
11167 4254 Acked Write 6 Done Sent Done false
11168 4255 Acked Write 10 Done Sent Done false
11169 4256 Acked Flush 0 Done Sent Done false
11170 4257 Acked Write 8 Done Sent Done false
11171 4258 Acked Write 1 Done Sent Done false
11172 4259 Acked Write 1 Done Sent Done false
11173 4260 Acked Read 5 Done Sent Done false
11174 4261 Acked Write 1 Done Sent Done false
11175 4262 Acked Read 7 Done Sent Done false
11176 4263 Acked Write 9 Done Sent Done false
11177 4264 Acked Write 6 Done Sent Done false
11178 4265 Acked Read 9 Done Sent Done false
11179 4266 Acked Write 4 Done Sent Done false
11180 4267 Acked Read 1 Done Sent Done false
11181 4268 Acked Read 2 Done Sent Done false
11182 4269 Acked Write 8 Done Sent Done false
11183 4270 Acked Write 4 Done Sent Done false
11184 4271 Acked Read 7 Done Sent Done false
11185 4272 Acked Read 3 Done Sent Done false
11186 4273 Acked Write 9 Done Sent Done false
11187 4274 Acked Write 8 Done Sent Done false
11188 4275 Acked Write 7 Done Sent Done false
11189 4276 Acked Write 8 Done Sent Done false
11190 4277 Acked Flush 0 Done Sent Done false
11191 4278 Acked Read 10 Done Sent Done false
11192 4279 Acked Read 5 Done Sent Done false
11193 4280 Acked Write 2 Done Sent Done false
11194 4281 Acked Read 7 Done Sent Done false
11195 4282 Acked Read 8 Done Sent Done false
11196 4283 Acked Flush 0 Done Sent Done false
11197 4284 Acked Write 8 Done Sent Done false
11198 4285 Acked Flush 0 Done Sent Done false
11199 4286 Acked Read 5 Done Sent Done false
11200 4287 Acked Read 4 Done Sent Done false
11201 4288 Acked Read 5 Done Sent Done false
11202 4289 Acked Write 3 Done Sent Done false
11203 4290 Acked Read 6 Done Sent Done false
11204 4291 Acked Write 4 Done Sent Done false
11205 4292 Acked Read 2 Done Sent Done false
11206 4293 Acked Write 8 Done Sent Done false
11207 4294 Acked Write 10 Done Sent Done false
11208 4295 Acked Read 6 Done Sent Done false
11209 4296 Acked Read 8 Done Sent Done false
11210 4297 Acked Write 6 Done Sent Done false
11211 4298 Acked Read 7 Done Sent Done false
11212 4299 Acked Read 2 Done Sent Done false
11213 4300 Acked Read 3 Done Sent Done false
11214 4301 Acked Read 5 Done Sent Done false
11215 4302 Acked Write 8 Done Sent Done false
11216 4303 Acked Read 7 Done Sent Done false
11217 4304 Acked Write 5 Done Sent Done false
11218 4305 Acked Write 1 Done Sent Done false
11219 4306 Acked Write 1 Done Sent Done false
11220 4307 Acked Write 2 Done Sent Done false
11221 4308 Acked Read 10 Done Sent Done false
11222 4309 Acked Read 5 Done Sent Done false
11223 4310 Acked Read 3 Done Sent Done false
11224 4311 Acked Read 7 Done Sent Done false
11225 4312 Acked Write 2 Done Sent Done false
11226 4313 Acked Read 7 Done Sent Done false
11227 4314 Acked Read 1 Done Sent Done false
11228 4315 Acked Read 10 Done Sent Done false
11229 4316 Acked Write 6 Done Sent Done false
11230 4317 Acked Read 8 Done Sent Done false
11231 4318 Acked Read 7 Done Sent Done false
11232 4319 Acked Write 1 Done Sent Done false
11233 STATES DS:0 DS:1 DS:2 TOTAL
11234 Sent 0 301 0 301
11235 Done 301 0 301 602
11236 Skipped 0 0 0 0
11237 Error 0 0 0 0
11238 Last Flush: 4285 4018 4285
11239 Downstairs last five completed: 4018 4017 4016 4015 4014
11240 Upstairs last five completed: 4319 4318 4317 4316 4315
11241 CLIENT: Up:0 ds:301 act:2
11242 [8820] got start action
11243 [8820] I am going to restart
11244 Starting downstairs at port 8820
11245 Make output file at "/tmp/dsc/downstairs-8820.txt"
11246 [8820][1] reports Starting
11247 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1213)
11248 [8820][1] reports Running
11249 ----------------------------------------------------------------
11250 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
11251 States: Active Offline Active
11252 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
11253 4019 Acked Read 1 Done Sent Done false
11254 4020 Acked Read 3 Done Sent Done false
11255 4021 Acked Read 10 Done Sent Done false
11256 4022 Acked Write 5 Done Sent Done false
11257 4023 Acked Read 9 Done Sent Done false
11258 4024 Acked Read 7 Done Sent Done false
11259 4025 Acked Read 2 Done Sent Done false
11260 4026 Acked Write 4 Done Sent Done false
11261 4027 Acked Write 1 Done Sent Done false
11262 4028 Acked Read 2 Done Sent Done false
11263 4029 Acked Read 8 Done Sent Done false
11264 4030 Acked Write 9 Done Sent Done false
11265 4031 Acked Read 1 Done Sent Done false
11266 4032 Acked Flush 0 Done Sent Done false
11267 4033 Acked Write 7 Done Sent Done false
11268 4034 Acked Write 5 Done Sent Done false
11269 4035 Acked Flush 0 Done Sent Done false
11270 4036 Acked Read 8 Done Sent Done false
11271 4037 Acked Read 10 Done Sent Done false
11272 4038 Acked Read 4 Done Sent Done false
11273 4039 Acked Read 1 Done Sent Done false
11274 4040 Acked Read 10 Done Sent Done false
11275 4041 Acked Write 4 Done Sent Done false
11276 4042 Acked Write 3 Done Sent Done false
11277 4043 Acked Read 3 Done Sent Done false
11278 4044 Acked Read 6 Done Sent Done false
11279 4045 Acked Write 4 Done Sent Done false
11280 4046 Acked Write 2 Done Sent Done false
11281 4047 Acked Write 8 Done Sent Done false
11282 4048 Acked Write 4 Done Sent Done false
11283 4049 Acked Read 1 Done Sent Done false
11284 4050 Acked Read 2 Done Sent Done false
11285 4051 Acked Flush 0 Done Sent Done false
11286 4052 Acked Write 4 Done Sent Done false
11287 4053 Acked Read 1 Done Sent Done false
11288 4054 Acked Write 4 Done Sent Done false
11289 4055 Acked Read 8 Done Sent Done false
11290 4056 Acked Write 3 Done Sent Done false
11291 4057 Acked Read 6 Done Sent Done false
11292 4058 Acked Write 3 Done Sent Done false
11293 4059 Acked Write 10 Done Sent Done false
11294 4060 Acked Read 5 Done Sent Done false
11295 4061 Acked Write 1 Done Sent Done false
11296 4062 Acked Write 1 Done Sent Done false
11297 4063 Acked Write 1 Done Sent Done false
11298 4064 Acked Read 10 Done Sent Done false
11299 4065 Acked Read 5 Done Sent Done false
11300 4066 Acked Write 9 Done Sent Done false
11301 4067 Acked Read 4 Done Sent Done false
11302 4068 Acked Write 4 Done Sent Done false
11303 4069 Acked Write 7 Done Sent Done false
11304 4070 Acked Write 10 Done Sent Done false
11305 4071 Acked Read 2 Done Sent Done false
11306 4072 Acked Read 9 Done Sent Done false
11307 4073 Acked Read 1 Done Sent Done false
11308 4074 Acked Write 4 Done Sent Done false
11309 4075 Acked Read 9 Done Sent Done false
11310 4076 Acked Write 4 Done Sent Done false
11311 4077 Acked Read 3 Done Sent Done false
11312 4078 Acked Read 8 Done Sent Done false
11313 4079 Acked Flush 0 Done Sent Done false
11314 4080 Acked Write 6 Done Sent Done false
11315 4081 Acked Read 3 Done Sent Done false
11316 4082 Acked Write 8 Done Sent Done false
11317 4083 Acked Write 10 Done Sent Done false
11318 4084 Acked Read 2 Done Sent Done false
11319 4085 Acked Write 6 Done Sent Done false
11320 4086 Acked Write 5 Done Sent Done false
11321 4087 Acked Read 2 Done Sent Done false
11322 4088 Acked Write 1 Done Sent Done false
11323 4089 Acked Read 8 Done Sent Done false
11324 4090 Acked Read 8 Done Sent Done false
11325 4091 Acked Write 7 Done Sent Done false
11326 4092 Acked Read 6 Done Sent Done false
11327 4093 Acked Read 10 Done Sent Done false
11328 4094 Acked Write 2 Done Sent Done false
11329 4095 Acked Write 10 Done Sent Done false
11330 4096 Acked Write 5 Done Sent Done false
11331 4097 Acked Write 4 Done Sent Done false
11332 4098 Acked Write 9 Done Sent Done false
11333 4099 Acked Write 6 Done Sent Done false
11334 4100 Acked Write 10 Done Sent Done false
11335 4101 Acked Read 3 Done Sent Done false
11336 4102 Acked Write 5 Done Sent Done false
11337 4103 Acked Read 3 Done Sent Done false
11338 4104 Acked Write 3 Done Sent Done false
11339 4105 Acked Write 2 Done Sent Done false
11340 4106 Acked Write 6 Done Sent Done false
11341 4107 Acked Read 8 Done Sent Done false
11342 4108 Acked Write 5 Done Sent Done false
11343 4109 Acked Read 4 Done Sent Done false
11344 4110 Acked Read 6 Done Sent Done false
11345 4111 Acked Read 6 Done Sent Done false
11346 4112 Acked Write 5 Done Sent Done false
11347 4113 Acked Read 2 Done Sent Done false
11348 4114 Acked Write 1 Done Sent Done false
11349 4115 Acked Read 3 Done Sent Done false
11350 4116 Acked Read 8 Done Sent Done false
11351 4117 Acked Write 7 Done Sent Done false
11352 4118 Acked Read 2 Done Sent Done false
11353 4119 Acked Read 5 Done Sent Done false
11354 4120 Acked Flush 0 Done Sent Done false
11355 4121 Acked Read 5 Done Sent Done false
11356 4122 Acked Read 7 Done Sent Done false
11357 4123 Acked Write 8 Done Sent Done false
11358 4124 Acked Read 7 Done Sent Done false
11359 4125 Acked Write 10 Done Sent Done false
11360 4126 Acked Read 8 Done Sent Done false
11361 4127 Acked Read 10 Done Sent Done false
11362 4128 Acked Write 5 Done Sent Done false
11363 4129 Acked Write 3 Done Sent Done false
11364 4130 Acked Read 2 Done Sent Done false
11365 4131 Acked Write 4 Done Sent Done false
11366 4132 Acked Write 1 Done Sent Done false
11367 4133 Acked Read 4 Done Sent Done false
11368 4134 Acked Write 2 Done Sent Done false
11369 4135 Acked Read 3 Done Sent Done false
11370 4136 Acked Read 2 Done Sent Done false
11371 4137 Acked Read 6 Done Sent Done false
11372 4138 Acked Write 9 Done Sent Done false
11373 4139 Acked Read 6 Done Sent Done false
11374 4140 Acked Read 10 Done Sent Done false
11375 4141 Acked Read 1 Done Sent Done false
11376 4142 Acked Write 2 Done Sent Done false
11377 4143 Acked Write 1 Done Sent Done false
11378 4144 Acked Read 2 Done Sent Done false
11379 4145 Acked Write 1 Done Sent Done false
11380 4146 Acked Read 6 Done Sent Done false
11381 4147 Acked Read 6 Done Sent Done false
11382 4148 Acked Read 8 Done Sent Done false
11383 4149 Acked Write 3 Done Sent Done false
11384 4150 Acked Read 10 Done Sent Done false
11385 4151 Acked Read 3 Done Sent Done false
11386 4152 Acked Write 6 Done Sent Done false
11387 4153 Acked Read 8 Done Sent Done false
11388 4154 Acked Write 5 Done Sent Done false
11389 4155 Acked Write 3 Done Sent Done false
11390 4156 Acked Read 8 Done Sent Done false
11391 4157 Acked Read 9 Done Sent Done false
11392 4158 Acked Read 2 Done Sent Done false
11393 4159 Acked Write 7 Done Sent Done false
11394 4160 Acked Read 10 Done Sent Done false
11395 4161 Acked Read 6 Done Sent Done false
11396 4162 Acked Write 4 Done Sent Done false
11397 4163 Acked Write 2 Done Sent Done false
11398 4164 Acked Read 6 Done Sent Done false
11399 4165 Acked Write 1 Done Sent Done false
11400 4166 Acked Write 5 Done Sent Done false
11401 4167 Acked Read 9 Done Sent Done false
11402 4168 Acked Write 8 Done Sent Done false
11403 4169 Acked Read 3 Done Sent Done false
11404 4170 Acked Write 3 Done Sent Done false
11405 4171 Acked Flush 0 Done Sent Done false
11406 4172 Acked Read 7 Done Sent Done false
11407 4173 Acked Write 9 Done Sent Done false
11408 4174 Acked Write 3 Done Sent Done false
11409 4175 Acked Read 4 Done Sent Done false
11410 4176 Acked Read 2 Done Sent Done false
11411 4177 Acked Read 1 Done Sent Done false
11412 4178 Acked Read 4 Done Sent Done false
11413 4179 Acked Read 9 Done Sent Done false
11414 4180 Acked Write 10 Done Sent Done false
11415 4181 Acked Write 9 Done Sent Done false
11416 4182 Acked Read 7 Done Sent Done false
11417 4183 Acked Write 5 Done Sent Done false
11418 4184 Acked Read 7 Done Sent Done false
11419 4185 Acked Write 3 Done Sent Done false
11420 4186 Acked Write 7 Done Sent Done false
11421 4187 Acked Write 3 Done Sent Done false
11422 4188 Acked Read 7 Done Sent Done false
11423 4189 Acked Flush 0 Done Sent Done false
11424 4190 Acked Read 1 Done Sent Done false
11425 4191 Acked Read 3 Done Sent Done false
11426 4192 Acked Write 9 Done Sent Done false
11427 4193 Acked Read 6 Done Sent Done false
11428 4194 Acked Write 6 Done Sent Done false
11429 4195 Acked Read 8 Done Sent Done false
11430 4196 Acked Write 2 Done Sent Done false
11431 4197 Acked Read 9 Done Sent Done false
11432 4198 Acked Write 8 Done Sent Done false
11433 4199 Acked Write 4 Done Sent Done false
11434 4200 Acked Read 7 Done Sent Done false
11435 4201 Acked Write 6 Done Sent Done false
11436 4202 Acked Read 1 Done Sent Done false
11437 4203 Acked Read 4 Done Sent Done false
11438 4204 Acked Read 10 Done Sent Done false
11439 4205 Acked Write 7 Done Sent Done false
11440 4206 Acked Write 7 Done Sent Done false
11441 4207 Acked Write 2 Done Sent Done false
11442 4208 Acked Read 7 Done Sent Done false
11443 4209 Acked Write 3 Done Sent Done false
11444 4210 Acked Write 2 Done Sent Done false
11445 4211 Acked Write 2 Done Sent Done false
11446 4212 Acked Read 8 Done Sent Done false
11447 4213 Acked Write 7 Done Sent Done false
11448 4214 Acked Read 7 Done Sent Done false
11449 4215 Acked Write 3 Done Sent Done false
11450 4216 Acked Read 2 Done Sent Done false
11451 4217 Acked Write 5 Done Sent Done false
11452 4218 Acked Read 9 Done Sent Done false
11453 4219 Acked Read 10 Done Sent Done false
11454 4220 Acked Read 5 Done Sent Done false
11455 4221 Acked Write 8 Done Sent Done false
11456 4222 Acked Read 8 Done Sent Done false
11457 4223 Acked Write 7 Done Sent Done false
11458 4224 Acked Read 5 Done Sent Done false
11459 4225 Acked Read 2 Done Sent Done false
11460 4226 Acked Read 3 Done Sent Done false
11461 4227 Acked Write 9 Done Sent Done false
11462 4228 Acked Read 6 Done Sent Done false
11463 4229 Acked Write 1 Done Sent Done false
11464 4230 Acked Flush 0 Done Sent Done false
11465 4231 Acked Read 9 Done Sent Done false
11466 4232 Acked Write 10 Done Sent Done false
11467 4233 Acked Read 5 Done Sent Done false
11468 4234 Acked Read 1 Done Sent Done false
11469 4235 Acked Write 3 Done Sent Done false
11470 4236 Acked Read 8 Done Sent Done false
11471 4237 Acked Write 10 Done Sent Done false
11472 4238 Acked Read 4 Done Sent Done false
11473 4239 Acked Flush 0 Done Sent Done false
11474 4240 Acked Write 5 Done Sent Done false
11475 4241 Acked Write 8 Done Sent Done false
11476 4242 Acked Write 3 Done Sent Done false
11477 4243 Acked Write 6 Done Sent Done false
11478 4244 Acked Write 7 Done Sent Done false
11479 4245 Acked Read 7 Done Sent Done false
11480 4246 Acked Write 2 Done Sent Done false
11481 4247 Acked Flush 0 Done Sent Done false
11482 4248 Acked Read 1 Done Sent Done false
11483 4249 Acked Flush 0 Done Sent Done false
11484 4250 Acked Write 3 Done Sent Done false
11485 4251 Acked Flush 0 Done Sent Done false
11486 4252 Acked Write 5 Done Sent Done false
11487 4253 Acked Read 10 Done Sent Done false
11488 4254 Acked Write 6 Done Sent Done false
11489 4255 Acked Write 10 Done Sent Done false
11490 4256 Acked Flush 0 Done Sent Done false
11491 4257 Acked Write 8 Done Sent Done false
11492 4258 Acked Write 1 Done Sent Done false
11493 4259 Acked Write 1 Done Sent Done false
11494 4260 Acked Read 5 Done Sent Done false
11495 4261 Acked Write 1 Done Sent Done false
11496 4262 Acked Read 7 Done Sent Done false
11497 4263 Acked Write 9 Done Sent Done false
11498 4264 Acked Write 6 Done Sent Done false
11499 4265 Acked Read 9 Done Sent Done false
11500 4266 Acked Write 4 Done Sent Done false
11501 4267 Acked Read 1 Done Sent Done false
11502 4268 Acked Read 2 Done Sent Done false
11503 4269 Acked Write 8 Done Sent Done false
11504 4270 Acked Write 4 Done Sent Done false
11505 4271 Acked Read 7 Done Sent Done false
11506 4272 Acked Read 3 Done Sent Done false
11507 4273 Acked Write 9 Done Sent Done false
11508 4274 Acked Write 8 Done Sent Done false
11509 4275 Acked Write 7 Done Sent Done false
11510 4276 Acked Write 8 Done Sent Done false
11511 4277 Acked Flush 0 Done Sent Done false
11512 4278 Acked Read 10 Done Sent Done false
11513 4279 Acked Read 5 Done Sent Done false
11514 4280 Acked Write 2 Done Sent Done false
11515 4281 Acked Read 7 Done Sent Done false
11516 4282 Acked Read 8 Done Sent Done false
11517 4283 Acked Flush 0 Done Sent Done false
11518 4284 Acked Write 8 Done Sent Done false
11519 4285 Acked Flush 0 Done Sent Done false
11520 4286 Acked Read 5 Done Sent Done false
11521 4287 Acked Read 4 Done Sent Done false
11522 4288 Acked Read 5 Done Sent Done false
11523 4289 Acked Write 3 Done Sent Done false
11524 4290 Acked Read 6 Done Sent Done false
11525 4291 Acked Write 4 Done Sent Done false
11526 4292 Acked Read 2 Done Sent Done false
11527 4293 Acked Write 8 Done Sent Done false
11528 4294 Acked Write 10 Done Sent Done false
11529 4295 Acked Read 6 Done Sent Done false
11530 4296 Acked Read 8 Done Sent Done false
11531 4297 Acked Write 6 Done Sent Done false
11532 4298 Acked Read 7 Done Sent Done false
11533 4299 Acked Read 2 Done Sent Done false
11534 4300 Acked Read 3 Done Sent Done false
11535 4301 Acked Read 5 Done Sent Done false
11536 4302 Acked Write 8 Done Sent Done false
11537 4303 Acked Read 7 Done Sent Done false
11538 4304 Acked Write 5 Done Sent Done false
11539 4305 Acked Write 1 Done Sent Done false
11540 4306 Acked Write 1 Done Sent Done false
11541 4307 Acked Write 2 Done Sent Done false
11542 4308 Acked Read 10 Done Sent Done false
11543 4309 Acked Read 5 Done Sent Done false
11544 4310 Acked Read 3 Done Sent Done false
11545 4311 Acked Read 7 Done Sent Done false
11546 4312 Acked Write 2 Done Sent Done false
11547 4313 Acked Read 7 Done Sent Done false
11548 4314 Acked Read 1 Done Sent Done false
11549 4315 Acked Read 10 Done Sent Done false
11550 4316 Acked Write 6 Done Sent Done false
11551 4317 Acked Read 8 Done Sent Done false
11552 4318 Acked Read 7 Done Sent Done false
11553 4319 Acked Write 1 Done Sent Done false
11554 4320 Acked Flush 0 Done Sent Done false
11555 STATES DS:0 DS:1 DS:2 TOTAL
11556 Sent 0 302 0 302
11557 Done 302 0 302 604
11558 Skipped 0 0 0 0
11559 Error 0 0 0 0
11560 Last Flush: 4320 4018 4320
11561 Downstairs last five completed: 4018 4017 4016 4015 4014
11562 Upstairs last five completed: 4320 4319 4318 4317 4316
11563 CLIENT: Up:0 ds:302 act:2
115642024-11-28T04:52:43.441ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
115652024-11-28T04:52:43.441ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:42808) = io task client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
115662024-11-28T04:52:43.442ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
115672024-11-28T04:52:43.442ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
115682024-11-28T04:52:43.442ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
115692024-11-28T04:52:43.442ZINFOcrucible: send last flush ID to this DS: 4018 = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
115702024-11-28T04:52:43.443ZINFOcrucible: Replied this last flush ID: 4018 = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
115712024-11-28T04:52:43.443ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
115722024-11-28T04:52:43.443ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
115732024-11-28T04:52:43.443ZINFOcrucible: [1] client re-new 302 jobs since flush 4018 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
115742024-11-28T04:52:43.443ZINFOcrucible: [1] Replayed 302 jobs since flush: 4018 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
11575 ----------------------------------------------------------------
11576 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
11577 Downstairs last five completed: 4320 4319 4318 4317 4316
11578 Upstairs last five completed: 4320 4319 4318 4317 4316
11579 CLIENT: Up:0 ds:0 act:3
11580 Replay: All jobs finished
11581 Main task has work to do, go find it
11582 got dsc Stop(2)
11583 stop 2
11584 Nov 28 04:52:45.678 INFO request completed[8830] Got stop action so:false kr:false
11585 , latency_us: 360, response_code: 204, uri: /stop/cid/2, method: POST, req_id: e6a23776-4371-4946-9d24-84ef457e0f83, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
115862024-11-28T04:52:45.679ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
115872024-11-28T04:52:45.679ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
115882024-11-28T04:52:45.679ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
115892024-11-28T04:52:45.679ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
115902024-11-28T04:52:45.679ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
11591 Nov 28 04:52:45.679 INFO request completed, latency_us: 130, response_code: 200, uri: /state/cid/2, method: GET{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2024-11-28T04:52:45.679700516Z","hostname":"w-01JDRHKEEFJ1228TFPSBEGXWHY","pid":1202,"":"io task","client":"2","":"downstairs","session_id":"4a2415b1-6594-436f-82e5-b8cf9a74eeaf"}
11592 , req_id: 03244ea7-0612-4497-80e5-821ad7a46f4d, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
11593 [8830] Exited with: ExitStatus(unix_wait_status(9))
11594 [8830][2] reports Exit
11595 Nov 28 04:52:49.681 INFO request completed, latency_us: 214, response_code: 200, uri: /state/cid/2, method: GET, req_id: 171db7b5-5e18-42ba-8adf-a729b5a7b8ba, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
11596 001/300 Write block 153 len 36864 data: 4 4 4 5 5 4 4 4 4
11597 002/300 Write block 3247 len 40960 data: 2 2 3 3 3 3 4 4 5 5
11598 003/300 Write block 2751 len 28672 data: 6 5 5 4 3 3 4
11599 004/300 Read block 2939 len 40960
11600 005/300 Read block 1561 len 16384
11601 006/300 Write block 3179 len 16384 data: 3 3 2 2
11602 007/300 Write block 4465 len 28672 data: 4 4 4 3 3 4 4
11603 008/300 Write block 4869 len 12288 data: 3 3 4
11604 009/300 Read block 2401 len 8192
11605 010/300 Write block 2491 len 28672 data: 3 3 3 2 2 2 2
11606 011/300 Read block 985 len 40960
11607 012/300 Read block 2258 len 24576
11608 013/300 Read block 4825 len 32768
11609 014/300 Read block 319 len 4096
11610 015/300 Read block 4011 len 12288
11611 016/300 Write block 3190 len 24576 data: 4 4 4 4 3 3
11612 017/300 Flush
11613 018/300 Flush
11614 019/300 Read block 4175 len 16384
11615 020/300 Write block 2182 len 4096 data: 2
11616 021/300 Read block 932 len 16384
11617 022/300 Read block 1462 len 16384
11618 023/300 Write block 4002 len 20480 data: 2 2 2 2 2
11619 024/300 Write block 4691 len 36864 data: 2 2 2 2 2 2 2 2 2
11620 025/300 Write block 848 len 20480 data: 4 3 4 5 5
11621 026/300 Write block 773 len 32768 data: 2 2 2 2 2 2 2 2
11622 027/300 Write block 2330 len 20480 data: 4 5 5 4 4
11623 028/300 Read block 4725 len 20480
11624 029/300 Flush
11625 030/300 Write block 524 len 8192 data: 3 2
11626 031/300 Read block 1076 len 32768
11627 032/300 Write block 734 len 20480 data: 4 3 4 3 3
11628 033/300 Write block 4531 len 8192 data: 4 4
11629 034/300 Read block 3628 len 12288
11630 035/300 Write block 3387 len 32768 data: 2 2 2 2 2 2 2 2
11631 036/300 Write block 3933 len 8192 data: 4 3
11632 037/300 Write block 3327 len 24576 data: 4 5 6 5 5 4
11633 038/300 Read block 4222 len 12288
11634 039/300 Read block 1172 len 12288
11635 040/300 Read block 1657 len 24576
11636 041/300 Read block 4660 len 24576
11637 042/300 Read block 1394 len 36864
11638 043/300 Write block 1706 len 28672 data: 9 10 11 12 10 9 9
11639 044/300 Write block 3447 len 8192 data: 4 4
11640 045/300 Read block 2184 len 32768
11641 046/300 Read block 2049 len 28672
11642 047/300 Write block 2567 len 20480 data: 4 4 4 5 6
11643 048/300 Write block 310 len 16384 data: 5 4 4 4
11644 049/300 Write block 2805 len 4096 data: 3
11645 050/300 Read block 3639 len 28672
11646 051/300 Read block 4824 len 40960
11647 052/300 Write block 2260 len 16384 data: 3 4 4 3
11648 053/300 Read block 476 len 40960
11649 054/300 Write block 2919 len 12288 data: 4 5 5
11650 055/300 Read block 4890 len 40960
11651 056/300 Write block 1703 len 36864 data: 6 6 8 10 11 12 13 11 10
11652 057/300 Read block 1291 len 24576
11653 058/300 Write block 1575 len 20480 data: 3 3 3 3 3
11654 059/300 Write block 1453 len 4096 data: 3
11655 060/300 Write block 2712 len 12288 data: 4 5 6
11656 061/300 Write block 2445 len 8192 data: 3 2
11657 062/300 Read block 2453 len 36864
11658 063/300 Flush
11659 064/300 Read block 2858 len 36864
11660 065/300 Read block 2642 len 32768
11661 066/300 Write block 981 len 36864 data: 3 3 3 5 5 4 4 4 4
11662 067/300 Read block 1218 len 20480
11663 068/300 Write block 4245 len 24576 data: 5 5 6 5 4 4
11664 069/300 Write block 3018 len 32768 data: 3 3 3 3 3 4 5 4
11665 070/300 Write block 3589 len 28672 data: 2 2 2 2 2 2 2
11666 071/300 Write block 352 len 28672 data: 3 3 4 4 4 3 2
11667 072/300 Write block 4629 len 4096 data: 4
11668 073/300 Read block 2177 len 16384
11669 074/300 Read block 4215 len 24576
11670 075/300 Write block 4835 len 28672 data: 4 3 3 2 2 2 2
11671 076/300 Write block 2713 len 36864 data: 6 7 6 6 6 5 5 4 4
11672 077/300 Read block 4905 len 4096
11673 078/300 Write block 1920 len 8192 data: 3 4
11674 079/300 Read block 1281 len 12288
11675 080/300 Write block 3861 len 24576 data: 4 4 3 3 3 4
11676 081/300 Read block 3572 len 28672
11677 082/300 Read block 2356 len 40960
11678 083/300 Write block 1259 len 36864 data: 2 3 3 3 3 3 4 4 4
11679 084/300 Read block 989 len 8192
11680 085/300 Write block 4535 len 4096 data: 4
11681 086/300 Read block 1465 len 36864
11682 087/300 Flush
11683 088/300 Write block 4300 len 32768 data: 4 4 4 4 4 4 3 2
11684 089/300 Read block 4028 len 16384
11685 090/300 Read block 1727 len 4096
11686 091/300 Read block 1289 len 12288
11687 092/300 Read block 2309 len 16384
11688 093/300 Read block 865 len 36864
11689 094/300 Flush
11690 095/300 Write block 1944 len 40960 data: 3 3 3 2 2 2 2 2 2 2
11691 096/300 Write block 3594 len 32768 data: 3 3 2 2 2 2 2 2
11692 097/300 Write block 2348 len 24576 data: 2 3 3 3 3 3
11693 098/300 Read block 4732 len 28672
11694 099/300 Read block 1410 len 20480
11695 100/300 Write block 2616 len 12288 data: 2 3 4
11696 101/300 Write block 1953 len 40960 data: 3 2 2 2 3 4 4 4 4 4
11697 102/300 Read block 3039 len 4096
11698 103/300 Write block 2166 len 36864 data: 2 2 4 4 4 3 4 4 4
11699 104/300 Write block 732 len 4096 data: 3
11700 105/300 Flush
11701 106/300 Write block 2092 len 8192 data: 3 4
11702 107/300 Write block 4246 len 28672 data: 6 7 6 5 5 3 3
11703 108/300 Read block 1282 len 12288
11704 109/300 Read block 2258 len 16384
11705 110/300 Read block 1597 len 12288
11706 111/300 Read block 1663 len 40960
11707 112/300 Read block 1235 len 16384
11708 113/300 Flush
11709 114/300 Write block 2690 len 4096 data: 3
11710 115/300 Read block 4629 len 40960
11711 116/300 Read block 2992 len 32768
11712 117/300 Write block 4268 len 32768 data: 5 5 5 3 3 3 2 2
11713 118/300 Write block 4078 len 28672 data: 6 7 6 6 5 4 5
11714 119/300 Write block 2118 len 36864 data: 4 3 2 2 2 3 3 3 4
11715 120/300 Write block 4815 len 28672 data: 2 2 2 2 2 2 2
11716 121/300 Read block 2715 len 16384
11717 122/300 Write block 3741 len 16384 data: 3 3 3 3
11718 123/300 Write block 4350 len 28672 data: 3 2 3 3 3 3 4
11719 124/300 Read block 860 len 12288
11720 125/300 Write block 3937 len 32768 data: 3 2 2 2 3 3 3 3
11721 126/300 Read block 2613 len 8192
11722 127/300 Read block 2114 len 4096
11723 128/300 Read block 2786 len 8192
11724 129/300 Read block 2599 len 8192
11725 130/300 Read block 3347 len 24576
11726 131/300 Write block 201 len 16384 data: 4 4 3 3
11727 132/300 Read block 4700 len 4096
11728 133/300 Flush
11729 134/300 Write block 3617 len 20480 data: 2 2 2 3 3
11730 135/300 Write block 795 len 36864 data: 3 3 2 2 2 2 2 2 3
11731 136/300 Write block 2646 len 20480 data: 2 3 3 4 4
11732 137/300 Read block 1778 len 8192
11733 138/300 Write block 4806 len 20480 data: 5 3 3 3 2
11734 139/300 Write block 4223 len 8192 data: 3 3
11735 140/300 Write block 629 len 20480 data: 4 4 5 6 6
11736 141/300 Write block 1775 len 12288 data: 4 5 4
11737 142/300 Read block 4912 len 20480
11738 143/300 Write block 4826 len 24576 data: 3 2 2 2 3 2
11739 144/300 Read block 3415 len 32768
11740 145/300 Read block 2770 len 4096
11741 146/300 Read block 2037 len 20480
11742 147/300 Read block 512 len 12288
11743 148/300 Read block 250 len 40960
11744 149/300 Read block 1789 len 4096
11745 150/300 Write block 1545 len 32768 data: 4 3 3 3 5 3 3 3
11746 151/300 Write block 2948 len 8192 data: 5 5
11747 152/300 Write block 3011 len 12288 data: 4 4 4
11748 153/300 Write block 1145 len 24576 data: 2 2 4 3 4 5
11749 154/300 Read block 1941 len 36864
11750 155/300 Write block 2137 len 20480 data: 2 2 2 2 3
11751 156/300 Read block 826 len 28672
11752 157/300 Write block 435 len 36864 data: 3 4 3 4 5 5 4 4 4
11753 158/300 Write block 1824 len 32768 data: 3 3 2 3 3 3 4 4
11754 159/300 Read block 3641 len 24576
11755 160/300 Flush
11756 161/300 Write block 2728 len 4096 data: 2
11757 162/300 Write block 4797 len 16384 data: 4 4 4 3
11758 163/300 Write block 625 len 12288 data: 5 6 4
11759 164/300 Read block 1396 len 36864
11760 165/300 Flush
11761 166/300 Read block 2828 len 8192
11762 167/300 Read block 3705 len 32768
11763 168/300 Read block 3109 len 16384
11764 169/300 Write block 4450 len 8192 data: 6 3
11765 170/300 Flush
11766 171/300 Write block 3922 len 8192 data: 3 4
11767 172/300 Read block 3378 len 32768
11768 173/300 Write block 2320 len 16384 data: 4 4 3 3
11769 174/300 Write block 4622 len 4096 data: 2
11770 175/300 Flush
11771 176/300 Read block 3567 len 28672
11772 177/300 Write block 890 len 12288 data: 2 3 3
11773 178/300 Flush
11774 179/300 Write block 1583 len 4096 data: 3
11775 180/300 Read block 4181 len 24576
11776 181/300 Read block 474 len 20480
11777 182/300 Read block 2682 len 36864
11778 183/300 Read block 1166 len 4096
11779 184/300 Read block 2705 len 32768
11780 185/300 Read block 3536 len 36864
11781 186/300 Read block 460 len 12288
11782 187/300 Write block 671 len 24576 data: 5 6 3 3 3 3
11783 188/300 Write block 2809 len 16384 data: 2 2 2 2
11784 189/300 Read block 1240 len 4096
11785 190/300 Write block 318 len 40960 data: 4 5 5 6 7 7 6 6 4 4
11786 191/300 Flush
11787 192/300 Read block 2225 len 28672
11788 193/300 Write block 4870 len 24576 data: 4 5 4 3 2 2
11789 194/300 Read block 4300 len 24576
11790 195/300 Write block 3648 len 16384 data: 3 5 6 5
11791 196/300 Read block 853 len 4096
11792 197/300 Write block 3036 len 20480 data: 5 5 5 6 4
11793 198/300 Read block 119 len 40960
11794 199/300 Write block 4732 len 36864 data: 2 2 2 2 2 3 3 3 3
11795 200/300 Read block 3206 len 24576
11796 201/300 Write block 375 len 20480 data: 6 5 5 6 7
11797 202/300 Flush
11798 203/300 Read block 154 len 20480
11799 204/300 Read block 48 len 20480
11800 205/300 Write block 429 len 40960 data: 2 2 2 3 3 3 4 5 4 5
11801 206/300 Write block 4736 len 20480 data: 3 4 4 4 4
11802 207/300 Read block 4974 len 32768
11803 208/300 Write block 894 len 20480 data: 2 2 2 2 3
11804 209/300 Read block 3610 len 12288
11805 210/300 Write block 4141 len 16384 data: 5 4 3 3
11806 211/300 Flush
11807 212/300 Write block 2008 len 20480 data: 2 2 2 2 2
11808 213/300 Flush
11809 214/300 Read block 3381 len 4096
11810 215/300 Read block 3005 len 12288
11811 216/300 Write block 2381 len 4096 data: 5
11812 217/300 Write block 656 len 32768 data: 5 6 6 9 8 8 9 8
11813 218/300 Read block 2723 len 20480
11814 219/300 Write block 191 len 20480 data: 6 6 6 5 5
11815 220/300 Write block 3255 len 40960 data: 6 6 5 5 3 4 4 3 4 4
11816 221/300 Write block 1297 len 32768 data: 2 2 2 2 2 2 3 3
11817 222/300 Read block 1208 len 16384
11818 223/300 Write block 4115 len 4096 data: 3
11819 224/300 Read block 3913 len 12288
11820 225/300 Write block 109 len 24576 data: 5 3 2 2 2 2
11821 226/300 Read block 430 len 16384
11822 227/300 Write block 466 len 28672 data: 3 2 3 4 4 4 5
11823 228/300 Write block 4069 len 32768 data: 3 3 3 3 2 2 3 3
11824 229/300 Read block 3178 len 8192
11825 230/300 Read block 3061 len 28672
11826 231/300 Write block 1218 len 32768 data: 3 4 5 5 5 5 5 4
11827 232/300 Write block 1222 len 28672 data: 6 6 6 5 4 4 3
11828 233/300 Write block 2195 len 20480 data: 6 5 4 3 3
11829 234/300 Read block 2409 len 32768
11830 235/300 Write block 2855 len 40960 data: 3 3 2 2 2 2 2 2 2 2
11831 236/300 Write block 4159 len 40960 data: 5 5 5 4 4 4 4 4 4 3
11832 237/300 Flush
11833 238/300 Write block 335 len 16384 data: 6 5 5 4
11834 239/300 Read block 296 len 32768
11835 240/300 Read block 697 len 28672
11836 241/300 Write block 2336 len 20480 data: 3 4 4 3 3
11837 242/300 Read block 196 len 12288
11838 243/300 Read block 693 len 16384
11839 244/300 Read block 3649 len 8192
11840 245/300 Write block 2272 len 28672 data: 4 3 3 3 2 2 2
11841 246/300 Read block 2993 len 40960
11842 247/300 Read block 3998 len 20480
11843 248/300 Read block 1587 len 32768
11844 249/300 Read block 888 len 24576
11845 250/300 Read block 3795 len 12288
11846 251/300 Read block 825 len 20480
11847 252/300 Read block 3626 len 20480
11848 253/300 Write block 630 len 12288 data: 5 6 7
11849 254/300 Read block 2719 len 8192
11850 255/300 Write block 17 len 40960 data: 7 6 4 4 3 3 3 3 4 3
11851 256/300 Read block 718 len 32768
11852 257/300 Read block 4593 len 36864
11853 258/300 Write block 4604 len 24576 data: 3 3 2 2 2 2
11854 259/300 Flush
11855 260/300 Write block 3303 len 8192 data: 4 5
11856 261/300 Write block 4127 len 40960 data: 4 4 4 4 4 4 4 4 3 4
11857 262/300 Read block 1086 len 16384
11858 263/300 Read block 4233 len 20480
11859 264/300 Read block 325 len 4096
11860 265/300 Write block 1326 len 32768 data: 3 2 2 3 3 3 2 3
11861 266/300 Read block 2604 len 28672
11862 267/300 Read block 3930 len 8192
11863 268/300 Read block 2350 len 32768
11864 269/300 Read block 255 len 36864
11865 270/300 Read block 3157 len 20480
11866 271/300 Read block 3667 len 40960
11867 272/300 Read block 1902 len 40960
11868 273/300 Read block 3489 len 20480
11869 274/300 Read block 466 len 20480
11870 275/300 Flush
11871 276/300 Write block 2134 len 32768 data: 2 2 2 3 3 3 3 4
11872 277/300 Write block 1672 len 32768 data: 3 2 2 2 2 2 2 2
11873 278/300 Flush
11874 279/300 Write block 817 len 16384 data: 5 5 4 3
11875 280/300 Write block 4887 len 12288 data: 4 4 4
11876 281/300 Write block 2647 len 28672 data: 4 4 5 5 3 3 4
11877 282/300 Write block 4025 len 36864 data: 2 2 2 2 2 2 3 3 5
11878 283/300 Read block 4572 len 4096
11879 284/300 Write block 3465 len 12288 data: 4 3 3
11880 285/300 Read block 2668 len 40960
11881 286/300 Read block 2924 len 20480
11882 287/300 Write block 3377 len 12288 data: 3 3 3
11883 288/300 Read block 940 len 12288
11884 289/300 Write block 104 len 20480 data: 3 3 3 4 4
11885 290/300 Write block 615 len 36864 data: 4 5 5 5 4 3 4 4 4
11886 291/300 Read block 101 len 20480
11887 292/300 Read block 1277 len 32768
11888 293/300 Write block 2829 len 32768 data: 3 3 3 3 3 3 2 2
11889 294/300 Read block 601 len 24576
11890 295/300 Flush
11891 296/300 Write block 1626 len 12288 data: 3 3 3
11892 297/300 Write block 2419 len 28672 data: 4 4 4 4 3 3 3
11893 298/300 Write block 4416 len 28672 data: 3 3 3 3 3 2 2
11894 299/300 Read block 2209 len 40960
11895 300/300 Read block 3544 len 28672
11896 301/300 Flush
11897 Main task has work to do, go find it
11898 got dsc Start(2)
11899 start 2
11900 Nov 28 04:52:49.914 INFO request completed, latency_us: 221, response_code: 204, uri: /start/cid/2, method: POST, req_id: be170604-8aeb-4ac7-95f3-35e245d8ef94, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
11901 Replay: started 2, returned:Ok(())
11902 ----------------------------------------------------------------
11903 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
11904 States: Active Active Offline
11905 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
11906 4321 Acked Write 9 Done Done Sent false
11907 4322 Acked Write 10 Done Done Sent false
11908 4323 Acked Write 7 Done Done Sent false
11909 4324 Acked Read 10 Done Done Sent false
11910 4325 Acked Read 4 Done Done Sent false
11911 4326 Acked Write 4 Done Done Sent false
11912 4327 Acked Write 7 Done Done Sent false
11913 4328 Acked Write 3 Done Done Sent false
11914 4329 Acked Read 2 Done Done Sent false
11915 4330 Acked Write 7 Done Done Sent false
11916 4331 Acked Read 10 Done Done Sent false
11917 4332 Acked Read 6 Done Done Sent false
11918 4333 Acked Read 8 Done Done Sent false
11919 4334 Acked Read 1 Done Done Sent false
11920 4335 Acked Read 3 Done Done Sent false
11921 4336 Acked Write 6 Done Done Sent false
11922 4337 Acked Flush 0 Done Done Sent false
11923 4338 Acked Flush 0 Done Done Sent false
11924 4339 Acked Read 4 Done Done Sent false
11925 4340 Acked Write 1 Done Done Sent false
11926 4341 Acked Read 4 Done Done Sent false
11927 4342 Acked Read 4 Done Done Sent false
11928 4343 Acked Write 5 Done Done Sent false
11929 4344 Acked Write 9 Done Done Sent false
11930 4345 Acked Write 5 Done Done Sent false
11931 4346 Acked Write 8 Done Done Sent false
11932 4347 Acked Write 5 Done Done Sent false
11933 4348 Acked Read 5 Done Done Sent false
11934 4349 Acked Flush 0 Done Done Sent false
11935 4350 Acked Write 2 Done Done Sent false
11936 4351 Acked Read 8 Done Done Sent false
11937 4352 Acked Write 5 Done Done Sent false
11938 4353 Acked Write 2 Done Done Sent false
11939 4354 Acked Read 3 Done Done Sent false
11940 4355 Acked Write 8 Done Done Sent false
11941 4356 Acked Write 2 Done Done Sent false
11942 4357 Acked Write 6 Done Done Sent false
11943 4358 Acked Read 3 Done Done Sent false
11944 4359 Acked Read 3 Done Done Sent false
11945 4360 Acked Read 6 Done Done Sent false
11946 4361 Acked Read 6 Done Done Sent false
11947 4362 Acked Read 9 Done Done Sent false
11948 4363 Acked Write 7 Done Done Sent false
11949 4364 Acked Write 2 Done Done Sent false
11950 4365 Acked Read 8 Done Done Sent false
11951 4366 Acked Read 7 Done Done Sent false
11952 4367 Acked Write 5 Done Done Sent false
11953 4368 Acked Write 4 Done Done Sent false
11954 4369 Acked Write 1 Done Done Sent false
11955 4370 Acked Read 7 Done Done Sent false
11956 4371 Acked Read 10 Done Done Sent false
11957 4372 Acked Write 4 Done Done Sent false
11958 4373 Acked Read 10 Done Done Sent false
11959 4374 Acked Write 3 Done Done Sent false
11960 4375 Acked Read 10 Done Done Sent false
11961 4376 Acked Write 9 Done Done Sent false
11962 4377 Acked Read 6 Done Done Sent false
11963 4378 Acked Write 5 Done Done Sent false
11964 4379 Acked Write 1 Done Done Sent false
11965 4380 Acked Write 3 Done Done Sent false
11966 4381 Acked Write 2 Done Done Sent false
11967 4382 Acked Read 9 Done Done Sent false
11968 4383 Acked Flush 0 Done Done Sent false
11969 4384 Acked Read 9 Done Done Sent false
11970 4385 Acked Read 8 Done Done Sent false
11971 4386 Acked Write 9 Done Done Sent false
11972 4387 Acked Read 5 Done Done Sent false
11973 4388 Acked Write 6 Done Done Sent false
11974 4389 Acked Write 8 Done Done Sent false
11975 4390 Acked Write 7 Done Done Sent false
11976 4391 Acked Write 7 Done Done Sent false
11977 4392 Acked Write 1 Done Done Sent false
11978 4393 Acked Read 4 Done Done Sent false
11979 4394 Acked Read 6 Done Done Sent false
11980 4395 Acked Write 7 Done Done Sent false
11981 4396 Acked Write 9 Done Done Sent false
11982 4397 Acked Read 1 Done Done Sent false
11983 4398 Acked Write 2 Done Done Sent false
11984 4399 Acked Read 3 Done Done Sent false
11985 4400 Acked Write 6 Done Done Sent false
11986 4401 Acked Read 7 Done Done Sent false
11987 4402 Acked Read 10 Done Done Sent false
11988 4403 Acked Write 9 Done Done Sent false
11989 4404 Acked Read 2 Done Done Sent false
11990 4405 Acked Write 1 Done Done Sent false
11991 4406 Acked Read 9 Done Done Sent false
11992 4407 Acked Flush 0 Done Done Sent false
11993 4408 Acked Write 8 Done Done Sent false
11994 4409 Acked Read 4 Done Done Sent false
11995 4410 Acked Read 1 Done Done Sent false
11996 4411 Acked Read 3 Done Done Sent false
11997 4412 Acked Read 4 Done Done Sent false
11998 4413 Acked Read 9 Done Done Sent false
11999 4414 Acked Flush 0 Done Done Sent false
12000 4415 Acked Write 10 Done Done Sent false
12001 4416 Acked Write 8 Done Done Sent false
12002 4417 Acked Write 6 Done Done Sent false
12003 4418 Acked Read 7 Done Done Sent false
12004 4419 Acked Read 5 Done Done Sent false
12005 4420 Acked Write 3 Done Done Sent false
12006 4421 Acked Write 10 Done Done Sent false
12007 4422 Acked Read 1 Done Done Sent false
12008 4423 Acked Write 9 Done Done Sent false
12009 4424 Acked Write 1 Done Done Sent false
12010 4425 Acked Flush 0 Done Done Sent false
12011 4426 Acked Write 2 Done Done Sent false
12012 4427 Acked Write 7 Done Done Sent false
12013 4428 Acked Read 3 Done Done Sent false
12014 4429 Acked Read 4 Done Done Sent false
12015 4430 Acked Read 3 Done Done Sent false
12016 4431 Acked Read 10 Done Done Sent false
12017 4432 Acked Read 4 Done Done Sent false
12018 4433 Acked Flush 0 Done Done Sent false
12019 4434 Acked Write 1 Done Done Sent false
12020 4435 Acked Read 10 Done Done Sent false
12021 4436 Acked Read 8 Done Done Sent false
12022 4437 Acked Write 8 Done Done Sent false
12023 4438 Acked Write 7 Done Done Sent false
12024 4439 Acked Write 9 Done Done Sent false
12025 4440 Acked Write 7 Done Done Sent false
12026 4441 Acked Read 4 Done Done Sent false
12027 4442 Acked Write 4 Done Done Sent false
12028 4443 Acked Write 7 Done Done Sent false
12029 4444 Acked Read 3 Done Done Sent false
12030 4445 Acked Write 8 Done Done Sent false
12031 4446 Acked Read 2 Done Done Sent false
12032 4447 Acked Read 1 Done Done Sent false
12033 4448 Acked Read 2 Done Done Sent false
12034 4449 Acked Read 2 Done Done Sent false
12035 4450 Acked Read 6 Done Done Sent false
12036 4451 Acked Write 4 Done Done Sent false
12037 4452 Acked Read 1 Done Done Sent false
12038 4453 Acked Flush 0 Done Done Sent false
12039 4454 Acked Write 5 Done Done Sent false
12040 4455 Acked Write 9 Done Done Sent false
12041 4456 Acked Write 5 Done Done Sent false
12042 4457 Acked Read 2 Done Done Sent false
12043 4458 Acked Write 5 Done Done Sent false
12044 4459 Acked Write 2 Done Done Sent false
12045 4460 Acked Write 5 Done Done Sent false
12046 4461 Acked Write 3 Done Done Sent false
12047 4462 Acked Read 5 Done Done Sent false
12048 4463 Acked Write 6 Done Done Sent false
12049 4464 Acked Read 8 Done Done Sent false
12050 4465 Acked Read 1 Done Done Sent false
12051 4466 Acked Read 5 Done Done Sent false
12052 4467 Acked Read 3 Done Done Sent false
12053 4468 Acked Read 10 Done Done Sent false
12054 4469 Acked Read 1 Done Done Sent false
12055 4470 Acked Write 8 Done Done Sent false
12056 4471 Acked Write 2 Done Done Sent false
12057 4472 Acked Write 3 Done Done Sent false
12058 4473 Acked Write 6 Done Done Sent false
12059 4474 Acked Read 9 Done Done Sent false
12060 4475 Acked Write 5 Done Done Sent false
12061 4476 Acked Read 7 Done Done Sent false
12062 4477 Acked Write 9 Done Done Sent false
12063 4478 Acked Write 8 Done Done Sent false
12064 4479 Acked Read 6 Done Done Sent false
12065 4480 Acked Flush 0 Done Done Sent false
12066 4481 Acked Write 1 Done Done Sent false
12067 4482 Acked Write 4 Done Done Sent false
12068 4483 Acked Write 3 Done Done Sent false
12069 4484 Acked Read 9 Done Done Sent false
12070 4485 Acked Flush 0 Done Done Sent false
12071 4486 Acked Read 2 Done Done Sent false
12072 4487 Acked Read 8 Done Done Sent false
12073 4488 Acked Read 4 Done Done Sent false
12074 4489 Acked Write 2 Done Done Sent false
12075 4490 Acked Flush 0 Done Done Sent false
12076 4491 Acked Write 2 Done Done Sent false
12077 4492 Acked Read 8 Done Done Sent false
12078 4493 Acked Write 4 Done Done Sent false
12079 4494 Acked Write 1 Done Done Sent false
12080 4495 Acked Flush 0 Done Done Sent false
12081 4496 Acked Read 7 Done Done Sent false
12082 4497 Acked Write 3 Done Done Sent false
12083 4498 Acked Flush 0 Done Done Sent false
12084 4499 Acked Write 1 Done Done Sent false
12085 4500 Acked Read 6 Done Done Sent false
12086 4501 Acked Read 5 Done Done Sent false
12087 4502 Acked Read 9 Done Done Sent false
12088 4503 Acked Read 1 Done Done Sent false
12089 4504 Acked Read 8 Done Done Sent false
12090 4505 Acked Read 9 Done Done Sent false
12091 4506 Acked Read 3 Done Done Sent false
12092 4507 Acked Write 6 Done Done Sent false
12093 4508 Acked Write 4 Done Done Sent false
12094 4509 Acked Read 1 Done Done Sent false
12095 4510 Acked Write 10 Done Done Sent false
12096 4511 Acked Flush 0 Done Done Sent false
12097 4512 Acked Read 7 Done Done Sent false
12098 4513 Acked Write 6 Done Done Sent false
12099 4514 Acked Read 6 Done Done Sent false
12100 4515 Acked Write 4 Done Done Sent false
12101 4516 Acked Read 1 Done Done Sent false
12102 4517 Acked Write 5 Done Done Sent false
12103 4518 Acked Read 10 Done Done Sent false
12104 4519 Acked Write 9 Done Done Sent false
12105 4520 Acked Read 6 Done Done Sent false
12106 4521 Acked Write 5 Done Done Sent false
12107 4522 Acked Flush 0 Done Done Sent false
12108 4523 Acked Read 5 Done Done Sent false
12109 4524 Acked Read 5 Done Done Sent false
12110 4525 Acked Write 10 Done Done Sent false
12111 4526 Acked Write 5 Done Done Sent false
12112 4527 Acked Read 8 Done Done Sent false
12113 4528 Acked Write 5 Done Done Sent false
12114 4529 Acked Read 3 Done Done Sent false
12115 4530 Acked Write 4 Done Done Sent false
12116 4531 Acked Flush 0 Done Done Sent false
12117 4532 Acked Write 5 Done Done Sent false
12118 4533 Acked Flush 0 Done Done Sent false
12119 4534 Acked Read 1 Done Done Sent false
12120 4535 Acked Read 3 Done Done Sent false
12121 4536 Acked Write 1 Done Done Sent false
12122 4537 Acked Write 8 Done Done Sent false
12123 4538 Acked Read 5 Done Done Sent false
12124 4539 Acked Write 5 Done Done Sent false
12125 4540 Acked Write 10 Done Done Sent false
12126 4541 Acked Write 8 Done Done Sent false
12127 4542 Acked Read 4 Done Done Sent false
12128 4543 Acked Write 1 Done Done Sent false
12129 4544 Acked Read 3 Done Done Sent false
12130 4545 Acked Write 6 Done Done Sent false
12131 4546 Acked Read 4 Done Done Sent false
12132 4547 Acked Write 7 Done Done Sent false
12133 4548 Acked Write 8 Done Done Sent false
12134 4549 Acked Read 2 Done Done Sent false
12135 4550 Acked Read 7 Done Done Sent false
12136 4551 Acked Write 8 Done Done Sent false
12137 4552 Acked Write 7 Done Done Sent false
12138 4553 Acked Write 5 Done Done Sent false
12139 4554 Acked Read 8 Done Done Sent false
12140 4555 Acked Write 10 Done Done Sent false
12141 4556 Acked Write 10 Done Done Sent false
12142 4557 Acked Flush 0 Done Done Sent false
12143 4558 Acked Write 4 Done Done Sent false
12144 4559 Acked Read 8 Done Done Sent false
12145 4560 Acked Read 7 Done Done Sent false
12146 4561 Acked Write 5 Done Done Sent false
12147 4562 Acked Read 3 Done Done Sent false
12148 4563 Acked Read 4 Done Done Sent false
12149 4564 Acked Read 2 Done Done Sent false
12150 4565 Acked Write 7 Done Done Sent false
12151 4566 Acked Read 10 Done Done Sent false
12152 4567 Acked Read 5 Done Done Sent false
12153 4568 Acked Read 8 Done Done Sent false
12154 4569 Acked Read 6 Done Done Sent false
12155 4570 Acked Read 3 Done Done Sent false
12156 4571 Acked Read 5 Done Done Sent false
12157 4572 Acked Read 5 Done Done Sent false
12158 4573 Acked Write 3 Done Done Sent false
12159 4574 Acked Read 2 Done Done Sent false
12160 4575 Acked Write 10 Done Done Sent false
12161 4576 Acked Read 8 Done Done Sent false
12162 4577 Acked Read 9 Done Done Sent false
12163 4578 Acked Write 6 Done Done Sent false
12164 4579 Acked Flush 0 Done Done Sent false
12165 4580 Acked Write 2 Done Done Sent false
12166 4581 Acked Write 10 Done Done Sent false
12167 4582 Acked Read 4 Done Done Sent false
12168 4583 Acked Read 5 Done Done Sent false
12169 4584 Acked Read 1 Done Done Sent false
12170 4585 Acked Write 8 Done Done Sent false
12171 4586 Acked Read 7 Done Done Sent false
12172 4587 Acked Read 2 Done Done Sent false
12173 4588 Acked Read 8 Done Done Sent false
12174 4589 Acked Read 9 Done Done Sent false
12175 4590 Acked Read 5 Done Done Sent false
12176 4591 Acked Read 10 Done Done Sent false
12177 4592 Acked Read 10 Done Done Sent false
12178 4593 Acked Read 5 Done Done Sent false
12179 4594 Acked Read 5 Done Done Sent false
12180 4595 Acked Flush 0 Done Done Sent false
12181 4596 Acked Write 8 Done Done Sent false
12182 4597 Acked Write 8 Done Done Sent false
12183 4598 Acked Flush 0 Done Done Sent false
12184 4599 Acked Write 4 Done Done Sent false
12185 4600 Acked Write 3 Done Done Sent false
12186 4601 Acked Write 7 Done Done Sent false
12187 4602 Acked Write 9 Done Done Sent false
12188 4603 Acked Read 1 Done Done Sent false
12189 4604 Acked Write 3 Done Done Sent false
12190 4605 Acked Read 10 Done Done Sent false
12191 4606 Acked Read 5 Done Done Sent false
12192 4607 Acked Write 3 Done Done Sent false
12193 4608 Acked Read 3 Done Done Sent false
12194 4609 Acked Write 5 Done Done Sent false
12195 4610 Acked Write 9 Done Done Sent false
12196 4611 Acked Read 5 Done Done Sent false
12197 4612 Acked Read 8 Done Done Sent false
12198 4613 Acked Write 8 Done Done Sent false
12199 4614 Acked Read 6 Done Done Sent false
12200 4615 Acked Flush 0 Done Done Sent false
12201 4616 Acked Write 3 Done Done Sent false
12202 4617 Acked Write 7 Done Done Sent false
12203 4618 Acked Write 7 Done Done Sent false
12204 4619 Acked Read 10 Done Done Sent false
12205 4620 Acked Read 7 Done Done Sent false
12206 4621 Acked Flush 0 Done Done Sent false
12207 STATES DS:0 DS:1 DS:2 TOTAL
12208 Sent 0 0 301 301
12209 Done 301 301 0 602
12210 Skipped 0 0 0 0
12211 Error 0 0 0 0
12212 Last Flush: 4621 4621 4320
12213 Downstairs last five completed: 4320 4319 4318 4317 4316
12214 Upstairs last five completed: 4621 4620 4619 4618 4617
12215 CLIENT: Up:0 ds:301 act:2
12216 [8830] got start action
12217 [8830] I am going to restart
12218 Starting downstairs at port 8830
12219 Make output file at "/tmp/dsc/downstairs-8830.txt"
12220 [8830][2] reports Starting
12221 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1214)
12222 [8830][2] reports Running
12223 ----------------------------------------------------------------
12224 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
12225 States: Active Active Offline
12226 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
12227 4321 Acked Write 9 Done Done Sent false
12228 4322 Acked Write 10 Done Done Sent false
12229 4323 Acked Write 7 Done Done Sent false
12230 4324 Acked Read 10 Done Done Sent false
12231 4325 Acked Read 4 Done Done Sent false
12232 4326 Acked Write 4 Done Done Sent false
12233 4327 Acked Write 7 Done Done Sent false
12234 4328 Acked Write 3 Done Done Sent false
12235 4329 Acked Read 2 Done Done Sent false
12236 4330 Acked Write 7 Done Done Sent false
12237 4331 Acked Read 10 Done Done Sent false
12238 4332 Acked Read 6 Done Done Sent false
12239 4333 Acked Read 8 Done Done Sent false
12240 4334 Acked Read 1 Done Done Sent false
12241 4335 Acked Read 3 Done Done Sent false
12242 4336 Acked Write 6 Done Done Sent false
12243 4337 Acked Flush 0 Done Done Sent false
12244 4338 Acked Flush 0 Done Done Sent false
12245 4339 Acked Read 4 Done Done Sent false
12246 4340 Acked Write 1 Done Done Sent false
12247 4341 Acked Read 4 Done Done Sent false
12248 4342 Acked Read 4 Done Done Sent false
12249 4343 Acked Write 5 Done Done Sent false
12250 4344 Acked Write 9 Done Done Sent false
12251 4345 Acked Write 5 Done Done Sent false
12252 4346 Acked Write 8 Done Done Sent false
12253 4347 Acked Write 5 Done Done Sent false
12254 4348 Acked Read 5 Done Done Sent false
12255 4349 Acked Flush 0 Done Done Sent false
12256 4350 Acked Write 2 Done Done Sent false
12257 4351 Acked Read 8 Done Done Sent false
12258 4352 Acked Write 5 Done Done Sent false
12259 4353 Acked Write 2 Done Done Sent false
12260 4354 Acked Read 3 Done Done Sent false
12261 4355 Acked Write 8 Done Done Sent false
12262 4356 Acked Write 2 Done Done Sent false
12263 4357 Acked Write 6 Done Done Sent false
12264 4358 Acked Read 3 Done Done Sent false
12265 4359 Acked Read 3 Done Done Sent false
12266 4360 Acked Read 6 Done Done Sent false
12267 4361 Acked Read 6 Done Done Sent false
12268 4362 Acked Read 9 Done Done Sent false
12269 4363 Acked Write 7 Done Done Sent false
12270 4364 Acked Write 2 Done Done Sent false
12271 4365 Acked Read 8 Done Done Sent false
12272 4366 Acked Read 7 Done Done Sent false
12273 4367 Acked Write 5 Done Done Sent false
12274 4368 Acked Write 4 Done Done Sent false
12275 4369 Acked Write 1 Done Done Sent false
12276 4370 Acked Read 7 Done Done Sent false
12277 4371 Acked Read 10 Done Done Sent false
12278 4372 Acked Write 4 Done Done Sent false
12279 4373 Acked Read 10 Done Done Sent false
12280 4374 Acked Write 3 Done Done Sent false
12281 4375 Acked Read 10 Done Done Sent false
12282 4376 Acked Write 9 Done Done Sent false
12283 4377 Acked Read 6 Done Done Sent false
12284 4378 Acked Write 5 Done Done Sent false
12285 4379 Acked Write 1 Done Done Sent false
12286 4380 Acked Write 3 Done Done Sent false
12287 4381 Acked Write 2 Done Done Sent false
12288 4382 Acked Read 9 Done Done Sent false
12289 4383 Acked Flush 0 Done Done Sent false
12290 4384 Acked Read 9 Done Done Sent false
12291 4385 Acked Read 8 Done Done Sent false
12292 4386 Acked Write 9 Done Done Sent false
12293 4387 Acked Read 5 Done Done Sent false
12294 4388 Acked Write 6 Done Done Sent false
12295 4389 Acked Write 8 Done Done Sent false
12296 4390 Acked Write 7 Done Done Sent false
12297 4391 Acked Write 7 Done Done Sent false
12298 4392 Acked Write 1 Done Done Sent false
12299 4393 Acked Read 4 Done Done Sent false
12300 4394 Acked Read 6 Done Done Sent false
12301 4395 Acked Write 7 Done Done Sent false
12302 4396 Acked Write 9 Done Done Sent false
12303 4397 Acked Read 1 Done Done Sent false
12304 4398 Acked Write 2 Done Done Sent false
12305 4399 Acked Read 3 Done Done Sent false
12306 4400 Acked Write 6 Done Done Sent false
12307 4401 Acked Read 7 Done Done Sent false
12308 4402 Acked Read 10 Done Done Sent false
12309 4403 Acked Write 9 Done Done Sent false
12310 4404 Acked Read 2 Done Done Sent false
12311 4405 Acked Write 1 Done Done Sent false
12312 4406 Acked Read 9 Done Done Sent false
12313 4407 Acked Flush 0 Done Done Sent false
12314 4408 Acked Write 8 Done Done Sent false
12315 4409 Acked Read 4 Done Done Sent false
12316 4410 Acked Read 1 Done Done Sent false
12317 4411 Acked Read 3 Done Done Sent false
12318 4412 Acked Read 4 Done Done Sent false
12319 4413 Acked Read 9 Done Done Sent false
12320 4414 Acked Flush 0 Done Done Sent false
12321 4415 Acked Write 10 Done Done Sent false
12322 4416 Acked Write 8 Done Done Sent false
12323 4417 Acked Write 6 Done Done Sent false
12324 4418 Acked Read 7 Done Done Sent false
12325 4419 Acked Read 5 Done Done Sent false
12326 4420 Acked Write 3 Done Done Sent false
12327 4421 Acked Write 10 Done Done Sent false
12328 4422 Acked Read 1 Done Done Sent false
12329 4423 Acked Write 9 Done Done Sent false
12330 4424 Acked Write 1 Done Done Sent false
12331 4425 Acked Flush 0 Done Done Sent false
12332 4426 Acked Write 2 Done Done Sent false
12333 4427 Acked Write 7 Done Done Sent false
12334 4428 Acked Read 3 Done Done Sent false
12335 4429 Acked Read 4 Done Done Sent false
12336 4430 Acked Read 3 Done Done Sent false
12337 4431 Acked Read 10 Done Done Sent false
12338 4432 Acked Read 4 Done Done Sent false
12339 4433 Acked Flush 0 Done Done Sent false
12340 4434 Acked Write 1 Done Done Sent false
12341 4435 Acked Read 10 Done Done Sent false
12342 4436 Acked Read 8 Done Done Sent false
12343 4437 Acked Write 8 Done Done Sent false
12344 4438 Acked Write 7 Done Done Sent false
12345 4439 Acked Write 9 Done Done Sent false
12346 4440 Acked Write 7 Done Done Sent false
12347 4441 Acked Read 4 Done Done Sent false
12348 4442 Acked Write 4 Done Done Sent false
12349 4443 Acked Write 7 Done Done Sent false
12350 4444 Acked Read 3 Done Done Sent false
12351 4445 Acked Write 8 Done Done Sent false
12352 4446 Acked Read 2 Done Done Sent false
12353 4447 Acked Read 1 Done Done Sent false
12354 4448 Acked Read 2 Done Done Sent false
12355 4449 Acked Read 2 Done Done Sent false
12356 4450 Acked Read 6 Done Done Sent false
12357 4451 Acked Write 4 Done Done Sent false
12358 4452 Acked Read 1 Done Done Sent false
12359 4453 Acked Flush 0 Done Done Sent false
12360 4454 Acked Write 5 Done Done Sent false
12361 4455 Acked Write 9 Done Done Sent false
12362 4456 Acked Write 5 Done Done Sent false
12363 4457 Acked Read 2 Done Done Sent false
12364 4458 Acked Write 5 Done Done Sent false
12365 4459 Acked Write 2 Done Done Sent false
12366 4460 Acked Write 5 Done Done Sent false
12367 4461 Acked Write 3 Done Done Sent false
12368 4462 Acked Read 5 Done Done Sent false
12369 4463 Acked Write 6 Done Done Sent false
12370 4464 Acked Read 8 Done Done Sent false
12371 4465 Acked Read 1 Done Done Sent false
12372 4466 Acked Read 5 Done Done Sent false
12373 4467 Acked Read 3 Done Done Sent false
12374 4468 Acked Read 10 Done Done Sent false
12375 4469 Acked Read 1 Done Done Sent false
12376 4470 Acked Write 8 Done Done Sent false
12377 4471 Acked Write 2 Done Done Sent false
12378 4472 Acked Write 3 Done Done Sent false
12379 4473 Acked Write 6 Done Done Sent false
12380 4474 Acked Read 9 Done Done Sent false
12381 4475 Acked Write 5 Done Done Sent false
12382 4476 Acked Read 7 Done Done Sent false
12383 4477 Acked Write 9 Done Done Sent false
12384 4478 Acked Write 8 Done Done Sent false
12385 4479 Acked Read 6 Done Done Sent false
12386 4480 Acked Flush 0 Done Done Sent false
12387 4481 Acked Write 1 Done Done Sent false
12388 4482 Acked Write 4 Done Done Sent false
12389 4483 Acked Write 3 Done Done Sent false
12390 4484 Acked Read 9 Done Done Sent false
12391 4485 Acked Flush 0 Done Done Sent false
12392 4486 Acked Read 2 Done Done Sent false
12393 4487 Acked Read 8 Done Done Sent false
12394 4488 Acked Read 4 Done Done Sent false
12395 4489 Acked Write 2 Done Done Sent false
12396 4490 Acked Flush 0 Done Done Sent false
12397 4491 Acked Write 2 Done Done Sent false
12398 4492 Acked Read 8 Done Done Sent false
12399 4493 Acked Write 4 Done Done Sent false
12400 4494 Acked Write 1 Done Done Sent false
12401 4495 Acked Flush 0 Done Done Sent false
12402 4496 Acked Read 7 Done Done Sent false
12403 4497 Acked Write 3 Done Done Sent false
12404 4498 Acked Flush 0 Done Done Sent false
12405 4499 Acked Write 1 Done Done Sent false
12406 4500 Acked Read 6 Done Done Sent false
12407 4501 Acked Read 5 Done Done Sent false
12408 4502 Acked Read 9 Done Done Sent false
12409 4503 Acked Read 1 Done Done Sent false
12410 4504 Acked Read 8 Done Done Sent false
12411 4505 Acked Read 9 Done Done Sent false
12412 4506 Acked Read 3 Done Done Sent false
12413 4507 Acked Write 6 Done Done Sent false
12414 4508 Acked Write 4 Done Done Sent false
12415 4509 Acked Read 1 Done Done Sent false
12416 4510 Acked Write 10 Done Done Sent false
12417 4511 Acked Flush 0 Done Done Sent false
12418 4512 Acked Read 7 Done Done Sent false
12419 4513 Acked Write 6 Done Done Sent false
12420 4514 Acked Read 6 Done Done Sent false
12421 4515 Acked Write 4 Done Done Sent false
12422 4516 Acked Read 1 Done Done Sent false
12423 4517 Acked Write 5 Done Done Sent false
12424 4518 Acked Read 10 Done Done Sent false
12425 4519 Acked Write 9 Done Done Sent false
12426 4520 Acked Read 6 Done Done Sent false
12427 4521 Acked Write 5 Done Done Sent false
12428 4522 Acked Flush 0 Done Done Sent false
12429 4523 Acked Read 5 Done Done Sent false
12430 4524 Acked Read 5 Done Done Sent false
12431 4525 Acked Write 10 Done Done Sent false
12432 4526 Acked Write 5 Done Done Sent false
12433 4527 Acked Read 8 Done Done Sent false
12434 4528 Acked Write 5 Done Done Sent false
12435 4529 Acked Read 3 Done Done Sent false
12436 4530 Acked Write 4 Done Done Sent false
12437 4531 Acked Flush 0 Done Done Sent false
12438 4532 Acked Write 5 Done Done Sent false
12439 4533 Acked Flush 0 Done Done Sent false
12440 4534 Acked Read 1 Done Done Sent false
12441 4535 Acked Read 3 Done Done Sent false
12442 4536 Acked Write 1 Done Done Sent false
12443 4537 Acked Write 8 Done Done Sent false
12444 4538 Acked Read 5 Done Done Sent false
12445 4539 Acked Write 5 Done Done Sent false
12446 4540 Acked Write 10 Done Done Sent false
12447 4541 Acked Write 8 Done Done Sent false
12448 4542 Acked Read 4 Done Done Sent false
12449 4543 Acked Write 1 Done Done Sent false
12450 4544 Acked Read 3 Done Done Sent false
12451 4545 Acked Write 6 Done Done Sent false
12452 4546 Acked Read 4 Done Done Sent false
12453 4547 Acked Write 7 Done Done Sent false
12454 4548 Acked Write 8 Done Done Sent false
12455 4549 Acked Read 2 Done Done Sent false
12456 4550 Acked Read 7 Done Done Sent false
12457 4551 Acked Write 8 Done Done Sent false
12458 4552 Acked Write 7 Done Done Sent false
12459 4553 Acked Write 5 Done Done Sent false
12460 4554 Acked Read 8 Done Done Sent false
12461 4555 Acked Write 10 Done Done Sent false
12462 4556 Acked Write 10 Done Done Sent false
12463 4557 Acked Flush 0 Done Done Sent false
12464 4558 Acked Write 4 Done Done Sent false
12465 4559 Acked Read 8 Done Done Sent false
12466 4560 Acked Read 7 Done Done Sent false
12467 4561 Acked Write 5 Done Done Sent false
12468 4562 Acked Read 3 Done Done Sent false
12469 4563 Acked Read 4 Done Done Sent false
12470 4564 Acked Read 2 Done Done Sent false
12471 4565 Acked Write 7 Done Done Sent false
12472 4566 Acked Read 10 Done Done Sent false
12473 4567 Acked Read 5 Done Done Sent false
12474 4568 Acked Read 8 Done Done Sent false
12475 4569 Acked Read 6 Done Done Sent false
12476 4570 Acked Read 3 Done Done Sent false
12477 4571 Acked Read 5 Done Done Sent false
12478 4572 Acked Read 5 Done Done Sent false
12479 4573 Acked Write 3 Done Done Sent false
12480 4574 Acked Read 2 Done Done Sent false
12481 4575 Acked Write 10 Done Done Sent false
12482 4576 Acked Read 8 Done Done Sent false
12483 4577 Acked Read 9 Done Done Sent false
12484 4578 Acked Write 6 Done Done Sent false
12485 4579 Acked Flush 0 Done Done Sent false
12486 4580 Acked Write 2 Done Done Sent false
12487 4581 Acked Write 10 Done Done Sent false
12488 4582 Acked Read 4 Done Done Sent false
12489 4583 Acked Read 5 Done Done Sent false
12490 4584 Acked Read 1 Done Done Sent false
12491 4585 Acked Write 8 Done Done Sent false
12492 4586 Acked Read 7 Done Done Sent false
12493 4587 Acked Read 2 Done Done Sent false
12494 4588 Acked Read 8 Done Done Sent false
12495 4589 Acked Read 9 Done Done Sent false
12496 4590 Acked Read 5 Done Done Sent false
12497 4591 Acked Read 10 Done Done Sent false
12498 4592 Acked Read 10 Done Done Sent false
12499 4593 Acked Read 5 Done Done Sent false
12500 4594 Acked Read 5 Done Done Sent false
12501 4595 Acked Flush 0 Done Done Sent false
12502 4596 Acked Write 8 Done Done Sent false
12503 4597 Acked Write 8 Done Done Sent false
12504 4598 Acked Flush 0 Done Done Sent false
12505 4599 Acked Write 4 Done Done Sent false
12506 4600 Acked Write 3 Done Done Sent false
12507 4601 Acked Write 7 Done Done Sent false
12508 4602 Acked Write 9 Done Done Sent false
12509 4603 Acked Read 1 Done Done Sent false
12510 4604 Acked Write 3 Done Done Sent false
12511 4605 Acked Read 10 Done Done Sent false
12512 4606 Acked Read 5 Done Done Sent false
12513 4607 Acked Write 3 Done Done Sent false
12514 4608 Acked Read 3 Done Done Sent false
12515 4609 Acked Write 5 Done Done Sent false
12516 4610 Acked Write 9 Done Done Sent false
12517 4611 Acked Read 5 Done Done Sent false
12518 4612 Acked Read 8 Done Done Sent false
12519 4613 Acked Write 8 Done Done Sent false
12520 4614 Acked Read 6 Done Done Sent false
12521 4615 Acked Flush 0 Done Done Sent false
12522 4616 Acked Write 3 Done Done Sent false
12523 4617 Acked Write 7 Done Done Sent false
12524 4618 Acked Write 7 Done Done Sent false
12525 4619 Acked Read 10 Done Done Sent false
12526 4620 Acked Read 7 Done Done Sent false
12527 4621 Acked Flush 0 Done Done Sent false
12528 STATES DS:0 DS:1 DS:2 TOTAL
12529 Sent 0 0 301 301
12530 Done 301 301 0 602
12531 Skipped 0 0 0 0
12532 Error 0 0 0 0
12533 Last Flush: 4621 4621 4320
12534 Downstairs last five completed: 4320 4319 4318 4317 4316
12535 Upstairs last five completed: 4621 4620 4619 4618 4617
12536 CLIENT: Up:0 ds:301 act:2
125372024-11-28T04:52:55.680ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
125382024-11-28T04:52:55.680ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:61894) = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
125392024-11-28T04:52:55.681ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
125402024-11-28T04:52:55.682ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
125412024-11-28T04:52:55.682ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
125422024-11-28T04:52:55.682ZINFOcrucible: send last flush ID to this DS: 4320 = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
125432024-11-28T04:52:55.682ZINFOcrucible: Replied this last flush ID: 4320 = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
125442024-11-28T04:52:55.682ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
125452024-11-28T04:52:55.682ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
125462024-11-28T04:52:55.682ZINFOcrucible: [2] client re-new 301 jobs since flush 4320 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
125472024-11-28T04:52:55.682ZINFOcrucible: [2] Replayed 301 jobs since flush: 4320 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
12548 ----------------------------------------------------------------
12549 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
12550 Downstairs last five completed: 4621 4620 4619 4618 4617
12551 Upstairs last five completed: 4621 4620 4619 4618 4617
12552 CLIENT: Up:0 ds:0 act:3
12553 Replay: All jobs finished
12554 Main task has work to do, go find it
12555 got dsc Stop(1)
12556 stop 1
12557 Nov 28 04:52:57.918 INFO[8820] Got stop action so:false kr:false
12558 request completed, latency_us: 386, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 36fde39c-c0cc-4f5f-b027-701dc4ffeeb7, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
12559 Nov 28 04:52:57.919 INFO request completed, latency_us: 118, response_code: 200, uri: /state/cid/1, method: GET, req_id: 7f9d6e7e-b964-4768-8e98-52f7ac63a674, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
125602024-11-28T04:52:57.919ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
125612024-11-28T04:52:57.919ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
125622024-11-28T04:52:57.919ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
125632024-11-28T04:52:57.919ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
125642024-11-28T04:52:57.919ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
125652024-11-28T04:52:57.919ZINFOcrucible: client task is exiting = io task client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
12566 [8820] Exited with: ExitStatus(unix_wait_status(9))
12567 [8820][1] reports Exit
12568 Nov 28 04:53:01.921 INFO request completed, latency_us: 199, response_code: 200, uri: /state/cid/1, method: GET, req_id: a3bf3b84-a886-49fe-ac4d-cc00c44c888f, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
12569 001/300 Flush
12570 002/300 Write block 2 len 40960 data: 2 2 3 3 3 3 4 7 7 7
12571 003/300 Write block 3251 len 36864 data: 4 4 5 5 7 7 6 6 4
12572 004/300 Write block 2259 len 12288 data: 3 4 5
12573 005/300 Write block 4121 len 40960 data: 6 7 3 3 4 4 5 5 5 5
12574 006/300 Flush
12575 007/300 Flush
12576 008/300 Write block 178 len 36864 data: 2 3 4 4 3 3 3 2 2
12577 009/300 Read block 3839 len 12288
12578 010/300 Write block 4431 len 20480 data: 4 4 3 2 2
12579 011/300 Read block 2382 len 12288
12580 012/300 Read block 2200 len 16384
12581 013/300 Read block 1987 len 40960
12582 014/300 Read block 4116 len 28672
12583 015/300 Write block 2551 len 40960 data: 3 3 3 3 3 3 4 3 3 3
12584 016/300 Read block 3034 len 16384
12585 017/300 Write block 806 len 12288 data: 3 3 3
12586 018/300 Write block 1516 len 20480 data: 3 3 3 3 3
12587 019/300 Read block 1274 len 24576
12588 020/300 Read block 3403 len 4096
12589 021/300 Write block 1258 len 40960 data: 3 3 4 4 4 4 4 5 5 5
12590 022/300 Write block 4653 len 28672 data: 3 2 3 3 3 4 4
12591 023/300 Read block 869 len 24576
12592 024/300 Read block 1589 len 28672
12593 025/300 Write block 2383 len 32768 data: 3 2 2 3 3 4 4 3
12594 026/300 Read block 372 len 20480
12595 027/300 Read block 3314 len 32768
12596 028/300 Write block 405 len 28672 data: 3 3 5 5 5 5 5
12597 029/300 Read block 423 len 4096
12598 030/300 Read block 3419 len 40960
12599 031/300 Read block 4102 len 8192
12600 032/300 Read block 3018 len 8192
12601 033/300 Read block 4852 len 40960
12602 034/300 Write block 4342 len 4096 data: 3
12603 035/300 Write block 2268 len 4096 data: 3
12604 036/300 Read block 2306 len 32768
12605 037/300 Write block 3403 len 32768 data: 4 5 4 4 4 3 3 3
12606 038/300 Write block 3336 len 36864 data: 3 3 2 2 3 3 3 2 2
12607 039/300 Write block 2973 len 12288 data: 3 4 5
12608 040/300 Write block 3952 len 28672 data: 5 5 4 4 4 4 4
12609 041/300 Read block 2084 len 12288
12610 042/300 Read block 2862 len 8192
12611 043/300 Read block 4832 len 36864
12612 044/300 Read block 2997 len 28672
12613 045/300 Read block 4227 len 40960
12614 046/300 Read block 3184 len 24576
12615 047/300 Write block 2602 len 40960 data: 2 2 2 2 2 2 2 2 2 2
12616 048/300 Write block 2012 len 12288 data: 3 2 2
12617 049/300 Read block 1287 len 32768
12618 050/300 Read block 460 len 16384
12619 051/300 Write block 1815 len 12288 data: 4 4 3
12620 052/300 Write block 684 len 20480 data: 3 4 4 3 3
12621 053/300 Read block 4254 len 36864
12622 054/300 Write block 1629 len 16384 data: 4 4 3 3
12623 055/300 Read block 3995 len 40960
12624 056/300 Read block 21 len 20480
12625 057/300 Flush
12626 058/300 Write block 1310 len 16384 data: 3 3 2 2
12627 059/300 Write block 3640 len 8192 data: 4 3
12628 060/300 Write block 211 len 4096 data: 5
12629 061/300 Write block 2384 len 8192 data: 3 3
12630 062/300 Flush
12631 063/300 Read block 1412 len 32768
12632 064/300 Write block 2080 len 36864 data: 4 4 3 4 5 5 4 3 3
12633 065/300 Flush
12634 066/300 Write block 2942 len 36864 data: 3 3 4 4 5 6 6 6 5
12635 067/300 Read block 2005 len 28672
12636 068/300 Read block 997 len 28672
12637 069/300 Write block 703 len 24576 data: 2 2 3 3 3 3
12638 070/300 Write block 1402 len 16384 data: 4 3 3 4
12639 071/300 Read block 835 len 32768
12640 072/300 Write block 2801 len 24576 data: 2 2 2 3 4 2
12641 073/300 Write block 4039 len 32768 data: 2 2 2 3 4 4 3 3
12642 074/300 Write block 2233 len 12288 data: 3 3 4
12643 075/300 Read block 4520 len 12288
12644 076/300 Write block 480 len 16384 data: 2 2 2 2
12645 077/300 Write block 652 len 24576 data: 4 4 4 5 6 7
12646 078/300 Read block 2685 len 24576
12647 079/300 Write block 958 len 12288 data: 3 3 3
12648 080/300 Read block 1429 len 40960
12649 081/300 Write block 4059 len 16384 data: 6 6 5 5
12650 082/300 Write block 572 len 16384 data: 2 2 2 2
12651 083/300 Write block 3798 len 4096 data: 5
12652 084/300 Read block 2464 len 16384
12653 085/300 Read block 3460 len 8192
12654 086/300 Write block 2123 len 20480 data: 4 4 4 5 5
12655 087/300 Read block 2894 len 40960
12656 088/300 Write block 2522 len 4096 data: 7
12657 089/300 Read block 394 len 20480
12658 090/300 Read block 2293 len 12288
12659 091/300 Write block 3252 len 28672 data: 5 6 6 8 8 7 7
12660 092/300 Write block 4792 len 36864 data: 3 3 3 3 3 5 5 5 4
12661 093/300 Read block 4767 len 32768
12662 094/300 Read block 929 len 40960
12663 095/300 Write block 3742 len 40960 data: 4 4 4 3 4 4 3 2 2 2
12664 096/300 Write block 3870 len 20480 data: 3 3 3 3 2
12665 097/300 Read block 579 len 16384
12666 098/300 Write block 2805 len 40960 data: 5 3 2 2 3 3 3 3 2 2
12667 099/300 Read block 3637 len 32768
12668 100/300 Write block 4088 len 24576 data: 4 4 5 5 5 5
12669 101/300 Read block 4878 len 24576
12670 102/300 Read block 1107 len 20480
12671 103/300 Read block 2668 len 36864
12672 104/300 Read block 4310 len 4096
12673 105/300 Read block 4535 len 20480
12674 106/300 Write block 3420 len 4096 data: 3
12675 107/300 Write block 1300 len 40960 data: 3 3 3 4 4 3 3 3 3 3
12676 108/300 Read block 1919 len 24576
12677 109/300 Read block 149 len 36864
12678 110/300 Read block 1990 len 16384
12679 111/300 Read block 4734 len 28672
12680 112/300 Read block 4679 len 28672
12681 113/300 Read block 515 len 4096
12682 114/300 Read block 4764 len 36864
12683 115/300 Write block 1132 len 12288 data: 2 2 3
12684 116/300 Read block 3990 len 36864
12685 117/300 Write block 3497 len 12288 data: 4 5 5
12686 118/300 Write block 3004 len 16384 data: 4 6 6 5
12687 119/300 Read block 3232 len 24576
12688 120/300 Read block 1190 len 12288
12689 121/300 Flush
12690 122/300 Flush
12691 123/300 Write block 236 len 32768 data: 2 2 2 2 2 3 3 4
12692 124/300 Read block 1059 len 16384
12693 125/300 Write block 3455 len 20480 data: 3 3 3 3 3
12694 126/300 Write block 4240 len 16384 data: 2 2 3 4
12695 127/300 Read block 288 len 40960
12696 128/300 Write block 4574 len 4096 data: 3
12697 129/300 Read block 3501 len 36864
12698 130/300 Write block 2506 len 40960 data: 3 4 3 3 3 3 3 3 4 4
12699 131/300 Read block 3774 len 28672
12700 132/300 Read block 2049 len 8192
12701 133/300 Write block 2190 len 4096 data: 3
12702 134/300 Write block 414 len 36864 data: 5 4 2 2 2 2 2 2 2
12703 135/300 Read block 4644 len 20480
12704 136/300 Read block 1173 len 36864
12705 137/300 Read block 447 len 16384
12706 138/300 Write block 4467 len 40960 data: 5 4 4 5 5 4 4 5 4 4
12707 139/300 Flush
12708 140/300 Read block 4768 len 40960
12709 141/300 Read block 4984 len 32768
12710 142/300 Write block 3722 len 28672 data: 2 2 2 2 2 2 2
12711 143/300 Write block 2750 len 28672 data: 6 7 6 6 5 4 4
12712 144/300 Read block 4860 len 4096
12713 145/300 Read block 2680 len 20480
12714 146/300 Write block 4383 len 32768 data: 3 4 3 3 4 3 3 3
12715 147/300 Read block 1078 len 32768
12716 148/300 Write block 3085 len 32768 data: 4 3 2 3 4 4 4 4
12717 149/300 Read block 4029 len 40960
12718 150/300 Write block 1682 len 24576 data: 2 2 3 3 3 3
12719 151/300 Write block 1119 len 16384 data: 2 2 2 3
12720 152/300 Flush
12721 153/300 Read block 696 len 32768
12722 154/300 Write block 3627 len 12288 data: 3 4 4
12723 155/300 Write block 776 len 12288 data: 3 3 3
12724 156/300 Write block 731 len 8192 data: 4 4
12725 157/300 Flush
12726 158/300 Flush
12727 159/300 Write block 2449 len 32768 data: 2 2 2 2 2 2 2 2
12728 160/300 Read block 2784 len 24576
12729 161/300 Read block 3120 len 8192
12730 162/300 Flush
12731 163/300 Read block 1913 len 20480
12732 164/300 Read block 257 len 4096
12733 165/300 Write block 3442 len 4096 data: 2
12734 166/300 Read block 52 len 40960
12735 167/300 Write block 703 len 12288 data: 3 3 4
12736 168/300 Write block 2397 len 16384 data: 6 4 3 2
12737 169/300 Write block 2916 len 8192 data: 2 3
12738 170/300 Write block 2192 len 4096 data: 4
12739 171/300 Flush
12740 172/300 Read block 893 len 4096
12741 173/300 Read block 1875 len 40960
12742 174/300 Read block 4433 len 40960
12743 175/300 Read block 1666 len 4096
12744 176/300 Write block 3644 len 32768 data: 4 4 3 3 4 6 7 6
12745 177/300 Flush
12746 178/300 Write block 1299 len 16384 data: 3 4 4 4
12747 179/300 Read block 4349 len 28672
12748 180/300 Read block 219 len 28672
12749 181/300 Flush
12750 182/300 Read block 4210 len 24576
12751 183/300 Write block 3245 len 4096 data: 3
12752 184/300 Flush
12753 185/300 Read block 992 len 20480
12754 186/300 Write block 4371 len 4096 data: 4
12755 187/300 Read block 843 len 12288
12756 188/300 Flush
12757 189/300 Write block 2471 len 36864 data: 4 4 4 4 4 5 3 3 3
12758 190/300 Read block 1717 len 16384
12759 191/300 Write block 3564 len 40960 data: 3 3 3 4 4 4 3 3 3 2
12760 192/300 Write block 3330 len 8192 data: 6 6
12761 193/300 Read block 1333 len 20480
12762 194/300 Write block 4445 len 40960 data: 6 8 7 6 6 7 4 4 3 4
12763 195/300 Write block 4550 len 32768 data: 4 3 4 3 3 3 2 2
12764 196/300 Write block 2701 len 8192 data: 4 4
12765 197/300 Write block 2296 len 8192 data: 2 3
12766 198/300 Read block 3548 len 12288
12767 199/300 Read block 1325 len 12288
12768 200/300 Read block 4677 len 20480
12769 201/300 Read block 1920 len 40960
12770 202/300 Write block 2344 len 4096 data: 3
12771 203/300 Write block 4853 len 8192 data: 5 5
12772 204/300 Write block 1696 len 40960 data: 3 2 3 3 3 3 4 7 7 9
12773 205/300 Write block 4503 len 32768 data: 3 3 3 3 3 3 2 2
12774 206/300 Read block 4684 len 24576
12775 207/300 Read block 4306 len 12288
12776 208/300 Write block 1464 len 24576 data: 3 3 3 4 4 3
12777 209/300 Write block 4806 len 8192 data: 6 4
12778 210/300 Read block 2020 len 32768
12779 211/300 Write block 4604 len 24576 data: 4 4 3 3 3 3
12780 212/300 Write block 4853 len 4096 data: 6
12781 213/300 Write block 702 len 20480 data: 2 4 4 5 4
12782 214/300 Read block 4140 len 16384
12783 215/300 Write block 3970 len 28672 data: 3 2 2 2 2 3 3
12784 216/300 Write block 1891 len 16384 data: 5 5 5 4
12785 217/300 Write block 1046 len 40960 data: 4 4 4 5 5 6 6 6 4 4
12786 218/300 Read block 4061 len 20480
12787 219/300 Read block 741 len 8192
12788 220/300 Flush
12789 221/300 Read block 4074 len 32768
12790 222/300 Read block 4674 len 32768
12791 223/300 Write block 4454 len 8192 data: 5 6
12792 224/300 Read block 4330 len 4096
12793 225/300 Read block 4118 len 8192
12794 226/300 Read block 905 len 28672
12795 227/300 Write block 1416 len 40960 data: 3 3 3 3 3 3 4 3 3 3
12796 228/300 Flush
12797 229/300 Write block 3314 len 36864 data: 3 4 5 5 5 3 3 3 3
12798 230/300 Write block 2906 len 32768 data: 2 2 2 2 2 2 2 2
12799 231/300 Flush
12800 232/300 Read block 3418 len 40960
12801 233/300 Write block 1829 len 20480 data: 4 5 5 4 4
12802 234/300 Read block 1534 len 40960
12803 235/300 Read block 4390 len 24576
12804 236/300 Read block 4668 len 40960
12805 237/300 Read block 2310 len 24576
12806 238/300 Read block 1120 len 8192
12807 239/300 Read block 4330 len 36864
12808 240/300 Read block 4183 len 8192
12809 241/300 Read block 3042 len 20480
12810 242/300 Read block 2131 len 24576
12811 243/300 Read block 4506 len 20480
12812 244/300 Read block 1033 len 12288
12813 245/300 Read block 4764 len 16384
12814 246/300 Write block 4486 len 40960 data: 2 2 2 2 2 2 3 3 2 2
12815 247/300 Read block 1473 len 8192
12816 248/300 Flush
12817 249/300 Write block 2229 len 40960 data: 4 4 4 3 4 4 5 5 4 2
12818 250/300 Flush
12819 251/300 Read block 1376 len 36864
12820 252/300 Read block 4181 len 32768
12821 253/300 Write block 4934 len 24576 data: 2 2 2 2 2 2
12822 254/300 Read block 3929 len 20480
12823 255/300 Flush
12824 256/300 Read block 621 len 4096
12825 257/300 Flush
12826 258/300 Read block 1605 len 8192
12827 259/300 Flush
12828 260/300 Flush
12829 261/300 Flush
12830 262/300 Read block 1056 len 8192
12831 263/300 Flush
12832 264/300 Read block 4573 len 24576
12833 265/300 Read block 3608 len 20480
12834 266/300 Read block 3614 len 4096
12835 267/300 Read block 241 len 16384
12836 268/300 Read block 996 len 32768
12837 269/300 Read block 4979 len 32768
12838 270/300 Read block 2301 len 20480
12839 271/300 Write block 204 len 36864 data: 4 3 3 3 4 4 5 6 5
12840 272/300 Read block 1109 len 12288
12841 273/300 Read block 1758 len 20480
12842 274/300 Write block 271 len 24576 data: 2 2 2 2 3 3
12843 275/300 Read block 3023 len 40960
12844 276/300 Flush
12845 277/300 Read block 2508 len 32768
12846 278/300 Write block 3172 len 20480 data: 4 5 5 5 5
12847 279/300 Write block 3502 len 40960 data: 5 5 4 5 6 6 6 5 5 4
12848 280/300 Read block 3058 len 36864
12849 281/300 Flush
12850 282/300 Write block 118 len 36864 data: 3 3 4 3 4 3 2 2 4
12851 283/300 Read block 218 len 28672
12852 284/300 Read block 3561 len 20480
12853 285/300 Read block 4499 len 16384
12854 286/300 Read block 914 len 40960
12855 287/300 Read block 1835 len 16384
12856 288/300 Read block 1807 len 4096
12857 289/300 Write block 3652 len 32768 data: 4 4 3 2 2 2 3 2
12858 290/300 Read block 3944 len 28672
12859 291/300 Read block 3669 len 32768
12860 292/300 Write block 1047 len 28672 data: 5 5 6 6 7 7 7
12861 293/300 Write block 991 len 32768 data: 3 3 2 2 2 3 3 3
12862 294/300 Read block 4809 len 12288
12863 295/300 Write block 1973 len 24576 data: 4 4 4 3 4 5
12864 296/300 Write block 1940 len 12288 data: 3 3 3
12865 297/300 Read block 1905 len 8192
12866 298/300 Write block 2128 len 20480 data: 4 4 3 2 2
12867 299/300 Read block 1518 len 20480
12868 300/300 Write block 4242 len 24576 data: 4 5 4 6 7 8
12869 301/300 Read block 3980 len 12288
12870 Main task has work to do, go find it
12871 got dsc Start(1)
12872 start 1
12873 Nov 28 04:53:02.162 INFO request completed, latency_us: 220, response_code: 204, uri: /start/cid/1, method: POST, req_id: ea017146-66ff-4007-b3f5-d27b03ebd53c, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
12874 Replay: started 1, returned:Ok(())
12875 ----------------------------------------------------------------
12876 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
12877 States: Active Offline Active
12878 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
12879 4622 Acked Flush 0 Done Sent Done false
12880 4623 Acked Write 10 Done Sent Done false
12881 4624 Acked Write 9 Done Sent Done false
12882 4625 Acked Write 3 Done Sent Done false
12883 4626 Acked Write 10 Done Sent Done false
12884 4627 Acked Flush 0 Done Sent Done false
12885 4628 Acked Flush 0 Done Sent Done false
12886 4629 Acked Write 9 Done Sent Done false
12887 4630 Acked Read 3 Done Sent Done false
12888 4631 Acked Write 5 Done Sent Done false
12889 4632 Acked Read 3 Done Sent Done false
12890 4633 Acked Read 4 Done Sent Done false
12891 4634 Acked Read 10 Done Sent Done false
12892 4635 Acked Read 7 Done Sent Done false
12893 4636 Acked Write 10 Done Sent Done false
12894 4637 Acked Read 4 Done Sent Done false
12895 4638 Acked Write 3 Done Sent Done false
12896 4639 Acked Write 5 Done Sent Done false
12897 4640 Acked Read 6 Done Sent Done false
12898 4641 Acked Read 1 Done Sent Done false
12899 4642 Acked Write 10 Done Sent Done false
12900 4643 Acked Write 7 Done Sent Done false
12901 4644 Acked Read 6 Done Sent Done false
12902 4645 Acked Read 7 Done Sent Done false
12903 4646 Acked Write 8 Done Sent Done false
12904 4647 Acked Read 5 Done Sent Done false
12905 4648 Acked Read 8 Done Sent Done false
12906 4649 Acked Write 7 Done Sent Done false
12907 4650 Acked Read 1 Done Sent Done false
12908 4651 Acked Read 10 Done Sent Done false
12909 4652 Acked Read 2 Done Sent Done false
12910 4653 Acked Read 2 Done Sent Done false
12911 4654 Acked Read 10 Done Sent Done false
12912 4655 Acked Write 1 Done Sent Done false
12913 4656 Acked Write 1 Done Sent Done false
12914 4657 Acked Read 8 Done Sent Done false
12915 4658 Acked Write 8 Done Sent Done false
12916 4659 Acked Write 9 Done Sent Done false
12917 4660 Acked Write 3 Done Sent Done false
12918 4661 Acked Write 7 Done Sent Done false
12919 4662 Acked Read 3 Done Sent Done false
12920 4663 Acked Read 2 Done Sent Done false
12921 4664 Acked Read 9 Done Sent Done false
12922 4665 Acked Read 7 Done Sent Done false
12923 4666 Acked Read 10 Done Sent Done false
12924 4667 Acked Read 6 Done Sent Done false
12925 4668 Acked Write 10 Done Sent Done false
12926 4669 Acked Write 3 Done Sent Done false
12927 4670 Acked Read 8 Done Sent Done false
12928 4671 Acked Read 4 Done Sent Done false
12929 4672 Acked Write 3 Done Sent Done false
12930 4673 Acked Write 5 Done Sent Done false
12931 4674 Acked Read 9 Done Sent Done false
12932 4675 Acked Write 4 Done Sent Done false
12933 4676 Acked Read 10 Done Sent Done false
12934 4677 Acked Read 5 Done Sent Done false
12935 4678 Acked Flush 0 Done Sent Done false
12936 4679 Acked Write 4 Done Sent Done false
12937 4680 Acked Write 2 Done Sent Done false
12938 4681 Acked Write 1 Done Sent Done false
12939 4682 Acked Write 2 Done Sent Done false
12940 4683 Acked Flush 0 Done Sent Done false
12941 4684 Acked Read 8 Done Sent Done false
12942 4685 Acked Write 9 Done Sent Done false
12943 4686 Acked Flush 0 Done Sent Done false
12944 4687 Acked Write 9 Done Sent Done false
12945 4688 Acked Read 7 Done Sent Done false
12946 4689 Acked Read 7 Done Sent Done false
12947 4690 Acked Write 6 Done Sent Done false
12948 4691 Acked Write 4 Done Sent Done false
12949 4692 Acked Read 8 Done Sent Done false
12950 4693 Acked Write 6 Done Sent Done false
12951 4694 Acked Write 8 Done Sent Done false
12952 4695 Acked Write 3 Done Sent Done false
12953 4696 Acked Read 3 Done Sent Done false
12954 4697 Acked Write 4 Done Sent Done false
12955 4698 Acked Write 6 Done Sent Done false
12956 4699 Acked Read 6 Done Sent Done false
12957 4700 Acked Write 3 Done Sent Done false
12958 4701 Acked Read 10 Done Sent Done false
12959 4702 Acked Write 4 Done Sent Done false
12960 4703 Acked Write 4 Done Sent Done false
12961 4704 Acked Write 1 Done Sent Done false
12962 4705 Acked Read 4 Done Sent Done false
12963 4706 Acked Read 2 Done Sent Done false
12964 4707 Acked Write 5 Done Sent Done false
12965 4708 Acked Read 10 Done Sent Done false
12966 4709 Acked Write 1 Done Sent Done false
12967 4710 Acked Read 5 Done Sent Done false
12968 4711 Acked Read 3 Done Sent Done false
12969 4712 Acked Write 7 Done Sent Done false
12970 4713 Acked Write 9 Done Sent Done false
12971 4714 Acked Read 8 Done Sent Done false
12972 4715 Acked Read 10 Done Sent Done false
12973 4716 Acked Write 10 Done Sent Done false
12974 4717 Acked Write 5 Done Sent Done false
12975 4718 Acked Read 4 Done Sent Done false
12976 4719 Acked Write 10 Done Sent Done false
12977 4720 Acked Read 8 Done Sent Done false
12978 4721 Acked Write 6 Done Sent Done false
12979 4722 Acked Read 6 Done Sent Done false
12980 4723 Acked Read 5 Done Sent Done false
12981 4724 Acked Read 9 Done Sent Done false
12982 4725 Acked Read 1 Done Sent Done false
12983 4726 Acked Read 5 Done Sent Done false
12984 4727 Acked Write 1 Done Sent Done false
12985 4728 Acked Write 10 Done Sent Done false
12986 4729 Acked Read 6 Done Sent Done false
12987 4730 Acked Read 9 Done Sent Done false
12988 4731 Acked Read 4 Done Sent Done false
12989 4732 Acked Read 7 Done Sent Done false
12990 4733 Acked Read 7 Done Sent Done false
12991 4734 Acked Read 1 Done Sent Done false
12992 4735 Acked Read 9 Done Sent Done false
12993 4736 Acked Write 3 Done Sent Done false
12994 4737 Acked Read 9 Done Sent Done false
12995 4738 Acked Write 3 Done Sent Done false
12996 4739 Acked Write 4 Done Sent Done false
12997 4740 Acked Read 6 Done Sent Done false
12998 4741 Acked Read 3 Done Sent Done false
12999 4742 Acked Flush 0 Done Sent Done false
13000 4743 Acked Flush 0 Done Sent Done false
13001 4744 Acked Write 8 Done Sent Done false
13002 4745 Acked Read 4 Done Sent Done false
13003 4746 Acked Write 5 Done Sent Done false
13004 4747 Acked Write 4 Done Sent Done false
13005 4748 Acked Read 10 Done Sent Done false
13006 4749 Acked Write 1 Done Sent Done false
13007 4750 Acked Read 9 Done Sent Done false
13008 4751 Acked Write 10 Done Sent Done false
13009 4752 Acked Read 7 Done Sent Done false
13010 4753 Acked Read 2 Done Sent Done false
13011 4754 Acked Write 1 Done Sent Done false
13012 4755 Acked Write 9 Done Sent Done false
13013 4756 Acked Read 5 Done Sent Done false
13014 4757 Acked Read 9 Done Sent Done false
13015 4758 Acked Read 4 Done Sent Done false
13016 4759 Acked Write 10 Done Sent Done false
13017 4760 Acked Flush 0 Done Sent Done false
13018 4761 Acked Read 10 Done Sent Done false
13019 4762 Acked Read 8 Done Sent Done false
13020 4763 Acked Write 7 Done Sent Done false
13021 4764 Acked Write 7 Done Sent Done false
13022 4765 Acked Read 1 Done Sent Done false
13023 4766 Acked Read 5 Done Sent Done false
13024 4767 Acked Write 8 Done Sent Done false
13025 4768 Acked Read 8 Done Sent Done false
13026 4769 Acked Write 8 Done Sent Done false
13027 4770 Acked Read 10 Done Sent Done false
13028 4771 Acked Write 6 Done Sent Done false
13029 4772 Acked Write 4 Done Sent Done false
13030 4773 Acked Flush 0 Done Sent Done false
13031 4774 Acked Read 8 Done Sent Done false
13032 4775 Acked Write 3 Done Sent Done false
13033 4776 Acked Write 3 Done Sent Done false
13034 4777 Acked Write 2 Done Sent Done false
13035 4778 Acked Flush 0 Done Sent Done false
13036 4779 Acked Flush 0 Done Sent Done false
13037 4780 Acked Write 8 Done Sent Done false
13038 4781 Acked Read 6 Done Sent Done false
13039 4782 Acked Read 2 Done Sent Done false
13040 4783 Acked Flush 0 Done Sent Done false
13041 4784 Acked Read 5 Done Sent Done false
13042 4785 Acked Read 1 Done Sent Done false
13043 4786 Acked Write 1 Done Sent Done false
13044 4787 Acked Read 10 Done Sent Done false
13045 4788 Acked Write 3 Done Sent Done false
13046 4789 Acked Write 4 Done Sent Done false
13047 4790 Acked Write 2 Done Sent Done false
13048 4791 Acked Write 1 Done Sent Done false
13049 4792 Acked Flush 0 Done Sent Done false
13050 4793 Acked Read 1 Done Sent Done false
13051 4794 Acked Read 10 Done Sent Done false
13052 4795 Acked Read 10 Done Sent Done false
13053 4796 Acked Read 1 Done Sent Done false
13054 4797 Acked Write 8 Done Sent Done false
13055 4798 Acked Flush 0 Done Sent Done false
13056 4799 Acked Write 4 Done Sent Done false
13057 4800 Acked Read 7 Done Sent Done false
13058 4801 Acked Read 7 Done Sent Done false
13059 4802 Acked Flush 0 Done Sent Done false
13060 4803 Acked Read 6 Done Sent Done false
13061 4804 Acked Write 1 Done Sent Done false
13062 4805 Acked Flush 0 Done Sent Done false
13063 4806 Acked Read 5 Done Sent Done false
13064 4807 Acked Write 1 Done Sent Done false
13065 4808 Acked Read 3 Done Sent Done false
13066 4809 Acked Flush 0 Done Sent Done false
13067 4810 Acked Write 9 Done Sent Done false
13068 4811 Acked Read 4 Done Sent Done false
13069 4812 Acked Write 10 Done Sent Done false
13070 4813 Acked Write 2 Done Sent Done false
13071 4814 Acked Read 5 Done Sent Done false
13072 4815 Acked Write 10 Done Sent Done false
13073 4816 Acked Write 8 Done Sent Done false
13074 4817 Acked Write 2 Done Sent Done false
13075 4818 Acked Write 2 Done Sent Done false
13076 4819 Acked Read 3 Done Sent Done false
13077 4820 Acked Read 3 Done Sent Done false
13078 4821 Acked Read 5 Done Sent Done false
13079 4822 Acked Read 10 Done Sent Done false
13080 4823 Acked Write 1 Done Sent Done false
13081 4824 Acked Write 2 Done Sent Done false
13082 4825 Acked Write 10 Done Sent Done false
13083 4826 Acked Write 8 Done Sent Done false
13084 4827 Acked Read 6 Done Sent Done false
13085 4828 Acked Read 3 Done Sent Done false
13086 4829 Acked Write 6 Done Sent Done false
13087 4830 Acked Write 2 Done Sent Done false
13088 4831 Acked Read 8 Done Sent Done false
13089 4832 Acked Write 6 Done Sent Done false
13090 4833 Acked Write 1 Done Sent Done false
13091 4834 Acked Write 5 Done Sent Done false
13092 4835 Acked Read 4 Done Sent Done false
13093 4836 Acked Write 7 Done Sent Done false
13094 4837 Acked Write 4 Done Sent Done false
13095 4838 Acked Write 10 Done Sent Done false
13096 4839 Acked Read 5 Done Sent Done false
13097 4840 Acked Read 2 Done Sent Done false
13098 4841 Acked Flush 0 Done Sent Done false
13099 4842 Acked Read 8 Done Sent Done false
13100 4843 Acked Read 8 Done Sent Done false
13101 4844 Acked Write 2 Done Sent Done false
13102 4845 Acked Read 1 Done Sent Done false
13103 4846 Acked Read 2 Done Sent Done false
13104 4847 Acked Read 7 Done Sent Done false
13105 4848 Acked Write 10 Done Sent Done false
13106 4849 Acked Flush 0 Done Sent Done false
13107 4850 Acked Write 9 Done Sent Done false
13108 4851 Acked Write 8 Done Sent Done false
13109 4852 Acked Flush 0 Done Sent Done false
13110 4853 Acked Read 10 Done Sent Done false
13111 4854 Acked Write 5 Done Sent Done false
13112 4855 Acked Read 10 Done Sent Done false
13113 4856 Acked Read 6 Done Sent Done false
13114 4857 Acked Read 10 Done Sent Done false
13115 4858 Acked Read 6 Done Sent Done false
13116 4859 Acked Read 2 Done Sent Done false
13117 4860 Acked Read 9 Done Sent Done false
13118 4861 Acked Read 2 Done Sent Done false
13119 4862 Acked Read 5 Done Sent Done false
13120 4863 Acked Read 6 Done Sent Done false
13121 4864 Acked Read 5 Done Sent Done false
13122 4865 Acked Read 3 Done Sent Done false
13123 4866 Acked Read 4 Done Sent Done false
13124 4867 Acked Write 10 Done Sent Done false
13125 4868 Acked Read 2 Done Sent Done false
13126 4869 Acked Flush 0 Done Sent Done false
13127 4870 Acked Write 10 Done Sent Done false
13128 4871 Acked Flush 0 Done Sent Done false
13129 4872 Acked Read 9 Done Sent Done false
13130 4873 Acked Read 8 Done Sent Done false
13131 4874 Acked Write 6 Done Sent Done false
13132 4875 Acked Read 5 Done Sent Done false
13133 4876 Acked Flush 0 Done Sent Done false
13134 4877 Acked Read 1 Done Sent Done false
13135 4878 Acked Flush 0 Done Sent Done false
13136 4879 Acked Read 2 Done Sent Done false
13137 4880 Acked Flush 0 Done Sent Done false
13138 4881 Acked Flush 0 Done Sent Done false
13139 4882 Acked Flush 0 Done Sent Done false
13140 4883 Acked Read 2 Done Sent Done false
13141 4884 Acked Flush 0 Done Sent Done false
13142 4885 Acked Read 6 Done Sent Done false
13143 4886 Acked Read 5 Done Sent Done false
13144 4887 Acked Read 1 Done Sent Done false
13145 4888 Acked Read 4 Done Sent Done false
13146 4889 Acked Read 8 Done Sent Done false
13147 4890 Acked Read 8 Done Sent Done false
13148 4891 Acked Read 5 Done Sent Done false
13149 4892 Acked Write 9 Done Sent Done false
13150 4893 Acked Read 3 Done Sent Done false
13151 4894 Acked Read 5 Done Sent Done false
13152 4895 Acked Write 6 Done Sent Done false
13153 4896 Acked Read 10 Done Sent Done false
13154 4897 Acked Flush 0 Done Sent Done false
13155 4898 Acked Read 8 Done Sent Done false
13156 4899 Acked Write 5 Done Sent Done false
13157 4900 Acked Write 10 Done Sent Done false
13158 4901 Acked Read 9 Done Sent Done false
13159 4902 Acked Flush 0 Done Sent Done false
13160 4903 Acked Write 9 Done Sent Done false
13161 4904 Acked Read 7 Done Sent Done false
13162 4905 Acked Read 5 Done Sent Done false
13163 4906 Acked Read 4 Done Sent Done false
13164 4907 Acked Read 10 Done Sent Done false
13165 4908 Acked Read 4 Done Sent Done false
13166 4909 Acked Read 1 Done Sent Done false
13167 4910 Acked Write 8 Done Sent Done false
13168 4911 Acked Read 7 Done Sent Done false
13169 4912 Acked Read 8 Done Sent Done false
13170 4913 Acked Write 7 Done Sent Done false
13171 4914 Acked Write 8 Done Sent Done false
13172 4915 Acked Read 3 Done Sent Done false
13173 4916 Acked Write 6 Done Sent Done false
13174 4917 Acked Write 3 Done Sent Done false
13175 4918 Acked Read 2 Done Sent Done false
13176 4919 Acked Write 5 Done Sent Done false
13177 4920 Acked Read 5 Done Sent Done false
13178 4921 Acked Write 6 Done Sent Done false
13179 4922 Acked Read 3 Done Sent Done false
13180 STATES DS:0 DS:1 DS:2 TOTAL
13181 Sent 0 301 0 301
13182 Done 301 0 301 602
13183 Skipped 0 0 0 0
13184 Error 0 0 0 0
13185 Last Flush: 4902 4621 4902
13186 Downstairs last five completed: 4621 4620 4619 4618 4617
13187 Upstairs last five completed: 4922 4921 4920 4919 4918
13188 CLIENT: Up:0 ds:301 act:2
13189 [8820] got start action
13190 [8820] I am going to restart
13191 Starting downstairs at port 8820
13192 Make output file at "/tmp/dsc/downstairs-8820.txt"
13193 [8820][1] reports Starting
13194 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1215)
13195 [8820][1] reports Running
13196 ----------------------------------------------------------------
13197 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
13198 States: Active Offline Active
13199 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
13200 4622 Acked Flush 0 Done Sent Done false
13201 4623 Acked Write 10 Done Sent Done false
13202 4624 Acked Write 9 Done Sent Done false
13203 4625 Acked Write 3 Done Sent Done false
13204 4626 Acked Write 10 Done Sent Done false
13205 4627 Acked Flush 0 Done Sent Done false
13206 4628 Acked Flush 0 Done Sent Done false
13207 4629 Acked Write 9 Done Sent Done false
13208 4630 Acked Read 3 Done Sent Done false
13209 4631 Acked Write 5 Done Sent Done false
13210 4632 Acked Read 3 Done Sent Done false
13211 4633 Acked Read 4 Done Sent Done false
13212 4634 Acked Read 10 Done Sent Done false
13213 4635 Acked Read 7 Done Sent Done false
13214 4636 Acked Write 10 Done Sent Done false
13215 4637 Acked Read 4 Done Sent Done false
13216 4638 Acked Write 3 Done Sent Done false
13217 4639 Acked Write 5 Done Sent Done false
13218 4640 Acked Read 6 Done Sent Done false
13219 4641 Acked Read 1 Done Sent Done false
13220 4642 Acked Write 10 Done Sent Done false
13221 4643 Acked Write 7 Done Sent Done false
13222 4644 Acked Read 6 Done Sent Done false
13223 4645 Acked Read 7 Done Sent Done false
13224 4646 Acked Write 8 Done Sent Done false
13225 4647 Acked Read 5 Done Sent Done false
13226 4648 Acked Read 8 Done Sent Done false
13227 4649 Acked Write 7 Done Sent Done false
13228 4650 Acked Read 1 Done Sent Done false
13229 4651 Acked Read 10 Done Sent Done false
13230 4652 Acked Read 2 Done Sent Done false
13231 4653 Acked Read 2 Done Sent Done false
13232 4654 Acked Read 10 Done Sent Done false
13233 4655 Acked Write 1 Done Sent Done false
13234 4656 Acked Write 1 Done Sent Done false
13235 4657 Acked Read 8 Done Sent Done false
13236 4658 Acked Write 8 Done Sent Done false
13237 4659 Acked Write 9 Done Sent Done false
13238 4660 Acked Write 3 Done Sent Done false
13239 4661 Acked Write 7 Done Sent Done false
13240 4662 Acked Read 3 Done Sent Done false
13241 4663 Acked Read 2 Done Sent Done false
13242 4664 Acked Read 9 Done Sent Done false
13243 4665 Acked Read 7 Done Sent Done false
13244 4666 Acked Read 10 Done Sent Done false
13245 4667 Acked Read 6 Done Sent Done false
13246 4668 Acked Write 10 Done Sent Done false
13247 4669 Acked Write 3 Done Sent Done false
13248 4670 Acked Read 8 Done Sent Done false
13249 4671 Acked Read 4 Done Sent Done false
13250 4672 Acked Write 3 Done Sent Done false
13251 4673 Acked Write 5 Done Sent Done false
13252 4674 Acked Read 9 Done Sent Done false
13253 4675 Acked Write 4 Done Sent Done false
13254 4676 Acked Read 10 Done Sent Done false
13255 4677 Acked Read 5 Done Sent Done false
13256 4678 Acked Flush 0 Done Sent Done false
13257 4679 Acked Write 4 Done Sent Done false
13258 4680 Acked Write 2 Done Sent Done false
13259 4681 Acked Write 1 Done Sent Done false
13260 4682 Acked Write 2 Done Sent Done false
13261 4683 Acked Flush 0 Done Sent Done false
13262 4684 Acked Read 8 Done Sent Done false
13263 4685 Acked Write 9 Done Sent Done false
13264 4686 Acked Flush 0 Done Sent Done false
13265 4687 Acked Write 9 Done Sent Done false
13266 4688 Acked Read 7 Done Sent Done false
13267 4689 Acked Read 7 Done Sent Done false
13268 4690 Acked Write 6 Done Sent Done false
13269 4691 Acked Write 4 Done Sent Done false
13270 4692 Acked Read 8 Done Sent Done false
13271 4693 Acked Write 6 Done Sent Done false
13272 4694 Acked Write 8 Done Sent Done false
13273 4695 Acked Write 3 Done Sent Done false
13274 4696 Acked Read 3 Done Sent Done false
13275 4697 Acked Write 4 Done Sent Done false
13276 4698 Acked Write 6 Done Sent Done false
13277 4699 Acked Read 6 Done Sent Done false
13278 4700 Acked Write 3 Done Sent Done false
13279 4701 Acked Read 10 Done Sent Done false
13280 4702 Acked Write 4 Done Sent Done false
13281 4703 Acked Write 4 Done Sent Done false
13282 4704 Acked Write 1 Done Sent Done false
13283 4705 Acked Read 4 Done Sent Done false
13284 4706 Acked Read 2 Done Sent Done false
13285 4707 Acked Write 5 Done Sent Done false
13286 4708 Acked Read 10 Done Sent Done false
13287 4709 Acked Write 1 Done Sent Done false
13288 4710 Acked Read 5 Done Sent Done false
13289 4711 Acked Read 3 Done Sent Done false
13290 4712 Acked Write 7 Done Sent Done false
13291 4713 Acked Write 9 Done Sent Done false
13292 4714 Acked Read 8 Done Sent Done false
13293 4715 Acked Read 10 Done Sent Done false
13294 4716 Acked Write 10 Done Sent Done false
13295 4717 Acked Write 5 Done Sent Done false
13296 4718 Acked Read 4 Done Sent Done false
13297 4719 Acked Write 10 Done Sent Done false
13298 4720 Acked Read 8 Done Sent Done false
13299 4721 Acked Write 6 Done Sent Done false
13300 4722 Acked Read 6 Done Sent Done false
13301 4723 Acked Read 5 Done Sent Done false
13302 4724 Acked Read 9 Done Sent Done false
13303 4725 Acked Read 1 Done Sent Done false
13304 4726 Acked Read 5 Done Sent Done false
13305 4727 Acked Write 1 Done Sent Done false
13306 4728 Acked Write 10 Done Sent Done false
13307 4729 Acked Read 6 Done Sent Done false
13308 4730 Acked Read 9 Done Sent Done false
13309 4731 Acked Read 4 Done Sent Done false
13310 4732 Acked Read 7 Done Sent Done false
13311 4733 Acked Read 7 Done Sent Done false
13312 4734 Acked Read 1 Done Sent Done false
13313 4735 Acked Read 9 Done Sent Done false
13314 4736 Acked Write 3 Done Sent Done false
13315 4737 Acked Read 9 Done Sent Done false
13316 4738 Acked Write 3 Done Sent Done false
13317 4739 Acked Write 4 Done Sent Done false
13318 4740 Acked Read 6 Done Sent Done false
13319 4741 Acked Read 3 Done Sent Done false
13320 4742 Acked Flush 0 Done Sent Done false
13321 4743 Acked Flush 0 Done Sent Done false
13322 4744 Acked Write 8 Done Sent Done false
13323 4745 Acked Read 4 Done Sent Done false
13324 4746 Acked Write 5 Done Sent Done false
13325 4747 Acked Write 4 Done Sent Done false
13326 4748 Acked Read 10 Done Sent Done false
13327 4749 Acked Write 1 Done Sent Done false
13328 4750 Acked Read 9 Done Sent Done false
13329 4751 Acked Write 10 Done Sent Done false
13330 4752 Acked Read 7 Done Sent Done false
13331 4753 Acked Read 2 Done Sent Done false
13332 4754 Acked Write 1 Done Sent Done false
13333 4755 Acked Write 9 Done Sent Done false
13334 4756 Acked Read 5 Done Sent Done false
13335 4757 Acked Read 9 Done Sent Done false
13336 4758 Acked Read 4 Done Sent Done false
13337 4759 Acked Write 10 Done Sent Done false
13338 4760 Acked Flush 0 Done Sent Done false
13339 4761 Acked Read 10 Done Sent Done false
13340 4762 Acked Read 8 Done Sent Done false
13341 4763 Acked Write 7 Done Sent Done false
13342 4764 Acked Write 7 Done Sent Done false
13343 4765 Acked Read 1 Done Sent Done false
13344 4766 Acked Read 5 Done Sent Done false
13345 4767 Acked Write 8 Done Sent Done false
13346 4768 Acked Read 8 Done Sent Done false
13347 4769 Acked Write 8 Done Sent Done false
13348 4770 Acked Read 10 Done Sent Done false
13349 4771 Acked Write 6 Done Sent Done false
13350 4772 Acked Write 4 Done Sent Done false
13351 4773 Acked Flush 0 Done Sent Done false
13352 4774 Acked Read 8 Done Sent Done false
13353 4775 Acked Write 3 Done Sent Done false
13354 4776 Acked Write 3 Done Sent Done false
13355 4777 Acked Write 2 Done Sent Done false
13356 4778 Acked Flush 0 Done Sent Done false
13357 4779 Acked Flush 0 Done Sent Done false
13358 4780 Acked Write 8 Done Sent Done false
13359 4781 Acked Read 6 Done Sent Done false
13360 4782 Acked Read 2 Done Sent Done false
13361 4783 Acked Flush 0 Done Sent Done false
13362 4784 Acked Read 5 Done Sent Done false
13363 4785 Acked Read 1 Done Sent Done false
13364 4786 Acked Write 1 Done Sent Done false
13365 4787 Acked Read 10 Done Sent Done false
13366 4788 Acked Write 3 Done Sent Done false
13367 4789 Acked Write 4 Done Sent Done false
13368 4790 Acked Write 2 Done Sent Done false
13369 4791 Acked Write 1 Done Sent Done false
13370 4792 Acked Flush 0 Done Sent Done false
13371 4793 Acked Read 1 Done Sent Done false
13372 4794 Acked Read 10 Done Sent Done false
13373 4795 Acked Read 10 Done Sent Done false
13374 4796 Acked Read 1 Done Sent Done false
13375 4797 Acked Write 8 Done Sent Done false
13376 4798 Acked Flush 0 Done Sent Done false
13377 4799 Acked Write 4 Done Sent Done false
13378 4800 Acked Read 7 Done Sent Done false
13379 4801 Acked Read 7 Done Sent Done false
13380 4802 Acked Flush 0 Done Sent Done false
13381 4803 Acked Read 6 Done Sent Done false
13382 4804 Acked Write 1 Done Sent Done false
13383 4805 Acked Flush 0 Done Sent Done false
13384 4806 Acked Read 5 Done Sent Done false
13385 4807 Acked Write 1 Done Sent Done false
13386 4808 Acked Read 3 Done Sent Done false
13387 4809 Acked Flush 0 Done Sent Done false
13388 4810 Acked Write 9 Done Sent Done false
13389 4811 Acked Read 4 Done Sent Done false
13390 4812 Acked Write 10 Done Sent Done false
13391 4813 Acked Write 2 Done Sent Done false
13392 4814 Acked Read 5 Done Sent Done false
13393 4815 Acked Write 10 Done Sent Done false
13394 4816 Acked Write 8 Done Sent Done false
13395 4817 Acked Write 2 Done Sent Done false
13396 4818 Acked Write 2 Done Sent Done false
13397 4819 Acked Read 3 Done Sent Done false
13398 4820 Acked Read 3 Done Sent Done false
13399 4821 Acked Read 5 Done Sent Done false
13400 4822 Acked Read 10 Done Sent Done false
13401 4823 Acked Write 1 Done Sent Done false
13402 4824 Acked Write 2 Done Sent Done false
13403 4825 Acked Write 10 Done Sent Done false
13404 4826 Acked Write 8 Done Sent Done false
13405 4827 Acked Read 6 Done Sent Done false
13406 4828 Acked Read 3 Done Sent Done false
13407 4829 Acked Write 6 Done Sent Done false
13408 4830 Acked Write 2 Done Sent Done false
13409 4831 Acked Read 8 Done Sent Done false
13410 4832 Acked Write 6 Done Sent Done false
13411 4833 Acked Write 1 Done Sent Done false
13412 4834 Acked Write 5 Done Sent Done false
13413 4835 Acked Read 4 Done Sent Done false
13414 4836 Acked Write 7 Done Sent Done false
13415 4837 Acked Write 4 Done Sent Done false
13416 4838 Acked Write 10 Done Sent Done false
13417 4839 Acked Read 5 Done Sent Done false
13418 4840 Acked Read 2 Done Sent Done false
13419 4841 Acked Flush 0 Done Sent Done false
13420 4842 Acked Read 8 Done Sent Done false
13421 4843 Acked Read 8 Done Sent Done false
13422 4844 Acked Write 2 Done Sent Done false
13423 4845 Acked Read 1 Done Sent Done false
13424 4846 Acked Read 2 Done Sent Done false
13425 4847 Acked Read 7 Done Sent Done false
13426 4848 Acked Write 10 Done Sent Done false
13427 4849 Acked Flush 0 Done Sent Done false
13428 4850 Acked Write 9 Done Sent Done false
13429 4851 Acked Write 8 Done Sent Done false
13430 4852 Acked Flush 0 Done Sent Done false
13431 4853 Acked Read 10 Done Sent Done false
13432 4854 Acked Write 5 Done Sent Done false
13433 4855 Acked Read 10 Done Sent Done false
13434 4856 Acked Read 6 Done Sent Done false
13435 4857 Acked Read 10 Done Sent Done false
13436 4858 Acked Read 6 Done Sent Done false
13437 4859 Acked Read 2 Done Sent Done false
13438 4860 Acked Read 9 Done Sent Done false
13439 4861 Acked Read 2 Done Sent Done false
13440 4862 Acked Read 5 Done Sent Done false
13441 4863 Acked Read 6 Done Sent Done false
13442 4864 Acked Read 5 Done Sent Done false
13443 4865 Acked Read 3 Done Sent Done false
13444 4866 Acked Read 4 Done Sent Done false
13445 4867 Acked Write 10 Done Sent Done false
13446 4868 Acked Read 2 Done Sent Done false
13447 4869 Acked Flush 0 Done Sent Done false
13448 4870 Acked Write 10 Done Sent Done false
13449 4871 Acked Flush 0 Done Sent Done false
13450 4872 Acked Read 9 Done Sent Done false
13451 4873 Acked Read 8 Done Sent Done false
13452 4874 Acked Write 6 Done Sent Done false
13453 4875 Acked Read 5 Done Sent Done false
13454 4876 Acked Flush 0 Done Sent Done false
13455 4877 Acked Read 1 Done Sent Done false
13456 4878 Acked Flush 0 Done Sent Done false
13457 4879 Acked Read 2 Done Sent Done false
13458 4880 Acked Flush 0 Done Sent Done false
13459 4881 Acked Flush 0 Done Sent Done false
13460 4882 Acked Flush 0 Done Sent Done false
13461 4883 Acked Read 2 Done Sent Done false
13462 4884 Acked Flush 0 Done Sent Done false
13463 4885 Acked Read 6 Done Sent Done false
13464 4886 Acked Read 5 Done Sent Done false
13465 4887 Acked Read 1 Done Sent Done false
13466 4888 Acked Read 4 Done Sent Done false
13467 4889 Acked Read 8 Done Sent Done false
13468 4890 Acked Read 8 Done Sent Done false
13469 4891 Acked Read 5 Done Sent Done false
13470 4892 Acked Write 9 Done Sent Done false
13471 4893 Acked Read 3 Done Sent Done false
13472 4894 Acked Read 5 Done Sent Done false
13473 4895 Acked Write 6 Done Sent Done false
13474 4896 Acked Read 10 Done Sent Done false
13475 4897 Acked Flush 0 Done Sent Done false
13476 4898 Acked Read 8 Done Sent Done false
13477 4899 Acked Write 5 Done Sent Done false
13478 4900 Acked Write 10 Done Sent Done false
13479 4901 Acked Read 9 Done Sent Done false
13480 4902 Acked Flush 0 Done Sent Done false
13481 4903 Acked Write 9 Done Sent Done false
13482 4904 Acked Read 7 Done Sent Done false
13483 4905 Acked Read 5 Done Sent Done false
13484 4906 Acked Read 4 Done Sent Done false
13485 4907 Acked Read 10 Done Sent Done false
13486 4908 Acked Read 4 Done Sent Done false
13487 4909 Acked Read 1 Done Sent Done false
13488 4910 Acked Write 8 Done Sent Done false
13489 4911 Acked Read 7 Done Sent Done false
13490 4912 Acked Read 8 Done Sent Done false
13491 4913 Acked Write 7 Done Sent Done false
13492 4914 Acked Write 8 Done Sent Done false
13493 4915 Acked Read 3 Done Sent Done false
13494 4916 Acked Write 6 Done Sent Done false
13495 4917 Acked Write 3 Done Sent Done false
13496 4918 Acked Read 2 Done Sent Done false
13497 4919 Acked Write 5 Done Sent Done false
13498 4920 Acked Read 5 Done Sent Done false
13499 4921 Acked Write 6 Done Sent Done false
13500 4922 Acked Read 3 Done Sent Done false
13501 4923 Acked Flush 0 Done Sent Done false
13502 STATES DS:0 DS:1 DS:2 TOTAL
13503 Sent 0 302 0 302
13504 Done 302 0 302 604
13505 Skipped 0 0 0 0
13506 Error 0 0 0 0
13507 Last Flush: 4923 4621 4923
13508 Downstairs last five completed: 4621 4620 4619 4618 4617
13509 Upstairs last five completed: 4923 4922 4921 4920 4919
13510 CLIENT: Up:0 ds:302 act:2
135112024-11-28T04:53:07.920ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
135122024-11-28T04:53:07.920ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:36792) = io task client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
135132024-11-28T04:53:07.921ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
135142024-11-28T04:53:07.922ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
135152024-11-28T04:53:07.922ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
135162024-11-28T04:53:07.922ZINFOcrucible: send last flush ID to this DS: 4621 = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
135172024-11-28T04:53:07.922ZINFOcrucible: Replied this last flush ID: 4621 = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
135182024-11-28T04:53:07.922ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
135192024-11-28T04:53:07.922ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
135202024-11-28T04:53:07.922ZINFOcrucible: [1] client re-new 302 jobs since flush 4621 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
135212024-11-28T04:53:07.922ZINFOcrucible: [1] Replayed 302 jobs since flush: 4621 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
13522 ----------------------------------------------------------------
13523 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
13524 Downstairs last five completed: 4923 4922 4921 4920 4919
13525 Upstairs last five completed: 4923 4922 4921 4920 4919
13526 CLIENT: Up:0 ds:0 act:3
13527 Replay: All jobs finished
13528 Main task has work to do, go find it
13529 got dsc Stop(2)
13530 stop 2
13531 [8830] Got stop action so:false kr:false
13532 Nov 28 04:53:10.168 INFO request completed, latency_us: 323, response_code: 204, uri: /stop/cid/2, method: POST, req_id: f8b8a045-b70a-4f21-bf7d-52ffe09b5a7e, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
135332024-11-28T04:53:10.168ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
135342024-11-28T04:53:10.168ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
135352024-11-28T04:53:10.168ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
13536 Nov 28 04:53:10.168 INFO request completed, latency_us: 123, response_code: 200, uri: /state/cid/2, method: GET, req_id: 9682bdc3-a86a-424e-ae4a-980acb807b22, remote_addr: 127.0.0.1{"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-28T04:53:10.168985174Z","hostname":"w-01JDRHKEEFJ1228TFPSBEGXWHY","pid":1202,"client":"2","":"downstairs","session_id":"4a2415b1-6594-436f-82e5-b8cf9a74eeaf"}
13537 :60418, local_addr: 127.0.0.1:9998
135382024-11-28T04:53:10.169ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
135392024-11-28T04:53:10.169ZINFOcrucible: client task is exiting = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
13540 [8830] Exited with: ExitStatus(unix_wait_status(9))
13541 [8830][2] reports Exit
13542 Nov 28 04:53:14.170 INFO request completed, latency_us: 213, response_code: 200, uri: /state/cid/2, method: GET, req_id: 4769b649-4d48-483f-9921-7fa8ac72b273, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
13543 001/300 Read block 669 len 40960
13544 002/300 Flush
13545 003/300 Read block 4989 len 12288
13546 004/300 Flush
13547 005/300 Flush
13548 006/300 Read block 1122 len 4096
13549 007/300 Read block 145 len 28672
13550 008/300 Write block 3064 len 20480 data: 4 4 3 2 2
13551 009/300 Read block 2213 len 24576
13552 010/300 Write block 2289 len 28672 data: 3 3 3 3 2 2 2
13553 011/300 Write block 1996 len 8192 data: 2 3
13554 012/300 Write block 1632 len 8192 data: 4 3
13555 013/300 Flush
13556 014/300 Write block 103 len 16384 data: 4 4 4 4
13557 015/300 Write block 4655 len 24576 data: 4 4 4 5 5 4
13558 016/300 Read block 2331 len 32768
13559 017/300 Read block 3915 len 24576
13560 018/300 Write block 1656 len 4096 data: 2
13561 019/300 Read block 1510 len 4096
13562 020/300 Read block 1033 len 20480
13563 021/300 Read block 4982 len 12288
13564 022/300 Write block 4445 len 8192 data: 7 9
13565 023/300 Write block 1933 len 28672 data: 3 3 3 3 3 3 3
13566 024/300 Write block 3572 len 20480 data: 4 3 2 2 2
13567 025/300 Read block 3590 len 16384
13568 026/300 Write block 3925 len 28672 data: 6 4 4 4 4 4 3
13569 027/300 Write block 3424 len 24576 data: 4 4 5 6 6 6
13570 028/300 Read block 1113 len 12288
13571 029/300 Write block 4084 len 40960 data: 6 4 4 5 5 5 6 6 6 6
13572 030/300 Flush
13573 031/300 Read block 1413 len 8192
13574 032/300 Write block 3804 len 32768 data: 3 3 4 4 3 2 3 3
13575 033/300 Write block 3152 len 24576 data: 4 4 5 4 4 4
13576 034/300 Write block 831 len 4096 data: 5
13577 035/300 Write block 3641 len 28672 data: 4 4 4 5 5 4 4
13578 036/300 Read block 3645 len 20480
13579 037/300 Read block 1982 len 4096
13580 038/300 Write block 3915 len 40960 data: 2 2 3 2 2 2 2 4 5 4
13581 039/300 Read block 2567 len 8192
13582 040/300 Read block 1589 len 20480
13583 041/300 Read block 4942 len 40960
13584 042/300 Write block 2963 len 24576 data: 6 5 6 5 4 3
13585 043/300 Read block 3829 len 36864
13586 044/300 Write block 3598 len 16384 data: 3 3 3 3
13587 045/300 Write block 3600 len 8192 data: 4 4
13588 046/300 Read block 3622 len 40960
13589 047/300 Write block 2170 len 16384 data: 5 4 5 5
13590 048/300 Write block 1285 len 20480 data: 4 3 3 3 3
13591 049/300 Read block 4627 len 8192
13592 050/300 Read block 4441 len 20480
13593 051/300 Write block 3819 len 24576 data: 5 5 5 5 5 5
13594 052/300 Flush
13595 053/300 Read block 182 len 28672
13596 054/300 Read block 4134 len 36864
13597 055/300 Read block 2967 len 12288
13598 056/300 Write block 999 len 40960 data: 3 3 3 2 2 2 2 2 2 2
13599 057/300 Flush
13600 058/300 Read block 1285 len 36864
13601 059/300 Flush
13602 060/300 Read block 420 len 36864
13603 061/300 Read block 3515 len 8192
13604 062/300 Read block 3123 len 28672
13605 063/300 Read block 4835 len 28672
13606 064/300 Write block 4753 len 20480 data: 3 3 3 2 2
13607 065/300 Write block 1954 len 32768 data: 3 3 3 4 5 5 5 5
13608 066/300 Write block 278 len 36864 data: 3 3 3 3 3 3 2 2 2
13609 067/300 Read block 2566 len 36864
13610 068/300 Read block 3138 len 20480
13611 069/300 Write block 3055 len 32768 data: 4 3 4 5 5 5 5 4
13612 070/300 Write block 1206 len 24576 data: 3 3 3 3 4 3
13613 071/300 Write block 1620 len 40960 data: 2 2 2 3 3 3 4 4 4 5
13614 072/300 Read block 2762 len 4096
13615 073/300 Read block 63 len 40960
13616 074/300 Read block 1741 len 16384
13617 075/300 Write block 2346 len 12288 data: 2 2 3
13618 076/300 Read block 3169 len 32768
13619 077/300 Write block 1858 len 36864 data: 4 3 3 3 2 2 2 2 2
13620 078/300 Read block 1003 len 24576
13621 079/300 Flush
13622 080/300 Write block 438 len 32768 data: 6 6 6 5 5 5 4 3
13623 081/300 Read block 2937 len 24576
13624 082/300 Flush
13625 083/300 Write block 4552 len 4096 data: 5
13626 084/300 Write block 1900 len 36864 data: 2 3 3 3 2 2 2 3 3
13627 085/300 Read block 182 len 24576
13628 086/300 Read block 4880 len 12288
13629 087/300 Read block 1302 len 12288
13630 088/300 Read block 825 len 32768
13631 089/300 Write block 2373 len 36864 data: 6 6 6 7 5 6 6 6 6
13632 090/300 Read block 4621 len 20480
13633 091/300 Write block 3688 len 40960 data: 4 3 3 3 3 5 5 5 5 5
13634 092/300 Read block 3845 len 36864
13635 093/300 Write block 69 len 40960 data: 3 2 2 2 2 2 2 2 2 2
13636 094/300 Read block 701 len 36864
13637 095/300 Read block 2690 len 20480
13638 096/300 Write block 1667 len 20480 data: 2 3 3 3 3
13639 097/300 Read block 1519 len 28672
13640 098/300 Write block 4091 len 32768 data: 7 7 7 5 5 6 6 5
13641 099/300 Write block 947 len 40960 data: 3 3 2 2 2 2 2 3 3 3
13642 100/300 Flush
13643 101/300 Write block 2154 len 16384 data: 2 2 2 4
13644 102/300 Write block 222 len 12288 data: 2 2 2
13645 103/300 Read block 3788 len 40960
13646 104/300 Read block 4761 len 24576
13647 105/300 Read block 4494 len 24576
13648 106/300 Flush
13649 107/300 Flush
13650 108/300 Read block 3395 len 16384
13651 109/300 Write block 7 len 36864 data: 4 5 8 8 8 5 5 7 7
13652 110/300 Read block 750 len 20480
13653 111/300 Flush
13654 112/300 Read block 2556 len 24576
13655 113/300 Write block 835 len 4096 data: 3
13656 114/300 Write block 311 len 32768 data: 5 5 5 4 3 3 4 5
13657 115/300 Read block 343 len 36864
13658 116/300 Write block 3207 len 24576 data: 5 4 5 4 4 3
13659 117/300 Read block 4048 len 40960
13660 118/300 Read block 3177 len 40960
13661 119/300 Read block 3176 len 32768
13662 120/300 Write block 25 len 24576 data: 5 4 3 3 3 5
13663 121/300 Flush
13664 122/300 Read block 2626 len 24576
13665 123/300 Write block 2373 len 16384 data: 7 7 7 8
13666 124/300 Flush
13667 125/300 Read block 1258 len 32768
13668 126/300 Write block 3887 len 12288 data: 4 4 4
13669 127/300 Read block 878 len 8192
13670 128/300 Write block 1861 len 12288 data: 4 3 3
13671 129/300 Read block 1419 len 20480
13672 130/300 Read block 4683 len 8192
13673 131/300 Write block 4766 len 28672 data: 2 2 4 4 4 4 3
13674 132/300 Read block 1947 len 20480
13675 133/300 Read block 4989 len 8192
13676 134/300 Read block 174 len 36864
13677 135/300 Flush
13678 136/300 Write block 3221 len 16384 data: 5 4 4 4
13679 137/300 Write block 4576 len 24576 data: 4 4 4 3 3 3
13680 138/300 Read block 1394 len 12288
13681 139/300 Read block 2013 len 32768
13682 140/300 Write block 4404 len 24576 data: 2 2 3 4 3 3
13683 141/300 Write block 468 len 20480 data: 4 5 5 5 6
13684 142/300 Read block 4693 len 8192
13685 143/300 Flush
13686 144/300 Flush
13687 145/300 Read block 254 len 40960
13688 146/300 Write block 859 len 4096 data: 5
13689 147/300 Write block 3780 len 40960 data: 4 3 3 3 2 2 2 2 3 4
13690 148/300 Read block 1068 len 8192
13691 149/300 Write block 2143 len 24576 data: 3 3 3 3 3 3
13692 150/300 Read block 1714 len 40960
13693 151/300 Write block 2598 len 8192 data: 3 4
13694 152/300 Write block 2994 len 16384 data: 2 2 2 2
13695 153/300 Read block 2209 len 16384
13696 154/300 Write block 3200 len 8192 data: 3 3
13697 155/300 Read block 1592 len 36864
13698 156/300 Write block 4237 len 8192 data: 4 3
13699 157/300 Read block 2131 len 20480
13700 158/300 Read block 1964 len 24576
13701 159/300 Write block 809 len 36864 data: 3 4 3 4 4 7 6 6 6
13702 160/300 Write block 3342 len 24576 data: 4 3 3 2 2 2
13703 161/300 Read block 450 len 36864
13704 162/300 Write block 4728 len 28672 data: 4 4 4 2 3 3 3
13705 163/300 Read block 3512 len 4096
13706 164/300 Read block 2063 len 8192
13707 165/300 Read block 2751 len 36864
13708 166/300 Read block 695 len 32768
13709 167/300 Read block 3793 len 24576
13710 168/300 Flush
13711 169/300 Read block 3465 len 24576
13712 170/300 Read block 3473 len 40960
13713 171/300 Read block 4186 len 32768
13714 172/300 Read block 4277 len 12288
13715 173/300 Write block 2632 len 16384 data: 3 2 2 2
13716 174/300 Read block 1198 len 20480
13717 175/300 Read block 4707 len 20480
13718 176/300 Write block 2158 len 36864 data: 4 4 4 4 4 4 2 2 3
13719 177/300 Flush
13720 178/300 Read block 4222 len 28672
13721 179/300 Write block 370 len 4096 data: 5
13722 180/300 Write block 2317 len 20480 data: 3 3 4 5 5
13723 181/300 Read block 1396 len 24576
13724 182/300 Read block 817 len 16384
13725 183/300 Write block 4572 len 24576 data: 3 3 4 4 5 5
13726 184/300 Read block 2604 len 24576
13727 185/300 Write block 116 len 24576 data: 3 3 4 4 5 4
13728 186/300 Write block 1946 len 32768 data: 4 3 3 3 3 3 3 4
13729 187/300 Flush
13730 188/300 Write block 4052 len 12288 data: 2 2 3
13731 189/300 Write block 4329 len 4096 data: 4
13732 190/300 Read block 3418 len 28672
13733 191/300 Read block 246 len 8192
13734 192/300 Write block 2429 len 4096 data: 2
13735 193/300 Read block 3349 len 8192
13736 194/300 Flush
13737 195/300 Flush
13738 196/300 Read block 4261 len 8192
13739 197/300 Write block 2735 len 28672 data: 3 3 3 3 2 2 3
13740 198/300 Write block 693 len 16384 data: 4 4 3 3
13741 199/300 Read block 2054 len 12288
13742 200/300 Write block 1674 len 8192 data: 3 3
13743 201/300 Read block 2273 len 28672
13744 202/300 Read block 73 len 36864
13745 203/300 Flush
13746 204/300 Write block 1190 len 12288 data: 3 2 2
13747 205/300 Write block 4773 len 40960 data: 4 5 4 3 3 3 3 3 3 3
13748 206/300 Read block 3560 len 40960
13749 207/300 Read block 2079 len 32768
13750 208/300 Read block 4835 len 20480
13751 209/300 Read block 3183 len 28672
13752 210/300 Read block 4483 len 40960
13753 211/300 Write block 2750 len 12288 data: 7 8 7
13754 212/300 Read block 4883 len 8192
13755 213/300 Write block 2333 len 24576 data: 5 5 3 4 5 5
13756 214/300 Flush
13757 215/300 Write block 4179 len 40960 data: 4 4 4 4 4 3 3 2 3 3
13758 216/300 Read block 288 len 36864
13759 217/300 Flush
13760 218/300 Write block 2632 len 4096 data: 4
13761 219/300 Write block 1819 len 12288 data: 3 4 3
13762 220/300 Read block 2962 len 40960
13763 221/300 Write block 238 len 12288 data: 3 3 3
13764 222/300 Write block 1060 len 40960 data: 2 2 3 3 3 2 2 3 3 3
13765 223/300 Read block 2639 len 16384
13766 224/300 Read block 941 len 8192
13767 225/300 Write block 4923 len 40960 data: 4 4 4 3 3 2 4 4 3 3
13768 226/300 Write block 890 len 28672 data: 3 4 4 2 3 3 3
13769 227/300 Read block 3588 len 36864
13770 228/300 Flush
13771 229/300 Read block 42 len 8192
13772 230/300 Write block 184 len 20480 data: 4 3 3 3 5
13773 231/300 Read block 1814 len 4096
13774 232/300 Write block 2874 len 36864 data: 3 3 4 5 5 5 5 5 4
13775 233/300 Write block 1261 len 20480 data: 5 5 5 5 6
13776 234/300 Write block 1892 len 36864 data: 6 6 5 4 3 3 3 3 3
13777 235/300 Write block 1465 len 28672 data: 4 4 5 5 4 3 3
13778 236/300 Write block 2334 len 28672 data: 6 4 5 6 6 4 4
13779 237/300 Write block 3338 len 4096 data: 3
13780 238/300 Read block 4227 len 12288
13781 239/300 Write block 3630 len 8192 data: 4 4
13782 240/300 Read block 4418 len 20480
13783 241/300 Read block 4666 len 20480
13784 242/300 Read block 894 len 24576
13785 243/300 Read block 647 len 28672
13786 244/300 Read block 3448 len 24576
13787 245/300 Read block 662 len 36864
13788 246/300 Read block 1400 len 4096
13789 247/300 Read block 3538 len 4096
13790 248/300 Read block 3666 len 16384
13791 249/300 Read block 2003 len 28672
13792 250/300 Write block 769 len 8192 data: 3 3
13793 251/300 Write block 2790 len 16384 data: 9 6 6 4
13794 252/300 Read block 3491 len 8192
13795 253/300 Write block 2719 len 40960 data: 6 5 5 2 2 2 2 2 2 3
13796 254/300 Read block 4709 len 28672
13797 255/300 Write block 3667 len 36864 data: 3 4 5 5 4 4 3 4 4
13798 256/300 Write block 1150 len 32768 data: 6 5 4 4 3 2 3 3
13799 257/300 Write block 1152 len 16384 data: 5 5 4 3
13800 258/300 Read block 2789 len 32768
13801 259/300 Flush
13802 260/300 Read block 3225 len 40960
13803 261/300 Read block 3117 len 36864
13804 262/300 Read block 3054 len 8192
13805 263/300 Write block 4491 len 16384 data: 3 4 4 3
13806 264/300 Write block 101 len 40960 data: 3 3 5 5 5 5 5 5 6 4
13807 265/300 Read block 3354 len 8192
13808 266/300 Read block 1921 len 40960
13809 267/300 Read block 3302 len 20480
13810 268/300 Read block 4105 len 32768
13811 269/300 Read block 4518 len 32768
13812 270/300 Write block 1983 len 8192 data: 4 4
13813 271/300 Flush
13814 272/300 Read block 1538 len 12288
13815 273/300 Read block 1868 len 16384
13816 274/300 Write block 3360 len 16384 data: 2 2 2 2
13817 275/300 Write block 221 len 12288 data: 3 3 3
13818 276/300 Read block 4983 len 8192
13819 277/300 Read block 935 len 32768
13820 278/300 Read block 4772 len 12288
13821 279/300 Read block 3620 len 28672
13822 280/300 Write block 3689 len 32768 data: 4 4 4 4 6 6 6 6
13823 281/300 Write block 4049 len 28672 data: 2 2 2 3 3 4 3
13824 282/300 Read block 2034 len 24576
13825 283/300 Read block 1641 len 12288
13826 284/300 Read block 4209 len 40960
13827 285/300 Read block 807 len 36864
13828 286/300 Write block 4228 len 16384 data: 4 3 3 3
13829 287/300 Write block 3239 len 4096 data: 5
13830 288/300 Read block 975 len 32768
13831 289/300 Write block 3849 len 28672 data: 3 3 3 3 3 4 4
13832 290/300 Write block 3811 len 12288 data: 4 4 4
13833 291/300 Write block 1218 len 16384 data: 4 5 6 6
13834 292/300 Read block 3674 len 40960
13835 293/300 Read block 106 len 28672
13836 294/300 Read block 1053 len 16384
13837 295/300 Write block 2685 len 32768 data: 4 4 4 4 4 4 2 2
13838 296/300 Flush
13839 297/300 Write block 1174 len 36864 data: 2 3 3 3 3 4 4 5 5
13840 298/300 Read block 4190 len 24576
13841 299/300 Write block 113 len 40960 data: 3 3 2 4 4 5 5 6 5 5
13842 300/300 Read block 3282 len 40960
13843 301/300 Read block 3469 len 8192
13844 Main task has work to do, go find it
13845 got dsc Start(2)
13846 start 2
13847 Nov 28 04:53:14.413 INFO request completed, latency_us: 239, response_code: 204, uri: /start/cid/2, method: POST, req_id: 6e0c9386-ebad-44e6-adba-dac7d7e4d653, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
13848 Replay: started 2, returned:Ok(())
13849 ----------------------------------------------------------------
13850 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
13851 States: Active Active Offline
13852 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
13853 4924 Acked Read 10 Done Done Sent false
13854 4925 Acked Flush 0 Done Done Sent false
13855 4926 Acked Read 3 Done Done Sent false
13856 4927 Acked Flush 0 Done Done Sent false
13857 4928 Acked Flush 0 Done Done Sent false
13858 4929 Acked Read 1 Done Done Sent false
13859 4930 Acked Read 7 Done Done Sent false
13860 4931 Acked Write 5 Done Done Sent false
13861 4932 Acked Read 6 Done Done Sent false
13862 4933 Acked Write 7 Done Done Sent false
13863 4934 Acked Write 2 Done Done Sent false
13864 4935 Acked Write 2 Done Done Sent false
13865 4936 Acked Flush 0 Done Done Sent false
13866 4937 Acked Write 4 Done Done Sent false
13867 4938 Acked Write 6 Done Done Sent false
13868 4939 Acked Read 8 Done Done Sent false
13869 4940 Acked Read 6 Done Done Sent false
13870 4941 Acked Write 1 Done Done Sent false
13871 4942 Acked Read 1 Done Done Sent false
13872 4943 Acked Read 5 Done Done Sent false
13873 4944 Acked Read 3 Done Done Sent false
13874 4945 Acked Write 2 Done Done Sent false
13875 4946 Acked Write 7 Done Done Sent false
13876 4947 Acked Write 5 Done Done Sent false
13877 4948 Acked Read 4 Done Done Sent false
13878 4949 Acked Write 7 Done Done Sent false
13879 4950 Acked Write 6 Done Done Sent false
13880 4951 Acked Read 3 Done Done Sent false
13881 4952 Acked Write 10 Done Done Sent false
13882 4953 Acked Flush 0 Done Done Sent false
13883 4954 Acked Read 2 Done Done Sent false
13884 4955 Acked Write 8 Done Done Sent false
13885 4956 Acked Write 6 Done Done Sent false
13886 4957 Acked Write 1 Done Done Sent false
13887 4958 Acked Write 7 Done Done Sent false
13888 4959 Acked Read 5 Done Done Sent false
13889 4960 Acked Read 1 Done Done Sent false
13890 4961 Acked Write 10 Done Done Sent false
13891 4962 Acked Read 2 Done Done Sent false
13892 4963 Acked Read 5 Done Done Sent false
13893 4964 Acked Read 10 Done Done Sent false
13894 4965 Acked Write 6 Done Done Sent false
13895 4966 Acked Read 9 Done Done Sent false
13896 4967 Acked Write 4 Done Done Sent false
13897 4968 Acked Write 2 Done Done Sent false
13898 4969 Acked Read 10 Done Done Sent false
13899 4970 Acked Write 4 Done Done Sent false
13900 4971 Acked Write 5 Done Done Sent false
13901 4972 Acked Read 2 Done Done Sent false
13902 4973 Acked Read 5 Done Done Sent false
13903 4974 Acked Write 6 Done Done Sent false
13904 4975 Acked Flush 0 Done Done Sent false
13905 4976 Acked Read 7 Done Done Sent false
13906 4977 Acked Read 9 Done Done Sent false
13907 4978 Acked Read 3 Done Done Sent false
13908 4979 Acked Write 10 Done Done Sent false
13909 4980 Acked Flush 0 Done Done Sent false
13910 4981 Acked Read 9 Done Done Sent false
13911 4982 Acked Flush 0 Done Done Sent false
13912 4983 Acked Read 9 Done Done Sent false
13913 4984 Acked Read 2 Done Done Sent false
13914 4985 Acked Read 7 Done Done Sent false
13915 4986 Acked Read 7 Done Done Sent false
13916 4987 Acked Write 5 Done Done Sent false
13917 4988 Acked Write 8 Done Done Sent false
13918 4989 Acked Write 9 Done Done Sent false
13919 4990 Acked Read 9 Done Done Sent false
13920 4991 Acked Read 5 Done Done Sent false
13921 4992 Acked Write 8 Done Done Sent false
13922 4993 Acked Write 6 Done Done Sent false
13923 4994 Acked Write 10 Done Done Sent false
13924 4995 Acked Read 1 Done Done Sent false
13925 4996 Acked Read 10 Done Done Sent false
13926 4997 Acked Read 4 Done Done Sent false
13927 4998 Acked Write 3 Done Done Sent false
13928 4999 Acked Read 8 Done Done Sent false
13929 5000 Acked Write 9 Done Done Sent false
13930 5001 Acked Read 6 Done Done Sent false
13931 5002 Acked Flush 0 Done Done Sent false
13932 5003 Acked Write 8 Done Done Sent false
13933 5004 Acked Read 6 Done Done Sent false
13934 5005 Acked Flush 0 Done Done Sent false
13935 5006 Acked Write 1 Done Done Sent false
13936 5007 Acked Write 9 Done Done Sent false
13937 5008 Acked Read 6 Done Done Sent false
13938 5009 Acked Read 3 Done Done Sent false
13939 5010 Acked Read 3 Done Done Sent false
13940 5011 Acked Read 8 Done Done Sent false
13941 5012 Acked Write 9 Done Done Sent false
13942 5013 Acked Read 5 Done Done Sent false
13943 5014 Acked Write 10 Done Done Sent false
13944 5015 Acked Read 9 Done Done Sent false
13945 5016 Acked Write 10 Done Done Sent false
13946 5017 Acked Read 9 Done Done Sent false
13947 5018 Acked Read 5 Done Done Sent false
13948 5019 Acked Write 5 Done Done Sent false
13949 5020 Acked Read 7 Done Done Sent false
13950 5021 Acked Write 8 Done Done Sent false
13951 5022 Acked Write 10 Done Done Sent false
13952 5023 Acked Flush 0 Done Done Sent false
13953 5024 Acked Write 4 Done Done Sent false
13954 5025 Acked Write 3 Done Done Sent false
13955 5026 Acked Read 10 Done Done Sent false
13956 5027 Acked Read 6 Done Done Sent false
13957 5028 Acked Read 6 Done Done Sent false
13958 5029 Acked Flush 0 Done Done Sent false
13959 5030 Acked Flush 0 Done Done Sent false
13960 5031 Acked Read 4 Done Done Sent false
13961 5032 Acked Write 9 Done Done Sent false
13962 5033 Acked Read 5 Done Done Sent false
13963 5034 Acked Flush 0 Done Done Sent false
13964 5035 Acked Read 6 Done Done Sent false
13965 5036 Acked Write 1 Done Done Sent false
13966 5037 Acked Write 8 Done Done Sent false
13967 5038 Acked Read 9 Done Done Sent false
13968 5039 Acked Write 6 Done Done Sent false
13969 5040 Acked Read 10 Done Done Sent false
13970 5041 Acked Read 10 Done Done Sent false
13971 5042 Acked Read 8 Done Done Sent false
13972 5043 Acked Write 6 Done Done Sent false
13973 5044 Acked Flush 0 Done Done Sent false
13974 5045 Acked Read 6 Done Done Sent false
13975 5046 Acked Write 4 Done Done Sent false
13976 5047 Acked Flush 0 Done Done Sent false
13977 5048 Acked Read 8 Done Done Sent false
13978 5049 Acked Write 3 Done Done Sent false
13979 5050 Acked Read 2 Done Done Sent false
13980 5051 Acked Write 3 Done Done Sent false
13981 5052 Acked Read 5 Done Done Sent false
13982 5053 Acked Read 2 Done Done Sent false
13983 5054 Acked Write 7 Done Done Sent false
13984 5055 Acked Read 5 Done Done Sent false
13985 5056 Acked Read 2 Done Done Sent false
13986 5057 Acked Read 9 Done Done Sent false
13987 5058 Acked Flush 0 Done Done Sent false
13988 5059 Acked Write 4 Done Done Sent false
13989 5060 Acked Write 6 Done Done Sent false
13990 5061 Acked Read 3 Done Done Sent false
13991 5062 Acked Read 8 Done Done Sent false
13992 5063 Acked Write 6 Done Done Sent false
13993 5064 Acked Write 5 Done Done Sent false
13994 5065 Acked Read 2 Done Done Sent false
13995 5066 Acked Flush 0 Done Done Sent false
13996 5067 Acked Flush 0 Done Done Sent false
13997 5068 Acked Read 10 Done Done Sent false
13998 5069 Acked Write 1 Done Done Sent false
13999 5070 Acked Write 10 Done Done Sent false
14000 5071 Acked Read 2 Done Done Sent false
14001 5072 Acked Write 6 Done Done Sent false
14002 5073 Acked Read 10 Done Done Sent false
14003 5074 Acked Write 2 Done Done Sent false
14004 5075 Acked Write 4 Done Done Sent false
14005 5076 Acked Read 4 Done Done Sent false
14006 5077 Acked Write 2 Done Done Sent false
14007 5078 Acked Read 9 Done Done Sent false
14008 5079 Acked Write 2 Done Done Sent false
14009 5080 Acked Read 5 Done Done Sent false
14010 5081 Acked Read 6 Done Done Sent false
14011 5082 Acked Write 9 Done Done Sent false
14012 5083 Acked Write 6 Done Done Sent false
14013 5084 Acked Read 9 Done Done Sent false
14014 5085 Acked Write 7 Done Done Sent false
14015 5086 Acked Read 1 Done Done Sent false
14016 5087 Acked Read 2 Done Done Sent false
14017 5088 Acked Read 9 Done Done Sent false
14018 5089 Acked Read 8 Done Done Sent false
14019 5090 Acked Read 6 Done Done Sent false
14020 5091 Acked Flush 0 Done Done Sent false
14021 5092 Acked Read 6 Done Done Sent false
14022 5093 Acked Read 10 Done Done Sent false
14023 5094 Acked Read 8 Done Done Sent false
14024 5095 Acked Read 3 Done Done Sent false
14025 5096 Acked Write 4 Done Done Sent false
14026 5097 Acked Read 5 Done Done Sent false
14027 5098 Acked Read 5 Done Done Sent false
14028 5099 Acked Write 9 Done Done Sent false
14029 5100 Acked Flush 0 Done Done Sent false
14030 5101 Acked Read 7 Done Done Sent false
14031 5102 Acked Write 1 Done Done Sent false
14032 5103 Acked Write 5 Done Done Sent false
14033 5104 Acked Read 6 Done Done Sent false
14034 5105 Acked Read 4 Done Done Sent false
14035 5106 Acked Write 6 Done Done Sent false
14036 5107 Acked Read 6 Done Done Sent false
14037 5108 Acked Write 6 Done Done Sent false
14038 5109 Acked Write 8 Done Done Sent false
14039 5110 Acked Flush 0 Done Done Sent false
14040 5111 Acked Write 3 Done Done Sent false
14041 5112 Acked Write 1 Done Done Sent false
14042 5113 Acked Read 7 Done Done Sent false
14043 5114 Acked Read 2 Done Done Sent false
14044 5115 Acked Write 1 Done Done Sent false
14045 5116 Acked Read 2 Done Done Sent false
14046 5117 Acked Flush 0 Done Done Sent false
14047 5118 Acked Flush 0 Done Done Sent false
14048 5119 Acked Read 2 Done Done Sent false
14049 5120 Acked Write 7 Done Done Sent false
14050 5121 Acked Write 4 Done Done Sent false
14051 5122 Acked Read 3 Done Done Sent false
14052 5123 Acked Write 2 Done Done Sent false
14053 5124 Acked Read 7 Done Done Sent false
14054 5125 Acked Read 9 Done Done Sent false
14055 5126 Acked Flush 0 Done Done Sent false
14056 5127 Acked Write 3 Done Done Sent false
14057 5128 Acked Write 10 Done Done Sent false
14058 5129 Acked Read 10 Done Done Sent false
14059 5130 Acked Read 8 Done Done Sent false
14060 5131 Acked Read 5 Done Done Sent false
14061 5132 Acked Read 7 Done Done Sent false
14062 5133 Acked Read 10 Done Done Sent false
14063 5134 Acked Write 3 Done Done Sent false
14064 5135 Acked Read 2 Done Done Sent false
14065 5136 Acked Write 6 Done Done Sent false
14066 5137 Acked Flush 0 Done Done Sent false
14067 5138 Acked Write 10 Done Done Sent false
14068 5139 Acked Read 9 Done Done Sent false
14069 5140 Acked Flush 0 Done Done Sent false
14070 5141 Acked Write 1 Done Done Sent false
14071 5142 Acked Write 3 Done Done Sent false
14072 5143 Acked Read 10 Done Done Sent false
14073 5144 Acked Write 3 Done Done Sent false
14074 5145 Acked Write 10 Done Done Sent false
14075 5146 Acked Read 4 Done Done Sent false
14076 5147 Acked Read 2 Done Done Sent false
14077 5148 Acked Write 10 Done Done Sent false
14078 5149 Acked Write 7 Done Done Sent false
14079 5150 Acked Read 9 Done Done Sent false
14080 5151 Acked Flush 0 Done Done Sent false
14081 5152 Acked Read 2 Done Done Sent false
14082 5153 Acked Write 5 Done Done Sent false
14083 5154 Acked Read 1 Done Done Sent false
14084 5155 Acked Write 9 Done Done Sent false
14085 5156 Acked Write 5 Done Done Sent false
14086 5157 Acked Write 9 Done Done Sent false
14087 5158 Acked Write 7 Done Done Sent false
14088 5159 Acked Write 7 Done Done Sent false
14089 5160 Acked Write 1 Done Done Sent false
14090 5161 Acked Read 3 Done Done Sent false
14091 5162 Acked Write 2 Done Done Sent false
14092 5163 Acked Read 5 Done Done Sent false
14093 5164 Acked Read 5 Done Done Sent false
14094 5165 Acked Read 6 Done Done Sent false
14095 5166 Acked Read 7 Done Done Sent false
14096 5167 Acked Read 6 Done Done Sent false
14097 5168 Acked Read 9 Done Done Sent false
14098 5169 Acked Read 1 Done Done Sent false
14099 5170 Acked Read 1 Done Done Sent false
14100 5171 Acked Read 4 Done Done Sent false
14101 5172 Acked Read 7 Done Done Sent false
14102 5173 Acked Write 2 Done Done Sent false
14103 5174 Acked Write 4 Done Done Sent false
14104 5175 Acked Read 2 Done Done Sent false
14105 5176 Acked Write 10 Done Done Sent false
14106 5177 Acked Read 7 Done Done Sent false
14107 5178 Acked Write 9 Done Done Sent false
14108 5179 Acked Write 8 Done Done Sent false
14109 5180 Acked Write 4 Done Done Sent false
14110 5181 Acked Read 8 Done Done Sent false
14111 5182 Acked Flush 0 Done Done Sent false
14112 5183 Acked Read 10 Done Done Sent false
14113 5184 Acked Read 9 Done Done Sent false
14114 5185 Acked Read 2 Done Done Sent false
14115 5186 Acked Write 4 Done Done Sent false
14116 5187 Acked Write 10 Done Done Sent false
14117 5188 Acked Read 2 Done Done Sent false
14118 5189 Acked Read 10 Done Done Sent false
14119 5190 Acked Read 5 Done Done Sent false
14120 5191 Acked Read 8 Done Done Sent false
14121 5192 Acked Read 8 Done Done Sent false
14122 5193 Acked Write 2 Done Done Sent false
14123 5194 Acked Flush 0 Done Done Sent false
14124 5195 Acked Read 3 Done Done Sent false
14125 5196 Acked Read 4 Done Done Sent false
14126 5197 Acked Write 4 Done Done Sent false
14127 5198 Acked Write 3 Done Done Sent false
14128 5199 Acked Read 2 Done Done Sent false
14129 5200 Acked Read 8 Done Done Sent false
14130 5201 Acked Read 3 Done Done Sent false
14131 5202 Acked Read 7 Done Done Sent false
14132 5203 Acked Write 8 Done Done Sent false
14133 5204 Acked Write 7 Done Done Sent false
14134 5205 Acked Read 6 Done Done Sent false
14135 5206 Acked Read 3 Done Done Sent false
14136 5207 Acked Read 10 Done Done Sent false
14137 5208 Acked Read 9 Done Done Sent false
14138 5209 Acked Write 4 Done Done Sent false
14139 5210 Acked Write 1 Done Done Sent false
14140 5211 Acked Read 8 Done Done Sent false
14141 5212 Acked Write 7 Done Done Sent false
14142 5213 Acked Write 3 Done Done Sent false
14143 5214 Acked Write 4 Done Done Sent false
14144 5215 Acked Read 10 Done Done Sent false
14145 5216 Acked Read 7 Done Done Sent false
14146 5217 Acked Read 4 Done Done Sent false
14147 5218 Acked Write 8 Done Done Sent false
14148 5219 Acked Flush 0 Done Done Sent false
14149 5220 Acked Write 9 Done Done Sent false
14150 5221 Acked Read 6 Done Done Sent false
14151 5222 Acked Write 10 Done Done Sent false
14152 5223 Acked Read 10 Done Done Sent false
14153 5224 Acked Read 2 Done Done Sent false
14154 STATES DS:0 DS:1 DS:2 TOTAL
14155 Sent 0 0 301 301
14156 Done 301 301 0 602
14157 Skipped 0 0 0 0
14158 Error 0 0 0 0
14159 Last Flush: 5219 5219 4923
14160 Downstairs last five completed: 4923 4922 4921 4920 4919
14161 Upstairs last five completed: 5224 5223 5222 5221 5220
14162 CLIENT: Up:0 ds:301 act:2
14163 ----------------------------------------------------------------
14164 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
14165 States: Active Active Offline
14166 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
14167 4924 Acked Read 10 Done Done Sent false
14168 4925 Acked Flush 0 Done Done Sent false
14169 4926 Acked Read 3 Done Done Sent false
14170 4927 Acked Flush 0 Done Done Sent false
14171 4928 Acked Flush 0 Done Done Sent false
14172 4929 Acked Read 1 Done Done Sent false
14173 4930 Acked Read 7 Done Done Sent false
14174 4931 Acked Write 5 Done Done Sent false
14175 4932 Acked Read 6 Done Done Sent false
14176 4933 Acked Write 7 Done Done Sent false
14177 4934 Acked Write 2 Done Done Sent false
14178 4935 Acked Write 2 Done Done Sent false
14179 4936 Acked Flush 0 Done Done Sent false
14180 4937 Acked Write 4 Done Done Sent false
14181 4938 Acked Write 6 Done Done Sent false
14182 4939 Acked Read 8 Done Done Sent false
14183 4940 Acked Read 6 Done Done Sent false
14184 4941 Acked Write 1 Done Done Sent false
14185 4942 Acked Read 1 Done Done Sent false
14186 4943 Acked Read 5 Done Done Sent false
14187 4944 Acked Read 3 Done Done Sent false
14188 4945 Acked Write 2 Done Done Sent false
14189 4946 Acked Write 7 Done Done Sent false
14190 4947 Acked Write 5 Done Done Sent false
14191 4948 Acked Read 4 Done Done Sent false
14192 4949 Acked Write 7 Done Done Sent false
14193 4950 Acked Write 6 Done Done Sent false
14194 4951 Acked Read 3 Done Done Sent false
14195 4952 Acked Write 10 Done Done Sent false
14196 4953 Acked Flush 0 Done Done Sent false
14197 4954 Acked Read 2 Done Done Sent false
14198 4955 Acked Write 8 Done Done Sent false
14199 4956 Acked Write 6 Done Done Sent false
14200 4957 Acked Write 1 Done Done Sent false
14201 4958 Acked Write 7 Done Done Sent false
14202 4959 Acked Read 5 Done Done Sent false
14203 4960 Acked Read 1 Done Done Sent false
14204 4961 Acked Write 10 Done Done Sent false
14205 4962 Acked Read 2 Done Done Sent false
14206 4963 Acked Read 5 Done Done Sent false
14207 4964 Acked Read 10 Done Done Sent false
14208 4965 Acked Write 6 Done Done Sent false
14209 4966 Acked Read 9 Done Done Sent false
14210 4967 Acked Write 4 Done Done Sent false
14211 4968 Acked Write 2 Done Done Sent false
14212 4969 Acked Read 10 Done Done Sent false
14213 4970 Acked Write 4 Done Done Sent false
14214 4971 Acked Write 5 Done Done Sent false
14215 4972 Acked Read 2 Done Done Sent false
14216 4973 Acked Read 5 Done Done Sent false
14217 4974 Acked Write 6 Done Done Sent false
14218 4975 Acked Flush 0 Done Done Sent false
14219 4976 Acked Read 7 Done Done Sent false
14220 4977 Acked Read 9 Done Done Sent false
14221 4978 Acked Read 3 Done Done Sent false
14222 4979 Acked Write 10 Done Done Sent false
14223 4980 Acked Flush 0 Done Done Sent false
14224 4981 Acked Read 9 Done Done Sent false
14225 4982 Acked Flush 0 Done Done Sent false
14226 4983 Acked Read 9 Done Done Sent false
14227 4984 Acked Read 2 Done Done Sent false
14228 4985 Acked Read 7 Done Done Sent false
14229 4986 Acked Read 7 Done Done Sent false
14230 4987 Acked Write 5 Done Done Sent false
14231 4988 Acked Write 8 Done Done Sent false
14232 4989 Acked Write 9 Done Done Sent false
14233 4990 Acked Read 9 Done Done Sent false
14234 4991 Acked Read 5 Done Done Sent false
14235 4992 Acked Write 8 Done Done Sent false
14236 4993 Acked Write 6 Done Done Sent false
14237 4994 Acked Write 10 Done Done Sent false
14238 4995 Acked Read 1 Done Done Sent false
14239 4996 Acked Read 10 Done Done Sent false
14240 4997 Acked Read 4 Done Done Sent false
14241 4998 Acked Write 3 Done Done Sent false
14242 4999 Acked Read 8 Done Done Sent false
14243 5000 Acked Write 9 Done Done Sent false
14244 5001 Acked Read 6 Done Done Sent false
14245 5002 Acked Flush 0 Done Done Sent false
14246 5003 Acked Write 8 Done Done Sent false
14247 5004 Acked Read 6 Done Done Sent false
14248 5005 Acked Flush 0 Done Done Sent false
14249 5006 Acked Write 1 Done Done Sent false
14250 5007 Acked Write 9 Done Done Sent false
14251 5008 Acked Read 6 Done Done Sent false
14252 5009 Acked Read 3 Done Done Sent false
14253 5010 Acked Read 3 Done Done Sent false
14254 5011 Acked Read 8 Done Done Sent false
14255 5012 Acked Write 9 Done Done Sent false
14256 5013 Acked Read 5 Done Done Sent false
14257 5014 Acked Write 10 Done Done Sent false
14258 5015 Acked Read 9 Done Done Sent false
14259 5016 Acked Write 10 Done Done Sent false
14260 5017 Acked Read 9 Done Done Sent false
14261 5018 Acked Read 5 Done Done Sent false
14262 5019 Acked Write 5 Done Done Sent false
14263 5020 Acked Read 7 Done Done Sent false
14264 5021 Acked Write 8 Done Done Sent false
14265 5022 Acked Write 10 Done Done Sent false
14266 5023 Acked Flush 0 Done Done Sent false
14267 5024 Acked Write 4 Done Done Sent false
14268 5025 Acked Write 3 Done Done Sent false
14269 5026 Acked Read 10 Done Done Sent false
14270 5027 Acked Read 6 Done Done Sent false
14271 5028 Acked Read 6 Done Done Sent false
14272 5029 Acked Flush 0 Done Done Sent false
14273 5030 Acked Flush 0 Done Done Sent false
14274 5031 Acked Read 4 Done Done Sent false
14275 5032 Acked Write 9 Done Done Sent false
14276 5033 Acked Read 5 Done Done Sent false
14277 5034 Acked Flush 0 Done Done Sent false
14278 5035 Acked Read 6 Done Done Sent false
14279 5036 Acked Write 1 Done Done Sent false
14280 5037 Acked Write 8 Done Done Sent false
14281 5038 Acked Read 9 Done Done Sent false
14282 5039 Acked Write 6 Done Done Sent false
14283 5040 Acked Read 10 Done Done Sent false
14284 5041 Acked Read 10 Done Done Sent false
14285 5042 Acked Read 8 Done Done Sent false
14286 5043 Acked Write 6 Done Done Sent false
14287 5044 Acked Flush 0 Done Done Sent false
14288 5045 Acked Read 6 Done Done Sent false
14289 5046 Acked Write 4 Done Done Sent false
14290 5047 Acked Flush 0 Done Done Sent false
14291 5048 Acked Read 8 Done Done Sent false
14292 5049 Acked Write 3 Done Done Sent false
14293 5050 Acked Read 2 Done Done Sent false
14294 5051 Acked Write 3 Done Done Sent false
14295 5052 Acked Read 5 Done Done Sent false
14296 5053 Acked Read 2 Done Done Sent false
14297 5054 Acked Write 7 Done Done Sent false
14298 5055 Acked Read 5 Done Done Sent false
14299 5056 Acked Read 2 Done Done Sent false
14300 5057 Acked Read 9 Done Done Sent false
14301 5058 Acked Flush 0 Done Done Sent false
14302 5059 Acked Write 4 Done Done Sent false
14303 5060 Acked Write 6 Done Done Sent false
14304 5061 Acked Read 3 Done Done Sent false
14305 5062 Acked Read 8 Done Done Sent false
14306 5063 Acked Write 6 Done Done Sent false
14307 5064 Acked Write 5 Done Done Sent false
14308 5065 Acked Read 2 Done Done Sent false
14309 5066 Acked Flush 0 Done Done Sent false
14310 5067 Acked Flush 0 Done Done Sent false
14311 5068 Acked Read 10 Done Done Sent false
14312 5069 Acked Write 1 Done Done Sent false
14313 5070 Acked Write 10 Done Done Sent false
14314 5071 Acked Read 2 Done Done Sent false
14315 5072 Acked Write 6 Done Done Sent false
14316 5073 Acked Read 10 Done Done Sent false
14317 5074 Acked Write 2 Done Done Sent false
14318 5075 Acked Write 4 Done Done Sent false
14319 5076 Acked Read 4 Done Done Sent false
14320 5077 Acked Write 2 Done Done Sent false
14321 5078 Acked Read 9 Done Done Sent false
14322 5079 Acked Write 2 Done Done Sent false
14323 5080 Acked Read 5 Done Done Sent false
14324 5081 Acked Read 6 Done Done Sent false
14325 5082 Acked Write 9 Done Done Sent false
14326 5083 Acked Write 6 Done Done Sent false
14327 5084 Acked Read 9 Done Done Sent false
14328 5085 Acked Write 7 Done Done Sent false
14329 5086 Acked Read 1 Done Done Sent false
14330 5087 Acked Read 2 Done Done Sent false
14331 5088 Acked Read 9 Done Done Sent false
14332 5089 Acked Read 8 Done Done Sent false
14333 5090 Acked Read 6 Done Done Sent false
14334 5091 Acked Flush 0 Done Done Sent false
14335 5092 Acked Read 6 Done Done Sent false
14336 5093 Acked Read 10 Done Done Sent false
14337 5094 Acked Read 8 Done Done Sent false
14338 5095 Acked Read 3 Done Done Sent false
14339 5096 Acked Write 4 Done Done Sent false
14340 5097 Acked Read 5 Done Done Sent false
14341 5098 Acked Read 5 Done Done Sent false
14342 5099 Acked Write 9 Done Done Sent false
14343 5100 Acked Flush 0 Done Done Sent false
14344 5101 Acked Read 7 Done Done Sent false
14345 5102 Acked Write 1 Done Done Sent false
14346 5103 Acked Write 5 Done Done Sent false
14347 5104 Acked Read 6 Done Done Sent false
14348 5105 Acked Read 4 Done Done Sent false
14349 5106 Acked Write 6 Done Done Sent false
14350 5107 Acked Read 6 Done Done Sent false
14351 5108 Acked Write 6 Done Done Sent false
14352 5109 Acked Write 8 Done Done Sent false
14353 5110 Acked Flush 0 Done Done Sent false
14354 5111 Acked Write 3 Done Done Sent false
14355 5112 Acked Write 1 Done Done Sent false
14356 5113 Acked Read 7 Done Done Sent false
14357 5114 Acked Read 2 Done Done Sent false
14358 5115 Acked Write 1 Done Done Sent false
14359 5116 Acked Read 2 Done Done Sent false
14360 5117 Acked Flush 0 Done Done Sent false
14361 5118 Acked Flush 0 Done Done Sent false
14362 5119 Acked Read 2 Done Done Sent false
14363 5120 Acked Write 7 Done Done Sent false
14364 5121 Acked Write 4 Done Done Sent false
14365 5122 Acked Read 3 Done Done Sent false
14366 5123 Acked Write 2 Done Done Sent false
14367 5124 Acked Read 7 Done Done Sent false
14368 5125 Acked Read 9 Done Done Sent false
14369 5126 Acked Flush 0 Done Done Sent false
14370 5127 Acked Write 3 Done Done Sent false
14371 5128 Acked Write 10 Done Done Sent false
14372 5129 Acked Read 10 Done Done Sent false
14373 5130 Acked Read 8 Done Done Sent false
14374 5131 Acked Read 5 Done Done Sent false
14375 5132 Acked Read 7 Done Done Sent false
14376 5133 Acked Read 10 Done Done Sent false
14377 5134 Acked Write 3 Done Done Sent false
14378 5135 Acked Read 2 Done Done Sent false
14379 5136 Acked Write 6 Done Done Sent false
14380 5137 Acked Flush 0 Done Done Sent false
14381 5138 Acked Write 10 Done Done Sent false
14382 5139 Acked Read 9 Done Done Sent false
14383 5140 Acked Flush 0 Done Done Sent false
14384 5141 Acked Write 1 Done Done Sent false
14385 5142 Acked Write 3 Done Done Sent false
14386 5143 Acked Read 10 Done Done Sent false
14387 5144 Acked Write 3 Done Done Sent false
14388 5145 Acked Write 10 Done Done Sent false
14389 5146 Acked Read 4 Done Done Sent false
14390 5147 Acked Read 2 Done Done Sent false
14391 5148 Acked Write 10 Done Done Sent false
14392 5149 Acked Write 7 Done Done Sent false
14393 5150 Acked Read 9 Done Done Sent false
14394 5151 Acked Flush 0 Done Done Sent false
14395 5152 Acked Read 2 Done Done Sent false
14396 5153 Acked Write 5 Done Done Sent false
14397 5154 Acked Read 1 Done Done Sent false
14398 5155 Acked Write 9 Done Done Sent false
14399 5156 Acked Write 5 Done Done Sent false
14400 5157 Acked Write 9 Done Done Sent false
14401 5158 Acked Write 7 Done Done Sent false
14402 5159 Acked Write 7 Done Done Sent false
14403 5160 Acked Write 1 Done Done Sent false
14404 5161 Acked Read 3 Done Done Sent false
14405 5162 Acked Write 2 Done Done Sent false
14406 5163 Acked Read 5 Done Done Sent false
14407 5164 Acked Read 5 Done Done Sent false
14408 5165 Acked Read 6 Done Done Sent false
14409 5166 Acked Read 7 Done Done Sent false
14410 5167 Acked Read 6 Done Done Sent false
14411 5168 Acked Read 9 Done Done Sent false
14412 5169 Acked Read 1 Done Done Sent false
14413 5170 Acked Read 1 Done Done Sent false
14414 5171 Acked Read 4 Done Done Sent false
14415 5172 Acked Read 7 Done Done Sent false
14416 5173 Acked Write 2 Done Done Sent false
14417 5174 Acked Write 4 Done Done Sent false
14418 5175 Acked Read 2 Done Done Sent false
14419 5176 Acked Write 10 Done Done Sent false
14420 5177 Acked Read 7 Done Done Sent false
14421 5178 Acked Write 9 Done Done Sent false
14422 5179 Acked Write 8 Done Done Sent false
14423 5180 Acked Write 4 Done Done Sent false
14424 5181 Acked Read 8 Done Done Sent false
14425 5182 Acked Flush 0 Done Done Sent false
14426 5183 Acked Read 10 Done Done Sent false
14427 5184 Acked Read 9 Done Done Sent false
14428 5185 Acked Read 2 Done Done Sent false
14429 5186 Acked Write 4 Done Done Sent false
14430 5187 Acked Write 10 Done Done Sent false
14431 5188 Acked Read 2 Done Done Sent false
14432 5189 Acked Read 10 Done Done Sent false
14433 5190 Acked Read 5 Done Done Sent false
14434 5191 Acked Read 8 Done Done Sent false
14435 5192 Acked Read 8 Done Done Sent false
14436 5193 Acked Write 2 Done Done Sent false
14437 5194 Acked Flush 0 Done Done Sent false
14438 5195 Acked Read 3 Done Done Sent false
14439 5196 Acked Read 4 Done Done Sent false
14440 5197 Acked Write 4 Done Done Sent false
14441 5198 Acked Write 3 Done Done Sent false
14442 5199 Acked Read 2 Done Done Sent false
14443 5200 Acked Read 8 Done Done Sent false
14444 5201 Acked Read 3 Done Done Sent false
14445 5202 Acked Read 7 Done Done Sent false
14446 5203 Acked Write 8 Done Done Sent false
14447 5204 Acked Write 7 Done Done Sent false
14448 5205 Acked Read 6 Done Done Sent false
14449 5206 Acked Read 3 Done Done Sent false
14450 5207 Acked Read 10 Done Done Sent false
14451 5208 Acked Read 9 Done Done Sent false
14452 5209 Acked Write 4 Done Done Sent false
14453 5210 Acked Write 1 Done Done Sent false
14454 5211 Acked Read 8 Done Done Sent false
14455 5212 Acked Write 7 Done Done Sent false
14456 5213 Acked Write 3 Done Done Sent false
14457 5214 Acked Write 4 Done Done Sent false
14458 5215 Acked Read 10 Done Done Sent false
14459 5216 Acked Read 7 Done Done Sent false
14460 5217 Acked Read 4 Done Done Sent false
14461 5218 Acked Write 8 Done Done Sent false
14462 5219 Acked Flush 0 Done Done Sent false
14463 5220 Acked Write 9 Done Done Sent false
14464 5221 Acked Read 6 Done Done Sent false
14465 5222 Acked Write 10 Done Done Sent false
14466 5223 Acked Read 10 Done Done Sent false
14467 5224 Acked Read 2 Done Done Sent false
14468 5225 Acked Flush 0 Done Done Sent false
14469 STATES DS:0 DS:1 DS:2 TOTAL
14470 Sent 0 0 302 302
14471 Done 302 302 0 604
14472 Skipped 0 0 0 0
14473 Error 0 0 0 0
14474 Last Flush: 5225 5225 4923
14475 Downstairs last five completed: 4923 4922 4921 4920 4919
14476 Upstairs last five completed: 5225 5224 5223 5222 5221
14477 CLIENT: Up:0 ds:302 act:2
144782024-11-28T04:53:20.169ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
144792024-11-28T04:53:20.169ZWARNcrucible: ds_connection connect to 127.0.0.1:8830 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
144802024-11-28T04:53:20.169ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
144812024-11-28T04:53:20.169ZWARNcrucible: downstairs task for 2 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
144822024-11-28T04:53:20.169ZINFOcrucible: client task is exiting = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
14483 [8830] got start action
14484 [8830] I am going to restart
14485 Starting downstairs at port 8830
14486 Make output file at "/tmp/dsc/downstairs-8830.txt"
14487 [8830][2] reports Starting
14488 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1216)
14489 [8830][2] reports Running
14490 ----------------------------------------------------------------
14491 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
14492 States: Active Active Offline
14493 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
14494 4924 Acked Read 10 Done Done Sent false
14495 4925 Acked Flush 0 Done Done Sent false
14496 4926 Acked Read 3 Done Done Sent false
14497 4927 Acked Flush 0 Done Done Sent false
14498 4928 Acked Flush 0 Done Done Sent false
14499 4929 Acked Read 1 Done Done Sent false
14500 4930 Acked Read 7 Done Done Sent false
14501 4931 Acked Write 5 Done Done Sent false
14502 4932 Acked Read 6 Done Done Sent false
14503 4933 Acked Write 7 Done Done Sent false
14504 4934 Acked Write 2 Done Done Sent false
14505 4935 Acked Write 2 Done Done Sent false
14506 4936 Acked Flush 0 Done Done Sent false
14507 4937 Acked Write 4 Done Done Sent false
14508 4938 Acked Write 6 Done Done Sent false
14509 4939 Acked Read 8 Done Done Sent false
14510 4940 Acked Read 6 Done Done Sent false
14511 4941 Acked Write 1 Done Done Sent false
14512 4942 Acked Read 1 Done Done Sent false
14513 4943 Acked Read 5 Done Done Sent false
14514 4944 Acked Read 3 Done Done Sent false
14515 4945 Acked Write 2 Done Done Sent false
14516 4946 Acked Write 7 Done Done Sent false
14517 4947 Acked Write 5 Done Done Sent false
14518 4948 Acked Read 4 Done Done Sent false
14519 4949 Acked Write 7 Done Done Sent false
14520 4950 Acked Write 6 Done Done Sent false
14521 4951 Acked Read 3 Done Done Sent false
14522 4952 Acked Write 10 Done Done Sent false
14523 4953 Acked Flush 0 Done Done Sent false
14524 4954 Acked Read 2 Done Done Sent false
14525 4955 Acked Write 8 Done Done Sent false
14526 4956 Acked Write 6 Done Done Sent false
14527 4957 Acked Write 1 Done Done Sent false
14528 4958 Acked Write 7 Done Done Sent false
14529 4959 Acked Read 5 Done Done Sent false
14530 4960 Acked Read 1 Done Done Sent false
14531 4961 Acked Write 10 Done Done Sent false
14532 4962 Acked Read 2 Done Done Sent false
14533 4963 Acked Read 5 Done Done Sent false
14534 4964 Acked Read 10 Done Done Sent false
14535 4965 Acked Write 6 Done Done Sent false
14536 4966 Acked Read 9 Done Done Sent false
14537 4967 Acked Write 4 Done Done Sent false
14538 4968 Acked Write 2 Done Done Sent false
14539 4969 Acked Read 10 Done Done Sent false
14540 4970 Acked Write 4 Done Done Sent false
14541 4971 Acked Write 5 Done Done Sent false
14542 4972 Acked Read 2 Done Done Sent false
14543 4973 Acked Read 5 Done Done Sent false
14544 4974 Acked Write 6 Done Done Sent false
14545 4975 Acked Flush 0 Done Done Sent false
14546 4976 Acked Read 7 Done Done Sent false
14547 4977 Acked Read 9 Done Done Sent false
14548 4978 Acked Read 3 Done Done Sent false
14549 4979 Acked Write 10 Done Done Sent false
14550 4980 Acked Flush 0 Done Done Sent false
14551 4981 Acked Read 9 Done Done Sent false
14552 4982 Acked Flush 0 Done Done Sent false
14553 4983 Acked Read 9 Done Done Sent false
14554 4984 Acked Read 2 Done Done Sent false
14555 4985 Acked Read 7 Done Done Sent false
14556 4986 Acked Read 7 Done Done Sent false
14557 4987 Acked Write 5 Done Done Sent false
14558 4988 Acked Write 8 Done Done Sent false
14559 4989 Acked Write 9 Done Done Sent false
14560 4990 Acked Read 9 Done Done Sent false
14561 4991 Acked Read 5 Done Done Sent false
14562 4992 Acked Write 8 Done Done Sent false
14563 4993 Acked Write 6 Done Done Sent false
14564 4994 Acked Write 10 Done Done Sent false
14565 4995 Acked Read 1 Done Done Sent false
14566 4996 Acked Read 10 Done Done Sent false
14567 4997 Acked Read 4 Done Done Sent false
14568 4998 Acked Write 3 Done Done Sent false
14569 4999 Acked Read 8 Done Done Sent false
14570 5000 Acked Write 9 Done Done Sent false
14571 5001 Acked Read 6 Done Done Sent false
14572 5002 Acked Flush 0 Done Done Sent false
14573 5003 Acked Write 8 Done Done Sent false
14574 5004 Acked Read 6 Done Done Sent false
14575 5005 Acked Flush 0 Done Done Sent false
14576 5006 Acked Write 1 Done Done Sent false
14577 5007 Acked Write 9 Done Done Sent false
14578 5008 Acked Read 6 Done Done Sent false
14579 5009 Acked Read 3 Done Done Sent false
14580 5010 Acked Read 3 Done Done Sent false
14581 5011 Acked Read 8 Done Done Sent false
14582 5012 Acked Write 9 Done Done Sent false
14583 5013 Acked Read 5 Done Done Sent false
14584 5014 Acked Write 10 Done Done Sent false
14585 5015 Acked Read 9 Done Done Sent false
14586 5016 Acked Write 10 Done Done Sent false
14587 5017 Acked Read 9 Done Done Sent false
14588 5018 Acked Read 5 Done Done Sent false
14589 5019 Acked Write 5 Done Done Sent false
14590 5020 Acked Read 7 Done Done Sent false
14591 5021 Acked Write 8 Done Done Sent false
14592 5022 Acked Write 10 Done Done Sent false
14593 5023 Acked Flush 0 Done Done Sent false
14594 5024 Acked Write 4 Done Done Sent false
14595 5025 Acked Write 3 Done Done Sent false
14596 5026 Acked Read 10 Done Done Sent false
14597 5027 Acked Read 6 Done Done Sent false
14598 5028 Acked Read 6 Done Done Sent false
14599 5029 Acked Flush 0 Done Done Sent false
14600 5030 Acked Flush 0 Done Done Sent false
14601 5031 Acked Read 4 Done Done Sent false
14602 5032 Acked Write 9 Done Done Sent false
14603 5033 Acked Read 5 Done Done Sent false
14604 5034 Acked Flush 0 Done Done Sent false
14605 5035 Acked Read 6 Done Done Sent false
14606 5036 Acked Write 1 Done Done Sent false
14607 5037 Acked Write 8 Done Done Sent false
14608 5038 Acked Read 9 Done Done Sent false
14609 5039 Acked Write 6 Done Done Sent false
14610 5040 Acked Read 10 Done Done Sent false
14611 5041 Acked Read 10 Done Done Sent false
14612 5042 Acked Read 8 Done Done Sent false
14613 5043 Acked Write 6 Done Done Sent false
14614 5044 Acked Flush 0 Done Done Sent false
14615 5045 Acked Read 6 Done Done Sent false
14616 5046 Acked Write 4 Done Done Sent false
14617 5047 Acked Flush 0 Done Done Sent false
14618 5048 Acked Read 8 Done Done Sent false
14619 5049 Acked Write 3 Done Done Sent false
14620 5050 Acked Read 2 Done Done Sent false
14621 5051 Acked Write 3 Done Done Sent false
14622 5052 Acked Read 5 Done Done Sent false
14623 5053 Acked Read 2 Done Done Sent false
14624 5054 Acked Write 7 Done Done Sent false
14625 5055 Acked Read 5 Done Done Sent false
14626 5056 Acked Read 2 Done Done Sent false
14627 5057 Acked Read 9 Done Done Sent false
14628 5058 Acked Flush 0 Done Done Sent false
14629 5059 Acked Write 4 Done Done Sent false
14630 5060 Acked Write 6 Done Done Sent false
14631 5061 Acked Read 3 Done Done Sent false
14632 5062 Acked Read 8 Done Done Sent false
14633 5063 Acked Write 6 Done Done Sent false
14634 5064 Acked Write 5 Done Done Sent false
14635 5065 Acked Read 2 Done Done Sent false
14636 5066 Acked Flush 0 Done Done Sent false
14637 5067 Acked Flush 0 Done Done Sent false
14638 5068 Acked Read 10 Done Done Sent false
14639 5069 Acked Write 1 Done Done Sent false
14640 5070 Acked Write 10 Done Done Sent false
14641 5071 Acked Read 2 Done Done Sent false
14642 5072 Acked Write 6 Done Done Sent false
14643 5073 Acked Read 10 Done Done Sent false
14644 5074 Acked Write 2 Done Done Sent false
14645 5075 Acked Write 4 Done Done Sent false
14646 5076 Acked Read 4 Done Done Sent false
14647 5077 Acked Write 2 Done Done Sent false
14648 5078 Acked Read 9 Done Done Sent false
14649 5079 Acked Write 2 Done Done Sent false
14650 5080 Acked Read 5 Done Done Sent false
14651 5081 Acked Read 6 Done Done Sent false
14652 5082 Acked Write 9 Done Done Sent false
14653 5083 Acked Write 6 Done Done Sent false
14654 5084 Acked Read 9 Done Done Sent false
14655 5085 Acked Write 7 Done Done Sent false
14656 5086 Acked Read 1 Done Done Sent false
14657 5087 Acked Read 2 Done Done Sent false
14658 5088 Acked Read 9 Done Done Sent false
14659 5089 Acked Read 8 Done Done Sent false
14660 5090 Acked Read 6 Done Done Sent false
14661 5091 Acked Flush 0 Done Done Sent false
14662 5092 Acked Read 6 Done Done Sent false
14663 5093 Acked Read 10 Done Done Sent false
14664 5094 Acked Read 8 Done Done Sent false
14665 5095 Acked Read 3 Done Done Sent false
14666 5096 Acked Write 4 Done Done Sent false
14667 5097 Acked Read 5 Done Done Sent false
14668 5098 Acked Read 5 Done Done Sent false
14669 5099 Acked Write 9 Done Done Sent false
14670 5100 Acked Flush 0 Done Done Sent false
14671 5101 Acked Read 7 Done Done Sent false
14672 5102 Acked Write 1 Done Done Sent false
14673 5103 Acked Write 5 Done Done Sent false
14674 5104 Acked Read 6 Done Done Sent false
14675 5105 Acked Read 4 Done Done Sent false
14676 5106 Acked Write 6 Done Done Sent false
14677 5107 Acked Read 6 Done Done Sent false
14678 5108 Acked Write 6 Done Done Sent false
14679 5109 Acked Write 8 Done Done Sent false
14680 5110 Acked Flush 0 Done Done Sent false
14681 5111 Acked Write 3 Done Done Sent false
14682 5112 Acked Write 1 Done Done Sent false
14683 5113 Acked Read 7 Done Done Sent false
14684 5114 Acked Read 2 Done Done Sent false
14685 5115 Acked Write 1 Done Done Sent false
14686 5116 Acked Read 2 Done Done Sent false
14687 5117 Acked Flush 0 Done Done Sent false
14688 5118 Acked Flush 0 Done Done Sent false
14689 5119 Acked Read 2 Done Done Sent false
14690 5120 Acked Write 7 Done Done Sent false
14691 5121 Acked Write 4 Done Done Sent false
14692 5122 Acked Read 3 Done Done Sent false
14693 5123 Acked Write 2 Done Done Sent false
14694 5124 Acked Read 7 Done Done Sent false
14695 5125 Acked Read 9 Done Done Sent false
14696 5126 Acked Flush 0 Done Done Sent false
14697 5127 Acked Write 3 Done Done Sent false
14698 5128 Acked Write 10 Done Done Sent false
14699 5129 Acked Read 10 Done Done Sent false
14700 5130 Acked Read 8 Done Done Sent false
14701 5131 Acked Read 5 Done Done Sent false
14702 5132 Acked Read 7 Done Done Sent false
14703 5133 Acked Read 10 Done Done Sent false
14704 5134 Acked Write 3 Done Done Sent false
14705 5135 Acked Read 2 Done Done Sent false
14706 5136 Acked Write 6 Done Done Sent false
14707 5137 Acked Flush 0 Done Done Sent false
14708 5138 Acked Write 10 Done Done Sent false
14709 5139 Acked Read 9 Done Done Sent false
14710 5140 Acked Flush 0 Done Done Sent false
14711 5141 Acked Write 1 Done Done Sent false
14712 5142 Acked Write 3 Done Done Sent false
14713 5143 Acked Read 10 Done Done Sent false
14714 5144 Acked Write 3 Done Done Sent false
14715 5145 Acked Write 10 Done Done Sent false
14716 5146 Acked Read 4 Done Done Sent false
14717 5147 Acked Read 2 Done Done Sent false
14718 5148 Acked Write 10 Done Done Sent false
14719 5149 Acked Write 7 Done Done Sent false
14720 5150 Acked Read 9 Done Done Sent false
14721 5151 Acked Flush 0 Done Done Sent false
14722 5152 Acked Read 2 Done Done Sent false
14723 5153 Acked Write 5 Done Done Sent false
14724 5154 Acked Read 1 Done Done Sent false
14725 5155 Acked Write 9 Done Done Sent false
14726 5156 Acked Write 5 Done Done Sent false
14727 5157 Acked Write 9 Done Done Sent false
14728 5158 Acked Write 7 Done Done Sent false
14729 5159 Acked Write 7 Done Done Sent false
14730 5160 Acked Write 1 Done Done Sent false
14731 5161 Acked Read 3 Done Done Sent false
14732 5162 Acked Write 2 Done Done Sent false
14733 5163 Acked Read 5 Done Done Sent false
14734 5164 Acked Read 5 Done Done Sent false
14735 5165 Acked Read 6 Done Done Sent false
14736 5166 Acked Read 7 Done Done Sent false
14737 5167 Acked Read 6 Done Done Sent false
14738 5168 Acked Read 9 Done Done Sent false
14739 5169 Acked Read 1 Done Done Sent false
14740 5170 Acked Read 1 Done Done Sent false
14741 5171 Acked Read 4 Done Done Sent false
14742 5172 Acked Read 7 Done Done Sent false
14743 5173 Acked Write 2 Done Done Sent false
14744 5174 Acked Write 4 Done Done Sent false
14745 5175 Acked Read 2 Done Done Sent false
14746 5176 Acked Write 10 Done Done Sent false
14747 5177 Acked Read 7 Done Done Sent false
14748 5178 Acked Write 9 Done Done Sent false
14749 5179 Acked Write 8 Done Done Sent false
14750 5180 Acked Write 4 Done Done Sent false
14751 5181 Acked Read 8 Done Done Sent false
14752 5182 Acked Flush 0 Done Done Sent false
14753 5183 Acked Read 10 Done Done Sent false
14754 5184 Acked Read 9 Done Done Sent false
14755 5185 Acked Read 2 Done Done Sent false
14756 5186 Acked Write 4 Done Done Sent false
14757 5187 Acked Write 10 Done Done Sent false
14758 5188 Acked Read 2 Done Done Sent false
14759 5189 Acked Read 10 Done Done Sent false
14760 5190 Acked Read 5 Done Done Sent false
14761 5191 Acked Read 8 Done Done Sent false
14762 5192 Acked Read 8 Done Done Sent false
14763 5193 Acked Write 2 Done Done Sent false
14764 5194 Acked Flush 0 Done Done Sent false
14765 5195 Acked Read 3 Done Done Sent false
14766 5196 Acked Read 4 Done Done Sent false
14767 5197 Acked Write 4 Done Done Sent false
14768 5198 Acked Write 3 Done Done Sent false
14769 5199 Acked Read 2 Done Done Sent false
14770 5200 Acked Read 8 Done Done Sent false
14771 5201 Acked Read 3 Done Done Sent false
14772 5202 Acked Read 7 Done Done Sent false
14773 5203 Acked Write 8 Done Done Sent false
14774 5204 Acked Write 7 Done Done Sent false
14775 5205 Acked Read 6 Done Done Sent false
14776 5206 Acked Read 3 Done Done Sent false
14777 5207 Acked Read 10 Done Done Sent false
14778 5208 Acked Read 9 Done Done Sent false
14779 5209 Acked Write 4 Done Done Sent false
14780 5210 Acked Write 1 Done Done Sent false
14781 5211 Acked Read 8 Done Done Sent false
14782 5212 Acked Write 7 Done Done Sent false
14783 5213 Acked Write 3 Done Done Sent false
14784 5214 Acked Write 4 Done Done Sent false
14785 5215 Acked Read 10 Done Done Sent false
14786 5216 Acked Read 7 Done Done Sent false
14787 5217 Acked Read 4 Done Done Sent false
14788 5218 Acked Write 8 Done Done Sent false
14789 5219 Acked Flush 0 Done Done Sent false
14790 5220 Acked Write 9 Done Done Sent false
14791 5221 Acked Read 6 Done Done Sent false
14792 5222 Acked Write 10 Done Done Sent false
14793 5223 Acked Read 10 Done Done Sent false
14794 5224 Acked Read 2 Done Done Sent false
14795 5225 Acked Flush 0 Done Done Sent false
14796 STATES DS:0 DS:1 DS:2 TOTAL
14797 Sent 0 0 302 302
14798 Done 302 302 0 604
14799 Skipped 0 0 0 0
14800 Error 0 0 0 0
14801 Last Flush: 5225 5225 4923
14802 Downstairs last five completed: 4923 4922 4921 4920 4919
14803 Upstairs last five completed: 5225 5224 5223 5222 5221
14804 CLIENT: Up:0 ds:302 act:2
14805 ----------------------------------------------------------------
14806 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
14807 States: Active Active Offline
14808 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
14809 4924 Acked Read 10 Done Done Sent false
14810 4925 Acked Flush 0 Done Done Sent false
14811 4926 Acked Read 3 Done Done Sent false
14812 4927 Acked Flush 0 Done Done Sent false
14813 4928 Acked Flush 0 Done Done Sent false
14814 4929 Acked Read 1 Done Done Sent false
14815 4930 Acked Read 7 Done Done Sent false
14816 4931 Acked Write 5 Done Done Sent false
14817 4932 Acked Read 6 Done Done Sent false
14818 4933 Acked Write 7 Done Done Sent false
14819 4934 Acked Write 2 Done Done Sent false
14820 4935 Acked Write 2 Done Done Sent false
14821 4936 Acked Flush 0 Done Done Sent false
14822 4937 Acked Write 4 Done Done Sent false
14823 4938 Acked Write 6 Done Done Sent false
14824 4939 Acked Read 8 Done Done Sent false
14825 4940 Acked Read 6 Done Done Sent false
14826 4941 Acked Write 1 Done Done Sent false
14827 4942 Acked Read 1 Done Done Sent false
14828 4943 Acked Read 5 Done Done Sent false
14829 4944 Acked Read 3 Done Done Sent false
14830 4945 Acked Write 2 Done Done Sent false
14831 4946 Acked Write 7 Done Done Sent false
14832 4947 Acked Write 5 Done Done Sent false
14833 4948 Acked Read 4 Done Done Sent false
14834 4949 Acked Write 7 Done Done Sent false
14835 4950 Acked Write 6 Done Done Sent false
14836 4951 Acked Read 3 Done Done Sent false
14837 4952 Acked Write 10 Done Done Sent false
14838 4953 Acked Flush 0 Done Done Sent false
14839 4954 Acked Read 2 Done Done Sent false
14840 4955 Acked Write 8 Done Done Sent false
14841 4956 Acked Write 6 Done Done Sent false
14842 4957 Acked Write 1 Done Done Sent false
14843 4958 Acked Write 7 Done Done Sent false
14844 4959 Acked Read 5 Done Done Sent false
14845 4960 Acked Read 1 Done Done Sent false
14846 4961 Acked Write 10 Done Done Sent false
14847 4962 Acked Read 2 Done Done Sent false
14848 4963 Acked Read 5 Done Done Sent false
14849 4964 Acked Read 10 Done Done Sent false
14850 4965 Acked Write 6 Done Done Sent false
14851 4966 Acked Read 9 Done Done Sent false
14852 4967 Acked Write 4 Done Done Sent false
14853 4968 Acked Write 2 Done Done Sent false
14854 4969 Acked Read 10 Done Done Sent false
14855 4970 Acked Write 4 Done Done Sent false
14856 4971 Acked Write 5 Done Done Sent false
14857 4972 Acked Read 2 Done Done Sent false
14858 4973 Acked Read 5 Done Done Sent false
14859 4974 Acked Write 6 Done Done Sent false
14860 4975 Acked Flush 0 Done Done Sent false
14861 4976 Acked Read 7 Done Done Sent false
14862 4977 Acked Read 9 Done Done Sent false
14863 4978 Acked Read 3 Done Done Sent false
14864 4979 Acked Write 10 Done Done Sent false
14865 4980 Acked Flush 0 Done Done Sent false
14866 4981 Acked Read 9 Done Done Sent false
14867 4982 Acked Flush 0 Done Done Sent false
14868 4983 Acked Read 9 Done Done Sent false
14869 4984 Acked Read 2 Done Done Sent false
14870 4985 Acked Read 7 Done Done Sent false
14871 4986 Acked Read 7 Done Done Sent false
14872 4987 Acked Write 5 Done Done Sent false
14873 4988 Acked Write 8 Done Done Sent false
14874 4989 Acked Write 9 Done Done Sent false
14875 4990 Acked Read 9 Done Done Sent false
14876 4991 Acked Read 5 Done Done Sent false
14877 4992 Acked Write 8 Done Done Sent false
14878 4993 Acked Write 6 Done Done Sent false
14879 4994 Acked Write 10 Done Done Sent false
14880 4995 Acked Read 1 Done Done Sent false
14881 4996 Acked Read 10 Done Done Sent false
14882 4997 Acked Read 4 Done Done Sent false
14883 4998 Acked Write 3 Done Done Sent false
14884 4999 Acked Read 8 Done Done Sent false
14885 5000 Acked Write 9 Done Done Sent false
14886 5001 Acked Read 6 Done Done Sent false
14887 5002 Acked Flush 0 Done Done Sent false
14888 5003 Acked Write 8 Done Done Sent false
14889 5004 Acked Read 6 Done Done Sent false
14890 5005 Acked Flush 0 Done Done Sent false
14891 5006 Acked Write 1 Done Done Sent false
14892 5007 Acked Write 9 Done Done Sent false
14893 5008 Acked Read 6 Done Done Sent false
14894 5009 Acked Read 3 Done Done Sent false
14895 5010 Acked Read 3 Done Done Sent false
14896 5011 Acked Read 8 Done Done Sent false
14897 5012 Acked Write 9 Done Done Sent false
14898 5013 Acked Read 5 Done Done Sent false
14899 5014 Acked Write 10 Done Done Sent false
14900 5015 Acked Read 9 Done Done Sent false
14901 5016 Acked Write 10 Done Done Sent false
14902 5017 Acked Read 9 Done Done Sent false
14903 5018 Acked Read 5 Done Done Sent false
14904 5019 Acked Write 5 Done Done Sent false
14905 5020 Acked Read 7 Done Done Sent false
14906 5021 Acked Write 8 Done Done Sent false
14907 5022 Acked Write 10 Done Done Sent false
14908 5023 Acked Flush 0 Done Done Sent false
14909 5024 Acked Write 4 Done Done Sent false
14910 5025 Acked Write 3 Done Done Sent false
14911 5026 Acked Read 10 Done Done Sent false
14912 5027 Acked Read 6 Done Done Sent false
14913 5028 Acked Read 6 Done Done Sent false
14914 5029 Acked Flush 0 Done Done Sent false
14915 5030 Acked Flush 0 Done Done Sent false
14916 5031 Acked Read 4 Done Done Sent false
14917 5032 Acked Write 9 Done Done Sent false
14918 5033 Acked Read 5 Done Done Sent false
14919 5034 Acked Flush 0 Done Done Sent false
14920 5035 Acked Read 6 Done Done Sent false
14921 5036 Acked Write 1 Done Done Sent false
14922 5037 Acked Write 8 Done Done Sent false
14923 5038 Acked Read 9 Done Done Sent false
14924 5039 Acked Write 6 Done Done Sent false
14925 5040 Acked Read 10 Done Done Sent false
14926 5041 Acked Read 10 Done Done Sent false
14927 5042 Acked Read 8 Done Done Sent false
14928 5043 Acked Write 6 Done Done Sent false
14929 5044 Acked Flush 0 Done Done Sent false
14930 5045 Acked Read 6 Done Done Sent false
14931 5046 Acked Write 4 Done Done Sent false
14932 5047 Acked Flush 0 Done Done Sent false
14933 5048 Acked Read 8 Done Done Sent false
14934 5049 Acked Write 3 Done Done Sent false
14935 5050 Acked Read 2 Done Done Sent false
14936 5051 Acked Write 3 Done Done Sent false
14937 5052 Acked Read 5 Done Done Sent false
14938 5053 Acked Read 2 Done Done Sent false
14939 5054 Acked Write 7 Done Done Sent false
14940 5055 Acked Read 5 Done Done Sent false
14941 5056 Acked Read 2 Done Done Sent false
14942 5057 Acked Read 9 Done Done Sent false
14943 5058 Acked Flush 0 Done Done Sent false
14944 5059 Acked Write 4 Done Done Sent false
14945 5060 Acked Write 6 Done Done Sent false
14946 5061 Acked Read 3 Done Done Sent false
14947 5062 Acked Read 8 Done Done Sent false
14948 5063 Acked Write 6 Done Done Sent false
14949 5064 Acked Write 5 Done Done Sent false
14950 5065 Acked Read 2 Done Done Sent false
14951 5066 Acked Flush 0 Done Done Sent false
14952 5067 Acked Flush 0 Done Done Sent false
14953 5068 Acked Read 10 Done Done Sent false
14954 5069 Acked Write 1 Done Done Sent false
14955 5070 Acked Write 10 Done Done Sent false
14956 5071 Acked Read 2 Done Done Sent false
14957 5072 Acked Write 6 Done Done Sent false
14958 5073 Acked Read 10 Done Done Sent false
14959 5074 Acked Write 2 Done Done Sent false
14960 5075 Acked Write 4 Done Done Sent false
14961 5076 Acked Read 4 Done Done Sent false
14962 5077 Acked Write 2 Done Done Sent false
14963 5078 Acked Read 9 Done Done Sent false
14964 5079 Acked Write 2 Done Done Sent false
14965 5080 Acked Read 5 Done Done Sent false
14966 5081 Acked Read 6 Done Done Sent false
14967 5082 Acked Write 9 Done Done Sent false
14968 5083 Acked Write 6 Done Done Sent false
14969 5084 Acked Read 9 Done Done Sent false
14970 5085 Acked Write 7 Done Done Sent false
14971 5086 Acked Read 1 Done Done Sent false
14972 5087 Acked Read 2 Done Done Sent false
14973 5088 Acked Read 9 Done Done Sent false
14974 5089 Acked Read 8 Done Done Sent false
14975 5090 Acked Read 6 Done Done Sent false
14976 5091 Acked Flush 0 Done Done Sent false
14977 5092 Acked Read 6 Done Done Sent false
14978 5093 Acked Read 10 Done Done Sent false
14979 5094 Acked Read 8 Done Done Sent false
14980 5095 Acked Read 3 Done Done Sent false
14981 5096 Acked Write 4 Done Done Sent false
14982 5097 Acked Read 5 Done Done Sent false
14983 5098 Acked Read 5 Done Done Sent false
14984 5099 Acked Write 9 Done Done Sent false
14985 5100 Acked Flush 0 Done Done Sent false
14986 5101 Acked Read 7 Done Done Sent false
14987 5102 Acked Write 1 Done Done Sent false
14988 5103 Acked Write 5 Done Done Sent false
14989 5104 Acked Read 6 Done Done Sent false
14990 5105 Acked Read 4 Done Done Sent false
14991 5106 Acked Write 6 Done Done Sent false
14992 5107 Acked Read 6 Done Done Sent false
14993 5108 Acked Write 6 Done Done Sent false
14994 5109 Acked Write 8 Done Done Sent false
14995 5110 Acked Flush 0 Done Done Sent false
14996 5111 Acked Write 3 Done Done Sent false
14997 5112 Acked Write 1 Done Done Sent false
14998 5113 Acked Read 7 Done Done Sent false
14999 5114 Acked Read 2 Done Done Sent false
15000 5115 Acked Write 1 Done Done Sent false
15001 5116 Acked Read 2 Done Done Sent false
15002 5117 Acked Flush 0 Done Done Sent false
15003 5118 Acked Flush 0 Done Done Sent false
15004 5119 Acked Read 2 Done Done Sent false
15005 5120 Acked Write 7 Done Done Sent false
15006 5121 Acked Write 4 Done Done Sent false
15007 5122 Acked Read 3 Done Done Sent false
15008 5123 Acked Write 2 Done Done Sent false
15009 5124 Acked Read 7 Done Done Sent false
15010 5125 Acked Read 9 Done Done Sent false
15011 5126 Acked Flush 0 Done Done Sent false
15012 5127 Acked Write 3 Done Done Sent false
15013 5128 Acked Write 10 Done Done Sent false
15014 5129 Acked Read 10 Done Done Sent false
15015 5130 Acked Read 8 Done Done Sent false
15016 5131 Acked Read 5 Done Done Sent false
15017 5132 Acked Read 7 Done Done Sent false
15018 5133 Acked Read 10 Done Done Sent false
15019 5134 Acked Write 3 Done Done Sent false
15020 5135 Acked Read 2 Done Done Sent false
15021 5136 Acked Write 6 Done Done Sent false
15022 5137 Acked Flush 0 Done Done Sent false
15023 5138 Acked Write 10 Done Done Sent false
15024 5139 Acked Read 9 Done Done Sent false
15025 5140 Acked Flush 0 Done Done Sent false
15026 5141 Acked Write 1 Done Done Sent false
15027 5142 Acked Write 3 Done Done Sent false
15028 5143 Acked Read 10 Done Done Sent false
15029 5144 Acked Write 3 Done Done Sent false
15030 5145 Acked Write 10 Done Done Sent false
15031 5146 Acked Read 4 Done Done Sent false
15032 5147 Acked Read 2 Done Done Sent false
15033 5148 Acked Write 10 Done Done Sent false
15034 5149 Acked Write 7 Done Done Sent false
15035 5150 Acked Read 9 Done Done Sent false
15036 5151 Acked Flush 0 Done Done Sent false
15037 5152 Acked Read 2 Done Done Sent false
15038 5153 Acked Write 5 Done Done Sent false
15039 5154 Acked Read 1 Done Done Sent false
15040 5155 Acked Write 9 Done Done Sent false
15041 5156 Acked Write 5 Done Done Sent false
15042 5157 Acked Write 9 Done Done Sent false
15043 5158 Acked Write 7 Done Done Sent false
15044 5159 Acked Write 7 Done Done Sent false
15045 5160 Acked Write 1 Done Done Sent false
15046 5161 Acked Read 3 Done Done Sent false
15047 5162 Acked Write 2 Done Done Sent false
15048 5163 Acked Read 5 Done Done Sent false
15049 5164 Acked Read 5 Done Done Sent false
15050 5165 Acked Read 6 Done Done Sent false
15051 5166 Acked Read 7 Done Done Sent false
15052 5167 Acked Read 6 Done Done Sent false
15053 5168 Acked Read 9 Done Done Sent false
15054 5169 Acked Read 1 Done Done Sent false
15055 5170 Acked Read 1 Done Done Sent false
15056 5171 Acked Read 4 Done Done Sent false
15057 5172 Acked Read 7 Done Done Sent false
15058 5173 Acked Write 2 Done Done Sent false
15059 5174 Acked Write 4 Done Done Sent false
15060 5175 Acked Read 2 Done Done Sent false
15061 5176 Acked Write 10 Done Done Sent false
15062 5177 Acked Read 7 Done Done Sent false
15063 5178 Acked Write 9 Done Done Sent false
15064 5179 Acked Write 8 Done Done Sent false
15065 5180 Acked Write 4 Done Done Sent false
15066 5181 Acked Read 8 Done Done Sent false
15067 5182 Acked Flush 0 Done Done Sent false
15068 5183 Acked Read 10 Done Done Sent false
15069 5184 Acked Read 9 Done Done Sent false
15070 5185 Acked Read 2 Done Done Sent false
15071 5186 Acked Write 4 Done Done Sent false
15072 5187 Acked Write 10 Done Done Sent false
15073 5188 Acked Read 2 Done Done Sent false
15074 5189 Acked Read 10 Done Done Sent false
15075 5190 Acked Read 5 Done Done Sent false
15076 5191 Acked Read 8 Done Done Sent false
15077 5192 Acked Read 8 Done Done Sent false
15078 5193 Acked Write 2 Done Done Sent false
15079 5194 Acked Flush 0 Done Done Sent false
15080 5195 Acked Read 3 Done Done Sent false
15081 5196 Acked Read 4 Done Done Sent false
15082 5197 Acked Write 4 Done Done Sent false
15083 5198 Acked Write 3 Done Done Sent false
15084 5199 Acked Read 2 Done Done Sent false
15085 5200 Acked Read 8 Done Done Sent false
15086 5201 Acked Read 3 Done Done Sent false
15087 5202 Acked Read 7 Done Done Sent false
15088 5203 Acked Write 8 Done Done Sent false
15089 5204 Acked Write 7 Done Done Sent false
15090 5205 Acked Read 6 Done Done Sent false
15091 5206 Acked Read 3 Done Done Sent false
15092 5207 Acked Read 10 Done Done Sent false
15093 5208 Acked Read 9 Done Done Sent false
15094 5209 Acked Write 4 Done Done Sent false
15095 5210 Acked Write 1 Done Done Sent false
15096 5211 Acked Read 8 Done Done Sent false
15097 5212 Acked Write 7 Done Done Sent false
15098 5213 Acked Write 3 Done Done Sent false
15099 5214 Acked Write 4 Done Done Sent false
15100 5215 Acked Read 10 Done Done Sent false
15101 5216 Acked Read 7 Done Done Sent false
15102 5217 Acked Read 4 Done Done Sent false
15103 5218 Acked Write 8 Done Done Sent false
15104 5219 Acked Flush 0 Done Done Sent false
15105 5220 Acked Write 9 Done Done Sent false
15106 5221 Acked Read 6 Done Done Sent false
15107 5222 Acked Write 10 Done Done Sent false
15108 5223 Acked Read 10 Done Done Sent false
15109 5224 Acked Read 2 Done Done Sent false
15110 5225 Acked Flush 0 Done Done Sent false
15111 STATES DS:0 DS:1 DS:2 TOTAL
15112 Sent 0 0 302 302
15113 Done 302 302 0 604
15114 Skipped 0 0 0 0
15115 Error 0 0 0 0
15116 Last Flush: 5225 5225 4923
15117 Downstairs last five completed: 4923 4922 4921 4920 4919
15118 Upstairs last five completed: 5225 5224 5223 5222 5221
15119 CLIENT: Up:0 ds:302 act:2
151202024-11-28T04:53:30.170ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
151212024-11-28T04:53:30.170ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:51363) = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
151222024-11-28T04:53:30.171ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
151232024-11-28T04:53:30.172ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
151242024-11-28T04:53:30.172ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
151252024-11-28T04:53:30.172ZINFOcrucible: send last flush ID to this DS: 4923 = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
151262024-11-28T04:53:30.172ZINFOcrucible: Replied this last flush ID: 4923 = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
151272024-11-28T04:53:30.172ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
151282024-11-28T04:53:30.172ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
151292024-11-28T04:53:30.172ZINFOcrucible: [2] client re-new 302 jobs since flush 4923 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
151302024-11-28T04:53:30.172ZINFOcrucible: [2] Replayed 302 jobs since flush: 4923 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
15131 ----------------------------------------------------------------
15132 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
15133 Downstairs last five completed: 5225 5224 5223 5222 5221
15134 Upstairs last five completed: 5225 5224 5223 5222 5221
15135 CLIENT: Up:0 ds:0 act:3
15136 Replay: All jobs finished
15137 Main task has work to do, go find it
15138 got dsc Stop(2)
15139 stop 2
15140 [8830] Got stop action so:false kr:false
15141 Nov 28 04:53:30.421 INFO request completed, latency_us: 314, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 5ae38276-3350-49a1-bc2e-1dce8d01e712, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
151422024-11-28T04:53:30.422ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
151432024-11-28T04:53:30.422ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
151442024-11-28T04:53:30.422ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
151452024-11-28T04:53:30.422ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
151462024-11-28T04:53:30.422ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
151472024-11-28T04:53:30.422ZINFOcrucible: client task is exiting = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
15148 Nov 28 04:53:30.422 INFO request completed, latency_us: 131, response_code: 200, uri: /state/cid/2, method: GET, req_id: 49af3530-6f71-4508-8261-64f785b5ff5c, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
15149 [8830] Exited with: ExitStatus(unix_wait_status(9))
15150 [8830][2] reports Exit
15151 Nov 28 04:53:34.424 INFO request completed, latency_us: 213, response_code: 200, uri: /state/cid/2, method: GET, req_id: 2b3f56be-949d-4726-b68c-e30908701c14, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
15152 001/300 Write block 4518 len 20480 data: 3 3 3 3 4
15153 002/300 Read block 3422 len 32768
15154 003/300 Write block 2576 len 32768 data: 3 3 3 3 3 3 3 3
15155 004/300 Flush
15156 005/300 Write block 1801 len 32768 data: 2 2 2 2 2 3 3 2
15157 006/300 Read block 2359 len 16384
15158 007/300 Read block 1318 len 40960
15159 008/300 Read block 1360 len 24576
15160 009/300 Write block 4588 len 4096 data: 2
15161 010/300 Write block 3341 len 20480 data: 4 5 4 4 3
15162 011/300 Write block 3695 len 16384 data: 7 7 6 5
15163 012/300 Read block 711 len 20480
15164 013/300 Read block 1699 len 8192
15165 014/300 Write block 2402 len 16384 data: 2 2 2 2
15166 015/300 Write block 3779 len 32768 data: 4 5 4 4 4 3 3 3
15167 016/300 Write block 2845 len 32768 data: 3 3 4 4 4 5 5 5
15168 017/300 Write block 2054 len 32768 data: 4 4 4 3 4 4 4 4
15169 018/300 Write block 2145 len 8192 data: 4 4
15170 019/300 Write block 1609 len 8192 data: 5 5
15171 020/300 Read block 2161 len 40960
15172 021/300 Read block 534 len 40960
15173 022/300 Read block 3736 len 16384
15174 023/300 Write block 2239 len 28672 data: 2 2 4 4 4 4 4
15175 024/300 Read block 3414 len 28672
15176 025/300 Read block 4640 len 4096
15177 026/300 Read block 4290 len 28672
15178 027/300 Write block 1423 len 40960 data: 4 4 4 3 4 4 4 4 3 3
15179 028/300 Read block 556 len 16384
15180 029/300 Read block 2839 len 28672
15181 030/300 Flush
15182 031/300 Read block 4696 len 20480
15183 032/300 Read block 4237 len 4096
15184 033/300 Read block 2514 len 36864
15185 034/300 Read block 2935 len 40960
15186 035/300 Flush
15187 036/300 Flush
15188 037/300 Read block 2480 len 16384
15189 038/300 Flush
15190 039/300 Read block 567 len 40960
15191 040/300 Write block 2233 len 40960 data: 5 5 6 6 5 3 3 3 5 5
15192 041/300 Flush
15193 042/300 Write block 1702 len 20480 data: 5 8 8 10 11
15194 043/300 Write block 517 len 4096 data: 4
15195 044/300 Read block 2224 len 40960
15196 045/300 Read block 1865 len 12288
15197 046/300 Write block 2213 len 28672 data: 5 5 3 3 3 3 2
15198 047/300 Flush
15199 048/300 Write block 2825 len 32768 data: 2 2 2 2 4 4 4 4
15200 049/300 Write block 2961 len 4096 data: 5
15201 050/300 Write block 3767 len 8192 data: 4 2
15202 051/300 Read block 2310 len 8192
15203 052/300 Write block 602 len 4096 data: 5
15204 053/300 Flush
15205 054/300 Read block 4894 len 24576
15206 055/300 Write block 2220 len 32768 data: 2 2 2 2 3 3 4 4
15207 056/300 Write block 3792 len 16384 data: 6 5 4 4
15208 057/300 Write block 2734 len 36864 data: 3 4 4 4 4 3 3 4 2
15209 058/300 Read block 60 len 20480
15210 059/300 Read block 4634 len 28672
15211 060/300 Write block 4532 len 36864 data: 5 2 4 5 4 3 3 2 2
15212 061/300 Read block 3709 len 16384
15213 062/300 Write block 2355 len 8192 data: 3 3
15214 063/300 Read block 1554 len 12288
15215 064/300 Read block 691 len 36864
15216 065/300 Read block 1290 len 24576
15217 066/300 Read block 3412 len 20480
15218 067/300 Read block 2465 len 12288
15219 068/300 Read block 2665 len 20480
15220 069/300 Read block 153 len 32768
15221 070/300 Read block 2402 len 40960
15222 071/300 Read block 3034 len 40960
15223 072/300 Read block 3796 len 32768
15224 073/300 Read block 274 len 24576
15225 074/300 Write block 3960 len 8192 data: 4 5
15226 075/300 Read block 100 len 24576
15227 076/300 Write block 3356 len 28672 data: 2 2 2 2 3 3 3
15228 077/300 Write block 332 len 32768 data: 2 2 4 7 6 6 5 4
15229 078/300 Write block 3002 len 32768 data: 4 4 5 7 7 6 5 4
15230 079/300 Write block 2717 len 4096 data: 7
15231 080/300 Write block 3353 len 32768 data: 3 3 2 3 3 3 3 4
15232 081/300 Write block 4268 len 12288 data: 6 6 6
15233 082/300 Write block 143 len 40960 data: 5 4 2 2 2 2 3 3 5 4
15234 083/300 Flush
15235 084/300 Write block 1403 len 28672 data: 4 4 5 4 3 2 2
15236 085/300 Write block 2696 len 40960 data: 5 5 4 3 3 5 5 5 4 5
15237 086/300 Write block 4596 len 40960 data: 5 5 5 5 5 4 4 3 5 5
15238 087/300 Write block 583 len 24576 data: 2 2 2 2 2 2
15239 088/300 Read block 1170 len 36864
15240 089/300 Read block 3489 len 40960
15241 090/300 Write block 4962 len 40960 data: 2 3 4 4 5 6 6 5 5 5
15242 091/300 Write block 3031 len 28672 data: 2 4 5 6 6 6 6
15243 092/300 Read block 1758 len 24576
15244 093/300 Write block 3317 len 8192 data: 6 6
15245 094/300 Read block 1246 len 40960
15246 095/300 Write block 468 len 12288 data: 5 6 6
15247 096/300 Read block 2728 len 8192
15248 097/300 Read block 4439 len 20480
15249 098/300 Write block 1057 len 20480 data: 3 2 3 3 3
15250 099/300 Write block 4978 len 16384 data: 3 3 4 4
15251 100/300 Read block 1863 len 28672
15252 101/300 Read block 2881 len 36864
15253 102/300 Write block 2462 len 40960 data: 4 5 4 4 3 4 4 4 4 5
15254 103/300 Read block 4533 len 36864
15255 104/300 Write block 57 len 28672 data: 4 3 3 4 5 4 4
15256 105/300 Write block 4889 len 40960 data: 5 3 4 3 3 3 3 3 2 2
15257 106/300 Flush
15258 107/300 Write block 4470 len 40960 data: 6 6 5 5 6 5 5 4 4 4
15259 108/300 Read block 4779 len 24576
15260 109/300 Write block 1015 len 12288 data: 3 3 2
15261 110/300 Write block 3905 len 24576 data: 5 3 3 3 3 2
15262 111/300 Write block 767 len 28672 data: 3 3 4 4 3 2 3
15263 112/300 Write block 1154 len 8192 data: 5 4
15264 113/300 Flush
15265 114/300 Read block 4392 len 24576
15266 115/300 Read block 1828 len 20480
15267 116/300 Write block 108 len 20480 data: 6 7 5 3 3
15268 117/300 Write block 595 len 40960 data: 5 6 7 6 6 4 5 6 4 3
15269 118/300 Write block 1616 len 40960 data: 3 3 3 2 3 3 3 4 4 4
15270 119/300 Read block 1436 len 8192
15271 120/300 Read block 1345 len 20480
15272 121/300 Flush
15273 122/300 Write block 1334 len 20480 data: 3 3 3 3 3
15274 123/300 Write block 2333 len 16384 data: 6 7 5 6
15275 124/300 Read block 1287 len 16384
15276 125/300 Read block 2531 len 20480
15277 126/300 Read block 183 len 20480
15278 127/300 Write block 2183 len 12288 data: 3 3 3
15279 128/300 Read block 786 len 8192
15280 129/300 Write block 1805 len 8192 data: 3 4
15281 130/300 Read block 334 len 12288
15282 131/300 Read block 3208 len 28672
15283 132/300 Read block 3084 len 4096
15284 133/300 Write block 338 len 16384 data: 6 5 5 6
15285 134/300 Flush
15286 135/300 Read block 491 len 20480
15287 136/300 Write block 2081 len 16384 data: 5 4 5 6
15288 137/300 Read block 4222 len 28672
15289 138/300 Read block 697 len 4096
15290 139/300 Write block 3756 len 16384 data: 2 2 2 3
15291 140/300 Write block 2656 len 24576 data: 3 3 3 3 3 3
15292 141/300 Read block 4120 len 20480
15293 142/300 Write block 868 len 36864 data: 4 4 4 4 4 5 4 3 4
15294 143/300 Read block 4279 len 16384
15295 144/300 Read block 1032 len 24576
15296 145/300 Write block 570 len 28672 data: 2 2 3 3 3 3 2
15297 146/300 Write block 1751 len 40960 data: 5 4 4 4 5 4 3 3 2 2
15298 147/300 Read block 2092 len 16384
15299 148/300 Flush
15300 149/300 Read block 3535 len 28672
15301 150/300 Write block 4028 len 12288 data: 3 3 3
15302 151/300 Read block 3093 len 20480
15303 152/300 Read block 2694 len 24576
15304 153/300 Read block 2509 len 12288
15305 154/300 Read block 3712 len 20480
15306 155/300 Write block 2248 len 12288 data: 4 4 3
15307 156/300 Write block 3341 len 28672 data: 5 6 5 5 4 3 3
15308 157/300 Write block 2008 len 40960 data: 3 3 3 3 4 3 3 2 2 2
15309 158/300 Read block 4227 len 8192
15310 159/300 Flush
15311 160/300 Write block 3592 len 40960 data: 3 3 4 4 3 3 4 4 5 5
15312 161/300 Read block 4994 len 4096
15313 162/300 Read block 777 len 32768
15314 163/300 Read block 3263 len 40960
15315 164/300 Read block 2036 len 8192
15316 165/300 Write block 4253 len 4096 data: 3
15317 166/300 Read block 756 len 36864
15318 167/300 Write block 1576 len 36864 data: 4 4 4 4 4 4 3 4 3
15319 168/300 Read block 387 len 24576
15320 169/300 Read block 3210 len 8192
15321 170/300 Read block 3739 len 8192
15322 171/300 Write block 1394 len 40960 data: 4 3 4 3 4 5 3 4 5 5
15323 172/300 Read block 524 len 28672
15324 173/300 Write block 4568 len 24576 data: 4 5 4 4 4 4
15325 174/300 Write block 272 len 40960 data: 3 3 3 4 4 3 4 4 4 4
15326 175/300 Write block 1463 len 40960 data: 4 4 5 5 6 6 5 4 4 3
15327 176/300 Write block 730 len 4096 data: 4
15328 177/300 Read block 3014 len 28672
15329 178/300 Write block 3300 len 12288 data: 2 2 3
15330 179/300 Write block 1935 len 4096 data: 4
15331 180/300 Read block 4036 len 8192
15332 181/300 Write block 1361 len 20480 data: 4 4 4 5 5
15333 182/300 Write block 4390 len 16384 data: 4 3 3 2
15334 183/300 Read block 4404 len 4096
15335 184/300 Read block 4411 len 12288
15336 185/300 Write block 3587 len 36864 data: 3 3 3 3 3 4 4 5 5
15337 186/300 Write block 4075 len 28672 data: 4 4 5 7 8 7 7
15338 187/300 Write block 4026 len 36864 data: 3 3 4 4 4 4 4 6 4
15339 188/300 Write block 1321 len 20480 data: 2 3 3 3 3
15340 189/300 Read block 2765 len 28672
15341 190/300 Write block 544 len 32768 data: 5 4 4 5 4 4 3 3
15342 191/300 Write block 135 len 32768 data: 4 4 4 4 4 5 5 5
15343 192/300 Write block 1568 len 16384 data: 2 2 2 2
15344 193/300 Read block 2265 len 8192
15345 194/300 Write block 1233 len 24576 data: 3 3 3 3 3 3
15346 195/300 Read block 1060 len 32768
15347 196/300 Read block 704 len 16384
15348 197/300 Read block 3335 len 32768
15349 198/300 Read block 2151 len 24576
15350 199/300 Read block 4920 len 20480
15351 200/300 Read block 1581 len 12288
15352 201/300 Write block 3674 len 32768 data: 5 5 4 3 2 2 2 2
15353 202/300 Read block 2110 len 8192
15354 203/300 Write block 3105 len 4096 data: 2
15355 204/300 Write block 1211 len 24576 data: 4 3 3 3 3 3
15356 205/300 Read block 3908 len 32768
15357 206/300 Write block 4436 len 40960 data: 2 2 2 2 2 2 3 3 5 8
15358 207/300 Read block 3127 len 36864
15359 208/300 Write block 3160 len 4096 data: 3
15360 209/300 Write block 1522 len 8192 data: 3 3
15361 210/300 Write block 2183 len 40960 data: 4 4 4 3 4 3 3 4 3 5
15362 211/300 Flush
15363 212/300 Flush
15364 213/300 Write block 2594 len 12288 data: 2 3 3
15365 214/300 Read block 2268 len 24576
15366 215/300 Write block 1980 len 36864 data: 5 5 6 5 5 4 3 2 3
15367 216/300 Read block 129 len 4096
15368 217/300 Write block 2661 len 12288 data: 4 4 4
15369 218/300 Write block 938 len 8192 data: 3 3
15370 219/300 Write block 4775 len 24576 data: 5 4 4 4 4 4
15371 220/300 Read block 1562 len 28672
15372 221/300 Read block 958 len 40960
15373 222/300 Write block 3587 len 8192 data: 4 4
15374 223/300 Read block 1041 len 16384
15375 224/300 Read block 1398 len 28672
15376 225/300 Read block 406 len 8192
15377 226/300 Write block 2469 len 32768 data: 5 5 6 5 5 5 5 6
15378 227/300 Read block 662 len 4096
15379 228/300 Flush
15380 229/300 Read block 4189 len 40960
15381 230/300 Read block 3397 len 28672
15382 231/300 Read block 2612 len 40960
15383 232/300 Read block 567 len 40960
15384 233/300 Read block 2229 len 16384
15385 234/300 Read block 3477 len 32768
15386 235/300 Read block 2003 len 24576
15387 236/300 Write block 970 len 8192 data: 5 5
15388 237/300 Read block 215 len 40960
15389 238/300 Write block 2873 len 36864 data: 3 4 4 5 6 6 6 6 6
15390 239/300 Write block 553 len 40960 data: 4 5 5 5 5 4 4 4 3 4
15391 240/300 Read block 4109 len 8192
15392 241/300 Read block 2656 len 20480
15393 242/300 Read block 4757 len 40960
15394 243/300 Write block 2587 len 12288 data: 4 3 2
15395 244/300 Read block 3651 len 24576
15396 245/300 Read block 2410 len 24576
15397 246/300 Read block 3940 len 36864
15398 247/300 Flush
15399 248/300 Read block 1543 len 36864
15400 249/300 Write block 2706 len 4096 data: 6
15401 250/300 Read block 1649 len 8192
15402 251/300 Read block 209 len 36864
15403 252/300 Read block 909 len 16384
15404 253/300 Read block 4806 len 24576
15405 254/300 Read block 3063 len 12288
15406 255/300 Flush
15407 256/300 Write block 2089 len 20480 data: 3 3 3 4 5
15408 257/300 Read block 1242 len 4096
15409 258/300 Write block 151 len 16384 data: 6 5 5 5
15410 259/300 Read block 3526 len 20480
15411 260/300 Read block 1101 len 4096
15412 261/300 Read block 4587 len 16384
15413 262/300 Read block 1423 len 12288
15414 263/300 Write block 338 len 28672 data: 7 6 6 7 6 5 3
15415 264/300 Write block 1479 len 28672 data: 4 4 4 4 3 2 3
15416 265/300 Read block 2236 len 4096
15417 266/300 Write block 4786 len 28672 data: 2 2 2 2 3 3 4
15418 267/300 Read block 4057 len 12288
15419 268/300 Write block 135 len 24576 data: 5 5 5 5 5 6
15420 269/300 Write block 109 len 8192 data: 8 6
15421 270/300 Write block 446 len 16384 data: 5 4 3 3
15422 271/300 Read block 2795 len 4096
15423 272/300 Flush
15424 273/300 Write block 458 len 4096 data: 2
15425 274/300 Flush
15426 275/300 Write block 3046 len 36864 data: 4 4 4 5 5 5 5 6 5
15427 276/300 Write block 4313 len 40960 data: 3 3 3 3 2 2 2 2 2 3
15428 277/300 Flush
15429 278/300 Read block 4148 len 32768
15430 279/300 Flush
15431 280/300 Read block 3377 len 32768
15432 281/300 Read block 4800 len 12288
15433 282/300 Write block 3479 len 40960 data: 4 4 3 3 3 3 4 4 4 4
15434 283/300 Write block 2419 len 20480 data: 5 5 5 5 4
15435 284/300 Read block 3848 len 8192
15436 285/300 Read block 4527 len 32768
15437 286/300 Read block 979 len 32768
15438 287/300 Write block 423 len 16384 data: 2 2 2 2
15439 288/300 Read block 1806 len 20480
15440 289/300 Write block 3738 len 4096 data: 3
15441 290/300 Write block 3765 len 36864 data: 4 4 5 3 2 2 2 3 3
15442 291/300 Read block 2687 len 20480
15443 292/300 Write block 256 len 40960 data: 3 3 4 3 4 5 5 4 4 4
15444 293/300 Write block 2015 len 40960 data: 3 3 3 2 2 2 4 5 5 5
15445 294/300 Read block 993 len 36864
15446 295/300 Write block 3699 len 24576 data: 5 3 3 3 3 3
15447 296/300 Write block 1038 len 24576 data: 2 2 3 3 3 3
15448 297/300 Write block 2293 len 40960 data: 3 3 3 3 4 3 2 2 2 2
15449 298/300 Read block 3380 len 32768
15450 299/300 Flush
15451 300/300 Write block 268 len 36864 data: 2 2 2 3 4 4 4 5 5
15452 301/300 Read block 2475 len 8192
15453 Main task has work to do, go find it
15454 got dsc Start(2)
15455 start 2
15456 Nov 28 04:53:34.663 INFO request completed, latency_us: 227, response_code: 204, uri: /start/cid/2, method: POST, req_id: 310a589c-7027-4ec3-a2f7-8c4879d2343f, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
15457 Replay: started 2, returned:Ok(())
15458 ----------------------------------------------------------------
15459 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
15460 States: Active Active Offline
15461 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
15462 5226 Acked Write 5 Done Done Sent false
15463 5227 Acked Read 8 Done Done Sent false
15464 5228 Acked Write 8 Done Done Sent false
15465 5229 Acked Flush 0 Done Done Sent false
15466 5230 Acked Write 8 Done Done Sent false
15467 5231 Acked Read 4 Done Done Sent false
15468 5232 Acked Read 10 Done Done Sent false
15469 5233 Acked Read 6 Done Done Sent false
15470 5234 Acked Write 1 Done Done Sent false
15471 5235 Acked Write 5 Done Done Sent false
15472 5236 Acked Write 4 Done Done Sent false
15473 5237 Acked Read 5 Done Done Sent false
15474 5238 Acked Read 2 Done Done Sent false
15475 5239 Acked Write 4 Done Done Sent false
15476 5240 Acked Write 8 Done Done Sent false
15477 5241 Acked Write 8 Done Done Sent false
15478 5242 Acked Write 8 Done Done Sent false
15479 5243 Acked Write 2 Done Done Sent false
15480 5244 Acked Write 2 Done Done Sent false
15481 5245 Acked Read 10 Done Done Sent false
15482 5246 Acked Read 10 Done Done Sent false
15483 5247 Acked Read 4 Done Done Sent false
15484 5248 Acked Write 7 Done Done Sent false
15485 5249 Acked Read 7 Done Done Sent false
15486 5250 Acked Read 1 Done Done Sent false
15487 5251 Acked Read 7 Done Done Sent false
15488 5252 Acked Write 10 Done Done Sent false
15489 5253 Acked Read 4 Done Done Sent false
15490 5254 Acked Read 7 Done Done Sent false
15491 5255 Acked Flush 0 Done Done Sent false
15492 5256 Acked Read 5 Done Done Sent false
15493 5257 Acked Read 1 Done Done Sent false
15494 5258 Acked Read 9 Done Done Sent false
15495 5259 Acked Read 10 Done Done Sent false
15496 5260 Acked Flush 0 Done Done Sent false
15497 5261 Acked Flush 0 Done Done Sent false
15498 5262 Acked Read 4 Done Done Sent false
15499 5263 Acked Flush 0 Done Done Sent false
15500 5264 Acked Read 10 Done Done Sent false
15501 5265 Acked Write 10 Done Done Sent false
15502 5266 Acked Flush 0 Done Done Sent false
15503 5267 Acked Write 5 Done Done Sent false
15504 5268 Acked Write 1 Done Done Sent false
15505 5269 Acked Read 10 Done Done Sent false
15506 5270 Acked Read 3 Done Done Sent false
15507 5271 Acked Write 7 Done Done Sent false
15508 5272 Acked Flush 0 Done Done Sent false
15509 5273 Acked Write 8 Done Done Sent false
15510 5274 Acked Write 1 Done Done Sent false
15511 5275 Acked Write 2 Done Done Sent false
15512 5276 Acked Read 2 Done Done Sent false
15513 5277 Acked Write 1 Done Done Sent false
15514 5278 Acked Flush 0 Done Done Sent false
15515 5279 Acked Read 6 Done Done Sent false
15516 5280 Acked Write 8 Done Done Sent false
15517 5281 Acked Write 4 Done Done Sent false
15518 5282 Acked Write 9 Done Done Sent false
15519 5283 Acked Read 5 Done Done Sent false
15520 5284 Acked Read 7 Done Done Sent false
15521 5285 Acked Write 9 Done Done Sent false
15522 5286 Acked Read 4 Done Done Sent false
15523 5287 Acked Write 2 Done Done Sent false
15524 5288 Acked Read 3 Done Done Sent false
15525 5289 Acked Read 9 Done Done Sent false
15526 5290 Acked Read 6 Done Done Sent false
15527 5291 Acked Read 5 Done Done Sent false
15528 5292 Acked Read 3 Done Done Sent false
15529 5293 Acked Read 5 Done Done Sent false
15530 5294 Acked Read 8 Done Done Sent false
15531 5295 Acked Read 10 Done Done Sent false
15532 5296 Acked Read 10 Done Done Sent false
15533 5297 Acked Read 8 Done Done Sent false
15534 5298 Acked Read 6 Done Done Sent false
15535 5299 Acked Write 2 Done Done Sent false
15536 5300 Acked Read 6 Done Done Sent false
15537 5301 Acked Write 7 Done Done Sent false
15538 5302 Acked Write 8 Done Done Sent false
15539 5303 Acked Write 8 Done Done Sent false
15540 5304 Acked Write 1 Done Done Sent false
15541 5305 Acked Write 8 Done Done Sent false
15542 5306 Acked Write 3 Done Done Sent false
15543 5307 Acked Write 10 Done Done Sent false
15544 5308 Acked Flush 0 Done Done Sent false
15545 5309 Acked Write 7 Done Done Sent false
15546 5310 Acked Write 10 Done Done Sent false
15547 5311 Acked Write 10 Done Done Sent false
15548 5312 Acked Write 6 Done Done Sent false
15549 5313 Acked Read 9 Done Done Sent false
15550 5314 Acked Read 10 Done Done Sent false
15551 5315 Acked Write 10 Done Done Sent false
15552 5316 Acked Write 7 Done Done Sent false
15553 5317 Acked Read 6 Done Done Sent false
15554 5318 Acked Write 2 Done Done Sent false
15555 5319 Acked Read 10 Done Done Sent false
15556 5320 Acked Write 3 Done Done Sent false
15557 5321 Acked Read 2 Done Done Sent false
15558 5322 Acked Read 5 Done Done Sent false
15559 5323 Acked Write 5 Done Done Sent false
15560 5324 Acked Write 4 Done Done Sent false
15561 5325 Acked Read 7 Done Done Sent false
15562 5326 Acked Read 9 Done Done Sent false
15563 5327 Acked Write 10 Done Done Sent false
15564 5328 Acked Read 9 Done Done Sent false
15565 5329 Acked Write 7 Done Done Sent false
15566 5330 Acked Write 10 Done Done Sent false
15567 5331 Acked Flush 0 Done Done Sent false
15568 5332 Acked Write 10 Done Done Sent false
15569 5333 Acked Read 6 Done Done Sent false
15570 5334 Acked Write 3 Done Done Sent false
15571 5335 Acked Write 6 Done Done Sent false
15572 5336 Acked Write 7 Done Done Sent false
15573 5337 Acked Write 2 Done Done Sent false
15574 5338 Acked Flush 0 Done Done Sent false
15575 5339 Acked Read 6 Done Done Sent false
15576 5340 Acked Read 5 Done Done Sent false
15577 5341 Acked Write 5 Done Done Sent false
15578 5342 Acked Write 10 Done Done Sent false
15579 5343 Acked Write 10 Done Done Sent false
15580 5344 Acked Read 2 Done Done Sent false
15581 5345 Acked Read 5 Done Done Sent false
15582 5346 Acked Flush 0 Done Done Sent false
15583 5347 Acked Write 5 Done Done Sent false
15584 5348 Acked Write 4 Done Done Sent false
15585 5349 Acked Read 4 Done Done Sent false
15586 5350 Acked Read 5 Done Done Sent false
15587 5351 Acked Read 5 Done Done Sent false
15588 5352 Acked Write 3 Done Done Sent false
15589 5353 Acked Read 2 Done Done Sent false
15590 5354 Acked Write 2 Done Done Sent false
15591 5355 Acked Read 3 Done Done Sent false
15592 5356 Acked Read 7 Done Done Sent false
15593 5357 Acked Read 1 Done Done Sent false
15594 5358 Acked Write 4 Done Done Sent false
15595 5359 Acked Flush 0 Done Done Sent false
15596 5360 Acked Read 5 Done Done Sent false
15597 5361 Acked Write 4 Done Done Sent false
15598 5362 Acked Read 7 Done Done Sent false
15599 5363 Acked Read 1 Done Done Sent false
15600 5364 Acked Write 4 Done Done Sent false
15601 5365 Acked Write 6 Done Done Sent false
15602 5366 Acked Read 5 Done Done Sent false
15603 5367 Acked Write 9 Done Done Sent false
15604 5368 Acked Read 4 Done Done Sent false
15605 5369 Acked Read 6 Done Done Sent false
15606 5370 Acked Write 7 Done Done Sent false
15607 5371 Acked Write 10 Done Done Sent false
15608 5372 Acked Read 4 Done Done Sent false
15609 5373 Acked Flush 0 Done Done Sent false
15610 5374 Acked Read 7 Done Done Sent false
15611 5375 Acked Write 3 Done Done Sent false
15612 5376 Acked Read 5 Done Done Sent false
15613 5377 Acked Read 6 Done Done Sent false
15614 5378 Acked Read 3 Done Done Sent false
15615 5379 Acked Read 5 Done Done Sent false
15616 5380 Acked Write 3 Done Done Sent false
15617 5381 Acked Write 7 Done Done Sent false
15618 5382 Acked Write 10 Done Done Sent false
15619 5383 Acked Read 2 Done Done Sent false
15620 5384 Acked Flush 0 Done Done Sent false
15621 5385 Acked Write 10 Done Done Sent false
15622 5386 Acked Read 1 Done Done Sent false
15623 5387 Acked Read 8 Done Done Sent false
15624 5388 Acked Read 10 Done Done Sent false
15625 5389 Acked Read 2 Done Done Sent false
15626 5390 Acked Write 1 Done Done Sent false
15627 5391 Acked Read 9 Done Done Sent false
15628 5392 Acked Write 9 Done Done Sent false
15629 5393 Acked Read 6 Done Done Sent false
15630 5394 Acked Read 2 Done Done Sent false
15631 5395 Acked Read 2 Done Done Sent false
15632 5396 Acked Write 10 Done Done Sent false
15633 5397 Acked Read 7 Done Done Sent false
15634 5398 Acked Write 6 Done Done Sent false
15635 5399 Acked Write 10 Done Done Sent false
15636 5400 Acked Write 10 Done Done Sent false
15637 5401 Acked Write 1 Done Done Sent false
15638 5402 Acked Read 7 Done Done Sent false
15639 5403 Acked Write 3 Done Done Sent false
15640 5404 Acked Write 1 Done Done Sent false
15641 5405 Acked Read 2 Done Done Sent false
15642 5406 Acked Write 5 Done Done Sent false
15643 5407 Acked Write 4 Done Done Sent false
15644 5408 Acked Read 1 Done Done Sent false
15645 5409 Acked Read 3 Done Done Sent false
15646 5410 Acked Write 9 Done Done Sent false
15647 5411 Acked Write 7 Done Done Sent false
15648 5412 Acked Write 9 Done Done Sent false
15649 5413 Acked Write 5 Done Done Sent false
15650 5414 Acked Read 7 Done Done Sent false
15651 5415 Acked Write 8 Done Done Sent false
15652 5416 Acked Write 8 Done Done Sent false
15653 5417 Acked Write 4 Done Done Sent false
15654 5418 Acked Read 2 Done Done Sent false
15655 5419 Acked Write 6 Done Done Sent false
15656 5420 Acked Read 8 Done Done Sent false
15657 5421 Acked Read 4 Done Done Sent false
15658 5422 Acked Read 8 Done Done Sent false
15659 5423 Acked Read 6 Done Done Sent false
15660 5424 Acked Read 5 Done Done Sent false
15661 5425 Acked Read 3 Done Done Sent false
15662 5426 Acked Write 8 Done Done Sent false
15663 5427 Acked Read 2 Done Done Sent false
15664 5428 Acked Write 1 Done Done Sent false
15665 5429 Acked Write 6 Done Done Sent false
15666 5430 Acked Read 8 Done Done Sent false
15667 5431 Acked Write 10 Done Done Sent false
15668 5432 Acked Read 9 Done Done Sent false
15669 5433 Acked Write 1 Done Done Sent false
15670 5434 Acked Write 2 Done Done Sent false
15671 5435 Acked Write 10 Done Done Sent false
15672 5436 Acked Flush 0 Done Done Sent false
15673 5437 Acked Flush 0 Done Done Sent false
15674 5438 Acked Write 3 Done Done Sent false
15675 5439 Acked Read 6 Done Done Sent false
15676 5440 Acked Write 9 Done Done Sent false
15677 5441 Acked Read 1 Done Done Sent false
15678 5442 Acked Write 3 Done Done Sent false
15679 5443 Acked Write 2 Done Done Sent false
15680 5444 Acked Write 6 Done Done Sent false
15681 5445 Acked Read 7 Done Done Sent false
15682 5446 Acked Read 10 Done Done Sent false
15683 5447 Acked Write 2 Done Done Sent false
15684 5448 Acked Read 4 Done Done Sent false
15685 5449 Acked Read 7 Done Done Sent false
15686 5450 Acked Read 2 Done Done Sent false
15687 5451 Acked Write 8 Done Done Sent false
15688 5452 Acked Read 1 Done Done Sent false
15689 5453 Acked Flush 0 Done Done Sent false
15690 5454 Acked Read 10 Done Done Sent false
15691 5455 Acked Read 7 Done Done Sent false
15692 5456 Acked Read 10 Done Done Sent false
15693 5457 Acked Read 10 Done Done Sent false
15694 5458 Acked Read 4 Done Done Sent false
15695 5459 Acked Read 8 Done Done Sent false
15696 5460 Acked Read 6 Done Done Sent false
15697 5461 Acked Write 2 Done Done Sent false
15698 5462 Acked Read 10 Done Done Sent false
15699 5463 Acked Write 9 Done Done Sent false
15700 5464 Acked Write 10 Done Done Sent false
15701 5465 Acked Read 2 Done Done Sent false
15702 5466 Acked Read 5 Done Done Sent false
15703 5467 Acked Read 10 Done Done Sent false
15704 5468 Acked Write 3 Done Done Sent false
15705 5469 Acked Read 6 Done Done Sent false
15706 5470 Acked Read 6 Done Done Sent false
15707 5471 Acked Read 9 Done Done Sent false
15708 5472 Acked Flush 0 Done Done Sent false
15709 5473 Acked Read 9 Done Done Sent false
15710 5474 Acked Write 1 Done Done Sent false
15711 5475 Acked Read 2 Done Done Sent false
15712 5476 Acked Read 9 Done Done Sent false
15713 5477 Acked Read 4 Done Done Sent false
15714 5478 Acked Read 6 Done Done Sent false
15715 5479 Acked Read 3 Done Done Sent false
15716 5480 Acked Flush 0 Done Done Sent false
15717 5481 Acked Write 5 Done Done Sent false
15718 5482 Acked Read 1 Done Done Sent false
15719 5483 Acked Write 4 Done Done Sent false
15720 5484 Acked Read 5 Done Done Sent false
15721 5485 Acked Read 1 Done Done Sent false
15722 5486 Acked Read 4 Done Done Sent false
15723 5487 Acked Read 3 Done Done Sent false
15724 5488 Acked Write 7 Done Done Sent false
15725 5489 Acked Write 7 Done Done Sent false
15726 5490 Acked Read 1 Done Done Sent false
15727 5491 Acked Write 7 Done Done Sent false
15728 5492 Acked Read 3 Done Done Sent false
15729 5493 Acked Write 6 Done Done Sent false
15730 5494 Acked Write 2 Done Done Sent false
15731 5495 Acked Write 4 Done Done Sent false
15732 5496 Acked Read 1 Done Done Sent false
15733 5497 Acked Flush 0 Done Done Sent false
15734 5498 Acked Write 1 Done Done Sent false
15735 5499 Acked Flush 0 Done Done Sent false
15736 5500 Acked Write 9 Done Done Sent false
15737 5501 Acked Write 10 Done Done Sent false
15738 5502 Acked Flush 0 Done Done Sent false
15739 5503 Acked Read 8 Done Done Sent false
15740 5504 Acked Flush 0 Done Done Sent false
15741 5505 Acked Read 8 Done Done Sent false
15742 5506 Acked Read 3 Done Done Sent false
15743 5507 Acked Write 10 Done Done Sent false
15744 5508 Acked Write 5 Done Done Sent false
15745 5509 Acked Read 2 Done Done Sent false
15746 5510 Acked Read 8 Done Done Sent false
15747 5511 Acked Read 8 Done Done Sent false
15748 5512 Acked Write 4 Done Done Sent false
15749 5513 Acked Read 5 Done Done Sent false
15750 5514 Acked Write 1 Done Done Sent false
15751 5515 Acked Write 9 Done Done Sent false
15752 5516 Acked Read 5 Done Done Sent false
15753 5517 Acked Write 10 Done Done Sent false
15754 5518 Acked Write 10 Done Done Sent false
15755 5519 Acked Read 9 Done Done Sent false
15756 5520 Acked Write 6 Done Done Sent false
15757 5521 Acked Write 6 Done Done Sent false
15758 5522 Acked Write 10 Done Done Sent false
15759 5523 Acked Read 8 Done Done Sent false
15760 5524 Acked Flush 0 Done Done Sent false
15761 5525 Acked Write 9 Done Done Sent false
15762 5526 Acked Read 2 Done Done Sent false
15763 STATES DS:0 DS:1 DS:2 TOTAL
15764 Sent 0 0 301 301
15765 Done 301 301 0 602
15766 Skipped 0 0 0 0
15767 Error 0 0 0 0
15768 Last Flush: 5524 5524 5225
15769 Downstairs last five completed: 5225 5224 5223 5222 5221
15770 Upstairs last five completed: 5526 5525 5524 5523 5522
15771 CLIENT: Up:0 ds:301 act:2
15772 ----------------------------------------------------------------
15773 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
15774 States: Active Active Offline
15775 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
15776 5226 Acked Write 5 Done Done Sent false
15777 5227 Acked Read 8 Done Done Sent false
15778 5228 Acked Write 8 Done Done Sent false
15779 5229 Acked Flush 0 Done Done Sent false
15780 5230 Acked Write 8 Done Done Sent false
15781 5231 Acked Read 4 Done Done Sent false
15782 5232 Acked Read 10 Done Done Sent false
15783 5233 Acked Read 6 Done Done Sent false
15784 5234 Acked Write 1 Done Done Sent false
15785 5235 Acked Write 5 Done Done Sent false
15786 5236 Acked Write 4 Done Done Sent false
15787 5237 Acked Read 5 Done Done Sent false
15788 5238 Acked Read 2 Done Done Sent false
15789 5239 Acked Write 4 Done Done Sent false
15790 5240 Acked Write 8 Done Done Sent false
15791 5241 Acked Write 8 Done Done Sent false
15792 5242 Acked Write 8 Done Done Sent false
15793 5243 Acked Write 2 Done Done Sent false
15794 5244 Acked Write 2 Done Done Sent false
15795 5245 Acked Read 10 Done Done Sent false
15796 5246 Acked Read 10 Done Done Sent false
15797 5247 Acked Read 4 Done Done Sent false
15798 5248 Acked Write 7 Done Done Sent false
15799 5249 Acked Read 7 Done Done Sent false
15800 5250 Acked Read 1 Done Done Sent false
15801 5251 Acked Read 7 Done Done Sent false
15802 5252 Acked Write 10 Done Done Sent false
15803 5253 Acked Read 4 Done Done Sent false
15804 5254 Acked Read 7 Done Done Sent false
15805 5255 Acked Flush 0 Done Done Sent false
15806 5256 Acked Read 5 Done Done Sent false
15807 5257 Acked Read 1 Done Done Sent false
15808 5258 Acked Read 9 Done Done Sent false
15809 5259 Acked Read 10 Done Done Sent false
15810 5260 Acked Flush 0 Done Done Sent false
15811 5261 Acked Flush 0 Done Done Sent false
15812 5262 Acked Read 4 Done Done Sent false
15813 5263 Acked Flush 0 Done Done Sent false
15814 5264 Acked Read 10 Done Done Sent false
15815 5265 Acked Write 10 Done Done Sent false
15816 5266 Acked Flush 0 Done Done Sent false
15817 5267 Acked Write 5 Done Done Sent false
15818 5268 Acked Write 1 Done Done Sent false
15819 5269 Acked Read 10 Done Done Sent false
15820 5270 Acked Read 3 Done Done Sent false
15821 5271 Acked Write 7 Done Done Sent false
15822 5272 Acked Flush 0 Done Done Sent false
15823 5273 Acked Write 8 Done Done Sent false
15824 5274 Acked Write 1 Done Done Sent false
15825 5275 Acked Write 2 Done Done Sent false
15826 5276 Acked Read 2 Done Done Sent false
15827 5277 Acked Write 1 Done Done Sent false
15828 5278 Acked Flush 0 Done Done Sent false
15829 5279 Acked Read 6 Done Done Sent false
15830 5280 Acked Write 8 Done Done Sent false
15831 5281 Acked Write 4 Done Done Sent false
15832 5282 Acked Write 9 Done Done Sent false
15833 5283 Acked Read 5 Done Done Sent false
15834 5284 Acked Read 7 Done Done Sent false
15835 5285 Acked Write 9 Done Done Sent false
15836 5286 Acked Read 4 Done Done Sent false
15837 5287 Acked Write 2 Done Done Sent false
15838 5288 Acked Read 3 Done Done Sent false
15839 5289 Acked Read 9 Done Done Sent false
15840 5290 Acked Read 6 Done Done Sent false
15841 5291 Acked Read 5 Done Done Sent false
15842 5292 Acked Read 3 Done Done Sent false
15843 5293 Acked Read 5 Done Done Sent false
15844 5294 Acked Read 8 Done Done Sent false
15845 5295 Acked Read 10 Done Done Sent false
15846 5296 Acked Read 10 Done Done Sent false
15847 5297 Acked Read 8 Done Done Sent false
15848 5298 Acked Read 6 Done Done Sent false
15849 5299 Acked Write 2 Done Done Sent false
15850 5300 Acked Read 6 Done Done Sent false
15851 5301 Acked Write 7 Done Done Sent false
15852 5302 Acked Write 8 Done Done Sent false
15853 5303 Acked Write 8 Done Done Sent false
15854 5304 Acked Write 1 Done Done Sent false
15855 5305 Acked Write 8 Done Done Sent false
15856 5306 Acked Write 3 Done Done Sent false
15857 5307 Acked Write 10 Done Done Sent false
15858 5308 Acked Flush 0 Done Done Sent false
15859 5309 Acked Write 7 Done Done Sent false
15860 5310 Acked Write 10 Done Done Sent false
15861 5311 Acked Write 10 Done Done Sent false
15862 5312 Acked Write 6 Done Done Sent false
15863 5313 Acked Read 9 Done Done Sent false
15864 5314 Acked Read 10 Done Done Sent false
15865 5315 Acked Write 10 Done Done Sent false
15866 5316 Acked Write 7 Done Done Sent false
15867 5317 Acked Read 6 Done Done Sent false
15868 5318 Acked Write 2 Done Done Sent false
15869 5319 Acked Read 10 Done Done Sent false
15870 5320 Acked Write 3 Done Done Sent false
15871 5321 Acked Read 2 Done Done Sent false
15872 5322 Acked Read 5 Done Done Sent false
15873 5323 Acked Write 5 Done Done Sent false
15874 5324 Acked Write 4 Done Done Sent false
15875 5325 Acked Read 7 Done Done Sent false
15876 5326 Acked Read 9 Done Done Sent false
15877 5327 Acked Write 10 Done Done Sent false
15878 5328 Acked Read 9 Done Done Sent false
15879 5329 Acked Write 7 Done Done Sent false
15880 5330 Acked Write 10 Done Done Sent false
15881 5331 Acked Flush 0 Done Done Sent false
15882 5332 Acked Write 10 Done Done Sent false
15883 5333 Acked Read 6 Done Done Sent false
15884 5334 Acked Write 3 Done Done Sent false
15885 5335 Acked Write 6 Done Done Sent false
15886 5336 Acked Write 7 Done Done Sent false
15887 5337 Acked Write 2 Done Done Sent false
15888 5338 Acked Flush 0 Done Done Sent false
15889 5339 Acked Read 6 Done Done Sent false
15890 5340 Acked Read 5 Done Done Sent false
15891 5341 Acked Write 5 Done Done Sent false
15892 5342 Acked Write 10 Done Done Sent false
15893 5343 Acked Write 10 Done Done Sent false
15894 5344 Acked Read 2 Done Done Sent false
15895 5345 Acked Read 5 Done Done Sent false
15896 5346 Acked Flush 0 Done Done Sent false
15897 5347 Acked Write 5 Done Done Sent false
15898 5348 Acked Write 4 Done Done Sent false
15899 5349 Acked Read 4 Done Done Sent false
15900 5350 Acked Read 5 Done Done Sent false
15901 5351 Acked Read 5 Done Done Sent false
15902 5352 Acked Write 3 Done Done Sent false
15903 5353 Acked Read 2 Done Done Sent false
15904 5354 Acked Write 2 Done Done Sent false
15905 5355 Acked Read 3 Done Done Sent false
15906 5356 Acked Read 7 Done Done Sent false
15907 5357 Acked Read 1 Done Done Sent false
15908 5358 Acked Write 4 Done Done Sent false
15909 5359 Acked Flush 0 Done Done Sent false
15910 5360 Acked Read 5 Done Done Sent false
15911 5361 Acked Write 4 Done Done Sent false
15912 5362 Acked Read 7 Done Done Sent false
15913 5363 Acked Read 1 Done Done Sent false
15914 5364 Acked Write 4 Done Done Sent false
15915 5365 Acked Write 6 Done Done Sent false
15916 5366 Acked Read 5 Done Done Sent false
15917 5367 Acked Write 9 Done Done Sent false
15918 5368 Acked Read 4 Done Done Sent false
15919 5369 Acked Read 6 Done Done Sent false
15920 5370 Acked Write 7 Done Done Sent false
15921 5371 Acked Write 10 Done Done Sent false
15922 5372 Acked Read 4 Done Done Sent false
15923 5373 Acked Flush 0 Done Done Sent false
15924 5374 Acked Read 7 Done Done Sent false
15925 5375 Acked Write 3 Done Done Sent false
15926 5376 Acked Read 5 Done Done Sent false
15927 5377 Acked Read 6 Done Done Sent false
15928 5378 Acked Read 3 Done Done Sent false
15929 5379 Acked Read 5 Done Done Sent false
15930 5380 Acked Write 3 Done Done Sent false
15931 5381 Acked Write 7 Done Done Sent false
15932 5382 Acked Write 10 Done Done Sent false
15933 5383 Acked Read 2 Done Done Sent false
15934 5384 Acked Flush 0 Done Done Sent false
15935 5385 Acked Write 10 Done Done Sent false
15936 5386 Acked Read 1 Done Done Sent false
15937 5387 Acked Read 8 Done Done Sent false
15938 5388 Acked Read 10 Done Done Sent false
15939 5389 Acked Read 2 Done Done Sent false
15940 5390 Acked Write 1 Done Done Sent false
15941 5391 Acked Read 9 Done Done Sent false
15942 5392 Acked Write 9 Done Done Sent false
15943 5393 Acked Read 6 Done Done Sent false
15944 5394 Acked Read 2 Done Done Sent false
15945 5395 Acked Read 2 Done Done Sent false
15946 5396 Acked Write 10 Done Done Sent false
15947 5397 Acked Read 7 Done Done Sent false
15948 5398 Acked Write 6 Done Done Sent false
15949 5399 Acked Write 10 Done Done Sent false
15950 5400 Acked Write 10 Done Done Sent false
15951 5401 Acked Write 1 Done Done Sent false
15952 5402 Acked Read 7 Done Done Sent false
15953 5403 Acked Write 3 Done Done Sent false
15954 5404 Acked Write 1 Done Done Sent false
15955 5405 Acked Read 2 Done Done Sent false
15956 5406 Acked Write 5 Done Done Sent false
15957 5407 Acked Write 4 Done Done Sent false
15958 5408 Acked Read 1 Done Done Sent false
15959 5409 Acked Read 3 Done Done Sent false
15960 5410 Acked Write 9 Done Done Sent false
15961 5411 Acked Write 7 Done Done Sent false
15962 5412 Acked Write 9 Done Done Sent false
15963 5413 Acked Write 5 Done Done Sent false
15964 5414 Acked Read 7 Done Done Sent false
15965 5415 Acked Write 8 Done Done Sent false
15966 5416 Acked Write 8 Done Done Sent false
15967 5417 Acked Write 4 Done Done Sent false
15968 5418 Acked Read 2 Done Done Sent false
15969 5419 Acked Write 6 Done Done Sent false
15970 5420 Acked Read 8 Done Done Sent false
15971 5421 Acked Read 4 Done Done Sent false
15972 5422 Acked Read 8 Done Done Sent false
15973 5423 Acked Read 6 Done Done Sent false
15974 5424 Acked Read 5 Done Done Sent false
15975 5425 Acked Read 3 Done Done Sent false
15976 5426 Acked Write 8 Done Done Sent false
15977 5427 Acked Read 2 Done Done Sent false
15978 5428 Acked Write 1 Done Done Sent false
15979 5429 Acked Write 6 Done Done Sent false
15980 5430 Acked Read 8 Done Done Sent false
15981 5431 Acked Write 10 Done Done Sent false
15982 5432 Acked Read 9 Done Done Sent false
15983 5433 Acked Write 1 Done Done Sent false
15984 5434 Acked Write 2 Done Done Sent false
15985 5435 Acked Write 10 Done Done Sent false
15986 5436 Acked Flush 0 Done Done Sent false
15987 5437 Acked Flush 0 Done Done Sent false
15988 5438 Acked Write 3 Done Done Sent false
15989 5439 Acked Read 6 Done Done Sent false
15990 5440 Acked Write 9 Done Done Sent false
15991 5441 Acked Read 1 Done Done Sent false
15992 5442 Acked Write 3 Done Done Sent false
15993 5443 Acked Write 2 Done Done Sent false
15994 5444 Acked Write 6 Done Done Sent false
15995 5445 Acked Read 7 Done Done Sent false
15996 5446 Acked Read 10 Done Done Sent false
15997 5447 Acked Write 2 Done Done Sent false
15998 5448 Acked Read 4 Done Done Sent false
15999 5449 Acked Read 7 Done Done Sent false
16000 5450 Acked Read 2 Done Done Sent false
16001 5451 Acked Write 8 Done Done Sent false
16002 5452 Acked Read 1 Done Done Sent false
16003 5453 Acked Flush 0 Done Done Sent false
16004 5454 Acked Read 10 Done Done Sent false
16005 5455 Acked Read 7 Done Done Sent false
16006 5456 Acked Read 10 Done Done Sent false
16007 5457 Acked Read 10 Done Done Sent false
16008 5458 Acked Read 4 Done Done Sent false
16009 5459 Acked Read 8 Done Done Sent false
16010 5460 Acked Read 6 Done Done Sent false
16011 5461 Acked Write 2 Done Done Sent false
16012 5462 Acked Read 10 Done Done Sent false
16013 5463 Acked Write 9 Done Done Sent false
16014 5464 Acked Write 10 Done Done Sent false
16015 5465 Acked Read 2 Done Done Sent false
16016 5466 Acked Read 5 Done Done Sent false
16017 5467 Acked Read 10 Done Done Sent false
16018 5468 Acked Write 3 Done Done Sent false
16019 5469 Acked Read 6 Done Done Sent false
16020 5470 Acked Read 6 Done Done Sent false
16021 5471 Acked Read 9 Done Done Sent false
16022 5472 Acked Flush 0 Done Done Sent false
16023 5473 Acked Read 9 Done Done Sent false
16024 5474 Acked Write 1 Done Done Sent false
16025 5475 Acked Read 2 Done Done Sent false
16026 5476 Acked Read 9 Done Done Sent false
16027 5477 Acked Read 4 Done Done Sent false
16028 5478 Acked Read 6 Done Done Sent false
16029 5479 Acked Read 3 Done Done Sent false
16030 5480 Acked Flush 0 Done Done Sent false
16031 5481 Acked Write 5 Done Done Sent false
16032 5482 Acked Read 1 Done Done Sent false
16033 5483 Acked Write 4 Done Done Sent false
16034 5484 Acked Read 5 Done Done Sent false
16035 5485 Acked Read 1 Done Done Sent false
16036 5486 Acked Read 4 Done Done Sent false
16037 5487 Acked Read 3 Done Done Sent false
16038 5488 Acked Write 7 Done Done Sent false
16039 5489 Acked Write 7 Done Done Sent false
16040 5490 Acked Read 1 Done Done Sent false
16041 5491 Acked Write 7 Done Done Sent false
16042 5492 Acked Read 3 Done Done Sent false
16043 5493 Acked Write 6 Done Done Sent false
16044 5494 Acked Write 2 Done Done Sent false
16045 5495 Acked Write 4 Done Done Sent false
16046 5496 Acked Read 1 Done Done Sent false
16047 5497 Acked Flush 0 Done Done Sent false
16048 5498 Acked Write 1 Done Done Sent false
16049 5499 Acked Flush 0 Done Done Sent false
16050 5500 Acked Write 9 Done Done Sent false
16051 5501 Acked Write 10 Done Done Sent false
16052 5502 Acked Flush 0 Done Done Sent false
16053 5503 Acked Read 8 Done Done Sent false
16054 5504 Acked Flush 0 Done Done Sent false
16055 5505 Acked Read 8 Done Done Sent false
16056 5506 Acked Read 3 Done Done Sent false
16057 5507 Acked Write 10 Done Done Sent false
16058 5508 Acked Write 5 Done Done Sent false
16059 5509 Acked Read 2 Done Done Sent false
16060 5510 Acked Read 8 Done Done Sent false
16061 5511 Acked Read 8 Done Done Sent false
16062 5512 Acked Write 4 Done Done Sent false
16063 5513 Acked Read 5 Done Done Sent false
16064 5514 Acked Write 1 Done Done Sent false
16065 5515 Acked Write 9 Done Done Sent false
16066 5516 Acked Read 5 Done Done Sent false
16067 5517 Acked Write 10 Done Done Sent false
16068 5518 Acked Write 10 Done Done Sent false
16069 5519 Acked Read 9 Done Done Sent false
16070 5520 Acked Write 6 Done Done Sent false
16071 5521 Acked Write 6 Done Done Sent false
16072 5522 Acked Write 10 Done Done Sent false
16073 5523 Acked Read 8 Done Done Sent false
16074 5524 Acked Flush 0 Done Done Sent false
16075 5525 Acked Write 9 Done Done Sent false
16076 5526 Acked Read 2 Done Done Sent false
16077 5527 Acked Flush 0 Done Done Sent false
16078 STATES DS:0 DS:1 DS:2 TOTAL
16079 Sent 0 0 302 302
16080 Done 302 302 0 604
16081 Skipped 0 0 0 0
16082 Error 0 0 0 0
16083 Last Flush: 5527 5527 5225
16084 Downstairs last five completed: 5225 5224 5223 5222 5221
16085 Upstairs last five completed: 5527 5526 5525 5524 5523
16086 CLIENT: Up:0 ds:302 act:2
160872024-11-28T04:53:40.423ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
160882024-11-28T04:53:40.423ZWARNcrucible: ds_connection connect to 127.0.0.1:8830 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
160892024-11-28T04:53:40.423ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
160902024-11-28T04:53:40.423ZWARNcrucible: downstairs task for 2 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
160912024-11-28T04:53:40.423ZINFOcrucible: client task is exiting = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
16092 [8830] got start action
16093 [8830] I am going to restart
16094 Starting downstairs at port 8830
16095 Make output file at "/tmp/dsc/downstairs-8830.txt"
16096 [8830][2] reports Starting
16097 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1217)
16098 [8830][2] reports Running
16099 ----------------------------------------------------------------
16100 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
16101 States: Active Active Offline
16102 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
16103 5226 Acked Write 5 Done Done Sent false
16104 5227 Acked Read 8 Done Done Sent false
16105 5228 Acked Write 8 Done Done Sent false
16106 5229 Acked Flush 0 Done Done Sent false
16107 5230 Acked Write 8 Done Done Sent false
16108 5231 Acked Read 4 Done Done Sent false
16109 5232 Acked Read 10 Done Done Sent false
16110 5233 Acked Read 6 Done Done Sent false
16111 5234 Acked Write 1 Done Done Sent false
16112 5235 Acked Write 5 Done Done Sent false
16113 5236 Acked Write 4 Done Done Sent false
16114 5237 Acked Read 5 Done Done Sent false
16115 5238 Acked Read 2 Done Done Sent false
16116 5239 Acked Write 4 Done Done Sent false
16117 5240 Acked Write 8 Done Done Sent false
16118 5241 Acked Write 8 Done Done Sent false
16119 5242 Acked Write 8 Done Done Sent false
16120 5243 Acked Write 2 Done Done Sent false
16121 5244 Acked Write 2 Done Done Sent false
16122 5245 Acked Read 10 Done Done Sent false
16123 5246 Acked Read 10 Done Done Sent false
16124 5247 Acked Read 4 Done Done Sent false
16125 5248 Acked Write 7 Done Done Sent false
16126 5249 Acked Read 7 Done Done Sent false
16127 5250 Acked Read 1 Done Done Sent false
16128 5251 Acked Read 7 Done Done Sent false
16129 5252 Acked Write 10 Done Done Sent false
16130 5253 Acked Read 4 Done Done Sent false
16131 5254 Acked Read 7 Done Done Sent false
16132 5255 Acked Flush 0 Done Done Sent false
16133 5256 Acked Read 5 Done Done Sent false
16134 5257 Acked Read 1 Done Done Sent false
16135 5258 Acked Read 9 Done Done Sent false
16136 5259 Acked Read 10 Done Done Sent false
16137 5260 Acked Flush 0 Done Done Sent false
16138 5261 Acked Flush 0 Done Done Sent false
16139 5262 Acked Read 4 Done Done Sent false
16140 5263 Acked Flush 0 Done Done Sent false
16141 5264 Acked Read 10 Done Done Sent false
16142 5265 Acked Write 10 Done Done Sent false
16143 5266 Acked Flush 0 Done Done Sent false
16144 5267 Acked Write 5 Done Done Sent false
16145 5268 Acked Write 1 Done Done Sent false
16146 5269 Acked Read 10 Done Done Sent false
16147 5270 Acked Read 3 Done Done Sent false
16148 5271 Acked Write 7 Done Done Sent false
16149 5272 Acked Flush 0 Done Done Sent false
16150 5273 Acked Write 8 Done Done Sent false
16151 5274 Acked Write 1 Done Done Sent false
16152 5275 Acked Write 2 Done Done Sent false
16153 5276 Acked Read 2 Done Done Sent false
16154 5277 Acked Write 1 Done Done Sent false
16155 5278 Acked Flush 0 Done Done Sent false
16156 5279 Acked Read 6 Done Done Sent false
16157 5280 Acked Write 8 Done Done Sent false
16158 5281 Acked Write 4 Done Done Sent false
16159 5282 Acked Write 9 Done Done Sent false
16160 5283 Acked Read 5 Done Done Sent false
16161 5284 Acked Read 7 Done Done Sent false
16162 5285 Acked Write 9 Done Done Sent false
16163 5286 Acked Read 4 Done Done Sent false
16164 5287 Acked Write 2 Done Done Sent false
16165 5288 Acked Read 3 Done Done Sent false
16166 5289 Acked Read 9 Done Done Sent false
16167 5290 Acked Read 6 Done Done Sent false
16168 5291 Acked Read 5 Done Done Sent false
16169 5292 Acked Read 3 Done Done Sent false
16170 5293 Acked Read 5 Done Done Sent false
16171 5294 Acked Read 8 Done Done Sent false
16172 5295 Acked Read 10 Done Done Sent false
16173 5296 Acked Read 10 Done Done Sent false
16174 5297 Acked Read 8 Done Done Sent false
16175 5298 Acked Read 6 Done Done Sent false
16176 5299 Acked Write 2 Done Done Sent false
16177 5300 Acked Read 6 Done Done Sent false
16178 5301 Acked Write 7 Done Done Sent false
16179 5302 Acked Write 8 Done Done Sent false
16180 5303 Acked Write 8 Done Done Sent false
16181 5304 Acked Write 1 Done Done Sent false
16182 5305 Acked Write 8 Done Done Sent false
16183 5306 Acked Write 3 Done Done Sent false
16184 5307 Acked Write 10 Done Done Sent false
16185 5308 Acked Flush 0 Done Done Sent false
16186 5309 Acked Write 7 Done Done Sent false
16187 5310 Acked Write 10 Done Done Sent false
16188 5311 Acked Write 10 Done Done Sent false
16189 5312 Acked Write 6 Done Done Sent false
16190 5313 Acked Read 9 Done Done Sent false
16191 5314 Acked Read 10 Done Done Sent false
16192 5315 Acked Write 10 Done Done Sent false
16193 5316 Acked Write 7 Done Done Sent false
16194 5317 Acked Read 6 Done Done Sent false
16195 5318 Acked Write 2 Done Done Sent false
16196 5319 Acked Read 10 Done Done Sent false
16197 5320 Acked Write 3 Done Done Sent false
16198 5321 Acked Read 2 Done Done Sent false
16199 5322 Acked Read 5 Done Done Sent false
16200 5323 Acked Write 5 Done Done Sent false
16201 5324 Acked Write 4 Done Done Sent false
16202 5325 Acked Read 7 Done Done Sent false
16203 5326 Acked Read 9 Done Done Sent false
16204 5327 Acked Write 10 Done Done Sent false
16205 5328 Acked Read 9 Done Done Sent false
16206 5329 Acked Write 7 Done Done Sent false
16207 5330 Acked Write 10 Done Done Sent false
16208 5331 Acked Flush 0 Done Done Sent false
16209 5332 Acked Write 10 Done Done Sent false
16210 5333 Acked Read 6 Done Done Sent false
16211 5334 Acked Write 3 Done Done Sent false
16212 5335 Acked Write 6 Done Done Sent false
16213 5336 Acked Write 7 Done Done Sent false
16214 5337 Acked Write 2 Done Done Sent false
16215 5338 Acked Flush 0 Done Done Sent false
16216 5339 Acked Read 6 Done Done Sent false
16217 5340 Acked Read 5 Done Done Sent false
16218 5341 Acked Write 5 Done Done Sent false
16219 5342 Acked Write 10 Done Done Sent false
16220 5343 Acked Write 10 Done Done Sent false
16221 5344 Acked Read 2 Done Done Sent false
16222 5345 Acked Read 5 Done Done Sent false
16223 5346 Acked Flush 0 Done Done Sent false
16224 5347 Acked Write 5 Done Done Sent false
16225 5348 Acked Write 4 Done Done Sent false
16226 5349 Acked Read 4 Done Done Sent false
16227 5350 Acked Read 5 Done Done Sent false
16228 5351 Acked Read 5 Done Done Sent false
16229 5352 Acked Write 3 Done Done Sent false
16230 5353 Acked Read 2 Done Done Sent false
16231 5354 Acked Write 2 Done Done Sent false
16232 5355 Acked Read 3 Done Done Sent false
16233 5356 Acked Read 7 Done Done Sent false
16234 5357 Acked Read 1 Done Done Sent false
16235 5358 Acked Write 4 Done Done Sent false
16236 5359 Acked Flush 0 Done Done Sent false
16237 5360 Acked Read 5 Done Done Sent false
16238 5361 Acked Write 4 Done Done Sent false
16239 5362 Acked Read 7 Done Done Sent false
16240 5363 Acked Read 1 Done Done Sent false
16241 5364 Acked Write 4 Done Done Sent false
16242 5365 Acked Write 6 Done Done Sent false
16243 5366 Acked Read 5 Done Done Sent false
16244 5367 Acked Write 9 Done Done Sent false
16245 5368 Acked Read 4 Done Done Sent false
16246 5369 Acked Read 6 Done Done Sent false
16247 5370 Acked Write 7 Done Done Sent false
16248 5371 Acked Write 10 Done Done Sent false
16249 5372 Acked Read 4 Done Done Sent false
16250 5373 Acked Flush 0 Done Done Sent false
16251 5374 Acked Read 7 Done Done Sent false
16252 5375 Acked Write 3 Done Done Sent false
16253 5376 Acked Read 5 Done Done Sent false
16254 5377 Acked Read 6 Done Done Sent false
16255 5378 Acked Read 3 Done Done Sent false
16256 5379 Acked Read 5 Done Done Sent false
16257 5380 Acked Write 3 Done Done Sent false
16258 5381 Acked Write 7 Done Done Sent false
16259 5382 Acked Write 10 Done Done Sent false
16260 5383 Acked Read 2 Done Done Sent false
16261 5384 Acked Flush 0 Done Done Sent false
16262 5385 Acked Write 10 Done Done Sent false
16263 5386 Acked Read 1 Done Done Sent false
16264 5387 Acked Read 8 Done Done Sent false
16265 5388 Acked Read 10 Done Done Sent false
16266 5389 Acked Read 2 Done Done Sent false
16267 5390 Acked Write 1 Done Done Sent false
16268 5391 Acked Read 9 Done Done Sent false
16269 5392 Acked Write 9 Done Done Sent false
16270 5393 Acked Read 6 Done Done Sent false
16271 5394 Acked Read 2 Done Done Sent false
16272 5395 Acked Read 2 Done Done Sent false
16273 5396 Acked Write 10 Done Done Sent false
16274 5397 Acked Read 7 Done Done Sent false
16275 5398 Acked Write 6 Done Done Sent false
16276 5399 Acked Write 10 Done Done Sent false
16277 5400 Acked Write 10 Done Done Sent false
16278 5401 Acked Write 1 Done Done Sent false
16279 5402 Acked Read 7 Done Done Sent false
16280 5403 Acked Write 3 Done Done Sent false
16281 5404 Acked Write 1 Done Done Sent false
16282 5405 Acked Read 2 Done Done Sent false
16283 5406 Acked Write 5 Done Done Sent false
16284 5407 Acked Write 4 Done Done Sent false
16285 5408 Acked Read 1 Done Done Sent false
16286 5409 Acked Read 3 Done Done Sent false
16287 5410 Acked Write 9 Done Done Sent false
16288 5411 Acked Write 7 Done Done Sent false
16289 5412 Acked Write 9 Done Done Sent false
16290 5413 Acked Write 5 Done Done Sent false
16291 5414 Acked Read 7 Done Done Sent false
16292 5415 Acked Write 8 Done Done Sent false
16293 5416 Acked Write 8 Done Done Sent false
16294 5417 Acked Write 4 Done Done Sent false
16295 5418 Acked Read 2 Done Done Sent false
16296 5419 Acked Write 6 Done Done Sent false
16297 5420 Acked Read 8 Done Done Sent false
16298 5421 Acked Read 4 Done Done Sent false
16299 5422 Acked Read 8 Done Done Sent false
16300 5423 Acked Read 6 Done Done Sent false
16301 5424 Acked Read 5 Done Done Sent false
16302 5425 Acked Read 3 Done Done Sent false
16303 5426 Acked Write 8 Done Done Sent false
16304 5427 Acked Read 2 Done Done Sent false
16305 5428 Acked Write 1 Done Done Sent false
16306 5429 Acked Write 6 Done Done Sent false
16307 5430 Acked Read 8 Done Done Sent false
16308 5431 Acked Write 10 Done Done Sent false
16309 5432 Acked Read 9 Done Done Sent false
16310 5433 Acked Write 1 Done Done Sent false
16311 5434 Acked Write 2 Done Done Sent false
16312 5435 Acked Write 10 Done Done Sent false
16313 5436 Acked Flush 0 Done Done Sent false
16314 5437 Acked Flush 0 Done Done Sent false
16315 5438 Acked Write 3 Done Done Sent false
16316 5439 Acked Read 6 Done Done Sent false
16317 5440 Acked Write 9 Done Done Sent false
16318 5441 Acked Read 1 Done Done Sent false
16319 5442 Acked Write 3 Done Done Sent false
16320 5443 Acked Write 2 Done Done Sent false
16321 5444 Acked Write 6 Done Done Sent false
16322 5445 Acked Read 7 Done Done Sent false
16323 5446 Acked Read 10 Done Done Sent false
16324 5447 Acked Write 2 Done Done Sent false
16325 5448 Acked Read 4 Done Done Sent false
16326 5449 Acked Read 7 Done Done Sent false
16327 5450 Acked Read 2 Done Done Sent false
16328 5451 Acked Write 8 Done Done Sent false
16329 5452 Acked Read 1 Done Done Sent false
16330 5453 Acked Flush 0 Done Done Sent false
16331 5454 Acked Read 10 Done Done Sent false
16332 5455 Acked Read 7 Done Done Sent false
16333 5456 Acked Read 10 Done Done Sent false
16334 5457 Acked Read 10 Done Done Sent false
16335 5458 Acked Read 4 Done Done Sent false
16336 5459 Acked Read 8 Done Done Sent false
16337 5460 Acked Read 6 Done Done Sent false
16338 5461 Acked Write 2 Done Done Sent false
16339 5462 Acked Read 10 Done Done Sent false
16340 5463 Acked Write 9 Done Done Sent false
16341 5464 Acked Write 10 Done Done Sent false
16342 5465 Acked Read 2 Done Done Sent false
16343 5466 Acked Read 5 Done Done Sent false
16344 5467 Acked Read 10 Done Done Sent false
16345 5468 Acked Write 3 Done Done Sent false
16346 5469 Acked Read 6 Done Done Sent false
16347 5470 Acked Read 6 Done Done Sent false
16348 5471 Acked Read 9 Done Done Sent false
16349 5472 Acked Flush 0 Done Done Sent false
16350 5473 Acked Read 9 Done Done Sent false
16351 5474 Acked Write 1 Done Done Sent false
16352 5475 Acked Read 2 Done Done Sent false
16353 5476 Acked Read 9 Done Done Sent false
16354 5477 Acked Read 4 Done Done Sent false
16355 5478 Acked Read 6 Done Done Sent false
16356 5479 Acked Read 3 Done Done Sent false
16357 5480 Acked Flush 0 Done Done Sent false
16358 5481 Acked Write 5 Done Done Sent false
16359 5482 Acked Read 1 Done Done Sent false
16360 5483 Acked Write 4 Done Done Sent false
16361 5484 Acked Read 5 Done Done Sent false
16362 5485 Acked Read 1 Done Done Sent false
16363 5486 Acked Read 4 Done Done Sent false
16364 5487 Acked Read 3 Done Done Sent false
16365 5488 Acked Write 7 Done Done Sent false
16366 5489 Acked Write 7 Done Done Sent false
16367 5490 Acked Read 1 Done Done Sent false
16368 5491 Acked Write 7 Done Done Sent false
16369 5492 Acked Read 3 Done Done Sent false
16370 5493 Acked Write 6 Done Done Sent false
16371 5494 Acked Write 2 Done Done Sent false
16372 5495 Acked Write 4 Done Done Sent false
16373 5496 Acked Read 1 Done Done Sent false
16374 5497 Acked Flush 0 Done Done Sent false
16375 5498 Acked Write 1 Done Done Sent false
16376 5499 Acked Flush 0 Done Done Sent false
16377 5500 Acked Write 9 Done Done Sent false
16378 5501 Acked Write 10 Done Done Sent false
16379 5502 Acked Flush 0 Done Done Sent false
16380 5503 Acked Read 8 Done Done Sent false
16381 5504 Acked Flush 0 Done Done Sent false
16382 5505 Acked Read 8 Done Done Sent false
16383 5506 Acked Read 3 Done Done Sent false
16384 5507 Acked Write 10 Done Done Sent false
16385 5508 Acked Write 5 Done Done Sent false
16386 5509 Acked Read 2 Done Done Sent false
16387 5510 Acked Read 8 Done Done Sent false
16388 5511 Acked Read 8 Done Done Sent false
16389 5512 Acked Write 4 Done Done Sent false
16390 5513 Acked Read 5 Done Done Sent false
16391 5514 Acked Write 1 Done Done Sent false
16392 5515 Acked Write 9 Done Done Sent false
16393 5516 Acked Read 5 Done Done Sent false
16394 5517 Acked Write 10 Done Done Sent false
16395 5518 Acked Write 10 Done Done Sent false
16396 5519 Acked Read 9 Done Done Sent false
16397 5520 Acked Write 6 Done Done Sent false
16398 5521 Acked Write 6 Done Done Sent false
16399 5522 Acked Write 10 Done Done Sent false
16400 5523 Acked Read 8 Done Done Sent false
16401 5524 Acked Flush 0 Done Done Sent false
16402 5525 Acked Write 9 Done Done Sent false
16403 5526 Acked Read 2 Done Done Sent false
16404 5527 Acked Flush 0 Done Done Sent false
16405 STATES DS:0 DS:1 DS:2 TOTAL
16406 Sent 0 0 302 302
16407 Done 302 302 0 604
16408 Skipped 0 0 0 0
16409 Error 0 0 0 0
16410 Last Flush: 5527 5527 5225
16411 Downstairs last five completed: 5225 5224 5223 5222 5221
16412 Upstairs last five completed: 5527 5526 5525 5524 5523
16413 CLIENT: Up:0 ds:302 act:2
16414 ----------------------------------------------------------------
16415 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
16416 States: Active Active Offline
16417 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
16418 5226 Acked Write 5 Done Done Sent false
16419 5227 Acked Read 8 Done Done Sent false
16420 5228 Acked Write 8 Done Done Sent false
16421 5229 Acked Flush 0 Done Done Sent false
16422 5230 Acked Write 8 Done Done Sent false
16423 5231 Acked Read 4 Done Done Sent false
16424 5232 Acked Read 10 Done Done Sent false
16425 5233 Acked Read 6 Done Done Sent false
16426 5234 Acked Write 1 Done Done Sent false
16427 5235 Acked Write 5 Done Done Sent false
16428 5236 Acked Write 4 Done Done Sent false
16429 5237 Acked Read 5 Done Done Sent false
16430 5238 Acked Read 2 Done Done Sent false
16431 5239 Acked Write 4 Done Done Sent false
16432 5240 Acked Write 8 Done Done Sent false
16433 5241 Acked Write 8 Done Done Sent false
16434 5242 Acked Write 8 Done Done Sent false
16435 5243 Acked Write 2 Done Done Sent false
16436 5244 Acked Write 2 Done Done Sent false
16437 5245 Acked Read 10 Done Done Sent false
16438 5246 Acked Read 10 Done Done Sent false
16439 5247 Acked Read 4 Done Done Sent false
16440 5248 Acked Write 7 Done Done Sent false
16441 5249 Acked Read 7 Done Done Sent false
16442 5250 Acked Read 1 Done Done Sent false
16443 5251 Acked Read 7 Done Done Sent false
16444 5252 Acked Write 10 Done Done Sent false
16445 5253 Acked Read 4 Done Done Sent false
16446 5254 Acked Read 7 Done Done Sent false
16447 5255 Acked Flush 0 Done Done Sent false
16448 5256 Acked Read 5 Done Done Sent false
16449 5257 Acked Read 1 Done Done Sent false
16450 5258 Acked Read 9 Done Done Sent false
16451 5259 Acked Read 10 Done Done Sent false
16452 5260 Acked Flush 0 Done Done Sent false
16453 5261 Acked Flush 0 Done Done Sent false
16454 5262 Acked Read 4 Done Done Sent false
16455 5263 Acked Flush 0 Done Done Sent false
16456 5264 Acked Read 10 Done Done Sent false
16457 5265 Acked Write 10 Done Done Sent false
16458 5266 Acked Flush 0 Done Done Sent false
16459 5267 Acked Write 5 Done Done Sent false
16460 5268 Acked Write 1 Done Done Sent false
16461 5269 Acked Read 10 Done Done Sent false
16462 5270 Acked Read 3 Done Done Sent false
16463 5271 Acked Write 7 Done Done Sent false
16464 5272 Acked Flush 0 Done Done Sent false
16465 5273 Acked Write 8 Done Done Sent false
16466 5274 Acked Write 1 Done Done Sent false
16467 5275 Acked Write 2 Done Done Sent false
16468 5276 Acked Read 2 Done Done Sent false
16469 5277 Acked Write 1 Done Done Sent false
16470 5278 Acked Flush 0 Done Done Sent false
16471 5279 Acked Read 6 Done Done Sent false
16472 5280 Acked Write 8 Done Done Sent false
16473 5281 Acked Write 4 Done Done Sent false
16474 5282 Acked Write 9 Done Done Sent false
16475 5283 Acked Read 5 Done Done Sent false
16476 5284 Acked Read 7 Done Done Sent false
16477 5285 Acked Write 9 Done Done Sent false
16478 5286 Acked Read 4 Done Done Sent false
16479 5287 Acked Write 2 Done Done Sent false
16480 5288 Acked Read 3 Done Done Sent false
16481 5289 Acked Read 9 Done Done Sent false
16482 5290 Acked Read 6 Done Done Sent false
16483 5291 Acked Read 5 Done Done Sent false
16484 5292 Acked Read 3 Done Done Sent false
16485 5293 Acked Read 5 Done Done Sent false
16486 5294 Acked Read 8 Done Done Sent false
16487 5295 Acked Read 10 Done Done Sent false
16488 5296 Acked Read 10 Done Done Sent false
16489 5297 Acked Read 8 Done Done Sent false
16490 5298 Acked Read 6 Done Done Sent false
16491 5299 Acked Write 2 Done Done Sent false
16492 5300 Acked Read 6 Done Done Sent false
16493 5301 Acked Write 7 Done Done Sent false
16494 5302 Acked Write 8 Done Done Sent false
16495 5303 Acked Write 8 Done Done Sent false
16496 5304 Acked Write 1 Done Done Sent false
16497 5305 Acked Write 8 Done Done Sent false
16498 5306 Acked Write 3 Done Done Sent false
16499 5307 Acked Write 10 Done Done Sent false
16500 5308 Acked Flush 0 Done Done Sent false
16501 5309 Acked Write 7 Done Done Sent false
16502 5310 Acked Write 10 Done Done Sent false
16503 5311 Acked Write 10 Done Done Sent false
16504 5312 Acked Write 6 Done Done Sent false
16505 5313 Acked Read 9 Done Done Sent false
16506 5314 Acked Read 10 Done Done Sent false
16507 5315 Acked Write 10 Done Done Sent false
16508 5316 Acked Write 7 Done Done Sent false
16509 5317 Acked Read 6 Done Done Sent false
16510 5318 Acked Write 2 Done Done Sent false
16511 5319 Acked Read 10 Done Done Sent false
16512 5320 Acked Write 3 Done Done Sent false
16513 5321 Acked Read 2 Done Done Sent false
16514 5322 Acked Read 5 Done Done Sent false
16515 5323 Acked Write 5 Done Done Sent false
16516 5324 Acked Write 4 Done Done Sent false
16517 5325 Acked Read 7 Done Done Sent false
16518 5326 Acked Read 9 Done Done Sent false
16519 5327 Acked Write 10 Done Done Sent false
16520 5328 Acked Read 9 Done Done Sent false
16521 5329 Acked Write 7 Done Done Sent false
16522 5330 Acked Write 10 Done Done Sent false
16523 5331 Acked Flush 0 Done Done Sent false
16524 5332 Acked Write 10 Done Done Sent false
16525 5333 Acked Read 6 Done Done Sent false
16526 5334 Acked Write 3 Done Done Sent false
16527 5335 Acked Write 6 Done Done Sent false
16528 5336 Acked Write 7 Done Done Sent false
16529 5337 Acked Write 2 Done Done Sent false
16530 5338 Acked Flush 0 Done Done Sent false
16531 5339 Acked Read 6 Done Done Sent false
16532 5340 Acked Read 5 Done Done Sent false
16533 5341 Acked Write 5 Done Done Sent false
16534 5342 Acked Write 10 Done Done Sent false
16535 5343 Acked Write 10 Done Done Sent false
16536 5344 Acked Read 2 Done Done Sent false
16537 5345 Acked Read 5 Done Done Sent false
16538 5346 Acked Flush 0 Done Done Sent false
16539 5347 Acked Write 5 Done Done Sent false
16540 5348 Acked Write 4 Done Done Sent false
16541 5349 Acked Read 4 Done Done Sent false
16542 5350 Acked Read 5 Done Done Sent false
16543 5351 Acked Read 5 Done Done Sent false
16544 5352 Acked Write 3 Done Done Sent false
16545 5353 Acked Read 2 Done Done Sent false
16546 5354 Acked Write 2 Done Done Sent false
16547 5355 Acked Read 3 Done Done Sent false
16548 5356 Acked Read 7 Done Done Sent false
16549 5357 Acked Read 1 Done Done Sent false
16550 5358 Acked Write 4 Done Done Sent false
16551 5359 Acked Flush 0 Done Done Sent false
16552 5360 Acked Read 5 Done Done Sent false
16553 5361 Acked Write 4 Done Done Sent false
16554 5362 Acked Read 7 Done Done Sent false
16555 5363 Acked Read 1 Done Done Sent false
16556 5364 Acked Write 4 Done Done Sent false
16557 5365 Acked Write 6 Done Done Sent false
16558 5366 Acked Read 5 Done Done Sent false
16559 5367 Acked Write 9 Done Done Sent false
16560 5368 Acked Read 4 Done Done Sent false
16561 5369 Acked Read 6 Done Done Sent false
16562 5370 Acked Write 7 Done Done Sent false
16563 5371 Acked Write 10 Done Done Sent false
16564 5372 Acked Read 4 Done Done Sent false
16565 5373 Acked Flush 0 Done Done Sent false
16566 5374 Acked Read 7 Done Done Sent false
16567 5375 Acked Write 3 Done Done Sent false
16568 5376 Acked Read 5 Done Done Sent false
16569 5377 Acked Read 6 Done Done Sent false
16570 5378 Acked Read 3 Done Done Sent false
16571 5379 Acked Read 5 Done Done Sent false
16572 5380 Acked Write 3 Done Done Sent false
16573 5381 Acked Write 7 Done Done Sent false
16574 5382 Acked Write 10 Done Done Sent false
16575 5383 Acked Read 2 Done Done Sent false
16576 5384 Acked Flush 0 Done Done Sent false
16577 5385 Acked Write 10 Done Done Sent false
16578 5386 Acked Read 1 Done Done Sent false
16579 5387 Acked Read 8 Done Done Sent false
16580 5388 Acked Read 10 Done Done Sent false
16581 5389 Acked Read 2 Done Done Sent false
16582 5390 Acked Write 1 Done Done Sent false
16583 5391 Acked Read 9 Done Done Sent false
16584 5392 Acked Write 9 Done Done Sent false
16585 5393 Acked Read 6 Done Done Sent false
16586 5394 Acked Read 2 Done Done Sent false
16587 5395 Acked Read 2 Done Done Sent false
16588 5396 Acked Write 10 Done Done Sent false
16589 5397 Acked Read 7 Done Done Sent false
16590 5398 Acked Write 6 Done Done Sent false
16591 5399 Acked Write 10 Done Done Sent false
16592 5400 Acked Write 10 Done Done Sent false
16593 5401 Acked Write 1 Done Done Sent false
16594 5402 Acked Read 7 Done Done Sent false
16595 5403 Acked Write 3 Done Done Sent false
16596 5404 Acked Write 1 Done Done Sent false
16597 5405 Acked Read 2 Done Done Sent false
16598 5406 Acked Write 5 Done Done Sent false
16599 5407 Acked Write 4 Done Done Sent false
16600 5408 Acked Read 1 Done Done Sent false
16601 5409 Acked Read 3 Done Done Sent false
16602 5410 Acked Write 9 Done Done Sent false
16603 5411 Acked Write 7 Done Done Sent false
16604 5412 Acked Write 9 Done Done Sent false
16605 5413 Acked Write 5 Done Done Sent false
16606 5414 Acked Read 7 Done Done Sent false
16607 5415 Acked Write 8 Done Done Sent false
16608 5416 Acked Write 8 Done Done Sent false
16609 5417 Acked Write 4 Done Done Sent false
16610 5418 Acked Read 2 Done Done Sent false
16611 5419 Acked Write 6 Done Done Sent false
16612 5420 Acked Read 8 Done Done Sent false
16613 5421 Acked Read 4 Done Done Sent false
16614 5422 Acked Read 8 Done Done Sent false
16615 5423 Acked Read 6 Done Done Sent false
16616 5424 Acked Read 5 Done Done Sent false
16617 5425 Acked Read 3 Done Done Sent false
16618 5426 Acked Write 8 Done Done Sent false
16619 5427 Acked Read 2 Done Done Sent false
16620 5428 Acked Write 1 Done Done Sent false
16621 5429 Acked Write 6 Done Done Sent false
16622 5430 Acked Read 8 Done Done Sent false
16623 5431 Acked Write 10 Done Done Sent false
16624 5432 Acked Read 9 Done Done Sent false
16625 5433 Acked Write 1 Done Done Sent false
16626 5434 Acked Write 2 Done Done Sent false
16627 5435 Acked Write 10 Done Done Sent false
16628 5436 Acked Flush 0 Done Done Sent false
16629 5437 Acked Flush 0 Done Done Sent false
16630 5438 Acked Write 3 Done Done Sent false
16631 5439 Acked Read 6 Done Done Sent false
16632 5440 Acked Write 9 Done Done Sent false
16633 5441 Acked Read 1 Done Done Sent false
16634 5442 Acked Write 3 Done Done Sent false
16635 5443 Acked Write 2 Done Done Sent false
16636 5444 Acked Write 6 Done Done Sent false
16637 5445 Acked Read 7 Done Done Sent false
16638 5446 Acked Read 10 Done Done Sent false
16639 5447 Acked Write 2 Done Done Sent false
16640 5448 Acked Read 4 Done Done Sent false
16641 5449 Acked Read 7 Done Done Sent false
16642 5450 Acked Read 2 Done Done Sent false
16643 5451 Acked Write 8 Done Done Sent false
16644 5452 Acked Read 1 Done Done Sent false
16645 5453 Acked Flush 0 Done Done Sent false
16646 5454 Acked Read 10 Done Done Sent false
16647 5455 Acked Read 7 Done Done Sent false
16648 5456 Acked Read 10 Done Done Sent false
16649 5457 Acked Read 10 Done Done Sent false
16650 5458 Acked Read 4 Done Done Sent false
16651 5459 Acked Read 8 Done Done Sent false
16652 5460 Acked Read 6 Done Done Sent false
16653 5461 Acked Write 2 Done Done Sent false
16654 5462 Acked Read 10 Done Done Sent false
16655 5463 Acked Write 9 Done Done Sent false
16656 5464 Acked Write 10 Done Done Sent false
16657 5465 Acked Read 2 Done Done Sent false
16658 5466 Acked Read 5 Done Done Sent false
16659 5467 Acked Read 10 Done Done Sent false
16660 5468 Acked Write 3 Done Done Sent false
16661 5469 Acked Read 6 Done Done Sent false
16662 5470 Acked Read 6 Done Done Sent false
16663 5471 Acked Read 9 Done Done Sent false
16664 5472 Acked Flush 0 Done Done Sent false
16665 5473 Acked Read 9 Done Done Sent false
16666 5474 Acked Write 1 Done Done Sent false
16667 5475 Acked Read 2 Done Done Sent false
16668 5476 Acked Read 9 Done Done Sent false
16669 5477 Acked Read 4 Done Done Sent false
16670 5478 Acked Read 6 Done Done Sent false
16671 5479 Acked Read 3 Done Done Sent false
16672 5480 Acked Flush 0 Done Done Sent false
16673 5481 Acked Write 5 Done Done Sent false
16674 5482 Acked Read 1 Done Done Sent false
16675 5483 Acked Write 4 Done Done Sent false
16676 5484 Acked Read 5 Done Done Sent false
16677 5485 Acked Read 1 Done Done Sent false
16678 5486 Acked Read 4 Done Done Sent false
16679 5487 Acked Read 3 Done Done Sent false
16680 5488 Acked Write 7 Done Done Sent false
16681 5489 Acked Write 7 Done Done Sent false
16682 5490 Acked Read 1 Done Done Sent false
16683 5491 Acked Write 7 Done Done Sent false
16684 5492 Acked Read 3 Done Done Sent false
16685 5493 Acked Write 6 Done Done Sent false
16686 5494 Acked Write 2 Done Done Sent false
16687 5495 Acked Write 4 Done Done Sent false
16688 5496 Acked Read 1 Done Done Sent false
16689 5497 Acked Flush 0 Done Done Sent false
16690 5498 Acked Write 1 Done Done Sent false
16691 5499 Acked Flush 0 Done Done Sent false
16692 5500 Acked Write 9 Done Done Sent false
16693 5501 Acked Write 10 Done Done Sent false
16694 5502 Acked Flush 0 Done Done Sent false
16695 5503 Acked Read 8 Done Done Sent false
16696 5504 Acked Flush 0 Done Done Sent false
16697 5505 Acked Read 8 Done Done Sent false
16698 5506 Acked Read 3 Done Done Sent false
16699 5507 Acked Write 10 Done Done Sent false
16700 5508 Acked Write 5 Done Done Sent false
16701 5509 Acked Read 2 Done Done Sent false
16702 5510 Acked Read 8 Done Done Sent false
16703 5511 Acked Read 8 Done Done Sent false
16704 5512 Acked Write 4 Done Done Sent false
16705 5513 Acked Read 5 Done Done Sent false
16706 5514 Acked Write 1 Done Done Sent false
16707 5515 Acked Write 9 Done Done Sent false
16708 5516 Acked Read 5 Done Done Sent false
16709 5517 Acked Write 10 Done Done Sent false
16710 5518 Acked Write 10 Done Done Sent false
16711 5519 Acked Read 9 Done Done Sent false
16712 5520 Acked Write 6 Done Done Sent false
16713 5521 Acked Write 6 Done Done Sent false
16714 5522 Acked Write 10 Done Done Sent false
16715 5523 Acked Read 8 Done Done Sent false
16716 5524 Acked Flush 0 Done Done Sent false
16717 5525 Acked Write 9 Done Done Sent false
16718 5526 Acked Read 2 Done Done Sent false
16719 5527 Acked Flush 0 Done Done Sent false
16720 STATES DS:0 DS:1 DS:2 TOTAL
16721 Sent 0 0 302 302
16722 Done 302 302 0 604
16723 Skipped 0 0 0 0
16724 Error 0 0 0 0
16725 Last Flush: 5527 5527 5225
16726 Downstairs last five completed: 5225 5224 5223 5222 5221
16727 Upstairs last five completed: 5527 5526 5525 5524 5523
16728 CLIENT: Up:0 ds:302 act:2
167292024-11-28T04:53:50.424ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
167302024-11-28T04:53:50.424ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:62219) = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
167312024-11-28T04:53:50.426ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
167322024-11-28T04:53:50.426ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
167332024-11-28T04:53:50.426ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
167342024-11-28T04:53:50.426ZINFOcrucible: send last flush ID to this DS: 5225 = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
167352024-11-28T04:53:50.426ZINFOcrucible: Replied this last flush ID: 5225 = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
167362024-11-28T04:53:50.426ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
167372024-11-28T04:53:50.426ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
167382024-11-28T04:53:50.426ZINFOcrucible: [2] client re-new 302 jobs since flush 5225 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
167392024-11-28T04:53:50.427ZINFOcrucible: [2] Replayed 302 jobs since flush: 5225 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
16740 ----------------------------------------------------------------
16741 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
16742 Downstairs last five completed: 5527 5526 5525 5524 5523
16743 Upstairs last five completed: 5527 5526 5525 5524 5523
16744 CLIENT: Up:0 ds:0 act:3
16745 Replay: All jobs finished
16746 Main task has work to do, go find it
16747 got dsc Stop(1)
16748 stop 1
16749 [8820] Got stop action so:false kr:false
16750 Nov 28 04:53:50.672 INFO request completed, latency_us: 328, response_code: 204, uri: /stop/cid/1, method: POST, req_id: bd0043aa-f4e1-432f-8e7e-8a6f68b05b39, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
167512024-11-28T04:53:50.673ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
16752 Nov 28 04:53:50.673{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2024-11-28T04:53:50.673787285Z","hostname":"w-01JDRHKEEFJ1228TFPSBEGXWHY","pid":1202,"":"io task","client":"1","":"downstairs","session_id":"4a2415b1-6594-436f-82e5-b8cf9a74eeaf"}
16753 INFO request completed, latency_us: 112, response_code: 200, uri:{"msg":"downstairs task for 1 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2024-11-28T04:53:50.673832049Z","hostname":"w-01JDRHKEEFJ1228TFPSBEGXWHY","pid":1202,"session_id":"4a2415b1-6594-436f-82e5-b8cf9a74eeaf"}
16754 /state/cid/1, method: GET, req_id: b46f13e2-0499-4fcc-9d7c-f2f6e7defe64, remote_addr: 127.0.0.1:60418{"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-28T04:53:50.673860995Z","hostname":"w-01JDRHKEEFJ1228TFPSBEGXWHY","pid":1202,"client":"1","":"downstairs","session_id":"4a2415b1-6594-436f-82e5-b8cf9a74eeaf"}
16755 , local_addr: 127.0.0.1:9998
167562024-11-28T04:53:50.673ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
167572024-11-28T04:53:50.673ZINFOcrucible: client task is exiting = io task client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
16758 [8820] Exited with: ExitStatus(unix_wait_status(9))
16759 [8820][1] reports Exit
16760 Nov 28 04:53:54.675 INFO request completed, latency_us: 236, response_code: 200, uri: /state/cid/1, method: GET, req_id: 14af68f6-d428-459f-b780-629067ab3708, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
16761 001/300 Write block 4703 len 20480 data: 2 3 3 3 3
16762 002/300 Read block 1482 len 4096
16763 003/300 Read block 2681 len 32768
16764 004/300 Write block 3857 len 8192 data: 3 4
16765 005/300 Write block 2471 len 8192 data: 7 6
16766 006/300 Write block 1705 len 24576 data: 11 12 12 13 14 12
16767 007/300 Read block 308 len 36864
16768 008/300 Write block 1399 len 20480 data: 6 4 5 6 6
16769 009/300 Read block 4221 len 20480
16770 010/300 Read block 1535 len 16384
16771 011/300 Read block 4853 len 20480
16772 012/300 Read block 714 len 40960
16773 013/300 Write block 4071 len 28672 data: 4 4 3 3 5 5 6
16774 014/300 Read block 2152 len 36864
16775 015/300 Write block 1082 len 4096 data: 4
16776 016/300 Write block 1272 len 40960 data: 6 7 7 5 4 4 4 4 5 5
16777 017/300 Flush
16778 018/300 Read block 2062 len 32768
16779 019/300 Read block 848 len 24576
16780 020/300 Read block 85 len 40960
16781 021/300 Read block 1397 len 16384
16782 022/300 Read block 4908 len 28672
16783 023/300 Read block 435 len 36864
16784 024/300 Read block 4155 len 24576
16785 025/300 Write block 3695 len 4096 data: 8
16786 026/300 Read block 3700 len 40960
16787 027/300 Write block 1181 len 12288 data: 6 6 5
16788 028/300 Write block 293 len 28672 data: 4 4 3 3 3 3 2
16789 029/300 Read block 2441 len 32768
16790 030/300 Read block 2690 len 8192
16791 031/300 Read block 929 len 40960
16792 032/300 Write block 2317 len 16384 data: 4 4 5 6
16793 033/300 Flush
16794 034/300 Read block 4721 len 12288
16795 035/300 Write block 4170 len 8192 data: 2 2
16796 036/300 Read block 1374 len 20480
16797 037/300 Write block 2567 len 36864 data: 5 5 5 6 7 5 4 4 4
16798 038/300 Write block 3614 len 40960 data: 4 3 3 3 3 3 4 4 3 3
16799 039/300 Flush
16800 040/300 Read block 1203 len 36864
16801 041/300 Read block 2876 len 32768
16802 042/300 Write block 1123 len 16384 data: 3 5 5 4
16803 043/300 Flush
16804 044/300 Flush
16805 045/300 Write block 394 len 16384 data: 3 3 3 3
16806 046/300 Read block 939 len 28672
16807 047/300 Read block 1864 len 16384
16808 048/300 Write block 2688 len 20480 data: 5 5 5 3 3
16809 049/300 Read block 1541 len 24576
16810 050/300 Write block 1167 len 28672 data: 2 2 2 2 2 2 2
16811 051/300 Write block 2540 len 28672 data: 4 4 4 2 2 2 2
16812 052/300 Read block 3348 len 16384
16813 053/300 Write block 2257 len 36864 data: 3 2 4 5 6 5 4 4 4
16814 054/300 Write block 4848 len 40960 data: 6 7 6 7 6 7 6 5 4 5
16815 055/300 Read block 1006 len 28672
16816 056/300 Read block 2692 len 24576
16817 057/300 Read block 3788 len 16384
16818 058/300 Read block 4633 len 12288
16819 059/300 Write block 7 len 8192 data: 5 6
16820 060/300 Read block 2131 len 8192
16821 061/300 Write block 1717 len 20480 data: 4 4 3 3 2
16822 062/300 Write block 636 len 40960 data: 8 7 7 7 4 4 4 4 4 4
16823 063/300 Write block 2770 len 40960 data: 3 4 4 6 5 6 6 5 5 5
16824 064/300 Read block 4035 len 24576
16825 065/300 Read block 4610 len 24576
16826 066/300 Write block 4646 len 4096 data: 5
16827 067/300 Read block 4402 len 20480
16828 068/300 Read block 4645 len 24576
16829 069/300 Read block 174 len 24576
16830 070/300 Read block 4344 len 28672
16831 071/300 Flush
16832 072/300 Write block 4843 len 16384 data: 3 4 5 5
16833 073/300 Read block 570 len 24576
16834 074/300 Read block 1260 len 32768
16835 075/300 Read block 473 len 12288
16836 076/300 Write block 2711 len 8192 data: 3 5
16837 077/300 Write block 849 len 4096 data: 4
16838 078/300 Write block 3432 len 4096 data: 6
16839 079/300 Read block 628 len 28672
16840 080/300 Read block 4829 len 40960
16841 081/300 Write block 543 len 36864 data: 5 6 5 5 6 5 5 4 4
16842 082/300 Write block 847 len 24576 data: 4 5 5 5 6 6
16843 083/300 Write block 24 len 28672 data: 4 6 5 4 4 4 6
16844 084/300 Read block 3778 len 28672
16845 085/300 Read block 1214 len 20480
16846 086/300 Write block 3815 len 28672 data: 4 4 3 3 6 6 6
16847 087/300 Read block 1347 len 40960
16848 088/300 Read block 2494 len 4096
16849 089/300 Read block 838 len 28672
16850 090/300 Write block 4087 len 24576 data: 6 6 6 7 8 8
16851 091/300 Write block 302 len 32768 data: 2 2 3 3 4 4 4 4
16852 092/300 Read block 2101 len 8192
16853 093/300 Write block 1046 len 16384 data: 5 6 6 7
16854 094/300 Read block 1869 len 28672
16855 095/300 Read block 3721 len 28672
16856 096/300 Read block 4879 len 32768
16857 097/300 Write block 2735 len 24576 data: 5 5 5 5 4 4
16858 098/300 Read block 157 len 40960
16859 099/300 Write block 770 len 28672 data: 5 4 3 4 3 3 4
16860 100/300 Write block 2950 len 16384 data: 6 5 5 2
16861 101/300 Write block 3931 len 32768 data: 4 3 5 4 3 3 4 3
16862 102/300 Write block 553 len 4096 data: 5
16863 103/300 Read block 1335 len 4096
16864 104/300 Read block 1174 len 20480
16865 105/300 Read block 2437 len 4096
16866 106/300 Write block 2297 len 24576 data: 5 4 3 3 3 3
16867 107/300 Write block 2092 len 36864 data: 5 6 4 3 3 3 3 2 3
16868 108/300 Flush
16869 109/300 Write block 3820 len 16384 data: 7 7 6 6
16870 110/300 Read block 4102 len 24576
16871 111/300 Read block 2923 len 16384
16872 112/300 Read block 247 len 16384
16873 113/300 Write block 2957 len 16384 data: 2 3 4 5
16874 114/300 Flush
16875 115/300 Flush
16876 116/300 Write block 2941 len 4096 data: 3
16877 117/300 Flush
16878 118/300 Read block 4168 len 24576
16879 119/300 Read block 3562 len 8192
16880 120/300 Read block 4810 len 40960
16881 121/300 Flush
16882 122/300 Flush
16883 123/300 Write block 4107 len 12288 data: 3 3 3
16884 124/300 Read block 17 len 40960
16885 125/300 Write block 3746 len 16384 data: 5 5 4 3
16886 126/300 Write block 4310 len 32768 data: 3 3 3 4 4 4 4 3
16887 127/300 Read block 1073 len 4096
16888 128/300 Write block 2506 len 4096 data: 4
16889 129/300 Read block 1296 len 28672
16890 130/300 Write block 3872 len 40960 data: 4 4 3 4 4 3 3 4 4 3
16891 131/300 Flush
16892 132/300 Read block 1770 len 40960
16893 133/300 Flush
16894 134/300 Read block 1900 len 24576
16895 135/300 Write block 1921 len 12288 data: 5 3 2
16896 136/300 Write block 60 len 8192 data: 5 6
16897 137/300 Write block 4500 len 40960 data: 2 2 2 4 4 4 4 4 4 3
16898 138/300 Write block 2555 len 36864 data: 4 4 5 4 4 4 3 3 4
16899 139/300 Read block 1232 len 16384
16900 140/300 Read block 408 len 36864
16901 141/300 Read block 2274 len 36864
16902 142/300 Write block 1258 len 4096 data: 4
16903 143/300 Write block 2738 len 16384 data: 6 5 5 5
16904 144/300 Read block 2393 len 24576
16905 145/300 Read block 1251 len 36864
16906 146/300 Flush
16907 147/300 Read block 1337 len 12288
16908 148/300 Write block 405 len 36864 data: 4 4 6 6 6 6 6 5 5
16909 149/300 Write block 989 len 28672 data: 5 4 4 4 3 3 3
16910 150/300 Write block 2248 len 20480 data: 5 5 4 3 3
16911 151/300 Flush
16912 152/300 Read block 3151 len 32768
16913 153/300 Read block 307 len 36864
16914 154/300 Write block 4449 len 36864 data: 7 8 5 5 4 6 7 6 6
16915 155/300 Read block 4626 len 40960
16916 156/300 Write block 3516 len 28672 data: 4 3 3 4 3 3 3
16917 157/300 Read block 2081 len 20480
16918 158/300 Read block 440 len 16384
16919 159/300 Write block 3489 len 20480 data: 3 3 4 4 4
16920 160/300 Read block 4226 len 20480
16921 161/300 Write block 3599 len 24576 data: 5 6 6 2 2 3
16922 162/300 Read block 4410 len 12288
16923 163/300 Write block 3139 len 40960 data: 4 5 5 2 2 2 2 2 2 2
16924 164/300 Read block 4917 len 24576
16925 165/300 Read block 2258 len 24576
16926 166/300 Read block 480 len 4096
16927 167/300 Read block 2909 len 20480
16928 168/300 Read block 2776 len 28672
16929 169/300 Read block 2240 len 24576
16930 170/300 Read block 4366 len 16384
16931 171/300 Read block 591 len 12288
16932 172/300 Write block 4338 len 12288 data: 2 2 3
16933 173/300 Read block 302 len 28672
16934 174/300 Flush
16935 175/300 Read block 2551 len 4096
16936 176/300 Write block 4363 len 16384 data: 6 6 6 5
16937 177/300 Flush
16938 178/300 Write block 2985 len 20480 data: 2 2 3 3 3
16939 179/300 Read block 2951 len 36864
16940 180/300 Read block 695 len 4096
16941 181/300 Flush
16942 182/300 Read block 4891 len 8192
16943 183/300 Write block 3293 len 24576 data: 3 3 2 2 2 2
16944 184/300 Read block 4412 len 40960
16945 185/300 Flush
16946 186/300 Read block 1732 len 8192
16947 187/300 Read block 3119 len 36864
16948 188/300 Read block 1741 len 16384
16949 189/300 Write block 865 len 8192 data: 6 5
16950 190/300 Write block 1246 len 36864 data: 4 3 2 2 3 4 3 4 3
16951 191/300 Read block 995 len 20480
16952 192/300 Flush
16953 193/300 Write block 270 len 24576 data: 3 4 5 5 5 6
16954 194/300 Write block 3967 len 24576 data: 4 3 3 4 3 3
16955 195/300 Write block 4625 len 8192 data: 4 5
16956 196/300 Read block 4378 len 40960
16957 197/300 Write block 3174 len 16384 data: 6 6 6 6
16958 198/300 Read block 146 len 24576
16959 199/300 Write block 416 len 4096 data: 3
16960 200/300 Flush
16961 201/300 Read block 3736 len 36864
16962 202/300 Write block 4099 len 8192 data: 4 2
16963 203/300 Write block 4689 len 40960 data: 2 2 3 3 3 3 3 3 3 3
16964 204/300 Write block 3202 len 16384 data: 2 3 3 5
16965 205/300 Write block 2097 len 28672 data: 4 4 3 4 3 4 4
16966 206/300 Write block 952 len 8192 data: 3 3
16967 207/300 Read block 1588 len 36864
16968 208/300 Write block 2012 len 4096 data: 5
16969 209/300 Read block 4481 len 24576
16970 210/300 Write block 1419 len 8192 data: 4 4
16971 211/300 Write block 3850 len 4096 data: 4
16972 212/300 Read block 1364 len 20480
16973 213/300 Write block 4376 len 40960 data: 3 4 4 4 4 3 3 4 5 4
16974 214/300 Read block 4014 len 20480
16975 215/300 Flush
16976 216/300 Read block 4351 len 40960
16977 217/300 Read block 1587 len 20480
16978 218/300 Write block 2222 len 20480 data: 3 3 4 4 5
16979 219/300 Write block 397 len 20480 data: 4 3 3 3 3
16980 220/300 Read block 4053 len 24576
16981 221/300 Read block 1420 len 28672
16982 222/300 Read block 1332 len 16384
16983 223/300 Write block 4198 len 24576 data: 2 3 3 3 3 3
16984 224/300 Read block 3534 len 8192
16985 225/300 Read block 4593 len 16384
16986 226/300 Read block 3191 len 40960
16987 227/300 Write block 1288 len 8192 data: 4 4
16988 228/300 Read block 2029 len 36864
16989 229/300 Flush
16990 230/300 Write block 3559 len 28672 data: 4 4 3 3 2 4 4
16991 231/300 Read block 2914 len 8192
16992 232/300 Read block 2733 len 24576
16993 233/300 Read block 1063 len 12288
16994 234/300 Read block 3802 len 36864
16995 235/300 Write block 1682 len 12288 data: 3 3 4
16996 236/300 Read block 3812 len 36864
16997 237/300 Write block 3618 len 32768 data: 4 4 5 5 4 4 3 3
16998 238/300 Read block 4734 len 36864
16999 239/300 Read block 5 len 12288
17000 240/300 Write block 4427 len 36864 data: 3 3 3 4 5 5 4 3 3
17001 241/300 Flush
17002 242/300 Read block 1181 len 4096
17003 243/300 Write block 1221 len 24576 data: 7 7 7 7 6 5
17004 244/300 Read block 2901 len 4096
17005 245/300 Flush
17006 246/300 Write block 4381 len 4096 data: 4
17007 247/300 Write block 4820 len 36864 data: 3 3 2 2 3 3 4 3 3
17008 248/300 Flush
17009 249/300 Read block 960 len 40960
17010 250/300 Write block 2968 len 20480 data: 4 3 2 2 3
17011 251/300 Read block 2708 len 40960
17012 252/300 Write block 1236 len 32768 data: 4 4 4 3 3 3 4 5
17013 253/300 Read block 2211 len 36864
17014 254/300 Write block 406 len 16384 data: 5 7 7 7
17015 255/300 Read block 1649 len 16384
17016 256/300 Read block 1046 len 32768
17017 257/300 Write block 3693 len 20480 data: 7 7 9 8 7
17018 258/300 Write block 2556 len 12288 data: 5 6 5
17019 259/300 Flush
17020 260/300 Write block 3022 len 40960 data: 4 5 6 5 3 3 2 2 2 3
17021 261/300 Write block 2806 len 8192 data: 4 3
17022 262/300 Read block 1056 len 20480
17023 263/300 Read block 447 len 8192
17024 264/300 Read block 3876 len 28672
17025 265/300 Write block 4318 len 40960 data: 3 3 3 3 4 2 2 2 2 3
17026 266/300 Read block 1491 len 4096
17027 267/300 Write block 1764 len 8192 data: 4 3
17028 268/300 Read block 4719 len 32768
17029 269/300 Read block 2263 len 16384
17030 270/300 Write block 67 len 12288 data: 3 3 4
17031 271/300 Read block 1290 len 12288
17032 272/300 Read block 1390 len 12288
17033 273/300 Read block 4486 len 40960
17034 274/300 Write block 2620 len 28672 data: 3 3 3 4 4 4 3
17035 275/300 Read block 2557 len 12288
17036 276/300 Read block 4265 len 16384
17037 277/300 Read block 3964 len 36864
17038 278/300 Read block 292 len 32768
17039 279/300 Read block 2319 len 36864
17040 280/300 Flush
17041 281/300 Write block 3461 len 32768 data: 3 3 3 2 5 4 4 3
17042 282/300 Flush
17043 283/300 Write block 3137 len 40960 data: 4 5 5 6 6 3 3 3 3 3
17044 284/300 Write block 2327 len 8192 data: 3 3
17045 285/300 Write block 4987 len 32768 data: 4 3 4 5 5 5 4 4
17046 286/300 Read block 1601 len 20480
17047 287/300 Flush
17048 288/300 Read block 2605 len 16384
17049 289/300 Write block 2816 len 4096 data: 2
17050 290/300 Read block 3942 len 4096
17051 291/300 Read block 1976 len 12288
17052 292/300 Write block 2868 len 28672 data: 2 2 2 2 2 4 5
17053 293/300 Write block 4190 len 4096 data: 3
17054 294/300 Write block 3925 len 12288 data: 7 5 5
17055 295/300 Write block 4058 len 28672 data: 3 7 7 6 6 4 4
17056 296/300 Read block 4716 len 28672
17057 297/300 Read block 4265 len 36864
17058 298/300 Read block 1834 len 24576
17059 299/300 Read block 4579 len 24576
17060 300/300 Write block 417 len 40960 data: 3 3 3 3 3 3 3 3 3 3
17061 301/300 Write block 4449 len 24576 data: 8 9 6 6 5 7
17062 Main task has work to do, go find it
17063 got dsc Start(1)
17064 start 1
17065 Nov 28 04:53:54.917 INFO request completed, latency_us: 295, response_code: 204, uri: /start/cid/1, method: POST, req_id: 26ba6ed8-6896-4596-b8d6-6a7f0d76a229, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
17066 Replay: started 1, returned:Ok(())
17067 ----------------------------------------------------------------
17068 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
17069 States: Active Offline Active
17070 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
17071 5528 Acked Write 5 Done Sent Done false
17072 5529 Acked Read 1 Done Sent Done false
17073 5530 Acked Read 8 Done Sent Done false
17074 5531 Acked Write 2 Done Sent Done false
17075 5532 Acked Write 2 Done Sent Done false
17076 5533 Acked Write 6 Done Sent Done false
17077 5534 Acked Read 9 Done Sent Done false
17078 5535 Acked Write 5 Done Sent Done false
17079 5536 Acked Read 5 Done Sent Done false
17080 5537 Acked Read 4 Done Sent Done false
17081 5538 Acked Read 5 Done Sent Done false
17082 5539 Acked Read 10 Done Sent Done false
17083 5540 Acked Write 7 Done Sent Done false
17084 5541 Acked Read 9 Done Sent Done false
17085 5542 Acked Write 1 Done Sent Done false
17086 5543 Acked Write 10 Done Sent Done false
17087 5544 Acked Flush 0 Done Sent Done false
17088 5545 Acked Read 8 Done Sent Done false
17089 5546 Acked Read 6 Done Sent Done false
17090 5547 Acked Read 10 Done Sent Done false
17091 5548 Acked Read 4 Done Sent Done false
17092 5549 Acked Read 7 Done Sent Done false
17093 5550 Acked Read 9 Done Sent Done false
17094 5551 Acked Read 6 Done Sent Done false
17095 5552 Acked Write 1 Done Sent Done false
17096 5553 Acked Read 10 Done Sent Done false
17097 5554 Acked Write 3 Done Sent Done false
17098 5555 Acked Write 7 Done Sent Done false
17099 5556 Acked Read 8 Done Sent Done false
17100 5557 Acked Read 2 Done Sent Done false
17101 5558 Acked Read 10 Done Sent Done false
17102 5559 Acked Write 4 Done Sent Done false
17103 5560 Acked Flush 0 Done Sent Done false
17104 5561 Acked Read 3 Done Sent Done false
17105 5562 Acked Write 2 Done Sent Done false
17106 5563 Acked Read 5 Done Sent Done false
17107 5564 Acked Write 9 Done Sent Done false
17108 5565 Acked Write 10 Done Sent Done false
17109 5566 Acked Flush 0 Done Sent Done false
17110 5567 Acked Read 9 Done Sent Done false
17111 5568 Acked Read 8 Done Sent Done false
17112 5569 Acked Write 4 Done Sent Done false
17113 5570 Acked Flush 0 Done Sent Done false
17114 5571 Acked Flush 0 Done Sent Done false
17115 5572 Acked Write 4 Done Sent Done false
17116 5573 Acked Read 7 Done Sent Done false
17117 5574 Acked Read 4 Done Sent Done false
17118 5575 Acked Write 5 Done Sent Done false
17119 5576 Acked Read 6 Done Sent Done false
17120 5577 Acked Write 7 Done Sent Done false
17121 5578 Acked Write 7 Done Sent Done false
17122 5579 Acked Read 4 Done Sent Done false
17123 5580 Acked Write 9 Done Sent Done false
17124 5581 Acked Write 10 Done Sent Done false
17125 5582 Acked Read 7 Done Sent Done false
17126 5583 Acked Read 6 Done Sent Done false
17127 5584 Acked Read 4 Done Sent Done false
17128 5585 Acked Read 3 Done Sent Done false
17129 5586 Acked Write 2 Done Sent Done false
17130 5587 Acked Read 2 Done Sent Done false
17131 5588 Acked Write 5 Done Sent Done false
17132 5589 Acked Write 10 Done Sent Done false
17133 5590 Acked Write 10 Done Sent Done false
17134 5591 Acked Read 6 Done Sent Done false
17135 5592 Acked Read 6 Done Sent Done false
17136 5593 Acked Write 1 Done Sent Done false
17137 5594 Acked Read 5 Done Sent Done false
17138 5595 Acked Read 6 Done Sent Done false
17139 5596 Acked Read 6 Done Sent Done false
17140 5597 Acked Read 7 Done Sent Done false
17141 5598 Acked Flush 0 Done Sent Done false
17142 5599 Acked Write 4 Done Sent Done false
17143 5600 Acked Read 6 Done Sent Done false
17144 5601 Acked Read 8 Done Sent Done false
17145 5602 Acked Read 3 Done Sent Done false
17146 5603 Acked Write 2 Done Sent Done false
17147 5604 Acked Write 1 Done Sent Done false
17148 5605 Acked Write 1 Done Sent Done false
17149 5606 Acked Read 7 Done Sent Done false
17150 5607 Acked Read 10 Done Sent Done false
17151 5608 Acked Write 9 Done Sent Done false
17152 5609 Acked Write 6 Done Sent Done false
17153 5610 Acked Write 7 Done Sent Done false
17154 5611 Acked Read 7 Done Sent Done false
17155 5612 Acked Read 5 Done Sent Done false
17156 5613 Acked Write 7 Done Sent Done false
17157 5614 Acked Read 10 Done Sent Done false
17158 5615 Acked Read 1 Done Sent Done false
17159 5616 Acked Read 7 Done Sent Done false
17160 5617 Acked Write 6 Done Sent Done false
17161 5618 Acked Write 8 Done Sent Done false
17162 5619 Acked Read 2 Done Sent Done false
17163 5620 Acked Write 4 Done Sent Done false
17164 5621 Acked Read 7 Done Sent Done false
17165 5622 Acked Read 7 Done Sent Done false
17166 5623 Acked Read 8 Done Sent Done false
17167 5624 Acked Write 6 Done Sent Done false
17168 5625 Acked Read 10 Done Sent Done false
17169 5626 Acked Write 7 Done Sent Done false
17170 5627 Acked Write 4 Done Sent Done false
17171 5628 Acked Write 8 Done Sent Done false
17172 5629 Acked Write 1 Done Sent Done false
17173 5630 Acked Read 1 Done Sent Done false
17174 5631 Acked Read 5 Done Sent Done false
17175 5632 Acked Read 1 Done Sent Done false
17176 5633 Acked Write 6 Done Sent Done false
17177 5634 Acked Write 9 Done Sent Done false
17178 5635 Acked Flush 0 Done Sent Done false
17179 5636 Acked Write 4 Done Sent Done false
17180 5637 Acked Read 6 Done Sent Done false
17181 5638 Acked Read 4 Done Sent Done false
17182 5639 Acked Read 4 Done Sent Done false
17183 5640 Acked Write 4 Done Sent Done false
17184 5641 Acked Flush 0 Done Sent Done false
17185 5642 Acked Flush 0 Done Sent Done false
17186 5643 Acked Write 1 Done Sent Done false
17187 5644 Acked Flush 0 Done Sent Done false
17188 5645 Acked Read 6 Done Sent Done false
17189 5646 Acked Read 2 Done Sent Done false
17190 5647 Acked Read 10 Done Sent Done false
17191 5648 Acked Flush 0 Done Sent Done false
17192 5649 Acked Flush 0 Done Sent Done false
17193 5650 Acked Write 3 Done Sent Done false
17194 5651 Acked Read 10 Done Sent Done false
17195 5652 Acked Write 4 Done Sent Done false
17196 5653 Acked Write 8 Done Sent Done false
17197 5654 Acked Read 1 Done Sent Done false
17198 5655 Acked Write 1 Done Sent Done false
17199 5656 Acked Read 7 Done Sent Done false
17200 5657 Acked Write 10 Done Sent Done false
17201 5658 Acked Flush 0 Done Sent Done false
17202 5659 Acked Read 10 Done Sent Done false
17203 5660 Acked Flush 0 Done Sent Done false
17204 5661 Acked Read 6 Done Sent Done false
17205 5662 Acked Write 3 Done Sent Done false
17206 5663 Acked Write 2 Done Sent Done false
17207 5664 Acked Write 10 Done Sent Done false
17208 5665 Acked Write 9 Done Sent Done false
17209 5666 Acked Read 4 Done Sent Done false
17210 5667 Acked Read 9 Done Sent Done false
17211 5668 Acked Read 9 Done Sent Done false
17212 5669 Acked Write 1 Done Sent Done false
17213 5670 Acked Write 4 Done Sent Done false
17214 5671 Acked Read 6 Done Sent Done false
17215 5672 Acked Read 9 Done Sent Done false
17216 5673 Acked Flush 0 Done Sent Done false
17217 5674 Acked Read 3 Done Sent Done false
17218 5675 Acked Write 9 Done Sent Done false
17219 5676 Acked Write 7 Done Sent Done false
17220 5677 Acked Write 5 Done Sent Done false
17221 5678 Acked Flush 0 Done Sent Done false
17222 5679 Acked Read 8 Done Sent Done false
17223 5680 Acked Read 9 Done Sent Done false
17224 5681 Acked Write 9 Done Sent Done false
17225 5682 Acked Read 10 Done Sent Done false
17226 5683 Acked Write 7 Done Sent Done false
17227 5684 Acked Read 5 Done Sent Done false
17228 5685 Acked Read 4 Done Sent Done false
17229 5686 Acked Write 5 Done Sent Done false
17230 5687 Acked Read 5 Done Sent Done false
17231 5688 Acked Write 6 Done Sent Done false
17232 5689 Acked Read 3 Done Sent Done false
17233 5690 Acked Write 10 Done Sent Done false
17234 5691 Acked Read 6 Done Sent Done false
17235 5692 Acked Read 6 Done Sent Done false
17236 5693 Acked Read 1 Done Sent Done false
17237 5694 Acked Read 5 Done Sent Done false
17238 5695 Acked Read 7 Done Sent Done false
17239 5696 Acked Read 6 Done Sent Done false
17240 5697 Acked Read 4 Done Sent Done false
17241 5698 Acked Read 3 Done Sent Done false
17242 5699 Acked Write 3 Done Sent Done false
17243 5700 Acked Read 7 Done Sent Done false
17244 5701 Acked Flush 0 Done Sent Done false
17245 5702 Acked Read 1 Done Sent Done false
17246 5703 Acked Write 4 Done Sent Done false
17247 5704 Acked Flush 0 Done Sent Done false
17248 5705 Acked Write 5 Done Sent Done false
17249 5706 Acked Read 9 Done Sent Done false
17250 5707 Acked Read 1 Done Sent Done false
17251 5708 Acked Flush 0 Done Sent Done false
17252 5709 Acked Read 2 Done Sent Done false
17253 5710 Acked Write 6 Done Sent Done false
17254 5711 Acked Read 10 Done Sent Done false
17255 5712 Acked Flush 0 Done Sent Done false
17256 5713 Acked Read 2 Done Sent Done false
17257 5714 Acked Read 9 Done Sent Done false
17258 5715 Acked Read 4 Done Sent Done false
17259 5716 Acked Write 2 Done Sent Done false
17260 5717 Acked Write 9 Done Sent Done false
17261 5718 Acked Read 5 Done Sent Done false
17262 5719 Acked Flush 0 Done Sent Done false
17263 5720 Acked Write 6 Done Sent Done false
17264 5721 Acked Write 6 Done Sent Done false
17265 5722 Acked Write 2 Done Sent Done false
17266 5723 Acked Read 10 Done Sent Done false
17267 5724 Acked Write 4 Done Sent Done false
17268 5725 Acked Read 6 Done Sent Done false
17269 5726 Acked Write 1 Done Sent Done false
17270 5727 Acked Flush 0 Done Sent Done false
17271 5728 Acked Read 9 Done Sent Done false
17272 5729 Acked Write 2 Done Sent Done false
17273 5730 Acked Write 10 Done Sent Done false
17274 5731 Acked Write 4 Done Sent Done false
17275 5732 Acked Write 7 Done Sent Done false
17276 5733 Acked Write 2 Done Sent Done false
17277 5734 Acked Read 9 Done Sent Done false
17278 5735 Acked Write 1 Done Sent Done false
17279 5736 Acked Read 6 Done Sent Done false
17280 5737 Acked Write 2 Done Sent Done false
17281 5738 Acked Write 1 Done Sent Done false
17282 5739 Acked Read 5 Done Sent Done false
17283 5740 Acked Write 10 Done Sent Done false
17284 5741 Acked Read 5 Done Sent Done false
17285 5742 Acked Flush 0 Done Sent Done false
17286 5743 Acked Read 10 Done Sent Done false
17287 5744 Acked Read 5 Done Sent Done false
17288 5745 Acked Write 5 Done Sent Done false
17289 5746 Acked Write 5 Done Sent Done false
17290 5747 Acked Read 6 Done Sent Done false
17291 5748 Acked Read 7 Done Sent Done false
17292 5749 Acked Read 4 Done Sent Done false
17293 5750 Acked Write 6 Done Sent Done false
17294 5751 Acked Read 2 Done Sent Done false
17295 5752 Acked Read 4 Done Sent Done false
17296 5753 Acked Read 10 Done Sent Done false
17297 5754 Acked Write 2 Done Sent Done false
17298 5755 Acked Read 9 Done Sent Done false
17299 5756 Acked Flush 0 Done Sent Done false
17300 5757 Acked Write 7 Done Sent Done false
17301 5758 Acked Read 2 Done Sent Done false
17302 5759 Acked Read 6 Done Sent Done false
17303 5760 Acked Read 3 Done Sent Done false
17304 5761 Acked Read 9 Done Sent Done false
17305 5762 Acked Write 3 Done Sent Done false
17306 5763 Acked Read 9 Done Sent Done false
17307 5764 Acked Write 8 Done Sent Done false
17308 5765 Acked Read 9 Done Sent Done false
17309 5766 Acked Read 3 Done Sent Done false
17310 5767 Acked Write 9 Done Sent Done false
17311 5768 Acked Flush 0 Done Sent Done false
17312 5769 Acked Read 1 Done Sent Done false
17313 5770 Acked Write 6 Done Sent Done false
17314 5771 Acked Read 1 Done Sent Done false
17315 5772 Acked Flush 0 Done Sent Done false
17316 5773 Acked Write 1 Done Sent Done false
17317 5774 Acked Write 9 Done Sent Done false
17318 5775 Acked Flush 0 Done Sent Done false
17319 5776 Acked Read 10 Done Sent Done false
17320 5777 Acked Write 5 Done Sent Done false
17321 5778 Acked Read 10 Done Sent Done false
17322 5779 Acked Write 8 Done Sent Done false
17323 5780 Acked Read 9 Done Sent Done false
17324 5781 Acked Write 4 Done Sent Done false
17325 5782 Acked Read 4 Done Sent Done false
17326 5783 Acked Read 8 Done Sent Done false
17327 5784 Acked Write 5 Done Sent Done false
17328 5785 Acked Write 3 Done Sent Done false
17329 5786 Acked Flush 0 Done Sent Done false
17330 5787 Acked Write 10 Done Sent Done false
17331 5788 Acked Write 2 Done Sent Done false
17332 5789 Acked Read 5 Done Sent Done false
17333 5790 Acked Read 2 Done Sent Done false
17334 5791 Acked Read 7 Done Sent Done false
17335 5792 Acked Write 10 Done Sent Done false
17336 5793 Acked Read 1 Done Sent Done false
17337 5794 Acked Write 2 Done Sent Done false
17338 5795 Acked Read 8 Done Sent Done false
17339 5796 Acked Read 4 Done Sent Done false
17340 5797 Acked Write 3 Done Sent Done false
17341 5798 Acked Read 3 Done Sent Done false
17342 5799 Acked Read 3 Done Sent Done false
17343 5800 Acked Read 10 Done Sent Done false
17344 5801 Acked Write 7 Done Sent Done false
17345 5802 Acked Read 3 Done Sent Done false
17346 5803 Acked Read 4 Done Sent Done false
17347 5804 Acked Read 9 Done Sent Done false
17348 5805 Acked Read 8 Done Sent Done false
17349 5806 Acked Read 9 Done Sent Done false
17350 5807 Acked Flush 0 Done Sent Done false
17351 5808 Acked Write 8 Done Sent Done false
17352 5809 Acked Flush 0 Done Sent Done false
17353 5810 Acked Write 10 Done Sent Done false
17354 5811 Acked Write 2 Done Sent Done false
17355 5812 Acked Write 8 Done Sent Done false
17356 5813 Acked Read 5 Done Sent Done false
17357 5814 Acked Flush 0 Done Sent Done false
17358 5815 Acked Read 4 Done Sent Done false
17359 5816 Acked Write 1 Done Sent Done false
17360 5817 Acked Read 1 Done Sent Done false
17361 5818 Acked Read 3 Done Sent Done false
17362 5819 Acked Write 7 Done Sent Done false
17363 5820 Acked Write 1 Done Sent Done false
17364 5821 Acked Write 3 Done Sent Done false
17365 5822 Acked Write 7 Done Sent Done false
17366 5823 Acked Read 7 Done Sent Done false
17367 5824 Acked Read 9 Done Sent Done false
17368 5825 Acked Read 6 Done Sent Done false
17369 5826 Acked Read 6 Done Sent Done false
17370 5827 Acked Write 10 Done Sent Done false
17371 5828 Acked Write 6 Done Sent Done false
17372 STATES DS:0 DS:1 DS:2 TOTAL
17373 Sent 0 301 0 301
17374 Done 301 0 301 602
17375 Skipped 0 0 0 0
17376 Error 0 0 0 0
17377 Last Flush: 5814 5527 5814
17378 Downstairs last five completed: 5527 5526 5525 5524 5523
17379 Upstairs last five completed: 5828 5827 5826 5825 5824
17380 CLIENT: Up:0 ds:301 act:2
17381 [8820] got start action
17382 [8820] I am going to restart
17383 Starting downstairs at port 8820
17384 Make output file at "/tmp/dsc/downstairs-8820.txt"
17385 [8820][1] reports Starting
17386 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1218)
17387 [8820][1] reports Running
17388 ----------------------------------------------------------------
17389 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
17390 States: Active Offline Active
17391 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
17392 5528 Acked Write 5 Done Sent Done false
17393 5529 Acked Read 1 Done Sent Done false
17394 5530 Acked Read 8 Done Sent Done false
17395 5531 Acked Write 2 Done Sent Done false
17396 5532 Acked Write 2 Done Sent Done false
17397 5533 Acked Write 6 Done Sent Done false
17398 5534 Acked Read 9 Done Sent Done false
17399 5535 Acked Write 5 Done Sent Done false
17400 5536 Acked Read 5 Done Sent Done false
17401 5537 Acked Read 4 Done Sent Done false
17402 5538 Acked Read 5 Done Sent Done false
17403 5539 Acked Read 10 Done Sent Done false
17404 5540 Acked Write 7 Done Sent Done false
17405 5541 Acked Read 9 Done Sent Done false
17406 5542 Acked Write 1 Done Sent Done false
17407 5543 Acked Write 10 Done Sent Done false
17408 5544 Acked Flush 0 Done Sent Done false
17409 5545 Acked Read 8 Done Sent Done false
17410 5546 Acked Read 6 Done Sent Done false
17411 5547 Acked Read 10 Done Sent Done false
17412 5548 Acked Read 4 Done Sent Done false
17413 5549 Acked Read 7 Done Sent Done false
17414 5550 Acked Read 9 Done Sent Done false
17415 5551 Acked Read 6 Done Sent Done false
17416 5552 Acked Write 1 Done Sent Done false
17417 5553 Acked Read 10 Done Sent Done false
17418 5554 Acked Write 3 Done Sent Done false
17419 5555 Acked Write 7 Done Sent Done false
17420 5556 Acked Read 8 Done Sent Done false
17421 5557 Acked Read 2 Done Sent Done false
17422 5558 Acked Read 10 Done Sent Done false
17423 5559 Acked Write 4 Done Sent Done false
17424 5560 Acked Flush 0 Done Sent Done false
17425 5561 Acked Read 3 Done Sent Done false
17426 5562 Acked Write 2 Done Sent Done false
17427 5563 Acked Read 5 Done Sent Done false
17428 5564 Acked Write 9 Done Sent Done false
17429 5565 Acked Write 10 Done Sent Done false
17430 5566 Acked Flush 0 Done Sent Done false
17431 5567 Acked Read 9 Done Sent Done false
17432 5568 Acked Read 8 Done Sent Done false
17433 5569 Acked Write 4 Done Sent Done false
17434 5570 Acked Flush 0 Done Sent Done false
17435 5571 Acked Flush 0 Done Sent Done false
17436 5572 Acked Write 4 Done Sent Done false
17437 5573 Acked Read 7 Done Sent Done false
17438 5574 Acked Read 4 Done Sent Done false
17439 5575 Acked Write 5 Done Sent Done false
17440 5576 Acked Read 6 Done Sent Done false
17441 5577 Acked Write 7 Done Sent Done false
17442 5578 Acked Write 7 Done Sent Done false
17443 5579 Acked Read 4 Done Sent Done false
17444 5580 Acked Write 9 Done Sent Done false
17445 5581 Acked Write 10 Done Sent Done false
17446 5582 Acked Read 7 Done Sent Done false
17447 5583 Acked Read 6 Done Sent Done false
17448 5584 Acked Read 4 Done Sent Done false
17449 5585 Acked Read 3 Done Sent Done false
17450 5586 Acked Write 2 Done Sent Done false
17451 5587 Acked Read 2 Done Sent Done false
17452 5588 Acked Write 5 Done Sent Done false
17453 5589 Acked Write 10 Done Sent Done false
17454 5590 Acked Write 10 Done Sent Done false
17455 5591 Acked Read 6 Done Sent Done false
17456 5592 Acked Read 6 Done Sent Done false
17457 5593 Acked Write 1 Done Sent Done false
17458 5594 Acked Read 5 Done Sent Done false
17459 5595 Acked Read 6 Done Sent Done false
17460 5596 Acked Read 6 Done Sent Done false
17461 5597 Acked Read 7 Done Sent Done false
17462 5598 Acked Flush 0 Done Sent Done false
17463 5599 Acked Write 4 Done Sent Done false
17464 5600 Acked Read 6 Done Sent Done false
17465 5601 Acked Read 8 Done Sent Done false
17466 5602 Acked Read 3 Done Sent Done false
17467 5603 Acked Write 2 Done Sent Done false
17468 5604 Acked Write 1 Done Sent Done false
17469 5605 Acked Write 1 Done Sent Done false
17470 5606 Acked Read 7 Done Sent Done false
17471 5607 Acked Read 10 Done Sent Done false
17472 5608 Acked Write 9 Done Sent Done false
17473 5609 Acked Write 6 Done Sent Done false
17474 5610 Acked Write 7 Done Sent Done false
17475 5611 Acked Read 7 Done Sent Done false
17476 5612 Acked Read 5 Done Sent Done false
17477 5613 Acked Write 7 Done Sent Done false
17478 5614 Acked Read 10 Done Sent Done false
17479 5615 Acked Read 1 Done Sent Done false
17480 5616 Acked Read 7 Done Sent Done false
17481 5617 Acked Write 6 Done Sent Done false
17482 5618 Acked Write 8 Done Sent Done false
17483 5619 Acked Read 2 Done Sent Done false
17484 5620 Acked Write 4 Done Sent Done false
17485 5621 Acked Read 7 Done Sent Done false
17486 5622 Acked Read 7 Done Sent Done false
17487 5623 Acked Read 8 Done Sent Done false
17488 5624 Acked Write 6 Done Sent Done false
17489 5625 Acked Read 10 Done Sent Done false
17490 5626 Acked Write 7 Done Sent Done false
17491 5627 Acked Write 4 Done Sent Done false
17492 5628 Acked Write 8 Done Sent Done false
17493 5629 Acked Write 1 Done Sent Done false
17494 5630 Acked Read 1 Done Sent Done false
17495 5631 Acked Read 5 Done Sent Done false
17496 5632 Acked Read 1 Done Sent Done false
17497 5633 Acked Write 6 Done Sent Done false
17498 5634 Acked Write 9 Done Sent Done false
17499 5635 Acked Flush 0 Done Sent Done false
17500 5636 Acked Write 4 Done Sent Done false
17501 5637 Acked Read 6 Done Sent Done false
17502 5638 Acked Read 4 Done Sent Done false
17503 5639 Acked Read 4 Done Sent Done false
17504 5640 Acked Write 4 Done Sent Done false
17505 5641 Acked Flush 0 Done Sent Done false
17506 5642 Acked Flush 0 Done Sent Done false
17507 5643 Acked Write 1 Done Sent Done false
17508 5644 Acked Flush 0 Done Sent Done false
17509 5645 Acked Read 6 Done Sent Done false
17510 5646 Acked Read 2 Done Sent Done false
17511 5647 Acked Read 10 Done Sent Done false
17512 5648 Acked Flush 0 Done Sent Done false
17513 5649 Acked Flush 0 Done Sent Done false
17514 5650 Acked Write 3 Done Sent Done false
17515 5651 Acked Read 10 Done Sent Done false
17516 5652 Acked Write 4 Done Sent Done false
17517 5653 Acked Write 8 Done Sent Done false
17518 5654 Acked Read 1 Done Sent Done false
17519 5655 Acked Write 1 Done Sent Done false
17520 5656 Acked Read 7 Done Sent Done false
17521 5657 Acked Write 10 Done Sent Done false
17522 5658 Acked Flush 0 Done Sent Done false
17523 5659 Acked Read 10 Done Sent Done false
17524 5660 Acked Flush 0 Done Sent Done false
17525 5661 Acked Read 6 Done Sent Done false
17526 5662 Acked Write 3 Done Sent Done false
17527 5663 Acked Write 2 Done Sent Done false
17528 5664 Acked Write 10 Done Sent Done false
17529 5665 Acked Write 9 Done Sent Done false
17530 5666 Acked Read 4 Done Sent Done false
17531 5667 Acked Read 9 Done Sent Done false
17532 5668 Acked Read 9 Done Sent Done false
17533 5669 Acked Write 1 Done Sent Done false
17534 5670 Acked Write 4 Done Sent Done false
17535 5671 Acked Read 6 Done Sent Done false
17536 5672 Acked Read 9 Done Sent Done false
17537 5673 Acked Flush 0 Done Sent Done false
17538 5674 Acked Read 3 Done Sent Done false
17539 5675 Acked Write 9 Done Sent Done false
17540 5676 Acked Write 7 Done Sent Done false
17541 5677 Acked Write 5 Done Sent Done false
17542 5678 Acked Flush 0 Done Sent Done false
17543 5679 Acked Read 8 Done Sent Done false
17544 5680 Acked Read 9 Done Sent Done false
17545 5681 Acked Write 9 Done Sent Done false
17546 5682 Acked Read 10 Done Sent Done false
17547 5683 Acked Write 7 Done Sent Done false
17548 5684 Acked Read 5 Done Sent Done false
17549 5685 Acked Read 4 Done Sent Done false
17550 5686 Acked Write 5 Done Sent Done false
17551 5687 Acked Read 5 Done Sent Done false
17552 5688 Acked Write 6 Done Sent Done false
17553 5689 Acked Read 3 Done Sent Done false
17554 5690 Acked Write 10 Done Sent Done false
17555 5691 Acked Read 6 Done Sent Done false
17556 5692 Acked Read 6 Done Sent Done false
17557 5693 Acked Read 1 Done Sent Done false
17558 5694 Acked Read 5 Done Sent Done false
17559 5695 Acked Read 7 Done Sent Done false
17560 5696 Acked Read 6 Done Sent Done false
17561 5697 Acked Read 4 Done Sent Done false
17562 5698 Acked Read 3 Done Sent Done false
17563 5699 Acked Write 3 Done Sent Done false
17564 5700 Acked Read 7 Done Sent Done false
17565 5701 Acked Flush 0 Done Sent Done false
17566 5702 Acked Read 1 Done Sent Done false
17567 5703 Acked Write 4 Done Sent Done false
17568 5704 Acked Flush 0 Done Sent Done false
17569 5705 Acked Write 5 Done Sent Done false
17570 5706 Acked Read 9 Done Sent Done false
17571 5707 Acked Read 1 Done Sent Done false
17572 5708 Acked Flush 0 Done Sent Done false
17573 5709 Acked Read 2 Done Sent Done false
17574 5710 Acked Write 6 Done Sent Done false
17575 5711 Acked Read 10 Done Sent Done false
17576 5712 Acked Flush 0 Done Sent Done false
17577 5713 Acked Read 2 Done Sent Done false
17578 5714 Acked Read 9 Done Sent Done false
17579 5715 Acked Read 4 Done Sent Done false
17580 5716 Acked Write 2 Done Sent Done false
17581 5717 Acked Write 9 Done Sent Done false
17582 5718 Acked Read 5 Done Sent Done false
17583 5719 Acked Flush 0 Done Sent Done false
17584 5720 Acked Write 6 Done Sent Done false
17585 5721 Acked Write 6 Done Sent Done false
17586 5722 Acked Write 2 Done Sent Done false
17587 5723 Acked Read 10 Done Sent Done false
17588 5724 Acked Write 4 Done Sent Done false
17589 5725 Acked Read 6 Done Sent Done false
17590 5726 Acked Write 1 Done Sent Done false
17591 5727 Acked Flush 0 Done Sent Done false
17592 5728 Acked Read 9 Done Sent Done false
17593 5729 Acked Write 2 Done Sent Done false
17594 5730 Acked Write 10 Done Sent Done false
17595 5731 Acked Write 4 Done Sent Done false
17596 5732 Acked Write 7 Done Sent Done false
17597 5733 Acked Write 2 Done Sent Done false
17598 5734 Acked Read 9 Done Sent Done false
17599 5735 Acked Write 1 Done Sent Done false
17600 5736 Acked Read 6 Done Sent Done false
17601 5737 Acked Write 2 Done Sent Done false
17602 5738 Acked Write 1 Done Sent Done false
17603 5739 Acked Read 5 Done Sent Done false
17604 5740 Acked Write 10 Done Sent Done false
17605 5741 Acked Read 5 Done Sent Done false
17606 5742 Acked Flush 0 Done Sent Done false
17607 5743 Acked Read 10 Done Sent Done false
17608 5744 Acked Read 5 Done Sent Done false
17609 5745 Acked Write 5 Done Sent Done false
17610 5746 Acked Write 5 Done Sent Done false
17611 5747 Acked Read 6 Done Sent Done false
17612 5748 Acked Read 7 Done Sent Done false
17613 5749 Acked Read 4 Done Sent Done false
17614 5750 Acked Write 6 Done Sent Done false
17615 5751 Acked Read 2 Done Sent Done false
17616 5752 Acked Read 4 Done Sent Done false
17617 5753 Acked Read 10 Done Sent Done false
17618 5754 Acked Write 2 Done Sent Done false
17619 5755 Acked Read 9 Done Sent Done false
17620 5756 Acked Flush 0 Done Sent Done false
17621 5757 Acked Write 7 Done Sent Done false
17622 5758 Acked Read 2 Done Sent Done false
17623 5759 Acked Read 6 Done Sent Done false
17624 5760 Acked Read 3 Done Sent Done false
17625 5761 Acked Read 9 Done Sent Done false
17626 5762 Acked Write 3 Done Sent Done false
17627 5763 Acked Read 9 Done Sent Done false
17628 5764 Acked Write 8 Done Sent Done false
17629 5765 Acked Read 9 Done Sent Done false
17630 5766 Acked Read 3 Done Sent Done false
17631 5767 Acked Write 9 Done Sent Done false
17632 5768 Acked Flush 0 Done Sent Done false
17633 5769 Acked Read 1 Done Sent Done false
17634 5770 Acked Write 6 Done Sent Done false
17635 5771 Acked Read 1 Done Sent Done false
17636 5772 Acked Flush 0 Done Sent Done false
17637 5773 Acked Write 1 Done Sent Done false
17638 5774 Acked Write 9 Done Sent Done false
17639 5775 Acked Flush 0 Done Sent Done false
17640 5776 Acked Read 10 Done Sent Done false
17641 5777 Acked Write 5 Done Sent Done false
17642 5778 Acked Read 10 Done Sent Done false
17643 5779 Acked Write 8 Done Sent Done false
17644 5780 Acked Read 9 Done Sent Done false
17645 5781 Acked Write 4 Done Sent Done false
17646 5782 Acked Read 4 Done Sent Done false
17647 5783 Acked Read 8 Done Sent Done false
17648 5784 Acked Write 5 Done Sent Done false
17649 5785 Acked Write 3 Done Sent Done false
17650 5786 Acked Flush 0 Done Sent Done false
17651 5787 Acked Write 10 Done Sent Done false
17652 5788 Acked Write 2 Done Sent Done false
17653 5789 Acked Read 5 Done Sent Done false
17654 5790 Acked Read 2 Done Sent Done false
17655 5791 Acked Read 7 Done Sent Done false
17656 5792 Acked Write 10 Done Sent Done false
17657 5793 Acked Read 1 Done Sent Done false
17658 5794 Acked Write 2 Done Sent Done false
17659 5795 Acked Read 8 Done Sent Done false
17660 5796 Acked Read 4 Done Sent Done false
17661 5797 Acked Write 3 Done Sent Done false
17662 5798 Acked Read 3 Done Sent Done false
17663 5799 Acked Read 3 Done Sent Done false
17664 5800 Acked Read 10 Done Sent Done false
17665 5801 Acked Write 7 Done Sent Done false
17666 5802 Acked Read 3 Done Sent Done false
17667 5803 Acked Read 4 Done Sent Done false
17668 5804 Acked Read 9 Done Sent Done false
17669 5805 Acked Read 8 Done Sent Done false
17670 5806 Acked Read 9 Done Sent Done false
17671 5807 Acked Flush 0 Done Sent Done false
17672 5808 Acked Write 8 Done Sent Done false
17673 5809 Acked Flush 0 Done Sent Done false
17674 5810 Acked Write 10 Done Sent Done false
17675 5811 Acked Write 2 Done Sent Done false
17676 5812 Acked Write 8 Done Sent Done false
17677 5813 Acked Read 5 Done Sent Done false
17678 5814 Acked Flush 0 Done Sent Done false
17679 5815 Acked Read 4 Done Sent Done false
17680 5816 Acked Write 1 Done Sent Done false
17681 5817 Acked Read 1 Done Sent Done false
17682 5818 Acked Read 3 Done Sent Done false
17683 5819 Acked Write 7 Done Sent Done false
17684 5820 Acked Write 1 Done Sent Done false
17685 5821 Acked Write 3 Done Sent Done false
17686 5822 Acked Write 7 Done Sent Done false
17687 5823 Acked Read 7 Done Sent Done false
17688 5824 Acked Read 9 Done Sent Done false
17689 5825 Acked Read 6 Done Sent Done false
17690 5826 Acked Read 6 Done Sent Done false
17691 5827 Acked Write 10 Done Sent Done false
17692 5828 Acked Write 6 Done Sent Done false
17693 5829 Acked Flush 0 Done Sent Done false
17694 STATES DS:0 DS:1 DS:2 TOTAL
17695 Sent 0 302 0 302
17696 Done 302 0 302 604
17697 Skipped 0 0 0 0
17698 Error 0 0 0 0
17699 Last Flush: 5829 5527 5829
17700 Downstairs last five completed: 5527 5526 5525 5524 5523
17701 Upstairs last five completed: 5829 5828 5827 5826 5825
17702 CLIENT: Up:0 ds:302 act:2
177032024-11-28T04:54:00.675ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
177042024-11-28T04:54:00.675ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:54202) = io task client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
177052024-11-28T04:54:00.676ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
177062024-11-28T04:54:00.676ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
177072024-11-28T04:54:00.676ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
177082024-11-28T04:54:00.676ZINFOcrucible: send last flush ID to this DS: 5527 = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
177092024-11-28T04:54:00.677ZINFOcrucible: Replied this last flush ID: 5527 = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
177102024-11-28T04:54:00.677ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
177112024-11-28T04:54:00.677ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
177122024-11-28T04:54:00.677ZINFOcrucible: [1] client re-new 302 jobs since flush 5527 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
177132024-11-28T04:54:00.677ZINFOcrucible: [1] Replayed 302 jobs since flush: 5527 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
17714 ----------------------------------------------------------------
17715 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
17716 Downstairs last five completed: 5829 5828 5827 5826 5825
17717 Upstairs last five completed: 5829 5828 5827 5826 5825
17718 CLIENT: Up:0 ds:0 act:3
17719 Replay: All jobs finished
17720 Main task has work to do, go find it
17721 got dsc Stop(1)
17722 stop 1
17723 [8820] Got stop action so:false kr:false
17724 Nov 28 04:54:02.923 INFO request completed, latency_us: 335, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 730d0dde-9237-4628-a91e-95f1970d79fe, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
177252024-11-28T04:54:02.924ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
177262024-11-28T04:54:02.924ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
177272024-11-28T04:54:02.924ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
177282024-11-28T04:54:02.924ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
177292024-11-28T04:54:02.924ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
177302024-11-28T04:54:02.924ZINFOcrucible: client task is exiting = io task client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
17731 Nov 28 04:54:02.924 INFO request completed, latency_us: 162, response_code: 200, uri: /state/cid/1, method: GET, req_id: 1471090d-c7fb-4fbd-81d4-3cb45774ac6a, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
17732 [8820] Exited with: ExitStatus(unix_wait_status(9))
17733 [8820][1] reports Exit
17734 Nov 28 04:54:06.926 INFO request completed, latency_us: 195, response_code: 200, uri: /state/cid/1, method: GET, req_id: 298e5eed-ec97-4060-945b-3d24ff65c16a, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
17735 001/300 Read block 4893 len 32768
17736 002/300 Read block 384 len 36864
17737 003/300 Read block 3207 len 16384
17738 004/300 Read block 1185 len 36864
17739 005/300 Write block 2900 len 32768 data: 3 4 4 3 3 3 3 3
17740 006/300 Read block 2371 len 32768
17741 007/300 Write block 4054 len 16384 data: 5 4 3 3
17742 008/300 Write block 3995 len 12288 data: 4 4 4
17743 009/300 Flush
17744 010/300 Write block 62 len 20480 data: 5 5 4 3 3
17745 011/300 Read block 2637 len 4096
17746 012/300 Read block 4481 len 12288
17747 013/300 Write block 436 len 4096 data: 6
17748 014/300 Write block 4081 len 24576 data: 8 6 5 7 5 5
17749 015/300 Read block 4730 len 4096
17750 016/300 Read block 837 len 4096
17751 017/300 Read block 2534 len 32768
17752 018/300 Read block 97 len 8192
17753 019/300 Read block 4754 len 20480
17754 020/300 Read block 752 len 36864
17755 021/300 Read block 4892 len 16384
17756 022/300 Flush
17757 023/300 Write block 2866 len 28672 data: 2 2 3 3 3 3 3
17758 024/300 Write block 1719 len 32768 data: 4 4 3 2 2 3 4 2
17759 025/300 Read block 960 len 16384
17760 026/300 Write block 2682 len 40960 data: 3 3 3 5 5 5 6 6 6 4
17761 027/300 Read block 1386 len 4096
17762 028/300 Write block 2758 len 8192 data: 4 4
17763 029/300 Write block 4506 len 40960 data: 5 5 5 4 3 2 2 2 2 3
17764 030/300 Read block 2214 len 20480
17765 031/300 Read block 2741 len 24576
17766 032/300 Write block 4564 len 24576 data: 5 6 6 5 5 6
17767 033/300 Write block 3344 len 28672 data: 6 5 4 4 2 2 2
17768 034/300 Write block 819 len 36864 data: 5 4 4 3 3 3 4 6 6
17769 035/300 Write block 609 len 12288 data: 3 3 2
17770 036/300 Write block 2757 len 4096 data: 5
17771 037/300 Read block 4861 len 28672
17772 038/300 Read block 3903 len 40960
17773 039/300 Write block 1808 len 40960 data: 3 2 2 3 3 3 4 5 5 4
17774 040/300 Read block 450 len 28672
17775 041/300 Write block 2736 len 32768 data: 6 6 7 6 6 6 3 3
17776 042/300 Flush
17777 043/300 Read block 2415 len 4096
17778 044/300 Read block 4737 len 12288
17779 045/300 Write block 2393 len 4096 data: 4
17780 046/300 Flush
17781 047/300 Read block 3917 len 12288
17782 048/300 Read block 1991 len 16384
17783 049/300 Read block 804 len 28672
17784 050/300 Read block 4942 len 4096
17785 051/300 Flush
17786 052/300 Write block 523 len 20480 data: 3 4 3 3 4
17787 053/300 Flush
17788 054/300 Flush
17789 055/300 Write block 2671 len 24576 data: 4 4 3 3 3 3
17790 056/300 Flush
17791 057/300 Read block 1680 len 24576
17792 058/300 Read block 634 len 40960
17793 059/300 Write block 1046 len 8192 data: 6 7
17794 060/300 Write block 3691 len 4096 data: 5
17795 061/300 Read block 2350 len 4096
17796 062/300 Write block 3674 len 36864 data: 6 6 5 4 3 3 3 3 2
17797 063/300 Write block 2859 len 12288 data: 3 3 3
17798 064/300 Read block 1383 len 40960
17799 065/300 Flush
17800 066/300 Write block 4932 len 32768 data: 4 3 3 3 3 3 3 3
17801 067/300 Flush
17802 068/300 Read block 2566 len 24576
17803 069/300 Write block 4462 len 4096 data: 3
17804 070/300 Write block 3461 len 28672 data: 4 4 4 3 6 5 5
17805 071/300 Write block 4480 len 12288 data: 4 3 3
17806 072/300 Read block 1665 len 16384
17807 073/300 Write block 2784 len 24576 data: 5 4 4 5 8 9
17808 074/300 Flush
17809 075/300 Read block 155 len 16384
17810 076/300 Read block 2918 len 24576
17811 077/300 Read block 1424 len 28672
17812 078/300 Read block 70 len 36864
17813 079/300 Read block 920 len 40960
17814 080/300 Write block 2271 len 36864 data: 3 5 4 4 4 3 3 3 3
17815 081/300 Write block 3601 len 24576 data: 7 3 3 4 3 3
17816 082/300 Read block 820 len 24576
17817 083/300 Write block 1610 len 12288 data: 6 5 5
17818 084/300 Read block 2121 len 24576
17819 085/300 Flush
17820 086/300 Write block 1258 len 4096 data: 5
17821 087/300 Read block 2145 len 4096
17822 088/300 Write block 1176 len 40960 data: 4 4 4 5 5 7 7 6 4 3
17823 089/300 Read block 1654 len 28672
17824 090/300 Write block 3730 len 24576 data: 2 2 3 3 3 3
17825 091/300 Write block 4755 len 40960 data: 4 3 3 3 3 3 3 3 2 2
17826 092/300 Flush
17827 093/300 Write block 1272 len 4096 data: 7
17828 094/300 Read block 4199 len 16384
17829 095/300 Flush
17830 096/300 Read block 1364 len 24576
17831 097/300 Write block 4458 len 28672 data: 6 5 4 4 4 4 4
17832 098/300 Read block 3535 len 36864
17833 099/300 Read block 153 len 12288
17834 100/300 Write block 1529 len 20480 data: 4 2 3 3 4
17835 101/300 Write block 801 len 32768 data: 3 3 4 3 3 4 4 4
17836 102/300 Read block 3220 len 32768
17837 103/300 Write block 1927 len 12288 data: 2 2 2
17838 104/300 Read block 3337 len 32768
17839 105/300 Write block 464 len 32768 data: 3 3 4 3 6 7 7 6
17840 106/300 Flush
17841 107/300 Read block 3598 len 36864
17842 108/300 Write block 4527 len 16384 data: 4 3 4 4
17843 109/300 Read block 3799 len 12288
17844 110/300 Write block 2808 len 8192 data: 3 4
17845 111/300 Read block 4305 len 12288
17846 112/300 Write block 2339 len 8192 data: 5 5
17847 113/300 Read block 3601 len 4096
17848 114/300 Read block 2631 len 12288
17849 115/300 Read block 1996 len 12288
17850 116/300 Write block 83 len 12288 data: 4 3 4
17851 117/300 Write block 2392 len 16384 data: 4 5 5 6
17852 118/300 Write block 4795 len 36864 data: 4 4 6 6 6 5 4 4 5
17853 119/300 Write block 522 len 4096 data: 3
17854 120/300 Flush
17855 121/300 Write block 2929 len 4096 data: 4
17856 122/300 Write block 4222 len 16384 data: 3 4 4 3
17857 123/300 Read block 1313 len 20480
17858 124/300 Write block 3194 len 20480 data: 4 4 2 2 3
17859 125/300 Write block 1654 len 12288 data: 2 2 3
17860 126/300 Write block 3364 len 24576 data: 2 2 2 2 2 2
17861 127/300 Write block 885 len 32768 data: 5 4 4 3 2 4 5 5
17862 128/300 Write block 4207 len 4096 data: 2
17863 129/300 Read block 3265 len 28672
17864 130/300 Write block 4094 len 8192 data: 6 6
17865 131/300 Write block 2345 len 28672 data: 2 3 3 4 4 4 4
17866 132/300 Read block 4916 len 12288
17867 133/300 Flush
17868 134/300 Read block 2607 len 16384
17869 135/300 Read block 2091 len 24576
17870 136/300 Read block 4833 len 28672
17871 137/300 Write block 3297 len 12288 data: 3 3 2
17872 138/300 Write block 527 len 12288 data: 5 3 3
17873 139/300 Read block 1073 len 20480
17874 140/300 Flush
17875 141/300 Read block 903 len 16384
17876 142/300 Read block 2792 len 32768
17877 143/300 Read block 4960 len 16384
17878 144/300 Flush
17879 145/300 Read block 1013 len 28672
17880 146/300 Read block 1072 len 28672
17881 147/300 Read block 4750 len 32768
17882 148/300 Write block 2923 len 16384 data: 5 5 4 5
17883 149/300 Read block 4145 len 12288
17884 150/300 Read block 1231 len 24576
17885 151/300 Read block 1233 len 24576
17886 152/300 Write block 2815 len 24576 data: 2 3 2 2 2 2
17887 153/300 Write block 1314 len 8192 data: 2 2
17888 154/300 Read block 721 len 12288
17889 155/300 Write block 4264 len 16384 data: 3 3 6 5
17890 156/300 Write block 4520 len 40960 data: 4 4 5 5 6 6 5 5 4 5
17891 157/300 Flush
17892 158/300 Read block 2637 len 28672
17893 159/300 Write block 1532 len 4096 data: 4
17894 160/300 Write block 2917 len 20480 data: 4 3 5 6 6
17895 161/300 Write block 3307 len 36864 data: 4 4 4 4 4 3 3 4 5
17896 162/300 Write block 2548 len 40960 data: 2 2 3 4 4 4 4 5 6 7
17897 163/300 Write block 3164 len 40960 data: 4 5 5 4 4 4 3 4 5 6
17898 164/300 Write block 1198 len 20480 data: 4 2 2 2 2
17899 165/300 Flush
17900 166/300 Write block 3464 len 32768 data: 4 7 6 6 4 2 2 2
17901 167/300 Read block 2377 len 36864
17902 168/300 Flush
17903 169/300 Read block 2575 len 8192
17904 170/300 Read block 3862 len 20480
17905 171/300 Read block 1614 len 36864
17906 172/300 Write block 1817 len 36864 data: 5 3 4 5 4 3 3 4 4
17907 173/300 Read block 4523 len 24576
17908 174/300 Read block 750 len 28672
17909 175/300 Flush
17910 176/300 Write block 1721 len 12288 data: 4 3 3
17911 177/300 Write block 1674 len 24576 data: 4 4 3 3 3 3
17912 178/300 Write block 4529 len 32768 data: 6 5 5 6 3 5 6 5
17913 179/300 Read block 3180 len 12288
17914 180/300 Write block 3980 len 8192 data: 3 3
17915 181/300 Write block 1261 len 24576 data: 6 6 6 6 7 6
17916 182/300 Read block 3627 len 8192
17917 183/300 Read block 4019 len 28672
17918 184/300 Write block 1427 len 12288 data: 5 5 5
17919 185/300 Read block 1422 len 12288
17920 186/300 Write block 4789 len 20480 data: 3 4 4 5 4
17921 187/300 Write block 640 len 24576 data: 5 5 5 5 5 5
17922 188/300 Write block 1060 len 8192 data: 4 4
17923 189/300 Write block 1640 len 4096 data: 3
17924 190/300 Write block 1631 len 4096 data: 4
17925 191/300 Read block 1728 len 24576
17926 192/300 Write block 557 len 16384 data: 6 5 5 5
17927 193/300 Read block 3276 len 20480
17928 194/300 Read block 1014 len 16384
17929 195/300 Read block 2340 len 32768
17930 196/300 Read block 4657 len 28672
17931 197/300 Flush
17932 198/300 Flush
17933 199/300 Read block 4238 len 4096
17934 200/300 Read block 4136 len 16384
17935 201/300 Read block 4114 len 8192
17936 202/300 Write block 3340 len 16384 data: 4 6 7 6
17937 203/300 Read block 4504 len 36864
17938 204/300 Read block 3202 len 16384
17939 205/300 Write block 1318 len 20480 data: 2 2 2 3 4
17940 206/300 Write block 1336 len 16384 data: 4 4 4 4
17941 207/300 Read block 1851 len 24576
17942 208/300 Write block 413 len 16384 data: 6 6 5 4
17943 209/300 Read block 9 len 36864
17944 210/300 Read block 1211 len 20480
17945 211/300 Read block 4994 len 20480
17946 212/300 Write block 1204 len 32768 data: 3 3 4 4 4 4 5 5
17947 213/300 Read block 1183 len 4096
17948 214/300 Write block 2953 len 8192 data: 3 2
17949 215/300 Read block 4069 len 16384
17950 216/300 Read block 1408 len 12288
17951 217/300 Write block 2775 len 32768 data: 7 7 6 6 6 5 4 3
17952 218/300 Read block 3632 len 24576
17953 219/300 Read block 1405 len 16384
17954 220/300 Read block 3888 len 32768
17955 221/300 Read block 488 len 40960
17956 222/300 Read block 3257 len 8192
17957 223/300 Read block 4048 len 4096
17958 224/300 Write block 943 len 28672 data: 2 3 3 3 4 4 3
17959 225/300 Read block 656 len 40960
17960 226/300 Write block 2253 len 12288 data: 3 3 4
17961 227/300 Read block 1940 len 20480
17962 228/300 Flush
17963 229/300 Write block 4927 len 8192 data: 4 3
17964 230/300 Read block 3815 len 8192
17965 231/300 Read block 725 len 12288
17966 232/300 Read block 1176 len 12288
17967 233/300 Write block 2065 len 36864 data: 5 5 5 4 4 4 3 3 2
17968 234/300 Flush
17969 235/300 Write block 4954 len 36864 data: 5 5 4 4 5 4 3 3 3
17970 236/300 Write block 3527 len 8192 data: 4 4
17971 237/300 Write block 1254 len 20480 data: 4 3 4 3 6
17972 238/300 Write block 377 len 32768 data: 6 7 8 7 6 7 7 5
17973 239/300 Write block 2075 len 40960 data: 2 3 3 4 4 5 6 5 6 7
17974 240/300 Flush
17975 241/300 Flush
17976 242/300 Write block 1640 len 40960 data: 4 3 3 3 3 2 2 2 2 2
17977 243/300 Read block 2975 len 16384
17978 244/300 Write block 3845 len 8192 data: 3 3
17979 245/300 Read block 2013 len 4096
17980 246/300 Read block 236 len 4096
17981 247/300 Write block 1783 len 32768 data: 5 5 4 5 4 3 5 5
17982 248/300 Read block 1602 len 8192
17983 249/300 Write block 1092 len 20480 data: 4 3 3 3 4
17984 250/300 Write block 2218 len 4096 data: 4
17985 251/300 Read block 2924 len 8192
17986 252/300 Write block 2955 len 28672 data: 2 2 3 4 5 6 6
17987 253/300 Write block 489 len 20480 data: 3 3 3 3 2
17988 254/300 Read block 132 len 36864
17989 255/300 Write block 3997 len 24576 data: 5 3 3 3 2 3
17990 256/300 Write block 524 len 24576 data: 5 4 4 6 4 4
17991 257/300 Write block 610 len 40960 data: 4 3 4 4 4 5 6 6 6 5
17992 258/300 Write block 4596 len 28672 data: 6 6 6 6 6 5 5
17993 259/300 Read block 279 len 4096
17994 260/300 Read block 2234 len 40960
17995 261/300 Write block 457 len 20480 data: 3 3 3 3 3
17996 262/300 Read block 3372 len 36864
17997 263/300 Read block 2516 len 16384
17998 264/300 Write block 3809 len 16384 data: 3 4 5 5
17999 265/300 Read block 1705 len 16384
18000 266/300 Write block 2766 len 40960 data: 3 2 3 3 4 5 5 7 6 8
18001 267/300 Read block 4053 len 32768
18002 268/300 Read block 4472 len 40960
18003 269/300 Read block 992 len 4096
18004 270/300 Read block 4847 len 28672
18005 271/300 Read block 3124 len 40960
18006 272/300 Write block 3637 len 36864 data: 6 5 5 5 5 5 5 6 6
18007 273/300 Write block 620 len 12288 data: 4 5 5
18008 274/300 Write block 2585 len 32768 data: 3 3 5 4 3 2 2 2
18009 275/300 Read block 3222 len 36864
18010 276/300 Write block 1336 len 40960 data: 5 5 5 5 4 5 6 5 5 3
18011 277/300 Write block 3797 len 36864 data: 4 6 4 4 4 4 4 4 4
18012 278/300 Read block 953 len 16384
18013 279/300 Write block 577 len 36864 data: 2 2 2 2 2 2 3 3 3
18014 280/300 Write block 4704 len 8192 data: 4 4
18015 281/300 Write block 775 len 20480 data: 4 5 4 4 3
18016 282/300 Read block 2802 len 32768
18017 283/300 Write block 3909 len 20480 data: 4 3 3 3 2
18018 284/300 Write block 3680 len 28672 data: 4 4 3 2 2 2 3
18019 285/300 Read block 4214 len 20480
18020 286/300 Write block 1080 len 40960 data: 4 5 5 4 4 3 3 4 4 5
18021 287/300 Read block 2530 len 32768
18022 288/300 Flush
18023 289/300 Write block 3623 len 32768 data: 5 4 4 3 4 5 5 5
18024 290/300 Write block 3577 len 20480 data: 2 2 3 4 4
18025 291/300 Read block 270 len 16384
18026 292/300 Write block 2868 len 16384 data: 4 4 4 4
18027 293/300 Read block 770 len 8192
18028 294/300 Read block 179 len 20480
18029 295/300 Write block 2138 len 4096 data: 4
18030 296/300 Read block 1916 len 16384
18031 297/300 Write block 786 len 36864 data: 3 3 3 2 2 2 2 3 3
18032 298/300 Write block 3890 len 24576 data: 4 4 3 4 3 3
18033 299/300 Read block 3985 len 20480
18034 300/300 Flush
18035 301/300 Read block 1248 len 32768
18036 Main task has work to do, go find it
18037 got dsc Start(1)
18038 start 1
18039 Nov 28 04:54:07.152 INFO request completed, latency_us: 211, response_code: 204, uri: /start/cid/1, method: POST, req_id: 63ead7c9-3d29-4b3d-9fa7-28772fed060a, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
18040 Replay: started 1, returned:Ok(())
18041 ----------------------------------------------------------------
18042 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
18043 States: Active Offline Active
18044 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
18045 5830 Acked Read 8 Done Sent Done false
18046 5831 Acked Read 9 Done Sent Done false
18047 5832 Acked Read 4 Done Sent Done false
18048 5833 Acked Read 9 Done Sent Done false
18049 5834 Acked Write 8 Done Sent Done false
18050 5835 Acked Read 8 Done Sent Done false
18051 5836 Acked Write 4 Done Sent Done false
18052 5837 Acked Write 3 Done Sent Done false
18053 5838 Acked Flush 0 Done Sent Done false
18054 5839 Acked Write 5 Done Sent Done false
18055 5840 Acked Read 1 Done Sent Done false
18056 5841 Acked Read 3 Done Sent Done false
18057 5842 Acked Write 1 Done Sent Done false
18058 5843 Acked Write 6 Done Sent Done false
18059 5844 Acked Read 1 Done Sent Done false
18060 5845 Acked Read 1 Done Sent Done false
18061 5846 Acked Read 8 Done Sent Done false
18062 5847 Acked Read 2 Done Sent Done false
18063 5848 Acked Read 5 Done Sent Done false
18064 5849 Acked Read 9 Done Sent Done false
18065 5850 Acked Read 4 Done Sent Done false
18066 5851 Acked Flush 0 Done Sent Done false
18067 5852 Acked Write 7 Done Sent Done false
18068 5853 Acked Write 8 Done Sent Done false
18069 5854 Acked Read 4 Done Sent Done false
18070 5855 Acked Write 10 Done Sent Done false
18071 5856 Acked Read 1 Done Sent Done false
18072 5857 Acked Write 2 Done Sent Done false
18073 5858 Acked Write 10 Done Sent Done false
18074 5859 Acked Read 5 Done Sent Done false
18075 5860 Acked Read 6 Done Sent Done false
18076 5861 Acked Write 6 Done Sent Done false
18077 5862 Acked Write 7 Done Sent Done false
18078 5863 Acked Write 9 Done Sent Done false
18079 5864 Acked Write 3 Done Sent Done false
18080 5865 Acked Write 1 Done Sent Done false
18081 5866 Acked Read 7 Done Sent Done false
18082 5867 Acked Read 10 Done Sent Done false
18083 5868 Acked Write 10 Done Sent Done false
18084 5869 Acked Read 7 Done Sent Done false
18085 5870 Acked Write 8 Done Sent Done false
18086 5871 Acked Flush 0 Done Sent Done false
18087 5872 Acked Read 1 Done Sent Done false
18088 5873 Acked Read 3 Done Sent Done false
18089 5874 Acked Write 1 Done Sent Done false
18090 5875 Acked Flush 0 Done Sent Done false
18091 5876 Acked Read 3 Done Sent Done false
18092 5877 Acked Read 4 Done Sent Done false
18093 5878 Acked Read 7 Done Sent Done false
18094 5879 Acked Read 1 Done Sent Done false
18095 5880 Acked Flush 0 Done Sent Done false
18096 5881 Acked Write 5 Done Sent Done false
18097 5882 Acked Flush 0 Done Sent Done false
18098 5883 Acked Flush 0 Done Sent Done false
18099 5884 Acked Write 6 Done Sent Done false
18100 5885 Acked Flush 0 Done Sent Done false
18101 5886 Acked Read 6 Done Sent Done false
18102 5887 Acked Read 10 Done Sent Done false
18103 5888 Acked Write 2 Done Sent Done false
18104 5889 Acked Write 1 Done Sent Done false
18105 5890 Acked Read 1 Done Sent Done false
18106 5891 Acked Write 9 Done Sent Done false
18107 5892 Acked Write 3 Done Sent Done false
18108 5893 Acked Read 10 Done Sent Done false
18109 5894 Acked Flush 0 Done Sent Done false
18110 5895 Acked Write 8 Done Sent Done false
18111 5896 Acked Flush 0 Done Sent Done false
18112 5897 Acked Read 6 Done Sent Done false
18113 5898 Acked Write 1 Done Sent Done false
18114 5899 Acked Write 7 Done Sent Done false
18115 5900 Acked Write 3 Done Sent Done false
18116 5901 Acked Read 4 Done Sent Done false
18117 5902 Acked Write 6 Done Sent Done false
18118 5903 Acked Flush 0 Done Sent Done false
18119 5904 Acked Read 4 Done Sent Done false
18120 5905 Acked Read 6 Done Sent Done false
18121 5906 Acked Read 7 Done Sent Done false
18122 5907 Acked Read 9 Done Sent Done false
18123 5908 Acked Read 10 Done Sent Done false
18124 5909 Acked Write 9 Done Sent Done false
18125 5910 Acked Write 6 Done Sent Done false
18126 5911 Acked Read 6 Done Sent Done false
18127 5912 Acked Write 3 Done Sent Done false
18128 5913 Acked Read 6 Done Sent Done false
18129 5914 Acked Flush 0 Done Sent Done false
18130 5915 Acked Write 1 Done Sent Done false
18131 5916 Acked Read 1 Done Sent Done false
18132 5917 Acked Write 10 Done Sent Done false
18133 5918 Acked Read 7 Done Sent Done false
18134 5919 Acked Write 6 Done Sent Done false
18135 5920 Acked Write 10 Done Sent Done false
18136 5921 Acked Flush 0 Done Sent Done false
18137 5922 Acked Write 1 Done Sent Done false
18138 5923 Acked Read 4 Done Sent Done false
18139 5924 Acked Flush 0 Done Sent Done false
18140 5925 Acked Read 6 Done Sent Done false
18141 5926 Acked Write 7 Done Sent Done false
18142 5927 Acked Read 9 Done Sent Done false
18143 5928 Acked Read 3 Done Sent Done false
18144 5929 Acked Write 5 Done Sent Done false
18145 5930 Acked Write 8 Done Sent Done false
18146 5931 Acked Read 8 Done Sent Done false
18147 5932 Acked Write 3 Done Sent Done false
18148 5933 Acked Read 8 Done Sent Done false
18149 5934 Acked Write 8 Done Sent Done false
18150 5935 Acked Flush 0 Done Sent Done false
18151 5936 Acked Read 9 Done Sent Done false
18152 5937 Acked Write 4 Done Sent Done false
18153 5938 Acked Read 3 Done Sent Done false
18154 5939 Acked Write 2 Done Sent Done false
18155 5940 Acked Read 3 Done Sent Done false
18156 5941 Acked Write 2 Done Sent Done false
18157 5942 Acked Read 1 Done Sent Done false
18158 5943 Acked Read 3 Done Sent Done false
18159 5944 Acked Read 3 Done Sent Done false
18160 5945 Acked Write 3 Done Sent Done false
18161 5946 Acked Write 4 Done Sent Done false
18162 5947 Acked Write 9 Done Sent Done false
18163 5948 Acked Write 1 Done Sent Done false
18164 5949 Acked Flush 0 Done Sent Done false
18165 5950 Acked Write 1 Done Sent Done false
18166 5951 Acked Write 4 Done Sent Done false
18167 5952 Acked Read 5 Done Sent Done false
18168 5953 Acked Write 5 Done Sent Done false
18169 5954 Acked Write 3 Done Sent Done false
18170 5955 Acked Write 6 Done Sent Done false
18171 5956 Acked Write 8 Done Sent Done false
18172 5957 Acked Write 1 Done Sent Done false
18173 5958 Acked Read 7 Done Sent Done false
18174 5959 Acked Write 2 Done Sent Done false
18175 5960 Acked Write 7 Done Sent Done false
18176 5961 Acked Read 3 Done Sent Done false
18177 5962 Acked Flush 0 Done Sent Done false
18178 5963 Acked Read 4 Done Sent Done false
18179 5964 Acked Read 6 Done Sent Done false
18180 5965 Acked Read 7 Done Sent Done false
18181 5966 Acked Write 3 Done Sent Done false
18182 5967 Acked Write 3 Done Sent Done false
18183 5968 Acked Read 5 Done Sent Done false
18184 5969 Acked Flush 0 Done Sent Done false
18185 5970 Acked Read 4 Done Sent Done false
18186 5971 Acked Read 8 Done Sent Done false
18187 5972 Acked Read 4 Done Sent Done false
18188 5973 Acked Flush 0 Done Sent Done false
18189 5974 Acked Read 7 Done Sent Done false
18190 5975 Acked Read 7 Done Sent Done false
18191 5976 Acked Read 8 Done Sent Done false
18192 5977 Acked Write 4 Done Sent Done false
18193 5978 Acked Read 3 Done Sent Done false
18194 5979 Acked Read 6 Done Sent Done false
18195 5980 Acked Read 6 Done Sent Done false
18196 5981 Acked Write 6 Done Sent Done false
18197 5982 Acked Write 2 Done Sent Done false
18198 5983 Acked Read 3 Done Sent Done false
18199 5984 Acked Write 4 Done Sent Done false
18200 5985 Acked Write 10 Done Sent Done false
18201 5986 Acked Flush 0 Done Sent Done false
18202 5987 Acked Read 7 Done Sent Done false
18203 5988 Acked Write 1 Done Sent Done false
18204 5989 Acked Write 5 Done Sent Done false
18205 5990 Acked Write 9 Done Sent Done false
18206 5991 Acked Write 10 Done Sent Done false
18207 5992 Acked Write 10 Done Sent Done false
18208 5993 Acked Write 5 Done Sent Done false
18209 5994 Acked Flush 0 Done Sent Done false
18210 5995 Acked Write 8 Done Sent Done false
18211 5996 Acked Read 9 Done Sent Done false
18212 5997 Acked Flush 0 Done Sent Done false
18213 5998 Acked Read 2 Done Sent Done false
18214 5999 Acked Read 5 Done Sent Done false
18215 6000 Acked Read 9 Done Sent Done false
18216 6001 Acked Write 9 Done Sent Done false
18217 6002 Acked Read 6 Done Sent Done false
18218 6003 Acked Read 7 Done Sent Done false
18219 6004 Acked Flush 0 Done Sent Done false
18220 6005 Acked Write 3 Done Sent Done false
18221 6006 Acked Write 6 Done Sent Done false
18222 6007 Acked Write 8 Done Sent Done false
18223 6008 Acked Read 3 Done Sent Done false
18224 6009 Acked Write 2 Done Sent Done false
18225 6010 Acked Write 6 Done Sent Done false
18226 6011 Acked Read 2 Done Sent Done false
18227 6012 Acked Read 7 Done Sent Done false
18228 6013 Acked Write 3 Done Sent Done false
18229 6014 Acked Read 3 Done Sent Done false
18230 6015 Acked Write 5 Done Sent Done false
18231 6016 Acked Write 6 Done Sent Done false
18232 6017 Acked Write 2 Done Sent Done false
18233 6018 Acked Write 1 Done Sent Done false
18234 6019 Acked Write 1 Done Sent Done false
18235 6020 Acked Read 6 Done Sent Done false
18236 6021 Acked Write 4 Done Sent Done false
18237 6022 Acked Read 5 Done Sent Done false
18238 6023 Acked Read 4 Done Sent Done false
18239 6024 Acked Read 8 Done Sent Done false
18240 6025 Acked Read 7 Done Sent Done false
18241 6026 Acked Flush 0 Done Sent Done false
18242 6027 Acked Flush 0 Done Sent Done false
18243 6028 Acked Read 1 Done Sent Done false
18244 6029 Acked Read 4 Done Sent Done false
18245 6030 Acked Read 2 Done Sent Done false
18246 6031 Acked Write 4 Done Sent Done false
18247 6032 Acked Read 9 Done Sent Done false
18248 6033 Acked Read 4 Done Sent Done false
18249 6034 Acked Write 5 Done Sent Done false
18250 6035 Acked Write 4 Done Sent Done false
18251 6036 Acked Read 6 Done Sent Done false
18252 6037 Acked Write 4 Done Sent Done false
18253 6038 Acked Read 9 Done Sent Done false
18254 6039 Acked Read 5 Done Sent Done false
18255 6040 Acked Read 5 Done Sent Done false
18256 6041 Acked Write 8 Done Sent Done false
18257 6042 Acked Read 1 Done Sent Done false
18258 6043 Acked Write 2 Done Sent Done false
18259 6044 Acked Read 4 Done Sent Done false
18260 6045 Acked Read 3 Done Sent Done false
18261 6046 Acked Write 8 Done Sent Done false
18262 6047 Acked Read 6 Done Sent Done false
18263 6048 Acked Read 4 Done Sent Done false
18264 6049 Acked Read 8 Done Sent Done false
18265 6050 Acked Read 10 Done Sent Done false
18266 6051 Acked Read 2 Done Sent Done false
18267 6052 Acked Read 1 Done Sent Done false
18268 6053 Acked Write 7 Done Sent Done false
18269 6054 Acked Read 10 Done Sent Done false
18270 6055 Acked Write 3 Done Sent Done false
18271 6056 Acked Read 5 Done Sent Done false
18272 6057 Acked Flush 0 Done Sent Done false
18273 6058 Acked Write 2 Done Sent Done false
18274 6059 Acked Read 2 Done Sent Done false
18275 6060 Acked Read 3 Done Sent Done false
18276 6061 Acked Read 3 Done Sent Done false
18277 6062 Acked Write 9 Done Sent Done false
18278 6063 Acked Flush 0 Done Sent Done false
18279 6064 Acked Write 9 Done Sent Done false
18280 6065 Acked Write 2 Done Sent Done false
18281 6066 Acked Write 5 Done Sent Done false
18282 6067 Acked Write 8 Done Sent Done false
18283 6068 Acked Write 10 Done Sent Done false
18284 6069 Acked Flush 0 Done Sent Done false
18285 6070 Acked Flush 0 Done Sent Done false
18286 6071 Acked Write 10 Done Sent Done false
18287 6072 Acked Read 4 Done Sent Done false
18288 6073 Acked Write 2 Done Sent Done false
18289 6074 Acked Read 1 Done Sent Done false
18290 6075 Acked Read 1 Done Sent Done false
18291 6076 Acked Write 8 Done Sent Done false
18292 6077 Acked Read 2 Done Sent Done false
18293 6078 Acked Write 5 Done Sent Done false
18294 6079 Acked Write 1 Done Sent Done false
18295 6080 Acked Read 2 Done Sent Done false
18296 6081 Acked Write 7 Done Sent Done false
18297 6082 Acked Write 5 Done Sent Done false
18298 6083 Acked Read 9 Done Sent Done false
18299 6084 Acked Write 6 Done Sent Done false
18300 6085 Acked Write 6 Done Sent Done false
18301 6086 Acked Write 10 Done Sent Done false
18302 6087 Acked Write 7 Done Sent Done false
18303 6088 Acked Read 1 Done Sent Done false
18304 6089 Acked Read 10 Done Sent Done false
18305 6090 Acked Write 5 Done Sent Done false
18306 6091 Acked Read 9 Done Sent Done false
18307 6092 Acked Read 4 Done Sent Done false
18308 6093 Acked Write 4 Done Sent Done false
18309 6094 Acked Read 4 Done Sent Done false
18310 6095 Acked Write 10 Done Sent Done false
18311 6096 Acked Read 8 Done Sent Done false
18312 6097 Acked Read 10 Done Sent Done false
18313 6098 Acked Read 1 Done Sent Done false
18314 6099 Acked Read 7 Done Sent Done false
18315 6100 Acked Read 10 Done Sent Done false
18316 6101 Acked Write 9 Done Sent Done false
18317 6102 Acked Write 3 Done Sent Done false
18318 6103 Acked Write 8 Done Sent Done false
18319 6104 Acked Read 9 Done Sent Done false
18320 6105 Acked Write 10 Done Sent Done false
18321 6106 Acked Write 9 Done Sent Done false
18322 6107 Acked Read 4 Done Sent Done false
18323 6108 Acked Write 9 Done Sent Done false
18324 6109 Acked Write 2 Done Sent Done false
18325 6110 Acked Write 5 Done Sent Done false
18326 6111 Acked Read 8 Done Sent Done false
18327 6112 Acked Write 5 Done Sent Done false
18328 6113 Acked Write 7 Done Sent Done false
18329 6114 Acked Read 5 Done Sent Done false
18330 6115 Acked Write 10 Done Sent Done false
18331 6116 Acked Read 8 Done Sent Done false
18332 6117 Acked Flush 0 Done Sent Done false
18333 6118 Acked Write 8 Done Sent Done false
18334 6119 Acked Write 5 Done Sent Done false
18335 6120 Acked Read 4 Done Sent Done false
18336 6121 Acked Write 4 Done Sent Done false
18337 6122 Acked Read 2 Done Sent Done false
18338 6123 Acked Read 5 Done Sent Done false
18339 6124 Acked Write 1 Done Sent Done false
18340 6125 Acked Read 4 Done Sent Done false
18341 6126 Acked Write 9 Done Sent Done false
18342 6127 Acked Write 6 Done Sent Done false
18343 6128 Acked Read 5 Done Sent Done false
18344 6129 Acked Flush 0 Done Sent Done false
18345 6130 Acked Read 8 Done Sent Done false
18346 STATES DS:0 DS:1 DS:2 TOTAL
18347 Sent 0 301 0 301
18348 Done 301 0 301 602
18349 Skipped 0 0 0 0
18350 Error 0 0 0 0
18351 Last Flush: 6129 5829 6129
18352 Downstairs last five completed: 5829 5828 5827 5826 5825
18353 Upstairs last five completed: 6130 6129 6128 6127 6126
18354 CLIENT: Up:0 ds:301 act:2
18355 ----------------------------------------------------------------
18356 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
18357 States: Active Offline Active
18358 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
18359 5830 Acked Read 8 Done Sent Done false
18360 5831 Acked Read 9 Done Sent Done false
18361 5832 Acked Read 4 Done Sent Done false
18362 5833 Acked Read 9 Done Sent Done false
18363 5834 Acked Write 8 Done Sent Done false
18364 5835 Acked Read 8 Done Sent Done false
18365 5836 Acked Write 4 Done Sent Done false
18366 5837 Acked Write 3 Done Sent Done false
18367 5838 Acked Flush 0 Done Sent Done false
18368 5839 Acked Write 5 Done Sent Done false
18369 5840 Acked Read 1 Done Sent Done false
18370 5841 Acked Read 3 Done Sent Done false
18371 5842 Acked Write 1 Done Sent Done false
18372 5843 Acked Write 6 Done Sent Done false
18373 5844 Acked Read 1 Done Sent Done false
18374 5845 Acked Read 1 Done Sent Done false
18375 5846 Acked Read 8 Done Sent Done false
18376 5847 Acked Read 2 Done Sent Done false
18377 5848 Acked Read 5 Done Sent Done false
18378 5849 Acked Read 9 Done Sent Done false
18379 5850 Acked Read 4 Done Sent Done false
18380 5851 Acked Flush 0 Done Sent Done false
18381 5852 Acked Write 7 Done Sent Done false
18382 5853 Acked Write 8 Done Sent Done false
18383 5854 Acked Read 4 Done Sent Done false
18384 5855 Acked Write 10 Done Sent Done false
18385 5856 Acked Read 1 Done Sent Done false
18386 5857 Acked Write 2 Done Sent Done false
18387 5858 Acked Write 10 Done Sent Done false
18388 5859 Acked Read 5 Done Sent Done false
18389 5860 Acked Read 6 Done Sent Done false
18390 5861 Acked Write 6 Done Sent Done false
18391 5862 Acked Write 7 Done Sent Done false
18392 5863 Acked Write 9 Done Sent Done false
18393 5864 Acked Write 3 Done Sent Done false
18394 5865 Acked Write 1 Done Sent Done false
18395 5866 Acked Read 7 Done Sent Done false
18396 5867 Acked Read 10 Done Sent Done false
18397 5868 Acked Write 10 Done Sent Done false
18398 5869 Acked Read 7 Done Sent Done false
18399 5870 Acked Write 8 Done Sent Done false
18400 5871 Acked Flush 0 Done Sent Done false
18401 5872 Acked Read 1 Done Sent Done false
18402 5873 Acked Read 3 Done Sent Done false
18403 5874 Acked Write 1 Done Sent Done false
18404 5875 Acked Flush 0 Done Sent Done false
18405 5876 Acked Read 3 Done Sent Done false
18406 5877 Acked Read 4 Done Sent Done false
18407 5878 Acked Read 7 Done Sent Done false
18408 5879 Acked Read 1 Done Sent Done false
18409 5880 Acked Flush 0 Done Sent Done false
18410 5881 Acked Write 5 Done Sent Done false
18411 5882 Acked Flush 0 Done Sent Done false
18412 5883 Acked Flush 0 Done Sent Done false
18413 5884 Acked Write 6 Done Sent Done false
18414 5885 Acked Flush 0 Done Sent Done false
18415 5886 Acked Read 6 Done Sent Done false
18416 5887 Acked Read 10 Done Sent Done false
18417 5888 Acked Write 2 Done Sent Done false
18418 5889 Acked Write 1 Done Sent Done false
18419 5890 Acked Read 1 Done Sent Done false
18420 5891 Acked Write 9 Done Sent Done false
18421 5892 Acked Write 3 Done Sent Done false
18422 5893 Acked Read 10 Done Sent Done false
18423 5894 Acked Flush 0 Done Sent Done false
18424 5895 Acked Write 8 Done Sent Done false
18425 5896 Acked Flush 0 Done Sent Done false
18426 5897 Acked Read 6 Done Sent Done false
18427 5898 Acked Write 1 Done Sent Done false
18428 5899 Acked Write 7 Done Sent Done false
18429 5900 Acked Write 3 Done Sent Done false
18430 5901 Acked Read 4 Done Sent Done false
18431 5902 Acked Write 6 Done Sent Done false
18432 5903 Acked Flush 0 Done Sent Done false
18433 5904 Acked Read 4 Done Sent Done false
18434 5905 Acked Read 6 Done Sent Done false
18435 5906 Acked Read 7 Done Sent Done false
18436 5907 Acked Read 9 Done Sent Done false
18437 5908 Acked Read 10 Done Sent Done false
18438 5909 Acked Write 9 Done Sent Done false
18439 5910 Acked Write 6 Done Sent Done false
18440 5911 Acked Read 6 Done Sent Done false
18441 5912 Acked Write 3 Done Sent Done false
18442 5913 Acked Read 6 Done Sent Done false
18443 5914 Acked Flush 0 Done Sent Done false
18444 5915 Acked Write 1 Done Sent Done false
18445 5916 Acked Read 1 Done Sent Done false
18446 5917 Acked Write 10 Done Sent Done false
18447 5918 Acked Read 7 Done Sent Done false
18448 5919 Acked Write 6 Done Sent Done false
18449 5920 Acked Write 10 Done Sent Done false
18450 5921 Acked Flush 0 Done Sent Done false
18451 5922 Acked Write 1 Done Sent Done false
18452 5923 Acked Read 4 Done Sent Done false
18453 5924 Acked Flush 0 Done Sent Done false
18454 5925 Acked Read 6 Done Sent Done false
18455 5926 Acked Write 7 Done Sent Done false
18456 5927 Acked Read 9 Done Sent Done false
18457 5928 Acked Read 3 Done Sent Done false
18458 5929 Acked Write 5 Done Sent Done false
18459 5930 Acked Write 8 Done Sent Done false
18460 5931 Acked Read 8 Done Sent Done false
18461 5932 Acked Write 3 Done Sent Done false
18462 5933 Acked Read 8 Done Sent Done false
18463 5934 Acked Write 8 Done Sent Done false
18464 5935 Acked Flush 0 Done Sent Done false
18465 5936 Acked Read 9 Done Sent Done false
18466 5937 Acked Write 4 Done Sent Done false
18467 5938 Acked Read 3 Done Sent Done false
18468 5939 Acked Write 2 Done Sent Done false
18469 5940 Acked Read 3 Done Sent Done false
18470 5941 Acked Write 2 Done Sent Done false
18471 5942 Acked Read 1 Done Sent Done false
18472 5943 Acked Read 3 Done Sent Done false
18473 5944 Acked Read 3 Done Sent Done false
18474 5945 Acked Write 3 Done Sent Done false
18475 5946 Acked Write 4 Done Sent Done false
18476 5947 Acked Write 9 Done Sent Done false
18477 5948 Acked Write 1 Done Sent Done false
18478 5949 Acked Flush 0 Done Sent Done false
18479 5950 Acked Write 1 Done Sent Done false
18480 5951 Acked Write 4 Done Sent Done false
18481 5952 Acked Read 5 Done Sent Done false
18482 5953 Acked Write 5 Done Sent Done false
18483 5954 Acked Write 3 Done Sent Done false
18484 5955 Acked Write 6 Done Sent Done false
18485 5956 Acked Write 8 Done Sent Done false
18486 5957 Acked Write 1 Done Sent Done false
18487 5958 Acked Read 7 Done Sent Done false
18488 5959 Acked Write 2 Done Sent Done false
18489 5960 Acked Write 7 Done Sent Done false
18490 5961 Acked Read 3 Done Sent Done false
18491 5962 Acked Flush 0 Done Sent Done false
18492 5963 Acked Read 4 Done Sent Done false
18493 5964 Acked Read 6 Done Sent Done false
18494 5965 Acked Read 7 Done Sent Done false
18495 5966 Acked Write 3 Done Sent Done false
18496 5967 Acked Write 3 Done Sent Done false
18497 5968 Acked Read 5 Done Sent Done false
18498 5969 Acked Flush 0 Done Sent Done false
18499 5970 Acked Read 4 Done Sent Done false
18500 5971 Acked Read 8 Done Sent Done false
18501 5972 Acked Read 4 Done Sent Done false
18502 5973 Acked Flush 0 Done Sent Done false
18503 5974 Acked Read 7 Done Sent Done false
18504 5975 Acked Read 7 Done Sent Done false
18505 5976 Acked Read 8 Done Sent Done false
18506 5977 Acked Write 4 Done Sent Done false
18507 5978 Acked Read 3 Done Sent Done false
18508 5979 Acked Read 6 Done Sent Done false
18509 5980 Acked Read 6 Done Sent Done false
18510 5981 Acked Write 6 Done Sent Done false
18511 5982 Acked Write 2 Done Sent Done false
18512 5983 Acked Read 3 Done Sent Done false
18513 5984 Acked Write 4 Done Sent Done false
18514 5985 Acked Write 10 Done Sent Done false
18515 5986 Acked Flush 0 Done Sent Done false
18516 5987 Acked Read 7 Done Sent Done false
18517 5988 Acked Write 1 Done Sent Done false
18518 5989 Acked Write 5 Done Sent Done false
18519 5990 Acked Write 9 Done Sent Done false
18520 5991 Acked Write 10 Done Sent Done false
18521 5992 Acked Write 10 Done Sent Done false
18522 5993 Acked Write 5 Done Sent Done false
18523 5994 Acked Flush 0 Done Sent Done false
18524 5995 Acked Write 8 Done Sent Done false
18525 5996 Acked Read 9 Done Sent Done false
18526 5997 Acked Flush 0 Done Sent Done false
18527 5998 Acked Read 2 Done Sent Done false
18528 5999 Acked Read 5 Done Sent Done false
18529 6000 Acked Read 9 Done Sent Done false
18530 6001 Acked Write 9 Done Sent Done false
18531 6002 Acked Read 6 Done Sent Done false
18532 6003 Acked Read 7 Done Sent Done false
18533 6004 Acked Flush 0 Done Sent Done false
18534 6005 Acked Write 3 Done Sent Done false
18535 6006 Acked Write 6 Done Sent Done false
18536 6007 Acked Write 8 Done Sent Done false
18537 6008 Acked Read 3 Done Sent Done false
18538 6009 Acked Write 2 Done Sent Done false
18539 6010 Acked Write 6 Done Sent Done false
18540 6011 Acked Read 2 Done Sent Done false
18541 6012 Acked Read 7 Done Sent Done false
18542 6013 Acked Write 3 Done Sent Done false
18543 6014 Acked Read 3 Done Sent Done false
18544 6015 Acked Write 5 Done Sent Done false
18545 6016 Acked Write 6 Done Sent Done false
18546 6017 Acked Write 2 Done Sent Done false
18547 6018 Acked Write 1 Done Sent Done false
18548 6019 Acked Write 1 Done Sent Done false
18549 6020 Acked Read 6 Done Sent Done false
18550 6021 Acked Write 4 Done Sent Done false
18551 6022 Acked Read 5 Done Sent Done false
18552 6023 Acked Read 4 Done Sent Done false
18553 6024 Acked Read 8 Done Sent Done false
18554 6025 Acked Read 7 Done Sent Done false
18555 6026 Acked Flush 0 Done Sent Done false
18556 6027 Acked Flush 0 Done Sent Done false
18557 6028 Acked Read 1 Done Sent Done false
18558 6029 Acked Read 4 Done Sent Done false
18559 6030 Acked Read 2 Done Sent Done false
18560 6031 Acked Write 4 Done Sent Done false
18561 6032 Acked Read 9 Done Sent Done false
18562 6033 Acked Read 4 Done Sent Done false
18563 6034 Acked Write 5 Done Sent Done false
18564 6035 Acked Write 4 Done Sent Done false
18565 6036 Acked Read 6 Done Sent Done false
18566 6037 Acked Write 4 Done Sent Done false
18567 6038 Acked Read 9 Done Sent Done false
18568 6039 Acked Read 5 Done Sent Done false
18569 6040 Acked Read 5 Done Sent Done false
18570 6041 Acked Write 8 Done Sent Done false
18571 6042 Acked Read 1 Done Sent Done false
18572 6043 Acked Write 2 Done Sent Done false
18573 6044 Acked Read 4 Done Sent Done false
18574 6045 Acked Read 3 Done Sent Done false
18575 6046 Acked Write 8 Done Sent Done false
18576 6047 Acked Read 6 Done Sent Done false
18577 6048 Acked Read 4 Done Sent Done false
18578 6049 Acked Read 8 Done Sent Done false
18579 6050 Acked Read 10 Done Sent Done false
18580 6051 Acked Read 2 Done Sent Done false
18581 6052 Acked Read 1 Done Sent Done false
18582 6053 Acked Write 7 Done Sent Done false
18583 6054 Acked Read 10 Done Sent Done false
18584 6055 Acked Write 3 Done Sent Done false
18585 6056 Acked Read 5 Done Sent Done false
18586 6057 Acked Flush 0 Done Sent Done false
18587 6058 Acked Write 2 Done Sent Done false
18588 6059 Acked Read 2 Done Sent Done false
18589 6060 Acked Read 3 Done Sent Done false
18590 6061 Acked Read 3 Done Sent Done false
18591 6062 Acked Write 9 Done Sent Done false
18592 6063 Acked Flush 0 Done Sent Done false
18593 6064 Acked Write 9 Done Sent Done false
18594 6065 Acked Write 2 Done Sent Done false
18595 6066 Acked Write 5 Done Sent Done false
18596 6067 Acked Write 8 Done Sent Done false
18597 6068 Acked Write 10 Done Sent Done false
18598 6069 Acked Flush 0 Done Sent Done false
18599 6070 Acked Flush 0 Done Sent Done false
18600 6071 Acked Write 10 Done Sent Done false
18601 6072 Acked Read 4 Done Sent Done false
18602 6073 Acked Write 2 Done Sent Done false
18603 6074 Acked Read 1 Done Sent Done false
18604 6075 Acked Read 1 Done Sent Done false
18605 6076 Acked Write 8 Done Sent Done false
18606 6077 Acked Read 2 Done Sent Done false
18607 6078 Acked Write 5 Done Sent Done false
18608 6079 Acked Write 1 Done Sent Done false
18609 6080 Acked Read 2 Done Sent Done false
18610 6081 Acked Write 7 Done Sent Done false
18611 6082 Acked Write 5 Done Sent Done false
18612 6083 Acked Read 9 Done Sent Done false
18613 6084 Acked Write 6 Done Sent Done false
18614 6085 Acked Write 6 Done Sent Done false
18615 6086 Acked Write 10 Done Sent Done false
18616 6087 Acked Write 7 Done Sent Done false
18617 6088 Acked Read 1 Done Sent Done false
18618 6089 Acked Read 10 Done Sent Done false
18619 6090 Acked Write 5 Done Sent Done false
18620 6091 Acked Read 9 Done Sent Done false
18621 6092 Acked Read 4 Done Sent Done false
18622 6093 Acked Write 4 Done Sent Done false
18623 6094 Acked Read 4 Done Sent Done false
18624 6095 Acked Write 10 Done Sent Done false
18625 6096 Acked Read 8 Done Sent Done false
18626 6097 Acked Read 10 Done Sent Done false
18627 6098 Acked Read 1 Done Sent Done false
18628 6099 Acked Read 7 Done Sent Done false
18629 6100 Acked Read 10 Done Sent Done false
18630 6101 Acked Write 9 Done Sent Done false
18631 6102 Acked Write 3 Done Sent Done false
18632 6103 Acked Write 8 Done Sent Done false
18633 6104 Acked Read 9 Done Sent Done false
18634 6105 Acked Write 10 Done Sent Done false
18635 6106 Acked Write 9 Done Sent Done false
18636 6107 Acked Read 4 Done Sent Done false
18637 6108 Acked Write 9 Done Sent Done false
18638 6109 Acked Write 2 Done Sent Done false
18639 6110 Acked Write 5 Done Sent Done false
18640 6111 Acked Read 8 Done Sent Done false
18641 6112 Acked Write 5 Done Sent Done false
18642 6113 Acked Write 7 Done Sent Done false
18643 6114 Acked Read 5 Done Sent Done false
18644 6115 Acked Write 10 Done Sent Done false
18645 6116 Acked Read 8 Done Sent Done false
18646 6117 Acked Flush 0 Done Sent Done false
18647 6118 Acked Write 8 Done Sent Done false
18648 6119 Acked Write 5 Done Sent Done false
18649 6120 Acked Read 4 Done Sent Done false
18650 6121 Acked Write 4 Done Sent Done false
18651 6122 Acked Read 2 Done Sent Done false
18652 6123 Acked Read 5 Done Sent Done false
18653 6124 Acked Write 1 Done Sent Done false
18654 6125 Acked Read 4 Done Sent Done false
18655 6126 Acked Write 9 Done Sent Done false
18656 6127 Acked Write 6 Done Sent Done false
18657 6128 Acked Read 5 Done Sent Done false
18658 6129 Acked Flush 0 Done Sent Done false
18659 6130 Acked Read 8 Done Sent Done false
18660 6131 Acked Flush 0 Done Sent Done false
18661 STATES DS:0 DS:1 DS:2 TOTAL
18662 Sent 0 302 0 302
18663 Done 302 0 302 604
18664 Skipped 0 0 0 0
18665 Error 0 0 0 0
18666 Last Flush: 6131 5829 6131
18667 Downstairs last five completed: 5829 5828 5827 5826 5825
18668 Upstairs last five completed: 6131 6130 6129 6128 6127
18669 CLIENT: Up:0 ds:302 act:2
186702024-11-28T04:54:12.924ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
186712024-11-28T04:54:12.924ZWARNcrucible: ds_connection connect to 127.0.0.1:8820 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
186722024-11-28T04:54:12.925ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
186732024-11-28T04:54:12.925ZWARNcrucible: downstairs task for 1 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
186742024-11-28T04:54:12.925ZINFOcrucible: client task is exiting = io task client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
18675 [8820] got start action
18676 [8820] I am going to restart
18677 Starting downstairs at port 8820
18678 Make output file at "/tmp/dsc/downstairs-8820.txt"
18679 [8820][1] reports Starting
18680 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1219)
18681 [8820][1] reports Running
18682 ----------------------------------------------------------------
18683 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
18684 States: Active Offline Active
18685 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
18686 5830 Acked Read 8 Done Sent Done false
18687 5831 Acked Read 9 Done Sent Done false
18688 5832 Acked Read 4 Done Sent Done false
18689 5833 Acked Read 9 Done Sent Done false
18690 5834 Acked Write 8 Done Sent Done false
18691 5835 Acked Read 8 Done Sent Done false
18692 5836 Acked Write 4 Done Sent Done false
18693 5837 Acked Write 3 Done Sent Done false
18694 5838 Acked Flush 0 Done Sent Done false
18695 5839 Acked Write 5 Done Sent Done false
18696 5840 Acked Read 1 Done Sent Done false
18697 5841 Acked Read 3 Done Sent Done false
18698 5842 Acked Write 1 Done Sent Done false
18699 5843 Acked Write 6 Done Sent Done false
18700 5844 Acked Read 1 Done Sent Done false
18701 5845 Acked Read 1 Done Sent Done false
18702 5846 Acked Read 8 Done Sent Done false
18703 5847 Acked Read 2 Done Sent Done false
18704 5848 Acked Read 5 Done Sent Done false
18705 5849 Acked Read 9 Done Sent Done false
18706 5850 Acked Read 4 Done Sent Done false
18707 5851 Acked Flush 0 Done Sent Done false
18708 5852 Acked Write 7 Done Sent Done false
18709 5853 Acked Write 8 Done Sent Done false
18710 5854 Acked Read 4 Done Sent Done false
18711 5855 Acked Write 10 Done Sent Done false
18712 5856 Acked Read 1 Done Sent Done false
18713 5857 Acked Write 2 Done Sent Done false
18714 5858 Acked Write 10 Done Sent Done false
18715 5859 Acked Read 5 Done Sent Done false
18716 5860 Acked Read 6 Done Sent Done false
18717 5861 Acked Write 6 Done Sent Done false
18718 5862 Acked Write 7 Done Sent Done false
18719 5863 Acked Write 9 Done Sent Done false
18720 5864 Acked Write 3 Done Sent Done false
18721 5865 Acked Write 1 Done Sent Done false
18722 5866 Acked Read 7 Done Sent Done false
18723 5867 Acked Read 10 Done Sent Done false
18724 5868 Acked Write 10 Done Sent Done false
18725 5869 Acked Read 7 Done Sent Done false
18726 5870 Acked Write 8 Done Sent Done false
18727 5871 Acked Flush 0 Done Sent Done false
18728 5872 Acked Read 1 Done Sent Done false
18729 5873 Acked Read 3 Done Sent Done false
18730 5874 Acked Write 1 Done Sent Done false
18731 5875 Acked Flush 0 Done Sent Done false
18732 5876 Acked Read 3 Done Sent Done false
18733 5877 Acked Read 4 Done Sent Done false
18734 5878 Acked Read 7 Done Sent Done false
18735 5879 Acked Read 1 Done Sent Done false
18736 5880 Acked Flush 0 Done Sent Done false
18737 5881 Acked Write 5 Done Sent Done false
18738 5882 Acked Flush 0 Done Sent Done false
18739 5883 Acked Flush 0 Done Sent Done false
18740 5884 Acked Write 6 Done Sent Done false
18741 5885 Acked Flush 0 Done Sent Done false
18742 5886 Acked Read 6 Done Sent Done false
18743 5887 Acked Read 10 Done Sent Done false
18744 5888 Acked Write 2 Done Sent Done false
18745 5889 Acked Write 1 Done Sent Done false
18746 5890 Acked Read 1 Done Sent Done false
18747 5891 Acked Write 9 Done Sent Done false
18748 5892 Acked Write 3 Done Sent Done false
18749 5893 Acked Read 10 Done Sent Done false
18750 5894 Acked Flush 0 Done Sent Done false
18751 5895 Acked Write 8 Done Sent Done false
18752 5896 Acked Flush 0 Done Sent Done false
18753 5897 Acked Read 6 Done Sent Done false
18754 5898 Acked Write 1 Done Sent Done false
18755 5899 Acked Write 7 Done Sent Done false
18756 5900 Acked Write 3 Done Sent Done false
18757 5901 Acked Read 4 Done Sent Done false
18758 5902 Acked Write 6 Done Sent Done false
18759 5903 Acked Flush 0 Done Sent Done false
18760 5904 Acked Read 4 Done Sent Done false
18761 5905 Acked Read 6 Done Sent Done false
18762 5906 Acked Read 7 Done Sent Done false
18763 5907 Acked Read 9 Done Sent Done false
18764 5908 Acked Read 10 Done Sent Done false
18765 5909 Acked Write 9 Done Sent Done false
18766 5910 Acked Write 6 Done Sent Done false
18767 5911 Acked Read 6 Done Sent Done false
18768 5912 Acked Write 3 Done Sent Done false
18769 5913 Acked Read 6 Done Sent Done false
18770 5914 Acked Flush 0 Done Sent Done false
18771 5915 Acked Write 1 Done Sent Done false
18772 5916 Acked Read 1 Done Sent Done false
18773 5917 Acked Write 10 Done Sent Done false
18774 5918 Acked Read 7 Done Sent Done false
18775 5919 Acked Write 6 Done Sent Done false
18776 5920 Acked Write 10 Done Sent Done false
18777 5921 Acked Flush 0 Done Sent Done false
18778 5922 Acked Write 1 Done Sent Done false
18779 5923 Acked Read 4 Done Sent Done false
18780 5924 Acked Flush 0 Done Sent Done false
18781 5925 Acked Read 6 Done Sent Done false
18782 5926 Acked Write 7 Done Sent Done false
18783 5927 Acked Read 9 Done Sent Done false
18784 5928 Acked Read 3 Done Sent Done false
18785 5929 Acked Write 5 Done Sent Done false
18786 5930 Acked Write 8 Done Sent Done false
18787 5931 Acked Read 8 Done Sent Done false
18788 5932 Acked Write 3 Done Sent Done false
18789 5933 Acked Read 8 Done Sent Done false
18790 5934 Acked Write 8 Done Sent Done false
18791 5935 Acked Flush 0 Done Sent Done false
18792 5936 Acked Read 9 Done Sent Done false
18793 5937 Acked Write 4 Done Sent Done false
18794 5938 Acked Read 3 Done Sent Done false
18795 5939 Acked Write 2 Done Sent Done false
18796 5940 Acked Read 3 Done Sent Done false
18797 5941 Acked Write 2 Done Sent Done false
18798 5942 Acked Read 1 Done Sent Done false
18799 5943 Acked Read 3 Done Sent Done false
18800 5944 Acked Read 3 Done Sent Done false
18801 5945 Acked Write 3 Done Sent Done false
18802 5946 Acked Write 4 Done Sent Done false
18803 5947 Acked Write 9 Done Sent Done false
18804 5948 Acked Write 1 Done Sent Done false
18805 5949 Acked Flush 0 Done Sent Done false
18806 5950 Acked Write 1 Done Sent Done false
18807 5951 Acked Write 4 Done Sent Done false
18808 5952 Acked Read 5 Done Sent Done false
18809 5953 Acked Write 5 Done Sent Done false
18810 5954 Acked Write 3 Done Sent Done false
18811 5955 Acked Write 6 Done Sent Done false
18812 5956 Acked Write 8 Done Sent Done false
18813 5957 Acked Write 1 Done Sent Done false
18814 5958 Acked Read 7 Done Sent Done false
18815 5959 Acked Write 2 Done Sent Done false
18816 5960 Acked Write 7 Done Sent Done false
18817 5961 Acked Read 3 Done Sent Done false
18818 5962 Acked Flush 0 Done Sent Done false
18819 5963 Acked Read 4 Done Sent Done false
18820 5964 Acked Read 6 Done Sent Done false
18821 5965 Acked Read 7 Done Sent Done false
18822 5966 Acked Write 3 Done Sent Done false
18823 5967 Acked Write 3 Done Sent Done false
18824 5968 Acked Read 5 Done Sent Done false
18825 5969 Acked Flush 0 Done Sent Done false
18826 5970 Acked Read 4 Done Sent Done false
18827 5971 Acked Read 8 Done Sent Done false
18828 5972 Acked Read 4 Done Sent Done false
18829 5973 Acked Flush 0 Done Sent Done false
18830 5974 Acked Read 7 Done Sent Done false
18831 5975 Acked Read 7 Done Sent Done false
18832 5976 Acked Read 8 Done Sent Done false
18833 5977 Acked Write 4 Done Sent Done false
18834 5978 Acked Read 3 Done Sent Done false
18835 5979 Acked Read 6 Done Sent Done false
18836 5980 Acked Read 6 Done Sent Done false
18837 5981 Acked Write 6 Done Sent Done false
18838 5982 Acked Write 2 Done Sent Done false
18839 5983 Acked Read 3 Done Sent Done false
18840 5984 Acked Write 4 Done Sent Done false
18841 5985 Acked Write 10 Done Sent Done false
18842 5986 Acked Flush 0 Done Sent Done false
18843 5987 Acked Read 7 Done Sent Done false
18844 5988 Acked Write 1 Done Sent Done false
18845 5989 Acked Write 5 Done Sent Done false
18846 5990 Acked Write 9 Done Sent Done false
18847 5991 Acked Write 10 Done Sent Done false
18848 5992 Acked Write 10 Done Sent Done false
18849 5993 Acked Write 5 Done Sent Done false
18850 5994 Acked Flush 0 Done Sent Done false
18851 5995 Acked Write 8 Done Sent Done false
18852 5996 Acked Read 9 Done Sent Done false
18853 5997 Acked Flush 0 Done Sent Done false
18854 5998 Acked Read 2 Done Sent Done false
18855 5999 Acked Read 5 Done Sent Done false
18856 6000 Acked Read 9 Done Sent Done false
18857 6001 Acked Write 9 Done Sent Done false
18858 6002 Acked Read 6 Done Sent Done false
18859 6003 Acked Read 7 Done Sent Done false
18860 6004 Acked Flush 0 Done Sent Done false
18861 6005 Acked Write 3 Done Sent Done false
18862 6006 Acked Write 6 Done Sent Done false
18863 6007 Acked Write 8 Done Sent Done false
18864 6008 Acked Read 3 Done Sent Done false
18865 6009 Acked Write 2 Done Sent Done false
18866 6010 Acked Write 6 Done Sent Done false
18867 6011 Acked Read 2 Done Sent Done false
18868 6012 Acked Read 7 Done Sent Done false
18869 6013 Acked Write 3 Done Sent Done false
18870 6014 Acked Read 3 Done Sent Done false
18871 6015 Acked Write 5 Done Sent Done false
18872 6016 Acked Write 6 Done Sent Done false
18873 6017 Acked Write 2 Done Sent Done false
18874 6018 Acked Write 1 Done Sent Done false
18875 6019 Acked Write 1 Done Sent Done false
18876 6020 Acked Read 6 Done Sent Done false
18877 6021 Acked Write 4 Done Sent Done false
18878 6022 Acked Read 5 Done Sent Done false
18879 6023 Acked Read 4 Done Sent Done false
18880 6024 Acked Read 8 Done Sent Done false
18881 6025 Acked Read 7 Done Sent Done false
18882 6026 Acked Flush 0 Done Sent Done false
18883 6027 Acked Flush 0 Done Sent Done false
18884 6028 Acked Read 1 Done Sent Done false
18885 6029 Acked Read 4 Done Sent Done false
18886 6030 Acked Read 2 Done Sent Done false
18887 6031 Acked Write 4 Done Sent Done false
18888 6032 Acked Read 9 Done Sent Done false
18889 6033 Acked Read 4 Done Sent Done false
18890 6034 Acked Write 5 Done Sent Done false
18891 6035 Acked Write 4 Done Sent Done false
18892 6036 Acked Read 6 Done Sent Done false
18893 6037 Acked Write 4 Done Sent Done false
18894 6038 Acked Read 9 Done Sent Done false
18895 6039 Acked Read 5 Done Sent Done false
18896 6040 Acked Read 5 Done Sent Done false
18897 6041 Acked Write 8 Done Sent Done false
18898 6042 Acked Read 1 Done Sent Done false
18899 6043 Acked Write 2 Done Sent Done false
18900 6044 Acked Read 4 Done Sent Done false
18901 6045 Acked Read 3 Done Sent Done false
18902 6046 Acked Write 8 Done Sent Done false
18903 6047 Acked Read 6 Done Sent Done false
18904 6048 Acked Read 4 Done Sent Done false
18905 6049 Acked Read 8 Done Sent Done false
18906 6050 Acked Read 10 Done Sent Done false
18907 6051 Acked Read 2 Done Sent Done false
18908 6052 Acked Read 1 Done Sent Done false
18909 6053 Acked Write 7 Done Sent Done false
18910 6054 Acked Read 10 Done Sent Done false
18911 6055 Acked Write 3 Done Sent Done false
18912 6056 Acked Read 5 Done Sent Done false
18913 6057 Acked Flush 0 Done Sent Done false
18914 6058 Acked Write 2 Done Sent Done false
18915 6059 Acked Read 2 Done Sent Done false
18916 6060 Acked Read 3 Done Sent Done false
18917 6061 Acked Read 3 Done Sent Done false
18918 6062 Acked Write 9 Done Sent Done false
18919 6063 Acked Flush 0 Done Sent Done false
18920 6064 Acked Write 9 Done Sent Done false
18921 6065 Acked Write 2 Done Sent Done false
18922 6066 Acked Write 5 Done Sent Done false
18923 6067 Acked Write 8 Done Sent Done false
18924 6068 Acked Write 10 Done Sent Done false
18925 6069 Acked Flush 0 Done Sent Done false
18926 6070 Acked Flush 0 Done Sent Done false
18927 6071 Acked Write 10 Done Sent Done false
18928 6072 Acked Read 4 Done Sent Done false
18929 6073 Acked Write 2 Done Sent Done false
18930 6074 Acked Read 1 Done Sent Done false
18931 6075 Acked Read 1 Done Sent Done false
18932 6076 Acked Write 8 Done Sent Done false
18933 6077 Acked Read 2 Done Sent Done false
18934 6078 Acked Write 5 Done Sent Done false
18935 6079 Acked Write 1 Done Sent Done false
18936 6080 Acked Read 2 Done Sent Done false
18937 6081 Acked Write 7 Done Sent Done false
18938 6082 Acked Write 5 Done Sent Done false
18939 6083 Acked Read 9 Done Sent Done false
18940 6084 Acked Write 6 Done Sent Done false
18941 6085 Acked Write 6 Done Sent Done false
18942 6086 Acked Write 10 Done Sent Done false
18943 6087 Acked Write 7 Done Sent Done false
18944 6088 Acked Read 1 Done Sent Done false
18945 6089 Acked Read 10 Done Sent Done false
18946 6090 Acked Write 5 Done Sent Done false
18947 6091 Acked Read 9 Done Sent Done false
18948 6092 Acked Read 4 Done Sent Done false
18949 6093 Acked Write 4 Done Sent Done false
18950 6094 Acked Read 4 Done Sent Done false
18951 6095 Acked Write 10 Done Sent Done false
18952 6096 Acked Read 8 Done Sent Done false
18953 6097 Acked Read 10 Done Sent Done false
18954 6098 Acked Read 1 Done Sent Done false
18955 6099 Acked Read 7 Done Sent Done false
18956 6100 Acked Read 10 Done Sent Done false
18957 6101 Acked Write 9 Done Sent Done false
18958 6102 Acked Write 3 Done Sent Done false
18959 6103 Acked Write 8 Done Sent Done false
18960 6104 Acked Read 9 Done Sent Done false
18961 6105 Acked Write 10 Done Sent Done false
18962 6106 Acked Write 9 Done Sent Done false
18963 6107 Acked Read 4 Done Sent Done false
18964 6108 Acked Write 9 Done Sent Done false
18965 6109 Acked Write 2 Done Sent Done false
18966 6110 Acked Write 5 Done Sent Done false
18967 6111 Acked Read 8 Done Sent Done false
18968 6112 Acked Write 5 Done Sent Done false
18969 6113 Acked Write 7 Done Sent Done false
18970 6114 Acked Read 5 Done Sent Done false
18971 6115 Acked Write 10 Done Sent Done false
18972 6116 Acked Read 8 Done Sent Done false
18973 6117 Acked Flush 0 Done Sent Done false
18974 6118 Acked Write 8 Done Sent Done false
18975 6119 Acked Write 5 Done Sent Done false
18976 6120 Acked Read 4 Done Sent Done false
18977 6121 Acked Write 4 Done Sent Done false
18978 6122 Acked Read 2 Done Sent Done false
18979 6123 Acked Read 5 Done Sent Done false
18980 6124 Acked Write 1 Done Sent Done false
18981 6125 Acked Read 4 Done Sent Done false
18982 6126 Acked Write 9 Done Sent Done false
18983 6127 Acked Write 6 Done Sent Done false
18984 6128 Acked Read 5 Done Sent Done false
18985 6129 Acked Flush 0 Done Sent Done false
18986 6130 Acked Read 8 Done Sent Done false
18987 6131 Acked Flush 0 Done Sent Done false
18988 STATES DS:0 DS:1 DS:2 TOTAL
18989 Sent 0 302 0 302
18990 Done 302 0 302 604
18991 Skipped 0 0 0 0
18992 Error 0 0 0 0
18993 Last Flush: 6131 5829 6131
18994 Downstairs last five completed: 5829 5828 5827 5826 5825
18995 Upstairs last five completed: 6131 6130 6129 6128 6127
18996 CLIENT: Up:0 ds:302 act:2
18997 ----------------------------------------------------------------
18998 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
18999 States: Active Offline Active
19000 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
19001 5830 Acked Read 8 Done Sent Done false
19002 5831 Acked Read 9 Done Sent Done false
19003 5832 Acked Read 4 Done Sent Done false
19004 5833 Acked Read 9 Done Sent Done false
19005 5834 Acked Write 8 Done Sent Done false
19006 5835 Acked Read 8 Done Sent Done false
19007 5836 Acked Write 4 Done Sent Done false
19008 5837 Acked Write 3 Done Sent Done false
19009 5838 Acked Flush 0 Done Sent Done false
19010 5839 Acked Write 5 Done Sent Done false
19011 5840 Acked Read 1 Done Sent Done false
19012 5841 Acked Read 3 Done Sent Done false
19013 5842 Acked Write 1 Done Sent Done false
19014 5843 Acked Write 6 Done Sent Done false
19015 5844 Acked Read 1 Done Sent Done false
19016 5845 Acked Read 1 Done Sent Done false
19017 5846 Acked Read 8 Done Sent Done false
19018 5847 Acked Read 2 Done Sent Done false
19019 5848 Acked Read 5 Done Sent Done false
19020 5849 Acked Read 9 Done Sent Done false
19021 5850 Acked Read 4 Done Sent Done false
19022 5851 Acked Flush 0 Done Sent Done false
19023 5852 Acked Write 7 Done Sent Done false
19024 5853 Acked Write 8 Done Sent Done false
19025 5854 Acked Read 4 Done Sent Done false
19026 5855 Acked Write 10 Done Sent Done false
19027 5856 Acked Read 1 Done Sent Done false
19028 5857 Acked Write 2 Done Sent Done false
19029 5858 Acked Write 10 Done Sent Done false
19030 5859 Acked Read 5 Done Sent Done false
19031 5860 Acked Read 6 Done Sent Done false
19032 5861 Acked Write 6 Done Sent Done false
19033 5862 Acked Write 7 Done Sent Done false
19034 5863 Acked Write 9 Done Sent Done false
19035 5864 Acked Write 3 Done Sent Done false
19036 5865 Acked Write 1 Done Sent Done false
19037 5866 Acked Read 7 Done Sent Done false
19038 5867 Acked Read 10 Done Sent Done false
19039 5868 Acked Write 10 Done Sent Done false
19040 5869 Acked Read 7 Done Sent Done false
19041 5870 Acked Write 8 Done Sent Done false
19042 5871 Acked Flush 0 Done Sent Done false
19043 5872 Acked Read 1 Done Sent Done false
19044 5873 Acked Read 3 Done Sent Done false
19045 5874 Acked Write 1 Done Sent Done false
19046 5875 Acked Flush 0 Done Sent Done false
19047 5876 Acked Read 3 Done Sent Done false
19048 5877 Acked Read 4 Done Sent Done false
19049 5878 Acked Read 7 Done Sent Done false
19050 5879 Acked Read 1 Done Sent Done false
19051 5880 Acked Flush 0 Done Sent Done false
19052 5881 Acked Write 5 Done Sent Done false
19053 5882 Acked Flush 0 Done Sent Done false
19054 5883 Acked Flush 0 Done Sent Done false
19055 5884 Acked Write 6 Done Sent Done false
19056 5885 Acked Flush 0 Done Sent Done false
19057 5886 Acked Read 6 Done Sent Done false
19058 5887 Acked Read 10 Done Sent Done false
19059 5888 Acked Write 2 Done Sent Done false
19060 5889 Acked Write 1 Done Sent Done false
19061 5890 Acked Read 1 Done Sent Done false
19062 5891 Acked Write 9 Done Sent Done false
19063 5892 Acked Write 3 Done Sent Done false
19064 5893 Acked Read 10 Done Sent Done false
19065 5894 Acked Flush 0 Done Sent Done false
19066 5895 Acked Write 8 Done Sent Done false
19067 5896 Acked Flush 0 Done Sent Done false
19068 5897 Acked Read 6 Done Sent Done false
19069 5898 Acked Write 1 Done Sent Done false
19070 5899 Acked Write 7 Done Sent Done false
19071 5900 Acked Write 3 Done Sent Done false
19072 5901 Acked Read 4 Done Sent Done false
19073 5902 Acked Write 6 Done Sent Done false
19074 5903 Acked Flush 0 Done Sent Done false
19075 5904 Acked Read 4 Done Sent Done false
19076 5905 Acked Read 6 Done Sent Done false
19077 5906 Acked Read 7 Done Sent Done false
19078 5907 Acked Read 9 Done Sent Done false
19079 5908 Acked Read 10 Done Sent Done false
19080 5909 Acked Write 9 Done Sent Done false
19081 5910 Acked Write 6 Done Sent Done false
19082 5911 Acked Read 6 Done Sent Done false
19083 5912 Acked Write 3 Done Sent Done false
19084 5913 Acked Read 6 Done Sent Done false
19085 5914 Acked Flush 0 Done Sent Done false
19086 5915 Acked Write 1 Done Sent Done false
19087 5916 Acked Read 1 Done Sent Done false
19088 5917 Acked Write 10 Done Sent Done false
19089 5918 Acked Read 7 Done Sent Done false
19090 5919 Acked Write 6 Done Sent Done false
19091 5920 Acked Write 10 Done Sent Done false
19092 5921 Acked Flush 0 Done Sent Done false
19093 5922 Acked Write 1 Done Sent Done false
19094 5923 Acked Read 4 Done Sent Done false
19095 5924 Acked Flush 0 Done Sent Done false
19096 5925 Acked Read 6 Done Sent Done false
19097 5926 Acked Write 7 Done Sent Done false
19098 5927 Acked Read 9 Done Sent Done false
19099 5928 Acked Read 3 Done Sent Done false
19100 5929 Acked Write 5 Done Sent Done false
19101 5930 Acked Write 8 Done Sent Done false
19102 5931 Acked Read 8 Done Sent Done false
19103 5932 Acked Write 3 Done Sent Done false
19104 5933 Acked Read 8 Done Sent Done false
19105 5934 Acked Write 8 Done Sent Done false
19106 5935 Acked Flush 0 Done Sent Done false
19107 5936 Acked Read 9 Done Sent Done false
19108 5937 Acked Write 4 Done Sent Done false
19109 5938 Acked Read 3 Done Sent Done false
19110 5939 Acked Write 2 Done Sent Done false
19111 5940 Acked Read 3 Done Sent Done false
19112 5941 Acked Write 2 Done Sent Done false
19113 5942 Acked Read 1 Done Sent Done false
19114 5943 Acked Read 3 Done Sent Done false
19115 5944 Acked Read 3 Done Sent Done false
19116 5945 Acked Write 3 Done Sent Done false
19117 5946 Acked Write 4 Done Sent Done false
19118 5947 Acked Write 9 Done Sent Done false
19119 5948 Acked Write 1 Done Sent Done false
19120 5949 Acked Flush 0 Done Sent Done false
19121 5950 Acked Write 1 Done Sent Done false
19122 5951 Acked Write 4 Done Sent Done false
19123 5952 Acked Read 5 Done Sent Done false
19124 5953 Acked Write 5 Done Sent Done false
19125 5954 Acked Write 3 Done Sent Done false
19126 5955 Acked Write 6 Done Sent Done false
19127 5956 Acked Write 8 Done Sent Done false
19128 5957 Acked Write 1 Done Sent Done false
19129 5958 Acked Read 7 Done Sent Done false
19130 5959 Acked Write 2 Done Sent Done false
19131 5960 Acked Write 7 Done Sent Done false
19132 5961 Acked Read 3 Done Sent Done false
19133 5962 Acked Flush 0 Done Sent Done false
19134 5963 Acked Read 4 Done Sent Done false
19135 5964 Acked Read 6 Done Sent Done false
19136 5965 Acked Read 7 Done Sent Done false
19137 5966 Acked Write 3 Done Sent Done false
19138 5967 Acked Write 3 Done Sent Done false
19139 5968 Acked Read 5 Done Sent Done false
19140 5969 Acked Flush 0 Done Sent Done false
19141 5970 Acked Read 4 Done Sent Done false
19142 5971 Acked Read 8 Done Sent Done false
19143 5972 Acked Read 4 Done Sent Done false
19144 5973 Acked Flush 0 Done Sent Done false
19145 5974 Acked Read 7 Done Sent Done false
19146 5975 Acked Read 7 Done Sent Done false
19147 5976 Acked Read 8 Done Sent Done false
19148 5977 Acked Write 4 Done Sent Done false
19149 5978 Acked Read 3 Done Sent Done false
19150 5979 Acked Read 6 Done Sent Done false
19151 5980 Acked Read 6 Done Sent Done false
19152 5981 Acked Write 6 Done Sent Done false
19153 5982 Acked Write 2 Done Sent Done false
19154 5983 Acked Read 3 Done Sent Done false
19155 5984 Acked Write 4 Done Sent Done false
19156 5985 Acked Write 10 Done Sent Done false
19157 5986 Acked Flush 0 Done Sent Done false
19158 5987 Acked Read 7 Done Sent Done false
19159 5988 Acked Write 1 Done Sent Done false
19160 5989 Acked Write 5 Done Sent Done false
19161 5990 Acked Write 9 Done Sent Done false
19162 5991 Acked Write 10 Done Sent Done false
19163 5992 Acked Write 10 Done Sent Done false
19164 5993 Acked Write 5 Done Sent Done false
19165 5994 Acked Flush 0 Done Sent Done false
19166 5995 Acked Write 8 Done Sent Done false
19167 5996 Acked Read 9 Done Sent Done false
19168 5997 Acked Flush 0 Done Sent Done false
19169 5998 Acked Read 2 Done Sent Done false
19170 5999 Acked Read 5 Done Sent Done false
19171 6000 Acked Read 9 Done Sent Done false
19172 6001 Acked Write 9 Done Sent Done false
19173 6002 Acked Read 6 Done Sent Done false
19174 6003 Acked Read 7 Done Sent Done false
19175 6004 Acked Flush 0 Done Sent Done false
19176 6005 Acked Write 3 Done Sent Done false
19177 6006 Acked Write 6 Done Sent Done false
19178 6007 Acked Write 8 Done Sent Done false
19179 6008 Acked Read 3 Done Sent Done false
19180 6009 Acked Write 2 Done Sent Done false
19181 6010 Acked Write 6 Done Sent Done false
19182 6011 Acked Read 2 Done Sent Done false
19183 6012 Acked Read 7 Done Sent Done false
19184 6013 Acked Write 3 Done Sent Done false
19185 6014 Acked Read 3 Done Sent Done false
19186 6015 Acked Write 5 Done Sent Done false
19187 6016 Acked Write 6 Done Sent Done false
19188 6017 Acked Write 2 Done Sent Done false
19189 6018 Acked Write 1 Done Sent Done false
19190 6019 Acked Write 1 Done Sent Done false
19191 6020 Acked Read 6 Done Sent Done false
19192 6021 Acked Write 4 Done Sent Done false
19193 6022 Acked Read 5 Done Sent Done false
19194 6023 Acked Read 4 Done Sent Done false
19195 6024 Acked Read 8 Done Sent Done false
19196 6025 Acked Read 7 Done Sent Done false
19197 6026 Acked Flush 0 Done Sent Done false
19198 6027 Acked Flush 0 Done Sent Done false
19199 6028 Acked Read 1 Done Sent Done false
19200 6029 Acked Read 4 Done Sent Done false
19201 6030 Acked Read 2 Done Sent Done false
19202 6031 Acked Write 4 Done Sent Done false
19203 6032 Acked Read 9 Done Sent Done false
19204 6033 Acked Read 4 Done Sent Done false
19205 6034 Acked Write 5 Done Sent Done false
19206 6035 Acked Write 4 Done Sent Done false
19207 6036 Acked Read 6 Done Sent Done false
19208 6037 Acked Write 4 Done Sent Done false
19209 6038 Acked Read 9 Done Sent Done false
19210 6039 Acked Read 5 Done Sent Done false
19211 6040 Acked Read 5 Done Sent Done false
19212 6041 Acked Write 8 Done Sent Done false
19213 6042 Acked Read 1 Done Sent Done false
19214 6043 Acked Write 2 Done Sent Done false
19215 6044 Acked Read 4 Done Sent Done false
19216 6045 Acked Read 3 Done Sent Done false
19217 6046 Acked Write 8 Done Sent Done false
19218 6047 Acked Read 6 Done Sent Done false
19219 6048 Acked Read 4 Done Sent Done false
19220 6049 Acked Read 8 Done Sent Done false
19221 6050 Acked Read 10 Done Sent Done false
19222 6051 Acked Read 2 Done Sent Done false
19223 6052 Acked Read 1 Done Sent Done false
19224 6053 Acked Write 7 Done Sent Done false
19225 6054 Acked Read 10 Done Sent Done false
19226 6055 Acked Write 3 Done Sent Done false
19227 6056 Acked Read 5 Done Sent Done false
19228 6057 Acked Flush 0 Done Sent Done false
19229 6058 Acked Write 2 Done Sent Done false
19230 6059 Acked Read 2 Done Sent Done false
19231 6060 Acked Read 3 Done Sent Done false
19232 6061 Acked Read 3 Done Sent Done false
19233 6062 Acked Write 9 Done Sent Done false
19234 6063 Acked Flush 0 Done Sent Done false
19235 6064 Acked Write 9 Done Sent Done false
19236 6065 Acked Write 2 Done Sent Done false
19237 6066 Acked Write 5 Done Sent Done false
19238 6067 Acked Write 8 Done Sent Done false
19239 6068 Acked Write 10 Done Sent Done false
19240 6069 Acked Flush 0 Done Sent Done false
19241 6070 Acked Flush 0 Done Sent Done false
19242 6071 Acked Write 10 Done Sent Done false
19243 6072 Acked Read 4 Done Sent Done false
19244 6073 Acked Write 2 Done Sent Done false
19245 6074 Acked Read 1 Done Sent Done false
19246 6075 Acked Read 1 Done Sent Done false
19247 6076 Acked Write 8 Done Sent Done false
19248 6077 Acked Read 2 Done Sent Done false
19249 6078 Acked Write 5 Done Sent Done false
19250 6079 Acked Write 1 Done Sent Done false
19251 6080 Acked Read 2 Done Sent Done false
19252 6081 Acked Write 7 Done Sent Done false
19253 6082 Acked Write 5 Done Sent Done false
19254 6083 Acked Read 9 Done Sent Done false
19255 6084 Acked Write 6 Done Sent Done false
19256 6085 Acked Write 6 Done Sent Done false
19257 6086 Acked Write 10 Done Sent Done false
19258 6087 Acked Write 7 Done Sent Done false
19259 6088 Acked Read 1 Done Sent Done false
19260 6089 Acked Read 10 Done Sent Done false
19261 6090 Acked Write 5 Done Sent Done false
19262 6091 Acked Read 9 Done Sent Done false
19263 6092 Acked Read 4 Done Sent Done false
19264 6093 Acked Write 4 Done Sent Done false
19265 6094 Acked Read 4 Done Sent Done false
19266 6095 Acked Write 10 Done Sent Done false
19267 6096 Acked Read 8 Done Sent Done false
19268 6097 Acked Read 10 Done Sent Done false
19269 6098 Acked Read 1 Done Sent Done false
19270 6099 Acked Read 7 Done Sent Done false
19271 6100 Acked Read 10 Done Sent Done false
19272 6101 Acked Write 9 Done Sent Done false
19273 6102 Acked Write 3 Done Sent Done false
19274 6103 Acked Write 8 Done Sent Done false
19275 6104 Acked Read 9 Done Sent Done false
19276 6105 Acked Write 10 Done Sent Done false
19277 6106 Acked Write 9 Done Sent Done false
19278 6107 Acked Read 4 Done Sent Done false
19279 6108 Acked Write 9 Done Sent Done false
19280 6109 Acked Write 2 Done Sent Done false
19281 6110 Acked Write 5 Done Sent Done false
19282 6111 Acked Read 8 Done Sent Done false
19283 6112 Acked Write 5 Done Sent Done false
19284 6113 Acked Write 7 Done Sent Done false
19285 6114 Acked Read 5 Done Sent Done false
19286 6115 Acked Write 10 Done Sent Done false
19287 6116 Acked Read 8 Done Sent Done false
19288 6117 Acked Flush 0 Done Sent Done false
19289 6118 Acked Write 8 Done Sent Done false
19290 6119 Acked Write 5 Done Sent Done false
19291 6120 Acked Read 4 Done Sent Done false
19292 6121 Acked Write 4 Done Sent Done false
19293 6122 Acked Read 2 Done Sent Done false
19294 6123 Acked Read 5 Done Sent Done false
19295 6124 Acked Write 1 Done Sent Done false
19296 6125 Acked Read 4 Done Sent Done false
19297 6126 Acked Write 9 Done Sent Done false
19298 6127 Acked Write 6 Done Sent Done false
19299 6128 Acked Read 5 Done Sent Done false
19300 6129 Acked Flush 0 Done Sent Done false
19301 6130 Acked Read 8 Done Sent Done false
19302 6131 Acked Flush 0 Done Sent Done false
19303 STATES DS:0 DS:1 DS:2 TOTAL
19304 Sent 0 302 0 302
19305 Done 302 0 302 604
19306 Skipped 0 0 0 0
19307 Error 0 0 0 0
19308 Last Flush: 6131 5829 6131
19309 Downstairs last five completed: 5829 5828 5827 5826 5825
19310 Upstairs last five completed: 6131 6130 6129 6128 6127
19311 CLIENT: Up:0 ds:302 act:2
193122024-11-28T04:54:22.926ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
193132024-11-28T04:54:22.926ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:39219) = io task client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
193142024-11-28T04:54:22.927ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
193152024-11-28T04:54:22.928ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
193162024-11-28T04:54:22.928ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
193172024-11-28T04:54:22.928ZINFOcrucible: send last flush ID to this DS: 5829 = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
193182024-11-28T04:54:22.928ZINFOcrucible: Replied this last flush ID: 5829 = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
193192024-11-28T04:54:22.928ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
193202024-11-28T04:54:22.928ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
193212024-11-28T04:54:22.928ZINFOcrucible: [1] client re-new 302 jobs since flush 5829 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
193222024-11-28T04:54:22.928ZINFOcrucible: [1] Replayed 302 jobs since flush: 5829 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
19323 ----------------------------------------------------------------
19324 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
19325 Downstairs last five completed: 6131 6130 6129 6128 6127
19326 Upstairs last five completed: 6131 6130 6129 6128 6127
19327 CLIENT: Up:0 ds:0 act:3
19328 Replay: All jobs finished
19329 Main task has work to do, go find it
19330 got dsc Stop(1)
19331 stop 1
19332 [8820] Got stop action so:false kr:false
19333 Nov 28 04:54:23.160 INFO request completed, latency_us: 332, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 81833072-ba29-4d1a-a615-9e3bd515c256, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
193342024-11-28T04:54:23.161ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
193352024-11-28T04:54:23.161ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
193362024-11-28T04:54:23.161ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
193372024-11-28T04:54:23.161ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
19338 Nov 28 04:54:23.161 INFO request completed{"msg":"[1] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-28T04:54:23.16135532Z","hostname":"w-01JDRHKEEFJ1228TFPSBEGXWHY","pid":1202,"client":"1","":"downstairs","session_id":"4a2415b1-6594-436f-82e5-b8cf9a74eeaf"}
19339 , latency_us: 130, response_code: 200, uri: /state/cid/1, method: GET, req_id: 942ab6b6-6468-454f-b167-af46a3531eca, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
193402024-11-28T04:54:23.161ZINFOcrucible: client task is exiting = io task client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
19341 [8820] Exited with: ExitStatus(unix_wait_status(9))
19342 [8820][1] reports Exit
19343 Nov 28 04:54:27.163 INFO request completed, latency_us: 216, response_code: 200, uri: /state/cid/1, method: GET, req_id: 44b1e4b1-5b86-4e3f-9678-2dcd434e4db4, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
19344 001/300 Flush
19345 002/300 Write block 118 len 20480 data: 6 6 7 6 6
19346 003/300 Read block 884 len 24576
19347 004/300 Read block 3419 len 40960
19348 005/300 Read block 1771 len 12288
19349 006/300 Read block 762 len 28672
19350 007/300 Flush
19351 008/300 Flush
19352 009/300 Read block 72 len 4096
19353 010/300 Read block 2151 len 8192
19354 011/300 Read block 400 len 16384
19355 012/300 Read block 1699 len 20480
19356 013/300 Write block 4388 len 12288 data: 4 4 5
19357 014/300 Flush
19358 015/300 Read block 3864 len 32768
19359 016/300 Write block 3672 len 4096 data: 5
19360 017/300 Read block 4613 len 4096
19361 018/300 Read block 1380 len 36864
19362 019/300 Write block 148 len 40960 data: 3 4 4 7 6 6 6 5 6 6
19363 020/300 Write block 4475 len 12288 data: 6 6 5
19364 021/300 Read block 4134 len 40960
19365 022/300 Read block 149 len 28672
19366 023/300 Read block 1583 len 4096
19367 024/300 Read block 287 len 32768
19368 025/300 Flush
19369 026/300 Read block 266 len 8192
19370 027/300 Write block 380 len 28672 data: 8 7 8 8 6 3 3
19371 028/300 Read block 2692 len 12288
19372 029/300 Read block 140 len 28672
19373 030/300 Read block 3073 len 16384
19374 031/300 Flush
19375 032/300 Flush
19376 033/300 Read block 1332 len 40960
19377 034/300 Read block 1750 len 20480
19378 035/300 Read block 1044 len 24576
19379 036/300 Write block 2290 len 16384 data: 4 4 4 4
19380 037/300 Write block 4882 len 20480 data: 4 5 4 4 4
19381 038/300 Read block 4072 len 16384
19382 039/300 Read block 3617 len 20480
19383 040/300 Read block 2418 len 40960
19384 041/300 Write block 363 len 20480 data: 3 4 5 7 6
19385 042/300 Write block 3511 len 16384 data: 5 4 4 4
19386 043/300 Write block 2953 len 32768 data: 4 3 3 3 4 5 6 7
19387 044/300 Read block 4882 len 28672
19388 045/300 Write block 4622 len 24576 data: 3 2 3 5 6 4
19389 046/300 Write block 3156 len 28672 data: 5 5 4 5 4 3 3
19390 047/300 Read block 824 len 28672
19391 048/300 Write block 2417 len 40960 data: 2 2 6 6 6 6 5 4 4 3
19392 049/300 Read block 455 len 4096
19393 050/300 Flush
19394 051/300 Flush
19395 052/300 Read block 2621 len 28672
19396 053/300 Write block 1772 len 40960 data: 4 3 3 5 6 5 4 4 4 5
19397 054/300 Read block 3821 len 20480
19398 055/300 Write block 4772 len 20480 data: 4 5 6 6 5
19399 056/300 Read block 517 len 24576
19400 057/300 Read block 3160 len 4096
19401 058/300 Write block 2760 len 24576 data: 3 3 3 3 3 3
19402 059/300 Read block 546 len 4096
19403 060/300 Read block 2519 len 4096
19404 061/300 Write block 4913 len 24576 data: 2 3 3 3 3 4
19405 062/300 Write block 51 len 4096 data: 5
19406 063/300 Read block 3445 len 12288
19407 064/300 Write block 1213 len 40960 data: 4 4 4 4 3 5 6 7 8 8
19408 065/300 Flush
19409 066/300 Write block 952 len 28672 data: 4 4 4 4 4 3 4
19410 067/300 Flush
19411 068/300 Write block 3801 len 4096 data: 5
19412 069/300 Read block 1356 len 40960
19413 070/300 Read block 4448 len 4096
19414 071/300 Write block 78 len 12288 data: 3 2 2
19415 072/300 Write block 718 len 20480 data: 3 3 3 2 2
19416 073/300 Read block 927 len 36864
19417 074/300 Read block 107 len 36864
19418 075/300 Read block 298 len 16384
19419 076/300 Read block 3610 len 16384
19420 077/300 Read block 4364 len 32768
19421 078/300 Read block 798 len 24576
19422 079/300 Read block 1074 len 36864
19423 080/300 Read block 3885 len 16384
19424 081/300 Read block 4804 len 28672
19425 082/300 Write block 3956 len 32768 data: 5 5 5 4 5 6 5 4
19426 083/300 Write block 4259 len 20480 data: 3 3 2 2 2
19427 084/300 Write block 1275 len 12288 data: 6 5 5
19428 085/300 Write block 4841 len 28672 data: 3 3 4 5 6 6 5
19429 086/300 Flush
19430 087/300 Write block 4229 len 4096 data: 4
19431 088/300 Write block 2347 len 16384 data: 4 5 5 5
19432 089/300 Read block 2328 len 12288
19433 090/300 Read block 994 len 32768
19434 091/300 Write block 4853 len 28672 data: 8 7 6 5 6 4 4
19435 092/300 Read block 3826 len 12288
19436 093/300 Flush
19437 094/300 Flush
19438 095/300 Write block 1710 len 4096 data: 13
19439 096/300 Write block 2292 len 36864 data: 5 5 4 4 4 6 5 4 4
19440 097/300 Write block 4066 len 12288 data: 4 4 3
19441 098/300 Write block 4090 len 36864 data: 8 9 9 8 7 7 7 7 6
19442 099/300 Read block 4875 len 4096
19443 100/300 Write block 3460 len 24576 data: 3 5 5 5 5 8
19444 101/300 Write block 2377 len 32768 data: 6 7 7 7 7 4 4 4
19445 102/300 Write block 2930 len 40960 data: 4 3 3 3 2 3 2 2 2 2
19446 103/300 Write block 2018 len 16384 data: 3 3 3 5
19447 104/300 Read block 416 len 28672
19448 105/300 Read block 671 len 36864
19449 106/300 Read block 4233 len 12288
19450 107/300 Read block 2464 len 40960
19451 108/300 Flush
19452 109/300 Read block 3682 len 4096
19453 110/300 Write block 705 len 12288 data: 6 5 4
19454 111/300 Read block 1591 len 36864
19455 112/300 Write block 534 len 4096 data: 4
19456 113/300 Read block 709 len 4096
19457 114/300 Write block 602 len 16384 data: 7 5 4 3
19458 115/300 Read block 3302 len 8192
19459 116/300 Flush
19460 117/300 Write block 4466 len 12288 data: 5 6 5
19461 118/300 Read block 2077 len 20480
19462 119/300 Read block 3764 len 32768
19463 120/300 Read block 1908 len 32768
19464 121/300 Read block 2286 len 28672
19465 122/300 Write block 2931 len 16384 data: 4 4 4 3
19466 123/300 Read block 2201 len 8192
19467 124/300 Write block 870 len 28672 data: 5 5 5 6 5 4 5
19468 125/300 Read block 804 len 40960
19469 126/300 Write block 1452 len 8192 data: 3 4
19470 127/300 Read block 4678 len 8192
19471 128/300 Read block 3685 len 24576
19472 129/300 Read block 358 len 28672
19473 130/300 Write block 2124 len 24576 data: 5 5 6 6 5 5
19474 131/300 Read block 3855 len 32768
19475 132/300 Flush
19476 133/300 Write block 872 len 4096 data: 6
19477 134/300 Write block 4419 len 20480 data: 4 4 3 3 4
19478 135/300 Flush
19479 136/300 Write block 4454 len 24576 data: 8 8 7 7 7 6
19480 137/300 Read block 1293 len 24576
19481 138/300 Write block 1149 len 16384 data: 5 7 6 6
19482 139/300 Write block 803 len 28672 data: 5 4 4 5 5 5 4
19483 140/300 Read block 3776 len 36864
19484 141/300 Read block 2963 len 20480
19485 142/300 Read block 3229 len 32768
19486 143/300 Write block 1045 len 20480 data: 4 7 8 7 8
19487 144/300 Flush
19488 145/300 Write block 2327 len 36864 data: 4 4 3 5 6 6 7 8 6
19489 146/300 Read block 3492 len 8192
19490 147/300 Write block 1335 len 24576 data: 4 6 6 6 6 5
19491 148/300 Read block 4872 len 40960
19492 149/300 Write block 3601 len 12288 data: 8 4 4
19493 150/300 Write block 4951 len 20480 data: 5 5 4 6 6
19494 151/300 Read block 4559 len 24576
19495 152/300 Read block 2656 len 32768
19496 153/300 Read block 4134 len 40960
19497 154/300 Read block 910 len 40960
19498 155/300 Read block 1364 len 12288
19499 156/300 Write block 1436 len 36864 data: 4 3 2 2 2 2 2 2 2
19500 157/300 Read block 3369 len 12288
19501 158/300 Write block 992 len 40960 data: 5 4 4 4 4 4 4 4 4 4
19502 159/300 Read block 4205 len 4096
19503 160/300 Read block 4606 len 28672
19504 161/300 Read block 4400 len 40960
19505 162/300 Write block 2146 len 8192 data: 5 4
19506 163/300 Read block 1956 len 40960
19507 164/300 Read block 1844 len 16384
19508 165/300 Read block 4042 len 4096
19509 166/300 Read block 4126 len 32768
19510 167/300 Read block 1421 len 16384
19511 168/300 Write block 1523 len 40960 data: 4 4 3 3 3 3 5 3 4 5
19512 169/300 Read block 646 len 24576
19513 170/300 Read block 1151 len 32768
19514 171/300 Read block 3493 len 40960
19515 172/300 Write block 3672 len 4096 data: 6
19516 173/300 Write block 3293 len 24576 data: 4 4 3 3 4 4
19517 174/300 Write block 1649 len 36864 data: 3 2 2 2 2 3 3 4 2
19518 175/300 Read block 2875 len 20480
19519 176/300 Read block 1567 len 28672
19520 177/300 Flush
19521 178/300 Read block 3620 len 12288
19522 179/300 Write block 3377 len 8192 data: 4 4
19523 180/300 Read block 2589 len 4096
19524 181/300 Read block 1586 len 28672
19525 182/300 Read block 4026 len 32768
19526 183/300 Read block 578 len 12288
19527 184/300 Write block 2288 len 32768 data: 3 4 5 5 6 6 5 5
19528 185/300 Write block 2403 len 8192 data: 3 3
19529 186/300 Write block 3864 len 16384 data: 4 4 5 4
19530 187/300 Write block 3861 len 40960 data: 5 5 4 5 5 6 5 4 4 4
19531 188/300 Write block 1092 len 40960 data: 5 4 4 4 5 5 6 4 5 5
19532 189/300 Write block 1429 len 12288 data: 6 5 4
19533 190/300 Write block 2070 len 40960 data: 5 4 4 3 2 3 4 4 5 5
19534 191/300 Write block 1563 len 16384 data: 2 2 2 2
19535 192/300 Read block 2763 len 20480
19536 193/300 Write block 501 len 12288 data: 4 4 4
19537 194/300 Write block 1474 len 32768 data: 2 2 4 4 4 5 5 5
19538 195/300 Write block 3368 len 40960 data: 3 3 4 4 3 3 3 3 3 5
19539 196/300 Write block 474 len 12288 data: 6 6 4
19540 197/300 Read block 2689 len 40960
19541 198/300 Flush
19542 199/300 Write block 2248 len 12288 data: 6 6 5
19543 200/300 Read block 1783 len 24576
19544 201/300 Read block 4500 len 24576
19545 202/300 Read block 138 len 36864
19546 203/300 Read block 1950 len 32768
19547 204/300 Read block 4698 len 36864
19548 205/300 Write block 3647 len 32768 data: 5 5 7 8 7 5 5 4
19549 206/300 Write block 4521 len 28672 data: 5 6 6 7 7 6 6
19550 207/300 Read block 1486 len 40960
19551 208/300 Write block 1575 len 8192 data: 4 5
19552 209/300 Flush
19553 210/300 Flush
19554 211/300 Read block 1007 len 12288
19555 212/300 Read block 3657 len 16384
19556 213/300 Read block 4413 len 4096
19557 214/300 Write block 1287 len 24576 data: 4 5 5 2 2 2
19558 215/300 Read block 1102 len 4096
19559 216/300 Read block 4158 len 40960
19560 217/300 Read block 1129 len 4096
19561 218/300 Read block 4127 len 4096
19562 219/300 Read block 4728 len 8192
19563 220/300 Write block 3413 len 28672 data: 4 4 4 5 4 4 4
19564 221/300 Read block 4787 len 4096
19565 222/300 Write block 3081 len 28672 data: 5 5 4 5 5 4 3
19566 223/300 Write block 2844 len 16384 data: 2 4 4 5
19567 224/300 Write block 1313 len 16384 data: 3 3 3 2
19568 225/300 Write block 4210 len 16384 data: 3 3 3 4
19569 226/300 Write block 3298 len 28672 data: 5 3 3 3 4 5 6
19570 227/300 Write block 1545 len 24576 data: 5 4 4 4 6 4
19571 228/300 Flush
19572 229/300 Write block 3507 len 24576 data: 7 7 6 6 6 5
19573 230/300 Read block 1019 len 32768
19574 231/300 Write block 392 len 28672 data: 3 3 4 4 4 5 4
19575 232/300 Write block 3853 len 40960 data: 4 5 5 4 4 5 5 5 6 6
19576 233/300 Read block 1200 len 20480
19577 234/300 Read block 385 len 28672
19578 235/300 Write block 3607 len 20480 data: 3 4 4 3 2
19579 236/300 Read block 535 len 40960
19580 237/300 Read block 4168 len 16384
19581 238/300 Flush
19582 239/300 Read block 4656 len 12288
19583 240/300 Write block 504 len 28672 data: 4 5 7 7 6 4 4
19584 241/300 Write block 4937 len 40960 data: 4 4 4 2 2 2 2 2 2 2
19585 242/300 Read block 632 len 20480
19586 243/300 Read block 3082 len 40960
19587 244/300 Read block 3392 len 32768
19588 245/300 Flush
19589 246/300 Write block 1879 len 40960 data: 4 6 6 6 6 6 4 4 2 3
19590 247/300 Read block 883 len 24576
19591 248/300 Write block 4137 len 40960 data: 4 5 4 5 6 5 4 4 3 2
19592 249/300 Write block 3796 len 16384 data: 3 5 7 5
19593 250/300 Write block 2408 len 28672 data: 4 4 4 3 3 3 3
19594 251/300 Flush
19595 252/300 Read block 4795 len 36864
19596 253/300 Flush
19597 254/300 Write block 2498 len 16384 data: 3 3 3 3
19598 255/300 Read block 4838 len 8192
19599 256/300 Flush
19600 257/300 Read block 186 len 36864
19601 258/300 Read block 2739 len 16384
19602 259/300 Read block 4706 len 40960
19603 260/300 Read block 2700 len 16384
19604 261/300 Write block 2003 len 8192 data: 4 4
19605 262/300 Read block 3304 len 28672
19606 263/300 Write block 4064 len 8192 data: 5 5
19607 264/300 Read block 1891 len 16384
19608 265/300 Write block 4669 len 28672 data: 2 2 3 4 4 5 5
19609 266/300 Flush
19610 267/300 Flush
19611 268/300 Flush
19612 269/300 Read block 3254 len 16384
19613 270/300 Flush
19614 271/300 Read block 4382 len 4096
19615 272/300 Read block 1751 len 32768
19616 273/300 Read block 2012 len 28672
19617 274/300 Read block 72 len 20480
19618 275/300 Write block 3574 len 32768 data: 3 3 3 3 3 4 5 5
19619 276/300 Write block 1374 len 12288 data: 3 3 3
19620 277/300 Read block 4936 len 16384
19621 278/300 Write block 2359 len 12288 data: 4 4 4
19622 279/300 Write block 1413 len 16384 data: 2 3 3 4
19623 280/300 Read block 2430 len 32768
19624 281/300 Write block 1266 len 28672 data: 7 6 3 3 4 5 8
19625 282/300 Flush
19626 283/300 Flush
19627 284/300 Flush
19628 285/300 Write block 155 len 12288 data: 6 7 7
19629 286/300 Write block 4749 len 4096 data: 3
19630 287/300 Write block 3798 len 24576 data: 8 6 5 6 5 5
19631 288/300 Write block 2451 len 28672 data: 3 3 3 3 3 3 2
19632 289/300 Write block 1539 len 20480 data: 4 4 4 3 4
19633 290/300 Write block 898 len 24576 data: 4 3 4 4 4 4
19634 291/300 Read block 791 len 40960
19635 292/300 Read block 4517 len 40960
19636 293/300 Write block 4271 len 16384 data: 4 4 4 3
19637 294/300 Read block 1043 len 12288
19638 295/300 Read block 4269 len 32768
19639 296/300 Write block 37 len 40960 data: 4 3 3 2 3 3 3 3 4 5
19640 297/300 Write block 2860 len 36864 data: 4 4 3 3 3 2 3 3 5
19641 298/300 Read block 4863 len 8192
19642 299/300 Read block 225 len 28672
19643 300/300 Write block 1090 len 12288 data: 5 4 6
19644 301/300 Write block 3962 len 12288 data: 6 5 3
19645 Main task has work to do, go find it
19646 got dsc Start(1)
19647 start 1
19648 Nov 28 04:54:27.399 INFO request completed, latency_us: 226, response_code: 204, uri: /start/cid/1, method: POST, req_id: a2d2f0b6-5878-4dcb-bb7d-86eb6a05a032, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
19649 Replay: started 1, returned:Ok(())
19650 ----------------------------------------------------------------
19651 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
19652 States: Active Offline Active
19653 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
19654 6132 Acked Flush 0 Done Sent Done false
19655 6133 Acked Write 5 Done Sent Done false
19656 6134 Acked Read 6 Done Sent Done false
19657 6135 Acked Read 10 Done Sent Done false
19658 6136 Acked Read 3 Done Sent Done false
19659 6137 Acked Read 7 Done Sent Done false
19660 6138 Acked Flush 0 Done Sent Done false
19661 6139 Acked Flush 0 Done Sent Done false
19662 6140 Acked Read 1 Done Sent Done false
19663 6141 Acked Read 2 Done Sent Done false
19664 6142 Acked Read 4 Done Sent Done false
19665 6143 Acked Read 5 Done Sent Done false
19666 6144 Acked Write 3 Done Sent Done false
19667 6145 Acked Flush 0 Done Sent Done false
19668 6146 Acked Read 8 Done Sent Done false
19669 6147 Acked Write 1 Done Sent Done false
19670 6148 Acked Read 1 Done Sent Done false
19671 6149 Acked Read 9 Done Sent Done false
19672 6150 Acked Write 10 Done Sent Done false
19673 6151 Acked Write 3 Done Sent Done false
19674 6152 Acked Read 10 Done Sent Done false
19675 6153 Acked Read 7 Done Sent Done false
19676 6154 Acked Read 1 Done Sent Done false
19677 6155 Acked Read 8 Done Sent Done false
19678 6156 Acked Flush 0 Done Sent Done false
19679 6157 Acked Read 2 Done Sent Done false
19680 6158 Acked Write 7 Done Sent Done false
19681 6159 Acked Read 3 Done Sent Done false
19682 6160 Acked Read 7 Done Sent Done false
19683 6161 Acked Read 4 Done Sent Done false
19684 6162 Acked Flush 0 Done Sent Done false
19685 6163 Acked Flush 0 Done Sent Done false
19686 6164 Acked Read 10 Done Sent Done false
19687 6165 Acked Read 5 Done Sent Done false
19688 6166 Acked Read 6 Done Sent Done false
19689 6167 Acked Write 4 Done Sent Done false
19690 6168 Acked Write 5 Done Sent Done false
19691 6169 Acked Read 4 Done Sent Done false
19692 6170 Acked Read 5 Done Sent Done false
19693 6171 Acked Read 10 Done Sent Done false
19694 6172 Acked Write 5 Done Sent Done false
19695 6173 Acked Write 4 Done Sent Done false
19696 6174 Acked Write 8 Done Sent Done false
19697 6175 Acked Read 7 Done Sent Done false
19698 6176 Acked Write 6 Done Sent Done false
19699 6177 Acked Write 7 Done Sent Done false
19700 6178 Acked Read 7 Done Sent Done false
19701 6179 Acked Write 10 Done Sent Done false
19702 6180 Acked Read 1 Done Sent Done false
19703 6181 Acked Flush 0 Done Sent Done false
19704 6182 Acked Flush 0 Done Sent Done false
19705 6183 Acked Read 7 Done Sent Done false
19706 6184 Acked Write 10 Done Sent Done false
19707 6185 Acked Read 5 Done Sent Done false
19708 6186 Acked Write 5 Done Sent Done false
19709 6187 Acked Read 6 Done Sent Done false
19710 6188 Acked Read 1 Done Sent Done false
19711 6189 Acked Write 6 Done Sent Done false
19712 6190 Acked Read 1 Done Sent Done false
19713 6191 Acked Read 1 Done Sent Done false
19714 6192 Acked Write 6 Done Sent Done false
19715 6193 Acked Write 1 Done Sent Done false
19716 6194 Acked Read 3 Done Sent Done false
19717 6195 Acked Write 10 Done Sent Done false
19718 6196 Acked Flush 0 Done Sent Done false
19719 6197 Acked Write 7 Done Sent Done false
19720 6198 Acked Flush 0 Done Sent Done false
19721 6199 Acked Write 1 Done Sent Done false
19722 6200 Acked Read 10 Done Sent Done false
19723 6201 Acked Read 1 Done Sent Done false
19724 6202 Acked Write 3 Done Sent Done false
19725 6203 Acked Write 5 Done Sent Done false
19726 6204 Acked Read 9 Done Sent Done false
19727 6205 Acked Read 9 Done Sent Done false
19728 6206 Acked Read 4 Done Sent Done false
19729 6207 Acked Read 4 Done Sent Done false
19730 6208 Acked Read 8 Done Sent Done false
19731 6209 Acked Read 6 Done Sent Done false
19732 6210 Acked Read 9 Done Sent Done false
19733 6211 Acked Read 4 Done Sent Done false
19734 6212 Acked Read 7 Done Sent Done false
19735 6213 Acked Write 8 Done Sent Done false
19736 6214 Acked Write 5 Done Sent Done false
19737 6215 Acked Write 3 Done Sent Done false
19738 6216 Acked Write 7 Done Sent Done false
19739 6217 Acked Flush 0 Done Sent Done false
19740 6218 Acked Write 1 Done Sent Done false
19741 6219 Acked Write 4 Done Sent Done false
19742 6220 Acked Read 3 Done Sent Done false
19743 6221 Acked Read 8 Done Sent Done false
19744 6222 Acked Write 7 Done Sent Done false
19745 6223 Acked Read 3 Done Sent Done false
19746 6224 Acked Flush 0 Done Sent Done false
19747 6225 Acked Flush 0 Done Sent Done false
19748 6226 Acked Write 1 Done Sent Done false
19749 6227 Acked Write 9 Done Sent Done false
19750 6228 Acked Write 3 Done Sent Done false
19751 6229 Acked Write 9 Done Sent Done false
19752 6230 Acked Read 1 Done Sent Done false
19753 6231 Acked Write 6 Done Sent Done false
19754 6232 Acked Write 8 Done Sent Done false
19755 6233 Acked Write 10 Done Sent Done false
19756 6234 Acked Write 4 Done Sent Done false
19757 6235 Acked Read 7 Done Sent Done false
19758 6236 Acked Read 9 Done Sent Done false
19759 6237 Acked Read 3 Done Sent Done false
19760 6238 Acked Read 10 Done Sent Done false
19761 6239 Acked Flush 0 Done Sent Done false
19762 6240 Acked Read 1 Done Sent Done false
19763 6241 Acked Write 3 Done Sent Done false
19764 6242 Acked Read 9 Done Sent Done false
19765 6243 Acked Write 1 Done Sent Done false
19766 6244 Acked Read 1 Done Sent Done false
19767 6245 Acked Write 4 Done Sent Done false
19768 6246 Acked Read 2 Done Sent Done false
19769 6247 Acked Flush 0 Done Sent Done false
19770 6248 Acked Write 3 Done Sent Done false
19771 6249 Acked Read 5 Done Sent Done false
19772 6250 Acked Read 8 Done Sent Done false
19773 6251 Acked Read 8 Done Sent Done false
19774 6252 Acked Read 7 Done Sent Done false
19775 6253 Acked Write 4 Done Sent Done false
19776 6254 Acked Read 2 Done Sent Done false
19777 6255 Acked Write 7 Done Sent Done false
19778 6256 Acked Read 10 Done Sent Done false
19779 6257 Acked Write 2 Done Sent Done false
19780 6258 Acked Read 2 Done Sent Done false
19781 6259 Acked Read 6 Done Sent Done false
19782 6260 Acked Read 7 Done Sent Done false
19783 6261 Acked Write 6 Done Sent Done false
19784 6262 Acked Read 8 Done Sent Done false
19785 6263 Acked Flush 0 Done Sent Done false
19786 6264 Acked Write 1 Done Sent Done false
19787 6265 Acked Write 5 Done Sent Done false
19788 6266 Acked Flush 0 Done Sent Done false
19789 6267 Acked Write 6 Done Sent Done false
19790 6268 Acked Read 6 Done Sent Done false
19791 6269 Acked Write 4 Done Sent Done false
19792 6270 Acked Write 7 Done Sent Done false
19793 6271 Acked Read 9 Done Sent Done false
19794 6272 Acked Read 5 Done Sent Done false
19795 6273 Acked Read 8 Done Sent Done false
19796 6274 Acked Write 5 Done Sent Done false
19797 6275 Acked Flush 0 Done Sent Done false
19798 6276 Acked Write 9 Done Sent Done false
19799 6277 Acked Read 2 Done Sent Done false
19800 6278 Acked Write 6 Done Sent Done false
19801 6279 Acked Read 10 Done Sent Done false
19802 6280 Acked Write 3 Done Sent Done false
19803 6281 Acked Write 5 Done Sent Done false
19804 6282 Acked Read 6 Done Sent Done false
19805 6283 Acked Read 8 Done Sent Done false
19806 6284 Acked Read 10 Done Sent Done false
19807 6285 Acked Read 10 Done Sent Done false
19808 6286 Acked Read 3 Done Sent Done false
19809 6287 Acked Write 9 Done Sent Done false
19810 6288 Acked Read 3 Done Sent Done false
19811 6289 Acked Write 10 Done Sent Done false
19812 6290 Acked Read 1 Done Sent Done false
19813 6291 Acked Read 7 Done Sent Done false
19814 6292 Acked Read 10 Done Sent Done false
19815 6293 Acked Write 2 Done Sent Done false
19816 6294 Acked Read 10 Done Sent Done false
19817 6295 Acked Read 4 Done Sent Done false
19818 6296 Acked Read 1 Done Sent Done false
19819 6297 Acked Read 8 Done Sent Done false
19820 6298 Acked Read 4 Done Sent Done false
19821 6299 Acked Write 10 Done Sent Done false
19822 6300 Acked Read 6 Done Sent Done false
19823 6301 Acked Read 8 Done Sent Done false
19824 6302 Acked Read 10 Done Sent Done false
19825 6303 Acked Write 1 Done Sent Done false
19826 6304 Acked Write 6 Done Sent Done false
19827 6305 Acked Write 9 Done Sent Done false
19828 6306 Acked Read 5 Done Sent Done false
19829 6307 Acked Read 7 Done Sent Done false
19830 6308 Acked Flush 0 Done Sent Done false
19831 6309 Acked Read 3 Done Sent Done false
19832 6310 Acked Write 2 Done Sent Done false
19833 6311 Acked Read 1 Done Sent Done false
19834 6312 Acked Read 7 Done Sent Done false
19835 6313 Acked Read 8 Done Sent Done false
19836 6314 Acked Read 3 Done Sent Done false
19837 6315 Acked Write 8 Done Sent Done false
19838 6316 Acked Write 2 Done Sent Done false
19839 6317 Acked Write 4 Done Sent Done false
19840 6318 Acked Write 10 Done Sent Done false
19841 6319 Acked Write 10 Done Sent Done false
19842 6320 Acked Write 3 Done Sent Done false
19843 6321 Acked Write 10 Done Sent Done false
19844 6322 Acked Write 4 Done Sent Done false
19845 6323 Acked Read 5 Done Sent Done false
19846 6324 Acked Write 3 Done Sent Done false
19847 6325 Acked Write 8 Done Sent Done false
19848 6326 Acked Write 10 Done Sent Done false
19849 6327 Acked Write 3 Done Sent Done false
19850 6328 Acked Read 10 Done Sent Done false
19851 6329 Acked Flush 0 Done Sent Done false
19852 6330 Acked Write 3 Done Sent Done false
19853 6331 Acked Read 6 Done Sent Done false
19854 6332 Acked Read 6 Done Sent Done false
19855 6333 Acked Read 9 Done Sent Done false
19856 6334 Acked Read 8 Done Sent Done false
19857 6335 Acked Read 9 Done Sent Done false
19858 6336 Acked Write 8 Done Sent Done false
19859 6337 Acked Write 7 Done Sent Done false
19860 6338 Acked Read 10 Done Sent Done false
19861 6339 Acked Write 2 Done Sent Done false
19862 6340 Acked Flush 0 Done Sent Done false
19863 6341 Acked Flush 0 Done Sent Done false
19864 6342 Acked Read 3 Done Sent Done false
19865 6343 Acked Read 4 Done Sent Done false
19866 6344 Acked Read 1 Done Sent Done false
19867 6345 Acked Write 6 Done Sent Done false
19868 6346 Acked Read 1 Done Sent Done false
19869 6347 Acked Read 10 Done Sent Done false
19870 6348 Acked Read 1 Done Sent Done false
19871 6349 Acked Read 1 Done Sent Done false
19872 6350 Acked Read 2 Done Sent Done false
19873 6351 Acked Write 7 Done Sent Done false
19874 6352 Acked Read 1 Done Sent Done false
19875 6353 Acked Write 7 Done Sent Done false
19876 6354 Acked Write 4 Done Sent Done false
19877 6355 Acked Write 4 Done Sent Done false
19878 6356 Acked Write 4 Done Sent Done false
19879 6357 Acked Write 7 Done Sent Done false
19880 6358 Acked Write 6 Done Sent Done false
19881 6359 Acked Flush 0 Done Sent Done false
19882 6360 Acked Write 6 Done Sent Done false
19883 6361 Acked Read 8 Done Sent Done false
19884 6362 Acked Write 7 Done Sent Done false
19885 6363 Acked Write 10 Done Sent Done false
19886 6364 Acked Read 5 Done Sent Done false
19887 6365 Acked Read 7 Done Sent Done false
19888 6366 Acked Write 5 Done Sent Done false
19889 6367 Acked Read 10 Done Sent Done false
19890 6368 Acked Read 4 Done Sent Done false
19891 6369 Acked Flush 0 Done Sent Done false
19892 6370 Acked Read 3 Done Sent Done false
19893 6371 Acked Write 7 Done Sent Done false
19894 6372 Acked Write 10 Done Sent Done false
19895 6373 Acked Read 5 Done Sent Done false
19896 6374 Acked Read 10 Done Sent Done false
19897 6375 Acked Read 8 Done Sent Done false
19898 6376 Acked Flush 0 Done Sent Done false
19899 6377 Acked Write 10 Done Sent Done false
19900 6378 Acked Read 6 Done Sent Done false
19901 6379 Acked Write 10 Done Sent Done false
19902 6380 Acked Write 4 Done Sent Done false
19903 6381 Acked Write 7 Done Sent Done false
19904 6382 Acked Flush 0 Done Sent Done false
19905 6383 Acked Read 9 Done Sent Done false
19906 6384 Acked Flush 0 Done Sent Done false
19907 6385 Acked Write 4 Done Sent Done false
19908 6386 Acked Read 2 Done Sent Done false
19909 6387 Acked Flush 0 Done Sent Done false
19910 6388 Acked Read 9 Done Sent Done false
19911 6389 Acked Read 4 Done Sent Done false
19912 6390 Acked Read 10 Done Sent Done false
19913 6391 Acked Read 4 Done Sent Done false
19914 6392 Acked Write 2 Done Sent Done false
19915 6393 Acked Read 7 Done Sent Done false
19916 6394 Acked Write 2 Done Sent Done false
19917 6395 Acked Read 4 Done Sent Done false
19918 6396 Acked Write 7 Done Sent Done false
19919 6397 Acked Flush 0 Done Sent Done false
19920 6398 Acked Flush 0 Done Sent Done false
19921 6399 Acked Flush 0 Done Sent Done false
19922 6400 Acked Read 4 Done Sent Done false
19923 6401 Acked Flush 0 Done Sent Done false
19924 6402 Acked Read 1 Done Sent Done false
19925 6403 Acked Read 8 Done Sent Done false
19926 6404 Acked Read 7 Done Sent Done false
19927 6405 Acked Read 5 Done Sent Done false
19928 6406 Acked Write 8 Done Sent Done false
19929 6407 Acked Write 3 Done Sent Done false
19930 6408 Acked Read 4 Done Sent Done false
19931 6409 Acked Write 3 Done Sent Done false
19932 6410 Acked Write 4 Done Sent Done false
19933 6411 Acked Read 8 Done Sent Done false
19934 6412 Acked Write 7 Done Sent Done false
19935 6413 Acked Flush 0 Done Sent Done false
19936 6414 Acked Flush 0 Done Sent Done false
19937 6415 Acked Flush 0 Done Sent Done false
19938 6416 Acked Write 3 Done Sent Done false
19939 6417 Acked Write 1 Done Sent Done false
19940 6418 Acked Write 6 Done Sent Done false
19941 6419 Acked Write 7 Done Sent Done false
19942 6420 Acked Write 5 Done Sent Done false
19943 6421 Acked Write 6 Done Sent Done false
19944 6422 Acked Read 10 Done Sent Done false
19945 6423 Acked Read 10 Done Sent Done false
19946 6424 Acked Write 4 Done Sent Done false
19947 6425 Acked Read 3 Done Sent Done false
19948 6426 Acked Read 8 Done Sent Done false
19949 6427 Acked Write 10 Done Sent Done false
19950 6428 Acked Write 9 Done Sent Done false
19951 6429 Acked Read 2 Done Sent Done false
19952 6430 Acked Read 7 Done Sent Done false
19953 6431 Acked Write 3 Done Sent Done false
19954 6432 Acked Write 3 Done Sent Done false
19955 STATES DS:0 DS:1 DS:2 TOTAL
19956 Sent 0 301 0 301
19957 Done 301 0 301 602
19958 Skipped 0 0 0 0
19959 Error 0 0 0 0
19960 Last Flush: 6415 6131 6415
19961 Downstairs last five completed: 6131 6130 6129 6128 6127
19962 Upstairs last five completed: 6432 6431 6430 6429 6428
19963 CLIENT: Up:0 ds:301 act:2
19964 [8820] got start action
19965 [8820] I am going to restart
19966 Starting downstairs at port 8820
19967 Make output file at "/tmp/dsc/downstairs-8820.txt"
19968 [8820][1] reports Starting
19969 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1220)
19970 [8820][1] reports Running
19971 ----------------------------------------------------------------
19972 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
19973 States: Active Offline Active
19974 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
19975 6132 Acked Flush 0 Done Sent Done false
19976 6133 Acked Write 5 Done Sent Done false
19977 6134 Acked Read 6 Done Sent Done false
19978 6135 Acked Read 10 Done Sent Done false
19979 6136 Acked Read 3 Done Sent Done false
19980 6137 Acked Read 7 Done Sent Done false
19981 6138 Acked Flush 0 Done Sent Done false
19982 6139 Acked Flush 0 Done Sent Done false
19983 6140 Acked Read 1 Done Sent Done false
19984 6141 Acked Read 2 Done Sent Done false
19985 6142 Acked Read 4 Done Sent Done false
19986 6143 Acked Read 5 Done Sent Done false
19987 6144 Acked Write 3 Done Sent Done false
19988 6145 Acked Flush 0 Done Sent Done false
19989 6146 Acked Read 8 Done Sent Done false
19990 6147 Acked Write 1 Done Sent Done false
19991 6148 Acked Read 1 Done Sent Done false
19992 6149 Acked Read 9 Done Sent Done false
19993 6150 Acked Write 10 Done Sent Done false
19994 6151 Acked Write 3 Done Sent Done false
19995 6152 Acked Read 10 Done Sent Done false
19996 6153 Acked Read 7 Done Sent Done false
19997 6154 Acked Read 1 Done Sent Done false
19998 6155 Acked Read 8 Done Sent Done false
19999 6156 Acked Flush 0 Done Sent Done false
20000 6157 Acked Read 2 Done Sent Done false
20001 6158 Acked Write 7 Done Sent Done false
20002 6159 Acked Read 3 Done Sent Done false
20003 6160 Acked Read 7 Done Sent Done false
20004 6161 Acked Read 4 Done Sent Done false
20005 6162 Acked Flush 0 Done Sent Done false
20006 6163 Acked Flush 0 Done Sent Done false
20007 6164 Acked Read 10 Done Sent Done false
20008 6165 Acked Read 5 Done Sent Done false
20009 6166 Acked Read 6 Done Sent Done false
20010 6167 Acked Write 4 Done Sent Done false
20011 6168 Acked Write 5 Done Sent Done false
20012 6169 Acked Read 4 Done Sent Done false
20013 6170 Acked Read 5 Done Sent Done false
20014 6171 Acked Read 10 Done Sent Done false
20015 6172 Acked Write 5 Done Sent Done false
20016 6173 Acked Write 4 Done Sent Done false
20017 6174 Acked Write 8 Done Sent Done false
20018 6175 Acked Read 7 Done Sent Done false
20019 6176 Acked Write 6 Done Sent Done false
20020 6177 Acked Write 7 Done Sent Done false
20021 6178 Acked Read 7 Done Sent Done false
20022 6179 Acked Write 10 Done Sent Done false
20023 6180 Acked Read 1 Done Sent Done false
20024 6181 Acked Flush 0 Done Sent Done false
20025 6182 Acked Flush 0 Done Sent Done false
20026 6183 Acked Read 7 Done Sent Done false
20027 6184 Acked Write 10 Done Sent Done false
20028 6185 Acked Read 5 Done Sent Done false
20029 6186 Acked Write 5 Done Sent Done false
20030 6187 Acked Read 6 Done Sent Done false
20031 6188 Acked Read 1 Done Sent Done false
20032 6189 Acked Write 6 Done Sent Done false
20033 6190 Acked Read 1 Done Sent Done false
20034 6191 Acked Read 1 Done Sent Done false
20035 6192 Acked Write 6 Done Sent Done false
20036 6193 Acked Write 1 Done Sent Done false
20037 6194 Acked Read 3 Done Sent Done false
20038 6195 Acked Write 10 Done Sent Done false
20039 6196 Acked Flush 0 Done Sent Done false
20040 6197 Acked Write 7 Done Sent Done false
20041 6198 Acked Flush 0 Done Sent Done false
20042 6199 Acked Write 1 Done Sent Done false
20043 6200 Acked Read 10 Done Sent Done false
20044 6201 Acked Read 1 Done Sent Done false
20045 6202 Acked Write 3 Done Sent Done false
20046 6203 Acked Write 5 Done Sent Done false
20047 6204 Acked Read 9 Done Sent Done false
20048 6205 Acked Read 9 Done Sent Done false
20049 6206 Acked Read 4 Done Sent Done false
20050 6207 Acked Read 4 Done Sent Done false
20051 6208 Acked Read 8 Done Sent Done false
20052 6209 Acked Read 6 Done Sent Done false
20053 6210 Acked Read 9 Done Sent Done false
20054 6211 Acked Read 4 Done Sent Done false
20055 6212 Acked Read 7 Done Sent Done false
20056 6213 Acked Write 8 Done Sent Done false
20057 6214 Acked Write 5 Done Sent Done false
20058 6215 Acked Write 3 Done Sent Done false
20059 6216 Acked Write 7 Done Sent Done false
20060 6217 Acked Flush 0 Done Sent Done false
20061 6218 Acked Write 1 Done Sent Done false
20062 6219 Acked Write 4 Done Sent Done false
20063 6220 Acked Read 3 Done Sent Done false
20064 6221 Acked Read 8 Done Sent Done false
20065 6222 Acked Write 7 Done Sent Done false
20066 6223 Acked Read 3 Done Sent Done false
20067 6224 Acked Flush 0 Done Sent Done false
20068 6225 Acked Flush 0 Done Sent Done false
20069 6226 Acked Write 1 Done Sent Done false
20070 6227 Acked Write 9 Done Sent Done false
20071 6228 Acked Write 3 Done Sent Done false
20072 6229 Acked Write 9 Done Sent Done false
20073 6230 Acked Read 1 Done Sent Done false
20074 6231 Acked Write 6 Done Sent Done false
20075 6232 Acked Write 8 Done Sent Done false
20076 6233 Acked Write 10 Done Sent Done false
20077 6234 Acked Write 4 Done Sent Done false
20078 6235 Acked Read 7 Done Sent Done false
20079 6236 Acked Read 9 Done Sent Done false
20080 6237 Acked Read 3 Done Sent Done false
20081 6238 Acked Read 10 Done Sent Done false
20082 6239 Acked Flush 0 Done Sent Done false
20083 6240 Acked Read 1 Done Sent Done false
20084 6241 Acked Write 3 Done Sent Done false
20085 6242 Acked Read 9 Done Sent Done false
20086 6243 Acked Write 1 Done Sent Done false
20087 6244 Acked Read 1 Done Sent Done false
20088 6245 Acked Write 4 Done Sent Done false
20089 6246 Acked Read 2 Done Sent Done false
20090 6247 Acked Flush 0 Done Sent Done false
20091 6248 Acked Write 3 Done Sent Done false
20092 6249 Acked Read 5 Done Sent Done false
20093 6250 Acked Read 8 Done Sent Done false
20094 6251 Acked Read 8 Done Sent Done false
20095 6252 Acked Read 7 Done Sent Done false
20096 6253 Acked Write 4 Done Sent Done false
20097 6254 Acked Read 2 Done Sent Done false
20098 6255 Acked Write 7 Done Sent Done false
20099 6256 Acked Read 10 Done Sent Done false
20100 6257 Acked Write 2 Done Sent Done false
20101 6258 Acked Read 2 Done Sent Done false
20102 6259 Acked Read 6 Done Sent Done false
20103 6260 Acked Read 7 Done Sent Done false
20104 6261 Acked Write 6 Done Sent Done false
20105 6262 Acked Read 8 Done Sent Done false
20106 6263 Acked Flush 0 Done Sent Done false
20107 6264 Acked Write 1 Done Sent Done false
20108 6265 Acked Write 5 Done Sent Done false
20109 6266 Acked Flush 0 Done Sent Done false
20110 6267 Acked Write 6 Done Sent Done false
20111 6268 Acked Read 6 Done Sent Done false
20112 6269 Acked Write 4 Done Sent Done false
20113 6270 Acked Write 7 Done Sent Done false
20114 6271 Acked Read 9 Done Sent Done false
20115 6272 Acked Read 5 Done Sent Done false
20116 6273 Acked Read 8 Done Sent Done false
20117 6274 Acked Write 5 Done Sent Done false
20118 6275 Acked Flush 0 Done Sent Done false
20119 6276 Acked Write 9 Done Sent Done false
20120 6277 Acked Read 2 Done Sent Done false
20121 6278 Acked Write 6 Done Sent Done false
20122 6279 Acked Read 10 Done Sent Done false
20123 6280 Acked Write 3 Done Sent Done false
20124 6281 Acked Write 5 Done Sent Done false
20125 6282 Acked Read 6 Done Sent Done false
20126 6283 Acked Read 8 Done Sent Done false
20127 6284 Acked Read 10 Done Sent Done false
20128 6285 Acked Read 10 Done Sent Done false
20129 6286 Acked Read 3 Done Sent Done false
20130 6287 Acked Write 9 Done Sent Done false
20131 6288 Acked Read 3 Done Sent Done false
20132 6289 Acked Write 10 Done Sent Done false
20133 6290 Acked Read 1 Done Sent Done false
20134 6291 Acked Read 7 Done Sent Done false
20135 6292 Acked Read 10 Done Sent Done false
20136 6293 Acked Write 2 Done Sent Done false
20137 6294 Acked Read 10 Done Sent Done false
20138 6295 Acked Read 4 Done Sent Done false
20139 6296 Acked Read 1 Done Sent Done false
20140 6297 Acked Read 8 Done Sent Done false
20141 6298 Acked Read 4 Done Sent Done false
20142 6299 Acked Write 10 Done Sent Done false
20143 6300 Acked Read 6 Done Sent Done false
20144 6301 Acked Read 8 Done Sent Done false
20145 6302 Acked Read 10 Done Sent Done false
20146 6303 Acked Write 1 Done Sent Done false
20147 6304 Acked Write 6 Done Sent Done false
20148 6305 Acked Write 9 Done Sent Done false
20149 6306 Acked Read 5 Done Sent Done false
20150 6307 Acked Read 7 Done Sent Done false
20151 6308 Acked Flush 0 Done Sent Done false
20152 6309 Acked Read 3 Done Sent Done false
20153 6310 Acked Write 2 Done Sent Done false
20154 6311 Acked Read 1 Done Sent Done false
20155 6312 Acked Read 7 Done Sent Done false
20156 6313 Acked Read 8 Done Sent Done false
20157 6314 Acked Read 3 Done Sent Done false
20158 6315 Acked Write 8 Done Sent Done false
20159 6316 Acked Write 2 Done Sent Done false
20160 6317 Acked Write 4 Done Sent Done false
20161 6318 Acked Write 10 Done Sent Done false
20162 6319 Acked Write 10 Done Sent Done false
20163 6320 Acked Write 3 Done Sent Done false
20164 6321 Acked Write 10 Done Sent Done false
20165 6322 Acked Write 4 Done Sent Done false
20166 6323 Acked Read 5 Done Sent Done false
20167 6324 Acked Write 3 Done Sent Done false
20168 6325 Acked Write 8 Done Sent Done false
20169 6326 Acked Write 10 Done Sent Done false
20170 6327 Acked Write 3 Done Sent Done false
20171 6328 Acked Read 10 Done Sent Done false
20172 6329 Acked Flush 0 Done Sent Done false
20173 6330 Acked Write 3 Done Sent Done false
20174 6331 Acked Read 6 Done Sent Done false
20175 6332 Acked Read 6 Done Sent Done false
20176 6333 Acked Read 9 Done Sent Done false
20177 6334 Acked Read 8 Done Sent Done false
20178 6335 Acked Read 9 Done Sent Done false
20179 6336 Acked Write 8 Done Sent Done false
20180 6337 Acked Write 7 Done Sent Done false
20181 6338 Acked Read 10 Done Sent Done false
20182 6339 Acked Write 2 Done Sent Done false
20183 6340 Acked Flush 0 Done Sent Done false
20184 6341 Acked Flush 0 Done Sent Done false
20185 6342 Acked Read 3 Done Sent Done false
20186 6343 Acked Read 4 Done Sent Done false
20187 6344 Acked Read 1 Done Sent Done false
20188 6345 Acked Write 6 Done Sent Done false
20189 6346 Acked Read 1 Done Sent Done false
20190 6347 Acked Read 10 Done Sent Done false
20191 6348 Acked Read 1 Done Sent Done false
20192 6349 Acked Read 1 Done Sent Done false
20193 6350 Acked Read 2 Done Sent Done false
20194 6351 Acked Write 7 Done Sent Done false
20195 6352 Acked Read 1 Done Sent Done false
20196 6353 Acked Write 7 Done Sent Done false
20197 6354 Acked Write 4 Done Sent Done false
20198 6355 Acked Write 4 Done Sent Done false
20199 6356 Acked Write 4 Done Sent Done false
20200 6357 Acked Write 7 Done Sent Done false
20201 6358 Acked Write 6 Done Sent Done false
20202 6359 Acked Flush 0 Done Sent Done false
20203 6360 Acked Write 6 Done Sent Done false
20204 6361 Acked Read 8 Done Sent Done false
20205 6362 Acked Write 7 Done Sent Done false
20206 6363 Acked Write 10 Done Sent Done false
20207 6364 Acked Read 5 Done Sent Done false
20208 6365 Acked Read 7 Done Sent Done false
20209 6366 Acked Write 5 Done Sent Done false
20210 6367 Acked Read 10 Done Sent Done false
20211 6368 Acked Read 4 Done Sent Done false
20212 6369 Acked Flush 0 Done Sent Done false
20213 6370 Acked Read 3 Done Sent Done false
20214 6371 Acked Write 7 Done Sent Done false
20215 6372 Acked Write 10 Done Sent Done false
20216 6373 Acked Read 5 Done Sent Done false
20217 6374 Acked Read 10 Done Sent Done false
20218 6375 Acked Read 8 Done Sent Done false
20219 6376 Acked Flush 0 Done Sent Done false
20220 6377 Acked Write 10 Done Sent Done false
20221 6378 Acked Read 6 Done Sent Done false
20222 6379 Acked Write 10 Done Sent Done false
20223 6380 Acked Write 4 Done Sent Done false
20224 6381 Acked Write 7 Done Sent Done false
20225 6382 Acked Flush 0 Done Sent Done false
20226 6383 Acked Read 9 Done Sent Done false
20227 6384 Acked Flush 0 Done Sent Done false
20228 6385 Acked Write 4 Done Sent Done false
20229 6386 Acked Read 2 Done Sent Done false
20230 6387 Acked Flush 0 Done Sent Done false
20231 6388 Acked Read 9 Done Sent Done false
20232 6389 Acked Read 4 Done Sent Done false
20233 6390 Acked Read 10 Done Sent Done false
20234 6391 Acked Read 4 Done Sent Done false
20235 6392 Acked Write 2 Done Sent Done false
20236 6393 Acked Read 7 Done Sent Done false
20237 6394 Acked Write 2 Done Sent Done false
20238 6395 Acked Read 4 Done Sent Done false
20239 6396 Acked Write 7 Done Sent Done false
20240 6397 Acked Flush 0 Done Sent Done false
20241 6398 Acked Flush 0 Done Sent Done false
20242 6399 Acked Flush 0 Done Sent Done false
20243 6400 Acked Read 4 Done Sent Done false
20244 6401 Acked Flush 0 Done Sent Done false
20245 6402 Acked Read 1 Done Sent Done false
20246 6403 Acked Read 8 Done Sent Done false
20247 6404 Acked Read 7 Done Sent Done false
20248 6405 Acked Read 5 Done Sent Done false
20249 6406 Acked Write 8 Done Sent Done false
20250 6407 Acked Write 3 Done Sent Done false
20251 6408 Acked Read 4 Done Sent Done false
20252 6409 Acked Write 3 Done Sent Done false
20253 6410 Acked Write 4 Done Sent Done false
20254 6411 Acked Read 8 Done Sent Done false
20255 6412 Acked Write 7 Done Sent Done false
20256 6413 Acked Flush 0 Done Sent Done false
20257 6414 Acked Flush 0 Done Sent Done false
20258 6415 Acked Flush 0 Done Sent Done false
20259 6416 Acked Write 3 Done Sent Done false
20260 6417 Acked Write 1 Done Sent Done false
20261 6418 Acked Write 6 Done Sent Done false
20262 6419 Acked Write 7 Done Sent Done false
20263 6420 Acked Write 5 Done Sent Done false
20264 6421 Acked Write 6 Done Sent Done false
20265 6422 Acked Read 10 Done Sent Done false
20266 6423 Acked Read 10 Done Sent Done false
20267 6424 Acked Write 4 Done Sent Done false
20268 6425 Acked Read 3 Done Sent Done false
20269 6426 Acked Read 8 Done Sent Done false
20270 6427 Acked Write 10 Done Sent Done false
20271 6428 Acked Write 9 Done Sent Done false
20272 6429 Acked Read 2 Done Sent Done false
20273 6430 Acked Read 7 Done Sent Done false
20274 6431 Acked Write 3 Done Sent Done false
20275 6432 Acked Write 3 Done Sent Done false
20276 6433 Acked Flush 0 Done Sent Done false
20277 STATES DS:0 DS:1 DS:2 TOTAL
20278 Sent 0 302 0 302
20279 Done 302 0 302 604
20280 Skipped 0 0 0 0
20281 Error 0 0 0 0
20282 Last Flush: 6433 6131 6433
20283 Downstairs last five completed: 6131 6130 6129 6128 6127
20284 Upstairs last five completed: 6433 6432 6431 6430 6429
20285 CLIENT: Up:0 ds:302 act:2
202862024-11-28T04:54:33.162ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
202872024-11-28T04:54:33.162ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:62214) = io task client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
202882024-11-28T04:54:33.163ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
202892024-11-28T04:54:33.164ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
202902024-11-28T04:54:33.164ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
202912024-11-28T04:54:33.164ZINFOcrucible: send last flush ID to this DS: 6131 = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
202922024-11-28T04:54:33.164ZINFOcrucible: Replied this last flush ID: 6131 = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
202932024-11-28T04:54:33.164ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
202942024-11-28T04:54:33.164ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
202952024-11-28T04:54:33.164ZINFOcrucible: [1] client re-new 302 jobs since flush 6131 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
202962024-11-28T04:54:33.164ZINFOcrucible: [1] Replayed 302 jobs since flush: 6131 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
20297 ----------------------------------------------------------------
20298 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
20299 Downstairs last five completed: 6433 6432 6431 6430 6429
20300 Upstairs last five completed: 6433 6432 6431 6430 6429
20301 CLIENT: Up:0 ds:0 act:3
20302 Replay: All jobs finished
20303 Main task has work to do, go find it
20304 got dsc Stop(0)
20305 stop 0
20306 [8810] Got stop action so:false kr:false
20307 Nov 28 04:54:35.402 INFO request completed, latency_us: 340, response_code: 204, uri: /stop/cid/0, method: POST, req_id: ec145752-39ef-425e-bbc6-e52ff38defec, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
203082024-11-28T04:54:35.403ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
20309 Nov 28 04:54:35.403 {"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2024-11-28T04:54:35.403325806Z","hostname":"w-01JDRHKEEFJ1228TFPSBEGXWHY","pid":1202,"":"io task","client":"0","":"downstairs","session_id":"4a2415b1-6594-436f-82e5-b8cf9a74eeaf"}
20310 INFO request completed, latency_us: 151, response_code: 200, uri: /state/cid/0, method: GET, req_id: 525ba4da-2bae-419e-abbf-dba558a0d12b, remote_addr:{"msg":"downstairs task for 0 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2024-11-28T04:54:35.403352692Z","hostname":"w-01JDRHKEEFJ1228TFPSBEGXWHY","pid":1202,"session_id":"4a2415b1-6594-436f-82e5-b8cf9a74eeaf"}
20311 127.0.0.1:60418, local_addr: 127.0.0.1:9998
203122024-11-28T04:54:35.403ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
203132024-11-28T04:54:35.403ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
203142024-11-28T04:54:35.403ZINFOcrucible: client task is exiting = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
20315 [8810] Exited with: ExitStatus(unix_wait_status(9))
20316 [8810][0] reports Exit
20317 Nov 28 04:54:39.405 INFO request completed, latency_us: 209, response_code: 200, uri: /state/cid/0, method: GET, req_id: 63156b68-ecd5-45ad-86cf-3d3b3847264e, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
20318 001/300 Write block 1629 len 16384 data: 6 5 5 5
20319 002/300 Flush
20320 003/300 Read block 2469 len 8192
20321 004/300 Read block 2838 len 16384
20322 005/300 Read block 1791 len 40960
20323 006/300 Read block 2280 len 36864
20324 007/300 Read block 2095 len 8192
20325 008/300 Read block 3097 len 36864
20326 009/300 Read block 4486 len 8192
20327 010/300 Write block 2410 len 24576 data: 5 4 4 4 4 3
20328 011/300 Read block 917 len 40960
20329 012/300 Write block 237 len 4096 data: 3
20330 013/300 Flush
20331 014/300 Write block 3607 len 20480 data: 4 5 5 4 3
20332 015/300 Read block 3029 len 20480
20333 016/300 Write block 4830 len 36864 data: 4 3 2 3 4 5 4 4 3
20334 017/300 Read block 3915 len 24576
20335 018/300 Write block 4239 len 12288 data: 3 3 3
20336 019/300 Write block 896 len 32768 data: 4 3 5 4 5 5 5 5
20337 020/300 Write block 2553 len 8192 data: 5 5
20338 021/300 Read block 330 len 40960
20339 022/300 Read block 3817 len 16384
20340 023/300 Read block 169 len 32768
20341 024/300 Read block 4488 len 8192
20342 025/300 Read block 3840 len 16384
20343 026/300 Write block 2103 len 40960 data: 5 4 5 3 3 3 4 6 5 8
20344 027/300 Write block 3437 len 28672 data: 3 3 3 3 3 3 2
20345 028/300 Read block 4460 len 8192
20346 029/300 Read block 4131 len 12288
20347 030/300 Read block 193 len 16384
20348 031/300 Read block 2892 len 12288
20349 032/300 Write block 4633 len 24576 data: 3 4 3 3 3 4
20350 033/300 Read block 1204 len 32768
20351 034/300 Read block 3491 len 32768
20352 035/300 Read block 3043 len 8192
20353 036/300 Read block 2827 len 20480
20354 037/300 Read block 1321 len 24576
20355 038/300 Read block 2238 len 24576
20356 039/300 Flush
20357 040/300 Read block 333 len 4096
20358 041/300 Read block 2557 len 24576
20359 042/300 Write block 4859 len 8192 data: 5 4
20360 043/300 Write block 1694 len 4096 data: 3
20361 044/300 Read block 1986 len 40960
20362 045/300 Read block 1824 len 40960
20363 046/300 Write block 2185 len 32768 data: 5 4 5 4 4 5 4 6
20364 047/300 Read block 4774 len 32768
20365 048/300 Write block 4699 len 24576 data: 3 2 3 2 3 5
20366 049/300 Write block 526 len 4096 data: 5
20367 050/300 Read block 4904 len 4096
20368 051/300 Write block 624 len 16384 data: 4 6 7 5
20369 052/300 Read block 2502 len 16384
20370 053/300 Read block 362 len 24576
20371 054/300 Read block 2419 len 32768
20372 055/300 Read block 1939 len 28672
20373 056/300 Read block 4292 len 36864
20374 057/300 Read block 1213 len 16384
20375 058/300 Flush
20376 059/300 Write block 3615 len 32768 data: 4 4 4 5 5 6 6 5
20377 060/300 Read block 4727 len 8192
20378 061/300 Read block 1286 len 28672
20379 062/300 Write block 1382 len 24576 data: 4 4 4 4 3 3
20380 063/300 Read block 3695 len 12288
20381 064/300 Read block 2052 len 12288
20382 065/300 Write block 4433 len 36864 data: 5 4 4 3 3 3 3 3 3
20383 066/300 Read block 4590 len 36864
20384 067/300 Write block 1053 len 28672 data: 8 5 5 3 4 3 4
20385 068/300 Write block 1772 len 32768 data: 5 4 4 6 7 6 5 5
20386 069/300 Read block 3249 len 20480
20387 070/300 Write block 4557 len 4096 data: 3
20388 071/300 Read block 806 len 32768
20389 072/300 Write block 3752 len 24576 data: 2 3 2 2 3 3
20390 073/300 Read block 3187 len 36864
20391 074/300 Write block 4830 len 12288 data: 5 4 3
20392 075/300 Write block 288 len 40960 data: 4 4 4 3 3 5 5 4 4 4
20393 076/300 Read block 2458 len 4096
20394 077/300 Write block 3090 len 32768 data: 5 5 5 4 4 3 2 2
20395 078/300 Read block 1690 len 16384
20396 079/300 Write block 94 len 36864 data: 2 2 2 2 3 4 3 4 4
20397 080/300 Write block 2524 len 32768 data: 7 8 8 7 6 4 4 4
20398 081/300 Write block 812 len 4096 data: 5
20399 082/300 Flush
20400 083/300 Read block 3715 len 16384
20401 084/300 Read block 1438 len 20480
20402 085/300 Write block 4430 len 40960 data: 5 6 6 6 5 5 4 4 4 4
20403 086/300 Write block 719 len 36864 data: 4 4 3 3 3 2 2 2 3
20404 087/300 Write block 1070 len 36864 data: 3 3 3 4 4 7 6 6 4
20405 088/300 Read block 267 len 40960
20406 089/300 Read block 4845 len 36864
20407 090/300 Write block 147 len 8192 data: 3 4
20408 091/300 Write block 4582 len 40960 data: 5 5 5 3 3 2 3 2 2 2
20409 092/300 Write block 376 len 40960 data: 6 7 8 9 9 8 9 9 7 4
20410 093/300 Read block 2660 len 32768
20411 094/300 Read block 4975 len 12288
20412 095/300 Read block 786 len 12288
20413 096/300 Write block 189 len 20480 data: 5 6 7 7 7
20414 097/300 Read block 604 len 32768
20415 098/300 Write block 3314 len 24576 data: 5 6 6 7 7 4
20416 099/300 Write block 2394 len 28672 data: 6 7 6 7 5 4 3
20417 100/300 Write block 930 len 8192 data: 3 3
20418 101/300 Write block 2607 len 20480 data: 3 3 3 3 3
20419 102/300 Write block 643 len 28672 data: 6 6 6 3 3 3 5
20420 103/300 Write block 4959 len 28672 data: 5 4 4 4 4 5 5
20421 104/300 Write block 3835 len 28672 data: 3 2 3 3 3 3 3
20422 105/300 Read block 4439 len 36864
20423 106/300 Read block 3987 len 24576
20424 107/300 Write block 1891 len 16384 data: 6 7 7 6
20425 108/300 Write block 2124 len 12288 data: 6 6 7
20426 109/300 Read block 1383 len 28672
20427 110/300 Write block 1945 len 4096 data: 4
20428 111/300 Write block 2381 len 28672 data: 8 5 5 5 4 4 4
20429 112/300 Read block 656 len 32768
20430 113/300 Write block 27 len 28672 data: 5 5 5 7 6 5 5
20431 114/300 Read block 2243 len 36864
20432 115/300 Write block 4927 len 32768 data: 5 4 5 5 4 5 4 4
20433 116/300 Flush
20434 117/300 Read block 589 len 16384
20435 118/300 Read block 3506 len 12288
20436 119/300 Write block 2673 len 16384 data: 4 4 4 4
20437 120/300 Write block 4759 len 40960 data: 4 4 4 4 3 3 2 3 3 5
20438 121/300 Flush
20439 122/300 Read block 2788 len 32768
20440 123/300 Read block 4406 len 16384
20441 124/300 Flush
20442 125/300 Read block 480 len 28672
20443 126/300 Read block 802 len 32768
20444 127/300 Read block 2112 len 36864
20445 128/300 Write block 4793 len 32768 data: 5 4 5 5 7 7 7 6
20446 129/300 Read block 4927 len 36864
20447 130/300 Write block 1874 len 32768 data: 2 3 3 3 4 5 7 7
20448 131/300 Flush
20449 132/300 Read block 1823 len 40960
20450 133/300 Write block 4195 len 40960 data: 2 2 2 3 4 4 4 4 4 3
20451 134/300 Write block 3066 len 16384 data: 4 3 3 2
20452 135/300 Read block 4369 len 20480
20453 136/300 Read block 3992 len 40960
20454 137/300 Write block 2659 len 32768 data: 4 4 5 5 5 4 5 5
20455 138/300 Write block 4440 len 12288 data: 4 4 4
20456 139/300 Read block 2347 len 8192
20457 140/300 Write block 463 len 32768 data: 2 4 4 5 4 7 8 8
20458 141/300 Flush
20459 142/300 Write block 2907 len 40960 data: 4 3 3 3 3 3 3 2 2 3
20460 143/300 Write block 1893 len 12288 data: 8 7 5
20461 144/300 Write block 522 len 16384 data: 4 4 6 5
20462 145/300 Read block 2821 len 28672
20463 146/300 Read block 2369 len 20480
20464 147/300 Write block 4324 len 16384 data: 3 3 3 4
20465 148/300 Write block 1307 len 32768 data: 4 4 4 4 4 3 4 4
20466 149/300 Write block 4476 len 40960 data: 7 6 5 5 5 4 4 3 2 2
20467 150/300 Flush
20468 151/300 Read block 2361 len 16384
20469 152/300 Write block 4719 len 8192 data: 2 3
20470 153/300 Write block 1445 len 4096 data: 2
20471 154/300 Read block 2500 len 40960
20472 155/300 Read block 4433 len 4096
20473 156/300 Read block 3259 len 16384
20474 157/300 Write block 3768 len 4096 data: 4
20475 158/300 Read block 2710 len 4096
20476 159/300 Write block 810 len 32768 data: 5 4 6 5 8 7 7 7
20477 160/300 Write block 1086 len 12288 data: 4 5 5
20478 161/300 Read block 2898 len 40960
20479 162/300 Write block 29 len 40960 data: 6 8 7 6 6 4 4 4 5 4
20480 163/300 Write block 4170 len 28672 data: 3 3 3 3 3 2 3
20481 164/300 Read block 2470 len 32768
20482 165/300 Read block 4916 len 4096
20483 166/300 Write block 4 len 4096 data: 4
20484 167/300 Write block 4992 len 4096 data: 6
20485 168/300 Flush
20486 169/300 Read block 637 len 24576
20487 170/300 Read block 2487 len 20480
20488 171/300 Write block 1460 len 28672 data: 3 4 4 5 5 6 6
20489 172/300 Read block 1486 len 36864
20490 173/300 Write block 1065 len 32768 data: 3 3 4 4 4 4 4 4
20491 174/300 Read block 2352 len 8192
20492 175/300 Write block 4197 len 12288 data: 3 4 5
20493 176/300 Write block 3471 len 32768 data: 3 3 3 3 3 4 4 4
20494 177/300 Read block 3340 len 40960
20495 178/300 Write block 1660 len 32768 data: 3 2 2 2 2 2 2 3
20496 179/300 Flush
20497 180/300 Read block 1250 len 20480
20498 181/300 Read block 4351 len 8192
20499 182/300 Write block 4486 len 36864 data: 3 3 3 3 3 4 5 5 4
20500 183/300 Flush
20501 184/300 Read block 358 len 16384
20502 185/300 Write block 997 len 28672 data: 5 5 5 5 5 3 3
20503 186/300 Read block 398 len 20480
20504 187/300 Write block 3344 len 24576 data: 7 6 5 5 3 3
20505 188/300 Read block 4822 len 16384
20506 189/300 Write block 1015 len 32768 data: 4 4 3 2 3 3 4 3
20507 190/300 Flush
20508 191/300 Read block 3313 len 4096
20509 192/300 Write block 3425 len 16384 data: 5 6 7 7
20510 193/300 Read block 2379 len 28672
20511 194/300 Write block 3622 len 40960 data: 6 6 5 5 4 5 6 6 6 5
20512 195/300 Read block 3813 len 16384
20513 196/300 Read block 2639 len 40960
20514 197/300 Read block 1680 len 4096
20515 198/300 Read block 3952 len 24576
20516 199/300 Read block 3207 len 4096
20517 200/300 Read block 1043 len 36864
20518 201/300 Write block 4109 len 36864 data: 4 3 3 2 2 2 4 3 3
20519 202/300 Write block 4657 len 36864 data: 5 6 6 5 5 5 3 3 3
20520 203/300 Read block 2232 len 20480
20521 204/300 Read block 1812 len 8192
20522 205/300 Read block 3203 len 32768
20523 206/300 Write block 1573 len 32768 data: 2 2 5 6 5 5 5 5
20524 207/300 Write block 4772 len 32768 data: 5 6 7 7 6 5 5 5
20525 208/300 Read block 3177 len 24576
20526 209/300 Read block 3235 len 40960
20527 210/300 Write block 4163 len 40960 data: 5 5 5 5 5 4 2 4 4 4
20528 211/300 Flush
20529 212/300 Read block 3823 len 36864
20530 213/300 Read block 3131 len 20480
20531 214/300 Read block 4438 len 40960
20532 215/300 Flush
20533 216/300 Read block 2822 len 12288
20534 217/300 Flush
20535 218/300 Flush
20536 219/300 Read block 2237 len 16384
20537 220/300 Flush
20538 221/300 Write block 4749 len 16384 data: 4 3 3 3
20539 222/300 Read block 3852 len 36864
20540 223/300 Read block 359 len 28672
20541 224/300 Flush
20542 225/300 Flush
20543 226/300 Write block 2715 len 12288 data: 7 7 8
20544 227/300 Write block 1835 len 28672 data: 5 4 4 5 5 4 4
20545 228/300 Read block 2169 len 40960
20546 229/300 Write block 2575 len 8192 data: 5 4
20547 230/300 Flush
20548 231/300 Read block 4888 len 40960
20549 232/300 Read block 2074 len 24576
20550 233/300 Read block 4138 len 28672
20551 234/300 Flush
20552 235/300 Read block 2140 len 24576
20553 236/300 Write block 3732 len 32768 data: 4 4 4 4 3 2 4 3
20554 237/300 Write block 4306 len 24576 data: 4 3 2 3 4 4
20555 238/300 Write block 3608 len 12288 data: 6 6 5
20556 239/300 Write block 4823 len 28672 data: 3 4 4 5 4 4 3
20557 240/300 Flush
20558 241/300 Write block 4312 len 8192 data: 4 5
20559 242/300 Read block 2228 len 8192
20560 243/300 Read block 1122 len 8192
20561 244/300 Read block 4978 len 36864
20562 245/300 Write block 3133 len 4096 data: 4
20563 246/300 Write block 3719 len 24576 data: 3 3 2 3 3 3
20564 247/300 Read block 1496 len 24576
20565 248/300 Write block 800 len 4096 data: 3
20566 249/300 Write block 563 len 28672 data: 5 5 4 5 2 2 2
20567 250/300 Read block 3480 len 12288
20568 251/300 Read block 3800 len 4096
20569 252/300 Write block 988 len 40960 data: 5 6 5 5 6 5 5 5 5 6
20570 253/300 Write block 1434 len 28672 data: 3 3 5 4 3 3 3
20571 254/300 Write block 2977 len 8192 data: 5 5
20572 255/300 Read block 2095 len 36864
20573 256/300 Read block 3339 len 12288
20574 257/300 Read block 4967 len 40960
20575 258/300 Write block 2853 len 4096 data: 4
20576 259/300 Write block 1200 len 12288 data: 3 3 3
20577 260/300 Read block 4678 len 36864
20578 261/300 Flush
20579 262/300 Read block 2451 len 24576
20580 263/300 Read block 3400 len 32768
20581 264/300 Read block 100 len 24576
20582 265/300 Write block 3033 len 4096 data: 6
20583 266/300 Write block 725 len 32768 data: 3 3 4 3 4 5 5 5
20584 267/300 Write block 1535 len 4096 data: 4
20585 268/300 Read block 1427 len 32768
20586 269/300 Read block 1569 len 12288
20587 270/300 Read block 3111 len 28672
20588 271/300 Flush
20589 272/300 Write block 999 len 24576 data: 6 6 6 4 4 3
20590 273/300 Write block 1472 len 40960 data: 4 2 3 3 5 5 5 6 6 6
20591 274/300 Read block 2233 len 32768
20592 275/300 Read block 4382 len 4096
20593 276/300 Flush
20594 277/300 Read block 4910 len 20480
20595 278/300 Write block 4119 len 12288 data: 6 6 7
20596 279/300 Read block 1605 len 28672
20597 280/300 Write block 2886 len 28672 data: 2 3 3 3 4 4 4
20598 281/300 Write block 4330 len 28672 data: 3 3 3 2 2 2 2
20599 282/300 Write block 3634 len 16384 data: 5 5 5 7
20600 283/300 Read block 2307 len 16384
20601 284/300 Read block 4765 len 16384
20602 285/300 Write block 102 len 24576 data: 5 6 6 6 6 6
20603 286/300 Read block 4249 len 4096
20604 287/300 Write block 2500 len 20480 data: 4 4 3 3 3
20605 288/300 Write block 3190 len 4096 data: 5
20606 289/300 Write block 1071 len 28672 data: 5 5 5 5 8 7 7
20607 290/300 Read block 4729 len 4096
20608 291/300 Read block 895 len 16384
20609 292/300 Read block 706 len 36864
20610 293/300 Write block 4372 len 36864 data: 2 2 3 3 4 5 5 5 5
20611 294/300 Read block 3631 len 16384
20612 295/300 Write block 4816 len 8192 data: 3 3
20613 296/300 Write block 835 len 36864 data: 4 3 3 2 2 3 3 3 3
20614 297/300 Read block 976 len 36864
20615 298/300 Write block 2535 len 16384 data: 4 4 4 4
20616 299/300 Read block 2994 len 4096
20617 300/300 Read block 1618 len 20480
20618 301/300 Read block 940 len 20480
20619 Main task has work to do, go find it
20620 got dsc Start(0)
20621 start 0
20622 Nov 28 04:54:39.646 INFO request completed, latency_us: 222, response_code: 204, uri: /start/cid/0, method: POST, req_id: 19445713-c1e5-4207-a341-ec5b4392f3a0, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
20623 Replay: started 0, returned:Ok(())
20624 ----------------------------------------------------------------
20625 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
20626 States: Offline Active Active
20627 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
20628 6434 Acked Write 4 Sent Done Done false
20629 6435 Acked Flush 0 Sent Done Done false
20630 6436 Acked Read 2 Sent Done Done false
20631 6437 Acked Read 4 Sent Done Done false
20632 6438 Acked Read 10 Sent Done Done false
20633 6439 Acked Read 9 Sent Done Done false
20634 6440 Acked Read 2 Sent Done Done false
20635 6441 Acked Read 9 Sent Done Done false
20636 6442 Acked Read 2 Sent Done Done false
20637 6443 Acked Write 6 Sent Done Done false
20638 6444 Acked Read 10 Sent Done Done false
20639 6445 Acked Write 1 Sent Done Done false
20640 6446 Acked Flush 0 Sent Done Done false
20641 6447 Acked Write 5 Sent Done Done false
20642 6448 Acked Read 5 Sent Done Done false
20643 6449 Acked Write 9 Sent Done Done false
20644 6450 Acked Read 6 Sent Done Done false
20645 6451 Acked Write 3 Sent Done Done false
20646 6452 Acked Write 8 Sent Done Done false
20647 6453 Acked Write 2 Sent Done Done false
20648 6454 Acked Read 10 Sent Done Done false
20649 6455 Acked Read 4 Sent Done Done false
20650 6456 Acked Read 8 Sent Done Done false
20651 6457 Acked Read 2 Sent Done Done false
20652 6458 Acked Read 4 Sent Done Done false
20653 6459 Acked Write 10 Sent Done Done false
20654 6460 Acked Write 7 Sent Done Done false
20655 6461 Acked Read 2 Sent Done Done false
20656 6462 Acked Read 3 Sent Done Done false
20657 6463 Acked Read 4 Sent Done Done false
20658 6464 Acked Read 3 Sent Done Done false
20659 6465 Acked Write 6 Sent Done Done false
20660 6466 Acked Read 8 Sent Done Done false
20661 6467 Acked Read 8 Sent Done Done false
20662 6468 Acked Read 2 Sent Done Done false
20663 6469 Acked Read 5 Sent Done Done false
20664 6470 Acked Read 6 Sent Done Done false
20665 6471 Acked Read 6 Sent Done Done false
20666 6472 Acked Flush 0 Sent Done Done false
20667 6473 Acked Read 1 Sent Done Done false
20668 6474 Acked Read 6 Sent Done Done false
20669 6475 Acked Write 2 Sent Done Done false
20670 6476 Acked Write 1 Sent Done Done false
20671 6477 Acked Read 10 Sent Done Done false
20672 6478 Acked Read 10 Sent Done Done false
20673 6479 Acked Write 8 Sent Done Done false
20674 6480 Acked Read 8 Sent Done Done false
20675 6481 Acked Write 6 Sent Done Done false
20676 6482 Acked Write 1 Sent Done Done false
20677 6483 Acked Read 1 Sent Done Done false
20678 6484 Acked Write 4 Sent Done Done false
20679 6485 Acked Read 4 Sent Done Done false
20680 6486 Acked Read 6 Sent Done Done false
20681 6487 Acked Read 8 Sent Done Done false
20682 6488 Acked Read 7 Sent Done Done false
20683 6489 Acked Read 9 Sent Done Done false
20684 6490 Acked Read 4 Sent Done Done false
20685 6491 Acked Flush 0 Sent Done Done false
20686 6492 Acked Write 8 Sent Done Done false
20687 6493 Acked Read 2 Sent Done Done false
20688 6494 Acked Read 7 Sent Done Done false
20689 6495 Acked Write 6 Sent Done Done false
20690 6496 Acked Read 3 Sent Done Done false
20691 6497 Acked Read 3 Sent Done Done false
20692 6498 Acked Write 9 Sent Done Done false
20693 6499 Acked Read 9 Sent Done Done false
20694 6500 Acked Write 7 Sent Done Done false
20695 6501 Acked Write 8 Sent Done Done false
20696 6502 Acked Read 5 Sent Done Done false
20697 6503 Acked Write 1 Sent Done Done false
20698 6504 Acked Read 8 Sent Done Done false
20699 6505 Acked Write 6 Sent Done Done false
20700 6506 Acked Read 9 Sent Done Done false
20701 6507 Acked Write 3 Sent Done Done false
20702 6508 Acked Write 10 Sent Done Done false
20703 6509 Acked Read 1 Sent Done Done false
20704 6510 Acked Write 8 Sent Done Done false
20705 6511 Acked Read 4 Sent Done Done false
20706 6512 Acked Write 9 Sent Done Done false
20707 6513 Acked Write 8 Sent Done Done false
20708 6514 Acked Write 1 Sent Done Done false
20709 6515 Acked Flush 0 Sent Done Done false
20710 6516 Acked Read 4 Sent Done Done false
20711 6517 Acked Read 5 Sent Done Done false
20712 6518 Acked Write 10 Sent Done Done false
20713 6519 Acked Write 9 Sent Done Done false
20714 6520 Acked Write 9 Sent Done Done false
20715 6521 Acked Read 10 Sent Done Done false
20716 6522 Acked Read 9 Sent Done Done false
20717 6523 Acked Write 2 Sent Done Done false
20718 6524 Acked Write 10 Sent Done Done false
20719 6525 Acked Write 10 Sent Done Done false
20720 6526 Acked Read 8 Sent Done Done false
20721 6527 Acked Read 3 Sent Done Done false
20722 6528 Acked Read 3 Sent Done Done false
20723 6529 Acked Write 5 Sent Done Done false
20724 6530 Acked Read 8 Sent Done Done false
20725 6531 Acked Write 6 Sent Done Done false
20726 6532 Acked Write 7 Sent Done Done false
20727 6533 Acked Write 2 Sent Done Done false
20728 6534 Acked Write 5 Sent Done Done false
20729 6535 Acked Write 7 Sent Done Done false
20730 6536 Acked Write 7 Sent Done Done false
20731 6537 Acked Write 7 Sent Done Done false
20732 6538 Acked Read 9 Sent Done Done false
20733 6539 Acked Read 6 Sent Done Done false
20734 6540 Acked Write 4 Sent Done Done false
20735 6541 Acked Write 3 Sent Done Done false
20736 6542 Acked Read 7 Sent Done Done false
20737 6543 Acked Write 1 Sent Done Done false
20738 6544 Acked Write 7 Sent Done Done false
20739 6545 Acked Read 8 Sent Done Done false
20740 6546 Acked Write 7 Sent Done Done false
20741 6547 Acked Read 9 Sent Done Done false
20742 6548 Acked Write 8 Sent Done Done false
20743 6549 Acked Flush 0 Sent Done Done false
20744 6550 Acked Read 4 Sent Done Done false
20745 6551 Acked Read 3 Sent Done Done false
20746 6552 Acked Write 4 Sent Done Done false
20747 6553 Acked Write 10 Sent Done Done false
20748 6554 Acked Flush 0 Sent Done Done false
20749 6555 Acked Read 8 Sent Done Done false
20750 6556 Acked Read 4 Sent Done Done false
20751 6557 Acked Flush 0 Sent Done Done false
20752 6558 Acked Read 7 Sent Done Done false
20753 6559 Acked Read 8 Sent Done Done false
20754 6560 Acked Read 9 Sent Done Done false
20755 6561 Acked Write 8 Sent Done Done false
20756 6562 Acked Read 9 Sent Done Done false
20757 6563 Acked Write 8 Sent Done Done false
20758 6564 Acked Flush 0 Sent Done Done false
20759 6565 Acked Read 10 Sent Done Done false
20760 6566 Acked Write 10 Sent Done Done false
20761 6567 Acked Write 4 Sent Done Done false
20762 6568 Acked Read 5 Sent Done Done false
20763 6569 Acked Read 10 Sent Done Done false
20764 6570 Acked Write 8 Sent Done Done false
20765 6571 Acked Write 3 Sent Done Done false
20766 6572 Acked Read 2 Sent Done Done false
20767 6573 Acked Write 8 Sent Done Done false
20768 6574 Acked Flush 0 Sent Done Done false
20769 6575 Acked Write 10 Sent Done Done false
20770 6576 Acked Write 3 Sent Done Done false
20771 6577 Acked Write 4 Sent Done Done false
20772 6578 Acked Read 7 Sent Done Done false
20773 6579 Acked Read 5 Sent Done Done false
20774 6580 Acked Write 4 Sent Done Done false
20775 6581 Acked Write 8 Sent Done Done false
20776 6582 Acked Write 10 Sent Done Done false
20777 6583 Acked Flush 0 Sent Done Done false
20778 6584 Acked Read 4 Sent Done Done false
20779 6585 Acked Write 2 Sent Done Done false
20780 6586 Acked Write 1 Sent Done Done false
20781 6587 Acked Read 10 Sent Done Done false
20782 6588 Acked Read 1 Sent Done Done false
20783 6589 Acked Read 4 Sent Done Done false
20784 6590 Acked Write 1 Sent Done Done false
20785 6591 Acked Read 1 Sent Done Done false
20786 6592 Acked Write 8 Sent Done Done false
20787 6593 Acked Write 3 Sent Done Done false
20788 6594 Acked Read 10 Sent Done Done false
20789 6595 Acked Write 10 Sent Done Done false
20790 6596 Acked Write 7 Sent Done Done false
20791 6597 Acked Read 8 Sent Done Done false
20792 6598 Acked Read 1 Sent Done Done false
20793 6599 Acked Write 1 Sent Done Done false
20794 6600 Acked Write 1 Sent Done Done false
20795 6601 Acked Flush 0 Sent Done Done false
20796 6602 Acked Read 6 Sent Done Done false
20797 6603 Acked Read 5 Sent Done Done false
20798 6604 Acked Write 7 Sent Done Done false
20799 6605 Acked Read 9 Sent Done Done false
20800 6606 Acked Write 8 Sent Done Done false
20801 6607 Acked Read 2 Sent Done Done false
20802 6608 Acked Write 3 Sent Done Done false
20803 6609 Acked Write 8 Sent Done Done false
20804 6610 Acked Read 10 Sent Done Done false
20805 6611 Acked Write 8 Sent Done Done false
20806 6612 Acked Flush 0 Sent Done Done false
20807 6613 Acked Read 5 Sent Done Done false
20808 6614 Acked Read 2 Sent Done Done false
20809 6615 Acked Write 9 Sent Done Done false
20810 6616 Acked Flush 0 Sent Done Done false
20811 6617 Acked Read 4 Sent Done Done false
20812 6618 Acked Write 7 Sent Done Done false
20813 6619 Acked Read 5 Sent Done Done false
20814 6620 Acked Write 6 Sent Done Done false
20815 6621 Acked Read 4 Sent Done Done false
20816 6622 Acked Write 8 Sent Done Done false
20817 6623 Acked Flush 0 Sent Done Done false
20818 6624 Acked Read 1 Sent Done Done false
20819 6625 Acked Write 4 Sent Done Done false
20820 6626 Acked Read 7 Sent Done Done false
20821 6627 Acked Write 10 Sent Done Done false
20822 6628 Acked Read 4 Sent Done Done false
20823 6629 Acked Read 10 Sent Done Done false
20824 6630 Acked Read 1 Sent Done Done false
20825 6631 Acked Read 6 Sent Done Done false
20826 6632 Acked Read 1 Sent Done Done false
20827 6633 Acked Read 9 Sent Done Done false
20828 6634 Acked Write 9 Sent Done Done false
20829 6635 Acked Write 9 Sent Done Done false
20830 6636 Acked Read 5 Sent Done Done false
20831 6637 Acked Read 2 Sent Done Done false
20832 6638 Acked Read 8 Sent Done Done false
20833 6639 Acked Write 8 Sent Done Done false
20834 6640 Acked Write 8 Sent Done Done false
20835 6641 Acked Read 6 Sent Done Done false
20836 6642 Acked Read 10 Sent Done Done false
20837 6643 Acked Write 10 Sent Done Done false
20838 6644 Acked Flush 0 Sent Done Done false
20839 6645 Acked Read 9 Sent Done Done false
20840 6646 Acked Read 5 Sent Done Done false
20841 6647 Acked Read 10 Sent Done Done false
20842 6648 Acked Flush 0 Sent Done Done false
20843 6649 Acked Read 3 Sent Done Done false
20844 6650 Acked Flush 0 Sent Done Done false
20845 6651 Acked Flush 0 Sent Done Done false
20846 6652 Acked Read 4 Sent Done Done false
20847 6653 Acked Flush 0 Sent Done Done false
20848 6654 Acked Write 4 Sent Done Done false
20849 6655 Acked Read 9 Sent Done Done false
20850 6656 Acked Read 7 Sent Done Done false
20851 6657 Acked Flush 0 Sent Done Done false
20852 6658 Acked Flush 0 Sent Done Done false
20853 6659 Acked Write 3 Sent Done Done false
20854 6660 Acked Write 7 Sent Done Done false
20855 6661 Acked Read 10 Sent Done Done false
20856 6662 Acked Write 2 Sent Done Done false
20857 6663 Acked Flush 0 Sent Done Done false
20858 6664 Acked Read 10 Sent Done Done false
20859 6665 Acked Read 6 Sent Done Done false
20860 6666 Acked Read 7 Sent Done Done false
20861 6667 Acked Flush 0 Sent Done Done false
20862 6668 Acked Read 6 Sent Done Done false
20863 6669 Acked Write 8 Sent Done Done false
20864 6670 Acked Write 6 Sent Done Done false
20865 6671 Acked Write 3 Sent Done Done false
20866 6672 Acked Write 7 Sent Done Done false
20867 6673 Acked Flush 0 Sent Done Done false
20868 6674 Acked Write 2 Sent Done Done false
20869 6675 Acked Read 2 Sent Done Done false
20870 6676 Acked Read 2 Sent Done Done false
20871 6677 Acked Read 9 Sent Done Done false
20872 6678 Acked Write 1 Sent Done Done false
20873 6679 Acked Write 6 Sent Done Done false
20874 6680 Acked Read 6 Sent Done Done false
20875 6681 Acked Write 1 Sent Done Done false
20876 6682 Acked Write 7 Sent Done Done false
20877 6683 Acked Read 3 Sent Done Done false
20878 6684 Acked Read 1 Sent Done Done false
20879 6685 Acked Write 10 Sent Done Done false
20880 6686 Acked Write 7 Sent Done Done false
20881 6687 Acked Write 2 Sent Done Done false
20882 6688 Acked Read 9 Sent Done Done false
20883 6689 Acked Read 3 Sent Done Done false
20884 6690 Acked Read 10 Sent Done Done false
20885 6691 Acked Write 1 Sent Done Done false
20886 6692 Acked Write 3 Sent Done Done false
20887 6693 Acked Read 9 Sent Done Done false
20888 6694 Acked Flush 0 Sent Done Done false
20889 6695 Acked Read 6 Sent Done Done false
20890 6696 Acked Read 8 Sent Done Done false
20891 6697 Acked Read 6 Sent Done Done false
20892 6698 Acked Write 1 Sent Done Done false
20893 6699 Acked Write 8 Sent Done Done false
20894 6700 Acked Write 1 Sent Done Done false
20895 6701 Acked Read 8 Sent Done Done false
20896 6702 Acked Read 3 Sent Done Done false
20897 6703 Acked Read 7 Sent Done Done false
20898 6704 Acked Flush 0 Sent Done Done false
20899 6705 Acked Write 6 Sent Done Done false
20900 6706 Acked Write 10 Sent Done Done false
20901 6707 Acked Read 8 Sent Done Done false
20902 6708 Acked Read 1 Sent Done Done false
20903 6709 Acked Flush 0 Sent Done Done false
20904 6710 Acked Read 5 Sent Done Done false
20905 6711 Acked Write 3 Sent Done Done false
20906 6712 Acked Read 7 Sent Done Done false
20907 6713 Acked Write 7 Sent Done Done false
20908 6714 Acked Write 7 Sent Done Done false
20909 6715 Acked Write 4 Sent Done Done false
20910 6716 Acked Read 4 Sent Done Done false
20911 6717 Acked Read 4 Sent Done Done false
20912 6718 Acked Write 6 Sent Done Done false
20913 6719 Acked Read 1 Sent Done Done false
20914 6720 Acked Write 5 Sent Done Done false
20915 6721 Acked Write 1 Sent Done Done false
20916 6722 Acked Write 7 Sent Done Done false
20917 6723 Acked Read 1 Sent Done Done false
20918 6724 Acked Read 4 Sent Done Done false
20919 6725 Acked Read 9 Sent Done Done false
20920 6726 Acked Write 9 Sent Done Done false
20921 6727 Acked Read 4 Sent Done Done false
20922 6728 Acked Write 2 Sent Done Done false
20923 6729 Acked Write 9 Sent Done Done false
20924 6730 Acked Read 9 Sent Done Done false
20925 6731 Acked Write 4 Sent Done Done false
20926 6732 Acked Read 1 Sent Done Done false
20927 6733 Acked Read 5 Sent Done Done false
20928 6734 Acked Read 5 Sent Done Done false
20929 STATES DS:0 DS:1 DS:2 TOTAL
20930 Sent 301 0 0 301
20931 Done 0 301 301 602
20932 Skipped 0 0 0 0
20933 Error 0 0 0 0
20934 Last Flush: 6433 6709 6709
20935 Downstairs last five completed: 6433 6432 6431 6430 6429
20936 Upstairs last five completed: 6734 6733 6732 6731 6730
20937 CLIENT: Up:0 ds:301 act:2
20938 ----------------------------------------------------------------
20939 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
20940 States: Offline Active Active
20941 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
20942 6434 Acked Write 4 Sent Done Done false
20943 6435 Acked Flush 0 Sent Done Done false
20944 6436 Acked Read 2 Sent Done Done false
20945 6437 Acked Read 4 Sent Done Done false
20946 6438 Acked Read 10 Sent Done Done false
20947 6439 Acked Read 9 Sent Done Done false
20948 6440 Acked Read 2 Sent Done Done false
20949 6441 Acked Read 9 Sent Done Done false
20950 6442 Acked Read 2 Sent Done Done false
20951 6443 Acked Write 6 Sent Done Done false
20952 6444 Acked Read 10 Sent Done Done false
20953 6445 Acked Write 1 Sent Done Done false
20954 6446 Acked Flush 0 Sent Done Done false
20955 6447 Acked Write 5 Sent Done Done false
20956 6448 Acked Read 5 Sent Done Done false
20957 6449 Acked Write 9 Sent Done Done false
20958 6450 Acked Read 6 Sent Done Done false
20959 6451 Acked Write 3 Sent Done Done false
20960 6452 Acked Write 8 Sent Done Done false
20961 6453 Acked Write 2 Sent Done Done false
20962 6454 Acked Read 10 Sent Done Done false
20963 6455 Acked Read 4 Sent Done Done false
20964 6456 Acked Read 8 Sent Done Done false
20965 6457 Acked Read 2 Sent Done Done false
20966 6458 Acked Read 4 Sent Done Done false
20967 6459 Acked Write 10 Sent Done Done false
20968 6460 Acked Write 7 Sent Done Done false
20969 6461 Acked Read 2 Sent Done Done false
20970 6462 Acked Read 3 Sent Done Done false
20971 6463 Acked Read 4 Sent Done Done false
20972 6464 Acked Read 3 Sent Done Done false
20973 6465 Acked Write 6 Sent Done Done false
20974 6466 Acked Read 8 Sent Done Done false
20975 6467 Acked Read 8 Sent Done Done false
20976 6468 Acked Read 2 Sent Done Done false
20977 6469 Acked Read 5 Sent Done Done false
20978 6470 Acked Read 6 Sent Done Done false
20979 6471 Acked Read 6 Sent Done Done false
20980 6472 Acked Flush 0 Sent Done Done false
20981 6473 Acked Read 1 Sent Done Done false
20982 6474 Acked Read 6 Sent Done Done false
20983 6475 Acked Write 2 Sent Done Done false
20984 6476 Acked Write 1 Sent Done Done false
20985 6477 Acked Read 10 Sent Done Done false
20986 6478 Acked Read 10 Sent Done Done false
20987 6479 Acked Write 8 Sent Done Done false
20988 6480 Acked Read 8 Sent Done Done false
20989 6481 Acked Write 6 Sent Done Done false
20990 6482 Acked Write 1 Sent Done Done false
20991 6483 Acked Read 1 Sent Done Done false
20992 6484 Acked Write 4 Sent Done Done false
20993 6485 Acked Read 4 Sent Done Done false
20994 6486 Acked Read 6 Sent Done Done false
20995 6487 Acked Read 8 Sent Done Done false
20996 6488 Acked Read 7 Sent Done Done false
20997 6489 Acked Read 9 Sent Done Done false
20998 6490 Acked Read 4 Sent Done Done false
20999 6491 Acked Flush 0 Sent Done Done false
21000 6492 Acked Write 8 Sent Done Done false
21001 6493 Acked Read 2 Sent Done Done false
21002 6494 Acked Read 7 Sent Done Done false
21003 6495 Acked Write 6 Sent Done Done false
21004 6496 Acked Read 3 Sent Done Done false
21005 6497 Acked Read 3 Sent Done Done false
21006 6498 Acked Write 9 Sent Done Done false
21007 6499 Acked Read 9 Sent Done Done false
21008 6500 Acked Write 7 Sent Done Done false
21009 6501 Acked Write 8 Sent Done Done false
21010 6502 Acked Read 5 Sent Done Done false
21011 6503 Acked Write 1 Sent Done Done false
21012 6504 Acked Read 8 Sent Done Done false
21013 6505 Acked Write 6 Sent Done Done false
21014 6506 Acked Read 9 Sent Done Done false
21015 6507 Acked Write 3 Sent Done Done false
21016 6508 Acked Write 10 Sent Done Done false
21017 6509 Acked Read 1 Sent Done Done false
21018 6510 Acked Write 8 Sent Done Done false
21019 6511 Acked Read 4 Sent Done Done false
21020 6512 Acked Write 9 Sent Done Done false
21021 6513 Acked Write 8 Sent Done Done false
21022 6514 Acked Write 1 Sent Done Done false
21023 6515 Acked Flush 0 Sent Done Done false
21024 6516 Acked Read 4 Sent Done Done false
21025 6517 Acked Read 5 Sent Done Done false
21026 6518 Acked Write 10 Sent Done Done false
21027 6519 Acked Write 9 Sent Done Done false
21028 6520 Acked Write 9 Sent Done Done false
21029 6521 Acked Read 10 Sent Done Done false
21030 6522 Acked Read 9 Sent Done Done false
21031 6523 Acked Write 2 Sent Done Done false
21032 6524 Acked Write 10 Sent Done Done false
21033 6525 Acked Write 10 Sent Done Done false
21034 6526 Acked Read 8 Sent Done Done false
21035 6527 Acked Read 3 Sent Done Done false
21036 6528 Acked Read 3 Sent Done Done false
21037 6529 Acked Write 5 Sent Done Done false
21038 6530 Acked Read 8 Sent Done Done false
21039 6531 Acked Write 6 Sent Done Done false
21040 6532 Acked Write 7 Sent Done Done false
21041 6533 Acked Write 2 Sent Done Done false
21042 6534 Acked Write 5 Sent Done Done false
21043 6535 Acked Write 7 Sent Done Done false
21044 6536 Acked Write 7 Sent Done Done false
21045 6537 Acked Write 7 Sent Done Done false
21046 6538 Acked Read 9 Sent Done Done false
21047 6539 Acked Read 6 Sent Done Done false
21048 6540 Acked Write 4 Sent Done Done false
21049 6541 Acked Write 3 Sent Done Done false
21050 6542 Acked Read 7 Sent Done Done false
21051 6543 Acked Write 1 Sent Done Done false
21052 6544 Acked Write 7 Sent Done Done false
21053 6545 Acked Read 8 Sent Done Done false
21054 6546 Acked Write 7 Sent Done Done false
21055 6547 Acked Read 9 Sent Done Done false
21056 6548 Acked Write 8 Sent Done Done false
21057 6549 Acked Flush 0 Sent Done Done false
21058 6550 Acked Read 4 Sent Done Done false
21059 6551 Acked Read 3 Sent Done Done false
21060 6552 Acked Write 4 Sent Done Done false
21061 6553 Acked Write 10 Sent Done Done false
21062 6554 Acked Flush 0 Sent Done Done false
21063 6555 Acked Read 8 Sent Done Done false
21064 6556 Acked Read 4 Sent Done Done false
21065 6557 Acked Flush 0 Sent Done Done false
21066 6558 Acked Read 7 Sent Done Done false
21067 6559 Acked Read 8 Sent Done Done false
21068 6560 Acked Read 9 Sent Done Done false
21069 6561 Acked Write 8 Sent Done Done false
21070 6562 Acked Read 9 Sent Done Done false
21071 6563 Acked Write 8 Sent Done Done false
21072 6564 Acked Flush 0 Sent Done Done false
21073 6565 Acked Read 10 Sent Done Done false
21074 6566 Acked Write 10 Sent Done Done false
21075 6567 Acked Write 4 Sent Done Done false
21076 6568 Acked Read 5 Sent Done Done false
21077 6569 Acked Read 10 Sent Done Done false
21078 6570 Acked Write 8 Sent Done Done false
21079 6571 Acked Write 3 Sent Done Done false
21080 6572 Acked Read 2 Sent Done Done false
21081 6573 Acked Write 8 Sent Done Done false
21082 6574 Acked Flush 0 Sent Done Done false
21083 6575 Acked Write 10 Sent Done Done false
21084 6576 Acked Write 3 Sent Done Done false
21085 6577 Acked Write 4 Sent Done Done false
21086 6578 Acked Read 7 Sent Done Done false
21087 6579 Acked Read 5 Sent Done Done false
21088 6580 Acked Write 4 Sent Done Done false
21089 6581 Acked Write 8 Sent Done Done false
21090 6582 Acked Write 10 Sent Done Done false
21091 6583 Acked Flush 0 Sent Done Done false
21092 6584 Acked Read 4 Sent Done Done false
21093 6585 Acked Write 2 Sent Done Done false
21094 6586 Acked Write 1 Sent Done Done false
21095 6587 Acked Read 10 Sent Done Done false
21096 6588 Acked Read 1 Sent Done Done false
21097 6589 Acked Read 4 Sent Done Done false
21098 6590 Acked Write 1 Sent Done Done false
21099 6591 Acked Read 1 Sent Done Done false
21100 6592 Acked Write 8 Sent Done Done false
21101 6593 Acked Write 3 Sent Done Done false
21102 6594 Acked Read 10 Sent Done Done false
21103 6595 Acked Write 10 Sent Done Done false
21104 6596 Acked Write 7 Sent Done Done false
21105 6597 Acked Read 8 Sent Done Done false
21106 6598 Acked Read 1 Sent Done Done false
21107 6599 Acked Write 1 Sent Done Done false
21108 6600 Acked Write 1 Sent Done Done false
21109 6601 Acked Flush 0 Sent Done Done false
21110 6602 Acked Read 6 Sent Done Done false
21111 6603 Acked Read 5 Sent Done Done false
21112 6604 Acked Write 7 Sent Done Done false
21113 6605 Acked Read 9 Sent Done Done false
21114 6606 Acked Write 8 Sent Done Done false
21115 6607 Acked Read 2 Sent Done Done false
21116 6608 Acked Write 3 Sent Done Done false
21117 6609 Acked Write 8 Sent Done Done false
21118 6610 Acked Read 10 Sent Done Done false
21119 6611 Acked Write 8 Sent Done Done false
21120 6612 Acked Flush 0 Sent Done Done false
21121 6613 Acked Read 5 Sent Done Done false
21122 6614 Acked Read 2 Sent Done Done false
21123 6615 Acked Write 9 Sent Done Done false
21124 6616 Acked Flush 0 Sent Done Done false
21125 6617 Acked Read 4 Sent Done Done false
21126 6618 Acked Write 7 Sent Done Done false
21127 6619 Acked Read 5 Sent Done Done false
21128 6620 Acked Write 6 Sent Done Done false
21129 6621 Acked Read 4 Sent Done Done false
21130 6622 Acked Write 8 Sent Done Done false
21131 6623 Acked Flush 0 Sent Done Done false
21132 6624 Acked Read 1 Sent Done Done false
21133 6625 Acked Write 4 Sent Done Done false
21134 6626 Acked Read 7 Sent Done Done false
21135 6627 Acked Write 10 Sent Done Done false
21136 6628 Acked Read 4 Sent Done Done false
21137 6629 Acked Read 10 Sent Done Done false
21138 6630 Acked Read 1 Sent Done Done false
21139 6631 Acked Read 6 Sent Done Done false
21140 6632 Acked Read 1 Sent Done Done false
21141 6633 Acked Read 9 Sent Done Done false
21142 6634 Acked Write 9 Sent Done Done false
21143 6635 Acked Write 9 Sent Done Done false
21144 6636 Acked Read 5 Sent Done Done false
21145 6637 Acked Read 2 Sent Done Done false
21146 6638 Acked Read 8 Sent Done Done false
21147 6639 Acked Write 8 Sent Done Done false
21148 6640 Acked Write 8 Sent Done Done false
21149 6641 Acked Read 6 Sent Done Done false
21150 6642 Acked Read 10 Sent Done Done false
21151 6643 Acked Write 10 Sent Done Done false
21152 6644 Acked Flush 0 Sent Done Done false
21153 6645 Acked Read 9 Sent Done Done false
21154 6646 Acked Read 5 Sent Done Done false
21155 6647 Acked Read 10 Sent Done Done false
21156 6648 Acked Flush 0 Sent Done Done false
21157 6649 Acked Read 3 Sent Done Done false
21158 6650 Acked Flush 0 Sent Done Done false
21159 6651 Acked Flush 0 Sent Done Done false
21160 6652 Acked Read 4 Sent Done Done false
21161 6653 Acked Flush 0 Sent Done Done false
21162 6654 Acked Write 4 Sent Done Done false
21163 6655 Acked Read 9 Sent Done Done false
21164 6656 Acked Read 7 Sent Done Done false
21165 6657 Acked Flush 0 Sent Done Done false
21166 6658 Acked Flush 0 Sent Done Done false
21167 6659 Acked Write 3 Sent Done Done false
21168 6660 Acked Write 7 Sent Done Done false
21169 6661 Acked Read 10 Sent Done Done false
21170 6662 Acked Write 2 Sent Done Done false
21171 6663 Acked Flush 0 Sent Done Done false
21172 6664 Acked Read 10 Sent Done Done false
21173 6665 Acked Read 6 Sent Done Done false
21174 6666 Acked Read 7 Sent Done Done false
21175 6667 Acked Flush 0 Sent Done Done false
21176 6668 Acked Read 6 Sent Done Done false
21177 6669 Acked Write 8 Sent Done Done false
21178 6670 Acked Write 6 Sent Done Done false
21179 6671 Acked Write 3 Sent Done Done false
21180 6672 Acked Write 7 Sent Done Done false
21181 6673 Acked Flush 0 Sent Done Done false
21182 6674 Acked Write 2 Sent Done Done false
21183 6675 Acked Read 2 Sent Done Done false
21184 6676 Acked Read 2 Sent Done Done false
21185 6677 Acked Read 9 Sent Done Done false
21186 6678 Acked Write 1 Sent Done Done false
21187 6679 Acked Write 6 Sent Done Done false
21188 6680 Acked Read 6 Sent Done Done false
21189 6681 Acked Write 1 Sent Done Done false
21190 6682 Acked Write 7 Sent Done Done false
21191 6683 Acked Read 3 Sent Done Done false
21192 6684 Acked Read 1 Sent Done Done false
21193 6685 Acked Write 10 Sent Done Done false
21194 6686 Acked Write 7 Sent Done Done false
21195 6687 Acked Write 2 Sent Done Done false
21196 6688 Acked Read 9 Sent Done Done false
21197 6689 Acked Read 3 Sent Done Done false
21198 6690 Acked Read 10 Sent Done Done false
21199 6691 Acked Write 1 Sent Done Done false
21200 6692 Acked Write 3 Sent Done Done false
21201 6693 Acked Read 9 Sent Done Done false
21202 6694 Acked Flush 0 Sent Done Done false
21203 6695 Acked Read 6 Sent Done Done false
21204 6696 Acked Read 8 Sent Done Done false
21205 6697 Acked Read 6 Sent Done Done false
21206 6698 Acked Write 1 Sent Done Done false
21207 6699 Acked Write 8 Sent Done Done false
21208 6700 Acked Write 1 Sent Done Done false
21209 6701 Acked Read 8 Sent Done Done false
21210 6702 Acked Read 3 Sent Done Done false
21211 6703 Acked Read 7 Sent Done Done false
21212 6704 Acked Flush 0 Sent Done Done false
21213 6705 Acked Write 6 Sent Done Done false
21214 6706 Acked Write 10 Sent Done Done false
21215 6707 Acked Read 8 Sent Done Done false
21216 6708 Acked Read 1 Sent Done Done false
21217 6709 Acked Flush 0 Sent Done Done false
21218 6710 Acked Read 5 Sent Done Done false
21219 6711 Acked Write 3 Sent Done Done false
21220 6712 Acked Read 7 Sent Done Done false
21221 6713 Acked Write 7 Sent Done Done false
21222 6714 Acked Write 7 Sent Done Done false
21223 6715 Acked Write 4 Sent Done Done false
21224 6716 Acked Read 4 Sent Done Done false
21225 6717 Acked Read 4 Sent Done Done false
21226 6718 Acked Write 6 Sent Done Done false
21227 6719 Acked Read 1 Sent Done Done false
21228 6720 Acked Write 5 Sent Done Done false
21229 6721 Acked Write 1 Sent Done Done false
21230 6722 Acked Write 7 Sent Done Done false
21231 6723 Acked Read 1 Sent Done Done false
21232 6724 Acked Read 4 Sent Done Done false
21233 6725 Acked Read 9 Sent Done Done false
21234 6726 Acked Write 9 Sent Done Done false
21235 6727 Acked Read 4 Sent Done Done false
21236 6728 Acked Write 2 Sent Done Done false
21237 6729 Acked Write 9 Sent Done Done false
21238 6730 Acked Read 9 Sent Done Done false
21239 6731 Acked Write 4 Sent Done Done false
21240 6732 Acked Read 1 Sent Done Done false
21241 6733 Acked Read 5 Sent Done Done false
21242 6734 Acked Read 5 Sent Done Done false
21243 6735 Acked Flush 0 Sent Done Done false
21244 STATES DS:0 DS:1 DS:2 TOTAL
21245 Sent 302 0 0 302
21246 Done 0 302 302 604
21247 Skipped 0 0 0 0
21248 Error 0 0 0 0
21249 Last Flush: 6433 6735 6735
21250 Downstairs last five completed: 6433 6432 6431 6430 6429
21251 Upstairs last five completed: 6735 6734 6733 6732 6731
21252 CLIENT: Up:0 ds:302 act:2
212532024-11-28T04:54:45.404ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
212542024-11-28T04:54:45.404ZWARNcrucible: ds_connection connect to 127.0.0.1:8810 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
212552024-11-28T04:54:45.404ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
212562024-11-28T04:54:45.404ZWARNcrucible: downstairs task for 0 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
212572024-11-28T04:54:45.404ZINFOcrucible: client task is exiting = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
21258 [8810] got start action
21259 [8810] I am going to restart
21260 Starting downstairs at port 8810
21261 Make output file at "/tmp/dsc/downstairs-8810.txt"
21262 [8810][0] reports Starting
21263 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1221)
21264 [8810][0] reports Running
21265 ----------------------------------------------------------------
21266 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
21267 States: Offline Active Active
21268 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
21269 6434 Acked Write 4 Sent Done Done false
21270 6435 Acked Flush 0 Sent Done Done false
21271 6436 Acked Read 2 Sent Done Done false
21272 6437 Acked Read 4 Sent Done Done false
21273 6438 Acked Read 10 Sent Done Done false
21274 6439 Acked Read 9 Sent Done Done false
21275 6440 Acked Read 2 Sent Done Done false
21276 6441 Acked Read 9 Sent Done Done false
21277 6442 Acked Read 2 Sent Done Done false
21278 6443 Acked Write 6 Sent Done Done false
21279 6444 Acked Read 10 Sent Done Done false
21280 6445 Acked Write 1 Sent Done Done false
21281 6446 Acked Flush 0 Sent Done Done false
21282 6447 Acked Write 5 Sent Done Done false
21283 6448 Acked Read 5 Sent Done Done false
21284 6449 Acked Write 9 Sent Done Done false
21285 6450 Acked Read 6 Sent Done Done false
21286 6451 Acked Write 3 Sent Done Done false
21287 6452 Acked Write 8 Sent Done Done false
21288 6453 Acked Write 2 Sent Done Done false
21289 6454 Acked Read 10 Sent Done Done false
21290 6455 Acked Read 4 Sent Done Done false
21291 6456 Acked Read 8 Sent Done Done false
21292 6457 Acked Read 2 Sent Done Done false
21293 6458 Acked Read 4 Sent Done Done false
21294 6459 Acked Write 10 Sent Done Done false
21295 6460 Acked Write 7 Sent Done Done false
21296 6461 Acked Read 2 Sent Done Done false
21297 6462 Acked Read 3 Sent Done Done false
21298 6463 Acked Read 4 Sent Done Done false
21299 6464 Acked Read 3 Sent Done Done false
21300 6465 Acked Write 6 Sent Done Done false
21301 6466 Acked Read 8 Sent Done Done false
21302 6467 Acked Read 8 Sent Done Done false
21303 6468 Acked Read 2 Sent Done Done false
21304 6469 Acked Read 5 Sent Done Done false
21305 6470 Acked Read 6 Sent Done Done false
21306 6471 Acked Read 6 Sent Done Done false
21307 6472 Acked Flush 0 Sent Done Done false
21308 6473 Acked Read 1 Sent Done Done false
21309 6474 Acked Read 6 Sent Done Done false
21310 6475 Acked Write 2 Sent Done Done false
21311 6476 Acked Write 1 Sent Done Done false
21312 6477 Acked Read 10 Sent Done Done false
21313 6478 Acked Read 10 Sent Done Done false
21314 6479 Acked Write 8 Sent Done Done false
21315 6480 Acked Read 8 Sent Done Done false
21316 6481 Acked Write 6 Sent Done Done false
21317 6482 Acked Write 1 Sent Done Done false
21318 6483 Acked Read 1 Sent Done Done false
21319 6484 Acked Write 4 Sent Done Done false
21320 6485 Acked Read 4 Sent Done Done false
21321 6486 Acked Read 6 Sent Done Done false
21322 6487 Acked Read 8 Sent Done Done false
21323 6488 Acked Read 7 Sent Done Done false
21324 6489 Acked Read 9 Sent Done Done false
21325 6490 Acked Read 4 Sent Done Done false
21326 6491 Acked Flush 0 Sent Done Done false
21327 6492 Acked Write 8 Sent Done Done false
21328 6493 Acked Read 2 Sent Done Done false
21329 6494 Acked Read 7 Sent Done Done false
21330 6495 Acked Write 6 Sent Done Done false
21331 6496 Acked Read 3 Sent Done Done false
21332 6497 Acked Read 3 Sent Done Done false
21333 6498 Acked Write 9 Sent Done Done false
21334 6499 Acked Read 9 Sent Done Done false
21335 6500 Acked Write 7 Sent Done Done false
21336 6501 Acked Write 8 Sent Done Done false
21337 6502 Acked Read 5 Sent Done Done false
21338 6503 Acked Write 1 Sent Done Done false
21339 6504 Acked Read 8 Sent Done Done false
21340 6505 Acked Write 6 Sent Done Done false
21341 6506 Acked Read 9 Sent Done Done false
21342 6507 Acked Write 3 Sent Done Done false
21343 6508 Acked Write 10 Sent Done Done false
21344 6509 Acked Read 1 Sent Done Done false
21345 6510 Acked Write 8 Sent Done Done false
21346 6511 Acked Read 4 Sent Done Done false
21347 6512 Acked Write 9 Sent Done Done false
21348 6513 Acked Write 8 Sent Done Done false
21349 6514 Acked Write 1 Sent Done Done false
21350 6515 Acked Flush 0 Sent Done Done false
21351 6516 Acked Read 4 Sent Done Done false
21352 6517 Acked Read 5 Sent Done Done false
21353 6518 Acked Write 10 Sent Done Done false
21354 6519 Acked Write 9 Sent Done Done false
21355 6520 Acked Write 9 Sent Done Done false
21356 6521 Acked Read 10 Sent Done Done false
21357 6522 Acked Read 9 Sent Done Done false
21358 6523 Acked Write 2 Sent Done Done false
21359 6524 Acked Write 10 Sent Done Done false
21360 6525 Acked Write 10 Sent Done Done false
21361 6526 Acked Read 8 Sent Done Done false
21362 6527 Acked Read 3 Sent Done Done false
21363 6528 Acked Read 3 Sent Done Done false
21364 6529 Acked Write 5 Sent Done Done false
21365 6530 Acked Read 8 Sent Done Done false
21366 6531 Acked Write 6 Sent Done Done false
21367 6532 Acked Write 7 Sent Done Done false
21368 6533 Acked Write 2 Sent Done Done false
21369 6534 Acked Write 5 Sent Done Done false
21370 6535 Acked Write 7 Sent Done Done false
21371 6536 Acked Write 7 Sent Done Done false
21372 6537 Acked Write 7 Sent Done Done false
21373 6538 Acked Read 9 Sent Done Done false
21374 6539 Acked Read 6 Sent Done Done false
21375 6540 Acked Write 4 Sent Done Done false
21376 6541 Acked Write 3 Sent Done Done false
21377 6542 Acked Read 7 Sent Done Done false
21378 6543 Acked Write 1 Sent Done Done false
21379 6544 Acked Write 7 Sent Done Done false
21380 6545 Acked Read 8 Sent Done Done false
21381 6546 Acked Write 7 Sent Done Done false
21382 6547 Acked Read 9 Sent Done Done false
21383 6548 Acked Write 8 Sent Done Done false
21384 6549 Acked Flush 0 Sent Done Done false
21385 6550 Acked Read 4 Sent Done Done false
21386 6551 Acked Read 3 Sent Done Done false
21387 6552 Acked Write 4 Sent Done Done false
21388 6553 Acked Write 10 Sent Done Done false
21389 6554 Acked Flush 0 Sent Done Done false
21390 6555 Acked Read 8 Sent Done Done false
21391 6556 Acked Read 4 Sent Done Done false
21392 6557 Acked Flush 0 Sent Done Done false
21393 6558 Acked Read 7 Sent Done Done false
21394 6559 Acked Read 8 Sent Done Done false
21395 6560 Acked Read 9 Sent Done Done false
21396 6561 Acked Write 8 Sent Done Done false
21397 6562 Acked Read 9 Sent Done Done false
21398 6563 Acked Write 8 Sent Done Done false
21399 6564 Acked Flush 0 Sent Done Done false
21400 6565 Acked Read 10 Sent Done Done false
21401 6566 Acked Write 10 Sent Done Done false
21402 6567 Acked Write 4 Sent Done Done false
21403 6568 Acked Read 5 Sent Done Done false
21404 6569 Acked Read 10 Sent Done Done false
21405 6570 Acked Write 8 Sent Done Done false
21406 6571 Acked Write 3 Sent Done Done false
21407 6572 Acked Read 2 Sent Done Done false
21408 6573 Acked Write 8 Sent Done Done false
21409 6574 Acked Flush 0 Sent Done Done false
21410 6575 Acked Write 10 Sent Done Done false
21411 6576 Acked Write 3 Sent Done Done false
21412 6577 Acked Write 4 Sent Done Done false
21413 6578 Acked Read 7 Sent Done Done false
21414 6579 Acked Read 5 Sent Done Done false
21415 6580 Acked Write 4 Sent Done Done false
21416 6581 Acked Write 8 Sent Done Done false
21417 6582 Acked Write 10 Sent Done Done false
21418 6583 Acked Flush 0 Sent Done Done false
21419 6584 Acked Read 4 Sent Done Done false
21420 6585 Acked Write 2 Sent Done Done false
21421 6586 Acked Write 1 Sent Done Done false
21422 6587 Acked Read 10 Sent Done Done false
21423 6588 Acked Read 1 Sent Done Done false
21424 6589 Acked Read 4 Sent Done Done false
21425 6590 Acked Write 1 Sent Done Done false
21426 6591 Acked Read 1 Sent Done Done false
21427 6592 Acked Write 8 Sent Done Done false
21428 6593 Acked Write 3 Sent Done Done false
21429 6594 Acked Read 10 Sent Done Done false
21430 6595 Acked Write 10 Sent Done Done false
21431 6596 Acked Write 7 Sent Done Done false
21432 6597 Acked Read 8 Sent Done Done false
21433 6598 Acked Read 1 Sent Done Done false
21434 6599 Acked Write 1 Sent Done Done false
21435 6600 Acked Write 1 Sent Done Done false
21436 6601 Acked Flush 0 Sent Done Done false
21437 6602 Acked Read 6 Sent Done Done false
21438 6603 Acked Read 5 Sent Done Done false
21439 6604 Acked Write 7 Sent Done Done false
21440 6605 Acked Read 9 Sent Done Done false
21441 6606 Acked Write 8 Sent Done Done false
21442 6607 Acked Read 2 Sent Done Done false
21443 6608 Acked Write 3 Sent Done Done false
21444 6609 Acked Write 8 Sent Done Done false
21445 6610 Acked Read 10 Sent Done Done false
21446 6611 Acked Write 8 Sent Done Done false
21447 6612 Acked Flush 0 Sent Done Done false
21448 6613 Acked Read 5 Sent Done Done false
21449 6614 Acked Read 2 Sent Done Done false
21450 6615 Acked Write 9 Sent Done Done false
21451 6616 Acked Flush 0 Sent Done Done false
21452 6617 Acked Read 4 Sent Done Done false
21453 6618 Acked Write 7 Sent Done Done false
21454 6619 Acked Read 5 Sent Done Done false
21455 6620 Acked Write 6 Sent Done Done false
21456 6621 Acked Read 4 Sent Done Done false
21457 6622 Acked Write 8 Sent Done Done false
21458 6623 Acked Flush 0 Sent Done Done false
21459 6624 Acked Read 1 Sent Done Done false
21460 6625 Acked Write 4 Sent Done Done false
21461 6626 Acked Read 7 Sent Done Done false
21462 6627 Acked Write 10 Sent Done Done false
21463 6628 Acked Read 4 Sent Done Done false
21464 6629 Acked Read 10 Sent Done Done false
21465 6630 Acked Read 1 Sent Done Done false
21466 6631 Acked Read 6 Sent Done Done false
21467 6632 Acked Read 1 Sent Done Done false
21468 6633 Acked Read 9 Sent Done Done false
21469 6634 Acked Write 9 Sent Done Done false
21470 6635 Acked Write 9 Sent Done Done false
21471 6636 Acked Read 5 Sent Done Done false
21472 6637 Acked Read 2 Sent Done Done false
21473 6638 Acked Read 8 Sent Done Done false
21474 6639 Acked Write 8 Sent Done Done false
21475 6640 Acked Write 8 Sent Done Done false
21476 6641 Acked Read 6 Sent Done Done false
21477 6642 Acked Read 10 Sent Done Done false
21478 6643 Acked Write 10 Sent Done Done false
21479 6644 Acked Flush 0 Sent Done Done false
21480 6645 Acked Read 9 Sent Done Done false
21481 6646 Acked Read 5 Sent Done Done false
21482 6647 Acked Read 10 Sent Done Done false
21483 6648 Acked Flush 0 Sent Done Done false
21484 6649 Acked Read 3 Sent Done Done false
21485 6650 Acked Flush 0 Sent Done Done false
21486 6651 Acked Flush 0 Sent Done Done false
21487 6652 Acked Read 4 Sent Done Done false
21488 6653 Acked Flush 0 Sent Done Done false
21489 6654 Acked Write 4 Sent Done Done false
21490 6655 Acked Read 9 Sent Done Done false
21491 6656 Acked Read 7 Sent Done Done false
21492 6657 Acked Flush 0 Sent Done Done false
21493 6658 Acked Flush 0 Sent Done Done false
21494 6659 Acked Write 3 Sent Done Done false
21495 6660 Acked Write 7 Sent Done Done false
21496 6661 Acked Read 10 Sent Done Done false
21497 6662 Acked Write 2 Sent Done Done false
21498 6663 Acked Flush 0 Sent Done Done false
21499 6664 Acked Read 10 Sent Done Done false
21500 6665 Acked Read 6 Sent Done Done false
21501 6666 Acked Read 7 Sent Done Done false
21502 6667 Acked Flush 0 Sent Done Done false
21503 6668 Acked Read 6 Sent Done Done false
21504 6669 Acked Write 8 Sent Done Done false
21505 6670 Acked Write 6 Sent Done Done false
21506 6671 Acked Write 3 Sent Done Done false
21507 6672 Acked Write 7 Sent Done Done false
21508 6673 Acked Flush 0 Sent Done Done false
21509 6674 Acked Write 2 Sent Done Done false
21510 6675 Acked Read 2 Sent Done Done false
21511 6676 Acked Read 2 Sent Done Done false
21512 6677 Acked Read 9 Sent Done Done false
21513 6678 Acked Write 1 Sent Done Done false
21514 6679 Acked Write 6 Sent Done Done false
21515 6680 Acked Read 6 Sent Done Done false
21516 6681 Acked Write 1 Sent Done Done false
21517 6682 Acked Write 7 Sent Done Done false
21518 6683 Acked Read 3 Sent Done Done false
21519 6684 Acked Read 1 Sent Done Done false
21520 6685 Acked Write 10 Sent Done Done false
21521 6686 Acked Write 7 Sent Done Done false
21522 6687 Acked Write 2 Sent Done Done false
21523 6688 Acked Read 9 Sent Done Done false
21524 6689 Acked Read 3 Sent Done Done false
21525 6690 Acked Read 10 Sent Done Done false
21526 6691 Acked Write 1 Sent Done Done false
21527 6692 Acked Write 3 Sent Done Done false
21528 6693 Acked Read 9 Sent Done Done false
21529 6694 Acked Flush 0 Sent Done Done false
21530 6695 Acked Read 6 Sent Done Done false
21531 6696 Acked Read 8 Sent Done Done false
21532 6697 Acked Read 6 Sent Done Done false
21533 6698 Acked Write 1 Sent Done Done false
21534 6699 Acked Write 8 Sent Done Done false
21535 6700 Acked Write 1 Sent Done Done false
21536 6701 Acked Read 8 Sent Done Done false
21537 6702 Acked Read 3 Sent Done Done false
21538 6703 Acked Read 7 Sent Done Done false
21539 6704 Acked Flush 0 Sent Done Done false
21540 6705 Acked Write 6 Sent Done Done false
21541 6706 Acked Write 10 Sent Done Done false
21542 6707 Acked Read 8 Sent Done Done false
21543 6708 Acked Read 1 Sent Done Done false
21544 6709 Acked Flush 0 Sent Done Done false
21545 6710 Acked Read 5 Sent Done Done false
21546 6711 Acked Write 3 Sent Done Done false
21547 6712 Acked Read 7 Sent Done Done false
21548 6713 Acked Write 7 Sent Done Done false
21549 6714 Acked Write 7 Sent Done Done false
21550 6715 Acked Write 4 Sent Done Done false
21551 6716 Acked Read 4 Sent Done Done false
21552 6717 Acked Read 4 Sent Done Done false
21553 6718 Acked Write 6 Sent Done Done false
21554 6719 Acked Read 1 Sent Done Done false
21555 6720 Acked Write 5 Sent Done Done false
21556 6721 Acked Write 1 Sent Done Done false
21557 6722 Acked Write 7 Sent Done Done false
21558 6723 Acked Read 1 Sent Done Done false
21559 6724 Acked Read 4 Sent Done Done false
21560 6725 Acked Read 9 Sent Done Done false
21561 6726 Acked Write 9 Sent Done Done false
21562 6727 Acked Read 4 Sent Done Done false
21563 6728 Acked Write 2 Sent Done Done false
21564 6729 Acked Write 9 Sent Done Done false
21565 6730 Acked Read 9 Sent Done Done false
21566 6731 Acked Write 4 Sent Done Done false
21567 6732 Acked Read 1 Sent Done Done false
21568 6733 Acked Read 5 Sent Done Done false
21569 6734 Acked Read 5 Sent Done Done false
21570 6735 Acked Flush 0 Sent Done Done false
21571 STATES DS:0 DS:1 DS:2 TOTAL
21572 Sent 302 0 0 302
21573 Done 0 302 302 604
21574 Skipped 0 0 0 0
21575 Error 0 0 0 0
21576 Last Flush: 6433 6735 6735
21577 Downstairs last five completed: 6433 6432 6431 6430 6429
21578 Upstairs last five completed: 6735 6734 6733 6732 6731
21579 CLIENT: Up:0 ds:302 act:2
21580 ----------------------------------------------------------------
21581 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
21582 States: Offline Active Active
21583 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
21584 6434 Acked Write 4 Sent Done Done false
21585 6435 Acked Flush 0 Sent Done Done false
21586 6436 Acked Read 2 Sent Done Done false
21587 6437 Acked Read 4 Sent Done Done false
21588 6438 Acked Read 10 Sent Done Done false
21589 6439 Acked Read 9 Sent Done Done false
21590 6440 Acked Read 2 Sent Done Done false
21591 6441 Acked Read 9 Sent Done Done false
21592 6442 Acked Read 2 Sent Done Done false
21593 6443 Acked Write 6 Sent Done Done false
21594 6444 Acked Read 10 Sent Done Done false
21595 6445 Acked Write 1 Sent Done Done false
21596 6446 Acked Flush 0 Sent Done Done false
21597 6447 Acked Write 5 Sent Done Done false
21598 6448 Acked Read 5 Sent Done Done false
21599 6449 Acked Write 9 Sent Done Done false
21600 6450 Acked Read 6 Sent Done Done false
21601 6451 Acked Write 3 Sent Done Done false
21602 6452 Acked Write 8 Sent Done Done false
21603 6453 Acked Write 2 Sent Done Done false
21604 6454 Acked Read 10 Sent Done Done false
21605 6455 Acked Read 4 Sent Done Done false
21606 6456 Acked Read 8 Sent Done Done false
21607 6457 Acked Read 2 Sent Done Done false
21608 6458 Acked Read 4 Sent Done Done false
21609 6459 Acked Write 10 Sent Done Done false
21610 6460 Acked Write 7 Sent Done Done false
21611 6461 Acked Read 2 Sent Done Done false
21612 6462 Acked Read 3 Sent Done Done false
21613 6463 Acked Read 4 Sent Done Done false
21614 6464 Acked Read 3 Sent Done Done false
21615 6465 Acked Write 6 Sent Done Done false
21616 6466 Acked Read 8 Sent Done Done false
21617 6467 Acked Read 8 Sent Done Done false
21618 6468 Acked Read 2 Sent Done Done false
21619 6469 Acked Read 5 Sent Done Done false
21620 6470 Acked Read 6 Sent Done Done false
21621 6471 Acked Read 6 Sent Done Done false
21622 6472 Acked Flush 0 Sent Done Done false
21623 6473 Acked Read 1 Sent Done Done false
21624 6474 Acked Read 6 Sent Done Done false
21625 6475 Acked Write 2 Sent Done Done false
21626 6476 Acked Write 1 Sent Done Done false
21627 6477 Acked Read 10 Sent Done Done false
21628 6478 Acked Read 10 Sent Done Done false
21629 6479 Acked Write 8 Sent Done Done false
21630 6480 Acked Read 8 Sent Done Done false
21631 6481 Acked Write 6 Sent Done Done false
21632 6482 Acked Write 1 Sent Done Done false
21633 6483 Acked Read 1 Sent Done Done false
21634 6484 Acked Write 4 Sent Done Done false
21635 6485 Acked Read 4 Sent Done Done false
21636 6486 Acked Read 6 Sent Done Done false
21637 6487 Acked Read 8 Sent Done Done false
21638 6488 Acked Read 7 Sent Done Done false
21639 6489 Acked Read 9 Sent Done Done false
21640 6490 Acked Read 4 Sent Done Done false
21641 6491 Acked Flush 0 Sent Done Done false
21642 6492 Acked Write 8 Sent Done Done false
21643 6493 Acked Read 2 Sent Done Done false
21644 6494 Acked Read 7 Sent Done Done false
21645 6495 Acked Write 6 Sent Done Done false
21646 6496 Acked Read 3 Sent Done Done false
21647 6497 Acked Read 3 Sent Done Done false
21648 6498 Acked Write 9 Sent Done Done false
21649 6499 Acked Read 9 Sent Done Done false
21650 6500 Acked Write 7 Sent Done Done false
21651 6501 Acked Write 8 Sent Done Done false
21652 6502 Acked Read 5 Sent Done Done false
21653 6503 Acked Write 1 Sent Done Done false
21654 6504 Acked Read 8 Sent Done Done false
21655 6505 Acked Write 6 Sent Done Done false
21656 6506 Acked Read 9 Sent Done Done false
21657 6507 Acked Write 3 Sent Done Done false
21658 6508 Acked Write 10 Sent Done Done false
21659 6509 Acked Read 1 Sent Done Done false
21660 6510 Acked Write 8 Sent Done Done false
21661 6511 Acked Read 4 Sent Done Done false
21662 6512 Acked Write 9 Sent Done Done false
21663 6513 Acked Write 8 Sent Done Done false
21664 6514 Acked Write 1 Sent Done Done false
21665 6515 Acked Flush 0 Sent Done Done false
21666 6516 Acked Read 4 Sent Done Done false
21667 6517 Acked Read 5 Sent Done Done false
21668 6518 Acked Write 10 Sent Done Done false
21669 6519 Acked Write 9 Sent Done Done false
21670 6520 Acked Write 9 Sent Done Done false
21671 6521 Acked Read 10 Sent Done Done false
21672 6522 Acked Read 9 Sent Done Done false
21673 6523 Acked Write 2 Sent Done Done false
21674 6524 Acked Write 10 Sent Done Done false
21675 6525 Acked Write 10 Sent Done Done false
21676 6526 Acked Read 8 Sent Done Done false
21677 6527 Acked Read 3 Sent Done Done false
21678 6528 Acked Read 3 Sent Done Done false
21679 6529 Acked Write 5 Sent Done Done false
21680 6530 Acked Read 8 Sent Done Done false
21681 6531 Acked Write 6 Sent Done Done false
21682 6532 Acked Write 7 Sent Done Done false
21683 6533 Acked Write 2 Sent Done Done false
21684 6534 Acked Write 5 Sent Done Done false
21685 6535 Acked Write 7 Sent Done Done false
21686 6536 Acked Write 7 Sent Done Done false
21687 6537 Acked Write 7 Sent Done Done false
21688 6538 Acked Read 9 Sent Done Done false
21689 6539 Acked Read 6 Sent Done Done false
21690 6540 Acked Write 4 Sent Done Done false
21691 6541 Acked Write 3 Sent Done Done false
21692 6542 Acked Read 7 Sent Done Done false
21693 6543 Acked Write 1 Sent Done Done false
21694 6544 Acked Write 7 Sent Done Done false
21695 6545 Acked Read 8 Sent Done Done false
21696 6546 Acked Write 7 Sent Done Done false
21697 6547 Acked Read 9 Sent Done Done false
21698 6548 Acked Write 8 Sent Done Done false
21699 6549 Acked Flush 0 Sent Done Done false
21700 6550 Acked Read 4 Sent Done Done false
21701 6551 Acked Read 3 Sent Done Done false
21702 6552 Acked Write 4 Sent Done Done false
21703 6553 Acked Write 10 Sent Done Done false
21704 6554 Acked Flush 0 Sent Done Done false
21705 6555 Acked Read 8 Sent Done Done false
21706 6556 Acked Read 4 Sent Done Done false
21707 6557 Acked Flush 0 Sent Done Done false
21708 6558 Acked Read 7 Sent Done Done false
21709 6559 Acked Read 8 Sent Done Done false
21710 6560 Acked Read 9 Sent Done Done false
21711 6561 Acked Write 8 Sent Done Done false
21712 6562 Acked Read 9 Sent Done Done false
21713 6563 Acked Write 8 Sent Done Done false
21714 6564 Acked Flush 0 Sent Done Done false
21715 6565 Acked Read 10 Sent Done Done false
21716 6566 Acked Write 10 Sent Done Done false
21717 6567 Acked Write 4 Sent Done Done false
21718 6568 Acked Read 5 Sent Done Done false
21719 6569 Acked Read 10 Sent Done Done false
21720 6570 Acked Write 8 Sent Done Done false
21721 6571 Acked Write 3 Sent Done Done false
21722 6572 Acked Read 2 Sent Done Done false
21723 6573 Acked Write 8 Sent Done Done false
21724 6574 Acked Flush 0 Sent Done Done false
21725 6575 Acked Write 10 Sent Done Done false
21726 6576 Acked Write 3 Sent Done Done false
21727 6577 Acked Write 4 Sent Done Done false
21728 6578 Acked Read 7 Sent Done Done false
21729 6579 Acked Read 5 Sent Done Done false
21730 6580 Acked Write 4 Sent Done Done false
21731 6581 Acked Write 8 Sent Done Done false
21732 6582 Acked Write 10 Sent Done Done false
21733 6583 Acked Flush 0 Sent Done Done false
21734 6584 Acked Read 4 Sent Done Done false
21735 6585 Acked Write 2 Sent Done Done false
21736 6586 Acked Write 1 Sent Done Done false
21737 6587 Acked Read 10 Sent Done Done false
21738 6588 Acked Read 1 Sent Done Done false
21739 6589 Acked Read 4 Sent Done Done false
21740 6590 Acked Write 1 Sent Done Done false
21741 6591 Acked Read 1 Sent Done Done false
21742 6592 Acked Write 8 Sent Done Done false
21743 6593 Acked Write 3 Sent Done Done false
21744 6594 Acked Read 10 Sent Done Done false
21745 6595 Acked Write 10 Sent Done Done false
21746 6596 Acked Write 7 Sent Done Done false
21747 6597 Acked Read 8 Sent Done Done false
21748 6598 Acked Read 1 Sent Done Done false
21749 6599 Acked Write 1 Sent Done Done false
21750 6600 Acked Write 1 Sent Done Done false
21751 6601 Acked Flush 0 Sent Done Done false
21752 6602 Acked Read 6 Sent Done Done false
21753 6603 Acked Read 5 Sent Done Done false
21754 6604 Acked Write 7 Sent Done Done false
21755 6605 Acked Read 9 Sent Done Done false
21756 6606 Acked Write 8 Sent Done Done false
21757 6607 Acked Read 2 Sent Done Done false
21758 6608 Acked Write 3 Sent Done Done false
21759 6609 Acked Write 8 Sent Done Done false
21760 6610 Acked Read 10 Sent Done Done false
21761 6611 Acked Write 8 Sent Done Done false
21762 6612 Acked Flush 0 Sent Done Done false
21763 6613 Acked Read 5 Sent Done Done false
21764 6614 Acked Read 2 Sent Done Done false
21765 6615 Acked Write 9 Sent Done Done false
21766 6616 Acked Flush 0 Sent Done Done false
21767 6617 Acked Read 4 Sent Done Done false
21768 6618 Acked Write 7 Sent Done Done false
21769 6619 Acked Read 5 Sent Done Done false
21770 6620 Acked Write 6 Sent Done Done false
21771 6621 Acked Read 4 Sent Done Done false
21772 6622 Acked Write 8 Sent Done Done false
21773 6623 Acked Flush 0 Sent Done Done false
21774 6624 Acked Read 1 Sent Done Done false
21775 6625 Acked Write 4 Sent Done Done false
21776 6626 Acked Read 7 Sent Done Done false
21777 6627 Acked Write 10 Sent Done Done false
21778 6628 Acked Read 4 Sent Done Done false
21779 6629 Acked Read 10 Sent Done Done false
21780 6630 Acked Read 1 Sent Done Done false
21781 6631 Acked Read 6 Sent Done Done false
21782 6632 Acked Read 1 Sent Done Done false
21783 6633 Acked Read 9 Sent Done Done false
21784 6634 Acked Write 9 Sent Done Done false
21785 6635 Acked Write 9 Sent Done Done false
21786 6636 Acked Read 5 Sent Done Done false
21787 6637 Acked Read 2 Sent Done Done false
21788 6638 Acked Read 8 Sent Done Done false
21789 6639 Acked Write 8 Sent Done Done false
21790 6640 Acked Write 8 Sent Done Done false
21791 6641 Acked Read 6 Sent Done Done false
21792 6642 Acked Read 10 Sent Done Done false
21793 6643 Acked Write 10 Sent Done Done false
21794 6644 Acked Flush 0 Sent Done Done false
21795 6645 Acked Read 9 Sent Done Done false
21796 6646 Acked Read 5 Sent Done Done false
21797 6647 Acked Read 10 Sent Done Done false
21798 6648 Acked Flush 0 Sent Done Done false
21799 6649 Acked Read 3 Sent Done Done false
21800 6650 Acked Flush 0 Sent Done Done false
21801 6651 Acked Flush 0 Sent Done Done false
21802 6652 Acked Read 4 Sent Done Done false
21803 6653 Acked Flush 0 Sent Done Done false
21804 6654 Acked Write 4 Sent Done Done false
21805 6655 Acked Read 9 Sent Done Done false
21806 6656 Acked Read 7 Sent Done Done false
21807 6657 Acked Flush 0 Sent Done Done false
21808 6658 Acked Flush 0 Sent Done Done false
21809 6659 Acked Write 3 Sent Done Done false
21810 6660 Acked Write 7 Sent Done Done false
21811 6661 Acked Read 10 Sent Done Done false
21812 6662 Acked Write 2 Sent Done Done false
21813 6663 Acked Flush 0 Sent Done Done false
21814 6664 Acked Read 10 Sent Done Done false
21815 6665 Acked Read 6 Sent Done Done false
21816 6666 Acked Read 7 Sent Done Done false
21817 6667 Acked Flush 0 Sent Done Done false
21818 6668 Acked Read 6 Sent Done Done false
21819 6669 Acked Write 8 Sent Done Done false
21820 6670 Acked Write 6 Sent Done Done false
21821 6671 Acked Write 3 Sent Done Done false
21822 6672 Acked Write 7 Sent Done Done false
21823 6673 Acked Flush 0 Sent Done Done false
21824 6674 Acked Write 2 Sent Done Done false
21825 6675 Acked Read 2 Sent Done Done false
21826 6676 Acked Read 2 Sent Done Done false
21827 6677 Acked Read 9 Sent Done Done false
21828 6678 Acked Write 1 Sent Done Done false
21829 6679 Acked Write 6 Sent Done Done false
21830 6680 Acked Read 6 Sent Done Done false
21831 6681 Acked Write 1 Sent Done Done false
21832 6682 Acked Write 7 Sent Done Done false
21833 6683 Acked Read 3 Sent Done Done false
21834 6684 Acked Read 1 Sent Done Done false
21835 6685 Acked Write 10 Sent Done Done false
21836 6686 Acked Write 7 Sent Done Done false
21837 6687 Acked Write 2 Sent Done Done false
21838 6688 Acked Read 9 Sent Done Done false
21839 6689 Acked Read 3 Sent Done Done false
21840 6690 Acked Read 10 Sent Done Done false
21841 6691 Acked Write 1 Sent Done Done false
21842 6692 Acked Write 3 Sent Done Done false
21843 6693 Acked Read 9 Sent Done Done false
21844 6694 Acked Flush 0 Sent Done Done false
21845 6695 Acked Read 6 Sent Done Done false
21846 6696 Acked Read 8 Sent Done Done false
21847 6697 Acked Read 6 Sent Done Done false
21848 6698 Acked Write 1 Sent Done Done false
21849 6699 Acked Write 8 Sent Done Done false
21850 6700 Acked Write 1 Sent Done Done false
21851 6701 Acked Read 8 Sent Done Done false
21852 6702 Acked Read 3 Sent Done Done false
21853 6703 Acked Read 7 Sent Done Done false
21854 6704 Acked Flush 0 Sent Done Done false
21855 6705 Acked Write 6 Sent Done Done false
21856 6706 Acked Write 10 Sent Done Done false
21857 6707 Acked Read 8 Sent Done Done false
21858 6708 Acked Read 1 Sent Done Done false
21859 6709 Acked Flush 0 Sent Done Done false
21860 6710 Acked Read 5 Sent Done Done false
21861 6711 Acked Write 3 Sent Done Done false
21862 6712 Acked Read 7 Sent Done Done false
21863 6713 Acked Write 7 Sent Done Done false
21864 6714 Acked Write 7 Sent Done Done false
21865 6715 Acked Write 4 Sent Done Done false
21866 6716 Acked Read 4 Sent Done Done false
21867 6717 Acked Read 4 Sent Done Done false
21868 6718 Acked Write 6 Sent Done Done false
21869 6719 Acked Read 1 Sent Done Done false
21870 6720 Acked Write 5 Sent Done Done false
21871 6721 Acked Write 1 Sent Done Done false
21872 6722 Acked Write 7 Sent Done Done false
21873 6723 Acked Read 1 Sent Done Done false
21874 6724 Acked Read 4 Sent Done Done false
21875 6725 Acked Read 9 Sent Done Done false
21876 6726 Acked Write 9 Sent Done Done false
21877 6727 Acked Read 4 Sent Done Done false
21878 6728 Acked Write 2 Sent Done Done false
21879 6729 Acked Write 9 Sent Done Done false
21880 6730 Acked Read 9 Sent Done Done false
21881 6731 Acked Write 4 Sent Done Done false
21882 6732 Acked Read 1 Sent Done Done false
21883 6733 Acked Read 5 Sent Done Done false
21884 6734 Acked Read 5 Sent Done Done false
21885 6735 Acked Flush 0 Sent Done Done false
21886 STATES DS:0 DS:1 DS:2 TOTAL
21887 Sent 302 0 0 302
21888 Done 0 302 302 604
21889 Skipped 0 0 0 0
21890 Error 0 0 0 0
21891 Last Flush: 6433 6735 6735
21892 Downstairs last five completed: 6433 6432 6431 6430 6429
21893 Upstairs last five completed: 6735 6734 6733 6732 6731
21894 CLIENT: Up:0 ds:302 act:2
218952024-11-28T04:54:55.406ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
218962024-11-28T04:54:55.406ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:63320) = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
218972024-11-28T04:54:55.407ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
218982024-11-28T04:54:55.408ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
218992024-11-28T04:54:55.408ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
219002024-11-28T04:54:55.408ZINFOcrucible: send last flush ID to this DS: 6433 = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
219012024-11-28T04:54:55.408ZINFOcrucible: Replied this last flush ID: 6433 = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
219022024-11-28T04:54:55.408ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
219032024-11-28T04:54:55.408ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
219042024-11-28T04:54:55.408ZINFOcrucible: [0] client re-new 302 jobs since flush 6433 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
219052024-11-28T04:54:55.408ZINFOcrucible: [0] Replayed 302 jobs since flush: 6433 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
21906 ----------------------------------------------------------------
21907 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
21908 Downstairs last five completed: 6735 6734 6733 6732 6731
21909 Upstairs last five completed: 6735 6734 6733 6732 6731
21910 CLIENT: Up:0 ds:0 act:3
21911 Replay: All jobs finished
21912 Main task has work to do, go find it
21913 got dsc Stop(0)
21914 stop 0
21915 [8810] Got stop action so:false kr:false
21916 Nov 28 04:54:55.654 INFO request completed, latency_us: 295, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 75bdd95b-160c-4146-9655-6f6152615a38, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
219172024-11-28T04:54:55.655ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
219182024-11-28T04:54:55.655ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
219192024-11-28T04:54:55.655ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
219202024-11-28T04:54:55.655ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
219212024-11-28T04:54:55.655ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
219222024-11-28T04:54:55.655ZINFOcrucible: client task is exiting = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
21923 Nov 28 04:54:55.655 INFO request completed, latency_us: 128, response_code: 200, uri: /state/cid/0, method: GET, req_id: 2dc1be25-ab00-45ca-89e5-bbcd920fc630, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
21924 [8810] Exited with: ExitStatus(unix_wait_status(9))
21925 [8810][0] reports Exit
21926 Nov 28 04:54:59.657 INFO request completed, latency_us: 229, response_code: 200, uri: /state/cid/0, method: GET, req_id: 869977a2-7464-4f86-b120-02ec57e13d91, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
21927 001/300 Write block 3138 len 8192 data: 6 6
21928 002/300 Write block 1922 len 16384 data: 4 3 2 2
21929 003/300 Write block 770 len 12288 data: 6 5 4
21930 004/300 Write block 473 len 16384 data: 6 7 7 5
21931 005/300 Write block 1371 len 4096 data: 2
21932 006/300 Flush
21933 007/300 Write block 2551 len 16384 data: 5 5 6 6
21934 008/300 Write block 969 len 12288 data: 5 6 6
21935 009/300 Read block 1665 len 28672
21936 010/300 Write block 1912 len 12288 data: 2 2 3
21937 011/300 Read block 3922 len 32768
21938 012/300 Read block 4979 len 20480
21939 013/300 Read block 2207 len 40960
21940 014/300 Write block 4920 len 36864 data: 4 3 2 5 5 5 4 6 5
21941 015/300 Read block 4668 len 32768
21942 016/300 Read block 3013 len 8192
21943 017/300 Read block 2582 len 32768
21944 018/300 Write block 4600 len 28672 data: 7 6 6 4 6 6 4
21945 019/300 Write block 4342 len 12288 data: 4 3 4
21946 020/300 Read block 1784 len 4096
21947 021/300 Write block 4113 len 16384 data: 3 3 5 4
21948 022/300 Write block 4932 len 4096 data: 6
21949 023/300 Read block 1884 len 36864
21950 024/300 Read block 3688 len 8192
21951 025/300 Write block 349 len 8192 data: 3 3
21952 026/300 Write block 1106 len 36864 data: 3 3 5 4 4 4 3 3 3
21953 027/300 Write block 2662 len 12288 data: 6 6 5
21954 028/300 Write block 4194 len 36864 data: 3 3 3 4 5 6 5 5 5
21955 029/300 Write block 2779 len 28672 data: 7 6 5 4 3 6 5
21956 030/300 Read block 800 len 20480
21957 031/300 Flush
21958 032/300 Read block 2424 len 24576
21959 033/300 Read block 4662 len 24576
21960 034/300 Flush
21961 035/300 Write block 3434 len 24576 data: 5 4 5 4 4 4
21962 036/300 Read block 327 len 20480
21963 037/300 Read block 1627 len 8192
21964 038/300 Write block 969 len 40960 data: 6 7 7 5 4 4 3 3 3 4
21965 039/300 Read block 18 len 20480
21966 040/300 Read block 1658 len 4096
21967 041/300 Flush
21968 042/300 Read block 3485 len 32768
21969 043/300 Read block 525 len 4096
21970 044/300 Write block 4334 len 32768 data: 3 3 3 2 3 3 4 3
21971 045/300 Read block 3198 len 4096
21972 046/300 Read block 1516 len 8192
21973 047/300 Read block 3062 len 12288
21974 048/300 Write block 2239 len 4096 data: 4
21975 049/300 Read block 2059 len 8192
21976 050/300 Flush
21977 051/300 Flush
21978 052/300 Write block 3727 len 12288 data: 3 3 2
21979 053/300 Read block 469 len 16384
21980 054/300 Read block 2837 len 12288
21981 055/300 Flush
21982 056/300 Write block 2170 len 16384 data: 6 5 6 6
21983 057/300 Write block 2690 len 28672 data: 7 5 4 3 4 4 6
21984 058/300 Write block 2451 len 28672 data: 4 4 4 4 4 4 3
21985 059/300 Write block 3412 len 8192 data: 4 5
21986 060/300 Write block 1540 len 4096 data: 5
21987 061/300 Write block 65 len 28672 data: 4 4 4 4 5 3 3
21988 062/300 Flush
21989 063/300 Read block 4830 len 4096
21990 064/300 Read block 2669 len 24576
21991 065/300 Read block 391 len 28672
21992 066/300 Read block 1540 len 12288
21993 067/300 Flush
21994 068/300 Write block 407 len 24576 data: 8 8 8 7 7 6
21995 069/300 Write block 3488 len 4096 data: 5
21996 070/300 Write block 3983 len 36864 data: 2 2 2 4 3 3 3 4 4
21997 071/300 Read block 4674 len 4096
21998 072/300 Read block 4967 len 28672
21999 073/300 Read block 2955 len 24576
22000 074/300 Read block 2693 len 40960
22001 075/300 Read block 4058 len 24576
22002 076/300 Write block 1910 len 32768 data: 3 3 3 3 4 3 3 3
22003 077/300 Read block 4806 len 16384
22004 078/300 Read block 846 len 40960
22005 079/300 Read block 1815 len 32768
22006 080/300 Flush
22007 081/300 Write block 3144 len 40960 data: 4 4 4 3 3 2 5 4 5 5
22008 082/300 Read block 1749 len 20480
22009 083/300 Read block 3723 len 20480
22010 084/300 Read block 873 len 8192
22011 085/300 Read block 4512 len 12288
22012 086/300 Write block 892 len 20480 data: 6 3 4 4 5
22013 087/300 Write block 3468 len 40960 data: 5 3 3 4 4 4 4 4 5 5
22014 088/300 Write block 4140 len 8192 data: 6 7
22015 089/300 Read block 3073 len 28672
22016 090/300 Write block 4019 len 40960 data: 3 3 3 2 2 2 3 4 4 5
22017 091/300 Write block 2593 len 8192 data: 2 3
22018 092/300 Write block 3826 len 20480 data: 3 4 3 3 4
22019 093/300 Write block 1545 len 40960 data: 6 5 5 5 7 5 4 4 3 4
22020 094/300 Read block 1745 len 36864
22021 095/300 Write block 1204 len 4096 data: 4
22022 096/300 Flush
22023 097/300 Read block 2507 len 40960
22024 098/300 Flush
22025 099/300 Read block 4096 len 8192
22026 100/300 Write block 2979 len 12288 data: 5 5 4
22027 101/300 Flush
22028 102/300 Read block 1198 len 24576
22029 103/300 Read block 2978 len 28672
22030 104/300 Write block 1849 len 36864 data: 5 4 5 5 5 4 4 4 4
22031 105/300 Write block 3555 len 24576 data: 4 5 5 4 5 5
22032 106/300 Read block 3516 len 20480
22033 107/300 Write block 4643 len 8192 data: 6 4
22034 108/300 Read block 2177 len 16384
22035 109/300 Write block 4196 len 32768 data: 4 5 6 7 6 6 6 5
22036 110/300 Write block 2755 len 20480 data: 5 5 6 5 5
22037 111/300 Write block 1721 len 28672 data: 5 4 4 4 5 3 2
22038 112/300 Read block 1541 len 28672
22039 113/300 Read block 4868 len 16384
22040 114/300 Read block 3715 len 16384
22041 115/300 Write block 4702 len 4096 data: 3
22042 116/300 Read block 1054 len 40960
22043 117/300 Write block 1425 len 8192 data: 5 4
22044 118/300 Read block 1914 len 8192
22045 119/300 Read block 3588 len 16384
22046 120/300 Read block 1760 len 36864
22047 121/300 Write block 2816 len 8192 data: 4 3
22048 122/300 Write block 4723 len 20480 data: 4 4 5 5 4
22049 123/300 Write block 4068 len 4096 data: 4
22050 124/300 Read block 485 len 40960
22051 125/300 Read block 4279 len 40960
22052 126/300 Read block 4702 len 16384
22053 127/300 Write block 3805 len 4096 data: 5
22054 128/300 Read block 2050 len 20480
22055 129/300 Write block 753 len 36864 data: 3 4 3 3 4 4 5 5 5
22056 130/300 Read block 2522 len 4096
22057 131/300 Read block 1397 len 16384
22058 132/300 Write block 1201 len 36864 data: 4 4 3 5 4 5 5 5 5
22059 133/300 Read block 3350 len 12288
22060 134/300 Read block 144 len 8192
22061 135/300 Write block 2035 len 32768 data: 5 5 4 3 3 2 2 2
22062 136/300 Write block 1423 len 40960 data: 5 5 6 5 6 6 7 6 5 4
22063 137/300 Write block 2319 len 36864 data: 6 7 6 4 4 4 5 5 5
22064 138/300 Read block 1761 len 8192
22065 139/300 Read block 826 len 36864
22066 140/300 Read block 2190 len 36864
22067 141/300 Write block 3685 len 28672 data: 3 4 4 5 5 5 6
22068 142/300 Read block 2268 len 24576
22069 143/300 Read block 690 len 16384
22070 144/300 Read block 748 len 32768
22071 145/300 Read block 4390 len 16384
22072 146/300 Write block 4711 len 32768 data: 4 3 2 3 3 2 2 2
22073 147/300 Write block 4376 len 28672 data: 5 6 6 6 6 5 4
22074 148/300 Write block 2426 len 16384 data: 4 3 2 3
22075 149/300 Read block 3197 len 24576
22076 150/300 Write block 3049 len 24576 data: 6 6 6 6 7 6
22077 151/300 Read block 2077 len 12288
22078 152/300 Write block 2404 len 8192 data: 4 3
22079 153/300 Read block 4577 len 28672
22080 154/300 Write block 1494 len 40960 data: 5 5 4 6 4 4 4 4 3 2
22081 155/300 Write block 4894 len 32768 data: 4 4 4 3 3 3 3 4
22082 156/300 Write block 1626 len 32768 data: 5 5 5 7 6 6 6 4
22083 157/300 Read block 1806 len 4096
22084 158/300 Read block 1000 len 12288
22085 159/300 Write block 171 len 8192 data: 3 3
22086 160/300 Read block 1923 len 28672
22087 161/300 Read block 2663 len 36864
22088 162/300 Write block 1751 len 20480 data: 6 5 5 5 6
22089 163/300 Write block 466 len 32768 data: 6 5 8 9 9 7 7 7
22090 164/300 Write block 3075 len 16384 data: 4 4 4 4
22091 165/300 Write block 2783 len 16384 data: 4 7 6 5
22092 166/300 Read block 3143 len 4096
22093 167/300 Read block 3355 len 28672
22094 168/300 Read block 1830 len 16384
22095 169/300 Read block 2538 len 20480
22096 170/300 Read block 4036 len 20480
22097 171/300 Read block 2695 len 28672
22098 172/300 Write block 3743 len 24576 data: 5 5 4 6 6 5
22099 173/300 Write block 4493 len 28672 data: 6 5 3 2 3 2 2
22100 174/300 Flush
22101 175/300 Read block 1718 len 20480
22102 176/300 Read block 1934 len 32768
22103 177/300 Read block 3998 len 12288
22104 178/300 Write block 3175 len 4096 data: 7
22105 179/300 Read block 4811 len 36864
22106 180/300 Write block 1440 len 12288 data: 4 3 3
22107 181/300 Write block 4190 len 24576 data: 4 3 3 3 4 4
22108 182/300 Write block 4183 len 32768 data: 5 4 4 3 4 4 3 5
22109 183/300 Write block 1273 len 24576 data: 8 8 7 6 6 5
22110 184/300 Read block 4681 len 4096
22111 185/300 Write block 4360 len 20480 data: 6 7 6 7 7
22112 186/300 Write block 1114 len 16384 data: 4 3 3 3
22113 187/300 Read block 3704 len 16384
22114 188/300 Write block 3579 len 32768 data: 5 6 6 5 4 4 4 3
22115 189/300 Write block 1341 len 40960 data: 6 7 6 6 4 3 2 3 3 3
22116 190/300 Flush
22117 191/300 Read block 4346 len 32768
22118 192/300 Write block 3808 len 40960 data: 4 4 5 6 6 5 4 5 5 4
22119 193/300 Write block 82 len 20480 data: 4 5 4 5 3
22120 194/300 Write block 4879 len 4096 data: 2
22121 195/300 Flush
22122 196/300 Write block 2453 len 12288 data: 5 5 5
22123 197/300 Read block 2907 len 8192
22124 198/300 Write block 1726 len 28672 data: 4 3 2 2 2 3 3
22125 199/300 Write block 4181 len 24576 data: 5 5 6 5 5 4
22126 200/300 Flush
22127 201/300 Write block 675 len 12288 data: 4 4 3
22128 202/300 Write block 680 len 24576 data: 2 2 2 2 4 5
22129 203/300 Write block 4708 len 36864 data: 3 3 3 5 4 3 4 4 3
22130 204/300 Flush
22131 205/300 Write block 741 len 8192 data: 2 2
22132 206/300 Flush
22133 207/300 Read block 2869 len 20480
22134 208/300 Read block 4341 len 16384
22135 209/300 Flush
22136 210/300 Write block 1497 len 28672 data: 7 5 5 5 5 4 3
22137 211/300 Flush
22138 212/300 Read block 2496 len 12288
22139 213/300 Write block 3104 len 28672 data: 2 3 2 2 2 2 2
22140 214/300 Write block 1320 len 4096 data: 3
22141 215/300 Read block 3048 len 4096
22142 216/300 Read block 778 len 16384
22143 217/300 Write block 4975 len 28672 data: 2 3 3 4 4 5 5
22144 218/300 Read block 4911 len 12288
22145 219/300 Read block 0 len 36864
22146 220/300 Read block 4806 len 8192
22147 221/300 Write block 4798 len 4096 data: 8
22148 222/300 Read block 2070 len 24576
22149 223/300 Flush
22150 224/300 Write block 1298 len 20480 data: 3 4 5 5 5
22151 225/300 Write block 2170 len 36864 data: 7 6 7 7 5 4 3 4 4
22152 226/300 Write block 1818 len 8192 data: 4 5
22153 227/300 Read block 1906 len 32768
22154 228/300 Flush
22155 229/300 Read block 3226 len 28672
22156 230/300 Read block 3565 len 8192
22157 231/300 Write block 3239 len 24576 data: 6 4 4 4 3 3
22158 232/300 Write block 1721 len 12288 data: 6 5 5
22159 233/300 Read block 2766 len 16384
22160 234/300 Write block 2987 len 8192 data: 4 4
22161 235/300 Read block 382 len 12288
22162 236/300 Read block 11 len 36864
22163 237/300 Read block 4825 len 36864
22164 238/300 Read block 1829 len 20480
22165 239/300 Write block 4182 len 36864 data: 6 7 6 6 5 5 5 4 6
22166 240/300 Read block 4347 len 12288
22167 241/300 Read block 4352 len 24576
22168 242/300 Write block 3054 len 8192 data: 7 5
22169 243/300 Write block 1082 len 16384 data: 6 5 5 4
22170 244/300 Write block 1672 len 40960 data: 4 3 5 5 4 4 4 4 2 2
22171 245/300 Write block 4075 len 40960 data: 6 6 7 8 9 8 9 7 6 8
22172 246/300 Write block 2161 len 24576 data: 5 5 5 3 3 4
22173 247/300 Read block 57 len 8192
22174 248/300 Write block 3509 len 32768 data: 7 7 7 6 5 5 3 5
22175 249/300 Read block 3619 len 20480
22176 250/300 Write block 2381 len 20480 data: 9 6 6 6 5
22177 251/300 Read block 1207 len 40960
22178 252/300 Write block 3442 len 24576 data: 4 3 2 3 3 5
22179 253/300 Read block 1606 len 24576
22180 254/300 Read block 3800 len 36864
22181 255/300 Write block 505 len 24576 data: 6 8 8 7 5 5
22182 256/300 Write block 2062 len 20480 data: 4 4 4 6 6
22183 257/300 Write block 3333 len 40960 data: 4 3 3 4 4 4 3 5 7 8
22184 258/300 Read block 3562 len 12288
22185 259/300 Flush
22186 260/300 Write block 3726 len 24576 data: 3 4 4 3 3 3
22187 261/300 Read block 4151 len 16384
22188 262/300 Read block 1683 len 4096
22189 263/300 Write block 4411 len 36864 data: 2 2 2 2 2 4 4 4 5
22190 264/300 Write block 1576 len 40960 data: 7 6 6 6 6 5 4 5 4 3
22191 265/300 Flush
22192 266/300 Read block 472 len 20480
22193 267/300 Flush
22194 268/300 Write block 2412 len 12288 data: 5 5 5
22195 269/300 Write block 4451 len 4096 data: 7
22196 270/300 Read block 4997 len 8192
22197 271/300 Write block 337 len 8192 data: 7 8
22198 272/300 Flush
22199 273/300 Write block 1356 len 12288 data: 3 4 4
22200 274/300 Write block 1800 len 12288 data: 2 3 3
22201 275/300 Read block 4459 len 16384
22202 276/300 Read block 3528 len 24576
22203 277/300 Write block 2551 len 32768 data: 6 6 7 7 6 7 8 6
22204 278/300 Write block 481 len 4096 data: 3
22205 279/300 Read block 2010 len 12288
22206 280/300 Write block 2483 len 24576 data: 3 4 4 3 2 2
22207 281/300 Read block 4050 len 12288
22208 282/300 Write block 4870 len 24576 data: 5 6 5 4 3 3
22209 283/300 Read block 1454 len 8192
22210 284/300 Flush
22211 285/300 Write block 4845 len 36864 data: 7 7 6 7 8 7 8 7 9
22212 286/300 Read block 294 len 32768
22213 287/300 Write block 4656 len 36864 data: 5 6 7 7 6 6 6 4 4
22214 288/300 Write block 2667 len 28672 data: 4 4 4 5 5 5 5
22215 289/300 Read block 2754 len 4096
22216 290/300 Read block 1639 len 24576
22217 291/300 Read block 779 len 36864
22218 292/300 Read block 3099 len 4096
22219 293/300 Write block 3184 len 4096 data: 2
22220 294/300 Flush
22221 295/300 Read block 2684 len 8192
22222 296/300 Flush
22223 297/300 Read block 3007 len 32768
22224 298/300 Read block 4427 len 12288
22225 299/300 Read block 3934 len 12288
22226 300/300 Read block 2809 len 20480
22227 301/300 Read block 4431 len 32768
22228 Main task has work to do, go find it
22229 got dsc Start(0)
22230 start 0
22231 Nov 28 04:54:59.876 INFO request completed, latency_us: 195, response_code: 204, uri: /start/cid/0, method: POST, req_id: 2bc67ebb-02aa-41b4-aa83-d6ae4fd34232, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
22232 Replay: started 0, returned:Ok(())
22233 ----------------------------------------------------------------
22234 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
22235 States: Offline Active Active
22236 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
22237 6736 Acked Write 2 Sent Done Done false
22238 6737 Acked Write 4 Sent Done Done false
22239 6738 Acked Write 3 Sent Done Done false
22240 6739 Acked Write 4 Sent Done Done false
22241 6740 Acked Write 1 Sent Done Done false
22242 6741 Acked Flush 0 Sent Done Done false
22243 6742 Acked Write 4 Sent Done Done false
22244 6743 Acked Write 3 Sent Done Done false
22245 6744 Acked Read 7 Sent Done Done false
22246 6745 Acked Write 3 Sent Done Done false
22247 6746 Acked Read 8 Sent Done Done false
22248 6747 Acked Read 5 Sent Done Done false
22249 6748 Acked Read 10 Sent Done Done false
22250 6749 Acked Write 9 Sent Done Done false
22251 6750 Acked Read 8 Sent Done Done false
22252 6751 Acked Read 2 Sent Done Done false
22253 6752 Acked Read 8 Sent Done Done false
22254 6753 Acked Write 7 Sent Done Done false
22255 6754 Acked Write 3 Sent Done Done false
22256 6755 Acked Read 1 Sent Done Done false
22257 6756 Acked Write 4 Sent Done Done false
22258 6757 Acked Write 1 Sent Done Done false
22259 6758 Acked Read 9 Sent Done Done false
22260 6759 Acked Read 2 Sent Done Done false
22261 6760 Acked Write 2 Sent Done Done false
22262 6761 Acked Write 9 Sent Done Done false
22263 6762 Acked Write 3 Sent Done Done false
22264 6763 Acked Write 9 Sent Done Done false
22265 6764 Acked Write 7 Sent Done Done false
22266 6765 Acked Read 5 Sent Done Done false
22267 6766 Acked Flush 0 Sent Done Done false
22268 6767 Acked Read 6 Sent Done Done false
22269 6768 Acked Read 6 Sent Done Done false
22270 6769 Acked Flush 0 Sent Done Done false
22271 6770 Acked Write 6 Sent Done Done false
22272 6771 Acked Read 5 Sent Done Done false
22273 6772 Acked Read 2 Sent Done Done false
22274 6773 Acked Write 10 Sent Done Done false
22275 6774 Acked Read 5 Sent Done Done false
22276 6775 Acked Read 1 Sent Done Done false
22277 6776 Acked Flush 0 Sent Done Done false
22278 6777 Acked Read 8 Sent Done Done false
22279 6778 Acked Read 1 Sent Done Done false
22280 6779 Acked Write 8 Sent Done Done false
22281 6780 Acked Read 1 Sent Done Done false
22282 6781 Acked Read 2 Sent Done Done false
22283 6782 Acked Read 3 Sent Done Done false
22284 6783 Acked Write 1 Sent Done Done false
22285 6784 Acked Read 2 Sent Done Done false
22286 6785 Acked Flush 0 Sent Done Done false
22287 6786 Acked Flush 0 Sent Done Done false
22288 6787 Acked Write 3 Sent Done Done false
22289 6788 Acked Read 4 Sent Done Done false
22290 6789 Acked Read 3 Sent Done Done false
22291 6790 Acked Flush 0 Sent Done Done false
22292 6791 Acked Write 4 Sent Done Done false
22293 6792 Acked Write 7 Sent Done Done false
22294 6793 Acked Write 7 Sent Done Done false
22295 6794 Acked Write 2 Sent Done Done false
22296 6795 Acked Write 1 Sent Done Done false
22297 6796 Acked Write 7 Sent Done Done false
22298 6797 Acked Flush 0 Sent Done Done false
22299 6798 Acked Read 1 Sent Done Done false
22300 6799 Acked Read 6 Sent Done Done false
22301 6800 Acked Read 7 Sent Done Done false
22302 6801 Acked Read 3 Sent Done Done false
22303 6802 Acked Flush 0 Sent Done Done false
22304 6803 Acked Write 6 Sent Done Done false
22305 6804 Acked Write 1 Sent Done Done false
22306 6805 Acked Write 9 Sent Done Done false
22307 6806 Acked Read 1 Sent Done Done false
22308 6807 Acked Read 7 Sent Done Done false
22309 6808 Acked Read 6 Sent Done Done false
22310 6809 Acked Read 10 Sent Done Done false
22311 6810 Acked Read 6 Sent Done Done false
22312 6811 Acked Write 8 Sent Done Done false
22313 6812 Acked Read 4 Sent Done Done false
22314 6813 Acked Read 10 Sent Done Done false
22315 6814 Acked Read 8 Sent Done Done false
22316 6815 Acked Flush 0 Sent Done Done false
22317 6816 Acked Write 10 Sent Done Done false
22318 6817 Acked Read 5 Sent Done Done false
22319 6818 Acked Read 5 Sent Done Done false
22320 6819 Acked Read 2 Sent Done Done false
22321 6820 Acked Read 3 Sent Done Done false
22322 6821 Acked Write 5 Sent Done Done false
22323 6822 Acked Write 10 Sent Done Done false
22324 6823 Acked Write 2 Sent Done Done false
22325 6824 Acked Read 7 Sent Done Done false
22326 6825 Acked Write 10 Sent Done Done false
22327 6826 Acked Write 2 Sent Done Done false
22328 6827 Acked Write 5 Sent Done Done false
22329 6828 Acked Write 10 Sent Done Done false
22330 6829 Acked Read 9 Sent Done Done false
22331 6830 Acked Write 1 Sent Done Done false
22332 6831 Acked Flush 0 Sent Done Done false
22333 6832 Acked Read 10 Sent Done Done false
22334 6833 Acked Flush 0 Sent Done Done false
22335 6834 Acked Read 2 Sent Done Done false
22336 6835 Acked Write 3 Sent Done Done false
22337 6836 Acked Flush 0 Sent Done Done false
22338 6837 Acked Read 6 Sent Done Done false
22339 6838 Acked Read 7 Sent Done Done false
22340 6839 Acked Write 9 Sent Done Done false
22341 6840 Acked Write 6 Sent Done Done false
22342 6841 Acked Read 5 Sent Done Done false
22343 6842 Acked Write 2 Sent Done Done false
22344 6843 Acked Read 4 Sent Done Done false
22345 6844 Acked Write 8 Sent Done Done false
22346 6845 Acked Write 5 Sent Done Done false
22347 6846 Acked Write 7 Sent Done Done false
22348 6847 Acked Read 7 Sent Done Done false
22349 6848 Acked Read 4 Sent Done Done false
22350 6849 Acked Read 4 Sent Done Done false
22351 6850 Acked Write 1 Sent Done Done false
22352 6851 Acked Read 10 Sent Done Done false
22353 6852 Acked Write 2 Sent Done Done false
22354 6853 Acked Read 2 Sent Done Done false
22355 6854 Acked Read 4 Sent Done Done false
22356 6855 Acked Read 9 Sent Done Done false
22357 6856 Acked Write 2 Sent Done Done false
22358 6857 Acked Write 5 Sent Done Done false
22359 6858 Acked Write 1 Sent Done Done false
22360 6859 Acked Read 10 Sent Done Done false
22361 6860 Acked Read 10 Sent Done Done false
22362 6861 Acked Read 4 Sent Done Done false
22363 6862 Acked Write 1 Sent Done Done false
22364 6863 Acked Read 5 Sent Done Done false
22365 6864 Acked Write 9 Sent Done Done false
22366 6865 Acked Read 1 Sent Done Done false
22367 6866 Acked Read 4 Sent Done Done false
22368 6867 Acked Write 9 Sent Done Done false
22369 6868 Acked Read 3 Sent Done Done false
22370 6869 Acked Read 2 Sent Done Done false
22371 6870 Acked Write 8 Sent Done Done false
22372 6871 Acked Write 10 Sent Done Done false
22373 6872 Acked Write 9 Sent Done Done false
22374 6873 Acked Read 2 Sent Done Done false
22375 6874 Acked Read 9 Sent Done Done false
22376 6875 Acked Read 9 Sent Done Done false
22377 6876 Acked Write 7 Sent Done Done false
22378 6877 Acked Read 6 Sent Done Done false
22379 6878 Acked Read 4 Sent Done Done false
22380 6879 Acked Read 8 Sent Done Done false
22381 6880 Acked Read 4 Sent Done Done false
22382 6881 Acked Write 8 Sent Done Done false
22383 6882 Acked Write 7 Sent Done Done false
22384 6883 Acked Write 4 Sent Done Done false
22385 6884 Acked Read 6 Sent Done Done false
22386 6885 Acked Write 6 Sent Done Done false
22387 6886 Acked Read 3 Sent Done Done false
22388 6887 Acked Write 2 Sent Done Done false
22389 6888 Acked Read 7 Sent Done Done false
22390 6889 Acked Write 10 Sent Done Done false
22391 6890 Acked Write 8 Sent Done Done false
22392 6891 Acked Write 8 Sent Done Done false
22393 6892 Acked Read 1 Sent Done Done false
22394 6893 Acked Read 3 Sent Done Done false
22395 6894 Acked Write 2 Sent Done Done false
22396 6895 Acked Read 7 Sent Done Done false
22397 6896 Acked Read 9 Sent Done Done false
22398 6897 Acked Write 5 Sent Done Done false
22399 6898 Acked Write 8 Sent Done Done false
22400 6899 Acked Write 4 Sent Done Done false
22401 6900 Acked Write 4 Sent Done Done false
22402 6901 Acked Read 1 Sent Done Done false
22403 6902 Acked Read 7 Sent Done Done false
22404 6903 Acked Read 4 Sent Done Done false
22405 6904 Acked Read 5 Sent Done Done false
22406 6905 Acked Read 5 Sent Done Done false
22407 6906 Acked Read 7 Sent Done Done false
22408 6907 Acked Write 6 Sent Done Done false
22409 6908 Acked Write 7 Sent Done Done false
22410 6909 Acked Flush 0 Sent Done Done false
22411 6910 Acked Read 5 Sent Done Done false
22412 6911 Acked Read 8 Sent Done Done false
22413 6912 Acked Read 3 Sent Done Done false
22414 6913 Acked Write 1 Sent Done Done false
22415 6914 Acked Read 9 Sent Done Done false
22416 6915 Acked Write 3 Sent Done Done false
22417 6916 Acked Write 6 Sent Done Done false
22418 6917 Acked Write 8 Sent Done Done false
22419 6918 Acked Write 6 Sent Done Done false
22420 6919 Acked Read 1 Sent Done Done false
22421 6920 Acked Write 5 Sent Done Done false
22422 6921 Acked Write 4 Sent Done Done false
22423 6922 Acked Read 4 Sent Done Done false
22424 6923 Acked Write 8 Sent Done Done false
22425 6924 Acked Write 10 Sent Done Done false
22426 6925 Acked Flush 0 Sent Done Done false
22427 6926 Acked Read 8 Sent Done Done false
22428 6927 Acked Write 10 Sent Done Done false
22429 6928 Acked Write 5 Sent Done Done false
22430 6929 Acked Write 1 Sent Done Done false
22431 6930 Acked Flush 0 Sent Done Done false
22432 6931 Acked Write 3 Sent Done Done false
22433 6932 Acked Read 2 Sent Done Done false
22434 6933 Acked Write 7 Sent Done Done false
22435 6934 Acked Write 6 Sent Done Done false
22436 6935 Acked Flush 0 Sent Done Done false
22437 6936 Acked Write 3 Sent Done Done false
22438 6937 Acked Write 6 Sent Done Done false
22439 6938 Acked Write 9 Sent Done Done false
22440 6939 Acked Flush 0 Sent Done Done false
22441 6940 Acked Write 2 Sent Done Done false
22442 6941 Acked Flush 0 Sent Done Done false
22443 6942 Acked Read 5 Sent Done Done false
22444 6943 Acked Read 4 Sent Done Done false
22445 6944 Acked Flush 0 Sent Done Done false
22446 6945 Acked Write 7 Sent Done Done false
22447 6946 Acked Flush 0 Sent Done Done false
22448 6947 Acked Read 3 Sent Done Done false
22449 6948 Acked Write 7 Sent Done Done false
22450 6949 Acked Write 1 Sent Done Done false
22451 6950 Acked Read 1 Sent Done Done false
22452 6951 Acked Read 4 Sent Done Done false
22453 6952 Acked Write 7 Sent Done Done false
22454 6953 Acked Read 3 Sent Done Done false
22455 6954 Acked Read 9 Sent Done Done false
22456 6955 Acked Read 2 Sent Done Done false
22457 6956 Acked Write 1 Sent Done Done false
22458 6957 Acked Read 6 Sent Done Done false
22459 6958 Acked Flush 0 Sent Done Done false
22460 6959 Acked Write 5 Sent Done Done false
22461 6960 Acked Write 9 Sent Done Done false
22462 6961 Acked Write 2 Sent Done Done false
22463 6962 Acked Read 8 Sent Done Done false
22464 6963 Acked Flush 0 Sent Done Done false
22465 6964 Acked Read 7 Sent Done Done false
22466 6965 Acked Read 2 Sent Done Done false
22467 6966 Acked Write 6 Sent Done Done false
22468 6967 Acked Write 3 Sent Done Done false
22469 6968 Acked Read 4 Sent Done Done false
22470 6969 Acked Write 2 Sent Done Done false
22471 6970 Acked Read 3 Sent Done Done false
22472 6971 Acked Read 9 Sent Done Done false
22473 6972 Acked Read 9 Sent Done Done false
22474 6973 Acked Read 5 Sent Done Done false
22475 6974 Acked Write 9 Sent Done Done false
22476 6975 Acked Read 3 Sent Done Done false
22477 6976 Acked Read 6 Sent Done Done false
22478 6977 Acked Write 2 Sent Done Done false
22479 6978 Acked Write 4 Sent Done Done false
22480 6979 Acked Write 10 Sent Done Done false
22481 6980 Acked Write 10 Sent Done Done false
22482 6981 Acked Write 6 Sent Done Done false
22483 6982 Acked Read 2 Sent Done Done false
22484 6983 Acked Write 8 Sent Done Done false
22485 6984 Acked Read 5 Sent Done Done false
22486 6985 Acked Write 5 Sent Done Done false
22487 6986 Acked Read 10 Sent Done Done false
22488 6987 Acked Write 6 Sent Done Done false
22489 6988 Acked Read 6 Sent Done Done false
22490 6989 Acked Read 9 Sent Done Done false
22491 6990 Acked Write 6 Sent Done Done false
22492 6991 Acked Write 5 Sent Done Done false
22493 6992 Acked Write 10 Sent Done Done false
22494 6993 Acked Read 3 Sent Done Done false
22495 6994 Acked Flush 0 Sent Done Done false
22496 6995 Acked Write 6 Sent Done Done false
22497 6996 Acked Read 4 Sent Done Done false
22498 6997 Acked Read 1 Sent Done Done false
22499 6998 Acked Write 9 Sent Done Done false
22500 6999 Acked Write 10 Sent Done Done false
22501 7000 Acked Flush 0 Sent Done Done false
22502 7001 Acked Read 5 Sent Done Done false
22503 7002 Acked Flush 0 Sent Done Done false
22504 7003 Acked Write 3 Sent Done Done false
22505 7004 Acked Write 1 Sent Done Done false
22506 7005 Acked Read 2 Sent Done Done false
22507 7006 Acked Write 2 Sent Done Done false
22508 7007 Acked Flush 0 Sent Done Done false
22509 7008 Acked Write 3 Sent Done Done false
22510 7009 Acked Write 3 Sent Done Done false
22511 7010 Acked Read 4 Sent Done Done false
22512 7011 Acked Read 6 Sent Done Done false
22513 7012 Acked Write 8 Sent Done Done false
22514 7013 Acked Write 1 Sent Done Done false
22515 7014 Acked Read 3 Sent Done Done false
22516 7015 Acked Write 6 Sent Done Done false
22517 7016 Acked Read 3 Sent Done Done false
22518 7017 Acked Write 6 Sent Done Done false
22519 7018 Acked Read 2 Sent Done Done false
22520 7019 Acked Flush 0 Sent Done Done false
22521 7020 Acked Write 9 Sent Done Done false
22522 7021 Acked Read 8 Sent Done Done false
22523 7022 Acked Write 9 Sent Done Done false
22524 7023 Acked Write 7 Sent Done Done false
22525 7024 Acked Read 1 Sent Done Done false
22526 7025 Acked Read 6 Sent Done Done false
22527 7026 Acked Read 9 Sent Done Done false
22528 7027 Acked Read 1 Sent Done Done false
22529 7028 Acked Write 1 Sent Done Done false
22530 7029 Acked Flush 0 Sent Done Done false
22531 7030 Acked Read 2 Sent Done Done false
22532 7031 Acked Flush 0 Sent Done Done false
22533 7032 Acked Read 8 Sent Done Done false
22534 7033 Acked Read 3 Sent Done Done false
22535 7034 Acked Read 3 Sent Done Done false
22536 7035 Acked Read 5 Sent Done Done false
22537 7036 Acked Read 8 Sent Done Done false
22538 STATES DS:0 DS:1 DS:2 TOTAL
22539 Sent 301 0 0 301
22540 Done 0 301 301 602
22541 Skipped 0 0 0 0
22542 Error 0 0 0 0
22543 Last Flush: 6735 7031 7031
22544 Downstairs last five completed: 6735 6734 6733 6732 6731
22545 Upstairs last five completed: 7036 7035 7034 7033 7032
22546 CLIENT: Up:0 ds:301 act:2
22547 ----------------------------------------------------------------
22548 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
22549 States: Offline Active Active
22550 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
22551 6736 Acked Write 2 Sent Done Done false
22552 6737 Acked Write 4 Sent Done Done false
22553 6738 Acked Write 3 Sent Done Done false
22554 6739 Acked Write 4 Sent Done Done false
22555 6740 Acked Write 1 Sent Done Done false
22556 6741 Acked Flush 0 Sent Done Done false
22557 6742 Acked Write 4 Sent Done Done false
22558 6743 Acked Write 3 Sent Done Done false
22559 6744 Acked Read 7 Sent Done Done false
22560 6745 Acked Write 3 Sent Done Done false
22561 6746 Acked Read 8 Sent Done Done false
22562 6747 Acked Read 5 Sent Done Done false
22563 6748 Acked Read 10 Sent Done Done false
22564 6749 Acked Write 9 Sent Done Done false
22565 6750 Acked Read 8 Sent Done Done false
22566 6751 Acked Read 2 Sent Done Done false
22567 6752 Acked Read 8 Sent Done Done false
22568 6753 Acked Write 7 Sent Done Done false
22569 6754 Acked Write 3 Sent Done Done false
22570 6755 Acked Read 1 Sent Done Done false
22571 6756 Acked Write 4 Sent Done Done false
22572 6757 Acked Write 1 Sent Done Done false
22573 6758 Acked Read 9 Sent Done Done false
22574 6759 Acked Read 2 Sent Done Done false
22575 6760 Acked Write 2 Sent Done Done false
22576 6761 Acked Write 9 Sent Done Done false
22577 6762 Acked Write 3 Sent Done Done false
22578 6763 Acked Write 9 Sent Done Done false
22579 6764 Acked Write 7 Sent Done Done false
22580 6765 Acked Read 5 Sent Done Done false
22581 6766 Acked Flush 0 Sent Done Done false
22582 6767 Acked Read 6 Sent Done Done false
22583 6768 Acked Read 6 Sent Done Done false
22584 6769 Acked Flush 0 Sent Done Done false
22585 6770 Acked Write 6 Sent Done Done false
22586 6771 Acked Read 5 Sent Done Done false
22587 6772 Acked Read 2 Sent Done Done false
22588 6773 Acked Write 10 Sent Done Done false
22589 6774 Acked Read 5 Sent Done Done false
22590 6775 Acked Read 1 Sent Done Done false
22591 6776 Acked Flush 0 Sent Done Done false
22592 6777 Acked Read 8 Sent Done Done false
22593 6778 Acked Read 1 Sent Done Done false
22594 6779 Acked Write 8 Sent Done Done false
22595 6780 Acked Read 1 Sent Done Done false
22596 6781 Acked Read 2 Sent Done Done false
22597 6782 Acked Read 3 Sent Done Done false
22598 6783 Acked Write 1 Sent Done Done false
22599 6784 Acked Read 2 Sent Done Done false
22600 6785 Acked Flush 0 Sent Done Done false
22601 6786 Acked Flush 0 Sent Done Done false
22602 6787 Acked Write 3 Sent Done Done false
22603 6788 Acked Read 4 Sent Done Done false
22604 6789 Acked Read 3 Sent Done Done false
22605 6790 Acked Flush 0 Sent Done Done false
22606 6791 Acked Write 4 Sent Done Done false
22607 6792 Acked Write 7 Sent Done Done false
22608 6793 Acked Write 7 Sent Done Done false
22609 6794 Acked Write 2 Sent Done Done false
22610 6795 Acked Write 1 Sent Done Done false
22611 6796 Acked Write 7 Sent Done Done false
22612 6797 Acked Flush 0 Sent Done Done false
22613 6798 Acked Read 1 Sent Done Done false
22614 6799 Acked Read 6 Sent Done Done false
22615 6800 Acked Read 7 Sent Done Done false
22616 6801 Acked Read 3 Sent Done Done false
22617 6802 Acked Flush 0 Sent Done Done false
22618 6803 Acked Write 6 Sent Done Done false
22619 6804 Acked Write 1 Sent Done Done false
22620 6805 Acked Write 9 Sent Done Done false
22621 6806 Acked Read 1 Sent Done Done false
22622 6807 Acked Read 7 Sent Done Done false
22623 6808 Acked Read 6 Sent Done Done false
22624 6809 Acked Read 10 Sent Done Done false
22625 6810 Acked Read 6 Sent Done Done false
22626 6811 Acked Write 8 Sent Done Done false
22627 6812 Acked Read 4 Sent Done Done false
22628 6813 Acked Read 10 Sent Done Done false
22629 6814 Acked Read 8 Sent Done Done false
22630 6815 Acked Flush 0 Sent Done Done false
22631 6816 Acked Write 10 Sent Done Done false
22632 6817 Acked Read 5 Sent Done Done false
22633 6818 Acked Read 5 Sent Done Done false
22634 6819 Acked Read 2 Sent Done Done false
22635 6820 Acked Read 3 Sent Done Done false
22636 6821 Acked Write 5 Sent Done Done false
22637 6822 Acked Write 10 Sent Done Done false
22638 6823 Acked Write 2 Sent Done Done false
22639 6824 Acked Read 7 Sent Done Done false
22640 6825 Acked Write 10 Sent Done Done false
22641 6826 Acked Write 2 Sent Done Done false
22642 6827 Acked Write 5 Sent Done Done false
22643 6828 Acked Write 10 Sent Done Done false
22644 6829 Acked Read 9 Sent Done Done false
22645 6830 Acked Write 1 Sent Done Done false
22646 6831 Acked Flush 0 Sent Done Done false
22647 6832 Acked Read 10 Sent Done Done false
22648 6833 Acked Flush 0 Sent Done Done false
22649 6834 Acked Read 2 Sent Done Done false
22650 6835 Acked Write 3 Sent Done Done false
22651 6836 Acked Flush 0 Sent Done Done false
22652 6837 Acked Read 6 Sent Done Done false
22653 6838 Acked Read 7 Sent Done Done false
22654 6839 Acked Write 9 Sent Done Done false
22655 6840 Acked Write 6 Sent Done Done false
22656 6841 Acked Read 5 Sent Done Done false
22657 6842 Acked Write 2 Sent Done Done false
22658 6843 Acked Read 4 Sent Done Done false
22659 6844 Acked Write 8 Sent Done Done false
22660 6845 Acked Write 5 Sent Done Done false
22661 6846 Acked Write 7 Sent Done Done false
22662 6847 Acked Read 7 Sent Done Done false
22663 6848 Acked Read 4 Sent Done Done false
22664 6849 Acked Read 4 Sent Done Done false
22665 6850 Acked Write 1 Sent Done Done false
22666 6851 Acked Read 10 Sent Done Done false
22667 6852 Acked Write 2 Sent Done Done false
22668 6853 Acked Read 2 Sent Done Done false
22669 6854 Acked Read 4 Sent Done Done false
22670 6855 Acked Read 9 Sent Done Done false
22671 6856 Acked Write 2 Sent Done Done false
22672 6857 Acked Write 5 Sent Done Done false
22673 6858 Acked Write 1 Sent Done Done false
22674 6859 Acked Read 10 Sent Done Done false
22675 6860 Acked Read 10 Sent Done Done false
22676 6861 Acked Read 4 Sent Done Done false
22677 6862 Acked Write 1 Sent Done Done false
22678 6863 Acked Read 5 Sent Done Done false
22679 6864 Acked Write 9 Sent Done Done false
22680 6865 Acked Read 1 Sent Done Done false
22681 6866 Acked Read 4 Sent Done Done false
22682 6867 Acked Write 9 Sent Done Done false
22683 6868 Acked Read 3 Sent Done Done false
22684 6869 Acked Read 2 Sent Done Done false
22685 6870 Acked Write 8 Sent Done Done false
22686 6871 Acked Write 10 Sent Done Done false
22687 6872 Acked Write 9 Sent Done Done false
22688 6873 Acked Read 2 Sent Done Done false
22689 6874 Acked Read 9 Sent Done Done false
22690 6875 Acked Read 9 Sent Done Done false
22691 6876 Acked Write 7 Sent Done Done false
22692 6877 Acked Read 6 Sent Done Done false
22693 6878 Acked Read 4 Sent Done Done false
22694 6879 Acked Read 8 Sent Done Done false
22695 6880 Acked Read 4 Sent Done Done false
22696 6881 Acked Write 8 Sent Done Done false
22697 6882 Acked Write 7 Sent Done Done false
22698 6883 Acked Write 4 Sent Done Done false
22699 6884 Acked Read 6 Sent Done Done false
22700 6885 Acked Write 6 Sent Done Done false
22701 6886 Acked Read 3 Sent Done Done false
22702 6887 Acked Write 2 Sent Done Done false
22703 6888 Acked Read 7 Sent Done Done false
22704 6889 Acked Write 10 Sent Done Done false
22705 6890 Acked Write 8 Sent Done Done false
22706 6891 Acked Write 8 Sent Done Done false
22707 6892 Acked Read 1 Sent Done Done false
22708 6893 Acked Read 3 Sent Done Done false
22709 6894 Acked Write 2 Sent Done Done false
22710 6895 Acked Read 7 Sent Done Done false
22711 6896 Acked Read 9 Sent Done Done false
22712 6897 Acked Write 5 Sent Done Done false
22713 6898 Acked Write 8 Sent Done Done false
22714 6899 Acked Write 4 Sent Done Done false
22715 6900 Acked Write 4 Sent Done Done false
22716 6901 Acked Read 1 Sent Done Done false
22717 6902 Acked Read 7 Sent Done Done false
22718 6903 Acked Read 4 Sent Done Done false
22719 6904 Acked Read 5 Sent Done Done false
22720 6905 Acked Read 5 Sent Done Done false
22721 6906 Acked Read 7 Sent Done Done false
22722 6907 Acked Write 6 Sent Done Done false
22723 6908 Acked Write 7 Sent Done Done false
22724 6909 Acked Flush 0 Sent Done Done false
22725 6910 Acked Read 5 Sent Done Done false
22726 6911 Acked Read 8 Sent Done Done false
22727 6912 Acked Read 3 Sent Done Done false
22728 6913 Acked Write 1 Sent Done Done false
22729 6914 Acked Read 9 Sent Done Done false
22730 6915 Acked Write 3 Sent Done Done false
22731 6916 Acked Write 6 Sent Done Done false
22732 6917 Acked Write 8 Sent Done Done false
22733 6918 Acked Write 6 Sent Done Done false
22734 6919 Acked Read 1 Sent Done Done false
22735 6920 Acked Write 5 Sent Done Done false
22736 6921 Acked Write 4 Sent Done Done false
22737 6922 Acked Read 4 Sent Done Done false
22738 6923 Acked Write 8 Sent Done Done false
22739 6924 Acked Write 10 Sent Done Done false
22740 6925 Acked Flush 0 Sent Done Done false
22741 6926 Acked Read 8 Sent Done Done false
22742 6927 Acked Write 10 Sent Done Done false
22743 6928 Acked Write 5 Sent Done Done false
22744 6929 Acked Write 1 Sent Done Done false
22745 6930 Acked Flush 0 Sent Done Done false
22746 6931 Acked Write 3 Sent Done Done false
22747 6932 Acked Read 2 Sent Done Done false
22748 6933 Acked Write 7 Sent Done Done false
22749 6934 Acked Write 6 Sent Done Done false
22750 6935 Acked Flush 0 Sent Done Done false
22751 6936 Acked Write 3 Sent Done Done false
22752 6937 Acked Write 6 Sent Done Done false
22753 6938 Acked Write 9 Sent Done Done false
22754 6939 Acked Flush 0 Sent Done Done false
22755 6940 Acked Write 2 Sent Done Done false
22756 6941 Acked Flush 0 Sent Done Done false
22757 6942 Acked Read 5 Sent Done Done false
22758 6943 Acked Read 4 Sent Done Done false
22759 6944 Acked Flush 0 Sent Done Done false
22760 6945 Acked Write 7 Sent Done Done false
22761 6946 Acked Flush 0 Sent Done Done false
22762 6947 Acked Read 3 Sent Done Done false
22763 6948 Acked Write 7 Sent Done Done false
22764 6949 Acked Write 1 Sent Done Done false
22765 6950 Acked Read 1 Sent Done Done false
22766 6951 Acked Read 4 Sent Done Done false
22767 6952 Acked Write 7 Sent Done Done false
22768 6953 Acked Read 3 Sent Done Done false
22769 6954 Acked Read 9 Sent Done Done false
22770 6955 Acked Read 2 Sent Done Done false
22771 6956 Acked Write 1 Sent Done Done false
22772 6957 Acked Read 6 Sent Done Done false
22773 6958 Acked Flush 0 Sent Done Done false
22774 6959 Acked Write 5 Sent Done Done false
22775 6960 Acked Write 9 Sent Done Done false
22776 6961 Acked Write 2 Sent Done Done false
22777 6962 Acked Read 8 Sent Done Done false
22778 6963 Acked Flush 0 Sent Done Done false
22779 6964 Acked Read 7 Sent Done Done false
22780 6965 Acked Read 2 Sent Done Done false
22781 6966 Acked Write 6 Sent Done Done false
22782 6967 Acked Write 3 Sent Done Done false
22783 6968 Acked Read 4 Sent Done Done false
22784 6969 Acked Write 2 Sent Done Done false
22785 6970 Acked Read 3 Sent Done Done false
22786 6971 Acked Read 9 Sent Done Done false
22787 6972 Acked Read 9 Sent Done Done false
22788 6973 Acked Read 5 Sent Done Done false
22789 6974 Acked Write 9 Sent Done Done false
22790 6975 Acked Read 3 Sent Done Done false
22791 6976 Acked Read 6 Sent Done Done false
22792 6977 Acked Write 2 Sent Done Done false
22793 6978 Acked Write 4 Sent Done Done false
22794 6979 Acked Write 10 Sent Done Done false
22795 6980 Acked Write 10 Sent Done Done false
22796 6981 Acked Write 6 Sent Done Done false
22797 6982 Acked Read 2 Sent Done Done false
22798 6983 Acked Write 8 Sent Done Done false
22799 6984 Acked Read 5 Sent Done Done false
22800 6985 Acked Write 5 Sent Done Done false
22801 6986 Acked Read 10 Sent Done Done false
22802 6987 Acked Write 6 Sent Done Done false
22803 6988 Acked Read 6 Sent Done Done false
22804 6989 Acked Read 9 Sent Done Done false
22805 6990 Acked Write 6 Sent Done Done false
22806 6991 Acked Write 5 Sent Done Done false
22807 6992 Acked Write 10 Sent Done Done false
22808 6993 Acked Read 3 Sent Done Done false
22809 6994 Acked Flush 0 Sent Done Done false
22810 6995 Acked Write 6 Sent Done Done false
22811 6996 Acked Read 4 Sent Done Done false
22812 6997 Acked Read 1 Sent Done Done false
22813 6998 Acked Write 9 Sent Done Done false
22814 6999 Acked Write 10 Sent Done Done false
22815 7000 Acked Flush 0 Sent Done Done false
22816 7001 Acked Read 5 Sent Done Done false
22817 7002 Acked Flush 0 Sent Done Done false
22818 7003 Acked Write 3 Sent Done Done false
22819 7004 Acked Write 1 Sent Done Done false
22820 7005 Acked Read 2 Sent Done Done false
22821 7006 Acked Write 2 Sent Done Done false
22822 7007 Acked Flush 0 Sent Done Done false
22823 7008 Acked Write 3 Sent Done Done false
22824 7009 Acked Write 3 Sent Done Done false
22825 7010 Acked Read 4 Sent Done Done false
22826 7011 Acked Read 6 Sent Done Done false
22827 7012 Acked Write 8 Sent Done Done false
22828 7013 Acked Write 1 Sent Done Done false
22829 7014 Acked Read 3 Sent Done Done false
22830 7015 Acked Write 6 Sent Done Done false
22831 7016 Acked Read 3 Sent Done Done false
22832 7017 Acked Write 6 Sent Done Done false
22833 7018 Acked Read 2 Sent Done Done false
22834 7019 Acked Flush 0 Sent Done Done false
22835 7020 Acked Write 9 Sent Done Done false
22836 7021 Acked Read 8 Sent Done Done false
22837 7022 Acked Write 9 Sent Done Done false
22838 7023 Acked Write 7 Sent Done Done false
22839 7024 Acked Read 1 Sent Done Done false
22840 7025 Acked Read 6 Sent Done Done false
22841 7026 Acked Read 9 Sent Done Done false
22842 7027 Acked Read 1 Sent Done Done false
22843 7028 Acked Write 1 Sent Done Done false
22844 7029 Acked Flush 0 Sent Done Done false
22845 7030 Acked Read 2 Sent Done Done false
22846 7031 Acked Flush 0 Sent Done Done false
22847 7032 Acked Read 8 Sent Done Done false
22848 7033 Acked Read 3 Sent Done Done false
22849 7034 Acked Read 3 Sent Done Done false
22850 7035 Acked Read 5 Sent Done Done false
22851 7036 Acked Read 8 Sent Done Done false
22852 7037 Acked Flush 0 Sent Done Done false
22853 STATES DS:0 DS:1 DS:2 TOTAL
22854 Sent 302 0 0 302
22855 Done 0 302 302 604
22856 Skipped 0 0 0 0
22857 Error 0 0 0 0
22858 Last Flush: 6735 7037 7037
22859 Downstairs last five completed: 6735 6734 6733 6732 6731
22860 Upstairs last five completed: 7037 7036 7035 7034 7033
22861 CLIENT: Up:0 ds:302 act:2
228622024-11-28T04:55:05.657ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
228632024-11-28T04:55:05.657ZWARNcrucible: ds_connection connect to 127.0.0.1:8810 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
228642024-11-28T04:55:05.657ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
228652024-11-28T04:55:05.657ZWARNcrucible: downstairs task for 0 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
228662024-11-28T04:55:05.657ZINFOcrucible: client task is exiting = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
22867 ----------------------------------------------------------------
22868 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
22869 States: Offline Active Active
22870 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
22871 6736 Acked Write 2 Sent Done Done false
22872 6737 Acked Write 4 Sent Done Done false
22873 6738 Acked Write 3 Sent Done Done false
22874 6739 Acked Write 4 Sent Done Done false
22875 6740 Acked Write 1 Sent Done Done false
22876 6741 Acked Flush 0 Sent Done Done false
22877 6742 Acked Write 4 Sent Done Done false
22878 6743 Acked Write 3 Sent Done Done false
22879 6744 Acked Read 7 Sent Done Done false
22880 6745 Acked Write 3 Sent Done Done false
22881 6746 Acked Read 8 Sent Done Done false
22882 6747 Acked Read 5 Sent Done Done false
22883 6748 Acked Read 10 Sent Done Done false
22884 6749 Acked Write 9 Sent Done Done false
22885 6750 Acked Read 8 Sent Done Done false
22886 6751 Acked Read 2 Sent Done Done false
22887 6752 Acked Read 8 Sent Done Done false
22888 6753 Acked Write 7 Sent Done Done false
22889 6754 Acked Write 3 Sent Done Done false
22890 6755 Acked Read 1 Sent Done Done false
22891 6756 Acked Write 4 Sent Done Done false
22892 6757 Acked Write 1 Sent Done Done false
22893 6758 Acked Read 9 Sent Done Done false
22894 6759 Acked Read 2 Sent Done Done false
22895 6760 Acked Write 2 Sent Done Done false
22896 6761 Acked Write 9 Sent Done Done false
22897 6762 Acked Write 3 Sent Done Done false
22898 6763 Acked Write 9 Sent Done Done false
22899 6764 Acked Write 7 Sent Done Done false
22900 6765 Acked Read 5 Sent Done Done false
22901 6766 Acked Flush 0 Sent Done Done false
22902 6767 Acked Read 6 Sent Done Done false
22903 6768 Acked Read 6 Sent Done Done false
22904 6769 Acked Flush 0 Sent Done Done false
22905 6770 Acked Write 6 Sent Done Done false
22906 6771 Acked Read 5 Sent Done Done false
22907 6772 Acked Read 2 Sent Done Done false
22908 6773 Acked Write 10 Sent Done Done false
22909 6774 Acked Read 5 Sent Done Done false
22910 6775 Acked Read 1 Sent Done Done false
22911 6776 Acked Flush 0 Sent Done Done false
22912 6777 Acked Read 8 Sent Done Done false
22913 6778 Acked Read 1 Sent Done Done false
22914 6779 Acked Write 8 Sent Done Done false
22915 6780 Acked Read 1 Sent Done Done false
22916 6781 Acked Read 2 Sent Done Done false
22917 6782 Acked Read 3 Sent Done Done false
22918 6783 Acked Write 1 Sent Done Done false
22919 6784 Acked Read 2 Sent Done Done false
22920 6785 Acked Flush 0 Sent Done Done false
22921 6786 Acked Flush 0 Sent Done Done false
22922 6787 Acked Write 3 Sent Done Done false
22923 6788 Acked Read 4 Sent Done Done false
22924 6789 Acked Read 3 Sent Done Done false
22925 6790 Acked Flush 0 Sent Done Done false
22926 6791 Acked Write 4 Sent Done Done false
22927 6792 Acked Write 7 Sent Done Done false
22928 6793 Acked Write 7 Sent Done Done false
22929 6794 Acked Write 2 Sent Done Done false
22930 6795 Acked Write 1 Sent Done Done false
22931 6796 Acked Write 7 Sent Done Done false
22932 6797 Acked Flush 0 Sent Done Done false
22933 6798 Acked Read 1 Sent Done Done false
22934 6799 Acked Read 6 Sent Done Done false
22935 6800 Acked Read 7 Sent Done Done false
22936 6801 Acked Read 3 Sent Done Done false
22937 6802 Acked Flush 0 Sent Done Done false
22938 6803 Acked Write 6 Sent Done Done false
22939 6804 Acked Write 1 Sent Done Done false
22940 6805 Acked Write 9 Sent Done Done false
22941 6806 Acked Read 1 Sent Done Done false
22942 6807 Acked Read 7 Sent Done Done false
22943 6808 Acked Read 6 Sent Done Done false
22944 6809 Acked Read 10 Sent Done Done false
22945 6810 Acked Read 6 Sent Done Done false
22946 6811 Acked Write 8 Sent Done Done false
22947 6812 Acked Read 4 Sent Done Done false
22948 6813 Acked Read 10 Sent Done Done false
22949 6814 Acked Read 8 Sent Done Done false
22950 6815 Acked Flush 0 Sent Done Done false
22951 6816 Acked Write 10 Sent Done Done false
22952 6817 Acked Read 5 Sent Done Done false
22953 6818 Acked Read 5 Sent Done Done false
22954 6819 Acked Read 2 Sent Done Done false
22955 6820 Acked Read 3 Sent Done Done false
22956 6821 Acked Write 5 Sent Done Done false
22957 6822 Acked Write 10 Sent Done Done false
22958 6823 Acked Write 2 Sent Done Done false
22959 6824 Acked Read 7 Sent Done Done false
22960 6825 Acked Write 10 Sent Done Done false
22961 6826 Acked Write 2 Sent Done Done false
22962 6827 Acked Write 5 Sent Done Done false
22963 6828 Acked Write 10 Sent Done Done false
22964 6829 Acked Read 9 Sent Done Done false
22965 6830 Acked Write 1 Sent Done Done false
22966 6831 Acked Flush 0 Sent Done Done false
22967 6832 Acked Read 10 Sent Done Done false
22968 6833 Acked Flush 0 Sent Done Done false
22969 6834 Acked Read 2 Sent Done Done false
22970 6835 Acked Write 3 Sent Done Done false
22971 6836 Acked Flush 0 Sent Done Done false
22972 6837 Acked Read 6 Sent Done Done false
22973 6838 Acked Read 7 Sent Done Done false
22974 6839 Acked Write 9 Sent Done Done false
22975 6840 Acked Write 6 Sent Done Done false
22976 6841 Acked Read 5 Sent Done Done false
22977 6842 Acked Write 2 Sent Done Done false
22978 6843 Acked Read 4 Sent Done Done false
22979 6844 Acked Write 8 Sent Done Done false
22980 6845 Acked Write 5 Sent Done Done false
22981 6846 Acked Write 7 Sent Done Done false
22982 6847 Acked Read 7 Sent Done Done false
22983 6848 Acked Read 4 Sent Done Done false
22984 6849 Acked Read 4 Sent Done Done false
22985 6850 Acked Write 1 Sent Done Done false
22986 6851 Acked Read 10 Sent Done Done false
22987 6852 Acked Write 2 Sent Done Done false
22988 6853 Acked Read 2 Sent Done Done false
22989 6854 Acked Read 4 Sent Done Done false
22990 6855 Acked Read 9 Sent Done Done false
22991 6856 Acked Write 2 Sent Done Done false
22992 6857 Acked Write 5 Sent Done Done false
22993 6858 Acked Write 1 Sent Done Done false
22994 6859 Acked Read 10 Sent Done Done false
22995 6860 Acked Read 10 Sent Done Done false
22996 6861 Acked Read 4 Sent Done Done false
22997 6862 Acked Write 1 Sent Done Done false
22998 6863 Acked Read 5 Sent Done Done false
22999 6864 Acked Write 9 Sent Done Done false
23000 6865 Acked Read 1 Sent Done Done false
23001 6866 Acked Read 4 Sent Done Done false
23002 6867 Acked Write 9 Sent Done Done false
23003 6868 Acked Read 3 Sent Done Done false
23004 6869 Acked Read 2 Sent Done Done false
23005 6870 Acked Write 8 Sent Done Done false
23006 6871 Acked Write 10 Sent Done Done false
23007 6872 Acked Write 9 Sent Done Done false
23008 6873 Acked Read 2 Sent Done Done false
23009 6874 Acked Read 9 Sent Done Done false
23010 6875 Acked Read 9 Sent Done Done false
23011 6876 Acked Write 7 Sent Done Done false
23012 6877 Acked Read 6 Sent Done Done false
23013 6878 Acked Read 4 Sent Done Done false
23014 6879 Acked Read 8 Sent Done Done false
23015 6880 Acked Read 4 Sent Done Done false
23016 6881 Acked Write 8 Sent Done Done false
23017 6882 Acked Write 7 Sent Done Done false
23018 6883 Acked Write 4 Sent Done Done false
23019 6884 Acked Read 6 Sent Done Done false
23020 6885 Acked Write 6 Sent Done Done false
23021 6886 Acked Read 3 Sent Done Done false
23022 6887 Acked Write 2 Sent Done Done false
23023 6888 Acked Read 7 Sent Done Done false
23024 6889 Acked Write 10 Sent Done Done false
23025 6890 Acked Write 8 Sent Done Done false
23026 6891 Acked Write 8 Sent Done Done false
23027 6892 Acked Read 1 Sent Done Done false
23028 6893 Acked Read 3 Sent Done Done false
23029 6894 Acked Write 2 Sent Done Done false
23030 6895 Acked Read 7 Sent Done Done false
23031 6896 Acked Read 9 Sent Done Done false
23032 6897 Acked Write 5 Sent Done Done false
23033 6898 Acked Write 8 Sent Done Done false
23034 6899 Acked Write 4 Sent Done Done false
23035 6900 Acked Write 4 Sent Done Done false
23036 6901 Acked Read 1 Sent Done Done false
23037 6902 Acked Read 7 Sent Done Done false
23038 6903 Acked Read 4 Sent Done Done false
23039 6904 Acked Read 5 Sent Done Done false
23040 6905 Acked Read 5 Sent Done Done false
23041 6906 Acked Read 7 Sent Done Done false
23042 6907 Acked Write 6 Sent Done Done false
23043 6908 Acked Write 7 Sent Done Done false
23044 6909 Acked Flush 0 Sent Done Done false
23045 6910 Acked Read 5 Sent Done Done false
23046 6911 Acked Read 8 Sent Done Done false
23047 6912 Acked Read 3 Sent Done Done false
23048 6913 Acked Write 1 Sent Done Done false
23049 6914 Acked Read 9 Sent Done Done false
23050 6915 Acked Write 3 Sent Done Done false
23051 6916 Acked Write 6 Sent Done Done false
23052 6917 Acked Write 8 Sent Done Done false
23053 6918 Acked Write 6 Sent Done Done false
23054 6919 Acked Read 1 Sent Done Done false
23055 6920 Acked Write 5 Sent Done Done false
23056 6921 Acked Write 4 Sent Done Done false
23057 6922 Acked Read 4 Sent Done Done false
23058 6923 Acked Write 8 Sent Done Done false
23059 6924 Acked Write 10 Sent Done Done false
23060 6925 Acked Flush 0 Sent Done Done false
23061 6926 Acked Read 8 Sent Done Done false
23062 6927 Acked Write 10 Sent Done Done false
23063 6928 Acked Write 5 Sent Done Done false
23064 6929 Acked Write 1 Sent Done Done false
23065 6930 Acked Flush 0 Sent Done Done false
23066 6931 Acked Write 3 Sent Done Done false
23067 6932 Acked Read 2 Sent Done Done false
23068 6933 Acked Write 7 Sent Done Done false
23069 6934 Acked Write 6 Sent Done Done false
23070 6935 Acked Flush 0 Sent Done Done false
23071 6936 Acked Write 3 Sent Done Done false
23072 6937 Acked Write 6 Sent Done Done false
23073 6938 Acked Write 9 Sent Done Done false
23074 6939 Acked Flush 0 Sent Done Done false
23075 6940 Acked Write 2 Sent Done Done false
23076 6941 Acked Flush 0 Sent Done Done false
23077 6942 Acked Read 5 Sent Done Done false
23078 6943 Acked Read 4 Sent Done Done false
23079 6944 Acked Flush 0 Sent Done Done false
23080 6945 Acked Write 7 Sent Done Done false
23081 6946 Acked Flush 0 Sent Done Done false
23082 6947 Acked Read 3 Sent Done Done false
23083 6948 Acked Write 7 Sent Done Done false
23084 6949 Acked Write 1 Sent Done Done false
23085 6950 Acked Read 1 Sent Done Done false
23086 6951 Acked Read 4 Sent Done Done false
23087 6952 Acked Write 7 Sent Done Done false
23088 6953 Acked Read 3 Sent Done Done false
23089 6954 Acked Read 9 Sent Done Done false
23090 6955 Acked Read 2 Sent Done Done false
23091 6956 Acked Write 1 Sent Done Done false
23092 6957 Acked Read 6 Sent Done Done false
23093 6958 Acked Flush 0 Sent Done Done false
23094 6959 Acked Write 5 Sent Done Done false
23095 6960 Acked Write 9 Sent Done Done false
23096 6961 Acked Write 2 Sent Done Done false
23097 6962 Acked Read 8 Sent Done Done false
23098 6963 Acked Flush 0 Sent Done Done false
23099 6964 Acked Read 7 Sent Done Done false
23100 6965 Acked Read 2 Sent Done Done false
23101 6966 Acked Write 6 Sent Done Done false
23102 6967 Acked Write 3 Sent Done Done false
23103 6968 Acked Read 4 Sent Done Done false
23104 6969 Acked Write 2 Sent Done Done false
23105 6970 Acked Read 3 Sent Done Done false
23106 6971 Acked Read 9 Sent Done Done false
23107 6972 Acked Read 9 Sent Done Done false
23108 6973 Acked Read 5 Sent Done Done false
23109 6974 Acked Write 9 Sent Done Done false
23110 6975 Acked Read 3 Sent Done Done false
23111 6976 Acked Read 6 Sent Done Done false
23112 6977 Acked Write 2 Sent Done Done false
23113 6978 Acked Write 4 Sent Done Done false
23114 6979 Acked Write 10 Sent Done Done false
23115 6980 Acked Write 10 Sent Done Done false
23116 6981 Acked Write 6 Sent Done Done false
23117 6982 Acked Read 2 Sent Done Done false
23118 6983 Acked Write 8 Sent Done Done false
23119 6984 Acked Read 5 Sent Done Done false
23120 6985 Acked Write 5 Sent Done Done false
23121 6986 Acked Read 10 Sent Done Done false
23122 6987 Acked Write 6 Sent Done Done false
23123 6988 Acked Read 6 Sent Done Done false
23124 6989 Acked Read 9 Sent Done Done false
23125 6990 Acked Write 6 Sent Done Done false
23126 6991 Acked Write 5 Sent Done Done false
23127 6992 Acked Write 10 Sent Done Done false
23128 6993 Acked Read 3 Sent Done Done false
23129 6994 Acked Flush 0 Sent Done Done false
23130 6995 Acked Write 6 Sent Done Done false
23131 6996 Acked Read 4 Sent Done Done false
23132 6997 Acked Read 1 Sent Done Done false
23133 6998 Acked Write 9 Sent Done Done false
23134 6999 Acked Write 10 Sent Done Done false
23135 7000 Acked Flush 0 Sent Done Done false
23136 7001 Acked Read 5 Sent Done Done false
23137 7002 Acked Flush 0 Sent Done Done false
23138 7003 Acked Write 3 Sent Done Done false
23139 7004 Acked Write 1 Sent Done Done false
23140 7005 Acked Read 2 Sent Done Done false
23141 7006 Acked Write 2 Sent Done Done false
23142 7007 Acked Flush 0 Sent Done Done false
23143 7008 Acked Write 3 Sent Done Done false
23144 7009 Acked Write 3 Sent Done Done false
23145 7010 Acked Read 4 Sent Done Done false
23146 7011 Acked Read 6 Sent Done Done false
23147 7012 Acked Write 8 Sent Done Done false
23148 7013 Acked Write 1 Sent Done Done false
23149 7014 Acked Read 3 Sent Done Done false
23150 7015 Acked Write 6 Sent Done Done false
23151 7016 Acked Read 3 Sent Done Done false
23152 7017 Acked Write 6 Sent Done Done false
23153 7018 Acked Read 2 Sent Done Done false
23154 7019 Acked Flush 0 Sent Done Done false
23155 7020 Acked Write 9 Sent Done Done false
23156 7021 Acked Read 8 Sent Done Done false
23157 7022 Acked Write 9 Sent Done Done false
23158 7023 Acked Write 7 Sent Done Done false
23159 7024 Acked Read 1 Sent Done Done false
23160 7025 Acked Read 6 Sent Done Done false
23161 7026 Acked Read 9 Sent Done Done false
23162 7027 Acked Read 1 Sent Done Done false
23163 7028 Acked Write 1 Sent Done Done false
23164 7029 Acked Flush 0 Sent Done Done false
23165 7030 Acked Read 2 Sent Done Done false
23166 7031 Acked Flush 0 Sent Done Done false
23167 7032 Acked Read 8 Sent Done Done false
23168 7033 Acked Read 3 Sent Done Done false
23169 7034 Acked Read 3 Sent Done Done false
23170 7035 Acked Read 5 Sent Done Done false
23171 7036 Acked Read 8 Sent Done Done false
23172 7037 Acked Flush 0 Sent Done Done false
23173 STATES DS:0 DS:1 DS:2 TOTAL
23174 Sent 302 0 0 302
23175 Done 0 302 302 604
23176 Skipped 0 0 0 0
23177 Error 0 0 0 0
23178 Last Flush: 6735 7037 7037
23179 Downstairs last five completed: 6735 6734 6733 6732 6731
23180 Upstairs last five completed: 7037 7036 7035 7034 7033
23181 CLIENT: Up:0 ds:302 act:2
23182 [8810] got start action
23183 [8810] I am going to restart
23184 Starting downstairs at port 8810
23185 Make output file at "/tmp/dsc/downstairs-8810.txt"
23186 [8810][0] reports Starting
23187 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1222)
23188 [8810][0] reports Running
23189 ----------------------------------------------------------------
23190 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
23191 States: Offline Active Active
23192 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
23193 6736 Acked Write 2 Sent Done Done false
23194 6737 Acked Write 4 Sent Done Done false
23195 6738 Acked Write 3 Sent Done Done false
23196 6739 Acked Write 4 Sent Done Done false
23197 6740 Acked Write 1 Sent Done Done false
23198 6741 Acked Flush 0 Sent Done Done false
23199 6742 Acked Write 4 Sent Done Done false
23200 6743 Acked Write 3 Sent Done Done false
23201 6744 Acked Read 7 Sent Done Done false
23202 6745 Acked Write 3 Sent Done Done false
23203 6746 Acked Read 8 Sent Done Done false
23204 6747 Acked Read 5 Sent Done Done false
23205 6748 Acked Read 10 Sent Done Done false
23206 6749 Acked Write 9 Sent Done Done false
23207 6750 Acked Read 8 Sent Done Done false
23208 6751 Acked Read 2 Sent Done Done false
23209 6752 Acked Read 8 Sent Done Done false
23210 6753 Acked Write 7 Sent Done Done false
23211 6754 Acked Write 3 Sent Done Done false
23212 6755 Acked Read 1 Sent Done Done false
23213 6756 Acked Write 4 Sent Done Done false
23214 6757 Acked Write 1 Sent Done Done false
23215 6758 Acked Read 9 Sent Done Done false
23216 6759 Acked Read 2 Sent Done Done false
23217 6760 Acked Write 2 Sent Done Done false
23218 6761 Acked Write 9 Sent Done Done false
23219 6762 Acked Write 3 Sent Done Done false
23220 6763 Acked Write 9 Sent Done Done false
23221 6764 Acked Write 7 Sent Done Done false
23222 6765 Acked Read 5 Sent Done Done false
23223 6766 Acked Flush 0 Sent Done Done false
23224 6767 Acked Read 6 Sent Done Done false
23225 6768 Acked Read 6 Sent Done Done false
23226 6769 Acked Flush 0 Sent Done Done false
23227 6770 Acked Write 6 Sent Done Done false
23228 6771 Acked Read 5 Sent Done Done false
23229 6772 Acked Read 2 Sent Done Done false
23230 6773 Acked Write 10 Sent Done Done false
23231 6774 Acked Read 5 Sent Done Done false
23232 6775 Acked Read 1 Sent Done Done false
23233 6776 Acked Flush 0 Sent Done Done false
23234 6777 Acked Read 8 Sent Done Done false
23235 6778 Acked Read 1 Sent Done Done false
23236 6779 Acked Write 8 Sent Done Done false
23237 6780 Acked Read 1 Sent Done Done false
23238 6781 Acked Read 2 Sent Done Done false
23239 6782 Acked Read 3 Sent Done Done false
23240 6783 Acked Write 1 Sent Done Done false
23241 6784 Acked Read 2 Sent Done Done false
23242 6785 Acked Flush 0 Sent Done Done false
23243 6786 Acked Flush 0 Sent Done Done false
23244 6787 Acked Write 3 Sent Done Done false
23245 6788 Acked Read 4 Sent Done Done false
23246 6789 Acked Read 3 Sent Done Done false
23247 6790 Acked Flush 0 Sent Done Done false
23248 6791 Acked Write 4 Sent Done Done false
23249 6792 Acked Write 7 Sent Done Done false
23250 6793 Acked Write 7 Sent Done Done false
23251 6794 Acked Write 2 Sent Done Done false
23252 6795 Acked Write 1 Sent Done Done false
23253 6796 Acked Write 7 Sent Done Done false
23254 6797 Acked Flush 0 Sent Done Done false
23255 6798 Acked Read 1 Sent Done Done false
23256 6799 Acked Read 6 Sent Done Done false
23257 6800 Acked Read 7 Sent Done Done false
23258 6801 Acked Read 3 Sent Done Done false
23259 6802 Acked Flush 0 Sent Done Done false
23260 6803 Acked Write 6 Sent Done Done false
23261 6804 Acked Write 1 Sent Done Done false
23262 6805 Acked Write 9 Sent Done Done false
23263 6806 Acked Read 1 Sent Done Done false
23264 6807 Acked Read 7 Sent Done Done false
23265 6808 Acked Read 6 Sent Done Done false
23266 6809 Acked Read 10 Sent Done Done false
23267 6810 Acked Read 6 Sent Done Done false
23268 6811 Acked Write 8 Sent Done Done false
23269 6812 Acked Read 4 Sent Done Done false
23270 6813 Acked Read 10 Sent Done Done false
23271 6814 Acked Read 8 Sent Done Done false
23272 6815 Acked Flush 0 Sent Done Done false
23273 6816 Acked Write 10 Sent Done Done false
23274 6817 Acked Read 5 Sent Done Done false
23275 6818 Acked Read 5 Sent Done Done false
23276 6819 Acked Read 2 Sent Done Done false
23277 6820 Acked Read 3 Sent Done Done false
23278 6821 Acked Write 5 Sent Done Done false
23279 6822 Acked Write 10 Sent Done Done false
23280 6823 Acked Write 2 Sent Done Done false
23281 6824 Acked Read 7 Sent Done Done false
23282 6825 Acked Write 10 Sent Done Done false
23283 6826 Acked Write 2 Sent Done Done false
23284 6827 Acked Write 5 Sent Done Done false
23285 6828 Acked Write 10 Sent Done Done false
23286 6829 Acked Read 9 Sent Done Done false
23287 6830 Acked Write 1 Sent Done Done false
23288 6831 Acked Flush 0 Sent Done Done false
23289 6832 Acked Read 10 Sent Done Done false
23290 6833 Acked Flush 0 Sent Done Done false
23291 6834 Acked Read 2 Sent Done Done false
23292 6835 Acked Write 3 Sent Done Done false
23293 6836 Acked Flush 0 Sent Done Done false
23294 6837 Acked Read 6 Sent Done Done false
23295 6838 Acked Read 7 Sent Done Done false
23296 6839 Acked Write 9 Sent Done Done false
23297 6840 Acked Write 6 Sent Done Done false
23298 6841 Acked Read 5 Sent Done Done false
23299 6842 Acked Write 2 Sent Done Done false
23300 6843 Acked Read 4 Sent Done Done false
23301 6844 Acked Write 8 Sent Done Done false
23302 6845 Acked Write 5 Sent Done Done false
23303 6846 Acked Write 7 Sent Done Done false
23304 6847 Acked Read 7 Sent Done Done false
23305 6848 Acked Read 4 Sent Done Done false
23306 6849 Acked Read 4 Sent Done Done false
23307 6850 Acked Write 1 Sent Done Done false
23308 6851 Acked Read 10 Sent Done Done false
23309 6852 Acked Write 2 Sent Done Done false
23310 6853 Acked Read 2 Sent Done Done false
23311 6854 Acked Read 4 Sent Done Done false
23312 6855 Acked Read 9 Sent Done Done false
23313 6856 Acked Write 2 Sent Done Done false
23314 6857 Acked Write 5 Sent Done Done false
23315 6858 Acked Write 1 Sent Done Done false
23316 6859 Acked Read 10 Sent Done Done false
23317 6860 Acked Read 10 Sent Done Done false
23318 6861 Acked Read 4 Sent Done Done false
23319 6862 Acked Write 1 Sent Done Done false
23320 6863 Acked Read 5 Sent Done Done false
23321 6864 Acked Write 9 Sent Done Done false
23322 6865 Acked Read 1 Sent Done Done false
23323 6866 Acked Read 4 Sent Done Done false
23324 6867 Acked Write 9 Sent Done Done false
23325 6868 Acked Read 3 Sent Done Done false
23326 6869 Acked Read 2 Sent Done Done false
23327 6870 Acked Write 8 Sent Done Done false
23328 6871 Acked Write 10 Sent Done Done false
23329 6872 Acked Write 9 Sent Done Done false
23330 6873 Acked Read 2 Sent Done Done false
23331 6874 Acked Read 9 Sent Done Done false
23332 6875 Acked Read 9 Sent Done Done false
23333 6876 Acked Write 7 Sent Done Done false
23334 6877 Acked Read 6 Sent Done Done false
23335 6878 Acked Read 4 Sent Done Done false
23336 6879 Acked Read 8 Sent Done Done false
23337 6880 Acked Read 4 Sent Done Done false
23338 6881 Acked Write 8 Sent Done Done false
23339 6882 Acked Write 7 Sent Done Done false
23340 6883 Acked Write 4 Sent Done Done false
23341 6884 Acked Read 6 Sent Done Done false
23342 6885 Acked Write 6 Sent Done Done false
23343 6886 Acked Read 3 Sent Done Done false
23344 6887 Acked Write 2 Sent Done Done false
23345 6888 Acked Read 7 Sent Done Done false
23346 6889 Acked Write 10 Sent Done Done false
23347 6890 Acked Write 8 Sent Done Done false
23348 6891 Acked Write 8 Sent Done Done false
23349 6892 Acked Read 1 Sent Done Done false
23350 6893 Acked Read 3 Sent Done Done false
23351 6894 Acked Write 2 Sent Done Done false
23352 6895 Acked Read 7 Sent Done Done false
23353 6896 Acked Read 9 Sent Done Done false
23354 6897 Acked Write 5 Sent Done Done false
23355 6898 Acked Write 8 Sent Done Done false
23356 6899 Acked Write 4 Sent Done Done false
23357 6900 Acked Write 4 Sent Done Done false
23358 6901 Acked Read 1 Sent Done Done false
23359 6902 Acked Read 7 Sent Done Done false
23360 6903 Acked Read 4 Sent Done Done false
23361 6904 Acked Read 5 Sent Done Done false
23362 6905 Acked Read 5 Sent Done Done false
23363 6906 Acked Read 7 Sent Done Done false
23364 6907 Acked Write 6 Sent Done Done false
23365 6908 Acked Write 7 Sent Done Done false
23366 6909 Acked Flush 0 Sent Done Done false
23367 6910 Acked Read 5 Sent Done Done false
23368 6911 Acked Read 8 Sent Done Done false
23369 6912 Acked Read 3 Sent Done Done false
23370 6913 Acked Write 1 Sent Done Done false
23371 6914 Acked Read 9 Sent Done Done false
23372 6915 Acked Write 3 Sent Done Done false
23373 6916 Acked Write 6 Sent Done Done false
23374 6917 Acked Write 8 Sent Done Done false
23375 6918 Acked Write 6 Sent Done Done false
23376 6919 Acked Read 1 Sent Done Done false
23377 6920 Acked Write 5 Sent Done Done false
23378 6921 Acked Write 4 Sent Done Done false
23379 6922 Acked Read 4 Sent Done Done false
23380 6923 Acked Write 8 Sent Done Done false
23381 6924 Acked Write 10 Sent Done Done false
23382 6925 Acked Flush 0 Sent Done Done false
23383 6926 Acked Read 8 Sent Done Done false
23384 6927 Acked Write 10 Sent Done Done false
23385 6928 Acked Write 5 Sent Done Done false
23386 6929 Acked Write 1 Sent Done Done false
23387 6930 Acked Flush 0 Sent Done Done false
23388 6931 Acked Write 3 Sent Done Done false
23389 6932 Acked Read 2 Sent Done Done false
23390 6933 Acked Write 7 Sent Done Done false
23391 6934 Acked Write 6 Sent Done Done false
23392 6935 Acked Flush 0 Sent Done Done false
23393 6936 Acked Write 3 Sent Done Done false
23394 6937 Acked Write 6 Sent Done Done false
23395 6938 Acked Write 9 Sent Done Done false
23396 6939 Acked Flush 0 Sent Done Done false
23397 6940 Acked Write 2 Sent Done Done false
23398 6941 Acked Flush 0 Sent Done Done false
23399 6942 Acked Read 5 Sent Done Done false
23400 6943 Acked Read 4 Sent Done Done false
23401 6944 Acked Flush 0 Sent Done Done false
23402 6945 Acked Write 7 Sent Done Done false
23403 6946 Acked Flush 0 Sent Done Done false
23404 6947 Acked Read 3 Sent Done Done false
23405 6948 Acked Write 7 Sent Done Done false
23406 6949 Acked Write 1 Sent Done Done false
23407 6950 Acked Read 1 Sent Done Done false
23408 6951 Acked Read 4 Sent Done Done false
23409 6952 Acked Write 7 Sent Done Done false
23410 6953 Acked Read 3 Sent Done Done false
23411 6954 Acked Read 9 Sent Done Done false
23412 6955 Acked Read 2 Sent Done Done false
23413 6956 Acked Write 1 Sent Done Done false
23414 6957 Acked Read 6 Sent Done Done false
23415 6958 Acked Flush 0 Sent Done Done false
23416 6959 Acked Write 5 Sent Done Done false
23417 6960 Acked Write 9 Sent Done Done false
23418 6961 Acked Write 2 Sent Done Done false
23419 6962 Acked Read 8 Sent Done Done false
23420 6963 Acked Flush 0 Sent Done Done false
23421 6964 Acked Read 7 Sent Done Done false
23422 6965 Acked Read 2 Sent Done Done false
23423 6966 Acked Write 6 Sent Done Done false
23424 6967 Acked Write 3 Sent Done Done false
23425 6968 Acked Read 4 Sent Done Done false
23426 6969 Acked Write 2 Sent Done Done false
23427 6970 Acked Read 3 Sent Done Done false
23428 6971 Acked Read 9 Sent Done Done false
23429 6972 Acked Read 9 Sent Done Done false
23430 6973 Acked Read 5 Sent Done Done false
23431 6974 Acked Write 9 Sent Done Done false
23432 6975 Acked Read 3 Sent Done Done false
23433 6976 Acked Read 6 Sent Done Done false
23434 6977 Acked Write 2 Sent Done Done false
23435 6978 Acked Write 4 Sent Done Done false
23436 6979 Acked Write 10 Sent Done Done false
23437 6980 Acked Write 10 Sent Done Done false
23438 6981 Acked Write 6 Sent Done Done false
23439 6982 Acked Read 2 Sent Done Done false
23440 6983 Acked Write 8 Sent Done Done false
23441 6984 Acked Read 5 Sent Done Done false
23442 6985 Acked Write 5 Sent Done Done false
23443 6986 Acked Read 10 Sent Done Done false
23444 6987 Acked Write 6 Sent Done Done false
23445 6988 Acked Read 6 Sent Done Done false
23446 6989 Acked Read 9 Sent Done Done false
23447 6990 Acked Write 6 Sent Done Done false
23448 6991 Acked Write 5 Sent Done Done false
23449 6992 Acked Write 10 Sent Done Done false
23450 6993 Acked Read 3 Sent Done Done false
23451 6994 Acked Flush 0 Sent Done Done false
23452 6995 Acked Write 6 Sent Done Done false
23453 6996 Acked Read 4 Sent Done Done false
23454 6997 Acked Read 1 Sent Done Done false
23455 6998 Acked Write 9 Sent Done Done false
23456 6999 Acked Write 10 Sent Done Done false
23457 7000 Acked Flush 0 Sent Done Done false
23458 7001 Acked Read 5 Sent Done Done false
23459 7002 Acked Flush 0 Sent Done Done false
23460 7003 Acked Write 3 Sent Done Done false
23461 7004 Acked Write 1 Sent Done Done false
23462 7005 Acked Read 2 Sent Done Done false
23463 7006 Acked Write 2 Sent Done Done false
23464 7007 Acked Flush 0 Sent Done Done false
23465 7008 Acked Write 3 Sent Done Done false
23466 7009 Acked Write 3 Sent Done Done false
23467 7010 Acked Read 4 Sent Done Done false
23468 7011 Acked Read 6 Sent Done Done false
23469 7012 Acked Write 8 Sent Done Done false
23470 7013 Acked Write 1 Sent Done Done false
23471 7014 Acked Read 3 Sent Done Done false
23472 7015 Acked Write 6 Sent Done Done false
23473 7016 Acked Read 3 Sent Done Done false
23474 7017 Acked Write 6 Sent Done Done false
23475 7018 Acked Read 2 Sent Done Done false
23476 7019 Acked Flush 0 Sent Done Done false
23477 7020 Acked Write 9 Sent Done Done false
23478 7021 Acked Read 8 Sent Done Done false
23479 7022 Acked Write 9 Sent Done Done false
23480 7023 Acked Write 7 Sent Done Done false
23481 7024 Acked Read 1 Sent Done Done false
23482 7025 Acked Read 6 Sent Done Done false
23483 7026 Acked Read 9 Sent Done Done false
23484 7027 Acked Read 1 Sent Done Done false
23485 7028 Acked Write 1 Sent Done Done false
23486 7029 Acked Flush 0 Sent Done Done false
23487 7030 Acked Read 2 Sent Done Done false
23488 7031 Acked Flush 0 Sent Done Done false
23489 7032 Acked Read 8 Sent Done Done false
23490 7033 Acked Read 3 Sent Done Done false
23491 7034 Acked Read 3 Sent Done Done false
23492 7035 Acked Read 5 Sent Done Done false
23493 7036 Acked Read 8 Sent Done Done false
23494 7037 Acked Flush 0 Sent Done Done false
23495 STATES DS:0 DS:1 DS:2 TOTAL
23496 Sent 302 0 0 302
23497 Done 0 302 302 604
23498 Skipped 0 0 0 0
23499 Error 0 0 0 0
23500 Last Flush: 6735 7037 7037
23501 Downstairs last five completed: 6735 6734 6733 6732 6731
23502 Upstairs last five completed: 7037 7036 7035 7034 7033
23503 CLIENT: Up:0 ds:302 act:2
235042024-11-28T04:55:15.657ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
235052024-11-28T04:55:15.657ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:57751) = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
235062024-11-28T04:55:15.658ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
235072024-11-28T04:55:15.659ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
235082024-11-28T04:55:15.659ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
235092024-11-28T04:55:15.659ZINFOcrucible: send last flush ID to this DS: 6735 = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
235102024-11-28T04:55:15.659ZINFOcrucible: Replied this last flush ID: 6735 = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
235112024-11-28T04:55:15.659ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
235122024-11-28T04:55:15.659ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
235132024-11-28T04:55:15.659ZINFOcrucible: [0] client re-new 302 jobs since flush 6735 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
235142024-11-28T04:55:15.659ZINFOcrucible: [0] Replayed 302 jobs since flush: 6735 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
23515 ----------------------------------------------------------------
23516 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
23517 Downstairs last five completed: 7037 7036 7035 7034 7033
23518 Upstairs last five completed: 7037 7036 7035 7034 7033
23519 CLIENT: Up:0 ds:0 act:3
23520 Replay: All jobs finished
23521 Main task has work to do, go find it
23522 got dsc Stop(2)
23523 stop 2
23524 [8830] Got stop action so:false kr:false
23525 Nov 28 04:55:15.885 INFO request completed, latency_us: 364, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 1f3c71f6-7211-478c-9542-37ee655c4144, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
23526 Nov 28 04:55:15.886{"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2024-11-28T04:55:15.886314392Z","hostname":"w-01JDRHKEEFJ1228TFPSBEGXWHY","pid":1202,"":"io task","client":"2","":"downstairs","session_id":"4a2415b1-6594-436f-82e5-b8cf9a74eeaf"}
23527 INFO request completed, latency_us: 116, response_code: 200, uri: /state/cid/2, method: GET, {"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2024-11-28T04:55:15.88639727Z","hostname":"w-01JDRHKEEFJ1228TFPSBEGXWHY","pid":1202,"":"io task","client":"2","":"downstairs","session_id":"4a2415b1-6594-436f-82e5-b8cf9a74eeaf"}
23528 req_id: bebb81d3-f074-4db2-a8cd-8182e52edf9b, remote_addr: {"msg":"downstairs task for 2 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2024-11-28T04:55:15.886437715Z","hostname":"w-01JDRHKEEFJ1228TFPSBEGXWHY","pid":1202,"session_id":"4a2415b1-6594-436f-82e5-b8cf9a74eeaf"}
23529 127.0.0.1:60418, {"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-28T04:55:15.886507735Z","hostname":"w-01JDRHKEEFJ1228TFPSBEGXWHY","pid":1202,"client":"2","":"downstairs","session_id":"4a2415b1-6594-436f-82e5-b8cf9a74eeaf"}
23530 local_addr: 127.0.0.1:9998{"msg":"[2] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-28T04:55:15.886552239Z","hostname":"w-01JDRHKEEFJ1228TFPSBEGXWHY","pid":1202,"client":"2","":"downstairs","session_id":"4a2415b1-6594-436f-82e5-b8cf9a74eeaf"}
23531 
235322024-11-28T04:55:15.886ZINFOcrucible: client task is exiting = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
23533 [8830] Exited with: ExitStatus(unix_wait_status(9))
23534 [8830][2] reports Exit
23535 Nov 28 04:55:19.888 INFO request completed, latency_us: 197, response_code: 200, uri: /state/cid/2, method: GET, req_id: 4b93db1d-9a13-4cbc-b324-c0395a0ca897, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
23536 001/300 Write block 1831 len 16384 data: 6 5 5 4
23537 002/300 Flush
23538 003/300 Flush
23539 004/300 Flush
23540 005/300 Write block 2398 len 20480 data: 6 5 4 2 3
23541 006/300 Write block 864 len 32768 data: 5 7 6 6 5 5 6 6
23542 007/300 Read block 1416 len 16384
23543 008/300 Flush
23544 009/300 Read block 2792 len 24576
23545 010/300 Write block 2252 len 20480 data: 4 4 4 5 3
23546 011/300 Flush
23547 012/300 Write block 3212 len 32768 data: 4 4 4 5 5 6 7 5
23548 013/300 Write block 3491 len 8192 data: 5 5
23549 014/300 Flush
23550 015/300 Read block 1505 len 32768
23551 016/300 Read block 1104 len 20480
23552 017/300 Flush
23553 018/300 Write block 1735 len 32768 data: 3 3 4 3 3 3 2 4
23554 019/300 Write block 4702 len 40960 data: 4 4 6 5 4 4 4 4 4 6
23555 020/300 Read block 601 len 12288
23556 021/300 Flush
23557 022/300 Read block 110 len 4096
23558 023/300 Read block 4710 len 16384
23559 024/300 Write block 4838 len 8192 data: 4 3
23560 025/300 Read block 4791 len 16384
23561 026/300 Read block 1629 len 36864
23562 027/300 Flush
23563 028/300 Read block 1489 len 20480
23564 029/300 Read block 1819 len 32768
23565 030/300 Write block 4904 len 32768 data: 3 2 2 2 2 2 2 2
23566 031/300 Flush
23567 032/300 Flush
23568 033/300 Read block 1547 len 40960
23569 034/300 Read block 3254 len 32768
23570 035/300 Read block 377 len 28672
23571 036/300 Write block 1099 len 32768 data: 5 6 6 4 4 5 4 4
23572 037/300 Read block 1719 len 40960
23573 038/300 Read block 1262 len 40960
23574 039/300 Read block 4836 len 40960
23575 040/300 Write block 4362 len 32768 data: 7 8 8 7 6 3 3 3
23576 041/300 Read block 453 len 36864
23577 042/300 Read block 1178 len 28672
23578 043/300 Read block 2631 len 4096
23579 044/300 Read block 658 len 28672
23580 045/300 Flush
23581 046/300 Write block 2752 len 12288 data: 8 7 6
23582 047/300 Read block 1967 len 28672
23583 048/300 Write block 3352 len 32768 data: 3 4 4 3 4 4 4 4
23584 049/300 Flush
23585 050/300 Read block 768 len 8192
23586 051/300 Read block 3842 len 20480
23587 052/300 Write block 2515 len 12288 data: 5 2 4
23588 053/300 Write block 1192 len 24576 data: 3 3 3 3 4 4
23589 054/300 Flush
23590 055/300 Read block 1086 len 4096
23591 056/300 Write block 563 len 40960 data: 6 6 5 6 3 3 3 3 3 4
23592 057/300 Write block 4779 len 24576 data: 6 5 4 4 2 2
23593 058/300 Flush
23594 059/300 Write block 3236 len 28672 data: 3 4 4 7 5 5 5
23595 060/300 Read block 3282 len 32768
23596 061/300 Read block 1527 len 20480
23597 062/300 Write block 132 len 12288 data: 3 3 3
23598 063/300 Write block 3563 len 12288 data: 3 5 5
23599 064/300 Write block 4515 len 36864 data: 4 3 3 4 4 5 6 7 7
23600 065/300 Write block 590 len 4096 data: 3
23601 066/300 Flush
23602 067/300 Read block 4804 len 8192
23603 068/300 Read block 3694 len 28672
23604 069/300 Write block 3404 len 28672 data: 6 5 5 5 4 4 4
23605 070/300 Write block 2977 len 40960 data: 6 6 6 6 5 4 3 2 3 3
23606 071/300 Write block 2460 len 4096 data: 4
23607 072/300 Read block 4217 len 4096
23608 073/300 Write block 1060 len 16384 data: 5 5 4 4
23609 074/300 Write block 752 len 8192 data: 3 4
23610 075/300 Write block 4162 len 8192 data: 5 6
23611 076/300 Read block 1765 len 8192
23612 077/300 Write block 325 len 28672 data: 7 5 5 4 4 4 4
23613 078/300 Write block 4810 len 16384 data: 3 2 2 2
23614 079/300 Write block 2033 len 12288 data: 5 6 6
23615 080/300 Read block 3863 len 28672
23616 081/300 Read block 571 len 28672
23617 082/300 Read block 3702 len 28672
23618 083/300 Read block 2400 len 40960
23619 084/300 Write block 4207 len 16384 data: 3 3 3 4
23620 085/300 Write block 4479 len 28672 data: 6 6 5 5 4 3 3
23621 086/300 Read block 1749 len 4096
23622 087/300 Write block 2596 len 16384 data: 4 3 4 5
23623 088/300 Flush
23624 089/300 Read block 903 len 16384
23625 090/300 Write block 1993 len 16384 data: 2 2 2 3
23626 091/300 Write block 468 len 40960 data: 9 10 10 8 8 8 8 8 6 4
23627 092/300 Write block 3154 len 36864 data: 6 5 6 6 5 6 5 4 4
23628 093/300 Flush
23629 094/300 Read block 4220 len 28672
23630 095/300 Read block 1378 len 20480
23631 096/300 Read block 4400 len 36864
23632 097/300 Write block 3572 len 8192 data: 5 4
23633 098/300 Read block 19 len 24576
23634 099/300 Read block 3760 len 8192
23635 100/300 Read block 2116 len 4096
23636 101/300 Read block 3242 len 8192
23637 102/300 Read block 1516 len 40960
23638 103/300 Read block 2340 len 12288
23639 104/300 Read block 2073 len 8192
23640 105/300 Write block 852 len 36864 data: 7 4 4 3 3 3 5 6 4
23641 106/300 Read block 2101 len 4096
23642 107/300 Write block 2014 len 24576 data: 4 4 4 4 4 4
23643 108/300 Flush
23644 109/300 Write block 1847 len 20480 data: 5 4 6 5 6
23645 110/300 Read block 2503 len 40960
23646 111/300 Read block 4044 len 40960
23647 112/300 Read block 1476 len 36864
23648 113/300 Read block 953 len 40960
23649 114/300 Write block 2776 len 24576 data: 8 7 7 8 7 6
23650 115/300 Write block 4294 len 4096 data: 2
23651 116/300 Write block 4738 len 20480 data: 5 5 5 3 2
23652 117/300 Write block 1063 len 28672 data: 5 4 4 4 5 5 5
23653 118/300 Write block 3901 len 16384 data: 3 5 5 5
23654 119/300 Read block 3780 len 24576
23655 120/300 Write block 736 len 24576 data: 5 4 4 2 2 3
23656 121/300 Read block 666 len 36864
23657 122/300 Write block 3154 len 28672 data: 7 6 7 7 6 7 6
23658 123/300 Read block 1193 len 4096
23659 124/300 Write block 42 len 4096 data: 4
23660 125/300 Read block 239 len 8192
23661 126/300 Read block 4968 len 8192
23662 127/300 Write block 286 len 12288 data: 3 4 5
23663 128/300 Read block 2106 len 36864
23664 129/300 Read block 808 len 4096
23665 130/300 Read block 3185 len 12288
23666 131/300 Read block 2769 len 12288
23667 132/300 Read block 763 len 36864
23668 133/300 Read block 1590 len 16384
23669 134/300 Read block 2162 len 40960
23670 135/300 Flush
23671 136/300 Write block 253 len 28672 data: 3 2 2 4 4 5 4
23672 137/300 Write block 3790 len 20480 data: 5 6 7 6 5
23673 138/300 Read block 2955 len 20480
23674 139/300 Write block 4513 len 40960 data: 3 3 5 4 4 5 5 6 7 8
23675 140/300 Read block 2339 len 36864
23676 141/300 Write block 4781 len 20480 data: 5 5 3 3 2
23677 142/300 Read block 2037 len 40960
23678 143/300 Read block 399 len 20480
23679 144/300 Read block 3151 len 20480
23680 145/300 Write block 71 len 16384 data: 4 3 3 3
23681 146/300 Read block 2092 len 24576
23682 147/300 Read block 2749 len 36864
23683 148/300 Read block 448 len 32768
23684 149/300 Write block 2154 len 20480 data: 3 3 3 5 5
23685 150/300 Flush
23686 151/300 Read block 2521 len 32768
23687 152/300 Read block 2619 len 8192
23688 153/300 Read block 2076 len 12288
23689 154/300 Read block 1870 len 32768
23690 155/300 Read block 4942 len 40960
23691 156/300 Write block 3950 len 28672 data: 5 5 6 6 5 5 6
23692 157/300 Read block 475 len 36864
23693 158/300 Write block 1137 len 12288 data: 3 3 2
23694 159/300 Write block 2746 len 36864 data: 4 4 4 5 8 9 9 8 7
23695 160/300 Read block 3135 len 36864
23696 161/300 Flush
23697 162/300 Write block 3404 len 40960 data: 7 6 6 6 5 5 5 4 5 6
23698 163/300 Write block 1394 len 24576 data: 5 4 5 4 5 7
23699 164/300 Read block 3355 len 24576
23700 165/300 Write block 4823 len 32768 data: 4 5 5 6 5 5 4 6
23701 166/300 Write block 3472 len 40960 data: 5 5 5 5 6 6 5 5 5 4
23702 167/300 Read block 3095 len 12288
23703 168/300 Read block 4906 len 4096
23704 169/300 Flush
23705 170/300 Write block 3513 len 16384 data: 6 6 4 6
23706 171/300 Flush
23707 172/300 Write block 2079 len 4096 data: 6
23708 173/300 Read block 2133 len 28672
23709 174/300 Write block 4419 len 16384 data: 6 5 4 4
23710 175/300 Read block 3342 len 24576
23711 176/300 Read block 4854 len 28672
23712 177/300 Read block 3321 len 36864
23713 178/300 Write block 1846 len 12288 data: 4 6 5
23714 179/300 Write block 3671 len 28672 data: 5 7 4 7 7 6 5
23715 180/300 Read block 571 len 16384
23716 181/300 Write block 1415 len 28672 data: 4 5 4 4 5 5 4
23717 182/300 Read block 4351 len 36864
23718 183/300 Read block 2820 len 16384
23719 184/300 Read block 88 len 20480
23720 185/300 Flush
23721 186/300 Read block 1191 len 40960
23722 187/300 Read block 3815 len 28672
23723 188/300 Write block 3613 len 12288 data: 3 5 5
23724 189/300 Read block 365 len 8192
23725 190/300 Read block 1233 len 40960
23726 191/300 Flush
23727 192/300 Write block 1539 len 20480 data: 5 6 5 4 5
23728 193/300 Write block 3439 len 20480 data: 5 4 4 5 4
23729 194/300 Flush
23730 195/300 Write block 4548 len 20480 data: 4 5 5 4 6
23731 196/300 Read block 4719 len 20480
23732 197/300 Write block 2095 len 32768 data: 4 4 5 5 4 5 4 5
23733 198/300 Write block 1837 len 4096 data: 5
23734 199/300 Flush
23735 200/300 Write block 2165 len 16384 data: 4 5 3 5
23736 201/300 Read block 3833 len 8192
23737 202/300 Read block 4954 len 36864
23738 203/300 Write block 1630 len 36864 data: 7 7 7 5 3 2 2 2 2
23739 204/300 Read block 2488 len 32768
23740 205/300 Read block 1253 len 24576
23741 206/300 Flush
23742 207/300 Read block 3898 len 28672
23743 208/300 Write block 1648 len 8192 data: 3 4
23744 209/300 Write block 4079 len 4096 data: 10
23745 210/300 Read block 4236 len 36864
23746 211/300 Read block 4525 len 4096
23747 212/300 Read block 265 len 12288
23748 213/300 Write block 3816 len 12288 data: 6 5 4
23749 214/300 Flush
23750 215/300 Read block 35 len 16384
23751 216/300 Read block 1044 len 4096
23752 217/300 Read block 4859 len 12288
23753 218/300 Write block 3889 len 20480 data: 5 5 5 4 5
23754 219/300 Write block 3216 len 40960 data: 6 7 8 6 5 6 5 5 5 5
23755 220/300 Read block 366 len 24576
23756 221/300 Read block 4290 len 4096
23757 222/300 Read block 1301 len 20480
23758 223/300 Read block 2910 len 28672
23759 224/300 Read block 2169 len 8192
23760 225/300 Read block 665 len 8192
23761 226/300 Read block 1065 len 40960
23762 227/300 Read block 4441 len 36864
23763 228/300 Flush
23764 229/300 Read block 2722 len 32768
23765 230/300 Read block 4358 len 32768
23766 231/300 Write block 4767 len 28672 data: 4 6 5 5 5 6 7
23767 232/300 Read block 2928 len 20480
23768 233/300 Read block 1903 len 8192
23769 234/300 Read block 3279 len 4096
23770 235/300 Write block 2383 len 36864 data: 7 7 6 5 5 5 5 4 4
23771 236/300 Read block 2753 len 4096
23772 237/300 Write block 2272 len 36864 data: 6 5 5 5 4 4 4 4 3
23773 238/300 Read block 1169 len 40960
23774 239/300 Read block 3865 len 8192
23775 240/300 Read block 4596 len 20480
23776 241/300 Flush
23777 242/300 Write block 1458 len 32768 data: 2 3 4 5 5 6 6 7
23778 243/300 Write block 1820 len 32768 data: 6 5 4 4 5 5 3 4
23779 244/300 Read block 1191 len 12288
23780 245/300 Write block 1617 len 32768 data: 4 4 3 4 4 4 5 5
23781 246/300 Write block 2790 len 20480 data: 10 7 7 5 4
23782 247/300 Write block 1600 len 4096 data: 3
23783 248/300 Read block 508 len 24576
23784 249/300 Write block 22 len 40960 data: 4 4 5 7 6 6 6 7 9 8
23785 250/300 Read block 855 len 4096
23786 251/300 Read block 2777 len 8192
23787 252/300 Write block 4386 len 32768 data: 4 5 5 5 6 4 4 3
23788 253/300 Write block 3178 len 28672 data: 5 4 4 3 3 2 3
23789 254/300 Read block 3259 len 20480
23790 255/300 Read block 3774 len 28672
23791 256/300 Write block 2927 len 32768 data: 4 3 5 5 5 5 5 4
23792 257/300 Read block 1119 len 24576
23793 258/300 Read block 3041 len 32768
23794 259/300 Read block 2192 len 20480
23795 260/300 Read block 1933 len 24576
23796 261/300 Write block 3083 len 36864 data: 5 6 6 5 4 4 5 6 6
23797 262/300 Write block 3234 len 40960 data: 3 3 4 5 5 8 6 6 6 4
23798 263/300 Read block 857 len 16384
23799 264/300 Write block 332 len 8192 data: 3 3
23800 265/300 Write block 783 len 8192 data: 3 3
23801 266/300 Write block 3190 len 16384 data: 6 5 5 5
23802 267/300 Flush
23803 268/300 Write block 2514 len 8192 data: 5 6
23804 269/300 Read block 4982 len 32768
23805 270/300 Write block 3551 len 36864 data: 4 4 4 4 5 6 6 5 6
23806 271/300 Read block 4609 len 8192
23807 272/300 Flush
23808 273/300 Read block 2323 len 36864
23809 274/300 Read block 4438 len 36864
23810 275/300 Read block 2745 len 36864
23811 276/300 Read block 3499 len 32768
23812 277/300 Flush
23813 278/300 Write block 1556 len 16384 data: 4 4 3 3
23814 279/300 Write block 4906 len 40960 data: 3 3 3 3 3 3 3 3 4 4
23815 280/300 Flush
23816 281/300 Write block 1355 len 32768 data: 4 4 5 5 4 3 5 5
23817 282/300 Read block 3212 len 40960
23818 283/300 Read block 1729 len 16384
23819 284/300 Read block 4514 len 20480
23820 285/300 Read block 616 len 24576
23821 286/300 Read block 2701 len 28672
23822 287/300 Write block 3222 len 4096 data: 6
23823 288/300 Read block 4156 len 12288
23824 289/300 Read block 1897 len 36864
23825 290/300 Read block 313 len 4096
23826 291/300 Read block 3903 len 8192
23827 292/300 Read block 4350 len 20480
23828 293/300 Read block 4892 len 24576
23829 294/300 Write block 1534 len 32768 data: 4 5 4 5 5 6 7 6
23830 295/300 Write block 3594 len 4096 data: 6
23831 296/300 Write block 1356 len 40960 data: 5 6 6 5 4 6 6 5 6 6
23832 297/300 Write block 3243 len 32768 data: 5 4 4 2 3 3 4 4
23833 298/300 Read block 1678 len 20480
23834 299/300 Read block 2144 len 20480
23835 300/300 Read block 3529 len 4096
23836 301/300 Flush
23837 Main task has work to do, go find it
23838 got dsc Start(2)
23839 start 2
23840 Nov 28 04:55:20.128 INFO request completed, latency_us: 214, response_code: 204, uri: /start/cid/2, method: POST, req_id: 6419d701-252e-473b-b6b1-c160462ae708, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
23841 Replay: started 2, returned:Ok(())
23842 ----------------------------------------------------------------
23843 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
23844 States: Active Active Offline
23845 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
23846 7038 Acked Write 4 Done Done Sent false
23847 7039 Acked Flush 0 Done Done Sent false
23848 7040 Acked Flush 0 Done Done Sent false
23849 7041 Acked Flush 0 Done Done Sent false
23850 7042 Acked Write 5 Done Done Sent false
23851 7043 Acked Write 8 Done Done Sent false
23852 7044 Acked Read 4 Done Done Sent false
23853 7045 Acked Flush 0 Done Done Sent false
23854 7046 Acked Read 6 Done Done Sent false
23855 7047 Acked Write 5 Done Done Sent false
23856 7048 Acked Flush 0 Done Done Sent false
23857 7049 Acked Write 8 Done Done Sent false
23858 7050 Acked Write 2 Done Done Sent false
23859 7051 Acked Flush 0 Done Done Sent false
23860 7052 Acked Read 8 Done Done Sent false
23861 7053 Acked Read 5 Done Done Sent false
23862 7054 Acked Flush 0 Done Done Sent false
23863 7055 Acked Write 8 Done Done Sent false
23864 7056 Acked Write 10 Done Done Sent false
23865 7057 Acked Read 3 Done Done Sent false
23866 7058 Acked Flush 0 Done Done Sent false
23867 7059 Acked Read 1 Done Done Sent false
23868 7060 Acked Read 4 Done Done Sent false
23869 7061 Acked Write 2 Done Done Sent false
23870 7062 Acked Read 4 Done Done Sent false
23871 7063 Acked Read 9 Done Done Sent false
23872 7064 Acked Flush 0 Done Done Sent false
23873 7065 Acked Read 5 Done Done Sent false
23874 7066 Acked Read 8 Done Done Sent false
23875 7067 Acked Write 8 Done Done Sent false
23876 7068 Acked Flush 0 Done Done Sent false
23877 7069 Acked Flush 0 Done Done Sent false
23878 7070 Acked Read 10 Done Done Sent false
23879 7071 Acked Read 8 Done Done Sent false
23880 7072 Acked Read 7 Done Done Sent false
23881 7073 Acked Write 8 Done Done Sent false
23882 7074 Acked Read 10 Done Done Sent false
23883 7075 Acked Read 10 Done Done Sent false
23884 7076 Acked Read 10 Done Done Sent false
23885 7077 Acked Write 8 Done Done Sent false
23886 7078 Acked Read 9 Done Done Sent false
23887 7079 Acked Read 7 Done Done Sent false
23888 7080 Acked Read 1 Done Done Sent false
23889 7081 Acked Read 7 Done Done Sent false
23890 7082 Acked Flush 0 Done Done Sent false
23891 7083 Acked Write 3 Done Done Sent false
23892 7084 Acked Read 7 Done Done Sent false
23893 7085 Acked Write 8 Done Done Sent false
23894 7086 Acked Flush 0 Done Done Sent false
23895 7087 Acked Read 2 Done Done Sent false
23896 7088 Acked Read 5 Done Done Sent false
23897 7089 Acked Write 3 Done Done Sent false
23898 7090 Acked Write 6 Done Done Sent false
23899 7091 Acked Flush 0 Done Done Sent false
23900 7092 Acked Read 1 Done Done Sent false
23901 7093 Acked Write 10 Done Done Sent false
23902 7094 Acked Write 6 Done Done Sent false
23903 7095 Acked Flush 0 Done Done Sent false
23904 7096 Acked Write 7 Done Done Sent false
23905 7097 Acked Read 8 Done Done Sent false
23906 7098 Acked Read 5 Done Done Sent false
23907 7099 Acked Write 3 Done Done Sent false
23908 7100 Acked Write 3 Done Done Sent false
23909 7101 Acked Write 9 Done Done Sent false
23910 7102 Acked Write 1 Done Done Sent false
23911 7103 Acked Flush 0 Done Done Sent false
23912 7104 Acked Read 2 Done Done Sent false
23913 7105 Acked Read 7 Done Done Sent false
23914 7106 Acked Write 7 Done Done Sent false
23915 7107 Acked Write 10 Done Done Sent false
23916 7108 Acked Write 1 Done Done Sent false
23917 7109 Acked Read 1 Done Done Sent false
23918 7110 Acked Write 4 Done Done Sent false
23919 7111 Acked Write 2 Done Done Sent false
23920 7112 Acked Write 2 Done Done Sent false
23921 7113 Acked Read 2 Done Done Sent false
23922 7114 Acked Write 7 Done Done Sent false
23923 7115 Acked Write 4 Done Done Sent false
23924 7116 Acked Write 3 Done Done Sent false
23925 7117 Acked Read 7 Done Done Sent false
23926 7118 Acked Read 7 Done Done Sent false
23927 7119 Acked Read 7 Done Done Sent false
23928 7120 Acked Read 10 Done Done Sent false
23929 7121 Acked Write 4 Done Done Sent false
23930 7122 Acked Write 7 Done Done Sent false
23931 7123 Acked Read 1 Done Done Sent false
23932 7124 Acked Write 4 Done Done Sent false
23933 7125 Acked Flush 0 Done Done Sent false
23934 7126 Acked Read 4 Done Done Sent false
23935 7127 Acked Write 4 Done Done Sent false
23936 7128 Acked Write 10 Done Done Sent false
23937 7129 Acked Write 9 Done Done Sent false
23938 7130 Acked Flush 0 Done Done Sent false
23939 7131 Acked Read 7 Done Done Sent false
23940 7132 Acked Read 5 Done Done Sent false
23941 7133 Acked Read 9 Done Done Sent false
23942 7134 Acked Write 2 Done Done Sent false
23943 7135 Acked Read 6 Done Done Sent false
23944 7136 Acked Read 2 Done Done Sent false
23945 7137 Acked Read 1 Done Done Sent false
23946 7138 Acked Read 2 Done Done Sent false
23947 7139 Acked Read 10 Done Done Sent false
23948 7140 Acked Read 3 Done Done Sent false
23949 7141 Acked Read 2 Done Done Sent false
23950 7142 Acked Write 9 Done Done Sent false
23951 7143 Acked Read 1 Done Done Sent false
23952 7144 Acked Write 6 Done Done Sent false
23953 7145 Acked Flush 0 Done Done Sent false
23954 7146 Acked Write 5 Done Done Sent false
23955 7147 Acked Read 10 Done Done Sent false
23956 7148 Acked Read 10 Done Done Sent false
23957 7149 Acked Read 9 Done Done Sent false
23958 7150 Acked Read 10 Done Done Sent false
23959 7151 Acked Write 6 Done Done Sent false
23960 7152 Acked Write 1 Done Done Sent false
23961 7153 Acked Write 5 Done Done Sent false
23962 7154 Acked Write 7 Done Done Sent false
23963 7155 Acked Write 4 Done Done Sent false
23964 7156 Acked Read 6 Done Done Sent false
23965 7157 Acked Write 6 Done Done Sent false
23966 7158 Acked Read 9 Done Done Sent false
23967 7159 Acked Write 7 Done Done Sent false
23968 7160 Acked Read 1 Done Done Sent false
23969 7161 Acked Write 1 Done Done Sent false
23970 7162 Acked Read 2 Done Done Sent false
23971 7163 Acked Read 2 Done Done Sent false
23972 7164 Acked Write 3 Done Done Sent false
23973 7165 Acked Read 9 Done Done Sent false
23974 7166 Acked Read 1 Done Done Sent false
23975 7167 Acked Read 3 Done Done Sent false
23976 7168 Acked Read 3 Done Done Sent false
23977 7169 Acked Read 9 Done Done Sent false
23978 7170 Acked Read 4 Done Done Sent false
23979 7171 Acked Read 10 Done Done Sent false
23980 7172 Acked Flush 0 Done Done Sent false
23981 7173 Acked Write 7 Done Done Sent false
23982 7174 Acked Write 5 Done Done Sent false
23983 7175 Acked Read 5 Done Done Sent false
23984 7176 Acked Write 10 Done Done Sent false
23985 7177 Acked Read 9 Done Done Sent false
23986 7178 Acked Write 5 Done Done Sent false
23987 7179 Acked Read 10 Done Done Sent false
23988 7180 Acked Read 5 Done Done Sent false
23989 7181 Acked Read 5 Done Done Sent false
23990 7182 Acked Write 4 Done Done Sent false
23991 7183 Acked Read 6 Done Done Sent false
23992 7184 Acked Read 9 Done Done Sent false
23993 7185 Acked Read 8 Done Done Sent false
23994 7186 Acked Write 5 Done Done Sent false
23995 7187 Acked Flush 0 Done Done Sent false
23996 7188 Acked Read 8 Done Done Sent false
23997 7189 Acked Read 2 Done Done Sent false
23998 7190 Acked Read 3 Done Done Sent false
23999 7191 Acked Read 8 Done Done Sent false
24000 7192 Acked Read 10 Done Done Sent false
24001 7193 Acked Write 7 Done Done Sent false
24002 7194 Acked Read 9 Done Done Sent false
24003 7195 Acked Write 3 Done Done Sent false
24004 7196 Acked Write 9 Done Done Sent false
24005 7197 Acked Read 9 Done Done Sent false
24006 7198 Acked Flush 0 Done Done Sent false
24007 7199 Acked Write 10 Done Done Sent false
24008 7200 Acked Write 6 Done Done Sent false
24009 7201 Acked Read 6 Done Done Sent false
24010 7202 Acked Write 8 Done Done Sent false
24011 7203 Acked Write 10 Done Done Sent false
24012 7204 Acked Read 3 Done Done Sent false
24013 7205 Acked Read 1 Done Done Sent false
24014 7206 Acked Flush 0 Done Done Sent false
24015 7207 Acked Write 4 Done Done Sent false
24016 7208 Acked Flush 0 Done Done Sent false
24017 7209 Acked Write 1 Done Done Sent false
24018 7210 Acked Read 7 Done Done Sent false
24019 7211 Acked Write 4 Done Done Sent false
24020 7212 Acked Read 6 Done Done Sent false
24021 7213 Acked Read 7 Done Done Sent false
24022 7214 Acked Read 9 Done Done Sent false
24023 7215 Acked Write 3 Done Done Sent false
24024 7216 Acked Write 7 Done Done Sent false
24025 7217 Acked Read 4 Done Done Sent false
24026 7218 Acked Write 7 Done Done Sent false
24027 7219 Acked Read 9 Done Done Sent false
24028 7220 Acked Read 4 Done Done Sent false
24029 7221 Acked Read 5 Done Done Sent false
24030 7222 Acked Flush 0 Done Done Sent false
24031 7223 Acked Read 10 Done Done Sent false
24032 7224 Acked Read 7 Done Done Sent false
24033 7225 Acked Write 3 Done Done Sent false
24034 7226 Acked Read 2 Done Done Sent false
24035 7227 Acked Read 10 Done Done Sent false
24036 7228 Acked Flush 0 Done Done Sent false
24037 7229 Acked Write 5 Done Done Sent false
24038 7230 Acked Write 5 Done Done Sent false
24039 7231 Acked Flush 0 Done Done Sent false
24040 7232 Acked Write 5 Done Done Sent false
24041 7233 Acked Read 5 Done Done Sent false
24042 7234 Acked Write 8 Done Done Sent false
24043 7235 Acked Write 1 Done Done Sent false
24044 7236 Acked Flush 0 Done Done Sent false
24045 7237 Acked Write 4 Done Done Sent false
24046 7238 Acked Read 2 Done Done Sent false
24047 7239 Acked Read 9 Done Done Sent false
24048 7240 Acked Write 9 Done Done Sent false
24049 7241 Acked Read 8 Done Done Sent false
24050 7242 Acked Read 6 Done Done Sent false
24051 7243 Acked Flush 0 Done Done Sent false
24052 7244 Acked Read 7 Done Done Sent false
24053 7245 Acked Write 2 Done Done Sent false
24054 7246 Acked Write 1 Done Done Sent false
24055 7247 Acked Read 9 Done Done Sent false
24056 7248 Acked Read 1 Done Done Sent false
24057 7249 Acked Read 3 Done Done Sent false
24058 7250 Acked Write 3 Done Done Sent false
24059 7251 Acked Flush 0 Done Done Sent false
24060 7252 Acked Read 4 Done Done Sent false
24061 7253 Acked Read 1 Done Done Sent false
24062 7254 Acked Read 3 Done Done Sent false
24063 7255 Acked Write 5 Done Done Sent false
24064 7256 Acked Write 10 Done Done Sent false
24065 7257 Acked Read 6 Done Done Sent false
24066 7258 Acked Read 1 Done Done Sent false
24067 7259 Acked Read 5 Done Done Sent false
24068 7260 Acked Read 7 Done Done Sent false
24069 7261 Acked Read 2 Done Done Sent false
24070 7262 Acked Read 2 Done Done Sent false
24071 7263 Acked Read 10 Done Done Sent false
24072 7264 Acked Read 9 Done Done Sent false
24073 7265 Acked Flush 0 Done Done Sent false
24074 7266 Acked Read 8 Done Done Sent false
24075 7267 Acked Read 8 Done Done Sent false
24076 7268 Acked Write 7 Done Done Sent false
24077 7269 Acked Read 5 Done Done Sent false
24078 7270 Acked Read 2 Done Done Sent false
24079 7271 Acked Read 1 Done Done Sent false
24080 7272 Acked Write 9 Done Done Sent false
24081 7273 Acked Read 1 Done Done Sent false
24082 7274 Acked Write 9 Done Done Sent false
24083 7275 Acked Read 10 Done Done Sent false
24084 7276 Acked Read 2 Done Done Sent false
24085 7277 Acked Read 5 Done Done Sent false
24086 7278 Acked Flush 0 Done Done Sent false
24087 7279 Acked Write 8 Done Done Sent false
24088 7280 Acked Write 8 Done Done Sent false
24089 7281 Acked Read 3 Done Done Sent false
24090 7282 Acked Write 8 Done Done Sent false
24091 7283 Acked Write 5 Done Done Sent false
24092 7284 Acked Write 1 Done Done Sent false
24093 7285 Acked Read 6 Done Done Sent false
24094 7286 Acked Write 10 Done Done Sent false
24095 7287 Acked Read 1 Done Done Sent false
24096 7288 Acked Read 2 Done Done Sent false
24097 7289 Acked Write 8 Done Done Sent false
24098 7290 Acked Write 7 Done Done Sent false
24099 7291 Acked Read 5 Done Done Sent false
24100 7292 Acked Read 7 Done Done Sent false
24101 7293 Acked Write 8 Done Done Sent false
24102 7294 Acked Read 6 Done Done Sent false
24103 7295 Acked Read 8 Done Done Sent false
24104 7296 Acked Read 5 Done Done Sent false
24105 7297 Acked Read 6 Done Done Sent false
24106 7298 Acked Write 9 Done Done Sent false
24107 7299 Acked Write 10 Done Done Sent false
24108 7300 Acked Read 4 Done Done Sent false
24109 7301 Acked Write 2 Done Done Sent false
24110 7302 Acked Write 2 Done Done Sent false
24111 7303 Acked Write 4 Done Done Sent false
24112 7304 Acked Flush 0 Done Done Sent false
24113 7305 Acked Write 2 Done Done Sent false
24114 7306 Acked Read 8 Done Done Sent false
24115 7307 Acked Write 9 Done Done Sent false
24116 7308 Acked Read 2 Done Done Sent false
24117 7309 Acked Flush 0 Done Done Sent false
24118 7310 Acked Read 9 Done Done Sent false
24119 7311 Acked Read 9 Done Done Sent false
24120 7312 Acked Read 9 Done Done Sent false
24121 7313 Acked Read 8 Done Done Sent false
24122 7314 Acked Flush 0 Done Done Sent false
24123 7315 Acked Write 4 Done Done Sent false
24124 7316 Acked Write 10 Done Done Sent false
24125 7317 Acked Flush 0 Done Done Sent false
24126 7318 Acked Write 8 Done Done Sent false
24127 7319 Acked Read 10 Done Done Sent false
24128 7320 Acked Read 4 Done Done Sent false
24129 7321 Acked Read 5 Done Done Sent false
24130 7322 Acked Read 6 Done Done Sent false
24131 7323 Acked Read 7 Done Done Sent false
24132 7324 Acked Write 1 Done Done Sent false
24133 7325 Acked Read 3 Done Done Sent false
24134 7326 Acked Read 9 Done Done Sent false
24135 7327 Acked Read 1 Done Done Sent false
24136 7328 Acked Read 2 Done Done Sent false
24137 7329 Acked Read 5 Done Done Sent false
24138 7330 Acked Read 6 Done Done Sent false
24139 7331 Acked Write 8 Done Done Sent false
24140 7332 Acked Write 1 Done Done Sent false
24141 7333 Acked Write 10 Done Done Sent false
24142 7334 Acked Write 8 Done Done Sent false
24143 7335 Acked Read 5 Done Done Sent false
24144 7336 Acked Read 5 Done Done Sent false
24145 7337 Acked Read 1 Done Done Sent false
24146 7338 Acked Flush 0 Done Done Sent false
24147 STATES DS:0 DS:1 DS:2 TOTAL
24148 Sent 0 0 301 301
24149 Done 301 301 0 602
24150 Skipped 0 0 0 0
24151 Error 0 0 0 0
24152 Last Flush: 7338 7338 7037
24153 Downstairs last five completed: 7037 7036 7035 7034 7033
24154 Upstairs last five completed: 7338 7337 7336 7335 7334
24155 CLIENT: Up:0 ds:301 act:2
24156 ----------------------------------------------------------------
24157 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
24158 States: Active Active Offline
24159 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
24160 7038 Acked Write 4 Done Done Sent false
24161 7039 Acked Flush 0 Done Done Sent false
24162 7040 Acked Flush 0 Done Done Sent false
24163 7041 Acked Flush 0 Done Done Sent false
24164 7042 Acked Write 5 Done Done Sent false
24165 7043 Acked Write 8 Done Done Sent false
24166 7044 Acked Read 4 Done Done Sent false
24167 7045 Acked Flush 0 Done Done Sent false
24168 7046 Acked Read 6 Done Done Sent false
24169 7047 Acked Write 5 Done Done Sent false
24170 7048 Acked Flush 0 Done Done Sent false
24171 7049 Acked Write 8 Done Done Sent false
24172 7050 Acked Write 2 Done Done Sent false
24173 7051 Acked Flush 0 Done Done Sent false
24174 7052 Acked Read 8 Done Done Sent false
24175 7053 Acked Read 5 Done Done Sent false
24176 7054 Acked Flush 0 Done Done Sent false
24177 7055 Acked Write 8 Done Done Sent false
24178 7056 Acked Write 10 Done Done Sent false
24179 7057 Acked Read 3 Done Done Sent false
24180 7058 Acked Flush 0 Done Done Sent false
24181 7059 Acked Read 1 Done Done Sent false
24182 7060 Acked Read 4 Done Done Sent false
24183 7061 Acked Write 2 Done Done Sent false
24184 7062 Acked Read 4 Done Done Sent false
24185 7063 Acked Read 9 Done Done Sent false
24186 7064 Acked Flush 0 Done Done Sent false
24187 7065 Acked Read 5 Done Done Sent false
24188 7066 Acked Read 8 Done Done Sent false
24189 7067 Acked Write 8 Done Done Sent false
24190 7068 Acked Flush 0 Done Done Sent false
24191 7069 Acked Flush 0 Done Done Sent false
24192 7070 Acked Read 10 Done Done Sent false
24193 7071 Acked Read 8 Done Done Sent false
24194 7072 Acked Read 7 Done Done Sent false
24195 7073 Acked Write 8 Done Done Sent false
24196 7074 Acked Read 10 Done Done Sent false
24197 7075 Acked Read 10 Done Done Sent false
24198 7076 Acked Read 10 Done Done Sent false
24199 7077 Acked Write 8 Done Done Sent false
24200 7078 Acked Read 9 Done Done Sent false
24201 7079 Acked Read 7 Done Done Sent false
24202 7080 Acked Read 1 Done Done Sent false
24203 7081 Acked Read 7 Done Done Sent false
24204 7082 Acked Flush 0 Done Done Sent false
24205 7083 Acked Write 3 Done Done Sent false
24206 7084 Acked Read 7 Done Done Sent false
24207 7085 Acked Write 8 Done Done Sent false
24208 7086 Acked Flush 0 Done Done Sent false
24209 7087 Acked Read 2 Done Done Sent false
24210 7088 Acked Read 5 Done Done Sent false
24211 7089 Acked Write 3 Done Done Sent false
24212 7090 Acked Write 6 Done Done Sent false
24213 7091 Acked Flush 0 Done Done Sent false
24214 7092 Acked Read 1 Done Done Sent false
24215 7093 Acked Write 10 Done Done Sent false
24216 7094 Acked Write 6 Done Done Sent false
24217 7095 Acked Flush 0 Done Done Sent false
24218 7096 Acked Write 7 Done Done Sent false
24219 7097 Acked Read 8 Done Done Sent false
24220 7098 Acked Read 5 Done Done Sent false
24221 7099 Acked Write 3 Done Done Sent false
24222 7100 Acked Write 3 Done Done Sent false
24223 7101 Acked Write 9 Done Done Sent false
24224 7102 Acked Write 1 Done Done Sent false
24225 7103 Acked Flush 0 Done Done Sent false
24226 7104 Acked Read 2 Done Done Sent false
24227 7105 Acked Read 7 Done Done Sent false
24228 7106 Acked Write 7 Done Done Sent false
24229 7107 Acked Write 10 Done Done Sent false
24230 7108 Acked Write 1 Done Done Sent false
24231 7109 Acked Read 1 Done Done Sent false
24232 7110 Acked Write 4 Done Done Sent false
24233 7111 Acked Write 2 Done Done Sent false
24234 7112 Acked Write 2 Done Done Sent false
24235 7113 Acked Read 2 Done Done Sent false
24236 7114 Acked Write 7 Done Done Sent false
24237 7115 Acked Write 4 Done Done Sent false
24238 7116 Acked Write 3 Done Done Sent false
24239 7117 Acked Read 7 Done Done Sent false
24240 7118 Acked Read 7 Done Done Sent false
24241 7119 Acked Read 7 Done Done Sent false
24242 7120 Acked Read 10 Done Done Sent false
24243 7121 Acked Write 4 Done Done Sent false
24244 7122 Acked Write 7 Done Done Sent false
24245 7123 Acked Read 1 Done Done Sent false
24246 7124 Acked Write 4 Done Done Sent false
24247 7125 Acked Flush 0 Done Done Sent false
24248 7126 Acked Read 4 Done Done Sent false
24249 7127 Acked Write 4 Done Done Sent false
24250 7128 Acked Write 10 Done Done Sent false
24251 7129 Acked Write 9 Done Done Sent false
24252 7130 Acked Flush 0 Done Done Sent false
24253 7131 Acked Read 7 Done Done Sent false
24254 7132 Acked Read 5 Done Done Sent false
24255 7133 Acked Read 9 Done Done Sent false
24256 7134 Acked Write 2 Done Done Sent false
24257 7135 Acked Read 6 Done Done Sent false
24258 7136 Acked Read 2 Done Done Sent false
24259 7137 Acked Read 1 Done Done Sent false
24260 7138 Acked Read 2 Done Done Sent false
24261 7139 Acked Read 10 Done Done Sent false
24262 7140 Acked Read 3 Done Done Sent false
24263 7141 Acked Read 2 Done Done Sent false
24264 7142 Acked Write 9 Done Done Sent false
24265 7143 Acked Read 1 Done Done Sent false
24266 7144 Acked Write 6 Done Done Sent false
24267 7145 Acked Flush 0 Done Done Sent false
24268 7146 Acked Write 5 Done Done Sent false
24269 7147 Acked Read 10 Done Done Sent false
24270 7148 Acked Read 10 Done Done Sent false
24271 7149 Acked Read 9 Done Done Sent false
24272 7150 Acked Read 10 Done Done Sent false
24273 7151 Acked Write 6 Done Done Sent false
24274 7152 Acked Write 1 Done Done Sent false
24275 7153 Acked Write 5 Done Done Sent false
24276 7154 Acked Write 7 Done Done Sent false
24277 7155 Acked Write 4 Done Done Sent false
24278 7156 Acked Read 6 Done Done Sent false
24279 7157 Acked Write 6 Done Done Sent false
24280 7158 Acked Read 9 Done Done Sent false
24281 7159 Acked Write 7 Done Done Sent false
24282 7160 Acked Read 1 Done Done Sent false
24283 7161 Acked Write 1 Done Done Sent false
24284 7162 Acked Read 2 Done Done Sent false
24285 7163 Acked Read 2 Done Done Sent false
24286 7164 Acked Write 3 Done Done Sent false
24287 7165 Acked Read 9 Done Done Sent false
24288 7166 Acked Read 1 Done Done Sent false
24289 7167 Acked Read 3 Done Done Sent false
24290 7168 Acked Read 3 Done Done Sent false
24291 7169 Acked Read 9 Done Done Sent false
24292 7170 Acked Read 4 Done Done Sent false
24293 7171 Acked Read 10 Done Done Sent false
24294 7172 Acked Flush 0 Done Done Sent false
24295 7173 Acked Write 7 Done Done Sent false
24296 7174 Acked Write 5 Done Done Sent false
24297 7175 Acked Read 5 Done Done Sent false
24298 7176 Acked Write 10 Done Done Sent false
24299 7177 Acked Read 9 Done Done Sent false
24300 7178 Acked Write 5 Done Done Sent false
24301 7179 Acked Read 10 Done Done Sent false
24302 7180 Acked Read 5 Done Done Sent false
24303 7181 Acked Read 5 Done Done Sent false
24304 7182 Acked Write 4 Done Done Sent false
24305 7183 Acked Read 6 Done Done Sent false
24306 7184 Acked Read 9 Done Done Sent false
24307 7185 Acked Read 8 Done Done Sent false
24308 7186 Acked Write 5 Done Done Sent false
24309 7187 Acked Flush 0 Done Done Sent false
24310 7188 Acked Read 8 Done Done Sent false
24311 7189 Acked Read 2 Done Done Sent false
24312 7190 Acked Read 3 Done Done Sent false
24313 7191 Acked Read 8 Done Done Sent false
24314 7192 Acked Read 10 Done Done Sent false
24315 7193 Acked Write 7 Done Done Sent false
24316 7194 Acked Read 9 Done Done Sent false
24317 7195 Acked Write 3 Done Done Sent false
24318 7196 Acked Write 9 Done Done Sent false
24319 7197 Acked Read 9 Done Done Sent false
24320 7198 Acked Flush 0 Done Done Sent false
24321 7199 Acked Write 10 Done Done Sent false
24322 7200 Acked Write 6 Done Done Sent false
24323 7201 Acked Read 6 Done Done Sent false
24324 7202 Acked Write 8 Done Done Sent false
24325 7203 Acked Write 10 Done Done Sent false
24326 7204 Acked Read 3 Done Done Sent false
24327 7205 Acked Read 1 Done Done Sent false
24328 7206 Acked Flush 0 Done Done Sent false
24329 7207 Acked Write 4 Done Done Sent false
24330 7208 Acked Flush 0 Done Done Sent false
24331 7209 Acked Write 1 Done Done Sent false
24332 7210 Acked Read 7 Done Done Sent false
24333 7211 Acked Write 4 Done Done Sent false
24334 7212 Acked Read 6 Done Done Sent false
24335 7213 Acked Read 7 Done Done Sent false
24336 7214 Acked Read 9 Done Done Sent false
24337 7215 Acked Write 3 Done Done Sent false
24338 7216 Acked Write 7 Done Done Sent false
24339 7217 Acked Read 4 Done Done Sent false
24340 7218 Acked Write 7 Done Done Sent false
24341 7219 Acked Read 9 Done Done Sent false
24342 7220 Acked Read 4 Done Done Sent false
24343 7221 Acked Read 5 Done Done Sent false
24344 7222 Acked Flush 0 Done Done Sent false
24345 7223 Acked Read 10 Done Done Sent false
24346 7224 Acked Read 7 Done Done Sent false
24347 7225 Acked Write 3 Done Done Sent false
24348 7226 Acked Read 2 Done Done Sent false
24349 7227 Acked Read 10 Done Done Sent false
24350 7228 Acked Flush 0 Done Done Sent false
24351 7229 Acked Write 5 Done Done Sent false
24352 7230 Acked Write 5 Done Done Sent false
24353 7231 Acked Flush 0 Done Done Sent false
24354 7232 Acked Write 5 Done Done Sent false
24355 7233 Acked Read 5 Done Done Sent false
24356 7234 Acked Write 8 Done Done Sent false
24357 7235 Acked Write 1 Done Done Sent false
24358 7236 Acked Flush 0 Done Done Sent false
24359 7237 Acked Write 4 Done Done Sent false
24360 7238 Acked Read 2 Done Done Sent false
24361 7239 Acked Read 9 Done Done Sent false
24362 7240 Acked Write 9 Done Done Sent false
24363 7241 Acked Read 8 Done Done Sent false
24364 7242 Acked Read 6 Done Done Sent false
24365 7243 Acked Flush 0 Done Done Sent false
24366 7244 Acked Read 7 Done Done Sent false
24367 7245 Acked Write 2 Done Done Sent false
24368 7246 Acked Write 1 Done Done Sent false
24369 7247 Acked Read 9 Done Done Sent false
24370 7248 Acked Read 1 Done Done Sent false
24371 7249 Acked Read 3 Done Done Sent false
24372 7250 Acked Write 3 Done Done Sent false
24373 7251 Acked Flush 0 Done Done Sent false
24374 7252 Acked Read 4 Done Done Sent false
24375 7253 Acked Read 1 Done Done Sent false
24376 7254 Acked Read 3 Done Done Sent false
24377 7255 Acked Write 5 Done Done Sent false
24378 7256 Acked Write 10 Done Done Sent false
24379 7257 Acked Read 6 Done Done Sent false
24380 7258 Acked Read 1 Done Done Sent false
24381 7259 Acked Read 5 Done Done Sent false
24382 7260 Acked Read 7 Done Done Sent false
24383 7261 Acked Read 2 Done Done Sent false
24384 7262 Acked Read 2 Done Done Sent false
24385 7263 Acked Read 10 Done Done Sent false
24386 7264 Acked Read 9 Done Done Sent false
24387 7265 Acked Flush 0 Done Done Sent false
24388 7266 Acked Read 8 Done Done Sent false
24389 7267 Acked Read 8 Done Done Sent false
24390 7268 Acked Write 7 Done Done Sent false
24391 7269 Acked Read 5 Done Done Sent false
24392 7270 Acked Read 2 Done Done Sent false
24393 7271 Acked Read 1 Done Done Sent false
24394 7272 Acked Write 9 Done Done Sent false
24395 7273 Acked Read 1 Done Done Sent false
24396 7274 Acked Write 9 Done Done Sent false
24397 7275 Acked Read 10 Done Done Sent false
24398 7276 Acked Read 2 Done Done Sent false
24399 7277 Acked Read 5 Done Done Sent false
24400 7278 Acked Flush 0 Done Done Sent false
24401 7279 Acked Write 8 Done Done Sent false
24402 7280 Acked Write 8 Done Done Sent false
24403 7281 Acked Read 3 Done Done Sent false
24404 7282 Acked Write 8 Done Done Sent false
24405 7283 Acked Write 5 Done Done Sent false
24406 7284 Acked Write 1 Done Done Sent false
24407 7285 Acked Read 6 Done Done Sent false
24408 7286 Acked Write 10 Done Done Sent false
24409 7287 Acked Read 1 Done Done Sent false
24410 7288 Acked Read 2 Done Done Sent false
24411 7289 Acked Write 8 Done Done Sent false
24412 7290 Acked Write 7 Done Done Sent false
24413 7291 Acked Read 5 Done Done Sent false
24414 7292 Acked Read 7 Done Done Sent false
24415 7293 Acked Write 8 Done Done Sent false
24416 7294 Acked Read 6 Done Done Sent false
24417 7295 Acked Read 8 Done Done Sent false
24418 7296 Acked Read 5 Done Done Sent false
24419 7297 Acked Read 6 Done Done Sent false
24420 7298 Acked Write 9 Done Done Sent false
24421 7299 Acked Write 10 Done Done Sent false
24422 7300 Acked Read 4 Done Done Sent false
24423 7301 Acked Write 2 Done Done Sent false
24424 7302 Acked Write 2 Done Done Sent false
24425 7303 Acked Write 4 Done Done Sent false
24426 7304 Acked Flush 0 Done Done Sent false
24427 7305 Acked Write 2 Done Done Sent false
24428 7306 Acked Read 8 Done Done Sent false
24429 7307 Acked Write 9 Done Done Sent false
24430 7308 Acked Read 2 Done Done Sent false
24431 7309 Acked Flush 0 Done Done Sent false
24432 7310 Acked Read 9 Done Done Sent false
24433 7311 Acked Read 9 Done Done Sent false
24434 7312 Acked Read 9 Done Done Sent false
24435 7313 Acked Read 8 Done Done Sent false
24436 7314 Acked Flush 0 Done Done Sent false
24437 7315 Acked Write 4 Done Done Sent false
24438 7316 Acked Write 10 Done Done Sent false
24439 7317 Acked Flush 0 Done Done Sent false
24440 7318 Acked Write 8 Done Done Sent false
24441 7319 Acked Read 10 Done Done Sent false
24442 7320 Acked Read 4 Done Done Sent false
24443 7321 Acked Read 5 Done Done Sent false
24444 7322 Acked Read 6 Done Done Sent false
24445 7323 Acked Read 7 Done Done Sent false
24446 7324 Acked Write 1 Done Done Sent false
24447 7325 Acked Read 3 Done Done Sent false
24448 7326 Acked Read 9 Done Done Sent false
24449 7327 Acked Read 1 Done Done Sent false
24450 7328 Acked Read 2 Done Done Sent false
24451 7329 Acked Read 5 Done Done Sent false
24452 7330 Acked Read 6 Done Done Sent false
24453 7331 Acked Write 8 Done Done Sent false
24454 7332 Acked Write 1 Done Done Sent false
24455 7333 Acked Write 10 Done Done Sent false
24456 7334 Acked Write 8 Done Done Sent false
24457 7335 Acked Read 5 Done Done Sent false
24458 7336 Acked Read 5 Done Done Sent false
24459 7337 Acked Read 1 Done Done Sent false
24460 7338 Acked Flush 0 Done Done Sent false
24461 STATES DS:0 DS:1 DS:2 TOTAL
24462 Sent 0 0 301 301
24463 Done 301 301 0 602
24464 Skipped 0 0 0 0
24465 Error 0 0 0 0
24466 Last Flush: 7338 7338 7037
24467 Downstairs last five completed: 7037 7036 7035 7034 7033
24468 Upstairs last five completed: 7338 7337 7336 7335 7334
24469 CLIENT: Up:0 ds:301 act:2
244702024-11-28T04:55:25.887ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
244712024-11-28T04:55:25.887ZWARNcrucible: ds_connection connect to 127.0.0.1:8830 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
244722024-11-28T04:55:25.887ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
244732024-11-28T04:55:25.887ZWARNcrucible: downstairs task for 2 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
244742024-11-28T04:55:25.887ZINFOcrucible: client task is exiting = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
24475 [8830] got start action
24476 [8830] I am going to restart
24477 Starting downstairs at port 8830
24478 Make output file at "/tmp/dsc/downstairs-8830.txt"
24479 [8830][2] reports Starting
24480 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1223)
24481 [8830][2] reports Running
24482 ----------------------------------------------------------------
24483 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
24484 States: Active Active Offline
24485 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
24486 7038 Acked Write 4 Done Done Sent false
24487 7039 Acked Flush 0 Done Done Sent false
24488 7040 Acked Flush 0 Done Done Sent false
24489 7041 Acked Flush 0 Done Done Sent false
24490 7042 Acked Write 5 Done Done Sent false
24491 7043 Acked Write 8 Done Done Sent false
24492 7044 Acked Read 4 Done Done Sent false
24493 7045 Acked Flush 0 Done Done Sent false
24494 7046 Acked Read 6 Done Done Sent false
24495 7047 Acked Write 5 Done Done Sent false
24496 7048 Acked Flush 0 Done Done Sent false
24497 7049 Acked Write 8 Done Done Sent false
24498 7050 Acked Write 2 Done Done Sent false
24499 7051 Acked Flush 0 Done Done Sent false
24500 7052 Acked Read 8 Done Done Sent false
24501 7053 Acked Read 5 Done Done Sent false
24502 7054 Acked Flush 0 Done Done Sent false
24503 7055 Acked Write 8 Done Done Sent false
24504 7056 Acked Write 10 Done Done Sent false
24505 7057 Acked Read 3 Done Done Sent false
24506 7058 Acked Flush 0 Done Done Sent false
24507 7059 Acked Read 1 Done Done Sent false
24508 7060 Acked Read 4 Done Done Sent false
24509 7061 Acked Write 2 Done Done Sent false
24510 7062 Acked Read 4 Done Done Sent false
24511 7063 Acked Read 9 Done Done Sent false
24512 7064 Acked Flush 0 Done Done Sent false
24513 7065 Acked Read 5 Done Done Sent false
24514 7066 Acked Read 8 Done Done Sent false
24515 7067 Acked Write 8 Done Done Sent false
24516 7068 Acked Flush 0 Done Done Sent false
24517 7069 Acked Flush 0 Done Done Sent false
24518 7070 Acked Read 10 Done Done Sent false
24519 7071 Acked Read 8 Done Done Sent false
24520 7072 Acked Read 7 Done Done Sent false
24521 7073 Acked Write 8 Done Done Sent false
24522 7074 Acked Read 10 Done Done Sent false
24523 7075 Acked Read 10 Done Done Sent false
24524 7076 Acked Read 10 Done Done Sent false
24525 7077 Acked Write 8 Done Done Sent false
24526 7078 Acked Read 9 Done Done Sent false
24527 7079 Acked Read 7 Done Done Sent false
24528 7080 Acked Read 1 Done Done Sent false
24529 7081 Acked Read 7 Done Done Sent false
24530 7082 Acked Flush 0 Done Done Sent false
24531 7083 Acked Write 3 Done Done Sent false
24532 7084 Acked Read 7 Done Done Sent false
24533 7085 Acked Write 8 Done Done Sent false
24534 7086 Acked Flush 0 Done Done Sent false
24535 7087 Acked Read 2 Done Done Sent false
24536 7088 Acked Read 5 Done Done Sent false
24537 7089 Acked Write 3 Done Done Sent false
24538 7090 Acked Write 6 Done Done Sent false
24539 7091 Acked Flush 0 Done Done Sent false
24540 7092 Acked Read 1 Done Done Sent false
24541 7093 Acked Write 10 Done Done Sent false
24542 7094 Acked Write 6 Done Done Sent false
24543 7095 Acked Flush 0 Done Done Sent false
24544 7096 Acked Write 7 Done Done Sent false
24545 7097 Acked Read 8 Done Done Sent false
24546 7098 Acked Read 5 Done Done Sent false
24547 7099 Acked Write 3 Done Done Sent false
24548 7100 Acked Write 3 Done Done Sent false
24549 7101 Acked Write 9 Done Done Sent false
24550 7102 Acked Write 1 Done Done Sent false
24551 7103 Acked Flush 0 Done Done Sent false
24552 7104 Acked Read 2 Done Done Sent false
24553 7105 Acked Read 7 Done Done Sent false
24554 7106 Acked Write 7 Done Done Sent false
24555 7107 Acked Write 10 Done Done Sent false
24556 7108 Acked Write 1 Done Done Sent false
24557 7109 Acked Read 1 Done Done Sent false
24558 7110 Acked Write 4 Done Done Sent false
24559 7111 Acked Write 2 Done Done Sent false
24560 7112 Acked Write 2 Done Done Sent false
24561 7113 Acked Read 2 Done Done Sent false
24562 7114 Acked Write 7 Done Done Sent false
24563 7115 Acked Write 4 Done Done Sent false
24564 7116 Acked Write 3 Done Done Sent false
24565 7117 Acked Read 7 Done Done Sent false
24566 7118 Acked Read 7 Done Done Sent false
24567 7119 Acked Read 7 Done Done Sent false
24568 7120 Acked Read 10 Done Done Sent false
24569 7121 Acked Write 4 Done Done Sent false
24570 7122 Acked Write 7 Done Done Sent false
24571 7123 Acked Read 1 Done Done Sent false
24572 7124 Acked Write 4 Done Done Sent false
24573 7125 Acked Flush 0 Done Done Sent false
24574 7126 Acked Read 4 Done Done Sent false
24575 7127 Acked Write 4 Done Done Sent false
24576 7128 Acked Write 10 Done Done Sent false
24577 7129 Acked Write 9 Done Done Sent false
24578 7130 Acked Flush 0 Done Done Sent false
24579 7131 Acked Read 7 Done Done Sent false
24580 7132 Acked Read 5 Done Done Sent false
24581 7133 Acked Read 9 Done Done Sent false
24582 7134 Acked Write 2 Done Done Sent false
24583 7135 Acked Read 6 Done Done Sent false
24584 7136 Acked Read 2 Done Done Sent false
24585 7137 Acked Read 1 Done Done Sent false
24586 7138 Acked Read 2 Done Done Sent false
24587 7139 Acked Read 10 Done Done Sent false
24588 7140 Acked Read 3 Done Done Sent false
24589 7141 Acked Read 2 Done Done Sent false
24590 7142 Acked Write 9 Done Done Sent false
24591 7143 Acked Read 1 Done Done Sent false
24592 7144 Acked Write 6 Done Done Sent false
24593 7145 Acked Flush 0 Done Done Sent false
24594 7146 Acked Write 5 Done Done Sent false
24595 7147 Acked Read 10 Done Done Sent false
24596 7148 Acked Read 10 Done Done Sent false
24597 7149 Acked Read 9 Done Done Sent false
24598 7150 Acked Read 10 Done Done Sent false
24599 7151 Acked Write 6 Done Done Sent false
24600 7152 Acked Write 1 Done Done Sent false
24601 7153 Acked Write 5 Done Done Sent false
24602 7154 Acked Write 7 Done Done Sent false
24603 7155 Acked Write 4 Done Done Sent false
24604 7156 Acked Read 6 Done Done Sent false
24605 7157 Acked Write 6 Done Done Sent false
24606 7158 Acked Read 9 Done Done Sent false
24607 7159 Acked Write 7 Done Done Sent false
24608 7160 Acked Read 1 Done Done Sent false
24609 7161 Acked Write 1 Done Done Sent false
24610 7162 Acked Read 2 Done Done Sent false
24611 7163 Acked Read 2 Done Done Sent false
24612 7164 Acked Write 3 Done Done Sent false
24613 7165 Acked Read 9 Done Done Sent false
24614 7166 Acked Read 1 Done Done Sent false
24615 7167 Acked Read 3 Done Done Sent false
24616 7168 Acked Read 3 Done Done Sent false
24617 7169 Acked Read 9 Done Done Sent false
24618 7170 Acked Read 4 Done Done Sent false
24619 7171 Acked Read 10 Done Done Sent false
24620 7172 Acked Flush 0 Done Done Sent false
24621 7173 Acked Write 7 Done Done Sent false
24622 7174 Acked Write 5 Done Done Sent false
24623 7175 Acked Read 5 Done Done Sent false
24624 7176 Acked Write 10 Done Done Sent false
24625 7177 Acked Read 9 Done Done Sent false
24626 7178 Acked Write 5 Done Done Sent false
24627 7179 Acked Read 10 Done Done Sent false
24628 7180 Acked Read 5 Done Done Sent false
24629 7181 Acked Read 5 Done Done Sent false
24630 7182 Acked Write 4 Done Done Sent false
24631 7183 Acked Read 6 Done Done Sent false
24632 7184 Acked Read 9 Done Done Sent false
24633 7185 Acked Read 8 Done Done Sent false
24634 7186 Acked Write 5 Done Done Sent false
24635 7187 Acked Flush 0 Done Done Sent false
24636 7188 Acked Read 8 Done Done Sent false
24637 7189 Acked Read 2 Done Done Sent false
24638 7190 Acked Read 3 Done Done Sent false
24639 7191 Acked Read 8 Done Done Sent false
24640 7192 Acked Read 10 Done Done Sent false
24641 7193 Acked Write 7 Done Done Sent false
24642 7194 Acked Read 9 Done Done Sent false
24643 7195 Acked Write 3 Done Done Sent false
24644 7196 Acked Write 9 Done Done Sent false
24645 7197 Acked Read 9 Done Done Sent false
24646 7198 Acked Flush 0 Done Done Sent false
24647 7199 Acked Write 10 Done Done Sent false
24648 7200 Acked Write 6 Done Done Sent false
24649 7201 Acked Read 6 Done Done Sent false
24650 7202 Acked Write 8 Done Done Sent false
24651 7203 Acked Write 10 Done Done Sent false
24652 7204 Acked Read 3 Done Done Sent false
24653 7205 Acked Read 1 Done Done Sent false
24654 7206 Acked Flush 0 Done Done Sent false
24655 7207 Acked Write 4 Done Done Sent false
24656 7208 Acked Flush 0 Done Done Sent false
24657 7209 Acked Write 1 Done Done Sent false
24658 7210 Acked Read 7 Done Done Sent false
24659 7211 Acked Write 4 Done Done Sent false
24660 7212 Acked Read 6 Done Done Sent false
24661 7213 Acked Read 7 Done Done Sent false
24662 7214 Acked Read 9 Done Done Sent false
24663 7215 Acked Write 3 Done Done Sent false
24664 7216 Acked Write 7 Done Done Sent false
24665 7217 Acked Read 4 Done Done Sent false
24666 7218 Acked Write 7 Done Done Sent false
24667 7219 Acked Read 9 Done Done Sent false
24668 7220 Acked Read 4 Done Done Sent false
24669 7221 Acked Read 5 Done Done Sent false
24670 7222 Acked Flush 0 Done Done Sent false
24671 7223 Acked Read 10 Done Done Sent false
24672 7224 Acked Read 7 Done Done Sent false
24673 7225 Acked Write 3 Done Done Sent false
24674 7226 Acked Read 2 Done Done Sent false
24675 7227 Acked Read 10 Done Done Sent false
24676 7228 Acked Flush 0 Done Done Sent false
24677 7229 Acked Write 5 Done Done Sent false
24678 7230 Acked Write 5 Done Done Sent false
24679 7231 Acked Flush 0 Done Done Sent false
24680 7232 Acked Write 5 Done Done Sent false
24681 7233 Acked Read 5 Done Done Sent false
24682 7234 Acked Write 8 Done Done Sent false
24683 7235 Acked Write 1 Done Done Sent false
24684 7236 Acked Flush 0 Done Done Sent false
24685 7237 Acked Write 4 Done Done Sent false
24686 7238 Acked Read 2 Done Done Sent false
24687 7239 Acked Read 9 Done Done Sent false
24688 7240 Acked Write 9 Done Done Sent false
24689 7241 Acked Read 8 Done Done Sent false
24690 7242 Acked Read 6 Done Done Sent false
24691 7243 Acked Flush 0 Done Done Sent false
24692 7244 Acked Read 7 Done Done Sent false
24693 7245 Acked Write 2 Done Done Sent false
24694 7246 Acked Write 1 Done Done Sent false
24695 7247 Acked Read 9 Done Done Sent false
24696 7248 Acked Read 1 Done Done Sent false
24697 7249 Acked Read 3 Done Done Sent false
24698 7250 Acked Write 3 Done Done Sent false
24699 7251 Acked Flush 0 Done Done Sent false
24700 7252 Acked Read 4 Done Done Sent false
24701 7253 Acked Read 1 Done Done Sent false
24702 7254 Acked Read 3 Done Done Sent false
24703 7255 Acked Write 5 Done Done Sent false
24704 7256 Acked Write 10 Done Done Sent false
24705 7257 Acked Read 6 Done Done Sent false
24706 7258 Acked Read 1 Done Done Sent false
24707 7259 Acked Read 5 Done Done Sent false
24708 7260 Acked Read 7 Done Done Sent false
24709 7261 Acked Read 2 Done Done Sent false
24710 7262 Acked Read 2 Done Done Sent false
24711 7263 Acked Read 10 Done Done Sent false
24712 7264 Acked Read 9 Done Done Sent false
24713 7265 Acked Flush 0 Done Done Sent false
24714 7266 Acked Read 8 Done Done Sent false
24715 7267 Acked Read 8 Done Done Sent false
24716 7268 Acked Write 7 Done Done Sent false
24717 7269 Acked Read 5 Done Done Sent false
24718 7270 Acked Read 2 Done Done Sent false
24719 7271 Acked Read 1 Done Done Sent false
24720 7272 Acked Write 9 Done Done Sent false
24721 7273 Acked Read 1 Done Done Sent false
24722 7274 Acked Write 9 Done Done Sent false
24723 7275 Acked Read 10 Done Done Sent false
24724 7276 Acked Read 2 Done Done Sent false
24725 7277 Acked Read 5 Done Done Sent false
24726 7278 Acked Flush 0 Done Done Sent false
24727 7279 Acked Write 8 Done Done Sent false
24728 7280 Acked Write 8 Done Done Sent false
24729 7281 Acked Read 3 Done Done Sent false
24730 7282 Acked Write 8 Done Done Sent false
24731 7283 Acked Write 5 Done Done Sent false
24732 7284 Acked Write 1 Done Done Sent false
24733 7285 Acked Read 6 Done Done Sent false
24734 7286 Acked Write 10 Done Done Sent false
24735 7287 Acked Read 1 Done Done Sent false
24736 7288 Acked Read 2 Done Done Sent false
24737 7289 Acked Write 8 Done Done Sent false
24738 7290 Acked Write 7 Done Done Sent false
24739 7291 Acked Read 5 Done Done Sent false
24740 7292 Acked Read 7 Done Done Sent false
24741 7293 Acked Write 8 Done Done Sent false
24742 7294 Acked Read 6 Done Done Sent false
24743 7295 Acked Read 8 Done Done Sent false
24744 7296 Acked Read 5 Done Done Sent false
24745 7297 Acked Read 6 Done Done Sent false
24746 7298 Acked Write 9 Done Done Sent false
24747 7299 Acked Write 10 Done Done Sent false
24748 7300 Acked Read 4 Done Done Sent false
24749 7301 Acked Write 2 Done Done Sent false
24750 7302 Acked Write 2 Done Done Sent false
24751 7303 Acked Write 4 Done Done Sent false
24752 7304 Acked Flush 0 Done Done Sent false
24753 7305 Acked Write 2 Done Done Sent false
24754 7306 Acked Read 8 Done Done Sent false
24755 7307 Acked Write 9 Done Done Sent false
24756 7308 Acked Read 2 Done Done Sent false
24757 7309 Acked Flush 0 Done Done Sent false
24758 7310 Acked Read 9 Done Done Sent false
24759 7311 Acked Read 9 Done Done Sent false
24760 7312 Acked Read 9 Done Done Sent false
24761 7313 Acked Read 8 Done Done Sent false
24762 7314 Acked Flush 0 Done Done Sent false
24763 7315 Acked Write 4 Done Done Sent false
24764 7316 Acked Write 10 Done Done Sent false
24765 7317 Acked Flush 0 Done Done Sent false
24766 7318 Acked Write 8 Done Done Sent false
24767 7319 Acked Read 10 Done Done Sent false
24768 7320 Acked Read 4 Done Done Sent false
24769 7321 Acked Read 5 Done Done Sent false
24770 7322 Acked Read 6 Done Done Sent false
24771 7323 Acked Read 7 Done Done Sent false
24772 7324 Acked Write 1 Done Done Sent false
24773 7325 Acked Read 3 Done Done Sent false
24774 7326 Acked Read 9 Done Done Sent false
24775 7327 Acked Read 1 Done Done Sent false
24776 7328 Acked Read 2 Done Done Sent false
24777 7329 Acked Read 5 Done Done Sent false
24778 7330 Acked Read 6 Done Done Sent false
24779 7331 Acked Write 8 Done Done Sent false
24780 7332 Acked Write 1 Done Done Sent false
24781 7333 Acked Write 10 Done Done Sent false
24782 7334 Acked Write 8 Done Done Sent false
24783 7335 Acked Read 5 Done Done Sent false
24784 7336 Acked Read 5 Done Done Sent false
24785 7337 Acked Read 1 Done Done Sent false
24786 7338 Acked Flush 0 Done Done Sent false
24787 STATES DS:0 DS:1 DS:2 TOTAL
24788 Sent 0 0 301 301
24789 Done 301 301 0 602
24790 Skipped 0 0 0 0
24791 Error 0 0 0 0
24792 Last Flush: 7338 7338 7037
24793 Downstairs last five completed: 7037 7036 7035 7034 7033
24794 Upstairs last five completed: 7338 7337 7336 7335 7334
24795 CLIENT: Up:0 ds:301 act:2
24796 ----------------------------------------------------------------
24797 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
24798 States: Active Active Offline
24799 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
24800 7038 Acked Write 4 Done Done Sent false
24801 7039 Acked Flush 0 Done Done Sent false
24802 7040 Acked Flush 0 Done Done Sent false
24803 7041 Acked Flush 0 Done Done Sent false
24804 7042 Acked Write 5 Done Done Sent false
24805 7043 Acked Write 8 Done Done Sent false
24806 7044 Acked Read 4 Done Done Sent false
24807 7045 Acked Flush 0 Done Done Sent false
24808 7046 Acked Read 6 Done Done Sent false
24809 7047 Acked Write 5 Done Done Sent false
24810 7048 Acked Flush 0 Done Done Sent false
24811 7049 Acked Write 8 Done Done Sent false
24812 7050 Acked Write 2 Done Done Sent false
24813 7051 Acked Flush 0 Done Done Sent false
24814 7052 Acked Read 8 Done Done Sent false
24815 7053 Acked Read 5 Done Done Sent false
24816 7054 Acked Flush 0 Done Done Sent false
24817 7055 Acked Write 8 Done Done Sent false
24818 7056 Acked Write 10 Done Done Sent false
24819 7057 Acked Read 3 Done Done Sent false
24820 7058 Acked Flush 0 Done Done Sent false
24821 7059 Acked Read 1 Done Done Sent false
24822 7060 Acked Read 4 Done Done Sent false
24823 7061 Acked Write 2 Done Done Sent false
24824 7062 Acked Read 4 Done Done Sent false
24825 7063 Acked Read 9 Done Done Sent false
24826 7064 Acked Flush 0 Done Done Sent false
24827 7065 Acked Read 5 Done Done Sent false
24828 7066 Acked Read 8 Done Done Sent false
24829 7067 Acked Write 8 Done Done Sent false
24830 7068 Acked Flush 0 Done Done Sent false
24831 7069 Acked Flush 0 Done Done Sent false
24832 7070 Acked Read 10 Done Done Sent false
24833 7071 Acked Read 8 Done Done Sent false
24834 7072 Acked Read 7 Done Done Sent false
24835 7073 Acked Write 8 Done Done Sent false
24836 7074 Acked Read 10 Done Done Sent false
24837 7075 Acked Read 10 Done Done Sent false
24838 7076 Acked Read 10 Done Done Sent false
24839 7077 Acked Write 8 Done Done Sent false
24840 7078 Acked Read 9 Done Done Sent false
24841 7079 Acked Read 7 Done Done Sent false
24842 7080 Acked Read 1 Done Done Sent false
24843 7081 Acked Read 7 Done Done Sent false
24844 7082 Acked Flush 0 Done Done Sent false
24845 7083 Acked Write 3 Done Done Sent false
24846 7084 Acked Read 7 Done Done Sent false
24847 7085 Acked Write 8 Done Done Sent false
24848 7086 Acked Flush 0 Done Done Sent false
24849 7087 Acked Read 2 Done Done Sent false
24850 7088 Acked Read 5 Done Done Sent false
24851 7089 Acked Write 3 Done Done Sent false
24852 7090 Acked Write 6 Done Done Sent false
24853 7091 Acked Flush 0 Done Done Sent false
24854 7092 Acked Read 1 Done Done Sent false
24855 7093 Acked Write 10 Done Done Sent false
24856 7094 Acked Write 6 Done Done Sent false
24857 7095 Acked Flush 0 Done Done Sent false
24858 7096 Acked Write 7 Done Done Sent false
24859 7097 Acked Read 8 Done Done Sent false
24860 7098 Acked Read 5 Done Done Sent false
24861 7099 Acked Write 3 Done Done Sent false
24862 7100 Acked Write 3 Done Done Sent false
24863 7101 Acked Write 9 Done Done Sent false
24864 7102 Acked Write 1 Done Done Sent false
24865 7103 Acked Flush 0 Done Done Sent false
24866 7104 Acked Read 2 Done Done Sent false
24867 7105 Acked Read 7 Done Done Sent false
24868 7106 Acked Write 7 Done Done Sent false
24869 7107 Acked Write 10 Done Done Sent false
24870 7108 Acked Write 1 Done Done Sent false
24871 7109 Acked Read 1 Done Done Sent false
24872 7110 Acked Write 4 Done Done Sent false
24873 7111 Acked Write 2 Done Done Sent false
24874 7112 Acked Write 2 Done Done Sent false
24875 7113 Acked Read 2 Done Done Sent false
24876 7114 Acked Write 7 Done Done Sent false
24877 7115 Acked Write 4 Done Done Sent false
24878 7116 Acked Write 3 Done Done Sent false
24879 7117 Acked Read 7 Done Done Sent false
24880 7118 Acked Read 7 Done Done Sent false
24881 7119 Acked Read 7 Done Done Sent false
24882 7120 Acked Read 10 Done Done Sent false
24883 7121 Acked Write 4 Done Done Sent false
24884 7122 Acked Write 7 Done Done Sent false
24885 7123 Acked Read 1 Done Done Sent false
24886 7124 Acked Write 4 Done Done Sent false
24887 7125 Acked Flush 0 Done Done Sent false
24888 7126 Acked Read 4 Done Done Sent false
24889 7127 Acked Write 4 Done Done Sent false
24890 7128 Acked Write 10 Done Done Sent false
24891 7129 Acked Write 9 Done Done Sent false
24892 7130 Acked Flush 0 Done Done Sent false
24893 7131 Acked Read 7 Done Done Sent false
24894 7132 Acked Read 5 Done Done Sent false
24895 7133 Acked Read 9 Done Done Sent false
24896 7134 Acked Write 2 Done Done Sent false
24897 7135 Acked Read 6 Done Done Sent false
24898 7136 Acked Read 2 Done Done Sent false
24899 7137 Acked Read 1 Done Done Sent false
24900 7138 Acked Read 2 Done Done Sent false
24901 7139 Acked Read 10 Done Done Sent false
24902 7140 Acked Read 3 Done Done Sent false
24903 7141 Acked Read 2 Done Done Sent false
24904 7142 Acked Write 9 Done Done Sent false
24905 7143 Acked Read 1 Done Done Sent false
24906 7144 Acked Write 6 Done Done Sent false
24907 7145 Acked Flush 0 Done Done Sent false
24908 7146 Acked Write 5 Done Done Sent false
24909 7147 Acked Read 10 Done Done Sent false
24910 7148 Acked Read 10 Done Done Sent false
24911 7149 Acked Read 9 Done Done Sent false
24912 7150 Acked Read 10 Done Done Sent false
24913 7151 Acked Write 6 Done Done Sent false
24914 7152 Acked Write 1 Done Done Sent false
24915 7153 Acked Write 5 Done Done Sent false
24916 7154 Acked Write 7 Done Done Sent false
24917 7155 Acked Write 4 Done Done Sent false
24918 7156 Acked Read 6 Done Done Sent false
24919 7157 Acked Write 6 Done Done Sent false
24920 7158 Acked Read 9 Done Done Sent false
24921 7159 Acked Write 7 Done Done Sent false
24922 7160 Acked Read 1 Done Done Sent false
24923 7161 Acked Write 1 Done Done Sent false
24924 7162 Acked Read 2 Done Done Sent false
24925 7163 Acked Read 2 Done Done Sent false
24926 7164 Acked Write 3 Done Done Sent false
24927 7165 Acked Read 9 Done Done Sent false
24928 7166 Acked Read 1 Done Done Sent false
24929 7167 Acked Read 3 Done Done Sent false
24930 7168 Acked Read 3 Done Done Sent false
24931 7169 Acked Read 9 Done Done Sent false
24932 7170 Acked Read 4 Done Done Sent false
24933 7171 Acked Read 10 Done Done Sent false
24934 7172 Acked Flush 0 Done Done Sent false
24935 7173 Acked Write 7 Done Done Sent false
24936 7174 Acked Write 5 Done Done Sent false
24937 7175 Acked Read 5 Done Done Sent false
24938 7176 Acked Write 10 Done Done Sent false
24939 7177 Acked Read 9 Done Done Sent false
24940 7178 Acked Write 5 Done Done Sent false
24941 7179 Acked Read 10 Done Done Sent false
24942 7180 Acked Read 5 Done Done Sent false
24943 7181 Acked Read 5 Done Done Sent false
24944 7182 Acked Write 4 Done Done Sent false
24945 7183 Acked Read 6 Done Done Sent false
24946 7184 Acked Read 9 Done Done Sent false
24947 7185 Acked Read 8 Done Done Sent false
24948 7186 Acked Write 5 Done Done Sent false
24949 7187 Acked Flush 0 Done Done Sent false
24950 7188 Acked Read 8 Done Done Sent false
24951 7189 Acked Read 2 Done Done Sent false
24952 7190 Acked Read 3 Done Done Sent false
24953 7191 Acked Read 8 Done Done Sent false
24954 7192 Acked Read 10 Done Done Sent false
24955 7193 Acked Write 7 Done Done Sent false
24956 7194 Acked Read 9 Done Done Sent false
24957 7195 Acked Write 3 Done Done Sent false
24958 7196 Acked Write 9 Done Done Sent false
24959 7197 Acked Read 9 Done Done Sent false
24960 7198 Acked Flush 0 Done Done Sent false
24961 7199 Acked Write 10 Done Done Sent false
24962 7200 Acked Write 6 Done Done Sent false
24963 7201 Acked Read 6 Done Done Sent false
24964 7202 Acked Write 8 Done Done Sent false
24965 7203 Acked Write 10 Done Done Sent false
24966 7204 Acked Read 3 Done Done Sent false
24967 7205 Acked Read 1 Done Done Sent false
24968 7206 Acked Flush 0 Done Done Sent false
24969 7207 Acked Write 4 Done Done Sent false
24970 7208 Acked Flush 0 Done Done Sent false
24971 7209 Acked Write 1 Done Done Sent false
24972 7210 Acked Read 7 Done Done Sent false
24973 7211 Acked Write 4 Done Done Sent false
24974 7212 Acked Read 6 Done Done Sent false
24975 7213 Acked Read 7 Done Done Sent false
24976 7214 Acked Read 9 Done Done Sent false
24977 7215 Acked Write 3 Done Done Sent false
24978 7216 Acked Write 7 Done Done Sent false
24979 7217 Acked Read 4 Done Done Sent false
24980 7218 Acked Write 7 Done Done Sent false
24981 7219 Acked Read 9 Done Done Sent false
24982 7220 Acked Read 4 Done Done Sent false
24983 7221 Acked Read 5 Done Done Sent false
24984 7222 Acked Flush 0 Done Done Sent false
24985 7223 Acked Read 10 Done Done Sent false
24986 7224 Acked Read 7 Done Done Sent false
24987 7225 Acked Write 3 Done Done Sent false
24988 7226 Acked Read 2 Done Done Sent false
24989 7227 Acked Read 10 Done Done Sent false
24990 7228 Acked Flush 0 Done Done Sent false
24991 7229 Acked Write 5 Done Done Sent false
24992 7230 Acked Write 5 Done Done Sent false
24993 7231 Acked Flush 0 Done Done Sent false
24994 7232 Acked Write 5 Done Done Sent false
24995 7233 Acked Read 5 Done Done Sent false
24996 7234 Acked Write 8 Done Done Sent false
24997 7235 Acked Write 1 Done Done Sent false
24998 7236 Acked Flush 0 Done Done Sent false
24999 7237 Acked Write 4 Done Done Sent false
25000 7238 Acked Read 2 Done Done Sent false
25001 7239 Acked Read 9 Done Done Sent false
25002 7240 Acked Write 9 Done Done Sent false
25003 7241 Acked Read 8 Done Done Sent false
25004 7242 Acked Read 6 Done Done Sent false
25005 7243 Acked Flush 0 Done Done Sent false
25006 7244 Acked Read 7 Done Done Sent false
25007 7245 Acked Write 2 Done Done Sent false
25008 7246 Acked Write 1 Done Done Sent false
25009 7247 Acked Read 9 Done Done Sent false
25010 7248 Acked Read 1 Done Done Sent false
25011 7249 Acked Read 3 Done Done Sent false
25012 7250 Acked Write 3 Done Done Sent false
25013 7251 Acked Flush 0 Done Done Sent false
25014 7252 Acked Read 4 Done Done Sent false
25015 7253 Acked Read 1 Done Done Sent false
25016 7254 Acked Read 3 Done Done Sent false
25017 7255 Acked Write 5 Done Done Sent false
25018 7256 Acked Write 10 Done Done Sent false
25019 7257 Acked Read 6 Done Done Sent false
25020 7258 Acked Read 1 Done Done Sent false
25021 7259 Acked Read 5 Done Done Sent false
25022 7260 Acked Read 7 Done Done Sent false
25023 7261 Acked Read 2 Done Done Sent false
25024 7262 Acked Read 2 Done Done Sent false
25025 7263 Acked Read 10 Done Done Sent false
25026 7264 Acked Read 9 Done Done Sent false
25027 7265 Acked Flush 0 Done Done Sent false
25028 7266 Acked Read 8 Done Done Sent false
25029 7267 Acked Read 8 Done Done Sent false
25030 7268 Acked Write 7 Done Done Sent false
25031 7269 Acked Read 5 Done Done Sent false
25032 7270 Acked Read 2 Done Done Sent false
25033 7271 Acked Read 1 Done Done Sent false
25034 7272 Acked Write 9 Done Done Sent false
25035 7273 Acked Read 1 Done Done Sent false
25036 7274 Acked Write 9 Done Done Sent false
25037 7275 Acked Read 10 Done Done Sent false
25038 7276 Acked Read 2 Done Done Sent false
25039 7277 Acked Read 5 Done Done Sent false
25040 7278 Acked Flush 0 Done Done Sent false
25041 7279 Acked Write 8 Done Done Sent false
25042 7280 Acked Write 8 Done Done Sent false
25043 7281 Acked Read 3 Done Done Sent false
25044 7282 Acked Write 8 Done Done Sent false
25045 7283 Acked Write 5 Done Done Sent false
25046 7284 Acked Write 1 Done Done Sent false
25047 7285 Acked Read 6 Done Done Sent false
25048 7286 Acked Write 10 Done Done Sent false
25049 7287 Acked Read 1 Done Done Sent false
25050 7288 Acked Read 2 Done Done Sent false
25051 7289 Acked Write 8 Done Done Sent false
25052 7290 Acked Write 7 Done Done Sent false
25053 7291 Acked Read 5 Done Done Sent false
25054 7292 Acked Read 7 Done Done Sent false
25055 7293 Acked Write 8 Done Done Sent false
25056 7294 Acked Read 6 Done Done Sent false
25057 7295 Acked Read 8 Done Done Sent false
25058 7296 Acked Read 5 Done Done Sent false
25059 7297 Acked Read 6 Done Done Sent false
25060 7298 Acked Write 9 Done Done Sent false
25061 7299 Acked Write 10 Done Done Sent false
25062 7300 Acked Read 4 Done Done Sent false
25063 7301 Acked Write 2 Done Done Sent false
25064 7302 Acked Write 2 Done Done Sent false
25065 7303 Acked Write 4 Done Done Sent false
25066 7304 Acked Flush 0 Done Done Sent false
25067 7305 Acked Write 2 Done Done Sent false
25068 7306 Acked Read 8 Done Done Sent false
25069 7307 Acked Write 9 Done Done Sent false
25070 7308 Acked Read 2 Done Done Sent false
25071 7309 Acked Flush 0 Done Done Sent false
25072 7310 Acked Read 9 Done Done Sent false
25073 7311 Acked Read 9 Done Done Sent false
25074 7312 Acked Read 9 Done Done Sent false
25075 7313 Acked Read 8 Done Done Sent false
25076 7314 Acked Flush 0 Done Done Sent false
25077 7315 Acked Write 4 Done Done Sent false
25078 7316 Acked Write 10 Done Done Sent false
25079 7317 Acked Flush 0 Done Done Sent false
25080 7318 Acked Write 8 Done Done Sent false
25081 7319 Acked Read 10 Done Done Sent false
25082 7320 Acked Read 4 Done Done Sent false
25083 7321 Acked Read 5 Done Done Sent false
25084 7322 Acked Read 6 Done Done Sent false
25085 7323 Acked Read 7 Done Done Sent false
25086 7324 Acked Write 1 Done Done Sent false
25087 7325 Acked Read 3 Done Done Sent false
25088 7326 Acked Read 9 Done Done Sent false
25089 7327 Acked Read 1 Done Done Sent false
25090 7328 Acked Read 2 Done Done Sent false
25091 7329 Acked Read 5 Done Done Sent false
25092 7330 Acked Read 6 Done Done Sent false
25093 7331 Acked Write 8 Done Done Sent false
25094 7332 Acked Write 1 Done Done Sent false
25095 7333 Acked Write 10 Done Done Sent false
25096 7334 Acked Write 8 Done Done Sent false
25097 7335 Acked Read 5 Done Done Sent false
25098 7336 Acked Read 5 Done Done Sent false
25099 7337 Acked Read 1 Done Done Sent false
25100 7338 Acked Flush 0 Done Done Sent false
25101 STATES DS:0 DS:1 DS:2 TOTAL
25102 Sent 0 0 301 301
25103 Done 301 301 0 602
25104 Skipped 0 0 0 0
25105 Error 0 0 0 0
25106 Last Flush: 7338 7338 7037
25107 Downstairs last five completed: 7037 7036 7035 7034 7033
25108 Upstairs last five completed: 7338 7337 7336 7335 7334
25109 CLIENT: Up:0 ds:301 act:2
251102024-11-28T04:55:35.889ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
251112024-11-28T04:55:35.889ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:61724) = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
251122024-11-28T04:55:35.890ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
251132024-11-28T04:55:35.890ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
251142024-11-28T04:55:35.890ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
251152024-11-28T04:55:35.890ZINFOcrucible: send last flush ID to this DS: 7037 = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
251162024-11-28T04:55:35.890ZINFOcrucible: Replied this last flush ID: 7037 = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
251172024-11-28T04:55:35.890ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
251182024-11-28T04:55:35.890ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
251192024-11-28T04:55:35.890ZINFOcrucible: [2] client re-new 301 jobs since flush 7037 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
251202024-11-28T04:55:35.891ZINFOcrucible: [2] Replayed 301 jobs since flush: 7037 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
25121 ----------------------------------------------------------------
25122 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
25123 Downstairs last five completed: 7338 7337 7336 7335 7334
25124 Upstairs last five completed: 7338 7337 7336 7335 7334
25125 CLIENT: Up:0 ds:0 act:3
25126 Replay: All jobs finished
25127 Main task has work to do, go find it
25128 got dsc Stop(2)
25129 stop 2
25130 Nov 28 04:55:36.137 [8830] Got stop action so:false kr:false
25131 INFO request completed, latency_us: 333, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 9eef46c5-85d6-4049-9fad-b52a752fdf18, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
25132 Nov 28 04:55:36.138 INFO accepted connection, remote_addr: 127.0.0.1:33629, local_addr: 127.0.0.1:9998
251332024-11-28T04:55:36.138ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
251342024-11-28T04:55:36.138ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
251352024-11-28T04:55:36.138ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
25136 Nov 28 04:55:36.138 {"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-28T04:55:36.138769015Z","hostname":"w-01JDRHKEEFJ1228TFPSBEGXWHY","pid":1202,"client":"2","":"downstairs","session_id":"4a2415b1-6594-436f-82e5-b8cf9a74eeaf"}
25137 INFO request completed, latency_us: 115, response_code: 200, uri: /state/cid/2, method: {"msg":"[2] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-28T04:55:36.13880095Z","hostname":"w-01JDRHKEEFJ1228TFPSBEGXWHY","pid":1202,"client":"2","":"downstairs","session_id":"4a2415b1-6594-436f-82e5-b8cf9a74eeaf"}
25138 GET, req_id: d90e4c3a-7ab5-481a-85a2-3d01f1cfc555, remote_addr: 127.0.0.1:60418, local_addr{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2024-11-28T04:55:36.138869741Z","hostname":"w-01JDRHKEEFJ1228TFPSBEGXWHY","pid":1202,"":"io task","client":"2","":"downstairs","session_id":"4a2415b1-6594-436f-82e5-b8cf9a74eeaf"}
25139 : 127.0.0.1:9998
25140 [8830] Exited with: ExitStatus(unix_wait_status(9))
25141 [8830][2] reports Exit
25142 Nov 28 04:55:40.140 INFO request completed, latency_us: 199, response_code: 200, uri: /state/cid/2, method: GET, req_id: bc3e225a-ff3e-452b-8a4d-c41359f1c75e, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
25143 001/300 Read block 2155 len 8192
25144 002/300 Write block 4178 len 12288 data: 4 5 5
25145 003/300 Write block 744 len 32768 data: 3 3 3 3 3 3 3 3
25146 004/300 Write block 4238 len 24576 data: 4 4 4 4 5 6
25147 005/300 Read block 1399 len 24576
25148 006/300 Read block 4393 len 4096
25149 007/300 Read block 4400 len 24576
25150 008/300 Read block 308 len 32768
25151 009/300 Write block 4725 len 8192 data: 6 6
25152 010/300 Write block 2295 len 16384 data: 6 5 7 6
25153 011/300 Read block 2411 len 20480
25154 012/300 Read block 3142 len 12288
25155 013/300 Write block 1140 len 36864 data: 2 2 2 2 2 3 3 5 4
25156 014/300 Read block 958 len 16384
25157 015/300 Read block 3318 len 8192
25158 016/300 Write block 2070 len 32768 data: 6 5 5 4 3 4 5 5
25159 017/300 Write block 2376 len 20480 data: 9 7 8 8 8
25160 018/300 Write block 3786 len 36864 data: 4 3 4 5 6 7 8 7 6
25161 019/300 Read block 2592 len 32768
25162 020/300 Read block 1503 len 36864
25163 021/300 Write block 11 len 28672 data: 9 6 6 8 8 8 8
25164 022/300 Write block 2380 len 28672 data: 9 10 7 8 8 7 6
25165 023/300 Read block 4802 len 8192
25166 024/300 Read block 225 len 32768
25167 025/300 Read block 635 len 28672
25168 026/300 Write block 2699 len 36864 data: 4 4 6 6 6 5 6 7 4
25169 027/300 Write block 1433 len 16384 data: 3 4 4 6
25170 028/300 Write block 4561 len 8192 data: 4 5
25171 029/300 Read block 4519 len 4096
25172 030/300 Read block 1921 len 4096
25173 031/300 Read block 4581 len 28672
25174 032/300 Read block 59 len 28672
25175 033/300 Read block 2359 len 12288
25176 034/300 Read block 1081 len 28672
25177 035/300 Flush
25178 036/300 Write block 2547 len 40960 data: 2 3 3 4 7 7 8 8 7 8
25179 037/300 Read block 4069 len 4096
25180 038/300 Read block 1715 len 4096
25181 039/300 Write block 436 len 12288 data: 7 5 7
25182 040/300 Write block 1494 len 40960 data: 6 6 5 8 6 6 6 6 5 4
25183 041/300 Write block 745 len 32768 data: 4 4 4 4 4 4 4 4
25184 042/300 Write block 1923 len 4096 data: 4
25185 043/300 Read block 4575 len 20480
25186 044/300 Write block 1329 len 16384 data: 4 4 4 3
25187 045/300 Write block 3131 len 40960 data: 3 3 5 5 5 4 5 7 7 7
25188 046/300 Read block 2546 len 12288
25189 047/300 Read block 1916 len 20480
25190 048/300 Write block 2714 len 16384 data: 8 8 8 9
25191 049/300 Write block 4339 len 12288 data: 4 5 4
25192 050/300 Read block 3026 len 8192
25193 051/300 Read block 2834 len 4096
25194 052/300 Write block 109 len 24576 data: 9 7 4 4 4 4
25195 053/300 Read block 3753 len 32768
25196 054/300 Flush
25197 055/300 Read block 4133 len 16384
25198 056/300 Write block 844 len 24576 data: 3 4 5 5 6 6
25199 057/300 Read block 3105 len 20480
25200 058/300 Read block 1725 len 40960
25201 059/300 Read block 774 len 16384
25202 060/300 Read block 676 len 20480
25203 061/300 Read block 2315 len 36864
25204 062/300 Read block 1593 len 12288
25205 063/300 Write block 2051 len 28672 data: 4 4 4 5 5 5 4
25206 064/300 Read block 2526 len 28672
25207 065/300 Write block 4418 len 24576 data: 5 7 6 5 5 5
25208 066/300 Write block 2485 len 16384 data: 5 4 3 3
25209 067/300 Write block 761 len 8192 data: 6 5
25210 068/300 Read block 214 len 20480
25211 069/300 Write block 1055 len 24576 data: 6 4 5 4 5 6
25212 070/300 Read block 1775 len 8192
25213 071/300 Write block 1345 len 4096 data: 5
25214 072/300 Read block 710 len 32768
25215 073/300 Read block 2873 len 8192
25216 074/300 Write block 2970 len 28672 data: 3 3 4 4 5 6 5
25217 075/300 Read block 1660 len 20480
25218 076/300 Read block 4301 len 8192
25219 077/300 Read block 4513 len 24576
25220 078/300 Read block 2594 len 24576
25221 079/300 Write block 3803 len 28672 data: 6 5 6 5 5 5 5
25222 080/300 Write block 1386 len 24576 data: 4 4 2 2 4 4
25223 081/300 Write block 213 len 40960 data: 5 3 3 3 4 4 4 4 4 4
25224 082/300 Flush
25225 083/300 Flush
25226 084/300 Write block 2810 len 36864 data: 4 4 4 3 3 3 5 4 3
25227 085/300 Read block 2995 len 12288
25228 086/300 Read block 4916 len 16384
25229 087/300 Read block 574 len 20480
25230 088/300 Flush
25231 089/300 Write block 1745 len 20480 data: 4 4 4 4 5
25232 090/300 Write block 4295 len 16384 data: 2 3 3 3
25233 091/300 Read block 3439 len 4096
25234 092/300 Flush
25235 093/300 Flush
25236 094/300 Write block 36 len 36864 data: 5 6 5 4 3 4 5 4 4
25237 095/300 Read block 774 len 32768
25238 096/300 Read block 1357 len 20480
25239 097/300 Write block 4159 len 40960 data: 6 6 6 6 7 6 6 6 6 5
25240 098/300 Read block 4997 len 8192
25241 099/300 Read block 3319 len 12288
25242 100/300 Read block 4381 len 12288
25243 101/300 Write block 263 len 32768 data: 5 5 5 4 3 3 3 4
25244 102/300 Read block 2349 len 32768
25245 103/300 Write block 663 len 40960 data: 9 7 6 7 8 8 6 6 6 7
25246 104/300 Write block 3377 len 32768 data: 6 5 4 2 2 2 2 2
25247 105/300 Write block 3969 len 40960 data: 4 5 4 4 3 3 4 4 3 3
25248 106/300 Flush
25249 107/300 Read block 1469 len 4096
25250 108/300 Write block 1240 len 24576 data: 4 4 5 6 5 5
25251 109/300 Read block 2681 len 24576
25252 110/300 Read block 2583 len 32768
25253 111/300 Flush
25254 112/300 Write block 3148 len 4096 data: 4
25255 113/300 Flush
25256 114/300 Write block 941 len 12288 data: 2 2 3
25257 115/300 Flush
25258 116/300 Write block 120 len 32768 data: 8 7 7 4 3 3 5 4
25259 117/300 Flush
25260 118/300 Flush
25261 119/300 Read block 2958 len 8192
25262 120/300 Write block 2632 len 32768 data: 5 3 3 3 2 3 3 3
25263 121/300 Write block 1227 len 40960 data: 5 4 2 2 2 2 4 4 4 5
25264 122/300 Read block 4468 len 12288
25265 123/300 Flush
25266 124/300 Read block 3090 len 4096
25267 125/300 Write block 3278 len 8192 data: 3 3
25268 126/300 Write block 1991 len 32768 data: 2 2 3 3 3 4 4 3
25269 127/300 Write block 3593 len 32768 data: 5 7 6 4 4 5 6 7
25270 128/300 Read block 732 len 32768
25271 129/300 Write block 3339 len 16384 data: 4 6 8 9
25272 130/300 Read block 1907 len 8192
25273 131/300 Write block 2839 len 32768 data: 2 2 2 2 2 3 5 5
25274 132/300 Write block 1344 len 32768 data: 7 6 4 3 4 4 4 3
25275 133/300 Write block 2538 len 28672 data: 5 4 5 5 5 3 3
25276 134/300 Read block 2187 len 16384
25277 135/300 Read block 2784 len 40960
25278 136/300 Read block 4277 len 40960
25279 137/300 Write block 2562 len 32768 data: 4 5 5 5 5 6 6 6
25280 138/300 Write block 1534 len 28672 data: 5 6 5 6 6 7 8
25281 139/300 Read block 1016 len 24576
25282 140/300 Read block 1206 len 4096
25283 141/300 Read block 4010 len 24576
25284 142/300 Read block 3342 len 36864
25285 143/300 Flush
25286 144/300 Flush
25287 145/300 Write block 1841 len 16384 data: 5 4 4 5
25288 146/300 Read block 3612 len 24576
25289 147/300 Write block 4522 len 24576 data: 9 8 8 8 7 7
25290 148/300 Read block 395 len 20480
25291 149/300 Write block 394 len 20480 data: 5 5 5 6 5
25292 150/300 Write block 2068 len 28672 data: 5 5 7 6 6 5 4
25293 151/300 Flush
25294 152/300 Flush
25295 153/300 Write block 4473 len 4096 data: 6
25296 154/300 Read block 2533 len 40960
25297 155/300 Read block 3172 len 4096
25298 156/300 Write block 3801 len 24576 data: 7 6 7 6 7 6
25299 157/300 Read block 1120 len 20480
25300 158/300 Read block 55 len 40960
25301 159/300 Flush
25302 160/300 Read block 2255 len 32768
25303 161/300 Write block 1887 len 8192 data: 3 4
25304 162/300 Write block 333 len 40960 data: 4 5 8 7 8 9 7 7 8 7
25305 163/300 Read block 3378 len 16384
25306 164/300 Write block 107 len 12288 data: 7 7 10
25307 165/300 Write block 3308 len 40960 data: 5 5 5 5 4 4 6 7 7 8
25308 166/300 Write block 857 len 32768 data: 4 6 7 5 3 6 6 6
25309 167/300 Write block 2186 len 8192 data: 5 6
25310 168/300 Write block 772 len 16384 data: 5 5 4 5
25311 169/300 Read block 4099 len 32768
25312 170/300 Read block 1206 len 24576
25313 171/300 Write block 4993 len 28672 data: 5 5 3 3 4 4 3
25314 172/300 Write block 3844 len 8192 data: 3 4
25315 173/300 Write block 3178 len 8192 data: 6 5
25316 174/300 Write block 4641 len 12288 data: 5 6 7
25317 175/300 Flush
25318 176/300 Read block 3031 len 4096
25319 177/300 Read block 2385 len 8192
25320 178/300 Write block 4345 len 40960 data: 6 6 6 6 5 4 3 4 4 4
25321 179/300 Read block 1704 len 28672
25322 180/300 Write block 3782 len 36864 data: 5 5 4 4 5 4 5 6 7
25323 181/300 Read block 4809 len 20480
25324 182/300 Read block 456 len 4096
25325 183/300 Write block 3137 len 20480 data: 6 8 8 8 7
25326 184/300 Write block 3358 len 8192 data: 5 5
25327 185/300 Read block 4842 len 36864
25328 186/300 Read block 3883 len 28672
25329 187/300 Read block 351 len 20480
25330 188/300 Write block 1565 len 12288 data: 3 3 2
25331 189/300 Read block 457 len 8192
25332 190/300 Read block 4166 len 28672
25333 191/300 Read block 335 len 32768
25334 192/300 Flush
25335 193/300 Read block 1025 len 32768
25336 194/300 Read block 2182 len 4096
25337 195/300 Read block 4512 len 40960
25338 196/300 Write block 2717 len 20480 data: 10 6 7 6 6
25339 197/300 Write block 4957 len 40960 data: 5 6 6 5 5 5 5 6 6 6
25340 198/300 Read block 3398 len 32768
25341 199/300 Read block 2242 len 20480
25342 200/300 Read block 4607 len 12288
25343 201/300 Read block 3456 len 24576
25344 202/300 Write block 1083 len 4096 data: 6
25345 203/300 Flush
25346 204/300 Read block 877 len 36864
25347 205/300 Read block 2724 len 8192
25348 206/300 Read block 1006 len 40960
25349 207/300 Read block 2727 len 8192
25350 208/300 Read block 2776 len 8192
25351 209/300 Read block 912 len 20480
25352 210/300 Read block 1797 len 32768
25353 211/300 Write block 3425 len 20480 data: 6 7 8 8 7
25354 212/300 Write block 320 len 20480 data: 6 7 8 8 7
25355 213/300 Read block 1157 len 16384
25356 214/300 Read block 1366 len 12288
25357 215/300 Write block 3017 len 20480 data: 2 4 4 4 4
25358 216/300 Read block 877 len 32768
25359 217/300 Read block 991 len 8192
25360 218/300 Read block 2839 len 8192
25361 219/300 Write block 3945 len 4096 data: 3
25362 220/300 Read block 45 len 36864
25363 221/300 Write block 1459 len 40960 data: 4 5 6 6 7 7 8 7 7 7
25364 222/300 Read block 3430 len 24576
25365 223/300 Write block 2923 len 8192 data: 6 6
25366 224/300 Write block 4820 len 12288 data: 4 4 3
25367 225/300 Write block 2247 len 20480 data: 4 7 7 6 4
25368 226/300 Flush
25369 227/300 Write block 1784 len 28672 data: 6 5 6 5 4 6 6
25370 228/300 Read block 3071 len 32768
25371 229/300 Write block 1349 len 16384 data: 5 5 4 3
25372 230/300 Write block 1544 len 12288 data: 4 7 6
25373 231/300 Read block 4539 len 36864
25374 232/300 Read block 4164 len 40960
25375 233/300 Write block 132 len 24576 data: 4 4 4 6 6 6
25376 234/300 Write block 4673 len 16384 data: 5 6 6 6
25377 235/300 Write block 136 len 36864 data: 7 7 6 6 7 6 6 6 5
25378 236/300 Read block 121 len 32768
25379 237/300 Read block 234 len 16384
25380 238/300 Read block 4026 len 20480
25381 239/300 Write block 3969 len 28672 data: 5 6 5 5 4 4 5
25382 240/300 Write block 4801 len 36864 data: 5 5 6 5 7 7 5 4 4
25383 241/300 Read block 3167 len 4096
25384 242/300 Read block 4241 len 40960
25385 243/300 Write block 3334 len 12288 data: 4 4 5
25386 244/300 Write block 4349 len 24576 data: 6 5 4 5 5 5
25387 245/300 Read block 736 len 36864
25388 246/300 Write block 3805 len 20480 data: 8 7 6 6 6
25389 247/300 Read block 4942 len 8192
25390 248/300 Write block 894 len 8192 data: 5 5
25391 249/300 Write block 4937 len 28672 data: 5 5 5 3 3 3 3
25392 250/300 Read block 789 len 28672
25393 251/300 Write block 523 len 24576 data: 5 7 6 6 7 5
25394 252/300 Write block 1290 len 40960 data: 3 3 3 2 2 2 2 3 4 5
25395 253/300 Write block 278 len 12288 data: 5 5 5
25396 254/300 Read block 2439 len 8192
25397 255/300 Write block 1961 len 32768 data: 6 5 4 5 5 5 5 4
25398 256/300 Write block 2365 len 16384 data: 5 3 3 4
25399 257/300 Read block 1252 len 40960
25400 258/300 Write block 3947 len 40960 data: 4 4 4 6 6 7 7 6 6 7
25401 259/300 Write block 2014 len 4096 data: 5
25402 260/300 Read block 4768 len 32768
25403 261/300 Write block 3900 len 4096 data: 3
25404 262/300 Read block 4386 len 8192
25405 263/300 Read block 2956 len 8192
25406 264/300 Write block 1665 len 36864 data: 3 3 4 4 4 4 4 5 4
25407 265/300 Read block 3369 len 28672
25408 266/300 Read block 3344 len 12288
25409 267/300 Write block 3694 len 28672 data: 8 10 9 8 6 6 4
25410 268/300 Write block 3174 len 8192 data: 7 8
25411 269/300 Read block 1807 len 20480
25412 270/300 Flush
25413 271/300 Write block 4484 len 4096 data: 4
25414 272/300 Flush
25415 273/300 Read block 1229 len 32768
25416 274/300 Read block 3105 len 32768
25417 275/300 Write block 3615 len 32768 data: 6 5 5 6 6 7 7 7
25418 276/300 Read block 4466 len 16384
25419 277/300 Write block 738 len 28672 data: 5 3 3 4 3 3 4
25420 278/300 Write block 1391 len 8192 data: 5 4
25421 279/300 Write block 1020 len 16384 data: 4 5 4 3
25422 280/300 Read block 1035 len 8192
25423 281/300 Read block 2330 len 16384
25424 282/300 Read block 2713 len 40960
25425 283/300 Read block 1014 len 20480
25426 284/300 Read block 1027 len 32768
25427 285/300 Read block 1299 len 32768
25428 286/300 Read block 2615 len 4096
25429 287/300 Flush
25430 288/300 Write block 3525 len 12288 data: 3 3 5
25431 289/300 Write block 4111 len 24576 data: 4 3 4 4 6 5
25432 290/300 Write block 4195 len 12288 data: 5 5 6
25433 291/300 Read block 4379 len 24576
25434 292/300 Read block 4830 len 24576
25435 293/300 Read block 3069 len 12288
25436 294/300 Read block 1854 len 12288
25437 295/300 Flush
25438 296/300 Write block 4260 len 40960 data: 4 3 3 3 4 4 7 6 7 7
25439 297/300 Write block 3859 len 24576 data: 6 6 7 7 5 6
25440 298/300 Write block 505 len 32768 data: 7 9 9 8 6 6 5 5
25441 299/300 Write block 4196 len 28672 data: 6 7 7 8 7 7 7
25442 300/300 Flush
25443 301/300 Read block 1017 len 24576
25444 Main task has work to do, go find it
25445 got dsc Start(2)
25446 start 2
25447 Nov 28 04:55:40.369 INFO request completed, latency_us: 211, response_code: 204, uri: /start/cid/2, method: POST, req_id: 46c5766b-992e-41e5-9c91-3e4f03c0ef07, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
25448 Replay: started 2, returned:Ok(())
25449 ----------------------------------------------------------------
25450 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
25451 States: Active Active Offline
25452 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
25453 7339 Acked Read 2 Done Done Sent false
25454 7340 Acked Write 3 Done Done Sent false
25455 7341 Acked Write 8 Done Done Sent false
25456 7342 Acked Write 6 Done Done Sent false
25457 7343 Acked Read 6 Done Done Sent false
25458 7344 Acked Read 1 Done Done Sent false
25459 7345 Acked Read 6 Done Done Sent false
25460 7346 Acked Read 8 Done Done Sent false
25461 7347 Acked Write 2 Done Done Sent false
25462 7348 Acked Write 4 Done Done Sent false
25463 7349 Acked Read 5 Done Done Sent false
25464 7350 Acked Read 3 Done Done Sent false
25465 7351 Acked Write 9 Done Done Sent false
25466 7352 Acked Read 4 Done Done Sent false
25467 7353 Acked Read 2 Done Done Sent false
25468 7354 Acked Write 8 Done Done Sent false
25469 7355 Acked Write 5 Done Done Sent false
25470 7356 Acked Write 9 Done Done Sent false
25471 7357 Acked Read 8 Done Done Sent false
25472 7358 Acked Read 9 Done Done Sent false
25473 7359 Acked Write 7 Done Done Sent false
25474 7360 Acked Write 7 Done Done Sent false
25475 7361 Acked Read 2 Done Done Sent false
25476 7362 Acked Read 8 Done Done Sent false
25477 7363 Acked Read 7 Done Done Sent false
25478 7364 Acked Write 9 Done Done Sent false
25479 7365 Acked Write 4 Done Done Sent false
25480 7366 Acked Write 2 Done Done Sent false
25481 7367 Acked Read 1 Done Done Sent false
25482 7368 Acked Read 1 Done Done Sent false
25483 7369 Acked Read 7 Done Done Sent false
25484 7370 Acked Read 7 Done Done Sent false
25485 7371 Acked Read 3 Done Done Sent false
25486 7372 Acked Read 7 Done Done Sent false
25487 7373 Acked Flush 0 Done Done Sent false
25488 7374 Acked Write 10 Done Done Sent false
25489 7375 Acked Read 1 Done Done Sent false
25490 7376 Acked Read 1 Done Done Sent false
25491 7377 Acked Write 3 Done Done Sent false
25492 7378 Acked Write 10 Done Done Sent false
25493 7379 Acked Write 8 Done Done Sent false
25494 7380 Acked Write 1 Done Done Sent false
25495 7381 Acked Read 5 Done Done Sent false
25496 7382 Acked Write 4 Done Done Sent false
25497 7383 Acked Write 10 Done Done Sent false
25498 7384 Acked Read 3 Done Done Sent false
25499 7385 Acked Read 5 Done Done Sent false
25500 7386 Acked Write 4 Done Done Sent false
25501 7387 Acked Write 3 Done Done Sent false
25502 7388 Acked Read 2 Done Done Sent false
25503 7389 Acked Read 1 Done Done Sent false
25504 7390 Acked Write 6 Done Done Sent false
25505 7391 Acked Read 8 Done Done Sent false
25506 7392 Acked Flush 0 Done Done Sent false
25507 7393 Acked Read 4 Done Done Sent false
25508 7394 Acked Write 6 Done Done Sent false
25509 7395 Acked Read 5 Done Done Sent false
25510 7396 Acked Read 10 Done Done Sent false
25511 7397 Acked Read 4 Done Done Sent false
25512 7398 Acked Read 5 Done Done Sent false
25513 7399 Acked Read 9 Done Done Sent false
25514 7400 Acked Read 3 Done Done Sent false
25515 7401 Acked Write 7 Done Done Sent false
25516 7402 Acked Read 7 Done Done Sent false
25517 7403 Acked Write 6 Done Done Sent false
25518 7404 Acked Write 4 Done Done Sent false
25519 7405 Acked Write 2 Done Done Sent false
25520 7406 Acked Read 5 Done Done Sent false
25521 7407 Acked Write 6 Done Done Sent false
25522 7408 Acked Read 2 Done Done Sent false
25523 7409 Acked Write 1 Done Done Sent false
25524 7410 Acked Read 8 Done Done Sent false
25525 7411 Acked Read 2 Done Done Sent false
25526 7412 Acked Write 7 Done Done Sent false
25527 7413 Acked Read 5 Done Done Sent false
25528 7414 Acked Read 2 Done Done Sent false
25529 7415 Acked Read 6 Done Done Sent false
25530 7416 Acked Read 6 Done Done Sent false
25531 7417 Acked Write 7 Done Done Sent false
25532 7418 Acked Write 6 Done Done Sent false
25533 7419 Acked Write 10 Done Done Sent false
25534 7420 Acked Flush 0 Done Done Sent false
25535 7421 Acked Flush 0 Done Done Sent false
25536 7422 Acked Write 9 Done Done Sent false
25537 7423 Acked Read 3 Done Done Sent false
25538 7424 Acked Read 4 Done Done Sent false
25539 7425 Acked Read 5 Done Done Sent false
25540 7426 Acked Flush 0 Done Done Sent false
25541 7427 Acked Write 5 Done Done Sent false
25542 7428 Acked Write 4 Done Done Sent false
25543 7429 Acked Read 1 Done Done Sent false
25544 7430 Acked Flush 0 Done Done Sent false
25545 7431 Acked Flush 0 Done Done Sent false
25546 7432 Acked Write 9 Done Done Sent false
25547 7433 Acked Read 8 Done Done Sent false
25548 7434 Acked Read 5 Done Done Sent false
25549 7435 Acked Write 10 Done Done Sent false
25550 7436 Acked Read 2 Done Done Sent false
25551 7437 Acked Read 3 Done Done Sent false
25552 7438 Acked Read 3 Done Done Sent false
25553 7439 Acked Write 8 Done Done Sent false
25554 7440 Acked Read 8 Done Done Sent false
25555 7441 Acked Write 10 Done Done Sent false
25556 7442 Acked Write 8 Done Done Sent false
25557 7443 Acked Write 10 Done Done Sent false
25558 7444 Acked Flush 0 Done Done Sent false
25559 7445 Acked Read 1 Done Done Sent false
25560 7446 Acked Write 6 Done Done Sent false
25561 7447 Acked Read 6 Done Done Sent false
25562 7448 Acked Read 8 Done Done Sent false
25563 7449 Acked Flush 0 Done Done Sent false
25564 7450 Acked Write 1 Done Done Sent false
25565 7451 Acked Flush 0 Done Done Sent false
25566 7452 Acked Write 3 Done Done Sent false
25567 7453 Acked Flush 0 Done Done Sent false
25568 7454 Acked Write 8 Done Done Sent false
25569 7455 Acked Flush 0 Done Done Sent false
25570 7456 Acked Flush 0 Done Done Sent false
25571 7457 Acked Read 2 Done Done Sent false
25572 7458 Acked Write 8 Done Done Sent false
25573 7459 Acked Write 10 Done Done Sent false
25574 7460 Acked Read 3 Done Done Sent false
25575 7461 Acked Flush 0 Done Done Sent false
25576 7462 Acked Read 1 Done Done Sent false
25577 7463 Acked Write 2 Done Done Sent false
25578 7464 Acked Write 8 Done Done Sent false
25579 7465 Acked Write 8 Done Done Sent false
25580 7466 Acked Read 8 Done Done Sent false
25581 7467 Acked Write 4 Done Done Sent false
25582 7468 Acked Read 2 Done Done Sent false
25583 7469 Acked Write 8 Done Done Sent false
25584 7470 Acked Write 8 Done Done Sent false
25585 7471 Acked Write 7 Done Done Sent false
25586 7472 Acked Read 4 Done Done Sent false
25587 7473 Acked Read 10 Done Done Sent false
25588 7474 Acked Read 10 Done Done Sent false
25589 7475 Acked Write 8 Done Done Sent false
25590 7476 Acked Write 7 Done Done Sent false
25591 7477 Acked Read 6 Done Done Sent false
25592 7478 Acked Read 1 Done Done Sent false
25593 7479 Acked Read 6 Done Done Sent false
25594 7480 Acked Read 9 Done Done Sent false
25595 7481 Acked Flush 0 Done Done Sent false
25596 7482 Acked Flush 0 Done Done Sent false
25597 7483 Acked Write 4 Done Done Sent false
25598 7484 Acked Read 6 Done Done Sent false
25599 7485 Acked Write 6 Done Done Sent false
25600 7486 Acked Read 5 Done Done Sent false
25601 7487 Acked Write 5 Done Done Sent false
25602 7488 Acked Write 7 Done Done Sent false
25603 7489 Acked Flush 0 Done Done Sent false
25604 7490 Acked Flush 0 Done Done Sent false
25605 7491 Acked Write 1 Done Done Sent false
25606 7492 Acked Read 10 Done Done Sent false
25607 7493 Acked Read 1 Done Done Sent false
25608 7494 Acked Write 6 Done Done Sent false
25609 7495 Acked Read 5 Done Done Sent false
25610 7496 Acked Read 10 Done Done Sent false
25611 7497 Acked Flush 0 Done Done Sent false
25612 7498 Acked Read 8 Done Done Sent false
25613 7499 Acked Write 2 Done Done Sent false
25614 7500 Acked Write 10 Done Done Sent false
25615 7501 Acked Read 4 Done Done Sent false
25616 7502 Acked Write 3 Done Done Sent false
25617 7503 Acked Write 10 Done Done Sent false
25618 7504 Acked Write 8 Done Done Sent false
25619 7505 Acked Write 2 Done Done Sent false
25620 7506 Acked Write 4 Done Done Sent false
25621 7507 Acked Read 8 Done Done Sent false
25622 7508 Acked Read 6 Done Done Sent false
25623 7509 Acked Write 7 Done Done Sent false
25624 7510 Acked Write 2 Done Done Sent false
25625 7511 Acked Write 2 Done Done Sent false
25626 7512 Acked Write 3 Done Done Sent false
25627 7513 Acked Flush 0 Done Done Sent false
25628 7514 Acked Read 1 Done Done Sent false
25629 7515 Acked Read 2 Done Done Sent false
25630 7516 Acked Write 10 Done Done Sent false
25631 7517 Acked Read 7 Done Done Sent false
25632 7518 Acked Write 9 Done Done Sent false
25633 7519 Acked Read 5 Done Done Sent false
25634 7520 Acked Read 1 Done Done Sent false
25635 7521 Acked Write 5 Done Done Sent false
25636 7522 Acked Write 2 Done Done Sent false
25637 7523 Acked Read 9 Done Done Sent false
25638 7524 Acked Read 7 Done Done Sent false
25639 7525 Acked Read 5 Done Done Sent false
25640 7526 Acked Write 3 Done Done Sent false
25641 7527 Acked Read 2 Done Done Sent false
25642 7528 Acked Read 7 Done Done Sent false
25643 7529 Acked Read 8 Done Done Sent false
25644 7530 Acked Flush 0 Done Done Sent false
25645 7531 Acked Read 8 Done Done Sent false
25646 7532 Acked Read 1 Done Done Sent false
25647 7533 Acked Read 10 Done Done Sent false
25648 7534 Acked Write 5 Done Done Sent false
25649 7535 Acked Write 10 Done Done Sent false
25650 7536 Acked Read 8 Done Done Sent false
25651 7537 Acked Read 5 Done Done Sent false
25652 7538 Acked Read 3 Done Done Sent false
25653 7539 Acked Read 6 Done Done Sent false
25654 7540 Acked Write 1 Done Done Sent false
25655 7541 Acked Flush 0 Done Done Sent false
25656 7542 Acked Read 9 Done Done Sent false
25657 7543 Acked Read 2 Done Done Sent false
25658 7544 Acked Read 10 Done Done Sent false
25659 7545 Acked Read 2 Done Done Sent false
25660 7546 Acked Read 2 Done Done Sent false
25661 7547 Acked Read 5 Done Done Sent false
25662 7548 Acked Read 8 Done Done Sent false
25663 7549 Acked Write 5 Done Done Sent false
25664 7550 Acked Write 5 Done Done Sent false
25665 7551 Acked Read 4 Done Done Sent false
25666 7552 Acked Read 3 Done Done Sent false
25667 7553 Acked Write 5 Done Done Sent false
25668 7554 Acked Read 8 Done Done Sent false
25669 7555 Acked Read 2 Done Done Sent false
25670 7556 Acked Read 2 Done Done Sent false
25671 7557 Acked Write 1 Done Done Sent false
25672 7558 Acked Read 9 Done Done Sent false
25673 7559 Acked Write 10 Done Done Sent false
25674 7560 Acked Read 6 Done Done Sent false
25675 7561 Acked Write 2 Done Done Sent false
25676 7562 Acked Write 3 Done Done Sent false
25677 7563 Acked Write 5 Done Done Sent false
25678 7564 Acked Flush 0 Done Done Sent false
25679 7565 Acked Write 7 Done Done Sent false
25680 7566 Acked Read 8 Done Done Sent false
25681 7567 Acked Write 4 Done Done Sent false
25682 7568 Acked Write 3 Done Done Sent false
25683 7569 Acked Read 9 Done Done Sent false
25684 7570 Acked Read 10 Done Done Sent false
25685 7571 Acked Write 6 Done Done Sent false
25686 7572 Acked Write 4 Done Done Sent false
25687 7573 Acked Write 9 Done Done Sent false
25688 7574 Acked Read 8 Done Done Sent false
25689 7575 Acked Read 4 Done Done Sent false
25690 7576 Acked Read 5 Done Done Sent false
25691 7577 Acked Write 7 Done Done Sent false
25692 7578 Acked Write 9 Done Done Sent false
25693 7579 Acked Read 1 Done Done Sent false
25694 7580 Acked Read 10 Done Done Sent false
25695 7581 Acked Write 3 Done Done Sent false
25696 7582 Acked Write 6 Done Done Sent false
25697 7583 Acked Read 9 Done Done Sent false
25698 7584 Acked Write 5 Done Done Sent false
25699 7585 Acked Read 2 Done Done Sent false
25700 7586 Acked Write 2 Done Done Sent false
25701 7587 Acked Write 7 Done Done Sent false
25702 7588 Acked Read 7 Done Done Sent false
25703 7589 Acked Write 6 Done Done Sent false
25704 7590 Acked Write 10 Done Done Sent false
25705 7591 Acked Write 3 Done Done Sent false
25706 7592 Acked Read 2 Done Done Sent false
25707 7593 Acked Write 8 Done Done Sent false
25708 7594 Acked Write 4 Done Done Sent false
25709 7595 Acked Read 10 Done Done Sent false
25710 7596 Acked Write 10 Done Done Sent false
25711 7597 Acked Write 1 Done Done Sent false
25712 7598 Acked Read 8 Done Done Sent false
25713 7599 Acked Write 1 Done Done Sent false
25714 7600 Acked Read 2 Done Done Sent false
25715 7601 Acked Read 2 Done Done Sent false
25716 7602 Acked Write 9 Done Done Sent false
25717 7603 Acked Read 7 Done Done Sent false
25718 7604 Acked Read 3 Done Done Sent false
25719 7605 Acked Write 7 Done Done Sent false
25720 7606 Acked Write 2 Done Done Sent false
25721 7607 Acked Read 5 Done Done Sent false
25722 7608 Acked Flush 0 Done Done Sent false
25723 7609 Acked Write 1 Done Done Sent false
25724 7610 Acked Flush 0 Done Done Sent false
25725 7611 Acked Read 8 Done Done Sent false
25726 7612 Acked Read 8 Done Done Sent false
25727 7613 Acked Write 8 Done Done Sent false
25728 7614 Acked Read 4 Done Done Sent false
25729 7615 Acked Write 7 Done Done Sent false
25730 7616 Acked Write 2 Done Done Sent false
25731 7617 Acked Write 4 Done Done Sent false
25732 7618 Acked Read 2 Done Done Sent false
25733 7619 Acked Read 4 Done Done Sent false
25734 7620 Acked Read 10 Done Done Sent false
25735 7621 Acked Read 5 Done Done Sent false
25736 7622 Acked Read 8 Done Done Sent false
25737 7623 Acked Read 8 Done Done Sent false
25738 7624 Acked Read 1 Done Done Sent false
25739 7625 Acked Flush 0 Done Done Sent false
25740 7626 Acked Write 3 Done Done Sent false
25741 7627 Acked Write 6 Done Done Sent false
25742 7628 Acked Write 3 Done Done Sent false
25743 7629 Acked Read 6 Done Done Sent false
25744 7630 Acked Read 6 Done Done Sent false
25745 7631 Acked Read 3 Done Done Sent false
25746 7632 Acked Read 3 Done Done Sent false
25747 7633 Acked Flush 0 Done Done Sent false
25748 7634 Acked Write 10 Done Done Sent false
25749 7635 Acked Write 6 Done Done Sent false
25750 7636 Acked Write 8 Done Done Sent false
25751 7637 Acked Write 7 Done Done Sent false
25752 7638 Acked Flush 0 Done Done Sent false
25753 7639 Acked Read 6 Done Done Sent false
25754 STATES DS:0 DS:1 DS:2 TOTAL
25755 Sent 0 0 301 301
25756 Done 301 301 0 602
25757 Skipped 0 0 0 0
25758 Error 0 0 0 0
25759 Last Flush: 7638 7638 7338
25760 Downstairs last five completed: 7338 7337 7336 7335 7334
25761 Upstairs last five completed: 7639 7638 7637 7636 7635
25762 CLIENT: Up:0 ds:301 act:2
25763 [8830] got start action
25764 [8830] I am going to restart
25765 Starting downstairs at port 8830
25766 Make output file at "/tmp/dsc/downstairs-8830.txt"
25767 [8830][2] reports Starting
25768 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1224)
25769 [8830][2] reports Running
25770 ----------------------------------------------------------------
25771 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
25772 States: Active Active Offline
25773 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
25774 7339 Acked Read 2 Done Done Sent false
25775 7340 Acked Write 3 Done Done Sent false
25776 7341 Acked Write 8 Done Done Sent false
25777 7342 Acked Write 6 Done Done Sent false
25778 7343 Acked Read 6 Done Done Sent false
25779 7344 Acked Read 1 Done Done Sent false
25780 7345 Acked Read 6 Done Done Sent false
25781 7346 Acked Read 8 Done Done Sent false
25782 7347 Acked Write 2 Done Done Sent false
25783 7348 Acked Write 4 Done Done Sent false
25784 7349 Acked Read 5 Done Done Sent false
25785 7350 Acked Read 3 Done Done Sent false
25786 7351 Acked Write 9 Done Done Sent false
25787 7352 Acked Read 4 Done Done Sent false
25788 7353 Acked Read 2 Done Done Sent false
25789 7354 Acked Write 8 Done Done Sent false
25790 7355 Acked Write 5 Done Done Sent false
25791 7356 Acked Write 9 Done Done Sent false
25792 7357 Acked Read 8 Done Done Sent false
25793 7358 Acked Read 9 Done Done Sent false
25794 7359 Acked Write 7 Done Done Sent false
25795 7360 Acked Write 7 Done Done Sent false
25796 7361 Acked Read 2 Done Done Sent false
25797 7362 Acked Read 8 Done Done Sent false
25798 7363 Acked Read 7 Done Done Sent false
25799 7364 Acked Write 9 Done Done Sent false
25800 7365 Acked Write 4 Done Done Sent false
25801 7366 Acked Write 2 Done Done Sent false
25802 7367 Acked Read 1 Done Done Sent false
25803 7368 Acked Read 1 Done Done Sent false
25804 7369 Acked Read 7 Done Done Sent false
25805 7370 Acked Read 7 Done Done Sent false
25806 7371 Acked Read 3 Done Done Sent false
25807 7372 Acked Read 7 Done Done Sent false
25808 7373 Acked Flush 0 Done Done Sent false
25809 7374 Acked Write 10 Done Done Sent false
25810 7375 Acked Read 1 Done Done Sent false
25811 7376 Acked Read 1 Done Done Sent false
25812 7377 Acked Write 3 Done Done Sent false
25813 7378 Acked Write 10 Done Done Sent false
25814 7379 Acked Write 8 Done Done Sent false
25815 7380 Acked Write 1 Done Done Sent false
25816 7381 Acked Read 5 Done Done Sent false
25817 7382 Acked Write 4 Done Done Sent false
25818 7383 Acked Write 10 Done Done Sent false
25819 7384 Acked Read 3 Done Done Sent false
25820 7385 Acked Read 5 Done Done Sent false
25821 7386 Acked Write 4 Done Done Sent false
25822 7387 Acked Write 3 Done Done Sent false
25823 7388 Acked Read 2 Done Done Sent false
25824 7389 Acked Read 1 Done Done Sent false
25825 7390 Acked Write 6 Done Done Sent false
25826 7391 Acked Read 8 Done Done Sent false
25827 7392 Acked Flush 0 Done Done Sent false
25828 7393 Acked Read 4 Done Done Sent false
25829 7394 Acked Write 6 Done Done Sent false
25830 7395 Acked Read 5 Done Done Sent false
25831 7396 Acked Read 10 Done Done Sent false
25832 7397 Acked Read 4 Done Done Sent false
25833 7398 Acked Read 5 Done Done Sent false
25834 7399 Acked Read 9 Done Done Sent false
25835 7400 Acked Read 3 Done Done Sent false
25836 7401 Acked Write 7 Done Done Sent false
25837 7402 Acked Read 7 Done Done Sent false
25838 7403 Acked Write 6 Done Done Sent false
25839 7404 Acked Write 4 Done Done Sent false
25840 7405 Acked Write 2 Done Done Sent false
25841 7406 Acked Read 5 Done Done Sent false
25842 7407 Acked Write 6 Done Done Sent false
25843 7408 Acked Read 2 Done Done Sent false
25844 7409 Acked Write 1 Done Done Sent false
25845 7410 Acked Read 8 Done Done Sent false
25846 7411 Acked Read 2 Done Done Sent false
25847 7412 Acked Write 7 Done Done Sent false
25848 7413 Acked Read 5 Done Done Sent false
25849 7414 Acked Read 2 Done Done Sent false
25850 7415 Acked Read 6 Done Done Sent false
25851 7416 Acked Read 6 Done Done Sent false
25852 7417 Acked Write 7 Done Done Sent false
25853 7418 Acked Write 6 Done Done Sent false
25854 7419 Acked Write 10 Done Done Sent false
25855 7420 Acked Flush 0 Done Done Sent false
25856 7421 Acked Flush 0 Done Done Sent false
25857 7422 Acked Write 9 Done Done Sent false
25858 7423 Acked Read 3 Done Done Sent false
25859 7424 Acked Read 4 Done Done Sent false
25860 7425 Acked Read 5 Done Done Sent false
25861 7426 Acked Flush 0 Done Done Sent false
25862 7427 Acked Write 5 Done Done Sent false
25863 7428 Acked Write 4 Done Done Sent false
25864 7429 Acked Read 1 Done Done Sent false
25865 7430 Acked Flush 0 Done Done Sent false
25866 7431 Acked Flush 0 Done Done Sent false
25867 7432 Acked Write 9 Done Done Sent false
25868 7433 Acked Read 8 Done Done Sent false
25869 7434 Acked Read 5 Done Done Sent false
25870 7435 Acked Write 10 Done Done Sent false
25871 7436 Acked Read 2 Done Done Sent false
25872 7437 Acked Read 3 Done Done Sent false
25873 7438 Acked Read 3 Done Done Sent false
25874 7439 Acked Write 8 Done Done Sent false
25875 7440 Acked Read 8 Done Done Sent false
25876 7441 Acked Write 10 Done Done Sent false
25877 7442 Acked Write 8 Done Done Sent false
25878 7443 Acked Write 10 Done Done Sent false
25879 7444 Acked Flush 0 Done Done Sent false
25880 7445 Acked Read 1 Done Done Sent false
25881 7446 Acked Write 6 Done Done Sent false
25882 7447 Acked Read 6 Done Done Sent false
25883 7448 Acked Read 8 Done Done Sent false
25884 7449 Acked Flush 0 Done Done Sent false
25885 7450 Acked Write 1 Done Done Sent false
25886 7451 Acked Flush 0 Done Done Sent false
25887 7452 Acked Write 3 Done Done Sent false
25888 7453 Acked Flush 0 Done Done Sent false
25889 7454 Acked Write 8 Done Done Sent false
25890 7455 Acked Flush 0 Done Done Sent false
25891 7456 Acked Flush 0 Done Done Sent false
25892 7457 Acked Read 2 Done Done Sent false
25893 7458 Acked Write 8 Done Done Sent false
25894 7459 Acked Write 10 Done Done Sent false
25895 7460 Acked Read 3 Done Done Sent false
25896 7461 Acked Flush 0 Done Done Sent false
25897 7462 Acked Read 1 Done Done Sent false
25898 7463 Acked Write 2 Done Done Sent false
25899 7464 Acked Write 8 Done Done Sent false
25900 7465 Acked Write 8 Done Done Sent false
25901 7466 Acked Read 8 Done Done Sent false
25902 7467 Acked Write 4 Done Done Sent false
25903 7468 Acked Read 2 Done Done Sent false
25904 7469 Acked Write 8 Done Done Sent false
25905 7470 Acked Write 8 Done Done Sent false
25906 7471 Acked Write 7 Done Done Sent false
25907 7472 Acked Read 4 Done Done Sent false
25908 7473 Acked Read 10 Done Done Sent false
25909 7474 Acked Read 10 Done Done Sent false
25910 7475 Acked Write 8 Done Done Sent false
25911 7476 Acked Write 7 Done Done Sent false
25912 7477 Acked Read 6 Done Done Sent false
25913 7478 Acked Read 1 Done Done Sent false
25914 7479 Acked Read 6 Done Done Sent false
25915 7480 Acked Read 9 Done Done Sent false
25916 7481 Acked Flush 0 Done Done Sent false
25917 7482 Acked Flush 0 Done Done Sent false
25918 7483 Acked Write 4 Done Done Sent false
25919 7484 Acked Read 6 Done Done Sent false
25920 7485 Acked Write 6 Done Done Sent false
25921 7486 Acked Read 5 Done Done Sent false
25922 7487 Acked Write 5 Done Done Sent false
25923 7488 Acked Write 7 Done Done Sent false
25924 7489 Acked Flush 0 Done Done Sent false
25925 7490 Acked Flush 0 Done Done Sent false
25926 7491 Acked Write 1 Done Done Sent false
25927 7492 Acked Read 10 Done Done Sent false
25928 7493 Acked Read 1 Done Done Sent false
25929 7494 Acked Write 6 Done Done Sent false
25930 7495 Acked Read 5 Done Done Sent false
25931 7496 Acked Read 10 Done Done Sent false
25932 7497 Acked Flush 0 Done Done Sent false
25933 7498 Acked Read 8 Done Done Sent false
25934 7499 Acked Write 2 Done Done Sent false
25935 7500 Acked Write 10 Done Done Sent false
25936 7501 Acked Read 4 Done Done Sent false
25937 7502 Acked Write 3 Done Done Sent false
25938 7503 Acked Write 10 Done Done Sent false
25939 7504 Acked Write 8 Done Done Sent false
25940 7505 Acked Write 2 Done Done Sent false
25941 7506 Acked Write 4 Done Done Sent false
25942 7507 Acked Read 8 Done Done Sent false
25943 7508 Acked Read 6 Done Done Sent false
25944 7509 Acked Write 7 Done Done Sent false
25945 7510 Acked Write 2 Done Done Sent false
25946 7511 Acked Write 2 Done Done Sent false
25947 7512 Acked Write 3 Done Done Sent false
25948 7513 Acked Flush 0 Done Done Sent false
25949 7514 Acked Read 1 Done Done Sent false
25950 7515 Acked Read 2 Done Done Sent false
25951 7516 Acked Write 10 Done Done Sent false
25952 7517 Acked Read 7 Done Done Sent false
25953 7518 Acked Write 9 Done Done Sent false
25954 7519 Acked Read 5 Done Done Sent false
25955 7520 Acked Read 1 Done Done Sent false
25956 7521 Acked Write 5 Done Done Sent false
25957 7522 Acked Write 2 Done Done Sent false
25958 7523 Acked Read 9 Done Done Sent false
25959 7524 Acked Read 7 Done Done Sent false
25960 7525 Acked Read 5 Done Done Sent false
25961 7526 Acked Write 3 Done Done Sent false
25962 7527 Acked Read 2 Done Done Sent false
25963 7528 Acked Read 7 Done Done Sent false
25964 7529 Acked Read 8 Done Done Sent false
25965 7530 Acked Flush 0 Done Done Sent false
25966 7531 Acked Read 8 Done Done Sent false
25967 7532 Acked Read 1 Done Done Sent false
25968 7533 Acked Read 10 Done Done Sent false
25969 7534 Acked Write 5 Done Done Sent false
25970 7535 Acked Write 10 Done Done Sent false
25971 7536 Acked Read 8 Done Done Sent false
25972 7537 Acked Read 5 Done Done Sent false
25973 7538 Acked Read 3 Done Done Sent false
25974 7539 Acked Read 6 Done Done Sent false
25975 7540 Acked Write 1 Done Done Sent false
25976 7541 Acked Flush 0 Done Done Sent false
25977 7542 Acked Read 9 Done Done Sent false
25978 7543 Acked Read 2 Done Done Sent false
25979 7544 Acked Read 10 Done Done Sent false
25980 7545 Acked Read 2 Done Done Sent false
25981 7546 Acked Read 2 Done Done Sent false
25982 7547 Acked Read 5 Done Done Sent false
25983 7548 Acked Read 8 Done Done Sent false
25984 7549 Acked Write 5 Done Done Sent false
25985 7550 Acked Write 5 Done Done Sent false
25986 7551 Acked Read 4 Done Done Sent false
25987 7552 Acked Read 3 Done Done Sent false
25988 7553 Acked Write 5 Done Done Sent false
25989 7554 Acked Read 8 Done Done Sent false
25990 7555 Acked Read 2 Done Done Sent false
25991 7556 Acked Read 2 Done Done Sent false
25992 7557 Acked Write 1 Done Done Sent false
25993 7558 Acked Read 9 Done Done Sent false
25994 7559 Acked Write 10 Done Done Sent false
25995 7560 Acked Read 6 Done Done Sent false
25996 7561 Acked Write 2 Done Done Sent false
25997 7562 Acked Write 3 Done Done Sent false
25998 7563 Acked Write 5 Done Done Sent false
25999 7564 Acked Flush 0 Done Done Sent false
26000 7565 Acked Write 7 Done Done Sent false
26001 7566 Acked Read 8 Done Done Sent false
26002 7567 Acked Write 4 Done Done Sent false
26003 7568 Acked Write 3 Done Done Sent false
26004 7569 Acked Read 9 Done Done Sent false
26005 7570 Acked Read 10 Done Done Sent false
26006 7571 Acked Write 6 Done Done Sent false
26007 7572 Acked Write 4 Done Done Sent false
26008 7573 Acked Write 9 Done Done Sent false
26009 7574 Acked Read 8 Done Done Sent false
26010 7575 Acked Read 4 Done Done Sent false
26011 7576 Acked Read 5 Done Done Sent false
26012 7577 Acked Write 7 Done Done Sent false
26013 7578 Acked Write 9 Done Done Sent false
26014 7579 Acked Read 1 Done Done Sent false
26015 7580 Acked Read 10 Done Done Sent false
26016 7581 Acked Write 3 Done Done Sent false
26017 7582 Acked Write 6 Done Done Sent false
26018 7583 Acked Read 9 Done Done Sent false
26019 7584 Acked Write 5 Done Done Sent false
26020 7585 Acked Read 2 Done Done Sent false
26021 7586 Acked Write 2 Done Done Sent false
26022 7587 Acked Write 7 Done Done Sent false
26023 7588 Acked Read 7 Done Done Sent false
26024 7589 Acked Write 6 Done Done Sent false
26025 7590 Acked Write 10 Done Done Sent false
26026 7591 Acked Write 3 Done Done Sent false
26027 7592 Acked Read 2 Done Done Sent false
26028 7593 Acked Write 8 Done Done Sent false
26029 7594 Acked Write 4 Done Done Sent false
26030 7595 Acked Read 10 Done Done Sent false
26031 7596 Acked Write 10 Done Done Sent false
26032 7597 Acked Write 1 Done Done Sent false
26033 7598 Acked Read 8 Done Done Sent false
26034 7599 Acked Write 1 Done Done Sent false
26035 7600 Acked Read 2 Done Done Sent false
26036 7601 Acked Read 2 Done Done Sent false
26037 7602 Acked Write 9 Done Done Sent false
26038 7603 Acked Read 7 Done Done Sent false
26039 7604 Acked Read 3 Done Done Sent false
26040 7605 Acked Write 7 Done Done Sent false
26041 7606 Acked Write 2 Done Done Sent false
26042 7607 Acked Read 5 Done Done Sent false
26043 7608 Acked Flush 0 Done Done Sent false
26044 7609 Acked Write 1 Done Done Sent false
26045 7610 Acked Flush 0 Done Done Sent false
26046 7611 Acked Read 8 Done Done Sent false
26047 7612 Acked Read 8 Done Done Sent false
26048 7613 Acked Write 8 Done Done Sent false
26049 7614 Acked Read 4 Done Done Sent false
26050 7615 Acked Write 7 Done Done Sent false
26051 7616 Acked Write 2 Done Done Sent false
26052 7617 Acked Write 4 Done Done Sent false
26053 7618 Acked Read 2 Done Done Sent false
26054 7619 Acked Read 4 Done Done Sent false
26055 7620 Acked Read 10 Done Done Sent false
26056 7621 Acked Read 5 Done Done Sent false
26057 7622 Acked Read 8 Done Done Sent false
26058 7623 Acked Read 8 Done Done Sent false
26059 7624 Acked Read 1 Done Done Sent false
26060 7625 Acked Flush 0 Done Done Sent false
26061 7626 Acked Write 3 Done Done Sent false
26062 7627 Acked Write 6 Done Done Sent false
26063 7628 Acked Write 3 Done Done Sent false
26064 7629 Acked Read 6 Done Done Sent false
26065 7630 Acked Read 6 Done Done Sent false
26066 7631 Acked Read 3 Done Done Sent false
26067 7632 Acked Read 3 Done Done Sent false
26068 7633 Acked Flush 0 Done Done Sent false
26069 7634 Acked Write 10 Done Done Sent false
26070 7635 Acked Write 6 Done Done Sent false
26071 7636 Acked Write 8 Done Done Sent false
26072 7637 Acked Write 7 Done Done Sent false
26073 7638 Acked Flush 0 Done Done Sent false
26074 7639 Acked Read 6 Done Done Sent false
26075 7640 Acked Flush 0 Done Done Sent false
26076 STATES DS:0 DS:1 DS:2 TOTAL
26077 Sent 0 0 302 302
26078 Done 302 302 0 604
26079 Skipped 0 0 0 0
26080 Error 0 0 0 0
26081 Last Flush: 7640 7640 7338
26082 Downstairs last five completed: 7338 7337 7336 7335 7334
26083 Upstairs last five completed: 7640 7639 7638 7637 7636
26084 CLIENT: Up:0 ds:302 act:2
260852024-11-28T04:55:46.139ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
260862024-11-28T04:55:46.139ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:38017) = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
260872024-11-28T04:55:46.140ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
260882024-11-28T04:55:46.141ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
260892024-11-28T04:55:46.141ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
260902024-11-28T04:55:46.141ZINFOcrucible: send last flush ID to this DS: 7338 = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
260912024-11-28T04:55:46.141ZINFOcrucible: Replied this last flush ID: 7338 = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
260922024-11-28T04:55:46.141ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
260932024-11-28T04:55:46.141ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
260942024-11-28T04:55:46.141ZINFOcrucible: [2] client re-new 302 jobs since flush 7338 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
260952024-11-28T04:55:46.141ZINFOcrucible: [2] Replayed 302 jobs since flush: 7338 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
26096 ----------------------------------------------------------------
26097 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
26098 Downstairs last five completed: 7640 7639 7638 7637 7636
26099 Upstairs last five completed: 7640 7639 7638 7637 7636
26100 CLIENT: Up:0 ds:0 act:3
26101 Replay: All jobs finished
26102 Main task has work to do, go find it
26103 got dsc Stop(0)
26104 stop 0
26105 Nov 28 04:55:48.373 INFO request completed, latency_us: [8810] Got stop action so:false kr:false
26106 330, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 353fe304-fab5-4df9-859c-0e58702644fc, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
261072024-11-28T04:55:48.374ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
261082024-11-28T04:55:48.374ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
261092024-11-28T04:55:48.374ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
261102024-11-28T04:55:48.374ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
26111 Nov 28 04:55:48.374 INFO request completed{"msg":"[0] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-28T04:55:48.374829901Z","hostname":"w-01JDRHKEEFJ1228TFPSBEGXWHY","pid":1202,"client":"0","":"downstairs","session_id":"4a2415b1-6594-436f-82e5-b8cf9a74eeaf"}
26112 , latency_us: 132, response_code: 200, uri: /state/cid/0, method: GET, req_id: 714affbb-c84b-49ee-ae57-02972e5720a1, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
261132024-11-28T04:55:48.374ZINFOcrucible: client task is exiting = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
26114 [8810] Exited with: ExitStatus(unix_wait_status(9))
26115 [8810][0] reports Exit
26116 Nov 28 04:55:52.376 INFO request completed, latency_us: 201, response_code: 200, uri: /state/cid/0, method: GET, req_id: a5b33398-524b-4d8d-841d-a03cc1a04422, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
26117 001/300 Write block 1888 len 24576 data: 5 4 5 7 8 9
26118 002/300 Write block 4860 len 16384 data: 5 4 6 4
26119 003/300 Flush
26120 004/300 Write block 1098 len 16384 data: 7 6 7 7
26121 005/300 Write block 1896 len 40960 data: 4 4 4 4 4 4 4 4 3 3
26122 006/300 Read block 2035 len 24576
26123 007/300 Read block 1325 len 16384
26124 008/300 Read block 871 len 16384
26125 009/300 Write block 800 len 12288 data: 4 4 4
26126 010/300 Flush
26127 011/300 Read block 1383 len 40960
26128 012/300 Read block 3617 len 24576
26129 013/300 Write block 256 len 36864 data: 5 5 6 5 5 6 6 6 6
26130 014/300 Read block 814 len 40960
26131 015/300 Read block 4767 len 20480
26132 016/300 Read block 1388 len 16384
26133 017/300 Write block 4225 len 28672 data: 4 2 3 5 5 4 4
26134 018/300 Write block 593 len 36864 data: 4 4 6 7 8 7 7 5 6
26135 019/300 Read block 601 len 28672
26136 020/300 Read block 4784 len 12288
26137 021/300 Write block 2278 len 20480 data: 5 5 4 3 4
26138 022/300 Read block 348 len 28672
26139 023/300 Write block 2846 len 16384 data: 6 6 5 5
26140 024/300 Write block 2368 len 24576 data: 5 4 4 4 4 8
26141 025/300 Write block 2440 len 8192 data: 6 5
26142 026/300 Read block 4446 len 12288
26143 027/300 Write block 3713 len 4096 data: 2
26144 028/300 Read block 4859 len 20480
26145 029/300 Write block 4664 len 12288 data: 5 4 2
26146 030/300 Read block 252 len 32768
26147 031/300 Flush
26148 032/300 Read block 1313 len 36864
26149 033/300 Write block 4178 len 12288 data: 5 6 6
26150 034/300 Read block 2154 len 28672
26151 035/300 Read block 3600 len 12288
26152 036/300 Flush
26153 037/300 Write block 4447 len 40960 data: 8 7 9 10 8 7 6 9 9 8
26154 038/300 Read block 2740 len 16384
26155 039/300 Write block 4508 len 8192 data: 6 5
26156 040/300 Write block 4019 len 16384 data: 4 4 4 3
26157 041/300 Write block 4936 len 24576 data: 4 6 6 6 4 4
26158 042/300 Write block 4149 len 16384 data: 3 4 5 6
26159 043/300 Read block 2129 len 32768
26160 044/300 Read block 3303 len 4096
26161 045/300 Flush
26162 046/300 Read block 1386 len 12288
26163 047/300 Read block 3623 len 16384
26164 048/300 Read block 287 len 4096
26165 049/300 Read block 2003 len 12288
26166 050/300 Read block 4341 len 28672
26167 051/300 Read block 2860 len 12288
26168 052/300 Write block 1937 len 28672 data: 4 4 4 4 4 4 3
26169 053/300 Read block 4593 len 12288
26170 054/300 Read block 4284 len 16384
26171 055/300 Read block 3050 len 36864
26172 056/300 Read block 3855 len 20480
26173 057/300 Write block 3966 len 12288 data: 4 5 4
26174 058/300 Write block 3103 len 24576 data: 2 3 4 3 3 3
26175 059/300 Read block 2722 len 32768
26176 060/300 Read block 3772 len 8192
26177 061/300 Write block 1854 len 40960 data: 5 5 5 5 5 4 4 5 4 4
26178 062/300 Write block 1147 len 36864 data: 6 5 6 8 7 7 6 6 5
26179 063/300 Write block 2180 len 40960 data: 3 3 3 5 5 6 6 7 5 5
26180 064/300 Read block 1506 len 4096
26181 065/300 Write block 4621 len 32768 data: 3 4 3 4 6 7 5 4
26182 066/300 Read block 1915 len 40960
26183 067/300 Read block 1017 len 40960
26184 068/300 Write block 3300 len 16384 data: 4 4 5 6
26185 069/300 Read block 491 len 12288
26186 070/300 Read block 1318 len 32768
26187 071/300 Flush
26188 072/300 Flush
26189 073/300 Write block 4673 len 8192 data: 6 7
26190 074/300 Read block 4566 len 20480
26191 075/300 Write block 1964 len 8192 data: 6 6
26192 076/300 Read block 1237 len 12288
26193 077/300 Write block 3794 len 36864 data: 7 5 4 6 9 7 6 8 7
26194 078/300 Flush
26195 079/300 Flush
26196 080/300 Read block 2627 len 20480
26197 081/300 Read block 2574 len 20480
26198 082/300 Flush
26199 083/300 Flush
26200 084/300 Read block 348 len 20480
26201 085/300 Write block 3044 len 32768 data: 5 5 5 5 5 7 7 7
26202 086/300 Write block 3134 len 8192 data: 6 6
26203 087/300 Write block 976 len 24576 data: 4 4 5 3 3 4
26204 088/300 Read block 3855 len 28672
26205 089/300 Write block 2165 len 40960 data: 5 6 4 6 5 8 7 8 8 6
26206 090/300 Read block 3005 len 24576
26207 091/300 Read block 2224 len 16384
26208 092/300 Read block 3025 len 24576
26209 093/300 Write block 799 len 36864 data: 3 5 5 5 6 5 5 6 6
26210 094/300 Write block 4350 len 28672 data: 6 5 6 6 6 4 5
26211 095/300 Read block 3218 len 24576
26212 096/300 Read block 1443 len 40960
26213 097/300 Write block 1682 len 32768 data: 4 4 5 4 4 4 4 4
26214 098/300 Read block 2467 len 28672
26215 099/300 Flush
26216 100/300 Read block 2318 len 24576
26217 101/300 Write block 2505 len 8192 data: 2 5
26218 102/300 Read block 3151 len 20480
26219 103/300 Read block 496 len 12288
26220 104/300 Read block 3949 len 20480
26221 105/300 Write block 496 len 8192 data: 3 3
26222 106/300 Write block 2132 len 12288 data: 3 2 3
26223 107/300 Read block 3931 len 40960
26224 108/300 Write block 4626 len 24576 data: 8 6 5 5 5 4
26225 109/300 Read block 3579 len 8192
26226 110/300 Write block 826 len 12288 data: 7 7 7
26227 111/300 Flush
26228 112/300 Read block 4086 len 24576
26229 113/300 Read block 2242 len 28672
26230 114/300 Write block 3964 len 28672 data: 4 4 5 6 5 6 7
26231 115/300 Read block 2563 len 20480
26232 116/300 Write block 410 len 24576 data: 8 8 7 7 7 6
26233 117/300 Write block 3899 len 4096 data: 3
26234 118/300 Write block 2942 len 16384 data: 4 4 5 5
26235 119/300 Write block 2729 len 40960 data: 2 2 2 2 3 4 6 7 7 8
26236 120/300 Flush
26237 121/300 Read block 1754 len 32768
26238 122/300 Read block 1544 len 16384
26239 123/300 Write block 3388 len 40960 data: 3 3 3 3 3 3 3 2 3 3
26240 124/300 Read block 1398 len 4096
26241 125/300 Write block 1231 len 4096 data: 3
26242 126/300 Write block 2936 len 20480 data: 3 3 3 3 2
26243 127/300 Read block 2128 len 32768
26244 128/300 Write block 3849 len 4096 data: 4
26245 129/300 Write block 2071 len 24576 data: 7 7 6 5 5 6
26246 130/300 Read block 4615 len 24576
26247 131/300 Write block 1696 len 32768 data: 4 3 4 4 4 4 6 9
26248 132/300 Flush
26249 133/300 Read block 282 len 16384
26250 134/300 Read block 427 len 4096
26251 135/300 Read block 2487 len 20480
26252 136/300 Read block 206 len 28672
26253 137/300 Read block 2319 len 20480
26254 138/300 Read block 146 len 16384
26255 139/300 Write block 3726 len 12288 data: 4 5 5
26256 140/300 Read block 4624 len 24576
26257 141/300 Read block 1395 len 16384
26258 142/300 Read block 3339 len 28672
26259 143/300 Read block 3076 len 32768
26260 144/300 Read block 4533 len 16384
26261 145/300 Write block 3184 len 20480 data: 4 4 4 5 5
26262 146/300 Read block 4101 len 4096
26263 147/300 Read block 3220 len 4096
26264 148/300 Read block 253 len 32768
26265 149/300 Write block 4178 len 8192 data: 6 7
26266 150/300 Read block 3504 len 32768
26267 151/300 Flush
26268 152/300 Read block 4568 len 12288
26269 153/300 Write block 1294 len 16384 data: 3 3 3 4
26270 154/300 Flush
26271 155/300 Read block 4623 len 4096
26272 156/300 Write block 4076 len 24576 data: 7 8 9 11 9 10
26273 157/300 Read block 2948 len 4096
26274 158/300 Write block 1576 len 40960 data: 8 7 7 7 7 6 5 6 5 4
26275 159/300 Write block 3112 len 4096 data: 2
26276 160/300 Write block 4064 len 36864 data: 6 6 5 5 5 4 4 5 5
26277 161/300 Read block 1305 len 4096
26278 162/300 Read block 1772 len 16384
26279 163/300 Read block 4893 len 12288
26280 164/300 Read block 3029 len 12288
26281 165/300 Read block 3388 len 32768
26282 166/300 Write block 3700 len 32768 data: 5 4 4 4 4 3 2 2
26283 167/300 Read block 1947 len 40960
26284 168/300 Flush
26285 169/300 Write block 477 len 36864 data: 5 2 2 3 4 3 3 2 3
26286 170/300 Write block 1195 len 20480 data: 4 5 5 5 3
26287 171/300 Flush
26288 172/300 Read block 3756 len 40960
26289 173/300 Read block 1520 len 32768
26290 174/300 Read block 4850 len 8192
26291 175/300 Write block 498 len 36864 data: 3 3 3 5 5 5 5 8 10
26292 176/300 Write block 4588 len 20480 data: 4 3 3 3 3
26293 177/300 Write block 2187 len 16384 data: 8 6 6 6
26294 178/300 Read block 4545 len 20480
26295 179/300 Read block 382 len 32768
26296 180/300 Write block 774 len 32768 data: 5 6 6 5 5 4 3 3
26297 181/300 Write block 4089 len 24576 data: 7 9 10 10 9 8
26298 182/300 Read block 1002 len 16384
26299 183/300 Read block 4266 len 16384
26300 184/300 Read block 767 len 28672
26301 185/300 Write block 4162 len 40960 data: 7 8 7 7 7 7 6 3 5 5
26302 186/300 Read block 1012 len 40960
26303 187/300 Read block 2565 len 28672
26304 188/300 Read block 4910 len 12288
26305 189/300 Write block 34 len 28672 data: 5 5 6 7 6 5 4
26306 190/300 Write block 2122 len 12288 data: 3 5 7
26307 191/300 Write block 2768 len 28672 data: 4 4 5 6 6 8 7
26308 192/300 Flush
26309 193/300 Read block 2097 len 16384
26310 194/300 Write block 3364 len 4096 data: 3
26311 195/300 Read block 2672 len 28672
26312 196/300 Read block 503 len 16384
26313 197/300 Read block 2583 len 32768
26314 198/300 Flush
26315 199/300 Read block 2234 len 20480
26316 200/300 Read block 2727 len 16384
26317 201/300 Write block 4531 len 36864 data: 6 7 4 6 7 6 4 4 3
26318 202/300 Flush
26319 203/300 Write block 1806 len 20480 data: 5 4 4 3 3
26320 204/300 Read block 140 len 16384
26321 205/300 Read block 3597 len 16384
26322 206/300 Read block 3254 len 36864
26323 207/300 Read block 1187 len 40960
26324 208/300 Read block 702 len 24576
26325 209/300 Read block 1005 len 32768
26326 210/300 Read block 1735 len 8192
26327 211/300 Write block 2147 len 12288 data: 5 4 3
26328 212/300 Read block 3658 len 40960
26329 213/300 Write block 2813 len 12288 data: 4 4 4
26330 214/300 Read block 119 len 4096
26331 215/300 Read block 2716 len 36864
26332 216/300 Write block 2659 len 12288 data: 5 5 6
26333 217/300 Read block 131 len 36864
26334 218/300 Write block 1248 len 24576 data: 3 3 4 5 4 5
26335 219/300 Read block 4732 len 12288
26336 220/300 Read block 2358 len 28672
26337 221/300 Write block 564 len 40960 data: 7 6 7 4 4 4 4 4 5 4
26338 222/300 Read block 753 len 12288
26339 223/300 Read block 3108 len 8192
26340 224/300 Read block 619 len 4096
26341 225/300 Write block 4431 len 36864 data: 7 7 7 6 6 5 5 5 5
26342 226/300 Read block 2597 len 8192
26343 227/300 Write block 4910 len 4096 data: 4
26344 228/300 Flush
26345 229/300 Write block 393 len 36864 data: 4 6 6 6 7 6 4 4 4
26346 230/300 Read block 1072 len 28672
26347 231/300 Write block 4867 len 4096 data: 3
26348 232/300 Write block 2532 len 8192 data: 4 4
26349 233/300 Read block 1918 len 32768
26350 234/300 Read block 1856 len 16384
26351 235/300 Write block 2780 len 32768 data: 8 7 5 5 8 7 6 6
26352 236/300 Flush
26353 237/300 Write block 3306 len 36864 data: 4 5 6 6 6 6 5 5 7
26354 238/300 Write block 599 len 28672 data: 8 6 7 8 6 5 4
26355 239/300 Write block 3896 len 36864 data: 3 3 3 4 4 4 6 6 6
26356 240/300 Read block 4132 len 24576
26357 241/300 Read block 3190 len 36864
26358 242/300 Read block 3447 len 24576
26359 243/300 Write block 2822 len 4096 data: 2
26360 244/300 Read block 1473 len 8192
26361 245/300 Write block 2231 len 24576 data: 5 4 6 6 7 7
26362 246/300 Flush
26363 247/300 Read block 581 len 20480
26364 248/300 Write block 562 len 36864 data: 5 7 8 7 8 5 5 5 5
26365 249/300 Write block 4959 len 4096 data: 7
26366 250/300 Read block 4483 len 12288
26367 251/300 Write block 3022 len 24576 data: 5 6 7 6 4 4
26368 252/300 Write block 2012 len 20480 data: 6 4 6 5 5
26369 253/300 Flush
26370 254/300 Read block 2450 len 8192
26371 255/300 Write block 1139 len 8192 data: 3 3
26372 256/300 Write block 302 len 24576 data: 3 3 4 4 5 5
26373 257/300 Read block 375 len 16384
26374 258/300 Write block 3721 len 24576 data: 3 4 4 4 3 5
26375 259/300 Write block 2281 len 32768 data: 4 5 5 6 5 4 4 4
26376 260/300 Write block 602 len 36864 data: 9 7 6 5 3 3 3 4 5
26377 261/300 Flush
26378 262/300 Read block 2759 len 8192
26379 263/300 Write block 1912 len 32768 data: 4 4 5 4 4 4 3 3
26380 264/300 Write block 2417 len 24576 data: 3 3 7 7 7 7
26381 265/300 Flush
26382 266/300 Write block 4790 len 40960 data: 5 5 6 6 5 6 6 8 9 8
26383 267/300 Write block 510 len 24576 data: 7 6 6 6 5 5
26384 268/300 Read block 2028 len 28672
26385 269/300 Write block 2943 len 20480 data: 5 6 6 6 7
26386 270/300 Read block 209 len 28672
26387 271/300 Read block 4944 len 24576
26388 272/300 Read block 4118 len 24576
26389 273/300 Write block 4822 len 24576 data: 4 5 6 6 7 6
26390 274/300 Write block 4026 len 36864 data: 5 5 6 5 5 5 5 7 5
26391 275/300 Write block 3267 len 24576 data: 3 3 2 2 3 3
26392 276/300 Read block 2683 len 40960
26393 277/300 Write block 598 len 36864 data: 8 9 7 8 10 8 7 6 4
26394 278/300 Read block 1067 len 24576
26395 279/300 Read block 3201 len 24576
26396 280/300 Write block 2779 len 8192 data: 9 9
26397 281/300 Write block 2161 len 20480 data: 6 6 6 4 6
26398 282/300 Write block 4074 len 32768 data: 4 7 8 9 10 12 10 11
26399 283/300 Read block 4782 len 20480
26400 284/300 Read block 1381 len 24576
26401 285/300 Write block 2096 len 8192 data: 5 6
26402 286/300 Read block 798 len 28672
26403 287/300 Write block 1372 len 20480 data: 3 3 4 4 4
26404 288/300 Read block 293 len 40960
26405 289/300 Read block 2948 len 24576
26406 290/300 Write block 1504 len 28672 data: 3 3 3 2 3 3 3
26407 291/300 Write block 4092 len 4096 data: 11
26408 292/300 Read block 3097 len 32768
26409 293/300 Read block 783 len 24576
26410 294/300 Write block 3858 len 40960 data: 6 7 7 8 8 6 7 6 7 6
26411 295/300 Write block 4826 len 32768 data: 8 7 6 5 7 5 4 4
26412 296/300 Write block 87 len 36864 data: 3 5 5 4 4 2 2 3 3
26413 297/300 Write block 551 len 28672 data: 5 3 6 6 6 6 7
26414 298/300 Write block 604 len 8192 data: 8 7
26415 299/300 Flush
26416 300/300 Write block 3237 len 20480 data: 6 6 9 7 7
26417 301/300 Read block 2050 len 16384
26418 Main task has work to do, go find it
26419 got dsc Start(0)
26420 start 0
26421 Nov 28 04:55:52.611 INFO request completed, latency_us: 212, response_code: 204, uri: /start/cid/0, method: POST, req_id: 3cd4df16-cc67-43e1-8d00-a4dda71b9bc3, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
26422 Replay: started 0, returned:Ok(())
26423 ----------------------------------------------------------------
26424 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
26425 States: Offline Active Active
26426 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
26427 7641 Acked Write 6 Sent Done Done false
26428 7642 Acked Write 4 Sent Done Done false
26429 7643 Acked Flush 0 Sent Done Done false
26430 7644 Acked Write 4 Sent Done Done false
26431 7645 Acked Write 10 Sent Done Done false
26432 7646 Acked Read 6 Sent Done Done false
26433 7647 Acked Read 4 Sent Done Done false
26434 7648 Acked Read 4 Sent Done Done false
26435 7649 Acked Write 3 Sent Done Done false
26436 7650 Acked Flush 0 Sent Done Done false
26437 7651 Acked Read 10 Sent Done Done false
26438 7652 Acked Read 6 Sent Done Done false
26439 7653 Acked Write 9 Sent Done Done false
26440 7654 Acked Read 10 Sent Done Done false
26441 7655 Acked Read 5 Sent Done Done false
26442 7656 Acked Read 4 Sent Done Done false
26443 7657 Acked Write 7 Sent Done Done false
26444 7658 Acked Write 9 Sent Done Done false
26445 7659 Acked Read 7 Sent Done Done false
26446 7660 Acked Read 3 Sent Done Done false
26447 7661 Acked Write 5 Sent Done Done false
26448 7662 Acked Read 7 Sent Done Done false
26449 7663 Acked Write 4 Sent Done Done false
26450 7664 Acked Write 6 Sent Done Done false
26451 7665 Acked Write 2 Sent Done Done false
26452 7666 Acked Read 3 Sent Done Done false
26453 7667 Acked Write 1 Sent Done Done false
26454 7668 Acked Read 5 Sent Done Done false
26455 7669 Acked Write 3 Sent Done Done false
26456 7670 Acked Read 8 Sent Done Done false
26457 7671 Acked Flush 0 Sent Done Done false
26458 7672 Acked Read 9 Sent Done Done false
26459 7673 Acked Write 3 Sent Done Done false
26460 7674 Acked Read 7 Sent Done Done false
26461 7675 Acked Read 3 Sent Done Done false
26462 7676 Acked Flush 0 Sent Done Done false
26463 7677 Acked Write 10 Sent Done Done false
26464 7678 Acked Read 4 Sent Done Done false
26465 7679 Acked Write 2 Sent Done Done false
26466 7680 Acked Write 4 Sent Done Done false
26467 7681 Acked Write 6 Sent Done Done false
26468 7682 Acked Write 4 Sent Done Done false
26469 7683 Acked Read 8 Sent Done Done false
26470 7684 Acked Read 1 Sent Done Done false
26471 7685 Acked Flush 0 Sent Done Done false
26472 7686 Acked Read 3 Sent Done Done false
26473 7687 Acked Read 4 Sent Done Done false
26474 7688 Acked Read 1 Sent Done Done false
26475 7689 Acked Read 3 Sent Done Done false
26476 7690 Acked Read 7 Sent Done Done false
26477 7691 Acked Read 3 Sent Done Done false
26478 7692 Acked Write 7 Sent Done Done false
26479 7693 Acked Read 3 Sent Done Done false
26480 7694 Acked Read 4 Sent Done Done false
26481 7695 Acked Read 9 Sent Done Done false
26482 7696 Acked Read 5 Sent Done Done false
26483 7697 Acked Write 3 Sent Done Done false
26484 7698 Acked Write 6 Sent Done Done false
26485 7699 Acked Read 8 Sent Done Done false
26486 7700 Acked Read 2 Sent Done Done false
26487 7701 Acked Write 10 Sent Done Done false
26488 7702 Acked Write 9 Sent Done Done false
26489 7703 Acked Write 10 Sent Done Done false
26490 7704 Acked Read 1 Sent Done Done false
26491 7705 Acked Write 8 Sent Done Done false
26492 7706 Acked Read 10 Sent Done Done false
26493 7707 Acked Read 10 Sent Done Done false
26494 7708 Acked Write 4 Sent Done Done false
26495 7709 Acked Read 3 Sent Done Done false
26496 7710 Acked Read 8 Sent Done Done false
26497 7711 Acked Flush 0 Sent Done Done false
26498 7712 Acked Flush 0 Sent Done Done false
26499 7713 Acked Write 2 Sent Done Done false
26500 7714 Acked Read 5 Sent Done Done false
26501 7715 Acked Write 2 Sent Done Done false
26502 7716 Acked Read 3 Sent Done Done false
26503 7717 Acked Write 9 Sent Done Done false
26504 7718 Acked Flush 0 Sent Done Done false
26505 7719 Acked Flush 0 Sent Done Done false
26506 7720 Acked Read 5 Sent Done Done false
26507 7721 Acked Read 5 Sent Done Done false
26508 7722 Acked Flush 0 Sent Done Done false
26509 7723 Acked Flush 0 Sent Done Done false
26510 7724 Acked Read 5 Sent Done Done false
26511 7725 Acked Write 8 Sent Done Done false
26512 7726 Acked Write 2 Sent Done Done false
26513 7727 Acked Write 6 Sent Done Done false
26514 7728 Acked Read 7 Sent Done Done false
26515 7729 Acked Write 10 Sent Done Done false
26516 7730 Acked Read 6 Sent Done Done false
26517 7731 Acked Read 4 Sent Done Done false
26518 7732 Acked Read 6 Sent Done Done false
26519 7733 Acked Write 9 Sent Done Done false
26520 7734 Acked Write 7 Sent Done Done false
26521 7735 Acked Read 6 Sent Done Done false
26522 7736 Acked Read 10 Sent Done Done false
26523 7737 Acked Write 8 Sent Done Done false
26524 7738 Acked Read 7 Sent Done Done false
26525 7739 Acked Flush 0 Sent Done Done false
26526 7740 Acked Read 6 Sent Done Done false
26527 7741 Acked Write 2 Sent Done Done false
26528 7742 Acked Read 5 Sent Done Done false
26529 7743 Acked Read 3 Sent Done Done false
26530 7744 Acked Read 5 Sent Done Done false
26531 7745 Acked Write 2 Sent Done Done false
26532 7746 Acked Write 3 Sent Done Done false
26533 7747 Acked Read 10 Sent Done Done false
26534 7748 Acked Write 6 Sent Done Done false
26535 7749 Acked Read 2 Sent Done Done false
26536 7750 Acked Write 3 Sent Done Done false
26537 7751 Acked Flush 0 Sent Done Done false
26538 7752 Acked Read 6 Sent Done Done false
26539 7753 Acked Read 7 Sent Done Done false
26540 7754 Acked Write 7 Sent Done Done false
26541 7755 Acked Read 5 Sent Done Done false
26542 7756 Acked Write 6 Sent Done Done false
26543 7757 Acked Write 1 Sent Done Done false
26544 7758 Acked Write 4 Sent Done Done false
26545 7759 Acked Write 10 Sent Done Done false
26546 7760 Acked Flush 0 Sent Done Done false
26547 7761 Acked Read 8 Sent Done Done false
26548 7762 Acked Read 4 Sent Done Done false
26549 7763 Acked Write 10 Sent Done Done false
26550 7764 Acked Read 1 Sent Done Done false
26551 7765 Acked Write 1 Sent Done Done false
26552 7766 Acked Write 5 Sent Done Done false
26553 7767 Acked Read 8 Sent Done Done false
26554 7768 Acked Write 1 Sent Done Done false
26555 7769 Acked Write 6 Sent Done Done false
26556 7770 Acked Read 6 Sent Done Done false
26557 7771 Acked Write 8 Sent Done Done false
26558 7772 Acked Flush 0 Sent Done Done false
26559 7773 Acked Read 4 Sent Done Done false
26560 7774 Acked Read 1 Sent Done Done false
26561 7775 Acked Read 5 Sent Done Done false
26562 7776 Acked Read 7 Sent Done Done false
26563 7777 Acked Read 5 Sent Done Done false
26564 7778 Acked Read 4 Sent Done Done false
26565 7779 Acked Write 3 Sent Done Done false
26566 7780 Acked Read 6 Sent Done Done false
26567 7781 Acked Read 4 Sent Done Done false
26568 7782 Acked Read 7 Sent Done Done false
26569 7783 Acked Read 8 Sent Done Done false
26570 7784 Acked Read 4 Sent Done Done false
26571 7785 Acked Write 5 Sent Done Done false
26572 7786 Acked Read 1 Sent Done Done false
26573 7787 Acked Read 1 Sent Done Done false
26574 7788 Acked Read 8 Sent Done Done false
26575 7789 Acked Write 2 Sent Done Done false
26576 7790 Acked Read 8 Sent Done Done false
26577 7791 Acked Flush 0 Sent Done Done false
26578 7792 Acked Read 3 Sent Done Done false
26579 7793 Acked Write 4 Sent Done Done false
26580 7794 Acked Flush 0 Sent Done Done false
26581 7795 Acked Read 1 Sent Done Done false
26582 7796 Acked Write 6 Sent Done Done false
26583 7797 Acked Read 1 Sent Done Done false
26584 7798 Acked Write 10 Sent Done Done false
26585 7799 Acked Write 1 Sent Done Done false
26586 7800 Acked Write 9 Sent Done Done false
26587 7801 Acked Read 1 Sent Done Done false
26588 7802 Acked Read 4 Sent Done Done false
26589 7803 Acked Read 3 Sent Done Done false
26590 7804 Acked Read 3 Sent Done Done false
26591 7805 Acked Read 8 Sent Done Done false
26592 7806 Acked Write 8 Sent Done Done false
26593 7807 Acked Read 10 Sent Done Done false
26594 7808 Acked Flush 0 Sent Done Done false
26595 7809 Acked Write 9 Sent Done Done false
26596 7810 Acked Write 5 Sent Done Done false
26597 7811 Acked Flush 0 Sent Done Done false
26598 7812 Acked Read 10 Sent Done Done false
26599 7813 Acked Read 8 Sent Done Done false
26600 7814 Acked Read 2 Sent Done Done false
26601 7815 Acked Write 9 Sent Done Done false
26602 7816 Acked Write 5 Sent Done Done false
26603 7817 Acked Write 4 Sent Done Done false
26604 7818 Acked Read 5 Sent Done Done false
26605 7819 Acked Read 8 Sent Done Done false
26606 7820 Acked Write 8 Sent Done Done false
26607 7821 Acked Write 6 Sent Done Done false
26608 7822 Acked Read 4 Sent Done Done false
26609 7823 Acked Read 4 Sent Done Done false
26610 7824 Acked Read 7 Sent Done Done false
26611 7825 Acked Write 10 Sent Done Done false
26612 7826 Acked Read 10 Sent Done Done false
26613 7827 Acked Read 7 Sent Done Done false
26614 7828 Acked Read 3 Sent Done Done false
26615 7829 Acked Write 7 Sent Done Done false
26616 7830 Acked Write 3 Sent Done Done false
26617 7831 Acked Write 7 Sent Done Done false
26618 7832 Acked Flush 0 Sent Done Done false
26619 7833 Acked Read 4 Sent Done Done false
26620 7834 Acked Write 1 Sent Done Done false
26621 7835 Acked Read 7 Sent Done Done false
26622 7836 Acked Read 4 Sent Done Done false
26623 7837 Acked Read 8 Sent Done Done false
26624 7838 Acked Flush 0 Sent Done Done false
26625 7839 Acked Read 5 Sent Done Done false
26626 7840 Acked Read 4 Sent Done Done false
26627 7841 Acked Write 9 Sent Done Done false
26628 7842 Acked Flush 0 Sent Done Done false
26629 7843 Acked Write 5 Sent Done Done false
26630 7844 Acked Read 4 Sent Done Done false
26631 7845 Acked Read 4 Sent Done Done false
26632 7846 Acked Read 9 Sent Done Done false
26633 7847 Acked Read 10 Sent Done Done false
26634 7848 Acked Read 6 Sent Done Done false
26635 7849 Acked Read 8 Sent Done Done false
26636 7850 Acked Read 2 Sent Done Done false
26637 7851 Acked Write 3 Sent Done Done false
26638 7852 Acked Read 10 Sent Done Done false
26639 7853 Acked Write 3 Sent Done Done false
26640 7854 Acked Read 1 Sent Done Done false
26641 7855 Acked Read 9 Sent Done Done false
26642 7856 Acked Write 3 Sent Done Done false
26643 7857 Acked Read 9 Sent Done Done false
26644 7858 Acked Write 6 Sent Done Done false
26645 7859 Acked Read 3 Sent Done Done false
26646 7860 Acked Read 7 Sent Done Done false
26647 7861 Acked Write 10 Sent Done Done false
26648 7862 Acked Read 3 Sent Done Done false
26649 7863 Acked Read 2 Sent Done Done false
26650 7864 Acked Read 1 Sent Done Done false
26651 7865 Acked Write 9 Sent Done Done false
26652 7866 Acked Read 2 Sent Done Done false
26653 7867 Acked Write 1 Sent Done Done false
26654 7868 Acked Flush 0 Sent Done Done false
26655 7869 Acked Write 9 Sent Done Done false
26656 7870 Acked Read 7 Sent Done Done false
26657 7871 Acked Write 1 Sent Done Done false
26658 7872 Acked Write 2 Sent Done Done false
26659 7873 Acked Read 8 Sent Done Done false
26660 7874 Acked Read 4 Sent Done Done false
26661 7875 Acked Write 8 Sent Done Done false
26662 7876 Acked Flush 0 Sent Done Done false
26663 7877 Acked Write 9 Sent Done Done false
26664 7878 Acked Write 7 Sent Done Done false
26665 7879 Acked Write 9 Sent Done Done false
26666 7880 Acked Read 6 Sent Done Done false
26667 7881 Acked Read 9 Sent Done Done false
26668 7882 Acked Read 6 Sent Done Done false
26669 7883 Acked Write 1 Sent Done Done false
26670 7884 Acked Read 2 Sent Done Done false
26671 7885 Acked Write 6 Sent Done Done false
26672 7886 Acked Flush 0 Sent Done Done false
26673 7887 Acked Read 5 Sent Done Done false
26674 7888 Acked Write 9 Sent Done Done false
26675 7889 Acked Write 1 Sent Done Done false
26676 7890 Acked Read 3 Sent Done Done false
26677 7891 Acked Write 6 Sent Done Done false
26678 7892 Acked Write 5 Sent Done Done false
26679 7893 Acked Flush 0 Sent Done Done false
26680 7894 Acked Read 2 Sent Done Done false
26681 7895 Acked Write 2 Sent Done Done false
26682 7896 Acked Write 6 Sent Done Done false
26683 7897 Acked Read 4 Sent Done Done false
26684 7898 Acked Write 6 Sent Done Done false
26685 7899 Acked Write 8 Sent Done Done false
26686 7900 Acked Write 9 Sent Done Done false
26687 7901 Acked Flush 0 Sent Done Done false
26688 7902 Acked Read 2 Sent Done Done false
26689 7903 Acked Write 8 Sent Done Done false
26690 7904 Acked Write 6 Sent Done Done false
26691 7905 Acked Flush 0 Sent Done Done false
26692 7906 Acked Write 10 Sent Done Done false
26693 7907 Acked Write 6 Sent Done Done false
26694 7908 Acked Read 7 Sent Done Done false
26695 7909 Acked Write 5 Sent Done Done false
26696 7910 Acked Read 7 Sent Done Done false
26697 7911 Acked Read 6 Sent Done Done false
26698 7912 Acked Read 6 Sent Done Done false
26699 7913 Acked Write 6 Sent Done Done false
26700 7914 Acked Write 9 Sent Done Done false
26701 7915 Acked Write 6 Sent Done Done false
26702 7916 Acked Read 10 Sent Done Done false
26703 7917 Acked Write 9 Sent Done Done false
26704 7918 Acked Read 6 Sent Done Done false
26705 7919 Acked Read 6 Sent Done Done false
26706 7920 Acked Write 2 Sent Done Done false
26707 7921 Acked Write 5 Sent Done Done false
26708 7922 Acked Write 8 Sent Done Done false
26709 7923 Acked Read 5 Sent Done Done false
26710 7924 Acked Read 6 Sent Done Done false
26711 7925 Acked Write 2 Sent Done Done false
26712 7926 Acked Read 7 Sent Done Done false
26713 7927 Acked Write 5 Sent Done Done false
26714 7928 Acked Read 10 Sent Done Done false
26715 7929 Acked Read 6 Sent Done Done false
26716 7930 Acked Write 7 Sent Done Done false
26717 7931 Acked Write 1 Sent Done Done false
26718 7932 Acked Read 8 Sent Done Done false
26719 7933 Acked Read 6 Sent Done Done false
26720 7934 Acked Write 10 Sent Done Done false
26721 7935 Acked Write 8 Sent Done Done false
26722 7936 Acked Write 9 Sent Done Done false
26723 7937 Acked Write 7 Sent Done Done false
26724 7938 Acked Write 2 Sent Done Done false
26725 7939 Acked Flush 0 Sent Done Done false
26726 7940 Acked Write 5 Sent Done Done false
26727 7941 Acked Read 4 Sent Done Done false
26728 STATES DS:0 DS:1 DS:2 TOTAL
26729 Sent 301 0 0 301
26730 Done 0 301 301 602
26731 Skipped 0 0 0 0
26732 Error 0 0 0 0
26733 Last Flush: 7640 7939 7939
26734 Downstairs last five completed: 7640 7639 7638 7637 7636
26735 Upstairs last five completed: 7941 7940 7939 7938 7937
26736 CLIENT: Up:0 ds:301 act:2
26737 ----------------------------------------------------------------
26738 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
26739 States: Offline Active Active
26740 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
26741 7641 Acked Write 6 Sent Done Done false
26742 7642 Acked Write 4 Sent Done Done false
26743 7643 Acked Flush 0 Sent Done Done false
26744 7644 Acked Write 4 Sent Done Done false
26745 7645 Acked Write 10 Sent Done Done false
26746 7646 Acked Read 6 Sent Done Done false
26747 7647 Acked Read 4 Sent Done Done false
26748 7648 Acked Read 4 Sent Done Done false
26749 7649 Acked Write 3 Sent Done Done false
26750 7650 Acked Flush 0 Sent Done Done false
26751 7651 Acked Read 10 Sent Done Done false
26752 7652 Acked Read 6 Sent Done Done false
26753 7653 Acked Write 9 Sent Done Done false
26754 7654 Acked Read 10 Sent Done Done false
26755 7655 Acked Read 5 Sent Done Done false
26756 7656 Acked Read 4 Sent Done Done false
26757 7657 Acked Write 7 Sent Done Done false
26758 7658 Acked Write 9 Sent Done Done false
26759 7659 Acked Read 7 Sent Done Done false
26760 7660 Acked Read 3 Sent Done Done false
26761 7661 Acked Write 5 Sent Done Done false
26762 7662 Acked Read 7 Sent Done Done false
26763 7663 Acked Write 4 Sent Done Done false
26764 7664 Acked Write 6 Sent Done Done false
26765 7665 Acked Write 2 Sent Done Done false
26766 7666 Acked Read 3 Sent Done Done false
26767 7667 Acked Write 1 Sent Done Done false
26768 7668 Acked Read 5 Sent Done Done false
26769 7669 Acked Write 3 Sent Done Done false
26770 7670 Acked Read 8 Sent Done Done false
26771 7671 Acked Flush 0 Sent Done Done false
26772 7672 Acked Read 9 Sent Done Done false
26773 7673 Acked Write 3 Sent Done Done false
26774 7674 Acked Read 7 Sent Done Done false
26775 7675 Acked Read 3 Sent Done Done false
26776 7676 Acked Flush 0 Sent Done Done false
26777 7677 Acked Write 10 Sent Done Done false
26778 7678 Acked Read 4 Sent Done Done false
26779 7679 Acked Write 2 Sent Done Done false
26780 7680 Acked Write 4 Sent Done Done false
26781 7681 Acked Write 6 Sent Done Done false
26782 7682 Acked Write 4 Sent Done Done false
26783 7683 Acked Read 8 Sent Done Done false
26784 7684 Acked Read 1 Sent Done Done false
26785 7685 Acked Flush 0 Sent Done Done false
26786 7686 Acked Read 3 Sent Done Done false
26787 7687 Acked Read 4 Sent Done Done false
26788 7688 Acked Read 1 Sent Done Done false
26789 7689 Acked Read 3 Sent Done Done false
26790 7690 Acked Read 7 Sent Done Done false
26791 7691 Acked Read 3 Sent Done Done false
26792 7692 Acked Write 7 Sent Done Done false
26793 7693 Acked Read 3 Sent Done Done false
26794 7694 Acked Read 4 Sent Done Done false
26795 7695 Acked Read 9 Sent Done Done false
26796 7696 Acked Read 5 Sent Done Done false
26797 7697 Acked Write 3 Sent Done Done false
26798 7698 Acked Write 6 Sent Done Done false
26799 7699 Acked Read 8 Sent Done Done false
26800 7700 Acked Read 2 Sent Done Done false
26801 7701 Acked Write 10 Sent Done Done false
26802 7702 Acked Write 9 Sent Done Done false
26803 7703 Acked Write 10 Sent Done Done false
26804 7704 Acked Read 1 Sent Done Done false
26805 7705 Acked Write 8 Sent Done Done false
26806 7706 Acked Read 10 Sent Done Done false
26807 7707 Acked Read 10 Sent Done Done false
26808 7708 Acked Write 4 Sent Done Done false
26809 7709 Acked Read 3 Sent Done Done false
26810 7710 Acked Read 8 Sent Done Done false
26811 7711 Acked Flush 0 Sent Done Done false
26812 7712 Acked Flush 0 Sent Done Done false
26813 7713 Acked Write 2 Sent Done Done false
26814 7714 Acked Read 5 Sent Done Done false
26815 7715 Acked Write 2 Sent Done Done false
26816 7716 Acked Read 3 Sent Done Done false
26817 7717 Acked Write 9 Sent Done Done false
26818 7718 Acked Flush 0 Sent Done Done false
26819 7719 Acked Flush 0 Sent Done Done false
26820 7720 Acked Read 5 Sent Done Done false
26821 7721 Acked Read 5 Sent Done Done false
26822 7722 Acked Flush 0 Sent Done Done false
26823 7723 Acked Flush 0 Sent Done Done false
26824 7724 Acked Read 5 Sent Done Done false
26825 7725 Acked Write 8 Sent Done Done false
26826 7726 Acked Write 2 Sent Done Done false
26827 7727 Acked Write 6 Sent Done Done false
26828 7728 Acked Read 7 Sent Done Done false
26829 7729 Acked Write 10 Sent Done Done false
26830 7730 Acked Read 6 Sent Done Done false
26831 7731 Acked Read 4 Sent Done Done false
26832 7732 Acked Read 6 Sent Done Done false
26833 7733 Acked Write 9 Sent Done Done false
26834 7734 Acked Write 7 Sent Done Done false
26835 7735 Acked Read 6 Sent Done Done false
26836 7736 Acked Read 10 Sent Done Done false
26837 7737 Acked Write 8 Sent Done Done false
26838 7738 Acked Read 7 Sent Done Done false
26839 7739 Acked Flush 0 Sent Done Done false
26840 7740 Acked Read 6 Sent Done Done false
26841 7741 Acked Write 2 Sent Done Done false
26842 7742 Acked Read 5 Sent Done Done false
26843 7743 Acked Read 3 Sent Done Done false
26844 7744 Acked Read 5 Sent Done Done false
26845 7745 Acked Write 2 Sent Done Done false
26846 7746 Acked Write 3 Sent Done Done false
26847 7747 Acked Read 10 Sent Done Done false
26848 7748 Acked Write 6 Sent Done Done false
26849 7749 Acked Read 2 Sent Done Done false
26850 7750 Acked Write 3 Sent Done Done false
26851 7751 Acked Flush 0 Sent Done Done false
26852 7752 Acked Read 6 Sent Done Done false
26853 7753 Acked Read 7 Sent Done Done false
26854 7754 Acked Write 7 Sent Done Done false
26855 7755 Acked Read 5 Sent Done Done false
26856 7756 Acked Write 6 Sent Done Done false
26857 7757 Acked Write 1 Sent Done Done false
26858 7758 Acked Write 4 Sent Done Done false
26859 7759 Acked Write 10 Sent Done Done false
26860 7760 Acked Flush 0 Sent Done Done false
26861 7761 Acked Read 8 Sent Done Done false
26862 7762 Acked Read 4 Sent Done Done false
26863 7763 Acked Write 10 Sent Done Done false
26864 7764 Acked Read 1 Sent Done Done false
26865 7765 Acked Write 1 Sent Done Done false
26866 7766 Acked Write 5 Sent Done Done false
26867 7767 Acked Read 8 Sent Done Done false
26868 7768 Acked Write 1 Sent Done Done false
26869 7769 Acked Write 6 Sent Done Done false
26870 7770 Acked Read 6 Sent Done Done false
26871 7771 Acked Write 8 Sent Done Done false
26872 7772 Acked Flush 0 Sent Done Done false
26873 7773 Acked Read 4 Sent Done Done false
26874 7774 Acked Read 1 Sent Done Done false
26875 7775 Acked Read 5 Sent Done Done false
26876 7776 Acked Read 7 Sent Done Done false
26877 7777 Acked Read 5 Sent Done Done false
26878 7778 Acked Read 4 Sent Done Done false
26879 7779 Acked Write 3 Sent Done Done false
26880 7780 Acked Read 6 Sent Done Done false
26881 7781 Acked Read 4 Sent Done Done false
26882 7782 Acked Read 7 Sent Done Done false
26883 7783 Acked Read 8 Sent Done Done false
26884 7784 Acked Read 4 Sent Done Done false
26885 7785 Acked Write 5 Sent Done Done false
26886 7786 Acked Read 1 Sent Done Done false
26887 7787 Acked Read 1 Sent Done Done false
26888 7788 Acked Read 8 Sent Done Done false
26889 7789 Acked Write 2 Sent Done Done false
26890 7790 Acked Read 8 Sent Done Done false
26891 7791 Acked Flush 0 Sent Done Done false
26892 7792 Acked Read 3 Sent Done Done false
26893 7793 Acked Write 4 Sent Done Done false
26894 7794 Acked Flush 0 Sent Done Done false
26895 7795 Acked Read 1 Sent Done Done false
26896 7796 Acked Write 6 Sent Done Done false
26897 7797 Acked Read 1 Sent Done Done false
26898 7798 Acked Write 10 Sent Done Done false
26899 7799 Acked Write 1 Sent Done Done false
26900 7800 Acked Write 9 Sent Done Done false
26901 7801 Acked Read 1 Sent Done Done false
26902 7802 Acked Read 4 Sent Done Done false
26903 7803 Acked Read 3 Sent Done Done false
26904 7804 Acked Read 3 Sent Done Done false
26905 7805 Acked Read 8 Sent Done Done false
26906 7806 Acked Write 8 Sent Done Done false
26907 7807 Acked Read 10 Sent Done Done false
26908 7808 Acked Flush 0 Sent Done Done false
26909 7809 Acked Write 9 Sent Done Done false
26910 7810 Acked Write 5 Sent Done Done false
26911 7811 Acked Flush 0 Sent Done Done false
26912 7812 Acked Read 10 Sent Done Done false
26913 7813 Acked Read 8 Sent Done Done false
26914 7814 Acked Read 2 Sent Done Done false
26915 7815 Acked Write 9 Sent Done Done false
26916 7816 Acked Write 5 Sent Done Done false
26917 7817 Acked Write 4 Sent Done Done false
26918 7818 Acked Read 5 Sent Done Done false
26919 7819 Acked Read 8 Sent Done Done false
26920 7820 Acked Write 8 Sent Done Done false
26921 7821 Acked Write 6 Sent Done Done false
26922 7822 Acked Read 4 Sent Done Done false
26923 7823 Acked Read 4 Sent Done Done false
26924 7824 Acked Read 7 Sent Done Done false
26925 7825 Acked Write 10 Sent Done Done false
26926 7826 Acked Read 10 Sent Done Done false
26927 7827 Acked Read 7 Sent Done Done false
26928 7828 Acked Read 3 Sent Done Done false
26929 7829 Acked Write 7 Sent Done Done false
26930 7830 Acked Write 3 Sent Done Done false
26931 7831 Acked Write 7 Sent Done Done false
26932 7832 Acked Flush 0 Sent Done Done false
26933 7833 Acked Read 4 Sent Done Done false
26934 7834 Acked Write 1 Sent Done Done false
26935 7835 Acked Read 7 Sent Done Done false
26936 7836 Acked Read 4 Sent Done Done false
26937 7837 Acked Read 8 Sent Done Done false
26938 7838 Acked Flush 0 Sent Done Done false
26939 7839 Acked Read 5 Sent Done Done false
26940 7840 Acked Read 4 Sent Done Done false
26941 7841 Acked Write 9 Sent Done Done false
26942 7842 Acked Flush 0 Sent Done Done false
26943 7843 Acked Write 5 Sent Done Done false
26944 7844 Acked Read 4 Sent Done Done false
26945 7845 Acked Read 4 Sent Done Done false
26946 7846 Acked Read 9 Sent Done Done false
26947 7847 Acked Read 10 Sent Done Done false
26948 7848 Acked Read 6 Sent Done Done false
26949 7849 Acked Read 8 Sent Done Done false
26950 7850 Acked Read 2 Sent Done Done false
26951 7851 Acked Write 3 Sent Done Done false
26952 7852 Acked Read 10 Sent Done Done false
26953 7853 Acked Write 3 Sent Done Done false
26954 7854 Acked Read 1 Sent Done Done false
26955 7855 Acked Read 9 Sent Done Done false
26956 7856 Acked Write 3 Sent Done Done false
26957 7857 Acked Read 9 Sent Done Done false
26958 7858 Acked Write 6 Sent Done Done false
26959 7859 Acked Read 3 Sent Done Done false
26960 7860 Acked Read 7 Sent Done Done false
26961 7861 Acked Write 10 Sent Done Done false
26962 7862 Acked Read 3 Sent Done Done false
26963 7863 Acked Read 2 Sent Done Done false
26964 7864 Acked Read 1 Sent Done Done false
26965 7865 Acked Write 9 Sent Done Done false
26966 7866 Acked Read 2 Sent Done Done false
26967 7867 Acked Write 1 Sent Done Done false
26968 7868 Acked Flush 0 Sent Done Done false
26969 7869 Acked Write 9 Sent Done Done false
26970 7870 Acked Read 7 Sent Done Done false
26971 7871 Acked Write 1 Sent Done Done false
26972 7872 Acked Write 2 Sent Done Done false
26973 7873 Acked Read 8 Sent Done Done false
26974 7874 Acked Read 4 Sent Done Done false
26975 7875 Acked Write 8 Sent Done Done false
26976 7876 Acked Flush 0 Sent Done Done false
26977 7877 Acked Write 9 Sent Done Done false
26978 7878 Acked Write 7 Sent Done Done false
26979 7879 Acked Write 9 Sent Done Done false
26980 7880 Acked Read 6 Sent Done Done false
26981 7881 Acked Read 9 Sent Done Done false
26982 7882 Acked Read 6 Sent Done Done false
26983 7883 Acked Write 1 Sent Done Done false
26984 7884 Acked Read 2 Sent Done Done false
26985 7885 Acked Write 6 Sent Done Done false
26986 7886 Acked Flush 0 Sent Done Done false
26987 7887 Acked Read 5 Sent Done Done false
26988 7888 Acked Write 9 Sent Done Done false
26989 7889 Acked Write 1 Sent Done Done false
26990 7890 Acked Read 3 Sent Done Done false
26991 7891 Acked Write 6 Sent Done Done false
26992 7892 Acked Write 5 Sent Done Done false
26993 7893 Acked Flush 0 Sent Done Done false
26994 7894 Acked Read 2 Sent Done Done false
26995 7895 Acked Write 2 Sent Done Done false
26996 7896 Acked Write 6 Sent Done Done false
26997 7897 Acked Read 4 Sent Done Done false
26998 7898 Acked Write 6 Sent Done Done false
26999 7899 Acked Write 8 Sent Done Done false
27000 7900 Acked Write 9 Sent Done Done false
27001 7901 Acked Flush 0 Sent Done Done false
27002 7902 Acked Read 2 Sent Done Done false
27003 7903 Acked Write 8 Sent Done Done false
27004 7904 Acked Write 6 Sent Done Done false
27005 7905 Acked Flush 0 Sent Done Done false
27006 7906 Acked Write 10 Sent Done Done false
27007 7907 Acked Write 6 Sent Done Done false
27008 7908 Acked Read 7 Sent Done Done false
27009 7909 Acked Write 5 Sent Done Done false
27010 7910 Acked Read 7 Sent Done Done false
27011 7911 Acked Read 6 Sent Done Done false
27012 7912 Acked Read 6 Sent Done Done false
27013 7913 Acked Write 6 Sent Done Done false
27014 7914 Acked Write 9 Sent Done Done false
27015 7915 Acked Write 6 Sent Done Done false
27016 7916 Acked Read 10 Sent Done Done false
27017 7917 Acked Write 9 Sent Done Done false
27018 7918 Acked Read 6 Sent Done Done false
27019 7919 Acked Read 6 Sent Done Done false
27020 7920 Acked Write 2 Sent Done Done false
27021 7921 Acked Write 5 Sent Done Done false
27022 7922 Acked Write 8 Sent Done Done false
27023 7923 Acked Read 5 Sent Done Done false
27024 7924 Acked Read 6 Sent Done Done false
27025 7925 Acked Write 2 Sent Done Done false
27026 7926 Acked Read 7 Sent Done Done false
27027 7927 Acked Write 5 Sent Done Done false
27028 7928 Acked Read 10 Sent Done Done false
27029 7929 Acked Read 6 Sent Done Done false
27030 7930 Acked Write 7 Sent Done Done false
27031 7931 Acked Write 1 Sent Done Done false
27032 7932 Acked Read 8 Sent Done Done false
27033 7933 Acked Read 6 Sent Done Done false
27034 7934 Acked Write 10 Sent Done Done false
27035 7935 Acked Write 8 Sent Done Done false
27036 7936 Acked Write 9 Sent Done Done false
27037 7937 Acked Write 7 Sent Done Done false
27038 7938 Acked Write 2 Sent Done Done false
27039 7939 Acked Flush 0 Sent Done Done false
27040 7940 Acked Write 5 Sent Done Done false
27041 7941 Acked Read 4 Sent Done Done false
27042 7942 Acked Flush 0 Sent Done Done false
27043 STATES DS:0 DS:1 DS:2 TOTAL
27044 Sent 302 0 0 302
27045 Done 0 302 302 604
27046 Skipped 0 0 0 0
27047 Error 0 0 0 0
27048 Last Flush: 7640 7942 7942
27049 Downstairs last five completed: 7640 7639 7638 7637 7636
27050 Upstairs last five completed: 7942 7941 7940 7939 7938
27051 CLIENT: Up:0 ds:302 act:2
270522024-11-28T04:55:58.375ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
270532024-11-28T04:55:58.375ZWARNcrucible: ds_connection connect to 127.0.0.1:8810 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
270542024-11-28T04:55:58.375ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
270552024-11-28T04:55:58.375ZWARNcrucible: downstairs task for 0 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
270562024-11-28T04:55:58.375ZINFOcrucible: client task is exiting = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
27057 [8810] got start action
27058 [8810] I am going to restart
27059 Starting downstairs at port 8810
27060 Make output file at "/tmp/dsc/downstairs-8810.txt"
27061 [8810][0] reports Starting
27062 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1225)
27063 [8810][0] reports Running
27064 ----------------------------------------------------------------
27065 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
27066 States: Offline Active Active
27067 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
27068 7641 Acked Write 6 Sent Done Done false
27069 7642 Acked Write 4 Sent Done Done false
27070 7643 Acked Flush 0 Sent Done Done false
27071 7644 Acked Write 4 Sent Done Done false
27072 7645 Acked Write 10 Sent Done Done false
27073 7646 Acked Read 6 Sent Done Done false
27074 7647 Acked Read 4 Sent Done Done false
27075 7648 Acked Read 4 Sent Done Done false
27076 7649 Acked Write 3 Sent Done Done false
27077 7650 Acked Flush 0 Sent Done Done false
27078 7651 Acked Read 10 Sent Done Done false
27079 7652 Acked Read 6 Sent Done Done false
27080 7653 Acked Write 9 Sent Done Done false
27081 7654 Acked Read 10 Sent Done Done false
27082 7655 Acked Read 5 Sent Done Done false
27083 7656 Acked Read 4 Sent Done Done false
27084 7657 Acked Write 7 Sent Done Done false
27085 7658 Acked Write 9 Sent Done Done false
27086 7659 Acked Read 7 Sent Done Done false
27087 7660 Acked Read 3 Sent Done Done false
27088 7661 Acked Write 5 Sent Done Done false
27089 7662 Acked Read 7 Sent Done Done false
27090 7663 Acked Write 4 Sent Done Done false
27091 7664 Acked Write 6 Sent Done Done false
27092 7665 Acked Write 2 Sent Done Done false
27093 7666 Acked Read 3 Sent Done Done false
27094 7667 Acked Write 1 Sent Done Done false
27095 7668 Acked Read 5 Sent Done Done false
27096 7669 Acked Write 3 Sent Done Done false
27097 7670 Acked Read 8 Sent Done Done false
27098 7671 Acked Flush 0 Sent Done Done false
27099 7672 Acked Read 9 Sent Done Done false
27100 7673 Acked Write 3 Sent Done Done false
27101 7674 Acked Read 7 Sent Done Done false
27102 7675 Acked Read 3 Sent Done Done false
27103 7676 Acked Flush 0 Sent Done Done false
27104 7677 Acked Write 10 Sent Done Done false
27105 7678 Acked Read 4 Sent Done Done false
27106 7679 Acked Write 2 Sent Done Done false
27107 7680 Acked Write 4 Sent Done Done false
27108 7681 Acked Write 6 Sent Done Done false
27109 7682 Acked Write 4 Sent Done Done false
27110 7683 Acked Read 8 Sent Done Done false
27111 7684 Acked Read 1 Sent Done Done false
27112 7685 Acked Flush 0 Sent Done Done false
27113 7686 Acked Read 3 Sent Done Done false
27114 7687 Acked Read 4 Sent Done Done false
27115 7688 Acked Read 1 Sent Done Done false
27116 7689 Acked Read 3 Sent Done Done false
27117 7690 Acked Read 7 Sent Done Done false
27118 7691 Acked Read 3 Sent Done Done false
27119 7692 Acked Write 7 Sent Done Done false
27120 7693 Acked Read 3 Sent Done Done false
27121 7694 Acked Read 4 Sent Done Done false
27122 7695 Acked Read 9 Sent Done Done false
27123 7696 Acked Read 5 Sent Done Done false
27124 7697 Acked Write 3 Sent Done Done false
27125 7698 Acked Write 6 Sent Done Done false
27126 7699 Acked Read 8 Sent Done Done false
27127 7700 Acked Read 2 Sent Done Done false
27128 7701 Acked Write 10 Sent Done Done false
27129 7702 Acked Write 9 Sent Done Done false
27130 7703 Acked Write 10 Sent Done Done false
27131 7704 Acked Read 1 Sent Done Done false
27132 7705 Acked Write 8 Sent Done Done false
27133 7706 Acked Read 10 Sent Done Done false
27134 7707 Acked Read 10 Sent Done Done false
27135 7708 Acked Write 4 Sent Done Done false
27136 7709 Acked Read 3 Sent Done Done false
27137 7710 Acked Read 8 Sent Done Done false
27138 7711 Acked Flush 0 Sent Done Done false
27139 7712 Acked Flush 0 Sent Done Done false
27140 7713 Acked Write 2 Sent Done Done false
27141 7714 Acked Read 5 Sent Done Done false
27142 7715 Acked Write 2 Sent Done Done false
27143 7716 Acked Read 3 Sent Done Done false
27144 7717 Acked Write 9 Sent Done Done false
27145 7718 Acked Flush 0 Sent Done Done false
27146 7719 Acked Flush 0 Sent Done Done false
27147 7720 Acked Read 5 Sent Done Done false
27148 7721 Acked Read 5 Sent Done Done false
27149 7722 Acked Flush 0 Sent Done Done false
27150 7723 Acked Flush 0 Sent Done Done false
27151 7724 Acked Read 5 Sent Done Done false
27152 7725 Acked Write 8 Sent Done Done false
27153 7726 Acked Write 2 Sent Done Done false
27154 7727 Acked Write 6 Sent Done Done false
27155 7728 Acked Read 7 Sent Done Done false
27156 7729 Acked Write 10 Sent Done Done false
27157 7730 Acked Read 6 Sent Done Done false
27158 7731 Acked Read 4 Sent Done Done false
27159 7732 Acked Read 6 Sent Done Done false
27160 7733 Acked Write 9 Sent Done Done false
27161 7734 Acked Write 7 Sent Done Done false
27162 7735 Acked Read 6 Sent Done Done false
27163 7736 Acked Read 10 Sent Done Done false
27164 7737 Acked Write 8 Sent Done Done false
27165 7738 Acked Read 7 Sent Done Done false
27166 7739 Acked Flush 0 Sent Done Done false
27167 7740 Acked Read 6 Sent Done Done false
27168 7741 Acked Write 2 Sent Done Done false
27169 7742 Acked Read 5 Sent Done Done false
27170 7743 Acked Read 3 Sent Done Done false
27171 7744 Acked Read 5 Sent Done Done false
27172 7745 Acked Write 2 Sent Done Done false
27173 7746 Acked Write 3 Sent Done Done false
27174 7747 Acked Read 10 Sent Done Done false
27175 7748 Acked Write 6 Sent Done Done false
27176 7749 Acked Read 2 Sent Done Done false
27177 7750 Acked Write 3 Sent Done Done false
27178 7751 Acked Flush 0 Sent Done Done false
27179 7752 Acked Read 6 Sent Done Done false
27180 7753 Acked Read 7 Sent Done Done false
27181 7754 Acked Write 7 Sent Done Done false
27182 7755 Acked Read 5 Sent Done Done false
27183 7756 Acked Write 6 Sent Done Done false
27184 7757 Acked Write 1 Sent Done Done false
27185 7758 Acked Write 4 Sent Done Done false
27186 7759 Acked Write 10 Sent Done Done false
27187 7760 Acked Flush 0 Sent Done Done false
27188 7761 Acked Read 8 Sent Done Done false
27189 7762 Acked Read 4 Sent Done Done false
27190 7763 Acked Write 10 Sent Done Done false
27191 7764 Acked Read 1 Sent Done Done false
27192 7765 Acked Write 1 Sent Done Done false
27193 7766 Acked Write 5 Sent Done Done false
27194 7767 Acked Read 8 Sent Done Done false
27195 7768 Acked Write 1 Sent Done Done false
27196 7769 Acked Write 6 Sent Done Done false
27197 7770 Acked Read 6 Sent Done Done false
27198 7771 Acked Write 8 Sent Done Done false
27199 7772 Acked Flush 0 Sent Done Done false
27200 7773 Acked Read 4 Sent Done Done false
27201 7774 Acked Read 1 Sent Done Done false
27202 7775 Acked Read 5 Sent Done Done false
27203 7776 Acked Read 7 Sent Done Done false
27204 7777 Acked Read 5 Sent Done Done false
27205 7778 Acked Read 4 Sent Done Done false
27206 7779 Acked Write 3 Sent Done Done false
27207 7780 Acked Read 6 Sent Done Done false
27208 7781 Acked Read 4 Sent Done Done false
27209 7782 Acked Read 7 Sent Done Done false
27210 7783 Acked Read 8 Sent Done Done false
27211 7784 Acked Read 4 Sent Done Done false
27212 7785 Acked Write 5 Sent Done Done false
27213 7786 Acked Read 1 Sent Done Done false
27214 7787 Acked Read 1 Sent Done Done false
27215 7788 Acked Read 8 Sent Done Done false
27216 7789 Acked Write 2 Sent Done Done false
27217 7790 Acked Read 8 Sent Done Done false
27218 7791 Acked Flush 0 Sent Done Done false
27219 7792 Acked Read 3 Sent Done Done false
27220 7793 Acked Write 4 Sent Done Done false
27221 7794 Acked Flush 0 Sent Done Done false
27222 7795 Acked Read 1 Sent Done Done false
27223 7796 Acked Write 6 Sent Done Done false
27224 7797 Acked Read 1 Sent Done Done false
27225 7798 Acked Write 10 Sent Done Done false
27226 7799 Acked Write 1 Sent Done Done false
27227 7800 Acked Write 9 Sent Done Done false
27228 7801 Acked Read 1 Sent Done Done false
27229 7802 Acked Read 4 Sent Done Done false
27230 7803 Acked Read 3 Sent Done Done false
27231 7804 Acked Read 3 Sent Done Done false
27232 7805 Acked Read 8 Sent Done Done false
27233 7806 Acked Write 8 Sent Done Done false
27234 7807 Acked Read 10 Sent Done Done false
27235 7808 Acked Flush 0 Sent Done Done false
27236 7809 Acked Write 9 Sent Done Done false
27237 7810 Acked Write 5 Sent Done Done false
27238 7811 Acked Flush 0 Sent Done Done false
27239 7812 Acked Read 10 Sent Done Done false
27240 7813 Acked Read 8 Sent Done Done false
27241 7814 Acked Read 2 Sent Done Done false
27242 7815 Acked Write 9 Sent Done Done false
27243 7816 Acked Write 5 Sent Done Done false
27244 7817 Acked Write 4 Sent Done Done false
27245 7818 Acked Read 5 Sent Done Done false
27246 7819 Acked Read 8 Sent Done Done false
27247 7820 Acked Write 8 Sent Done Done false
27248 7821 Acked Write 6 Sent Done Done false
27249 7822 Acked Read 4 Sent Done Done false
27250 7823 Acked Read 4 Sent Done Done false
27251 7824 Acked Read 7 Sent Done Done false
27252 7825 Acked Write 10 Sent Done Done false
27253 7826 Acked Read 10 Sent Done Done false
27254 7827 Acked Read 7 Sent Done Done false
27255 7828 Acked Read 3 Sent Done Done false
27256 7829 Acked Write 7 Sent Done Done false
27257 7830 Acked Write 3 Sent Done Done false
27258 7831 Acked Write 7 Sent Done Done false
27259 7832 Acked Flush 0 Sent Done Done false
27260 7833 Acked Read 4 Sent Done Done false
27261 7834 Acked Write 1 Sent Done Done false
27262 7835 Acked Read 7 Sent Done Done false
27263 7836 Acked Read 4 Sent Done Done false
27264 7837 Acked Read 8 Sent Done Done false
27265 7838 Acked Flush 0 Sent Done Done false
27266 7839 Acked Read 5 Sent Done Done false
27267 7840 Acked Read 4 Sent Done Done false
27268 7841 Acked Write 9 Sent Done Done false
27269 7842 Acked Flush 0 Sent Done Done false
27270 7843 Acked Write 5 Sent Done Done false
27271 7844 Acked Read 4 Sent Done Done false
27272 7845 Acked Read 4 Sent Done Done false
27273 7846 Acked Read 9 Sent Done Done false
27274 7847 Acked Read 10 Sent Done Done false
27275 7848 Acked Read 6 Sent Done Done false
27276 7849 Acked Read 8 Sent Done Done false
27277 7850 Acked Read 2 Sent Done Done false
27278 7851 Acked Write 3 Sent Done Done false
27279 7852 Acked Read 10 Sent Done Done false
27280 7853 Acked Write 3 Sent Done Done false
27281 7854 Acked Read 1 Sent Done Done false
27282 7855 Acked Read 9 Sent Done Done false
27283 7856 Acked Write 3 Sent Done Done false
27284 7857 Acked Read 9 Sent Done Done false
27285 7858 Acked Write 6 Sent Done Done false
27286 7859 Acked Read 3 Sent Done Done false
27287 7860 Acked Read 7 Sent Done Done false
27288 7861 Acked Write 10 Sent Done Done false
27289 7862 Acked Read 3 Sent Done Done false
27290 7863 Acked Read 2 Sent Done Done false
27291 7864 Acked Read 1 Sent Done Done false
27292 7865 Acked Write 9 Sent Done Done false
27293 7866 Acked Read 2 Sent Done Done false
27294 7867 Acked Write 1 Sent Done Done false
27295 7868 Acked Flush 0 Sent Done Done false
27296 7869 Acked Write 9 Sent Done Done false
27297 7870 Acked Read 7 Sent Done Done false
27298 7871 Acked Write 1 Sent Done Done false
27299 7872 Acked Write 2 Sent Done Done false
27300 7873 Acked Read 8 Sent Done Done false
27301 7874 Acked Read 4 Sent Done Done false
27302 7875 Acked Write 8 Sent Done Done false
27303 7876 Acked Flush 0 Sent Done Done false
27304 7877 Acked Write 9 Sent Done Done false
27305 7878 Acked Write 7 Sent Done Done false
27306 7879 Acked Write 9 Sent Done Done false
27307 7880 Acked Read 6 Sent Done Done false
27308 7881 Acked Read 9 Sent Done Done false
27309 7882 Acked Read 6 Sent Done Done false
27310 7883 Acked Write 1 Sent Done Done false
27311 7884 Acked Read 2 Sent Done Done false
27312 7885 Acked Write 6 Sent Done Done false
27313 7886 Acked Flush 0 Sent Done Done false
27314 7887 Acked Read 5 Sent Done Done false
27315 7888 Acked Write 9 Sent Done Done false
27316 7889 Acked Write 1 Sent Done Done false
27317 7890 Acked Read 3 Sent Done Done false
27318 7891 Acked Write 6 Sent Done Done false
27319 7892 Acked Write 5 Sent Done Done false
27320 7893 Acked Flush 0 Sent Done Done false
27321 7894 Acked Read 2 Sent Done Done false
27322 7895 Acked Write 2 Sent Done Done false
27323 7896 Acked Write 6 Sent Done Done false
27324 7897 Acked Read 4 Sent Done Done false
27325 7898 Acked Write 6 Sent Done Done false
27326 7899 Acked Write 8 Sent Done Done false
27327 7900 Acked Write 9 Sent Done Done false
27328 7901 Acked Flush 0 Sent Done Done false
27329 7902 Acked Read 2 Sent Done Done false
27330 7903 Acked Write 8 Sent Done Done false
27331 7904 Acked Write 6 Sent Done Done false
27332 7905 Acked Flush 0 Sent Done Done false
27333 7906 Acked Write 10 Sent Done Done false
27334 7907 Acked Write 6 Sent Done Done false
27335 7908 Acked Read 7 Sent Done Done false
27336 7909 Acked Write 5 Sent Done Done false
27337 7910 Acked Read 7 Sent Done Done false
27338 7911 Acked Read 6 Sent Done Done false
27339 7912 Acked Read 6 Sent Done Done false
27340 7913 Acked Write 6 Sent Done Done false
27341 7914 Acked Write 9 Sent Done Done false
27342 7915 Acked Write 6 Sent Done Done false
27343 7916 Acked Read 10 Sent Done Done false
27344 7917 Acked Write 9 Sent Done Done false
27345 7918 Acked Read 6 Sent Done Done false
27346 7919 Acked Read 6 Sent Done Done false
27347 7920 Acked Write 2 Sent Done Done false
27348 7921 Acked Write 5 Sent Done Done false
27349 7922 Acked Write 8 Sent Done Done false
27350 7923 Acked Read 5 Sent Done Done false
27351 7924 Acked Read 6 Sent Done Done false
27352 7925 Acked Write 2 Sent Done Done false
27353 7926 Acked Read 7 Sent Done Done false
27354 7927 Acked Write 5 Sent Done Done false
27355 7928 Acked Read 10 Sent Done Done false
27356 7929 Acked Read 6 Sent Done Done false
27357 7930 Acked Write 7 Sent Done Done false
27358 7931 Acked Write 1 Sent Done Done false
27359 7932 Acked Read 8 Sent Done Done false
27360 7933 Acked Read 6 Sent Done Done false
27361 7934 Acked Write 10 Sent Done Done false
27362 7935 Acked Write 8 Sent Done Done false
27363 7936 Acked Write 9 Sent Done Done false
27364 7937 Acked Write 7 Sent Done Done false
27365 7938 Acked Write 2 Sent Done Done false
27366 7939 Acked Flush 0 Sent Done Done false
27367 7940 Acked Write 5 Sent Done Done false
27368 7941 Acked Read 4 Sent Done Done false
27369 7942 Acked Flush 0 Sent Done Done false
27370 STATES DS:0 DS:1 DS:2 TOTAL
27371 Sent 302 0 0 302
27372 Done 0 302 302 604
27373 Skipped 0 0 0 0
27374 Error 0 0 0 0
27375 Last Flush: 7640 7942 7942
27376 Downstairs last five completed: 7640 7639 7638 7637 7636
27377 Upstairs last five completed: 7942 7941 7940 7939 7938
27378 CLIENT: Up:0 ds:302 act:2
27379 ----------------------------------------------------------------
27380 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
27381 States: Offline Active Active
27382 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
27383 7641 Acked Write 6 Sent Done Done false
27384 7642 Acked Write 4 Sent Done Done false
27385 7643 Acked Flush 0 Sent Done Done false
27386 7644 Acked Write 4 Sent Done Done false
27387 7645 Acked Write 10 Sent Done Done false
27388 7646 Acked Read 6 Sent Done Done false
27389 7647 Acked Read 4 Sent Done Done false
27390 7648 Acked Read 4 Sent Done Done false
27391 7649 Acked Write 3 Sent Done Done false
27392 7650 Acked Flush 0 Sent Done Done false
27393 7651 Acked Read 10 Sent Done Done false
27394 7652 Acked Read 6 Sent Done Done false
27395 7653 Acked Write 9 Sent Done Done false
27396 7654 Acked Read 10 Sent Done Done false
27397 7655 Acked Read 5 Sent Done Done false
27398 7656 Acked Read 4 Sent Done Done false
27399 7657 Acked Write 7 Sent Done Done false
27400 7658 Acked Write 9 Sent Done Done false
27401 7659 Acked Read 7 Sent Done Done false
27402 7660 Acked Read 3 Sent Done Done false
27403 7661 Acked Write 5 Sent Done Done false
27404 7662 Acked Read 7 Sent Done Done false
27405 7663 Acked Write 4 Sent Done Done false
27406 7664 Acked Write 6 Sent Done Done false
27407 7665 Acked Write 2 Sent Done Done false
27408 7666 Acked Read 3 Sent Done Done false
27409 7667 Acked Write 1 Sent Done Done false
27410 7668 Acked Read 5 Sent Done Done false
27411 7669 Acked Write 3 Sent Done Done false
27412 7670 Acked Read 8 Sent Done Done false
27413 7671 Acked Flush 0 Sent Done Done false
27414 7672 Acked Read 9 Sent Done Done false
27415 7673 Acked Write 3 Sent Done Done false
27416 7674 Acked Read 7 Sent Done Done false
27417 7675 Acked Read 3 Sent Done Done false
27418 7676 Acked Flush 0 Sent Done Done false
27419 7677 Acked Write 10 Sent Done Done false
27420 7678 Acked Read 4 Sent Done Done false
27421 7679 Acked Write 2 Sent Done Done false
27422 7680 Acked Write 4 Sent Done Done false
27423 7681 Acked Write 6 Sent Done Done false
27424 7682 Acked Write 4 Sent Done Done false
27425 7683 Acked Read 8 Sent Done Done false
27426 7684 Acked Read 1 Sent Done Done false
27427 7685 Acked Flush 0 Sent Done Done false
27428 7686 Acked Read 3 Sent Done Done false
27429 7687 Acked Read 4 Sent Done Done false
27430 7688 Acked Read 1 Sent Done Done false
27431 7689 Acked Read 3 Sent Done Done false
27432 7690 Acked Read 7 Sent Done Done false
27433 7691 Acked Read 3 Sent Done Done false
27434 7692 Acked Write 7 Sent Done Done false
27435 7693 Acked Read 3 Sent Done Done false
27436 7694 Acked Read 4 Sent Done Done false
27437 7695 Acked Read 9 Sent Done Done false
27438 7696 Acked Read 5 Sent Done Done false
27439 7697 Acked Write 3 Sent Done Done false
27440 7698 Acked Write 6 Sent Done Done false
27441 7699 Acked Read 8 Sent Done Done false
27442 7700 Acked Read 2 Sent Done Done false
27443 7701 Acked Write 10 Sent Done Done false
27444 7702 Acked Write 9 Sent Done Done false
27445 7703 Acked Write 10 Sent Done Done false
27446 7704 Acked Read 1 Sent Done Done false
27447 7705 Acked Write 8 Sent Done Done false
27448 7706 Acked Read 10 Sent Done Done false
27449 7707 Acked Read 10 Sent Done Done false
27450 7708 Acked Write 4 Sent Done Done false
27451 7709 Acked Read 3 Sent Done Done false
27452 7710 Acked Read 8 Sent Done Done false
27453 7711 Acked Flush 0 Sent Done Done false
27454 7712 Acked Flush 0 Sent Done Done false
27455 7713 Acked Write 2 Sent Done Done false
27456 7714 Acked Read 5 Sent Done Done false
27457 7715 Acked Write 2 Sent Done Done false
27458 7716 Acked Read 3 Sent Done Done false
27459 7717 Acked Write 9 Sent Done Done false
27460 7718 Acked Flush 0 Sent Done Done false
27461 7719 Acked Flush 0 Sent Done Done false
27462 7720 Acked Read 5 Sent Done Done false
27463 7721 Acked Read 5 Sent Done Done false
27464 7722 Acked Flush 0 Sent Done Done false
27465 7723 Acked Flush 0 Sent Done Done false
27466 7724 Acked Read 5 Sent Done Done false
27467 7725 Acked Write 8 Sent Done Done false
27468 7726 Acked Write 2 Sent Done Done false
27469 7727 Acked Write 6 Sent Done Done false
27470 7728 Acked Read 7 Sent Done Done false
27471 7729 Acked Write 10 Sent Done Done false
27472 7730 Acked Read 6 Sent Done Done false
27473 7731 Acked Read 4 Sent Done Done false
27474 7732 Acked Read 6 Sent Done Done false
27475 7733 Acked Write 9 Sent Done Done false
27476 7734 Acked Write 7 Sent Done Done false
27477 7735 Acked Read 6 Sent Done Done false
27478 7736 Acked Read 10 Sent Done Done false
27479 7737 Acked Write 8 Sent Done Done false
27480 7738 Acked Read 7 Sent Done Done false
27481 7739 Acked Flush 0 Sent Done Done false
27482 7740 Acked Read 6 Sent Done Done false
27483 7741 Acked Write 2 Sent Done Done false
27484 7742 Acked Read 5 Sent Done Done false
27485 7743 Acked Read 3 Sent Done Done false
27486 7744 Acked Read 5 Sent Done Done false
27487 7745 Acked Write 2 Sent Done Done false
27488 7746 Acked Write 3 Sent Done Done false
27489 7747 Acked Read 10 Sent Done Done false
27490 7748 Acked Write 6 Sent Done Done false
27491 7749 Acked Read 2 Sent Done Done false
27492 7750 Acked Write 3 Sent Done Done false
27493 7751 Acked Flush 0 Sent Done Done false
27494 7752 Acked Read 6 Sent Done Done false
27495 7753 Acked Read 7 Sent Done Done false
27496 7754 Acked Write 7 Sent Done Done false
27497 7755 Acked Read 5 Sent Done Done false
27498 7756 Acked Write 6 Sent Done Done false
27499 7757 Acked Write 1 Sent Done Done false
27500 7758 Acked Write 4 Sent Done Done false
27501 7759 Acked Write 10 Sent Done Done false
27502 7760 Acked Flush 0 Sent Done Done false
27503 7761 Acked Read 8 Sent Done Done false
27504 7762 Acked Read 4 Sent Done Done false
27505 7763 Acked Write 10 Sent Done Done false
27506 7764 Acked Read 1 Sent Done Done false
27507 7765 Acked Write 1 Sent Done Done false
27508 7766 Acked Write 5 Sent Done Done false
27509 7767 Acked Read 8 Sent Done Done false
27510 7768 Acked Write 1 Sent Done Done false
27511 7769 Acked Write 6 Sent Done Done false
27512 7770 Acked Read 6 Sent Done Done false
27513 7771 Acked Write 8 Sent Done Done false
27514 7772 Acked Flush 0 Sent Done Done false
27515 7773 Acked Read 4 Sent Done Done false
27516 7774 Acked Read 1 Sent Done Done false
27517 7775 Acked Read 5 Sent Done Done false
27518 7776 Acked Read 7 Sent Done Done false
27519 7777 Acked Read 5 Sent Done Done false
27520 7778 Acked Read 4 Sent Done Done false
27521 7779 Acked Write 3 Sent Done Done false
27522 7780 Acked Read 6 Sent Done Done false
27523 7781 Acked Read 4 Sent Done Done false
27524 7782 Acked Read 7 Sent Done Done false
27525 7783 Acked Read 8 Sent Done Done false
27526 7784 Acked Read 4 Sent Done Done false
27527 7785 Acked Write 5 Sent Done Done false
27528 7786 Acked Read 1 Sent Done Done false
27529 7787 Acked Read 1 Sent Done Done false
27530 7788 Acked Read 8 Sent Done Done false
27531 7789 Acked Write 2 Sent Done Done false
27532 7790 Acked Read 8 Sent Done Done false
27533 7791 Acked Flush 0 Sent Done Done false
27534 7792 Acked Read 3 Sent Done Done false
27535 7793 Acked Write 4 Sent Done Done false
27536 7794 Acked Flush 0 Sent Done Done false
27537 7795 Acked Read 1 Sent Done Done false
27538 7796 Acked Write 6 Sent Done Done false
27539 7797 Acked Read 1 Sent Done Done false
27540 7798 Acked Write 10 Sent Done Done false
27541 7799 Acked Write 1 Sent Done Done false
27542 7800 Acked Write 9 Sent Done Done false
27543 7801 Acked Read 1 Sent Done Done false
27544 7802 Acked Read 4 Sent Done Done false
27545 7803 Acked Read 3 Sent Done Done false
27546 7804 Acked Read 3 Sent Done Done false
27547 7805 Acked Read 8 Sent Done Done false
27548 7806 Acked Write 8 Sent Done Done false
27549 7807 Acked Read 10 Sent Done Done false
27550 7808 Acked Flush 0 Sent Done Done false
27551 7809 Acked Write 9 Sent Done Done false
27552 7810 Acked Write 5 Sent Done Done false
27553 7811 Acked Flush 0 Sent Done Done false
27554 7812 Acked Read 10 Sent Done Done false
27555 7813 Acked Read 8 Sent Done Done false
27556 7814 Acked Read 2 Sent Done Done false
27557 7815 Acked Write 9 Sent Done Done false
27558 7816 Acked Write 5 Sent Done Done false
27559 7817 Acked Write 4 Sent Done Done false
27560 7818 Acked Read 5 Sent Done Done false
27561 7819 Acked Read 8 Sent Done Done false
27562 7820 Acked Write 8 Sent Done Done false
27563 7821 Acked Write 6 Sent Done Done false
27564 7822 Acked Read 4 Sent Done Done false
27565 7823 Acked Read 4 Sent Done Done false
27566 7824 Acked Read 7 Sent Done Done false
27567 7825 Acked Write 10 Sent Done Done false
27568 7826 Acked Read 10 Sent Done Done false
27569 7827 Acked Read 7 Sent Done Done false
27570 7828 Acked Read 3 Sent Done Done false
27571 7829 Acked Write 7 Sent Done Done false
27572 7830 Acked Write 3 Sent Done Done false
27573 7831 Acked Write 7 Sent Done Done false
27574 7832 Acked Flush 0 Sent Done Done false
27575 7833 Acked Read 4 Sent Done Done false
27576 7834 Acked Write 1 Sent Done Done false
27577 7835 Acked Read 7 Sent Done Done false
27578 7836 Acked Read 4 Sent Done Done false
27579 7837 Acked Read 8 Sent Done Done false
27580 7838 Acked Flush 0 Sent Done Done false
27581 7839 Acked Read 5 Sent Done Done false
27582 7840 Acked Read 4 Sent Done Done false
27583 7841 Acked Write 9 Sent Done Done false
27584 7842 Acked Flush 0 Sent Done Done false
27585 7843 Acked Write 5 Sent Done Done false
27586 7844 Acked Read 4 Sent Done Done false
27587 7845 Acked Read 4 Sent Done Done false
27588 7846 Acked Read 9 Sent Done Done false
27589 7847 Acked Read 10 Sent Done Done false
27590 7848 Acked Read 6 Sent Done Done false
27591 7849 Acked Read 8 Sent Done Done false
27592 7850 Acked Read 2 Sent Done Done false
27593 7851 Acked Write 3 Sent Done Done false
27594 7852 Acked Read 10 Sent Done Done false
27595 7853 Acked Write 3 Sent Done Done false
27596 7854 Acked Read 1 Sent Done Done false
27597 7855 Acked Read 9 Sent Done Done false
27598 7856 Acked Write 3 Sent Done Done false
27599 7857 Acked Read 9 Sent Done Done false
27600 7858 Acked Write 6 Sent Done Done false
27601 7859 Acked Read 3 Sent Done Done false
27602 7860 Acked Read 7 Sent Done Done false
27603 7861 Acked Write 10 Sent Done Done false
27604 7862 Acked Read 3 Sent Done Done false
27605 7863 Acked Read 2 Sent Done Done false
27606 7864 Acked Read 1 Sent Done Done false
27607 7865 Acked Write 9 Sent Done Done false
27608 7866 Acked Read 2 Sent Done Done false
27609 7867 Acked Write 1 Sent Done Done false
27610 7868 Acked Flush 0 Sent Done Done false
27611 7869 Acked Write 9 Sent Done Done false
27612 7870 Acked Read 7 Sent Done Done false
27613 7871 Acked Write 1 Sent Done Done false
27614 7872 Acked Write 2 Sent Done Done false
27615 7873 Acked Read 8 Sent Done Done false
27616 7874 Acked Read 4 Sent Done Done false
27617 7875 Acked Write 8 Sent Done Done false
27618 7876 Acked Flush 0 Sent Done Done false
27619 7877 Acked Write 9 Sent Done Done false
27620 7878 Acked Write 7 Sent Done Done false
27621 7879 Acked Write 9 Sent Done Done false
27622 7880 Acked Read 6 Sent Done Done false
27623 7881 Acked Read 9 Sent Done Done false
27624 7882 Acked Read 6 Sent Done Done false
27625 7883 Acked Write 1 Sent Done Done false
27626 7884 Acked Read 2 Sent Done Done false
27627 7885 Acked Write 6 Sent Done Done false
27628 7886 Acked Flush 0 Sent Done Done false
27629 7887 Acked Read 5 Sent Done Done false
27630 7888 Acked Write 9 Sent Done Done false
27631 7889 Acked Write 1 Sent Done Done false
27632 7890 Acked Read 3 Sent Done Done false
27633 7891 Acked Write 6 Sent Done Done false
27634 7892 Acked Write 5 Sent Done Done false
27635 7893 Acked Flush 0 Sent Done Done false
27636 7894 Acked Read 2 Sent Done Done false
27637 7895 Acked Write 2 Sent Done Done false
27638 7896 Acked Write 6 Sent Done Done false
27639 7897 Acked Read 4 Sent Done Done false
27640 7898 Acked Write 6 Sent Done Done false
27641 7899 Acked Write 8 Sent Done Done false
27642 7900 Acked Write 9 Sent Done Done false
27643 7901 Acked Flush 0 Sent Done Done false
27644 7902 Acked Read 2 Sent Done Done false
27645 7903 Acked Write 8 Sent Done Done false
27646 7904 Acked Write 6 Sent Done Done false
27647 7905 Acked Flush 0 Sent Done Done false
27648 7906 Acked Write 10 Sent Done Done false
27649 7907 Acked Write 6 Sent Done Done false
27650 7908 Acked Read 7 Sent Done Done false
27651 7909 Acked Write 5 Sent Done Done false
27652 7910 Acked Read 7 Sent Done Done false
27653 7911 Acked Read 6 Sent Done Done false
27654 7912 Acked Read 6 Sent Done Done false
27655 7913 Acked Write 6 Sent Done Done false
27656 7914 Acked Write 9 Sent Done Done false
27657 7915 Acked Write 6 Sent Done Done false
27658 7916 Acked Read 10 Sent Done Done false
27659 7917 Acked Write 9 Sent Done Done false
27660 7918 Acked Read 6 Sent Done Done false
27661 7919 Acked Read 6 Sent Done Done false
27662 7920 Acked Write 2 Sent Done Done false
27663 7921 Acked Write 5 Sent Done Done false
27664 7922 Acked Write 8 Sent Done Done false
27665 7923 Acked Read 5 Sent Done Done false
27666 7924 Acked Read 6 Sent Done Done false
27667 7925 Acked Write 2 Sent Done Done false
27668 7926 Acked Read 7 Sent Done Done false
27669 7927 Acked Write 5 Sent Done Done false
27670 7928 Acked Read 10 Sent Done Done false
27671 7929 Acked Read 6 Sent Done Done false
27672 7930 Acked Write 7 Sent Done Done false
27673 7931 Acked Write 1 Sent Done Done false
27674 7932 Acked Read 8 Sent Done Done false
27675 7933 Acked Read 6 Sent Done Done false
27676 7934 Acked Write 10 Sent Done Done false
27677 7935 Acked Write 8 Sent Done Done false
27678 7936 Acked Write 9 Sent Done Done false
27679 7937 Acked Write 7 Sent Done Done false
27680 7938 Acked Write 2 Sent Done Done false
27681 7939 Acked Flush 0 Sent Done Done false
27682 7940 Acked Write 5 Sent Done Done false
27683 7941 Acked Read 4 Sent Done Done false
27684 7942 Acked Flush 0 Sent Done Done false
27685 STATES DS:0 DS:1 DS:2 TOTAL
27686 Sent 302 0 0 302
27687 Done 0 302 302 604
27688 Skipped 0 0 0 0
27689 Error 0 0 0 0
27690 Last Flush: 7640 7942 7942
27691 Downstairs last five completed: 7640 7639 7638 7637 7636
27692 Upstairs last five completed: 7942 7941 7940 7939 7938
27693 CLIENT: Up:0 ds:302 act:2
276942024-11-28T04:56:08.377ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
276952024-11-28T04:56:08.377ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:38668) = io task client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
276962024-11-28T04:56:08.378ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
276972024-11-28T04:56:08.378ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
276982024-11-28T04:56:08.378ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
276992024-11-28T04:56:08.378ZINFOcrucible: send last flush ID to this DS: 7640 = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
277002024-11-28T04:56:08.378ZINFOcrucible: Replied this last flush ID: 7640 = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
277012024-11-28T04:56:08.378ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
277022024-11-28T04:56:08.378ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
277032024-11-28T04:56:08.378ZINFOcrucible: [0] client re-new 302 jobs since flush 7640 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
277042024-11-28T04:56:08.379ZINFOcrucible: [0] Replayed 302 jobs since flush: 7640 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
27705 ----------------------------------------------------------------
27706 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
27707 Downstairs last five completed: 7942 7941 7940 7939 7938
27708 Upstairs last five completed: 7942 7941 7940 7939 7938
27709 CLIENT: Up:0 ds:0 act:3
27710 Replay: All jobs finished
27711 Main task has work to do, go find it
27712 got dsc Stop(2)
27713 stop 2
27714 Nov 28 04:56:08.620 INFO request completed, latency_us: 347, response_code: 204, uri: /stop/cid/2, method[8830] Got stop action so:false kr:false
27715 : POST, req_id: 19082323-8b95-424c-8fe3-94a6a6c4d851, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
277162024-11-28T04:56:08.621ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
277172024-11-28T04:56:08.621ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
277182024-11-28T04:56:08.621ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
277192024-11-28T04:56:08.621ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
277202024-11-28T04:56:08.621ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
277212024-11-28T04:56:08.621ZINFOcrucible: client task is exiting = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
27722 Nov 28 04:56:08.621 INFO request completed, latency_us: 173, response_code: 200, uri: /state/cid/2, method: GET, req_id: 43f80031-326b-4d54-b2ca-972a78b3431e, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
27723 [8830] Exited with: ExitStatus(unix_wait_status(9))
27724 [8830][2] reports Exit
27725 Nov 28 04:56:12.623 INFO request completed, latency_us: 211, response_code: 200, uri: /state/cid/2, method: GET, req_id: 67df9121-7a50-4f90-b422-47cd81f748d3, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
27726 001/300 Write block 2337 len 16384 data: 7 7 6 6
27727 002/300 Read block 3237 len 32768
27728 003/300 Read block 397 len 32768
27729 004/300 Read block 1948 len 36864
27730 005/300 Read block 3016 len 28672
27731 006/300 Write block 257 len 12288 data: 6 7 6
27732 007/300 Read block 976 len 8192
27733 008/300 Read block 1985 len 36864
27734 009/300 Read block 2345 len 20480
27735 010/300 Read block 2809 len 24576
27736 011/300 Read block 4003 len 32768
27737 012/300 Flush
27738 013/300 Write block 4545 len 16384 data: 4 5 4 5
27739 014/300 Read block 287 len 40960
27740 015/300 Write block 4598 len 12288 data: 7 7 8
27741 016/300 Write block 3411 len 20480 data: 5 6 7 5 5
27742 017/300 Read block 4854 len 36864
27743 018/300 Flush
27744 019/300 Write block 1967 len 28672 data: 6 5 5 5 5 4 5
27745 020/300 Write block 1647 len 20480 data: 3 4 5 3 3
27746 021/300 Flush
27747 022/300 Write block 2210 len 16384 data: 6 5 5 6
27748 023/300 Read block 4774 len 36864
27749 024/300 Read block 3779 len 12288
27750 025/300 Read block 3780 len 16384
27751 026/300 Read block 747 len 36864
27752 027/300 Flush
27753 028/300 Read block 230 len 28672
27754 029/300 Read block 2518 len 36864
27755 030/300 Write block 3687 len 24576 data: 5 6 6 6 7 5
27756 031/300 Read block 4370 len 24576
27757 032/300 Read block 543 len 12288
27758 033/300 Read block 249 len 8192
27759 034/300 Read block 3081 len 16384
27760 035/300 Flush
27761 036/300 Read block 3714 len 20480
27762 037/300 Write block 4252 len 16384 data: 4 4 2 3
27763 038/300 Read block 3465 len 28672
27764 039/300 Read block 3163 len 4096
27765 040/300 Write block 120 len 40960 data: 9 8 8 5 4 4 6 5 4 4
27766 041/300 Read block 4982 len 24576
27767 042/300 Read block 825 len 16384
27768 043/300 Read block 2010 len 32768
27769 044/300 Write block 2793 len 16384 data: 6 5 3 2
27770 045/300 Write block 4059 len 40960 data: 8 8 7 7 5 7 7 6 6 6
27771 046/300 Write block 1475 len 20480 data: 4 6 6 6 7
27772 047/300 Write block 2782 len 28672 data: 6 6 9 8 7 7 9
27773 048/300 Write block 4203 len 32768 data: 6 4 3 2 4 4 4 5
27774 049/300 Write block 988 len 12288 data: 6 7 6
27775 050/300 Read block 1423 len 4096
27776 051/300 Write block 3903 len 24576 data: 7 7 6 4 4 4
27777 052/300 Write block 109 len 16384 data: 11 8 5 5
27778 053/300 Read block 4397 len 28672
27779 054/300 Write block 4704 len 40960 data: 7 6 5 5 5 5 5 7 5 4
27780 055/300 Write block 3304 len 36864 data: 7 5 5 6 7 7 7 7 6
27781 056/300 Write block 844 len 28672 data: 4 5 6 6 7 7 6
27782 057/300 Write block 3883 len 32768 data: 5 4 3 4 5 5 6 6
27783 058/300 Read block 1567 len 24576
27784 059/300 Flush
27785 060/300 Read block 3115 len 8192
27786 061/300 Write block 2054 len 36864 data: 6 6 6 5 5 5 5 5 5
27787 062/300 Write block 4893 len 24576 data: 4 5 5 5 4 4
27788 063/300 Read block 360 len 24576
27789 064/300 Read block 4370 len 40960
27790 065/300 Write block 2475 len 36864 data: 6 7 4 4 4 3 3 3 4
27791 066/300 Read block 2147 len 24576
27792 067/300 Write block 2720 len 20480 data: 7 7 3 3 3
27793 068/300 Write block 2914 len 12288 data: 3 3 4
27794 069/300 Read block 2334 len 4096
27795 070/300 Read block 895 len 24576
27796 071/300 Flush
27797 072/300 Read block 3936 len 32768
27798 073/300 Write block 1189 len 28672 data: 3 4 3 4 4 4 5
27799 074/300 Read block 2071 len 12288
27800 075/300 Read block 404 len 24576
27801 076/300 Flush
27802 077/300 Read block 4358 len 36864
27803 078/300 Read block 324 len 40960
27804 079/300 Write block 588 len 28672 data: 3 3 4 4 4 5 5
27805 080/300 Write block 1019 len 20480 data: 4 5 6 5 4
27806 081/300 Write block 1607 len 12288 data: 5 5 6
27807 082/300 Write block 149 len 16384 data: 5 5 8 7
27808 083/300 Flush
27809 084/300 Read block 2154 len 20480
27810 085/300 Read block 4469 len 28672
27811 086/300 Read block 4973 len 16384
27812 087/300 Write block 517 len 20480 data: 5 4 3 3 3
27813 088/300 Write block 848 len 36864 data: 8 8 7 7 8 5 5 4 4
27814 089/300 Write block 4165 len 28672 data: 8 8 8 7 4 6 6
27815 090/300 Write block 3768 len 24576 data: 5 3 3 3 4 4
27816 091/300 Write block 4529 len 24576 data: 7 6 7 8 5 7
27817 092/300 Read block 1466 len 4096
27818 093/300 Read block 2716 len 28672
27819 094/300 Write block 1972 len 36864 data: 5 6 5 5 4 5 6 5 6
27820 095/300 Read block 2318 len 32768
27821 096/300 Write block 419 len 20480 data: 4 4 4 4 4
27822 097/300 Read block 4351 len 24576
27823 098/300 Read block 3318 len 20480
27824 099/300 Write block 4239 len 24576 data: 5 5 5 6 7 5
27825 100/300 Write block 3846 len 24576 data: 4 3 3 5 5 4
27826 101/300 Write block 2158 len 4096 data: 6
27827 102/300 Read block 4564 len 36864
27828 103/300 Write block 2469 len 32768 data: 6 6 8 7 6 6 7 8
27829 104/300 Write block 1882 len 16384 data: 7 7 7 5
27830 105/300 Write block 348 len 40960 data: 3 4 4 3 4 4 5 5 5 4
27831 106/300 Write block 4548 len 28672 data: 6 6 6 5 7 4 4
27832 107/300 Write block 1837 len 36864 data: 6 6 6 5 6 5 5 6 4
27833 108/300 Write block 2207 len 24576 data: 6 6 6 7 6 6
27834 109/300 Read block 3870 len 8192
27835 110/300 Read block 662 len 32768
27836 111/300 Read block 103 len 20480
27837 112/300 Read block 2891 len 20480
27838 113/300 Write block 1026 len 4096 data: 5
27839 114/300 Write block 2661 len 40960 data: 7 7 7 6 6 6 5 5 5 6
27840 115/300 Read block 1282 len 36864
27841 116/300 Read block 1912 len 24576
27842 117/300 Write block 667 len 32768 data: 9 9 7 7 7 8 4 4
27843 118/300 Write block 605 len 32768 data: 8 5 4 4 5 6 4 5
27844 119/300 Write block 721 len 32768 data: 4 4 4 3 4 4 5 4
27845 120/300 Read block 1214 len 24576
27846 121/300 Write block 4960 len 32768 data: 6 6 6 6 7 7 7 7
27847 122/300 Write block 2532 len 16384 data: 5 5 4 5
27848 123/300 Write block 1841 len 24576 data: 7 6 6 7 5 5
27849 124/300 Read block 4887 len 8192
27850 125/300 Read block 534 len 12288
27851 126/300 Read block 4475 len 32768
27852 127/300 Read block 4093 len 28672
27853 128/300 Write block 2046 len 16384 data: 3 4 3 3
27854 129/300 Write block 1497 len 4096 data: 9
27855 130/300 Read block 535 len 32768
27856 131/300 Flush
27857 132/300 Write block 39 len 28672 data: 6 5 5 6 5 5 5
27858 133/300 Read block 3102 len 36864
27859 134/300 Read block 4936 len 36864
27860 135/300 Read block 4235 len 24576
27861 136/300 Read block 1897 len 36864
27862 137/300 Read block 3331 len 24576
27863 138/300 Write block 3930 len 36864 data: 5 5 4 6 5 4 4 5 4
27864 139/300 Write block 686 len 4096 data: 5
27865 140/300 Write block 3040 len 12288 data: 5 5 5
27866 141/300 Read block 287 len 36864
27867 142/300 Read block 4617 len 12288
27868 143/300 Read block 3406 len 20480
27869 144/300 Flush
27870 145/300 Flush
27871 146/300 Read block 3699 len 16384
27872 147/300 Write block 1968 len 4096 data: 6
27873 148/300 Read block 3718 len 8192
27874 149/300 Write block 164 len 24576 data: 7 5 6 4 4 4
27875 150/300 Read block 1002 len 36864
27876 151/300 Write block 2403 len 20480 data: 4 5 4 2 3
27877 152/300 Read block 699 len 40960
27878 153/300 Write block 528 len 24576 data: 6 5 4 5 4 4
27879 154/300 Write block 3088 len 20480 data: 5 6 7 7 6
27880 155/300 Flush
27881 156/300 Write block 1334 len 16384 data: 4 5 7 7
27882 157/300 Read block 4459 len 16384
27883 158/300 Write block 3403 len 28672 data: 5 8 7 7 7 6 6
27884 159/300 Read block 94 len 32768
27885 160/300 Write block 1266 len 32768 data: 8 7 4 4 5 6 9 9
27886 161/300 Write block 2441 len 4096 data: 6
27887 162/300 Write block 1088 len 28672 data: 6 6 6 5 7 5 5
27888 163/300 Write block 2787 len 16384 data: 8 10 10 11
27889 164/300 Write block 722 len 28672 data: 5 5 4 5 5 6 5
27890 165/300 Write block 2365 len 32768 data: 6 4 4 6 5 5 5 5
27891 166/300 Read block 2691 len 36864
27892 167/300 Read block 1329 len 20480
27893 168/300 Flush
27894 169/300 Read block 4072 len 4096
27895 170/300 Write block 3763 len 32768 data: 4 4 5 5 6 6 4 4
27896 171/300 Read block 1100 len 12288
27897 172/300 Read block 2641 len 16384
27898 173/300 Flush
27899 174/300 Read block 2139 len 36864
27900 175/300 Write block 2753 len 16384 data: 9 8 6 6
27901 176/300 Flush
27902 177/300 Read block 2026 len 32768
27903 178/300 Write block 276 len 20480 data: 6 4 6 6 6
27904 179/300 Read block 2086 len 20480
27905 180/300 Write block 1914 len 20480 data: 6 5 5 5 4
27906 181/300 Read block 3147 len 24576
27907 182/300 Write block 1132 len 40960 data: 3 3 4 3 3 4 4 4 4 3
27908 183/300 Read block 1236 len 40960
27909 184/300 Read block 3391 len 24576
27910 185/300 Read block 1933 len 20480
27911 186/300 Write block 2880 len 20480 data: 7 7 5 3 3
27912 187/300 Write block 4418 len 24576 data: 6 8 7 6 6 6
27913 188/300 Write block 1339 len 12288 data: 7 6 7
27914 189/300 Write block 1836 len 16384 data: 5 7 7 7
27915 190/300 Read block 52 len 40960
27916 191/300 Read block 1021 len 40960
27917 192/300 Write block 2390 len 20480 data: 5 5 5 6 7
27918 193/300 Write block 2175 len 36864 data: 5 4 5 5 4 4 4 4 6
27919 194/300 Read block 2470 len 40960
27920 195/300 Read block 1297 len 12288
27921 196/300 Write block 3660 len 20480 data: 2 2 3 3 3
27922 197/300 Read block 2496 len 40960
27923 198/300 Read block 250 len 40960
27924 199/300 Read block 4465 len 12288
27925 200/300 Read block 1988 len 24576
27926 201/300 Write block 3814 len 8192 data: 5 6
27927 202/300 Read block 2562 len 8192
27928 203/300 Write block 1452 len 32768 data: 4 5 3 3 2 2 3 5
27929 204/300 Write block 992 len 32768 data: 7 6 6 6 6 7 6 7
27930 205/300 Read block 4714 len 40960
27931 206/300 Write block 1544 len 36864 data: 5 8 7 6 6 8 6 5 5
27932 207/300 Write block 1062 len 12288 data: 5 6 5
27933 208/300 Write block 1524 len 12288 data: 5 4 4
27934 209/300 Write block 1972 len 20480 data: 6 7 6 6 5
27935 210/300 Write block 438 len 12288 data: 8 7 7
27936 211/300 Read block 1569 len 36864
27937 212/300 Read block 4750 len 12288
27938 213/300 Read block 4876 len 12288
27939 214/300 Flush
27940 215/300 Read block 3311 len 12288
27941 216/300 Flush
27942 217/300 Read block 92 len 28672
27943 218/300 Read block 4790 len 8192
27944 219/300 Write block 2915 len 36864 data: 4 5 5 4 6 7 7 5 7
27945 220/300 Read block 3618 len 12288
27946 221/300 Write block 797 len 32768 data: 3 3 4 6 6 6 7 6
27947 222/300 Write block 1172 len 4096 data: 3
27948 223/300 Read block 4355 len 24576
27949 224/300 Write block 4572 len 8192 data: 5 5
27950 225/300 Write block 2909 len 16384 data: 4 4 4 4
27951 226/300 Read block 4889 len 16384
27952 227/300 Read block 3454 len 4096
27953 228/300 Write block 705 len 8192 data: 7 6
27954 229/300 Read block 3469 len 12288
27955 230/300 Write block 1317 len 12288 data: 2 3 3
27956 231/300 Read block 268 len 4096
27957 232/300 Read block 4921 len 32768
27958 233/300 Read block 3760 len 32768
27959 234/300 Write block 1320 len 28672 data: 4 4 5 4 4 4 4
27960 235/300 Read block 3355 len 28672
27961 236/300 Flush
27962 237/300 Read block 3928 len 20480
27963 238/300 Read block 4909 len 12288
27964 239/300 Write block 2797 len 40960 data: 2 2 2 2 3 3 3 4 6 5
27965 240/300 Read block 2333 len 4096
27966 241/300 Read block 3254 len 28672
27967 242/300 Read block 816 len 4096
27968 243/300 Write block 3546 len 36864 data: 2 2 3 4 4 5 5 5 5
27969 244/300 Read block 4074 len 20480
27970 245/300 Read block 4367 len 20480
27971 246/300 Read block 569 len 40960
27972 247/300 Write block 4239 len 32768 data: 6 6 6 7 8 6 7 8
27973 248/300 Read block 2917 len 36864
27974 249/300 Read block 581 len 8192
27975 250/300 Flush
27976 251/300 Read block 972 len 36864
27977 252/300 Write block 48 len 28672 data: 8 9 7 6 6 5 5
27978 253/300 Read block 1904 len 8192
27979 254/300 Write block 3797 len 36864 data: 7 10 8 7 9 8 8 7 9
27980 255/300 Read block 3745 len 36864
27981 256/300 Read block 188 len 12288
27982 257/300 Write block 2035 len 24576 data: 7 6 5 4 4 3
27983 258/300 Write block 4968 len 32768 data: 7 6 6 6 4 4 3 3
27984 259/300 Read block 3168 len 4096
27985 260/300 Read block 3827 len 16384
27986 261/300 Read block 768 len 4096
27987 262/300 Read block 500 len 24576
27988 263/300 Flush
27989 264/300 Read block 505 len 32768
27990 265/300 Write block 3476 len 40960 data: 7 7 6 6 6 5 4 4 4 5
27991 266/300 Write block 3581 len 40960 data: 7 6 5 5 5 4 5 5 4 4
27992 267/300 Write block 381 len 24576 data: 9 10 10 8 5 4
27993 268/300 Write block 4257 len 8192 data: 3 2
27994 269/300 Flush
27995 270/300 Write block 2913 len 16384 data: 4 4 5 6
27996 271/300 Read block 52 len 36864
27997 272/300 Read block 3303 len 36864
27998 273/300 Write block 4632 len 40960 data: 3 4 5 4 4 4 5 4 3 6
27999 274/300 Flush
28000 275/300 Read block 1704 len 28672
28001 276/300 Read block 1579 len 40960
28002 277/300 Read block 1764 len 16384
28003 278/300 Read block 529 len 28672
28004 279/300 Write block 651 len 28672 data: 5 5 5 5 6 7 8
28005 280/300 Write block 4015 len 4096 data: 3
28006 281/300 Write block 447 len 12288 data: 5 4 4
28007 282/300 Write block 1925 len 4096 data: 3
28008 283/300 Flush
28009 284/300 Write block 3733 len 36864 data: 5 5 5 4 3 5 4 3 4
28010 285/300 Read block 4268 len 4096
28011 286/300 Read block 3735 len 20480
28012 287/300 Flush
28013 288/300 Write block 3870 len 16384 data: 5 4 5 5
28014 289/300 Write block 1860 len 8192 data: 5 6
28015 290/300 Write block 1617 len 4096 data: 5
28016 291/300 Read block 1770 len 8192
28017 292/300 Flush
28018 293/300 Read block 2057 len 40960
28019 294/300 Write block 4363 len 40960 data: 9 9 8 7 4 4 4 3 5 3
28020 295/300 Write block 2781 len 8192 data: 8 7
28021 296/300 Write block 3236 len 12288 data: 5 7 7
28022 297/300 Write block 2193 len 28672 data: 5 6 7 6 5 4 4
28023 298/300 Write block 4727 len 4096 data: 5
28024 299/300 Flush
28025 300/300 Write block 3930 len 40960 data: 6 6 5 7 6 5 5 6 5 3
28026 301/300 Read block 4427 len 24576
28027 Main task has work to do, go find it
28028 got dsc Start(2)
28029 start 2
28030 Nov 28 04:56:12.858 INFO request completed, latency_us: 214, response_code: 204, uri: /start/cid/2, method: POST, req_id: 549f5efe-0893-43a8-8cf3-fc74964f5fdd, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
28031 Replay: started 2, returned:Ok(())
28032 ----------------------------------------------------------------
28033 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
28034 States: Active Active Offline
28035 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
28036 7943 Acked Write 4 Done Done Sent false
28037 7944 Acked Read 8 Done Done Sent false
28038 7945 Acked Read 8 Done Done Sent false
28039 7946 Acked Read 9 Done Done Sent false
28040 7947 Acked Read 7 Done Done Sent false
28041 7948 Acked Write 3 Done Done Sent false
28042 7949 Acked Read 2 Done Done Sent false
28043 7950 Acked Read 9 Done Done Sent false
28044 7951 Acked Read 5 Done Done Sent false
28045 7952 Acked Read 6 Done Done Sent false
28046 7953 Acked Read 8 Done Done Sent false
28047 7954 Acked Flush 0 Done Done Sent false
28048 7955 Acked Write 4 Done Done Sent false
28049 7956 Acked Read 10 Done Done Sent false
28050 7957 Acked Write 3 Done Done Sent false
28051 7958 Acked Write 5 Done Done Sent false
28052 7959 Acked Read 9 Done Done Sent false
28053 7960 Acked Flush 0 Done Done Sent false
28054 7961 Acked Write 7 Done Done Sent false
28055 7962 Acked Write 5 Done Done Sent false
28056 7963 Acked Flush 0 Done Done Sent false
28057 7964 Acked Write 4 Done Done Sent false
28058 7965 Acked Read 9 Done Done Sent false
28059 7966 Acked Read 3 Done Done Sent false
28060 7967 Acked Read 4 Done Done Sent false
28061 7968 Acked Read 9 Done Done Sent false
28062 7969 Acked Flush 0 Done Done Sent false
28063 7970 Acked Read 7 Done Done Sent false
28064 7971 Acked Read 9 Done Done Sent false
28065 7972 Acked Write 6 Done Done Sent false
28066 7973 Acked Read 6 Done Done Sent false
28067 7974 Acked Read 3 Done Done Sent false
28068 7975 Acked Read 2 Done Done Sent false
28069 7976 Acked Read 4 Done Done Sent false
28070 7977 Acked Flush 0 Done Done Sent false
28071 7978 Acked Read 5 Done Done Sent false
28072 7979 Acked Write 4 Done Done Sent false
28073 7980 Acked Read 7 Done Done Sent false
28074 7981 Acked Read 1 Done Done Sent false
28075 7982 Acked Write 10 Done Done Sent false
28076 7983 Acked Read 6 Done Done Sent false
28077 7984 Acked Read 4 Done Done Sent false
28078 7985 Acked Read 8 Done Done Sent false
28079 7986 Acked Write 4 Done Done Sent false
28080 7987 Acked Write 10 Done Done Sent false
28081 7988 Acked Write 5 Done Done Sent false
28082 7989 Acked Write 7 Done Done Sent false
28083 7990 Acked Write 8 Done Done Sent false
28084 7991 Acked Write 3 Done Done Sent false
28085 7992 Acked Read 1 Done Done Sent false
28086 7993 Acked Write 6 Done Done Sent false
28087 7994 Acked Write 4 Done Done Sent false
28088 7995 Acked Read 7 Done Done Sent false
28089 7996 Acked Write 10 Done Done Sent false
28090 7997 Acked Write 9 Done Done Sent false
28091 7998 Acked Write 7 Done Done Sent false
28092 7999 Acked Write 8 Done Done Sent false
28093 8000 Acked Read 6 Done Done Sent false
28094 8001 Acked Flush 0 Done Done Sent false
28095 8002 Acked Read 2 Done Done Sent false
28096 8003 Acked Write 9 Done Done Sent false
28097 8004 Acked Write 6 Done Done Sent false
28098 8005 Acked Read 6 Done Done Sent false
28099 8006 Acked Read 10 Done Done Sent false
28100 8007 Acked Write 9 Done Done Sent false
28101 8008 Acked Read 6 Done Done Sent false
28102 8009 Acked Write 5 Done Done Sent false
28103 8010 Acked Write 3 Done Done Sent false
28104 8011 Acked Read 1 Done Done Sent false
28105 8012 Acked Read 6 Done Done Sent false
28106 8013 Acked Flush 0 Done Done Sent false
28107 8014 Acked Read 8 Done Done Sent false
28108 8015 Acked Write 7 Done Done Sent false
28109 8016 Acked Read 3 Done Done Sent false
28110 8017 Acked Read 6 Done Done Sent false
28111 8018 Acked Flush 0 Done Done Sent false
28112 8019 Acked Read 9 Done Done Sent false
28113 8020 Acked Read 10 Done Done Sent false
28114 8021 Acked Write 7 Done Done Sent false
28115 8022 Acked Write 5 Done Done Sent false
28116 8023 Acked Write 3 Done Done Sent false
28117 8024 Acked Write 4 Done Done Sent false
28118 8025 Acked Flush 0 Done Done Sent false
28119 8026 Acked Read 5 Done Done Sent false
28120 8027 Acked Read 7 Done Done Sent false
28121 8028 Acked Read 4 Done Done Sent false
28122 8029 Acked Write 5 Done Done Sent false
28123 8030 Acked Write 9 Done Done Sent false
28124 8031 Acked Write 7 Done Done Sent false
28125 8032 Acked Write 6 Done Done Sent false
28126 8033 Acked Write 6 Done Done Sent false
28127 8034 Acked Read 1 Done Done Sent false
28128 8035 Acked Read 7 Done Done Sent false
28129 8036 Acked Write 9 Done Done Sent false
28130 8037 Acked Read 8 Done Done Sent false
28131 8038 Acked Write 5 Done Done Sent false
28132 8039 Acked Read 6 Done Done Sent false
28133 8040 Acked Read 5 Done Done Sent false
28134 8041 Acked Write 6 Done Done Sent false
28135 8042 Acked Write 6 Done Done Sent false
28136 8043 Acked Write 1 Done Done Sent false
28137 8044 Acked Read 9 Done Done Sent false
28138 8045 Acked Write 8 Done Done Sent false
28139 8046 Acked Write 4 Done Done Sent false
28140 8047 Acked Write 10 Done Done Sent false
28141 8048 Acked Write 7 Done Done Sent false
28142 8049 Acked Write 9 Done Done Sent false
28143 8050 Acked Write 6 Done Done Sent false
28144 8051 Acked Read 2 Done Done Sent false
28145 8052 Acked Read 8 Done Done Sent false
28146 8053 Acked Read 5 Done Done Sent false
28147 8054 Acked Read 5 Done Done Sent false
28148 8055 Acked Write 1 Done Done Sent false
28149 8056 Acked Write 10 Done Done Sent false
28150 8057 Acked Read 9 Done Done Sent false
28151 8058 Acked Read 6 Done Done Sent false
28152 8059 Acked Write 8 Done Done Sent false
28153 8060 Acked Write 8 Done Done Sent false
28154 8061 Acked Write 8 Done Done Sent false
28155 8062 Acked Read 6 Done Done Sent false
28156 8063 Acked Write 8 Done Done Sent false
28157 8064 Acked Write 4 Done Done Sent false
28158 8065 Acked Write 6 Done Done Sent false
28159 8066 Acked Read 2 Done Done Sent false
28160 8067 Acked Read 3 Done Done Sent false
28161 8068 Acked Read 8 Done Done Sent false
28162 8069 Acked Read 7 Done Done Sent false
28163 8070 Acked Write 4 Done Done Sent false
28164 8071 Acked Write 1 Done Done Sent false
28165 8072 Acked Read 8 Done Done Sent false
28166 8073 Acked Flush 0 Done Done Sent false
28167 8074 Acked Write 7 Done Done Sent false
28168 8075 Acked Read 9 Done Done Sent false
28169 8076 Acked Read 9 Done Done Sent false
28170 8077 Acked Read 6 Done Done Sent false
28171 8078 Acked Read 9 Done Done Sent false
28172 8079 Acked Read 6 Done Done Sent false
28173 8080 Acked Write 9 Done Done Sent false
28174 8081 Acked Write 1 Done Done Sent false
28175 8082 Acked Write 3 Done Done Sent false
28176 8083 Acked Read 9 Done Done Sent false
28177 8084 Acked Read 3 Done Done Sent false
28178 8085 Acked Read 5 Done Done Sent false
28179 8086 Acked Flush 0 Done Done Sent false
28180 8087 Acked Flush 0 Done Done Sent false
28181 8088 Acked Read 4 Done Done Sent false
28182 8089 Acked Write 1 Done Done Sent false
28183 8090 Acked Read 2 Done Done Sent false
28184 8091 Acked Write 6 Done Done Sent false
28185 8092 Acked Read 9 Done Done Sent false
28186 8093 Acked Write 5 Done Done Sent false
28187 8094 Acked Read 10 Done Done Sent false
28188 8095 Acked Write 6 Done Done Sent false
28189 8096 Acked Write 5 Done Done Sent false
28190 8097 Acked Flush 0 Done Done Sent false
28191 8098 Acked Write 4 Done Done Sent false
28192 8099 Acked Read 4 Done Done Sent false
28193 8100 Acked Write 7 Done Done Sent false
28194 8101 Acked Read 8 Done Done Sent false
28195 8102 Acked Write 8 Done Done Sent false
28196 8103 Acked Write 1 Done Done Sent false
28197 8104 Acked Write 7 Done Done Sent false
28198 8105 Acked Write 4 Done Done Sent false
28199 8106 Acked Write 7 Done Done Sent false
28200 8107 Acked Write 8 Done Done Sent false
28201 8108 Acked Read 9 Done Done Sent false
28202 8109 Acked Read 5 Done Done Sent false
28203 8110 Acked Flush 0 Done Done Sent false
28204 8111 Acked Read 1 Done Done Sent false
28205 8112 Acked Write 8 Done Done Sent false
28206 8113 Acked Read 3 Done Done Sent false
28207 8114 Acked Read 4 Done Done Sent false
28208 8115 Acked Flush 0 Done Done Sent false
28209 8116 Acked Read 9 Done Done Sent false
28210 8117 Acked Write 4 Done Done Sent false
28211 8118 Acked Flush 0 Done Done Sent false
28212 8119 Acked Read 8 Done Done Sent false
28213 8120 Acked Write 5 Done Done Sent false
28214 8121 Acked Read 5 Done Done Sent false
28215 8122 Acked Write 5 Done Done Sent false
28216 8123 Acked Read 6 Done Done Sent false
28217 8124 Acked Write 10 Done Done Sent false
28218 8125 Acked Read 10 Done Done Sent false
28219 8126 Acked Read 6 Done Done Sent false
28220 8127 Acked Read 5 Done Done Sent false
28221 8128 Acked Write 5 Done Done Sent false
28222 8129 Acked Write 6 Done Done Sent false
28223 8130 Acked Write 3 Done Done Sent false
28224 8131 Acked Write 4 Done Done Sent false
28225 8132 Acked Read 10 Done Done Sent false
28226 8133 Acked Read 10 Done Done Sent false
28227 8134 Acked Write 5 Done Done Sent false
28228 8135 Acked Write 9 Done Done Sent false
28229 8136 Acked Read 10 Done Done Sent false
28230 8137 Acked Read 3 Done Done Sent false
28231 8138 Acked Write 5 Done Done Sent false
28232 8139 Acked Read 10 Done Done Sent false
28233 8140 Acked Read 10 Done Done Sent false
28234 8141 Acked Read 3 Done Done Sent false
28235 8142 Acked Read 6 Done Done Sent false
28236 8143 Acked Write 2 Done Done Sent false
28237 8144 Acked Read 2 Done Done Sent false
28238 8145 Acked Write 8 Done Done Sent false
28239 8146 Acked Write 8 Done Done Sent false
28240 8147 Acked Read 10 Done Done Sent false
28241 8148 Acked Write 9 Done Done Sent false
28242 8149 Acked Write 3 Done Done Sent false
28243 8150 Acked Write 3 Done Done Sent false
28244 8151 Acked Write 5 Done Done Sent false
28245 8152 Acked Write 3 Done Done Sent false
28246 8153 Acked Read 9 Done Done Sent false
28247 8154 Acked Read 3 Done Done Sent false
28248 8155 Acked Read 3 Done Done Sent false
28249 8156 Acked Flush 0 Done Done Sent false
28250 8157 Acked Read 3 Done Done Sent false
28251 8158 Acked Flush 0 Done Done Sent false
28252 8159 Acked Read 7 Done Done Sent false
28253 8160 Acked Read 2 Done Done Sent false
28254 8161 Acked Write 9 Done Done Sent false
28255 8162 Acked Read 3 Done Done Sent false
28256 8163 Acked Write 8 Done Done Sent false
28257 8164 Acked Write 1 Done Done Sent false
28258 8165 Acked Read 6 Done Done Sent false
28259 8166 Acked Write 2 Done Done Sent false
28260 8167 Acked Write 4 Done Done Sent false
28261 8168 Acked Read 4 Done Done Sent false
28262 8169 Acked Read 1 Done Done Sent false
28263 8170 Acked Write 2 Done Done Sent false
28264 8171 Acked Read 3 Done Done Sent false
28265 8172 Acked Write 3 Done Done Sent false
28266 8173 Acked Read 1 Done Done Sent false
28267 8174 Acked Read 8 Done Done Sent false
28268 8175 Acked Read 8 Done Done Sent false
28269 8176 Acked Write 7 Done Done Sent false
28270 8177 Acked Read 7 Done Done Sent false
28271 8178 Acked Flush 0 Done Done Sent false
28272 8179 Acked Read 5 Done Done Sent false
28273 8180 Acked Read 3 Done Done Sent false
28274 8181 Acked Write 10 Done Done Sent false
28275 8182 Acked Read 1 Done Done Sent false
28276 8183 Acked Read 7 Done Done Sent false
28277 8184 Acked Read 1 Done Done Sent false
28278 8185 Acked Write 9 Done Done Sent false
28279 8186 Acked Read 5 Done Done Sent false
28280 8187 Acked Read 5 Done Done Sent false
28281 8188 Acked Read 10 Done Done Sent false
28282 8189 Acked Write 8 Done Done Sent false
28283 8190 Acked Read 9 Done Done Sent false
28284 8191 Acked Read 2 Done Done Sent false
28285 8192 Acked Flush 0 Done Done Sent false
28286 8193 Acked Read 9 Done Done Sent false
28287 8194 Acked Write 7 Done Done Sent false
28288 8195 Acked Read 2 Done Done Sent false
28289 8196 Acked Write 9 Done Done Sent false
28290 8197 Acked Read 9 Done Done Sent false
28291 8198 Acked Read 3 Done Done Sent false
28292 8199 Acked Write 6 Done Done Sent false
28293 8200 Acked Write 8 Done Done Sent false
28294 8201 Acked Read 1 Done Done Sent false
28295 8202 Acked Read 4 Done Done Sent false
28296 8203 Acked Read 1 Done Done Sent false
28297 8204 Acked Read 6 Done Done Sent false
28298 8205 Acked Flush 0 Done Done Sent false
28299 8206 Acked Read 8 Done Done Sent false
28300 8207 Acked Write 10 Done Done Sent false
28301 8208 Acked Write 10 Done Done Sent false
28302 8209 Acked Write 6 Done Done Sent false
28303 8210 Acked Write 2 Done Done Sent false
28304 8211 Acked Flush 0 Done Done Sent false
28305 8212 Acked Write 4 Done Done Sent false
28306 8213 Acked Read 9 Done Done Sent false
28307 8214 Acked Read 9 Done Done Sent false
28308 8215 Acked Write 10 Done Done Sent false
28309 8216 Acked Flush 0 Done Done Sent false
28310 8217 Acked Read 7 Done Done Sent false
28311 8218 Acked Read 10 Done Done Sent false
28312 8219 Acked Read 4 Done Done Sent false
28313 8220 Acked Read 7 Done Done Sent false
28314 8221 Acked Write 7 Done Done Sent false
28315 8222 Acked Write 1 Done Done Sent false
28316 8223 Acked Write 3 Done Done Sent false
28317 8224 Acked Write 1 Done Done Sent false
28318 8225 Acked Flush 0 Done Done Sent false
28319 8226 Acked Write 9 Done Done Sent false
28320 8227 Acked Read 1 Done Done Sent false
28321 8228 Acked Read 5 Done Done Sent false
28322 8229 Acked Flush 0 Done Done Sent false
28323 8230 Acked Write 4 Done Done Sent false
28324 8231 Acked Write 2 Done Done Sent false
28325 8232 Acked Write 1 Done Done Sent false
28326 8233 Acked Read 2 Done Done Sent false
28327 8234 Acked Flush 0 Done Done Sent false
28328 8235 Acked Read 10 Done Done Sent false
28329 8236 Acked Write 10 Done Done Sent false
28330 8237 Acked Write 2 Done Done Sent false
28331 8238 Acked Write 3 Done Done Sent false
28332 8239 Acked Write 7 Done Done Sent false
28333 8240 Acked Write 1 Done Done Sent false
28334 8241 Acked Flush 0 Done Done Sent false
28335 8242 Acked Write 10 Done Done Sent false
28336 8243 Acked Read 6 Done Done Sent false
28337 STATES DS:0 DS:1 DS:2 TOTAL
28338 Sent 0 0 301 301
28339 Done 301 301 0 602
28340 Skipped 0 0 0 0
28341 Error 0 0 0 0
28342 Last Flush: 8241 8241 7942
28343 Downstairs last five completed: 7942 7941 7940 7939 7938
28344 Upstairs last five completed: 8243 8242 8241 8240 8239
28345 CLIENT: Up:0 ds:301 act:2
28346 ----------------------------------------------------------------
28347 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
28348 States: Active Active Offline
28349 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
28350 7943 Acked Write 4 Done Done Sent false
28351 7944 Acked Read 8 Done Done Sent false
28352 7945 Acked Read 8 Done Done Sent false
28353 7946 Acked Read 9 Done Done Sent false
28354 7947 Acked Read 7 Done Done Sent false
28355 7948 Acked Write 3 Done Done Sent false
28356 7949 Acked Read 2 Done Done Sent false
28357 7950 Acked Read 9 Done Done Sent false
28358 7951 Acked Read 5 Done Done Sent false
28359 7952 Acked Read 6 Done Done Sent false
28360 7953 Acked Read 8 Done Done Sent false
28361 7954 Acked Flush 0 Done Done Sent false
28362 7955 Acked Write 4 Done Done Sent false
28363 7956 Acked Read 10 Done Done Sent false
28364 7957 Acked Write 3 Done Done Sent false
28365 7958 Acked Write 5 Done Done Sent false
28366 7959 Acked Read 9 Done Done Sent false
28367 7960 Acked Flush 0 Done Done Sent false
28368 7961 Acked Write 7 Done Done Sent false
28369 7962 Acked Write 5 Done Done Sent false
28370 7963 Acked Flush 0 Done Done Sent false
28371 7964 Acked Write 4 Done Done Sent false
28372 7965 Acked Read 9 Done Done Sent false
28373 7966 Acked Read 3 Done Done Sent false
28374 7967 Acked Read 4 Done Done Sent false
28375 7968 Acked Read 9 Done Done Sent false
28376 7969 Acked Flush 0 Done Done Sent false
28377 7970 Acked Read 7 Done Done Sent false
28378 7971 Acked Read 9 Done Done Sent false
28379 7972 Acked Write 6 Done Done Sent false
28380 7973 Acked Read 6 Done Done Sent false
28381 7974 Acked Read 3 Done Done Sent false
28382 7975 Acked Read 2 Done Done Sent false
28383 7976 Acked Read 4 Done Done Sent false
28384 7977 Acked Flush 0 Done Done Sent false
28385 7978 Acked Read 5 Done Done Sent false
28386 7979 Acked Write 4 Done Done Sent false
28387 7980 Acked Read 7 Done Done Sent false
28388 7981 Acked Read 1 Done Done Sent false
28389 7982 Acked Write 10 Done Done Sent false
28390 7983 Acked Read 6 Done Done Sent false
28391 7984 Acked Read 4 Done Done Sent false
28392 7985 Acked Read 8 Done Done Sent false
28393 7986 Acked Write 4 Done Done Sent false
28394 7987 Acked Write 10 Done Done Sent false
28395 7988 Acked Write 5 Done Done Sent false
28396 7989 Acked Write 7 Done Done Sent false
28397 7990 Acked Write 8 Done Done Sent false
28398 7991 Acked Write 3 Done Done Sent false
28399 7992 Acked Read 1 Done Done Sent false
28400 7993 Acked Write 6 Done Done Sent false
28401 7994 Acked Write 4 Done Done Sent false
28402 7995 Acked Read 7 Done Done Sent false
28403 7996 Acked Write 10 Done Done Sent false
28404 7997 Acked Write 9 Done Done Sent false
28405 7998 Acked Write 7 Done Done Sent false
28406 7999 Acked Write 8 Done Done Sent false
28407 8000 Acked Read 6 Done Done Sent false
28408 8001 Acked Flush 0 Done Done Sent false
28409 8002 Acked Read 2 Done Done Sent false
28410 8003 Acked Write 9 Done Done Sent false
28411 8004 Acked Write 6 Done Done Sent false
28412 8005 Acked Read 6 Done Done Sent false
28413 8006 Acked Read 10 Done Done Sent false
28414 8007 Acked Write 9 Done Done Sent false
28415 8008 Acked Read 6 Done Done Sent false
28416 8009 Acked Write 5 Done Done Sent false
28417 8010 Acked Write 3 Done Done Sent false
28418 8011 Acked Read 1 Done Done Sent false
28419 8012 Acked Read 6 Done Done Sent false
28420 8013 Acked Flush 0 Done Done Sent false
28421 8014 Acked Read 8 Done Done Sent false
28422 8015 Acked Write 7 Done Done Sent false
28423 8016 Acked Read 3 Done Done Sent false
28424 8017 Acked Read 6 Done Done Sent false
28425 8018 Acked Flush 0 Done Done Sent false
28426 8019 Acked Read 9 Done Done Sent false
28427 8020 Acked Read 10 Done Done Sent false
28428 8021 Acked Write 7 Done Done Sent false
28429 8022 Acked Write 5 Done Done Sent false
28430 8023 Acked Write 3 Done Done Sent false
28431 8024 Acked Write 4 Done Done Sent false
28432 8025 Acked Flush 0 Done Done Sent false
28433 8026 Acked Read 5 Done Done Sent false
28434 8027 Acked Read 7 Done Done Sent false
28435 8028 Acked Read 4 Done Done Sent false
28436 8029 Acked Write 5 Done Done Sent false
28437 8030 Acked Write 9 Done Done Sent false
28438 8031 Acked Write 7 Done Done Sent false
28439 8032 Acked Write 6 Done Done Sent false
28440 8033 Acked Write 6 Done Done Sent false
28441 8034 Acked Read 1 Done Done Sent false
28442 8035 Acked Read 7 Done Done Sent false
28443 8036 Acked Write 9 Done Done Sent false
28444 8037 Acked Read 8 Done Done Sent false
28445 8038 Acked Write 5 Done Done Sent false
28446 8039 Acked Read 6 Done Done Sent false
28447 8040 Acked Read 5 Done Done Sent false
28448 8041 Acked Write 6 Done Done Sent false
28449 8042 Acked Write 6 Done Done Sent false
28450 8043 Acked Write 1 Done Done Sent false
28451 8044 Acked Read 9 Done Done Sent false
28452 8045 Acked Write 8 Done Done Sent false
28453 8046 Acked Write 4 Done Done Sent false
28454 8047 Acked Write 10 Done Done Sent false
28455 8048 Acked Write 7 Done Done Sent false
28456 8049 Acked Write 9 Done Done Sent false
28457 8050 Acked Write 6 Done Done Sent false
28458 8051 Acked Read 2 Done Done Sent false
28459 8052 Acked Read 8 Done Done Sent false
28460 8053 Acked Read 5 Done Done Sent false
28461 8054 Acked Read 5 Done Done Sent false
28462 8055 Acked Write 1 Done Done Sent false
28463 8056 Acked Write 10 Done Done Sent false
28464 8057 Acked Read 9 Done Done Sent false
28465 8058 Acked Read 6 Done Done Sent false
28466 8059 Acked Write 8 Done Done Sent false
28467 8060 Acked Write 8 Done Done Sent false
28468 8061 Acked Write 8 Done Done Sent false
28469 8062 Acked Read 6 Done Done Sent false
28470 8063 Acked Write 8 Done Done Sent false
28471 8064 Acked Write 4 Done Done Sent false
28472 8065 Acked Write 6 Done Done Sent false
28473 8066 Acked Read 2 Done Done Sent false
28474 8067 Acked Read 3 Done Done Sent false
28475 8068 Acked Read 8 Done Done Sent false
28476 8069 Acked Read 7 Done Done Sent false
28477 8070 Acked Write 4 Done Done Sent false
28478 8071 Acked Write 1 Done Done Sent false
28479 8072 Acked Read 8 Done Done Sent false
28480 8073 Acked Flush 0 Done Done Sent false
28481 8074 Acked Write 7 Done Done Sent false
28482 8075 Acked Read 9 Done Done Sent false
28483 8076 Acked Read 9 Done Done Sent false
28484 8077 Acked Read 6 Done Done Sent false
28485 8078 Acked Read 9 Done Done Sent false
28486 8079 Acked Read 6 Done Done Sent false
28487 8080 Acked Write 9 Done Done Sent false
28488 8081 Acked Write 1 Done Done Sent false
28489 8082 Acked Write 3 Done Done Sent false
28490 8083 Acked Read 9 Done Done Sent false
28491 8084 Acked Read 3 Done Done Sent false
28492 8085 Acked Read 5 Done Done Sent false
28493 8086 Acked Flush 0 Done Done Sent false
28494 8087 Acked Flush 0 Done Done Sent false
28495 8088 Acked Read 4 Done Done Sent false
28496 8089 Acked Write 1 Done Done Sent false
28497 8090 Acked Read 2 Done Done Sent false
28498 8091 Acked Write 6 Done Done Sent false
28499 8092 Acked Read 9 Done Done Sent false
28500 8093 Acked Write 5 Done Done Sent false
28501 8094 Acked Read 10 Done Done Sent false
28502 8095 Acked Write 6 Done Done Sent false
28503 8096 Acked Write 5 Done Done Sent false
28504 8097 Acked Flush 0 Done Done Sent false
28505 8098 Acked Write 4 Done Done Sent false
28506 8099 Acked Read 4 Done Done Sent false
28507 8100 Acked Write 7 Done Done Sent false
28508 8101 Acked Read 8 Done Done Sent false
28509 8102 Acked Write 8 Done Done Sent false
28510 8103 Acked Write 1 Done Done Sent false
28511 8104 Acked Write 7 Done Done Sent false
28512 8105 Acked Write 4 Done Done Sent false
28513 8106 Acked Write 7 Done Done Sent false
28514 8107 Acked Write 8 Done Done Sent false
28515 8108 Acked Read 9 Done Done Sent false
28516 8109 Acked Read 5 Done Done Sent false
28517 8110 Acked Flush 0 Done Done Sent false
28518 8111 Acked Read 1 Done Done Sent false
28519 8112 Acked Write 8 Done Done Sent false
28520 8113 Acked Read 3 Done Done Sent false
28521 8114 Acked Read 4 Done Done Sent false
28522 8115 Acked Flush 0 Done Done Sent false
28523 8116 Acked Read 9 Done Done Sent false
28524 8117 Acked Write 4 Done Done Sent false
28525 8118 Acked Flush 0 Done Done Sent false
28526 8119 Acked Read 8 Done Done Sent false
28527 8120 Acked Write 5 Done Done Sent false
28528 8121 Acked Read 5 Done Done Sent false
28529 8122 Acked Write 5 Done Done Sent false
28530 8123 Acked Read 6 Done Done Sent false
28531 8124 Acked Write 10 Done Done Sent false
28532 8125 Acked Read 10 Done Done Sent false
28533 8126 Acked Read 6 Done Done Sent false
28534 8127 Acked Read 5 Done Done Sent false
28535 8128 Acked Write 5 Done Done Sent false
28536 8129 Acked Write 6 Done Done Sent false
28537 8130 Acked Write 3 Done Done Sent false
28538 8131 Acked Write 4 Done Done Sent false
28539 8132 Acked Read 10 Done Done Sent false
28540 8133 Acked Read 10 Done Done Sent false
28541 8134 Acked Write 5 Done Done Sent false
28542 8135 Acked Write 9 Done Done Sent false
28543 8136 Acked Read 10 Done Done Sent false
28544 8137 Acked Read 3 Done Done Sent false
28545 8138 Acked Write 5 Done Done Sent false
28546 8139 Acked Read 10 Done Done Sent false
28547 8140 Acked Read 10 Done Done Sent false
28548 8141 Acked Read 3 Done Done Sent false
28549 8142 Acked Read 6 Done Done Sent false
28550 8143 Acked Write 2 Done Done Sent false
28551 8144 Acked Read 2 Done Done Sent false
28552 8145 Acked Write 8 Done Done Sent false
28553 8146 Acked Write 8 Done Done Sent false
28554 8147 Acked Read 10 Done Done Sent false
28555 8148 Acked Write 9 Done Done Sent false
28556 8149 Acked Write 3 Done Done Sent false
28557 8150 Acked Write 3 Done Done Sent false
28558 8151 Acked Write 5 Done Done Sent false
28559 8152 Acked Write 3 Done Done Sent false
28560 8153 Acked Read 9 Done Done Sent false
28561 8154 Acked Read 3 Done Done Sent false
28562 8155 Acked Read 3 Done Done Sent false
28563 8156 Acked Flush 0 Done Done Sent false
28564 8157 Acked Read 3 Done Done Sent false
28565 8158 Acked Flush 0 Done Done Sent false
28566 8159 Acked Read 7 Done Done Sent false
28567 8160 Acked Read 2 Done Done Sent false
28568 8161 Acked Write 9 Done Done Sent false
28569 8162 Acked Read 3 Done Done Sent false
28570 8163 Acked Write 8 Done Done Sent false
28571 8164 Acked Write 1 Done Done Sent false
28572 8165 Acked Read 6 Done Done Sent false
28573 8166 Acked Write 2 Done Done Sent false
28574 8167 Acked Write 4 Done Done Sent false
28575 8168 Acked Read 4 Done Done Sent false
28576 8169 Acked Read 1 Done Done Sent false
28577 8170 Acked Write 2 Done Done Sent false
28578 8171 Acked Read 3 Done Done Sent false
28579 8172 Acked Write 3 Done Done Sent false
28580 8173 Acked Read 1 Done Done Sent false
28581 8174 Acked Read 8 Done Done Sent false
28582 8175 Acked Read 8 Done Done Sent false
28583 8176 Acked Write 7 Done Done Sent false
28584 8177 Acked Read 7 Done Done Sent false
28585 8178 Acked Flush 0 Done Done Sent false
28586 8179 Acked Read 5 Done Done Sent false
28587 8180 Acked Read 3 Done Done Sent false
28588 8181 Acked Write 10 Done Done Sent false
28589 8182 Acked Read 1 Done Done Sent false
28590 8183 Acked Read 7 Done Done Sent false
28591 8184 Acked Read 1 Done Done Sent false
28592 8185 Acked Write 9 Done Done Sent false
28593 8186 Acked Read 5 Done Done Sent false
28594 8187 Acked Read 5 Done Done Sent false
28595 8188 Acked Read 10 Done Done Sent false
28596 8189 Acked Write 8 Done Done Sent false
28597 8190 Acked Read 9 Done Done Sent false
28598 8191 Acked Read 2 Done Done Sent false
28599 8192 Acked Flush 0 Done Done Sent false
28600 8193 Acked Read 9 Done Done Sent false
28601 8194 Acked Write 7 Done Done Sent false
28602 8195 Acked Read 2 Done Done Sent false
28603 8196 Acked Write 9 Done Done Sent false
28604 8197 Acked Read 9 Done Done Sent false
28605 8198 Acked Read 3 Done Done Sent false
28606 8199 Acked Write 6 Done Done Sent false
28607 8200 Acked Write 8 Done Done Sent false
28608 8201 Acked Read 1 Done Done Sent false
28609 8202 Acked Read 4 Done Done Sent false
28610 8203 Acked Read 1 Done Done Sent false
28611 8204 Acked Read 6 Done Done Sent false
28612 8205 Acked Flush 0 Done Done Sent false
28613 8206 Acked Read 8 Done Done Sent false
28614 8207 Acked Write 10 Done Done Sent false
28615 8208 Acked Write 10 Done Done Sent false
28616 8209 Acked Write 6 Done Done Sent false
28617 8210 Acked Write 2 Done Done Sent false
28618 8211 Acked Flush 0 Done Done Sent false
28619 8212 Acked Write 4 Done Done Sent false
28620 8213 Acked Read 9 Done Done Sent false
28621 8214 Acked Read 9 Done Done Sent false
28622 8215 Acked Write 10 Done Done Sent false
28623 8216 Acked Flush 0 Done Done Sent false
28624 8217 Acked Read 7 Done Done Sent false
28625 8218 Acked Read 10 Done Done Sent false
28626 8219 Acked Read 4 Done Done Sent false
28627 8220 Acked Read 7 Done Done Sent false
28628 8221 Acked Write 7 Done Done Sent false
28629 8222 Acked Write 1 Done Done Sent false
28630 8223 Acked Write 3 Done Done Sent false
28631 8224 Acked Write 1 Done Done Sent false
28632 8225 Acked Flush 0 Done Done Sent false
28633 8226 Acked Write 9 Done Done Sent false
28634 8227 Acked Read 1 Done Done Sent false
28635 8228 Acked Read 5 Done Done Sent false
28636 8229 Acked Flush 0 Done Done Sent false
28637 8230 Acked Write 4 Done Done Sent false
28638 8231 Acked Write 2 Done Done Sent false
28639 8232 Acked Write 1 Done Done Sent false
28640 8233 Acked Read 2 Done Done Sent false
28641 8234 Acked Flush 0 Done Done Sent false
28642 8235 Acked Read 10 Done Done Sent false
28643 8236 Acked Write 10 Done Done Sent false
28644 8237 Acked Write 2 Done Done Sent false
28645 8238 Acked Write 3 Done Done Sent false
28646 8239 Acked Write 7 Done Done Sent false
28647 8240 Acked Write 1 Done Done Sent false
28648 8241 Acked Flush 0 Done Done Sent false
28649 8242 Acked Write 10 Done Done Sent false
28650 8243 Acked Read 6 Done Done Sent false
28651 8244 Acked Flush 0 Done Done Sent false
28652 STATES DS:0 DS:1 DS:2 TOTAL
28653 Sent 0 0 302 302
28654 Done 302 302 0 604
28655 Skipped 0 0 0 0
28656 Error 0 0 0 0
28657 Last Flush: 8244 8244 7942
28658 Downstairs last five completed: 7942 7941 7940 7939 7938
28659 Upstairs last five completed: 8244 8243 8242 8241 8240
28660 CLIENT: Up:0 ds:302 act:2
286612024-11-28T04:56:18.623ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
286622024-11-28T04:56:18.623ZWARNcrucible: ds_connection connect to 127.0.0.1:8830 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
286632024-11-28T04:56:18.623ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
286642024-11-28T04:56:18.623ZWARNcrucible: downstairs task for 2 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
286652024-11-28T04:56:18.623ZINFOcrucible: client task is exiting = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
28666 [8830] got start action
28667 [8830] I am going to restart
28668 Starting downstairs at port 8830
28669 Make output file at "/tmp/dsc/downstairs-8830.txt"
28670 [8830][2] reports Starting
28671 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1226)
28672 [8830][2] reports Running
28673 ----------------------------------------------------------------
28674 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
28675 States: Active Active Offline
28676 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
28677 7943 Acked Write 4 Done Done Sent false
28678 7944 Acked Read 8 Done Done Sent false
28679 7945 Acked Read 8 Done Done Sent false
28680 7946 Acked Read 9 Done Done Sent false
28681 7947 Acked Read 7 Done Done Sent false
28682 7948 Acked Write 3 Done Done Sent false
28683 7949 Acked Read 2 Done Done Sent false
28684 7950 Acked Read 9 Done Done Sent false
28685 7951 Acked Read 5 Done Done Sent false
28686 7952 Acked Read 6 Done Done Sent false
28687 7953 Acked Read 8 Done Done Sent false
28688 7954 Acked Flush 0 Done Done Sent false
28689 7955 Acked Write 4 Done Done Sent false
28690 7956 Acked Read 10 Done Done Sent false
28691 7957 Acked Write 3 Done Done Sent false
28692 7958 Acked Write 5 Done Done Sent false
28693 7959 Acked Read 9 Done Done Sent false
28694 7960 Acked Flush 0 Done Done Sent false
28695 7961 Acked Write 7 Done Done Sent false
28696 7962 Acked Write 5 Done Done Sent false
28697 7963 Acked Flush 0 Done Done Sent false
28698 7964 Acked Write 4 Done Done Sent false
28699 7965 Acked Read 9 Done Done Sent false
28700 7966 Acked Read 3 Done Done Sent false
28701 7967 Acked Read 4 Done Done Sent false
28702 7968 Acked Read 9 Done Done Sent false
28703 7969 Acked Flush 0 Done Done Sent false
28704 7970 Acked Read 7 Done Done Sent false
28705 7971 Acked Read 9 Done Done Sent false
28706 7972 Acked Write 6 Done Done Sent false
28707 7973 Acked Read 6 Done Done Sent false
28708 7974 Acked Read 3 Done Done Sent false
28709 7975 Acked Read 2 Done Done Sent false
28710 7976 Acked Read 4 Done Done Sent false
28711 7977 Acked Flush 0 Done Done Sent false
28712 7978 Acked Read 5 Done Done Sent false
28713 7979 Acked Write 4 Done Done Sent false
28714 7980 Acked Read 7 Done Done Sent false
28715 7981 Acked Read 1 Done Done Sent false
28716 7982 Acked Write 10 Done Done Sent false
28717 7983 Acked Read 6 Done Done Sent false
28718 7984 Acked Read 4 Done Done Sent false
28719 7985 Acked Read 8 Done Done Sent false
28720 7986 Acked Write 4 Done Done Sent false
28721 7987 Acked Write 10 Done Done Sent false
28722 7988 Acked Write 5 Done Done Sent false
28723 7989 Acked Write 7 Done Done Sent false
28724 7990 Acked Write 8 Done Done Sent false
28725 7991 Acked Write 3 Done Done Sent false
28726 7992 Acked Read 1 Done Done Sent false
28727 7993 Acked Write 6 Done Done Sent false
28728 7994 Acked Write 4 Done Done Sent false
28729 7995 Acked Read 7 Done Done Sent false
28730 7996 Acked Write 10 Done Done Sent false
28731 7997 Acked Write 9 Done Done Sent false
28732 7998 Acked Write 7 Done Done Sent false
28733 7999 Acked Write 8 Done Done Sent false
28734 8000 Acked Read 6 Done Done Sent false
28735 8001 Acked Flush 0 Done Done Sent false
28736 8002 Acked Read 2 Done Done Sent false
28737 8003 Acked Write 9 Done Done Sent false
28738 8004 Acked Write 6 Done Done Sent false
28739 8005 Acked Read 6 Done Done Sent false
28740 8006 Acked Read 10 Done Done Sent false
28741 8007 Acked Write 9 Done Done Sent false
28742 8008 Acked Read 6 Done Done Sent false
28743 8009 Acked Write 5 Done Done Sent false
28744 8010 Acked Write 3 Done Done Sent false
28745 8011 Acked Read 1 Done Done Sent false
28746 8012 Acked Read 6 Done Done Sent false
28747 8013 Acked Flush 0 Done Done Sent false
28748 8014 Acked Read 8 Done Done Sent false
28749 8015 Acked Write 7 Done Done Sent false
28750 8016 Acked Read 3 Done Done Sent false
28751 8017 Acked Read 6 Done Done Sent false
28752 8018 Acked Flush 0 Done Done Sent false
28753 8019 Acked Read 9 Done Done Sent false
28754 8020 Acked Read 10 Done Done Sent false
28755 8021 Acked Write 7 Done Done Sent false
28756 8022 Acked Write 5 Done Done Sent false
28757 8023 Acked Write 3 Done Done Sent false
28758 8024 Acked Write 4 Done Done Sent false
28759 8025 Acked Flush 0 Done Done Sent false
28760 8026 Acked Read 5 Done Done Sent false
28761 8027 Acked Read 7 Done Done Sent false
28762 8028 Acked Read 4 Done Done Sent false
28763 8029 Acked Write 5 Done Done Sent false
28764 8030 Acked Write 9 Done Done Sent false
28765 8031 Acked Write 7 Done Done Sent false
28766 8032 Acked Write 6 Done Done Sent false
28767 8033 Acked Write 6 Done Done Sent false
28768 8034 Acked Read 1 Done Done Sent false
28769 8035 Acked Read 7 Done Done Sent false
28770 8036 Acked Write 9 Done Done Sent false
28771 8037 Acked Read 8 Done Done Sent false
28772 8038 Acked Write 5 Done Done Sent false
28773 8039 Acked Read 6 Done Done Sent false
28774 8040 Acked Read 5 Done Done Sent false
28775 8041 Acked Write 6 Done Done Sent false
28776 8042 Acked Write 6 Done Done Sent false
28777 8043 Acked Write 1 Done Done Sent false
28778 8044 Acked Read 9 Done Done Sent false
28779 8045 Acked Write 8 Done Done Sent false
28780 8046 Acked Write 4 Done Done Sent false
28781 8047 Acked Write 10 Done Done Sent false
28782 8048 Acked Write 7 Done Done Sent false
28783 8049 Acked Write 9 Done Done Sent false
28784 8050 Acked Write 6 Done Done Sent false
28785 8051 Acked Read 2 Done Done Sent false
28786 8052 Acked Read 8 Done Done Sent false
28787 8053 Acked Read 5 Done Done Sent false
28788 8054 Acked Read 5 Done Done Sent false
28789 8055 Acked Write 1 Done Done Sent false
28790 8056 Acked Write 10 Done Done Sent false
28791 8057 Acked Read 9 Done Done Sent false
28792 8058 Acked Read 6 Done Done Sent false
28793 8059 Acked Write 8 Done Done Sent false
28794 8060 Acked Write 8 Done Done Sent false
28795 8061 Acked Write 8 Done Done Sent false
28796 8062 Acked Read 6 Done Done Sent false
28797 8063 Acked Write 8 Done Done Sent false
28798 8064 Acked Write 4 Done Done Sent false
28799 8065 Acked Write 6 Done Done Sent false
28800 8066 Acked Read 2 Done Done Sent false
28801 8067 Acked Read 3 Done Done Sent false
28802 8068 Acked Read 8 Done Done Sent false
28803 8069 Acked Read 7 Done Done Sent false
28804 8070 Acked Write 4 Done Done Sent false
28805 8071 Acked Write 1 Done Done Sent false
28806 8072 Acked Read 8 Done Done Sent false
28807 8073 Acked Flush 0 Done Done Sent false
28808 8074 Acked Write 7 Done Done Sent false
28809 8075 Acked Read 9 Done Done Sent false
28810 8076 Acked Read 9 Done Done Sent false
28811 8077 Acked Read 6 Done Done Sent false
28812 8078 Acked Read 9 Done Done Sent false
28813 8079 Acked Read 6 Done Done Sent false
28814 8080 Acked Write 9 Done Done Sent false
28815 8081 Acked Write 1 Done Done Sent false
28816 8082 Acked Write 3 Done Done Sent false
28817 8083 Acked Read 9 Done Done Sent false
28818 8084 Acked Read 3 Done Done Sent false
28819 8085 Acked Read 5 Done Done Sent false
28820 8086 Acked Flush 0 Done Done Sent false
28821 8087 Acked Flush 0 Done Done Sent false
28822 8088 Acked Read 4 Done Done Sent false
28823 8089 Acked Write 1 Done Done Sent false
28824 8090 Acked Read 2 Done Done Sent false
28825 8091 Acked Write 6 Done Done Sent false
28826 8092 Acked Read 9 Done Done Sent false
28827 8093 Acked Write 5 Done Done Sent false
28828 8094 Acked Read 10 Done Done Sent false
28829 8095 Acked Write 6 Done Done Sent false
28830 8096 Acked Write 5 Done Done Sent false
28831 8097 Acked Flush 0 Done Done Sent false
28832 8098 Acked Write 4 Done Done Sent false
28833 8099 Acked Read 4 Done Done Sent false
28834 8100 Acked Write 7 Done Done Sent false
28835 8101 Acked Read 8 Done Done Sent false
28836 8102 Acked Write 8 Done Done Sent false
28837 8103 Acked Write 1 Done Done Sent false
28838 8104 Acked Write 7 Done Done Sent false
28839 8105 Acked Write 4 Done Done Sent false
28840 8106 Acked Write 7 Done Done Sent false
28841 8107 Acked Write 8 Done Done Sent false
28842 8108 Acked Read 9 Done Done Sent false
28843 8109 Acked Read 5 Done Done Sent false
28844 8110 Acked Flush 0 Done Done Sent false
28845 8111 Acked Read 1 Done Done Sent false
28846 8112 Acked Write 8 Done Done Sent false
28847 8113 Acked Read 3 Done Done Sent false
28848 8114 Acked Read 4 Done Done Sent false
28849 8115 Acked Flush 0 Done Done Sent false
28850 8116 Acked Read 9 Done Done Sent false
28851 8117 Acked Write 4 Done Done Sent false
28852 8118 Acked Flush 0 Done Done Sent false
28853 8119 Acked Read 8 Done Done Sent false
28854 8120 Acked Write 5 Done Done Sent false
28855 8121 Acked Read 5 Done Done Sent false
28856 8122 Acked Write 5 Done Done Sent false
28857 8123 Acked Read 6 Done Done Sent false
28858 8124 Acked Write 10 Done Done Sent false
28859 8125 Acked Read 10 Done Done Sent false
28860 8126 Acked Read 6 Done Done Sent false
28861 8127 Acked Read 5 Done Done Sent false
28862 8128 Acked Write 5 Done Done Sent false
28863 8129 Acked Write 6 Done Done Sent false
28864 8130 Acked Write 3 Done Done Sent false
28865 8131 Acked Write 4 Done Done Sent false
28866 8132 Acked Read 10 Done Done Sent false
28867 8133 Acked Read 10 Done Done Sent false
28868 8134 Acked Write 5 Done Done Sent false
28869 8135 Acked Write 9 Done Done Sent false
28870 8136 Acked Read 10 Done Done Sent false
28871 8137 Acked Read 3 Done Done Sent false
28872 8138 Acked Write 5 Done Done Sent false
28873 8139 Acked Read 10 Done Done Sent false
28874 8140 Acked Read 10 Done Done Sent false
28875 8141 Acked Read 3 Done Done Sent false
28876 8142 Acked Read 6 Done Done Sent false
28877 8143 Acked Write 2 Done Done Sent false
28878 8144 Acked Read 2 Done Done Sent false
28879 8145 Acked Write 8 Done Done Sent false
28880 8146 Acked Write 8 Done Done Sent false
28881 8147 Acked Read 10 Done Done Sent false
28882 8148 Acked Write 9 Done Done Sent false
28883 8149 Acked Write 3 Done Done Sent false
28884 8150 Acked Write 3 Done Done Sent false
28885 8151 Acked Write 5 Done Done Sent false
28886 8152 Acked Write 3 Done Done Sent false
28887 8153 Acked Read 9 Done Done Sent false
28888 8154 Acked Read 3 Done Done Sent false
28889 8155 Acked Read 3 Done Done Sent false
28890 8156 Acked Flush 0 Done Done Sent false
28891 8157 Acked Read 3 Done Done Sent false
28892 8158 Acked Flush 0 Done Done Sent false
28893 8159 Acked Read 7 Done Done Sent false
28894 8160 Acked Read 2 Done Done Sent false
28895 8161 Acked Write 9 Done Done Sent false
28896 8162 Acked Read 3 Done Done Sent false
28897 8163 Acked Write 8 Done Done Sent false
28898 8164 Acked Write 1 Done Done Sent false
28899 8165 Acked Read 6 Done Done Sent false
28900 8166 Acked Write 2 Done Done Sent false
28901 8167 Acked Write 4 Done Done Sent false
28902 8168 Acked Read 4 Done Done Sent false
28903 8169 Acked Read 1 Done Done Sent false
28904 8170 Acked Write 2 Done Done Sent false
28905 8171 Acked Read 3 Done Done Sent false
28906 8172 Acked Write 3 Done Done Sent false
28907 8173 Acked Read 1 Done Done Sent false
28908 8174 Acked Read 8 Done Done Sent false
28909 8175 Acked Read 8 Done Done Sent false
28910 8176 Acked Write 7 Done Done Sent false
28911 8177 Acked Read 7 Done Done Sent false
28912 8178 Acked Flush 0 Done Done Sent false
28913 8179 Acked Read 5 Done Done Sent false
28914 8180 Acked Read 3 Done Done Sent false
28915 8181 Acked Write 10 Done Done Sent false
28916 8182 Acked Read 1 Done Done Sent false
28917 8183 Acked Read 7 Done Done Sent false
28918 8184 Acked Read 1 Done Done Sent false
28919 8185 Acked Write 9 Done Done Sent false
28920 8186 Acked Read 5 Done Done Sent false
28921 8187 Acked Read 5 Done Done Sent false
28922 8188 Acked Read 10 Done Done Sent false
28923 8189 Acked Write 8 Done Done Sent false
28924 8190 Acked Read 9 Done Done Sent false
28925 8191 Acked Read 2 Done Done Sent false
28926 8192 Acked Flush 0 Done Done Sent false
28927 8193 Acked Read 9 Done Done Sent false
28928 8194 Acked Write 7 Done Done Sent false
28929 8195 Acked Read 2 Done Done Sent false
28930 8196 Acked Write 9 Done Done Sent false
28931 8197 Acked Read 9 Done Done Sent false
28932 8198 Acked Read 3 Done Done Sent false
28933 8199 Acked Write 6 Done Done Sent false
28934 8200 Acked Write 8 Done Done Sent false
28935 8201 Acked Read 1 Done Done Sent false
28936 8202 Acked Read 4 Done Done Sent false
28937 8203 Acked Read 1 Done Done Sent false
28938 8204 Acked Read 6 Done Done Sent false
28939 8205 Acked Flush 0 Done Done Sent false
28940 8206 Acked Read 8 Done Done Sent false
28941 8207 Acked Write 10 Done Done Sent false
28942 8208 Acked Write 10 Done Done Sent false
28943 8209 Acked Write 6 Done Done Sent false
28944 8210 Acked Write 2 Done Done Sent false
28945 8211 Acked Flush 0 Done Done Sent false
28946 8212 Acked Write 4 Done Done Sent false
28947 8213 Acked Read 9 Done Done Sent false
28948 8214 Acked Read 9 Done Done Sent false
28949 8215 Acked Write 10 Done Done Sent false
28950 8216 Acked Flush 0 Done Done Sent false
28951 8217 Acked Read 7 Done Done Sent false
28952 8218 Acked Read 10 Done Done Sent false
28953 8219 Acked Read 4 Done Done Sent false
28954 8220 Acked Read 7 Done Done Sent false
28955 8221 Acked Write 7 Done Done Sent false
28956 8222 Acked Write 1 Done Done Sent false
28957 8223 Acked Write 3 Done Done Sent false
28958 8224 Acked Write 1 Done Done Sent false
28959 8225 Acked Flush 0 Done Done Sent false
28960 8226 Acked Write 9 Done Done Sent false
28961 8227 Acked Read 1 Done Done Sent false
28962 8228 Acked Read 5 Done Done Sent false
28963 8229 Acked Flush 0 Done Done Sent false
28964 8230 Acked Write 4 Done Done Sent false
28965 8231 Acked Write 2 Done Done Sent false
28966 8232 Acked Write 1 Done Done Sent false
28967 8233 Acked Read 2 Done Done Sent false
28968 8234 Acked Flush 0 Done Done Sent false
28969 8235 Acked Read 10 Done Done Sent false
28970 8236 Acked Write 10 Done Done Sent false
28971 8237 Acked Write 2 Done Done Sent false
28972 8238 Acked Write 3 Done Done Sent false
28973 8239 Acked Write 7 Done Done Sent false
28974 8240 Acked Write 1 Done Done Sent false
28975 8241 Acked Flush 0 Done Done Sent false
28976 8242 Acked Write 10 Done Done Sent false
28977 8243 Acked Read 6 Done Done Sent false
28978 8244 Acked Flush 0 Done Done Sent false
28979 STATES DS:0 DS:1 DS:2 TOTAL
28980 Sent 0 0 302 302
28981 Done 302 302 0 604
28982 Skipped 0 0 0 0
28983 Error 0 0 0 0
28984 Last Flush: 8244 8244 7942
28985 Downstairs last five completed: 7942 7941 7940 7939 7938
28986 Upstairs last five completed: 8244 8243 8242 8241 8240
28987 CLIENT: Up:0 ds:302 act:2
28988 ----------------------------------------------------------------
28989 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
28990 States: Active Active Offline
28991 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
28992 7943 Acked Write 4 Done Done Sent false
28993 7944 Acked Read 8 Done Done Sent false
28994 7945 Acked Read 8 Done Done Sent false
28995 7946 Acked Read 9 Done Done Sent false
28996 7947 Acked Read 7 Done Done Sent false
28997 7948 Acked Write 3 Done Done Sent false
28998 7949 Acked Read 2 Done Done Sent false
28999 7950 Acked Read 9 Done Done Sent false
29000 7951 Acked Read 5 Done Done Sent false
29001 7952 Acked Read 6 Done Done Sent false
29002 7953 Acked Read 8 Done Done Sent false
29003 7954 Acked Flush 0 Done Done Sent false
29004 7955 Acked Write 4 Done Done Sent false
29005 7956 Acked Read 10 Done Done Sent false
29006 7957 Acked Write 3 Done Done Sent false
29007 7958 Acked Write 5 Done Done Sent false
29008 7959 Acked Read 9 Done Done Sent false
29009 7960 Acked Flush 0 Done Done Sent false
29010 7961 Acked Write 7 Done Done Sent false
29011 7962 Acked Write 5 Done Done Sent false
29012 7963 Acked Flush 0 Done Done Sent false
29013 7964 Acked Write 4 Done Done Sent false
29014 7965 Acked Read 9 Done Done Sent false
29015 7966 Acked Read 3 Done Done Sent false
29016 7967 Acked Read 4 Done Done Sent false
29017 7968 Acked Read 9 Done Done Sent false
29018 7969 Acked Flush 0 Done Done Sent false
29019 7970 Acked Read 7 Done Done Sent false
29020 7971 Acked Read 9 Done Done Sent false
29021 7972 Acked Write 6 Done Done Sent false
29022 7973 Acked Read 6 Done Done Sent false
29023 7974 Acked Read 3 Done Done Sent false
29024 7975 Acked Read 2 Done Done Sent false
29025 7976 Acked Read 4 Done Done Sent false
29026 7977 Acked Flush 0 Done Done Sent false
29027 7978 Acked Read 5 Done Done Sent false
29028 7979 Acked Write 4 Done Done Sent false
29029 7980 Acked Read 7 Done Done Sent false
29030 7981 Acked Read 1 Done Done Sent false
29031 7982 Acked Write 10 Done Done Sent false
29032 7983 Acked Read 6 Done Done Sent false
29033 7984 Acked Read 4 Done Done Sent false
29034 7985 Acked Read 8 Done Done Sent false
29035 7986 Acked Write 4 Done Done Sent false
29036 7987 Acked Write 10 Done Done Sent false
29037 7988 Acked Write 5 Done Done Sent false
29038 7989 Acked Write 7 Done Done Sent false
29039 7990 Acked Write 8 Done Done Sent false
29040 7991 Acked Write 3 Done Done Sent false
29041 7992 Acked Read 1 Done Done Sent false
29042 7993 Acked Write 6 Done Done Sent false
29043 7994 Acked Write 4 Done Done Sent false
29044 7995 Acked Read 7 Done Done Sent false
29045 7996 Acked Write 10 Done Done Sent false
29046 7997 Acked Write 9 Done Done Sent false
29047 7998 Acked Write 7 Done Done Sent false
29048 7999 Acked Write 8 Done Done Sent false
29049 8000 Acked Read 6 Done Done Sent false
29050 8001 Acked Flush 0 Done Done Sent false
29051 8002 Acked Read 2 Done Done Sent false
29052 8003 Acked Write 9 Done Done Sent false
29053 8004 Acked Write 6 Done Done Sent false
29054 8005 Acked Read 6 Done Done Sent false
29055 8006 Acked Read 10 Done Done Sent false
29056 8007 Acked Write 9 Done Done Sent false
29057 8008 Acked Read 6 Done Done Sent false
29058 8009 Acked Write 5 Done Done Sent false
29059 8010 Acked Write 3 Done Done Sent false
29060 8011 Acked Read 1 Done Done Sent false
29061 8012 Acked Read 6 Done Done Sent false
29062 8013 Acked Flush 0 Done Done Sent false
29063 8014 Acked Read 8 Done Done Sent false
29064 8015 Acked Write 7 Done Done Sent false
29065 8016 Acked Read 3 Done Done Sent false
29066 8017 Acked Read 6 Done Done Sent false
29067 8018 Acked Flush 0 Done Done Sent false
29068 8019 Acked Read 9 Done Done Sent false
29069 8020 Acked Read 10 Done Done Sent false
29070 8021 Acked Write 7 Done Done Sent false
29071 8022 Acked Write 5 Done Done Sent false
29072 8023 Acked Write 3 Done Done Sent false
29073 8024 Acked Write 4 Done Done Sent false
29074 8025 Acked Flush 0 Done Done Sent false
29075 8026 Acked Read 5 Done Done Sent false
29076 8027 Acked Read 7 Done Done Sent false
29077 8028 Acked Read 4 Done Done Sent false
29078 8029 Acked Write 5 Done Done Sent false
29079 8030 Acked Write 9 Done Done Sent false
29080 8031 Acked Write 7 Done Done Sent false
29081 8032 Acked Write 6 Done Done Sent false
29082 8033 Acked Write 6 Done Done Sent false
29083 8034 Acked Read 1 Done Done Sent false
29084 8035 Acked Read 7 Done Done Sent false
29085 8036 Acked Write 9 Done Done Sent false
29086 8037 Acked Read 8 Done Done Sent false
29087 8038 Acked Write 5 Done Done Sent false
29088 8039 Acked Read 6 Done Done Sent false
29089 8040 Acked Read 5 Done Done Sent false
29090 8041 Acked Write 6 Done Done Sent false
29091 8042 Acked Write 6 Done Done Sent false
29092 8043 Acked Write 1 Done Done Sent false
29093 8044 Acked Read 9 Done Done Sent false
29094 8045 Acked Write 8 Done Done Sent false
29095 8046 Acked Write 4 Done Done Sent false
29096 8047 Acked Write 10 Done Done Sent false
29097 8048 Acked Write 7 Done Done Sent false
29098 8049 Acked Write 9 Done Done Sent false
29099 8050 Acked Write 6 Done Done Sent false
29100 8051 Acked Read 2 Done Done Sent false
29101 8052 Acked Read 8 Done Done Sent false
29102 8053 Acked Read 5 Done Done Sent false
29103 8054 Acked Read 5 Done Done Sent false
29104 8055 Acked Write 1 Done Done Sent false
29105 8056 Acked Write 10 Done Done Sent false
29106 8057 Acked Read 9 Done Done Sent false
29107 8058 Acked Read 6 Done Done Sent false
29108 8059 Acked Write 8 Done Done Sent false
29109 8060 Acked Write 8 Done Done Sent false
29110 8061 Acked Write 8 Done Done Sent false
29111 8062 Acked Read 6 Done Done Sent false
29112 8063 Acked Write 8 Done Done Sent false
29113 8064 Acked Write 4 Done Done Sent false
29114 8065 Acked Write 6 Done Done Sent false
29115 8066 Acked Read 2 Done Done Sent false
29116 8067 Acked Read 3 Done Done Sent false
29117 8068 Acked Read 8 Done Done Sent false
29118 8069 Acked Read 7 Done Done Sent false
29119 8070 Acked Write 4 Done Done Sent false
29120 8071 Acked Write 1 Done Done Sent false
29121 8072 Acked Read 8 Done Done Sent false
29122 8073 Acked Flush 0 Done Done Sent false
29123 8074 Acked Write 7 Done Done Sent false
29124 8075 Acked Read 9 Done Done Sent false
29125 8076 Acked Read 9 Done Done Sent false
29126 8077 Acked Read 6 Done Done Sent false
29127 8078 Acked Read 9 Done Done Sent false
29128 8079 Acked Read 6 Done Done Sent false
29129 8080 Acked Write 9 Done Done Sent false
29130 8081 Acked Write 1 Done Done Sent false
29131 8082 Acked Write 3 Done Done Sent false
29132 8083 Acked Read 9 Done Done Sent false
29133 8084 Acked Read 3 Done Done Sent false
29134 8085 Acked Read 5 Done Done Sent false
29135 8086 Acked Flush 0 Done Done Sent false
29136 8087 Acked Flush 0 Done Done Sent false
29137 8088 Acked Read 4 Done Done Sent false
29138 8089 Acked Write 1 Done Done Sent false
29139 8090 Acked Read 2 Done Done Sent false
29140 8091 Acked Write 6 Done Done Sent false
29141 8092 Acked Read 9 Done Done Sent false
29142 8093 Acked Write 5 Done Done Sent false
29143 8094 Acked Read 10 Done Done Sent false
29144 8095 Acked Write 6 Done Done Sent false
29145 8096 Acked Write 5 Done Done Sent false
29146 8097 Acked Flush 0 Done Done Sent false
29147 8098 Acked Write 4 Done Done Sent false
29148 8099 Acked Read 4 Done Done Sent false
29149 8100 Acked Write 7 Done Done Sent false
29150 8101 Acked Read 8 Done Done Sent false
29151 8102 Acked Write 8 Done Done Sent false
29152 8103 Acked Write 1 Done Done Sent false
29153 8104 Acked Write 7 Done Done Sent false
29154 8105 Acked Write 4 Done Done Sent false
29155 8106 Acked Write 7 Done Done Sent false
29156 8107 Acked Write 8 Done Done Sent false
29157 8108 Acked Read 9 Done Done Sent false
29158 8109 Acked Read 5 Done Done Sent false
29159 8110 Acked Flush 0 Done Done Sent false
29160 8111 Acked Read 1 Done Done Sent false
29161 8112 Acked Write 8 Done Done Sent false
29162 8113 Acked Read 3 Done Done Sent false
29163 8114 Acked Read 4 Done Done Sent false
29164 8115 Acked Flush 0 Done Done Sent false
29165 8116 Acked Read 9 Done Done Sent false
29166 8117 Acked Write 4 Done Done Sent false
29167 8118 Acked Flush 0 Done Done Sent false
29168 8119 Acked Read 8 Done Done Sent false
29169 8120 Acked Write 5 Done Done Sent false
29170 8121 Acked Read 5 Done Done Sent false
29171 8122 Acked Write 5 Done Done Sent false
29172 8123 Acked Read 6 Done Done Sent false
29173 8124 Acked Write 10 Done Done Sent false
29174 8125 Acked Read 10 Done Done Sent false
29175 8126 Acked Read 6 Done Done Sent false
29176 8127 Acked Read 5 Done Done Sent false
29177 8128 Acked Write 5 Done Done Sent false
29178 8129 Acked Write 6 Done Done Sent false
29179 8130 Acked Write 3 Done Done Sent false
29180 8131 Acked Write 4 Done Done Sent false
29181 8132 Acked Read 10 Done Done Sent false
29182 8133 Acked Read 10 Done Done Sent false
29183 8134 Acked Write 5 Done Done Sent false
29184 8135 Acked Write 9 Done Done Sent false
29185 8136 Acked Read 10 Done Done Sent false
29186 8137 Acked Read 3 Done Done Sent false
29187 8138 Acked Write 5 Done Done Sent false
29188 8139 Acked Read 10 Done Done Sent false
29189 8140 Acked Read 10 Done Done Sent false
29190 8141 Acked Read 3 Done Done Sent false
29191 8142 Acked Read 6 Done Done Sent false
29192 8143 Acked Write 2 Done Done Sent false
29193 8144 Acked Read 2 Done Done Sent false
29194 8145 Acked Write 8 Done Done Sent false
29195 8146 Acked Write 8 Done Done Sent false
29196 8147 Acked Read 10 Done Done Sent false
29197 8148 Acked Write 9 Done Done Sent false
29198 8149 Acked Write 3 Done Done Sent false
29199 8150 Acked Write 3 Done Done Sent false
29200 8151 Acked Write 5 Done Done Sent false
29201 8152 Acked Write 3 Done Done Sent false
29202 8153 Acked Read 9 Done Done Sent false
29203 8154 Acked Read 3 Done Done Sent false
29204 8155 Acked Read 3 Done Done Sent false
29205 8156 Acked Flush 0 Done Done Sent false
29206 8157 Acked Read 3 Done Done Sent false
29207 8158 Acked Flush 0 Done Done Sent false
29208 8159 Acked Read 7 Done Done Sent false
29209 8160 Acked Read 2 Done Done Sent false
29210 8161 Acked Write 9 Done Done Sent false
29211 8162 Acked Read 3 Done Done Sent false
29212 8163 Acked Write 8 Done Done Sent false
29213 8164 Acked Write 1 Done Done Sent false
29214 8165 Acked Read 6 Done Done Sent false
29215 8166 Acked Write 2 Done Done Sent false
29216 8167 Acked Write 4 Done Done Sent false
29217 8168 Acked Read 4 Done Done Sent false
29218 8169 Acked Read 1 Done Done Sent false
29219 8170 Acked Write 2 Done Done Sent false
29220 8171 Acked Read 3 Done Done Sent false
29221 8172 Acked Write 3 Done Done Sent false
29222 8173 Acked Read 1 Done Done Sent false
29223 8174 Acked Read 8 Done Done Sent false
29224 8175 Acked Read 8 Done Done Sent false
29225 8176 Acked Write 7 Done Done Sent false
29226 8177 Acked Read 7 Done Done Sent false
29227 8178 Acked Flush 0 Done Done Sent false
29228 8179 Acked Read 5 Done Done Sent false
29229 8180 Acked Read 3 Done Done Sent false
29230 8181 Acked Write 10 Done Done Sent false
29231 8182 Acked Read 1 Done Done Sent false
29232 8183 Acked Read 7 Done Done Sent false
29233 8184 Acked Read 1 Done Done Sent false
29234 8185 Acked Write 9 Done Done Sent false
29235 8186 Acked Read 5 Done Done Sent false
29236 8187 Acked Read 5 Done Done Sent false
29237 8188 Acked Read 10 Done Done Sent false
29238 8189 Acked Write 8 Done Done Sent false
29239 8190 Acked Read 9 Done Done Sent false
29240 8191 Acked Read 2 Done Done Sent false
29241 8192 Acked Flush 0 Done Done Sent false
29242 8193 Acked Read 9 Done Done Sent false
29243 8194 Acked Write 7 Done Done Sent false
29244 8195 Acked Read 2 Done Done Sent false
29245 8196 Acked Write 9 Done Done Sent false
29246 8197 Acked Read 9 Done Done Sent false
29247 8198 Acked Read 3 Done Done Sent false
29248 8199 Acked Write 6 Done Done Sent false
29249 8200 Acked Write 8 Done Done Sent false
29250 8201 Acked Read 1 Done Done Sent false
29251 8202 Acked Read 4 Done Done Sent false
29252 8203 Acked Read 1 Done Done Sent false
29253 8204 Acked Read 6 Done Done Sent false
29254 8205 Acked Flush 0 Done Done Sent false
29255 8206 Acked Read 8 Done Done Sent false
29256 8207 Acked Write 10 Done Done Sent false
29257 8208 Acked Write 10 Done Done Sent false
29258 8209 Acked Write 6 Done Done Sent false
29259 8210 Acked Write 2 Done Done Sent false
29260 8211 Acked Flush 0 Done Done Sent false
29261 8212 Acked Write 4 Done Done Sent false
29262 8213 Acked Read 9 Done Done Sent false
29263 8214 Acked Read 9 Done Done Sent false
29264 8215 Acked Write 10 Done Done Sent false
29265 8216 Acked Flush 0 Done Done Sent false
29266 8217 Acked Read 7 Done Done Sent false
29267 8218 Acked Read 10 Done Done Sent false
29268 8219 Acked Read 4 Done Done Sent false
29269 8220 Acked Read 7 Done Done Sent false
29270 8221 Acked Write 7 Done Done Sent false
29271 8222 Acked Write 1 Done Done Sent false
29272 8223 Acked Write 3 Done Done Sent false
29273 8224 Acked Write 1 Done Done Sent false
29274 8225 Acked Flush 0 Done Done Sent false
29275 8226 Acked Write 9 Done Done Sent false
29276 8227 Acked Read 1 Done Done Sent false
29277 8228 Acked Read 5 Done Done Sent false
29278 8229 Acked Flush 0 Done Done Sent false
29279 8230 Acked Write 4 Done Done Sent false
29280 8231 Acked Write 2 Done Done Sent false
29281 8232 Acked Write 1 Done Done Sent false
29282 8233 Acked Read 2 Done Done Sent false
29283 8234 Acked Flush 0 Done Done Sent false
29284 8235 Acked Read 10 Done Done Sent false
29285 8236 Acked Write 10 Done Done Sent false
29286 8237 Acked Write 2 Done Done Sent false
29287 8238 Acked Write 3 Done Done Sent false
29288 8239 Acked Write 7 Done Done Sent false
29289 8240 Acked Write 1 Done Done Sent false
29290 8241 Acked Flush 0 Done Done Sent false
29291 8242 Acked Write 10 Done Done Sent false
29292 8243 Acked Read 6 Done Done Sent false
29293 8244 Acked Flush 0 Done Done Sent false
29294 STATES DS:0 DS:1 DS:2 TOTAL
29295 Sent 0 0 302 302
29296 Done 302 302 0 604
29297 Skipped 0 0 0 0
29298 Error 0 0 0 0
29299 Last Flush: 8244 8244 7942
29300 Downstairs last five completed: 7942 7941 7940 7939 7938
29301 Upstairs last five completed: 8244 8243 8242 8241 8240
29302 CLIENT: Up:0 ds:302 act:2
293032024-11-28T04:56:28.624ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
293042024-11-28T04:56:28.624ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:34132) = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
293052024-11-28T04:56:28.625ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
293062024-11-28T04:56:28.626ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
293072024-11-28T04:56:28.626ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
293082024-11-28T04:56:28.626ZINFOcrucible: send last flush ID to this DS: 7942 = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
293092024-11-28T04:56:28.626ZINFOcrucible: Replied this last flush ID: 7942 = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
293102024-11-28T04:56:28.626ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
293112024-11-28T04:56:28.626ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
293122024-11-28T04:56:28.626ZINFOcrucible: [2] client re-new 302 jobs since flush 7942 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
293132024-11-28T04:56:28.626ZINFOcrucible: [2] Replayed 302 jobs since flush: 7942 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
29314 ----------------------------------------------------------------
29315 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
29316 Downstairs last five completed: 8244 8243 8242 8241 8240
29317 Upstairs last five completed: 8244 8243 8242 8241 8240
29318 CLIENT: Up:0 ds:0 act:3
29319 Replay: All jobs finished
29320 Main task has work to do, go find it
29321 got dsc Stop(1)
29322 stop 1
29323 [8820] Got stop action so:false kr:false
29324 Nov 28 04:56:28.869 INFO request completed, latency_us: 308, response_code: 204, uri: /stop/cid/1, method: POST, req_id: e8b48ee8-cfb0-4107-a6b5-bcadbc0248e5, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
293252024-11-28T04:56:28.869ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
29326 Nov 28 04:56:28.869 INFO request completed, latency_us: 123{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2024-11-28T04:56:28.869760999Z","hostname":"w-01JDRHKEEFJ1228TFPSBEGXWHY","pid":1202,"":"io task","client":"1","":"downstairs","session_id":"4a2415b1-6594-436f-82e5-b8cf9a74eeaf"}
29327 , response_code: 200, {"msg":"downstairs task for 1 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2024-11-28T04:56:28.869802163Z","hostname":"w-01JDRHKEEFJ1228TFPSBEGXWHY","pid":1202,"session_id":"4a2415b1-6594-436f-82e5-b8cf9a74eeaf"}
29328 uri: /state/cid/1{"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-28T04:56:28.869828209Z","hostname":"w-01JDRHKEEFJ1228TFPSBEGXWHY","pid":1202,"client":"1","":"downstairs","session_id":"4a2415b1-6594-436f-82e5-b8cf9a74eeaf"}
29329 , method: GET, req_id: 88231a45-ead0-4351-bb25-65a38d433256, remote_addr: 127.0.0.1{"msg":"[1] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-28T04:56:28.869879942Z","hostname":"w-01JDRHKEEFJ1228TFPSBEGXWHY","pid":1202,"client":"1","":"downstairs","session_id":"4a2415b1-6594-436f-82e5-b8cf9a74eeaf"}
29330 :60418, local_addr: 127.0.0.1:9998
293312024-11-28T04:56:28.869ZINFOcrucible: client task is exiting = io task client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
29332 [8820] Exited with: ExitStatus(unix_wait_status(9))
29333 [8820][1] reports Exit
29334 Nov 28 04:56:32.871 INFO request completed, latency_us: 218, response_code: 200, uri: /state/cid/1, method: GET, req_id: ff9054cc-829a-43cf-8639-e0ac8a19690a, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
29335 001/300 Read block 3299 len 40960
29336 002/300 Read block 2971 len 24576
29337 003/300 Read block 2991 len 4096
29338 004/300 Write block 2819 len 4096 data: 3
29339 005/300 Read block 4390 len 16384
29340 006/300 Read block 4716 len 16384
29341 007/300 Write block 3480 len 4096 data: 7
29342 008/300 Write block 1548 len 28672 data: 7 9 7 6 6 4 5
29343 009/300 Read block 1956 len 20480
29344 010/300 Read block 2216 len 8192
29345 011/300 Read block 1755 len 40960
29346 012/300 Read block 3294 len 32768
29347 013/300 Write block 609 len 36864 data: 6 7 5 6 5 5 6 7 7
29348 014/300 Read block 1798 len 24576
29349 015/300 Read block 3709 len 32768
29350 016/300 Read block 1248 len 20480
29351 017/300 Read block 3039 len 4096
29352 018/300 Read block 2068 len 36864
29353 019/300 Read block 1865 len 28672
29354 020/300 Write block 3732 len 4096 data: 5
29355 021/300 Write block 3764 len 12288 data: 5 6 6
29356 022/300 Read block 438 len 40960
29357 023/300 Read block 4261 len 12288
29358 024/300 Read block 2858 len 16384
29359 025/300 Read block 4960 len 12288
29360 026/300 Flush
29361 027/300 Flush
29362 028/300 Flush
29363 029/300 Read block 361 len 12288
29364 030/300 Write block 3100 len 8192 data: 2 2
29365 031/300 Read block 4669 len 20480
29366 032/300 Flush
29367 033/300 Flush
29368 034/300 Read block 2804 len 20480
29369 035/300 Write block 1229 len 40960 data: 3 3 4 3 5 5 5 6 5 5
29370 036/300 Flush
29371 037/300 Flush
29372 038/300 Read block 2332 len 32768
29373 039/300 Read block 4377 len 28672
29374 040/300 Flush
29375 041/300 Write block 4831 len 16384 data: 6 5 5 5
29376 042/300 Read block 4815 len 36864
29377 043/300 Read block 417 len 24576
29378 044/300 Read block 1098 len 28672
29379 045/300 Read block 3731 len 8192
29380 046/300 Write block 2567 len 36864 data: 7 7 7 7 8 6 5 5 6
29381 047/300 Read block 1439 len 32768
29382 048/300 Write block 255 len 28672 data: 3 6 7 8 7 6 7
29383 049/300 Read block 1649 len 20480
29384 050/300 Write block 2641 len 28672 data: 3 2 3 2 2 3 5
29385 051/300 Read block 1552 len 32768
29386 052/300 Flush
29387 053/300 Write block 2207 len 28672 data: 7 7 7 8 7 7 7
29388 054/300 Read block 2381 len 28672
29389 055/300 Write block 2686 len 24576 data: 6 6 7 7 8 6
29390 056/300 Write block 4674 len 24576 data: 8 7 7 6 6 6
29391 057/300 Write block 2193 len 8192 data: 6 7
29392 058/300 Flush
29393 059/300 Read block 830 len 4096
29394 060/300 Read block 4610 len 8192
29395 061/300 Flush
29396 062/300 Write block 508 len 32768 data: 9 7 8 7 7 7 6 6
29397 063/300 Flush
29398 064/300 Read block 4011 len 28672
29399 065/300 Flush
29400 066/300 Read block 4753 len 8192
29401 067/300 Write block 4451 len 28672 data: 9 8 7 10 10 9 8
29402 068/300 Write block 3198 len 36864 data: 4 3 4 4 3 4 4 6 5
29403 069/300 Flush
29404 070/300 Read block 4730 len 32768
29405 071/300 Write block 1841 len 12288 data: 8 7 7
29406 072/300 Read block 908 len 32768
29407 073/300 Flush
29408 074/300 Read block 3250 len 20480
29409 075/300 Read block 4350 len 28672
29410 076/300 Write block 4458 len 28672 data: 8 7 5 5 5 5 5
29411 077/300 Write block 4278 len 24576 data: 2 2 2 3 4 4
29412 078/300 Write block 1549 len 12288 data: 10 8 7
29413 079/300 Flush
29414 080/300 Flush
29415 081/300 Flush
29416 082/300 Write block 538 len 32768 data: 5 5 5 4 5 6 7 6
29417 083/300 Read block 377 len 36864
29418 084/300 Flush
29419 085/300 Read block 4458 len 36864
29420 086/300 Read block 1171 len 40960
29421 087/300 Read block 4236 len 40960
29422 088/300 Write block 197 len 8192 data: 5 4
29423 089/300 Read block 2253 len 4096
29424 090/300 Read block 980 len 4096
29425 091/300 Flush
29426 092/300 Read block 1089 len 32768
29427 093/300 Flush
29428 094/300 Read block 3010 len 24576
29429 095/300 Read block 3485 len 24576
29430 096/300 Write block 2204 len 4096 data: 4
29431 097/300 Read block 2608 len 4096
29432 098/300 Write block 3060 len 36864 data: 6 6 5 4 5 5 5 4 4
29433 099/300 Write block 3602 len 24576 data: 5 5 5 4 4 5
29434 100/300 Read block 519 len 20480
29435 101/300 Read block 397 len 20480
29436 102/300 Write block 4666 len 32768 data: 3 2 2 3 3 4 5 7
29437 103/300 Write block 3276 len 28672 data: 3 3 4 4 4 4 4
29438 104/300 Read block 4311 len 8192
29439 105/300 Read block 4425 len 20480
29440 106/300 Write block 2808 len 16384 data: 4 5 5 5
29441 107/300 Write block 2755 len 28672 data: 7 7 7 6 6 4 4
29442 108/300 Write block 1617 len 8192 data: 6 5
29443 109/300 Write block 771 len 16384 data: 6 6 6 6
29444 110/300 Flush
29445 111/300 Read block 7 len 28672
29446 112/300 Read block 2964 len 20480
29447 113/300 Read block 3949 len 12288
29448 114/300 Write block 4280 len 40960 data: 3 4 5 5 2 2 2 3 4 4
29449 115/300 Flush
29450 116/300 Write block 1373 len 8192 data: 4 5
29451 117/300 Read block 1726 len 40960
29452 118/300 Write block 4126 len 12288 data: 5 6 6
29453 119/300 Read block 4783 len 20480
29454 120/300 Read block 1863 len 28672
29455 121/300 Write block 2286 len 24576 data: 5 5 5 5 6 6
29456 122/300 Write block 2612 len 4096 data: 3
29457 123/300 Write block 4914 len 20480 data: 5 5 4 4 5
29458 124/300 Read block 454 len 8192
29459 125/300 Read block 3222 len 24576
29460 126/300 Write block 4864 len 8192 data: 4 3
29461 127/300 Read block 1052 len 36864
29462 128/300 Read block 3063 len 16384
29463 129/300 Flush
29464 130/300 Write block 4011 len 4096 data: 2
29465 131/300 Read block 1283 len 12288
29466 132/300 Flush
29467 133/300 Write block 2829 len 16384 data: 5 5 5 5
29468 134/300 Read block 341 len 32768
29469 135/300 Read block 2625 len 24576
29470 136/300 Write block 2851 len 12288 data: 6 6 5
29471 137/300 Write block 2015 len 12288 data: 6 6 5
29472 138/300 Read block 1966 len 36864
29473 139/300 Read block 3689 len 40960
29474 140/300 Write block 2494 len 24576 data: 3 3 3 3 4 4
29475 141/300 Write block 4937 len 28672 data: 7 7 7 5 5 4 4
29476 142/300 Write block 303 len 24576 data: 4 5 5 6 6 5
29477 143/300 Write block 2082 len 36864 data: 6 7 8 6 5 4 4 4 4
29478 144/300 Write block 2052 len 8192 data: 5 5
29479 145/300 Write block 1325 len 32768 data: 5 5 3 3 5 5 5 4
29480 146/300 Write block 601 len 8192 data: 9 11
29481 147/300 Read block 4886 len 20480
29482 148/300 Write block 2023 len 12288 data: 6 6 4
29483 149/300 Read block 3965 len 36864
29484 150/300 Read block 2785 len 16384
29485 151/300 Write block 1898 len 36864 data: 5 5 5 5 5 5 4 4 3
29486 152/300 Read block 333 len 28672
29487 153/300 Read block 2501 len 32768
29488 154/300 Read block 4535 len 8192
29489 155/300 Read block 4457 len 12288
29490 156/300 Read block 802 len 36864
29491 157/300 Read block 3434 len 4096
29492 158/300 Read block 1599 len 40960
29493 159/300 Read block 3634 len 36864
29494 160/300 Write block 1065 len 28672 data: 5 5 6 6 6 5 6
29495 161/300 Read block 862 len 16384
29496 162/300 Flush
29497 163/300 Read block 3707 len 36864
29498 164/300 Read block 128 len 20480
29499 165/300 Write block 3215 len 16384 data: 6 7 8 9
29500 166/300 Write block 4865 len 28672 data: 4 3 4 3 4 6 7
29501 167/300 Read block 752 len 40960
29502 168/300 Write block 4761 len 36864 data: 5 5 4 4 3 4 5 7 6
29503 169/300 Read block 4748 len 8192
29504 170/300 Read block 3571 len 24576
29505 171/300 Read block 2293 len 32768
29506 172/300 Write block 3891 len 4096 data: 6
29507 173/300 Write block 599 len 20480 data: 10 8 10 12 9
29508 174/300 Write block 2602 len 12288 data: 3 3 3
29509 175/300 Write block 478 len 12288 data: 3 3 4
29510 176/300 Read block 1448 len 8192
29511 177/300 Write block 4479 len 4096 data: 7
29512 178/300 Write block 284 len 40960 data: 3 3 4 5 6 5 5 4 4 6
29513 179/300 Flush
29514 180/300 Read block 3969 len 16384
29515 181/300 Read block 2522 len 20480
29516 182/300 Write block 2449 len 40960 data: 3 3 5 5 6 6 6 5 4 2
29517 183/300 Flush
29518 184/300 Read block 365 len 24576
29519 185/300 Read block 3421 len 36864
29520 186/300 Flush
29521 187/300 Read block 1059 len 12288
29522 188/300 Read block 3193 len 16384
29523 189/300 Read block 2617 len 8192
29524 190/300 Write block 1837 len 16384 data: 8 8 8 6
29525 191/300 Write block 2711 len 32768 data: 4 6 7 9 9 9 11 7
29526 192/300 Read block 3501 len 20480
29527 193/300 Read block 299 len 8192
29528 194/300 Flush
29529 195/300 Write block 20 len 36864 data: 5 4 5 5 6 8 7 7 7
29530 196/300 Read block 2845 len 8192
29531 197/300 Write block 4866 len 20480 data: 4 5 4 5 7
29532 198/300 Read block 4137 len 20480
29533 199/300 Read block 1958 len 8192
29534 200/300 Read block 3607 len 12288
29535 201/300 Flush
29536 202/300 Read block 1426 len 36864
29537 203/300 Write block 2378 len 28672 data: 9 9 10 11 8 9 9
29538 204/300 Read block 3216 len 16384
29539 205/300 Read block 459 len 28672
29540 206/300 Read block 788 len 28672
29541 207/300 Read block 1750 len 8192
29542 208/300 Read block 851 len 24576
29543 209/300 Read block 3449 len 12288
29544 210/300 Read block 4570 len 4096
29545 211/300 Write block 4249 len 24576 data: 6 6 4 5 5 3
29546 212/300 Read block 4282 len 20480
29547 213/300 Write block 4480 len 32768 data: 7 6 6 5 5 4 4 4
29548 214/300 Write block 1495 len 28672 data: 7 6 10 7 7 7 7
29549 215/300 Read block 1531 len 16384
29550 216/300 Write block 4626 len 8192 data: 9 7
29551 217/300 Write block 1766 len 4096 data: 3
29552 218/300 Read block 2207 len 32768
29553 219/300 Read block 189 len 40960
29554 220/300 Read block 2091 len 8192
29555 221/300 Write block 1818 len 8192 data: 5 6
29556 222/300 Write block 4279 len 28672 data: 3 4 5 6 6 3 3
29557 223/300 Write block 3473 len 16384 data: 6 6 6 8
29558 224/300 Read block 3197 len 32768
29559 225/300 Write block 2671 len 36864 data: 6 6 6 5 5 5 3 3 3
29560 226/300 Flush
29561 227/300 Read block 3636 len 20480
29562 228/300 Flush
29563 229/300 Read block 68 len 4096
29564 230/300 Write block 2061 len 20480 data: 6 6 5 5 7
29565 231/300 Read block 3176 len 40960
29566 232/300 Read block 1965 len 40960
29567 233/300 Read block 4338 len 8192
29568 234/300 Write block 3508 len 12288 data: 8 8 8
29569 235/300 Read block 524 len 28672
29570 236/300 Read block 508 len 4096
29571 237/300 Read block 1724 len 12288
29572 238/300 Read block 3159 len 16384
29573 239/300 Write block 1268 len 8192 data: 5 5
29574 240/300 Read block 1275 len 28672
29575 241/300 Write block 3622 len 36864 data: 8 7 6 6 5 6 7 7 7
29576 242/300 Read block 3946 len 12288
29577 243/300 Read block 1203 len 40960
29578 244/300 Flush
29579 245/300 Read block 1445 len 24576
29580 246/300 Read block 2034 len 20480
29581 247/300 Write block 1743 len 32768 data: 5 4 5 5 5 5 6 4
29582 248/300 Write block 4828 len 12288 data: 7 6 8
29583 249/300 Write block 1489 len 4096 data: 4
29584 250/300 Read block 2625 len 28672
29585 251/300 Write block 624 len 40960 data: 5 7 8 6 3 5 6 7 8 7
29586 252/300 Read block 4754 len 28672
29587 253/300 Read block 2123 len 8192
29588 254/300 Read block 148 len 16384
29589 255/300 Read block 4924 len 40960
29590 256/300 Read block 3057 len 24576
29591 257/300 Read block 4446 len 12288
29592 258/300 Read block 524 len 8192
29593 259/300 Read block 3835 len 20480
29594 260/300 Write block 3312 len 24576 data: 7 6 8 8 8 9
29595 261/300 Write block 4063 len 40960 data: 6 8 8 7 7 7 5 5 6 6
29596 262/300 Write block 3855 len 32768 data: 6 5 5 7 8 8 9 9
29597 263/300 Write block 189 len 12288 data: 6 7 8
29598 264/300 Flush
29599 265/300 Write block 3767 len 28672 data: 7 7 5 5 4 5 5
29600 266/300 Read block 4617 len 24576
29601 267/300 Write block 771 len 32768 data: 7 7 7 7 7 7 6 6
29602 268/300 Read block 4415 len 36864
29603 269/300 Write block 3273 len 24576 data: 3 3 3 4 4 5
29604 270/300 Write block 1672 len 36864 data: 6 5 6 6 5 5 5 5 3
29605 271/300 Write block 4241 len 40960 data: 7 8 9 7 8 9 9 7 7 7
29606 272/300 Write block 363 len 36864 data: 4 5 6 8 7 6 5 6 4
29607 273/300 Read block 2722 len 16384
29608 274/300 Write block 1988 len 40960 data: 4 3 2 3 3 4 4 4 5 5
29609 275/300 Read block 4354 len 20480
29610 276/300 Write block 775 len 28672 data: 8 8 7 7 5 4 4
29611 277/300 Write block 4387 len 36864 data: 6 6 6 7 5 5 4 2 2
29612 278/300 Write block 2255 len 24576 data: 6 4 4 3 5 6
29613 279/300 Read block 332 len 12288
29614 280/300 Read block 688 len 36864
29615 281/300 Write block 1969 len 36864 data: 6 6 6 7 8 7 7 6 6
29616 282/300 Read block 3588 len 36864
29617 283/300 Read block 3606 len 40960
29618 284/300 Read block 1077 len 36864
29619 285/300 Read block 3835 len 24576
29620 286/300 Read block 2942 len 24576
29621 287/300 Read block 726 len 36864
29622 288/300 Write block 3428 len 32768 data: 9 8 6 5 7 6 6 5
29623 289/300 Write block 3351 len 4096 data: 3
29624 290/300 Read block 1713 len 12288
29625 291/300 Write block 2123 len 8192 data: 6 8
29626 292/300 Write block 963 len 28672 data: 5 5 6 6 7 6 7
29627 293/300 Write block 4068 len 20480 data: 8 6 6 7 7
29628 294/300 Read block 4156 len 20480
29629 295/300 Write block 1094 len 20480 data: 6 5 6 6 8
29630 296/300 Write block 1284 len 16384 data: 4 5 4 5
29631 297/300 Write block 3117 len 4096 data: 5
29632 298/300 Read block 3622 len 32768
29633 299/300 Read block 3025 len 36864
29634 300/300 Read block 2987 len 12288
29635 301/300 Write block 3769 len 40960 data: 6 6 5 6 6 3 3 4 3 4
29636 Main task has work to do, go find it
29637 got dsc Start(1)
29638 start 1
29639 Nov 28 04:56:33.110 INFO request completed, latency_us: 225, response_code: 204, uri: /start/cid/1, method: POST, req_id: 724ee14e-3da8-46d2-81aa-0cc9b1fb7cda, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
29640 Replay: started 1, returned:Ok(())
29641 ----------------------------------------------------------------
29642 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
29643 States: Active Offline Active
29644 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
29645 8245 Acked Read 10 Done Sent Done false
29646 8246 Acked Read 6 Done Sent Done false
29647 8247 Acked Read 1 Done Sent Done false
29648 8248 Acked Write 1 Done Sent Done false
29649 8249 Acked Read 4 Done Sent Done false
29650 8250 Acked Read 4 Done Sent Done false
29651 8251 Acked Write 1 Done Sent Done false
29652 8252 Acked Write 7 Done Sent Done false
29653 8253 Acked Read 5 Done Sent Done false
29654 8254 Acked Read 2 Done Sent Done false
29655 8255 Acked Read 10 Done Sent Done false
29656 8256 Acked Read 8 Done Sent Done false
29657 8257 Acked Write 9 Done Sent Done false
29658 8258 Acked Read 6 Done Sent Done false
29659 8259 Acked Read 8 Done Sent Done false
29660 8260 Acked Read 5 Done Sent Done false
29661 8261 Acked Read 1 Done Sent Done false
29662 8262 Acked Read 9 Done Sent Done false
29663 8263 Acked Read 7 Done Sent Done false
29664 8264 Acked Write 1 Done Sent Done false
29665 8265 Acked Write 3 Done Sent Done false
29666 8266 Acked Read 10 Done Sent Done false
29667 8267 Acked Read 3 Done Sent Done false
29668 8268 Acked Read 4 Done Sent Done false
29669 8269 Acked Read 3 Done Sent Done false
29670 8270 Acked Flush 0 Done Sent Done false
29671 8271 Acked Flush 0 Done Sent Done false
29672 8272 Acked Flush 0 Done Sent Done false
29673 8273 Acked Read 3 Done Sent Done false
29674 8274 Acked Write 2 Done Sent Done false
29675 8275 Acked Read 5 Done Sent Done false
29676 8276 Acked Flush 0 Done Sent Done false
29677 8277 Acked Flush 0 Done Sent Done false
29678 8278 Acked Read 5 Done Sent Done false
29679 8279 Acked Write 10 Done Sent Done false
29680 8280 Acked Flush 0 Done Sent Done false
29681 8281 Acked Flush 0 Done Sent Done false
29682 8282 Acked Read 8 Done Sent Done false
29683 8283 Acked Read 7 Done Sent Done false
29684 8284 Acked Flush 0 Done Sent Done false
29685 8285 Acked Write 4 Done Sent Done false
29686 8286 Acked Read 9 Done Sent Done false
29687 8287 Acked Read 6 Done Sent Done false
29688 8288 Acked Read 7 Done Sent Done false
29689 8289 Acked Read 2 Done Sent Done false
29690 8290 Acked Write 9 Done Sent Done false
29691 8291 Acked Read 8 Done Sent Done false
29692 8292 Acked Write 7 Done Sent Done false
29693 8293 Acked Read 5 Done Sent Done false
29694 8294 Acked Write 7 Done Sent Done false
29695 8295 Acked Read 8 Done Sent Done false
29696 8296 Acked Flush 0 Done Sent Done false
29697 8297 Acked Write 7 Done Sent Done false
29698 8298 Acked Read 7 Done Sent Done false
29699 8299 Acked Write 6 Done Sent Done false
29700 8300 Acked Write 6 Done Sent Done false
29701 8301 Acked Write 2 Done Sent Done false
29702 8302 Acked Flush 0 Done Sent Done false
29703 8303 Acked Read 1 Done Sent Done false
29704 8304 Acked Read 2 Done Sent Done false
29705 8305 Acked Flush 0 Done Sent Done false
29706 8306 Acked Write 8 Done Sent Done false
29707 8307 Acked Flush 0 Done Sent Done false
29708 8308 Acked Read 7 Done Sent Done false
29709 8309 Acked Flush 0 Done Sent Done false
29710 8310 Acked Read 2 Done Sent Done false
29711 8311 Acked Write 7 Done Sent Done false
29712 8312 Acked Write 9 Done Sent Done false
29713 8313 Acked Flush 0 Done Sent Done false
29714 8314 Acked Read 8 Done Sent Done false
29715 8315 Acked Write 3 Done Sent Done false
29716 8316 Acked Read 8 Done Sent Done false
29717 8317 Acked Flush 0 Done Sent Done false
29718 8318 Acked Read 5 Done Sent Done false
29719 8319 Acked Read 7 Done Sent Done false
29720 8320 Acked Write 7 Done Sent Done false
29721 8321 Acked Write 6 Done Sent Done false
29722 8322 Acked Write 3 Done Sent Done false
29723 8323 Acked Flush 0 Done Sent Done false
29724 8324 Acked Flush 0 Done Sent Done false
29725 8325 Acked Flush 0 Done Sent Done false
29726 8326 Acked Write 8 Done Sent Done false
29727 8327 Acked Read 9 Done Sent Done false
29728 8328 Acked Flush 0 Done Sent Done false
29729 8329 Acked Read 9 Done Sent Done false
29730 8330 Acked Read 10 Done Sent Done false
29731 8331 Acked Read 10 Done Sent Done false
29732 8332 Acked Write 2 Done Sent Done false
29733 8333 Acked Read 1 Done Sent Done false
29734 8334 Acked Read 1 Done Sent Done false
29735 8335 Acked Flush 0 Done Sent Done false
29736 8336 Acked Read 8 Done Sent Done false
29737 8337 Acked Flush 0 Done Sent Done false
29738 8338 Acked Read 6 Done Sent Done false
29739 8339 Acked Read 6 Done Sent Done false
29740 8340 Acked Write 1 Done Sent Done false
29741 8341 Acked Read 1 Done Sent Done false
29742 8342 Acked Write 9 Done Sent Done false
29743 8343 Acked Write 6 Done Sent Done false
29744 8344 Acked Read 5 Done Sent Done false
29745 8345 Acked Read 5 Done Sent Done false
29746 8346 Acked Write 8 Done Sent Done false
29747 8347 Acked Write 7 Done Sent Done false
29748 8348 Acked Read 2 Done Sent Done false
29749 8349 Acked Read 5 Done Sent Done false
29750 8350 Acked Write 4 Done Sent Done false
29751 8351 Acked Write 7 Done Sent Done false
29752 8352 Acked Write 2 Done Sent Done false
29753 8353 Acked Write 4 Done Sent Done false
29754 8354 Acked Flush 0 Done Sent Done false
29755 8355 Acked Read 7 Done Sent Done false
29756 8356 Acked Read 5 Done Sent Done false
29757 8357 Acked Read 3 Done Sent Done false
29758 8358 Acked Write 10 Done Sent Done false
29759 8359 Acked Flush 0 Done Sent Done false
29760 8360 Acked Write 2 Done Sent Done false
29761 8361 Acked Read 10 Done Sent Done false
29762 8362 Acked Write 3 Done Sent Done false
29763 8363 Acked Read 5 Done Sent Done false
29764 8364 Acked Read 7 Done Sent Done false
29765 8365 Acked Write 6 Done Sent Done false
29766 8366 Acked Write 1 Done Sent Done false
29767 8367 Acked Write 5 Done Sent Done false
29768 8368 Acked Read 2 Done Sent Done false
29769 8369 Acked Read 6 Done Sent Done false
29770 8370 Acked Write 2 Done Sent Done false
29771 8371 Acked Read 9 Done Sent Done false
29772 8372 Acked Read 4 Done Sent Done false
29773 8373 Acked Flush 0 Done Sent Done false
29774 8374 Acked Write 1 Done Sent Done false
29775 8375 Acked Read 3 Done Sent Done false
29776 8376 Acked Flush 0 Done Sent Done false
29777 8377 Acked Write 4 Done Sent Done false
29778 8378 Acked Read 8 Done Sent Done false
29779 8379 Acked Read 6 Done Sent Done false
29780 8380 Acked Write 3 Done Sent Done false
29781 8381 Acked Write 3 Done Sent Done false
29782 8382 Acked Read 9 Done Sent Done false
29783 8383 Acked Read 10 Done Sent Done false
29784 8384 Acked Write 6 Done Sent Done false
29785 8385 Acked Write 7 Done Sent Done false
29786 8386 Acked Write 6 Done Sent Done false
29787 8387 Acked Write 9 Done Sent Done false
29788 8388 Acked Write 2 Done Sent Done false
29789 8389 Acked Write 8 Done Sent Done false
29790 8390 Acked Write 2 Done Sent Done false
29791 8391 Acked Read 5 Done Sent Done false
29792 8392 Acked Write 3 Done Sent Done false
29793 8393 Acked Read 9 Done Sent Done false
29794 8394 Acked Read 4 Done Sent Done false
29795 8395 Acked Write 9 Done Sent Done false
29796 8396 Acked Read 7 Done Sent Done false
29797 8397 Acked Read 8 Done Sent Done false
29798 8398 Acked Read 2 Done Sent Done false
29799 8399 Acked Read 3 Done Sent Done false
29800 8400 Acked Read 9 Done Sent Done false
29801 8401 Acked Read 1 Done Sent Done false
29802 8402 Acked Read 10 Done Sent Done false
29803 8403 Acked Read 9 Done Sent Done false
29804 8404 Acked Write 7 Done Sent Done false
29805 8405 Acked Read 4 Done Sent Done false
29806 8406 Acked Flush 0 Done Sent Done false
29807 8407 Acked Read 9 Done Sent Done false
29808 8408 Acked Read 5 Done Sent Done false
29809 8409 Acked Write 4 Done Sent Done false
29810 8410 Acked Write 7 Done Sent Done false
29811 8411 Acked Read 10 Done Sent Done false
29812 8412 Acked Write 9 Done Sent Done false
29813 8413 Acked Read 2 Done Sent Done false
29814 8414 Acked Read 6 Done Sent Done false
29815 8415 Acked Read 8 Done Sent Done false
29816 8416 Acked Write 1 Done Sent Done false
29817 8417 Acked Write 5 Done Sent Done false
29818 8418 Acked Write 3 Done Sent Done false
29819 8419 Acked Write 3 Done Sent Done false
29820 8420 Acked Read 2 Done Sent Done false
29821 8421 Acked Write 1 Done Sent Done false
29822 8422 Acked Write 10 Done Sent Done false
29823 8423 Acked Flush 0 Done Sent Done false
29824 8424 Acked Read 4 Done Sent Done false
29825 8425 Acked Read 5 Done Sent Done false
29826 8426 Acked Write 10 Done Sent Done false
29827 8427 Acked Flush 0 Done Sent Done false
29828 8428 Acked Read 6 Done Sent Done false
29829 8429 Acked Read 9 Done Sent Done false
29830 8430 Acked Flush 0 Done Sent Done false
29831 8431 Acked Read 3 Done Sent Done false
29832 8432 Acked Read 4 Done Sent Done false
29833 8433 Acked Read 2 Done Sent Done false
29834 8434 Acked Write 4 Done Sent Done false
29835 8435 Acked Write 8 Done Sent Done false
29836 8436 Acked Read 5 Done Sent Done false
29837 8437 Acked Read 2 Done Sent Done false
29838 8438 Acked Flush 0 Done Sent Done false
29839 8439 Acked Write 9 Done Sent Done false
29840 8440 Acked Read 2 Done Sent Done false
29841 8441 Acked Write 5 Done Sent Done false
29842 8442 Acked Read 5 Done Sent Done false
29843 8443 Acked Read 2 Done Sent Done false
29844 8444 Acked Read 3 Done Sent Done false
29845 8445 Acked Flush 0 Done Sent Done false
29846 8446 Acked Read 9 Done Sent Done false
29847 8447 Acked Write 7 Done Sent Done false
29848 8448 Acked Read 4 Done Sent Done false
29849 8449 Acked Read 7 Done Sent Done false
29850 8450 Acked Read 7 Done Sent Done false
29851 8451 Acked Read 2 Done Sent Done false
29852 8452 Acked Read 6 Done Sent Done false
29853 8453 Acked Read 3 Done Sent Done false
29854 8454 Acked Read 1 Done Sent Done false
29855 8455 Acked Write 6 Done Sent Done false
29856 8456 Acked Read 5 Done Sent Done false
29857 8457 Acked Write 8 Done Sent Done false
29858 8458 Acked Write 7 Done Sent Done false
29859 8459 Acked Read 4 Done Sent Done false
29860 8460 Acked Write 2 Done Sent Done false
29861 8461 Acked Write 1 Done Sent Done false
29862 8462 Acked Read 8 Done Sent Done false
29863 8463 Acked Read 10 Done Sent Done false
29864 8464 Acked Read 2 Done Sent Done false
29865 8465 Acked Write 2 Done Sent Done false
29866 8466 Acked Write 7 Done Sent Done false
29867 8467 Acked Write 4 Done Sent Done false
29868 8468 Acked Read 8 Done Sent Done false
29869 8469 Acked Write 9 Done Sent Done false
29870 8470 Acked Flush 0 Done Sent Done false
29871 8471 Acked Read 5 Done Sent Done false
29872 8472 Acked Flush 0 Done Sent Done false
29873 8473 Acked Read 1 Done Sent Done false
29874 8474 Acked Write 5 Done Sent Done false
29875 8475 Acked Read 10 Done Sent Done false
29876 8476 Acked Read 10 Done Sent Done false
29877 8477 Acked Read 2 Done Sent Done false
29878 8478 Acked Write 3 Done Sent Done false
29879 8479 Acked Read 7 Done Sent Done false
29880 8480 Acked Read 1 Done Sent Done false
29881 8481 Acked Read 3 Done Sent Done false
29882 8482 Acked Read 4 Done Sent Done false
29883 8483 Acked Write 2 Done Sent Done false
29884 8484 Acked Read 7 Done Sent Done false
29885 8485 Acked Write 9 Done Sent Done false
29886 8486 Acked Read 3 Done Sent Done false
29887 8487 Acked Read 10 Done Sent Done false
29888 8488 Acked Flush 0 Done Sent Done false
29889 8489 Acked Read 6 Done Sent Done false
29890 8490 Acked Read 5 Done Sent Done false
29891 8491 Acked Write 8 Done Sent Done false
29892 8492 Acked Write 3 Done Sent Done false
29893 8493 Acked Write 1 Done Sent Done false
29894 8494 Acked Read 7 Done Sent Done false
29895 8495 Acked Write 10 Done Sent Done false
29896 8496 Acked Read 7 Done Sent Done false
29897 8497 Acked Read 2 Done Sent Done false
29898 8498 Acked Read 4 Done Sent Done false
29899 8499 Acked Read 10 Done Sent Done false
29900 8500 Acked Read 6 Done Sent Done false
29901 8501 Acked Read 3 Done Sent Done false
29902 8502 Acked Read 2 Done Sent Done false
29903 8503 Acked Read 5 Done Sent Done false
29904 8504 Acked Write 6 Done Sent Done false
29905 8505 Acked Write 10 Done Sent Done false
29906 8506 Acked Write 8 Done Sent Done false
29907 8507 Acked Write 3 Done Sent Done false
29908 8508 Acked Flush 0 Done Sent Done false
29909 8509 Acked Write 7 Done Sent Done false
29910 8510 Acked Read 6 Done Sent Done false
29911 8511 Acked Write 8 Done Sent Done false
29912 8512 Acked Read 9 Done Sent Done false
29913 8513 Acked Write 6 Done Sent Done false
29914 8514 Acked Write 9 Done Sent Done false
29915 8515 Acked Write 10 Done Sent Done false
29916 8516 Acked Write 9 Done Sent Done false
29917 8517 Acked Read 4 Done Sent Done false
29918 8518 Acked Write 10 Done Sent Done false
29919 8519 Acked Read 5 Done Sent Done false
29920 8520 Acked Write 7 Done Sent Done false
29921 8521 Acked Write 9 Done Sent Done false
29922 8522 Acked Write 6 Done Sent Done false
29923 8523 Acked Read 3 Done Sent Done false
29924 8524 Acked Read 9 Done Sent Done false
29925 8525 Acked Write 9 Done Sent Done false
29926 8526 Acked Read 9 Done Sent Done false
29927 8527 Acked Read 10 Done Sent Done false
29928 8528 Acked Read 9 Done Sent Done false
29929 8529 Acked Read 6 Done Sent Done false
29930 8530 Acked Read 6 Done Sent Done false
29931 8531 Acked Read 9 Done Sent Done false
29932 8532 Acked Write 8 Done Sent Done false
29933 8533 Acked Write 1 Done Sent Done false
29934 8534 Acked Read 3 Done Sent Done false
29935 8535 Acked Write 2 Done Sent Done false
29936 8536 Acked Write 7 Done Sent Done false
29937 8537 Acked Write 5 Done Sent Done false
29938 8538 Acked Read 5 Done Sent Done false
29939 8539 Acked Write 5 Done Sent Done false
29940 8540 Acked Write 4 Done Sent Done false
29941 8541 Acked Write 1 Done Sent Done false
29942 8542 Acked Read 8 Done Sent Done false
29943 8543 Acked Read 9 Done Sent Done false
29944 8544 Acked Read 3 Done Sent Done false
29945 8545 Acked Write 10 Done Sent Done false
29946 STATES DS:0 DS:1 DS:2 TOTAL
29947 Sent 0 301 0 301
29948 Done 301 0 301 602
29949 Skipped 0 0 0 0
29950 Error 0 0 0 0
29951 Last Flush: 8508 8244 8508
29952 Downstairs last five completed: 8244 8243 8242 8241 8240
29953 Upstairs last five completed: 8545 8544 8543 8542 8541
29954 CLIENT: Up:0 ds:301 act:2
29955 [8820] got start action
29956 [8820] I am going to restart
29957 Starting downstairs at port 8820
29958 Make output file at "/tmp/dsc/downstairs-8820.txt"
29959 [8820][1] reports Starting
29960 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1227)
29961 [8820][1] reports Running
29962 ----------------------------------------------------------------
29963 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
29964 States: Active Offline Active
29965 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
29966 8245 Acked Read 10 Done Sent Done false
29967 8246 Acked Read 6 Done Sent Done false
29968 8247 Acked Read 1 Done Sent Done false
29969 8248 Acked Write 1 Done Sent Done false
29970 8249 Acked Read 4 Done Sent Done false
29971 8250 Acked Read 4 Done Sent Done false
29972 8251 Acked Write 1 Done Sent Done false
29973 8252 Acked Write 7 Done Sent Done false
29974 8253 Acked Read 5 Done Sent Done false
29975 8254 Acked Read 2 Done Sent Done false
29976 8255 Acked Read 10 Done Sent Done false
29977 8256 Acked Read 8 Done Sent Done false
29978 8257 Acked Write 9 Done Sent Done false
29979 8258 Acked Read 6 Done Sent Done false
29980 8259 Acked Read 8 Done Sent Done false
29981 8260 Acked Read 5 Done Sent Done false
29982 8261 Acked Read 1 Done Sent Done false
29983 8262 Acked Read 9 Done Sent Done false
29984 8263 Acked Read 7 Done Sent Done false
29985 8264 Acked Write 1 Done Sent Done false
29986 8265 Acked Write 3 Done Sent Done false
29987 8266 Acked Read 10 Done Sent Done false
29988 8267 Acked Read 3 Done Sent Done false
29989 8268 Acked Read 4 Done Sent Done false
29990 8269 Acked Read 3 Done Sent Done false
29991 8270 Acked Flush 0 Done Sent Done false
29992 8271 Acked Flush 0 Done Sent Done false
29993 8272 Acked Flush 0 Done Sent Done false
29994 8273 Acked Read 3 Done Sent Done false
29995 8274 Acked Write 2 Done Sent Done false
29996 8275 Acked Read 5 Done Sent Done false
29997 8276 Acked Flush 0 Done Sent Done false
29998 8277 Acked Flush 0 Done Sent Done false
29999 8278 Acked Read 5 Done Sent Done false
30000 8279 Acked Write 10 Done Sent Done false
30001 8280 Acked Flush 0 Done Sent Done false
30002 8281 Acked Flush 0 Done Sent Done false
30003 8282 Acked Read 8 Done Sent Done false
30004 8283 Acked Read 7 Done Sent Done false
30005 8284 Acked Flush 0 Done Sent Done false
30006 8285 Acked Write 4 Done Sent Done false
30007 8286 Acked Read 9 Done Sent Done false
30008 8287 Acked Read 6 Done Sent Done false
30009 8288 Acked Read 7 Done Sent Done false
30010 8289 Acked Read 2 Done Sent Done false
30011 8290 Acked Write 9 Done Sent Done false
30012 8291 Acked Read 8 Done Sent Done false
30013 8292 Acked Write 7 Done Sent Done false
30014 8293 Acked Read 5 Done Sent Done false
30015 8294 Acked Write 7 Done Sent Done false
30016 8295 Acked Read 8 Done Sent Done false
30017 8296 Acked Flush 0 Done Sent Done false
30018 8297 Acked Write 7 Done Sent Done false
30019 8298 Acked Read 7 Done Sent Done false
30020 8299 Acked Write 6 Done Sent Done false
30021 8300 Acked Write 6 Done Sent Done false
30022 8301 Acked Write 2 Done Sent Done false
30023 8302 Acked Flush 0 Done Sent Done false
30024 8303 Acked Read 1 Done Sent Done false
30025 8304 Acked Read 2 Done Sent Done false
30026 8305 Acked Flush 0 Done Sent Done false
30027 8306 Acked Write 8 Done Sent Done false
30028 8307 Acked Flush 0 Done Sent Done false
30029 8308 Acked Read 7 Done Sent Done false
30030 8309 Acked Flush 0 Done Sent Done false
30031 8310 Acked Read 2 Done Sent Done false
30032 8311 Acked Write 7 Done Sent Done false
30033 8312 Acked Write 9 Done Sent Done false
30034 8313 Acked Flush 0 Done Sent Done false
30035 8314 Acked Read 8 Done Sent Done false
30036 8315 Acked Write 3 Done Sent Done false
30037 8316 Acked Read 8 Done Sent Done false
30038 8317 Acked Flush 0 Done Sent Done false
30039 8318 Acked Read 5 Done Sent Done false
30040 8319 Acked Read 7 Done Sent Done false
30041 8320 Acked Write 7 Done Sent Done false
30042 8321 Acked Write 6 Done Sent Done false
30043 8322 Acked Write 3 Done Sent Done false
30044 8323 Acked Flush 0 Done Sent Done false
30045 8324 Acked Flush 0 Done Sent Done false
30046 8325 Acked Flush 0 Done Sent Done false
30047 8326 Acked Write 8 Done Sent Done false
30048 8327 Acked Read 9 Done Sent Done false
30049 8328 Acked Flush 0 Done Sent Done false
30050 8329 Acked Read 9 Done Sent Done false
30051 8330 Acked Read 10 Done Sent Done false
30052 8331 Acked Read 10 Done Sent Done false
30053 8332 Acked Write 2 Done Sent Done false
30054 8333 Acked Read 1 Done Sent Done false
30055 8334 Acked Read 1 Done Sent Done false
30056 8335 Acked Flush 0 Done Sent Done false
30057 8336 Acked Read 8 Done Sent Done false
30058 8337 Acked Flush 0 Done Sent Done false
30059 8338 Acked Read 6 Done Sent Done false
30060 8339 Acked Read 6 Done Sent Done false
30061 8340 Acked Write 1 Done Sent Done false
30062 8341 Acked Read 1 Done Sent Done false
30063 8342 Acked Write 9 Done Sent Done false
30064 8343 Acked Write 6 Done Sent Done false
30065 8344 Acked Read 5 Done Sent Done false
30066 8345 Acked Read 5 Done Sent Done false
30067 8346 Acked Write 8 Done Sent Done false
30068 8347 Acked Write 7 Done Sent Done false
30069 8348 Acked Read 2 Done Sent Done false
30070 8349 Acked Read 5 Done Sent Done false
30071 8350 Acked Write 4 Done Sent Done false
30072 8351 Acked Write 7 Done Sent Done false
30073 8352 Acked Write 2 Done Sent Done false
30074 8353 Acked Write 4 Done Sent Done false
30075 8354 Acked Flush 0 Done Sent Done false
30076 8355 Acked Read 7 Done Sent Done false
30077 8356 Acked Read 5 Done Sent Done false
30078 8357 Acked Read 3 Done Sent Done false
30079 8358 Acked Write 10 Done Sent Done false
30080 8359 Acked Flush 0 Done Sent Done false
30081 8360 Acked Write 2 Done Sent Done false
30082 8361 Acked Read 10 Done Sent Done false
30083 8362 Acked Write 3 Done Sent Done false
30084 8363 Acked Read 5 Done Sent Done false
30085 8364 Acked Read 7 Done Sent Done false
30086 8365 Acked Write 6 Done Sent Done false
30087 8366 Acked Write 1 Done Sent Done false
30088 8367 Acked Write 5 Done Sent Done false
30089 8368 Acked Read 2 Done Sent Done false
30090 8369 Acked Read 6 Done Sent Done false
30091 8370 Acked Write 2 Done Sent Done false
30092 8371 Acked Read 9 Done Sent Done false
30093 8372 Acked Read 4 Done Sent Done false
30094 8373 Acked Flush 0 Done Sent Done false
30095 8374 Acked Write 1 Done Sent Done false
30096 8375 Acked Read 3 Done Sent Done false
30097 8376 Acked Flush 0 Done Sent Done false
30098 8377 Acked Write 4 Done Sent Done false
30099 8378 Acked Read 8 Done Sent Done false
30100 8379 Acked Read 6 Done Sent Done false
30101 8380 Acked Write 3 Done Sent Done false
30102 8381 Acked Write 3 Done Sent Done false
30103 8382 Acked Read 9 Done Sent Done false
30104 8383 Acked Read 10 Done Sent Done false
30105 8384 Acked Write 6 Done Sent Done false
30106 8385 Acked Write 7 Done Sent Done false
30107 8386 Acked Write 6 Done Sent Done false
30108 8387 Acked Write 9 Done Sent Done false
30109 8388 Acked Write 2 Done Sent Done false
30110 8389 Acked Write 8 Done Sent Done false
30111 8390 Acked Write 2 Done Sent Done false
30112 8391 Acked Read 5 Done Sent Done false
30113 8392 Acked Write 3 Done Sent Done false
30114 8393 Acked Read 9 Done Sent Done false
30115 8394 Acked Read 4 Done Sent Done false
30116 8395 Acked Write 9 Done Sent Done false
30117 8396 Acked Read 7 Done Sent Done false
30118 8397 Acked Read 8 Done Sent Done false
30119 8398 Acked Read 2 Done Sent Done false
30120 8399 Acked Read 3 Done Sent Done false
30121 8400 Acked Read 9 Done Sent Done false
30122 8401 Acked Read 1 Done Sent Done false
30123 8402 Acked Read 10 Done Sent Done false
30124 8403 Acked Read 9 Done Sent Done false
30125 8404 Acked Write 7 Done Sent Done false
30126 8405 Acked Read 4 Done Sent Done false
30127 8406 Acked Flush 0 Done Sent Done false
30128 8407 Acked Read 9 Done Sent Done false
30129 8408 Acked Read 5 Done Sent Done false
30130 8409 Acked Write 4 Done Sent Done false
30131 8410 Acked Write 7 Done Sent Done false
30132 8411 Acked Read 10 Done Sent Done false
30133 8412 Acked Write 9 Done Sent Done false
30134 8413 Acked Read 2 Done Sent Done false
30135 8414 Acked Read 6 Done Sent Done false
30136 8415 Acked Read 8 Done Sent Done false
30137 8416 Acked Write 1 Done Sent Done false
30138 8417 Acked Write 5 Done Sent Done false
30139 8418 Acked Write 3 Done Sent Done false
30140 8419 Acked Write 3 Done Sent Done false
30141 8420 Acked Read 2 Done Sent Done false
30142 8421 Acked Write 1 Done Sent Done false
30143 8422 Acked Write 10 Done Sent Done false
30144 8423 Acked Flush 0 Done Sent Done false
30145 8424 Acked Read 4 Done Sent Done false
30146 8425 Acked Read 5 Done Sent Done false
30147 8426 Acked Write 10 Done Sent Done false
30148 8427 Acked Flush 0 Done Sent Done false
30149 8428 Acked Read 6 Done Sent Done false
30150 8429 Acked Read 9 Done Sent Done false
30151 8430 Acked Flush 0 Done Sent Done false
30152 8431 Acked Read 3 Done Sent Done false
30153 8432 Acked Read 4 Done Sent Done false
30154 8433 Acked Read 2 Done Sent Done false
30155 8434 Acked Write 4 Done Sent Done false
30156 8435 Acked Write 8 Done Sent Done false
30157 8436 Acked Read 5 Done Sent Done false
30158 8437 Acked Read 2 Done Sent Done false
30159 8438 Acked Flush 0 Done Sent Done false
30160 8439 Acked Write 9 Done Sent Done false
30161 8440 Acked Read 2 Done Sent Done false
30162 8441 Acked Write 5 Done Sent Done false
30163 8442 Acked Read 5 Done Sent Done false
30164 8443 Acked Read 2 Done Sent Done false
30165 8444 Acked Read 3 Done Sent Done false
30166 8445 Acked Flush 0 Done Sent Done false
30167 8446 Acked Read 9 Done Sent Done false
30168 8447 Acked Write 7 Done Sent Done false
30169 8448 Acked Read 4 Done Sent Done false
30170 8449 Acked Read 7 Done Sent Done false
30171 8450 Acked Read 7 Done Sent Done false
30172 8451 Acked Read 2 Done Sent Done false
30173 8452 Acked Read 6 Done Sent Done false
30174 8453 Acked Read 3 Done Sent Done false
30175 8454 Acked Read 1 Done Sent Done false
30176 8455 Acked Write 6 Done Sent Done false
30177 8456 Acked Read 5 Done Sent Done false
30178 8457 Acked Write 8 Done Sent Done false
30179 8458 Acked Write 7 Done Sent Done false
30180 8459 Acked Read 4 Done Sent Done false
30181 8460 Acked Write 2 Done Sent Done false
30182 8461 Acked Write 1 Done Sent Done false
30183 8462 Acked Read 8 Done Sent Done false
30184 8463 Acked Read 10 Done Sent Done false
30185 8464 Acked Read 2 Done Sent Done false
30186 8465 Acked Write 2 Done Sent Done false
30187 8466 Acked Write 7 Done Sent Done false
30188 8467 Acked Write 4 Done Sent Done false
30189 8468 Acked Read 8 Done Sent Done false
30190 8469 Acked Write 9 Done Sent Done false
30191 8470 Acked Flush 0 Done Sent Done false
30192 8471 Acked Read 5 Done Sent Done false
30193 8472 Acked Flush 0 Done Sent Done false
30194 8473 Acked Read 1 Done Sent Done false
30195 8474 Acked Write 5 Done Sent Done false
30196 8475 Acked Read 10 Done Sent Done false
30197 8476 Acked Read 10 Done Sent Done false
30198 8477 Acked Read 2 Done Sent Done false
30199 8478 Acked Write 3 Done Sent Done false
30200 8479 Acked Read 7 Done Sent Done false
30201 8480 Acked Read 1 Done Sent Done false
30202 8481 Acked Read 3 Done Sent Done false
30203 8482 Acked Read 4 Done Sent Done false
30204 8483 Acked Write 2 Done Sent Done false
30205 8484 Acked Read 7 Done Sent Done false
30206 8485 Acked Write 9 Done Sent Done false
30207 8486 Acked Read 3 Done Sent Done false
30208 8487 Acked Read 10 Done Sent Done false
30209 8488 Acked Flush 0 Done Sent Done false
30210 8489 Acked Read 6 Done Sent Done false
30211 8490 Acked Read 5 Done Sent Done false
30212 8491 Acked Write 8 Done Sent Done false
30213 8492 Acked Write 3 Done Sent Done false
30214 8493 Acked Write 1 Done Sent Done false
30215 8494 Acked Read 7 Done Sent Done false
30216 8495 Acked Write 10 Done Sent Done false
30217 8496 Acked Read 7 Done Sent Done false
30218 8497 Acked Read 2 Done Sent Done false
30219 8498 Acked Read 4 Done Sent Done false
30220 8499 Acked Read 10 Done Sent Done false
30221 8500 Acked Read 6 Done Sent Done false
30222 8501 Acked Read 3 Done Sent Done false
30223 8502 Acked Read 2 Done Sent Done false
30224 8503 Acked Read 5 Done Sent Done false
30225 8504 Acked Write 6 Done Sent Done false
30226 8505 Acked Write 10 Done Sent Done false
30227 8506 Acked Write 8 Done Sent Done false
30228 8507 Acked Write 3 Done Sent Done false
30229 8508 Acked Flush 0 Done Sent Done false
30230 8509 Acked Write 7 Done Sent Done false
30231 8510 Acked Read 6 Done Sent Done false
30232 8511 Acked Write 8 Done Sent Done false
30233 8512 Acked Read 9 Done Sent Done false
30234 8513 Acked Write 6 Done Sent Done false
30235 8514 Acked Write 9 Done Sent Done false
30236 8515 Acked Write 10 Done Sent Done false
30237 8516 Acked Write 9 Done Sent Done false
30238 8517 Acked Read 4 Done Sent Done false
30239 8518 Acked Write 10 Done Sent Done false
30240 8519 Acked Read 5 Done Sent Done false
30241 8520 Acked Write 7 Done Sent Done false
30242 8521 Acked Write 9 Done Sent Done false
30243 8522 Acked Write 6 Done Sent Done false
30244 8523 Acked Read 3 Done Sent Done false
30245 8524 Acked Read 9 Done Sent Done false
30246 8525 Acked Write 9 Done Sent Done false
30247 8526 Acked Read 9 Done Sent Done false
30248 8527 Acked Read 10 Done Sent Done false
30249 8528 Acked Read 9 Done Sent Done false
30250 8529 Acked Read 6 Done Sent Done false
30251 8530 Acked Read 6 Done Sent Done false
30252 8531 Acked Read 9 Done Sent Done false
30253 8532 Acked Write 8 Done Sent Done false
30254 8533 Acked Write 1 Done Sent Done false
30255 8534 Acked Read 3 Done Sent Done false
30256 8535 Acked Write 2 Done Sent Done false
30257 8536 Acked Write 7 Done Sent Done false
30258 8537 Acked Write 5 Done Sent Done false
30259 8538 Acked Read 5 Done Sent Done false
30260 8539 Acked Write 5 Done Sent Done false
30261 8540 Acked Write 4 Done Sent Done false
30262 8541 Acked Write 1 Done Sent Done false
30263 8542 Acked Read 8 Done Sent Done false
30264 8543 Acked Read 9 Done Sent Done false
30265 8544 Acked Read 3 Done Sent Done false
30266 8545 Acked Write 10 Done Sent Done false
30267 8546 Acked Flush 0 Done Sent Done false
30268 STATES DS:0 DS:1 DS:2 TOTAL
30269 Sent 0 302 0 302
30270 Done 302 0 302 604
30271 Skipped 0 0 0 0
30272 Error 0 0 0 0
30273 Last Flush: 8546 8244 8546
30274 Downstairs last five completed: 8244 8243 8242 8241 8240
30275 Upstairs last five completed: 8546 8545 8544 8543 8542
30276 CLIENT: Up:0 ds:302 act:2
302772024-11-28T04:56:38.871ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
302782024-11-28T04:56:38.871ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:43282) = io task client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
302792024-11-28T04:56:38.872ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
302802024-11-28T04:56:38.872ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
302812024-11-28T04:56:38.872ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
302822024-11-28T04:56:38.872ZINFOcrucible: send last flush ID to this DS: 8244 = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
302832024-11-28T04:56:38.873ZINFOcrucible: Replied this last flush ID: 8244 = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
302842024-11-28T04:56:38.873ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
302852024-11-28T04:56:38.873ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
302862024-11-28T04:56:38.873ZINFOcrucible: [1] client re-new 302 jobs since flush 8244 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
302872024-11-28T04:56:38.873ZINFOcrucible: [1] Replayed 302 jobs since flush: 8244 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
30288 ----------------------------------------------------------------
30289 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
30290 Downstairs last five completed: 8546 8545 8544 8543 8542
30291 Upstairs last five completed: 8546 8545 8544 8543 8542
30292 CLIENT: Up:0 ds:0 act:3
30293 Replay: All jobs finished
30294 Main task has work to do, go find it
30295 got dsc Stop(2)
30296 stop 2
30297 Nov 28 04:56:41.116 INFO request completed, latency_us: 355, response_code: 204, uri:[8830] Got stop action so:false kr:false
30298 /stop/cid/2, method: POST, req_id: 56762780-4fef-4330-9da0-03cfb8b227cc, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
30299 Nov 28 04:56:41.116 INFO request completed, latency_us: 117, response_code: 200, uri: /state/cid/2, method: GET, req_id: 35fb6796-38e0-42c9-b051-af1a91ece0e9, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
303002024-11-28T04:56:41.116ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
303012024-11-28T04:56:41.116ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
303022024-11-28T04:56:41.116ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
303032024-11-28T04:56:41.117ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
303042024-11-28T04:56:41.117ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
303052024-11-28T04:56:41.117ZINFOcrucible: client task is exiting = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
30306 [8830] Exited with: ExitStatus(unix_wait_status(9))
30307 [8830][2] reports Exit
30308 Nov 28 04:56:45.119 INFO request completed, latency_us: 219, response_code: 200, uri: /state/cid/2, method: GET, req_id: e70f61ed-8c76-4704-b89d-e2914574e9d0, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
30309 001/300 Write block 3386 len 8192 data: 2 3
30310 002/300 Write block 3420 len 36864 data: 4 3 2 3 5 7 8 9 10
30311 003/300 Write block 3089 len 40960 data: 7 8 8 7 5 5 4 3 3 2
30312 004/300 Flush
30313 005/300 Write block 2701 len 40960 data: 7 7 7 6 7 8 5 4 4 4
30314 006/300 Read block 3971 len 16384
30315 007/300 Read block 2122 len 8192
30316 008/300 Read block 4998 len 8192
30317 009/300 Read block 4287 len 40960
30318 010/300 Write block 3172 len 12288 data: 6 7 8
30319 011/300 Read block 2672 len 16384
30320 012/300 Write block 4375 len 20480 data: 4 6 7 7 7
30321 013/300 Read block 4235 len 36864
30322 014/300 Read block 4129 len 20480
30323 015/300 Write block 3487 len 8192 data: 5 6
30324 016/300 Flush
30325 017/300 Write block 1496 len 24576 data: 7 11 8 8 8 8
30326 018/300 Flush
30327 019/300 Write block 4472 len 4096 data: 6
30328 020/300 Write block 3281 len 24576 data: 5 5 5 5 6 7
30329 021/300 Read block 1806 len 28672
30330 022/300 Flush
30331 023/300 Flush
30332 024/300 Read block 2028 len 16384
30333 025/300 Flush
30334 026/300 Write block 3031 len 40960 data: 4 5 7 7 7 7 7 6 7 6
30335 027/300 Read block 4768 len 28672
30336 028/300 Write block 882 len 8192 data: 5 5
30337 029/300 Write block 2423 len 24576 data: 6 5 5 5 4 3
30338 030/300 Write block 239 len 4096 data: 4
30339 031/300 Write block 4093 len 16384 data: 10 9 8 8
30340 032/300 Read block 98 len 8192
30341 033/300 Read block 936 len 36864
30342 034/300 Write block 1793 len 20480 data: 3 5 5 5 4
30343 035/300 Write block 3310 len 20480 data: 8 8 8 7 9
30344 036/300 Write block 2877 len 16384 data: 7 7 7 8
30345 037/300 Read block 1362 len 4096
30346 038/300 Read block 1032 len 40960
30347 039/300 Read block 4161 len 8192
30348 040/300 Write block 571 len 32768 data: 5 6 5 4 4 3 3 3
30349 041/300 Write block 4364 len 24576 data: 10 9 8 5 5 5
30350 042/300 Read block 1916 len 36864
30351 043/300 Read block 476 len 16384
30352 044/300 Read block 1942 len 12288
30353 045/300 Write block 4491 len 4096 data: 5
30354 046/300 Read block 2616 len 8192
30355 047/300 Write block 2219 len 16384 data: 3 3 3 4
30356 048/300 Write block 4866 len 4096 data: 5
30357 049/300 Write block 1352 len 40960 data: 4 3 3 5 6 7 7 6 5 7
30358 050/300 Write block 3111 len 8192 data: 2 3
30359 051/300 Write block 284 len 32768 data: 4 4 5 6 7 6 6 5
30360 052/300 Read block 3972 len 36864
30361 053/300 Read block 1468 len 8192
30362 054/300 Write block 4982 len 28672 data: 4 4 4 4 4 5 4
30363 055/300 Write block 3835 len 24576 data: 4 3 4 4 4 4
30364 056/300 Write block 2256 len 32768 data: 5 5 4 6 7 7 6 5
30365 057/300 Read block 4453 len 20480
30366 058/300 Write block 2246 len 28672 data: 4 5 8 8 7 5 5
30367 059/300 Read block 4973 len 20480
30368 060/300 Write block 4923 len 36864 data: 6 6 6 5 7 6 6 6 5
30369 061/300 Write block 1699 len 28672 data: 5 5 5 7 10 9 12
30370 062/300 Write block 1197 len 20480 data: 6 6 4 4 5
30371 063/300 Read block 1674 len 40960
30372 064/300 Write block 1428 len 4096 data: 7
30373 065/300 Write block 765 len 4096 data: 5
30374 066/300 Read block 3715 len 32768
30375 067/300 Write block 2329 len 40960 data: 4 6 7 7 8 9 7 7 8 8
30376 068/300 Write block 1298 len 36864 data: 5 6 6 6 6 5 5 4 4
30377 069/300 Write block 4541 len 32768 data: 2 2 2 2 5 6 5 7
30378 070/300 Write block 816 len 8192 data: 8 8
30379 071/300 Read block 833 len 28672
30380 072/300 Flush
30381 073/300 Read block 1215 len 24576
30382 074/300 Write block 3774 len 12288 data: 4 4 5
30383 075/300 Write block 4580 len 24576 data: 4 4 6 6 6 4
30384 076/300 Read block 2003 len 40960
30385 077/300 Read block 122 len 4096
30386 078/300 Read block 310 len 4096
30387 079/300 Read block 742 len 20480
30388 080/300 Read block 3690 len 24576
30389 081/300 Read block 3948 len 28672
30390 082/300 Read block 3583 len 12288
30391 083/300 Read block 2592 len 12288
30392 084/300 Read block 3185 len 20480
30393 085/300 Read block 4467 len 32768
30394 086/300 Read block 1670 len 4096
30395 087/300 Read block 4123 len 20480
30396 088/300 Read block 4237 len 20480
30397 089/300 Write block 4320 len 12288 data: 4 4 5
30398 090/300 Flush
30399 091/300 Read block 446 len 8192
30400 092/300 Read block 257 len 32768
30401 093/300 Read block 4711 len 20480
30402 094/300 Read block 1616 len 36864
30403 095/300 Read block 4917 len 16384
30404 096/300 Read block 3764 len 36864
30405 097/300 Read block 1130 len 20480
30406 098/300 Write block 4969 len 36864 data: 7 7 7 5 5 4 4 4 4
30407 099/300 Write block 2531 len 8192 data: 5 6
30408 100/300 Read block 4974 len 28672
30409 101/300 Write block 2657 len 16384 data: 4 4 6 6
30410 102/300 Read block 2866 len 32768
30411 103/300 Read block 1016 len 20480
30412 104/300 Read block 4446 len 8192
30413 105/300 Read block 4735 len 4096
30414 106/300 Write block 405 len 8192 data: 5 6
30415 107/300 Flush
30416 108/300 Read block 3968 len 16384
30417 109/300 Write block 124 len 12288 data: 5 5 7
30418 110/300 Read block 2124 len 8192
30419 111/300 Read block 3233 len 40960
30420 112/300 Read block 527 len 20480
30421 113/300 Read block 4841 len 8192
30422 114/300 Write block 979 len 8192 data: 4 4
30423 115/300 Write block 3959 len 20480 data: 5 6 7 7 6
30424 116/300 Read block 2694 len 4096
30425 117/300 Write block 4982 len 32768 data: 5 5 5 5 5 6 5 5
30426 118/300 Read block 293 len 40960
30427 119/300 Read block 1996 len 8192
30428 120/300 Write block 4409 len 24576 data: 4 3 3 3 3 3
30429 121/300 Write block 780 len 36864 data: 5 5 3 4 4 3 4 4 4
30430 122/300 Read block 1347 len 36864
30431 123/300 Write block 2325 len 4096 data: 6
30432 124/300 Write block 2513 len 24576 data: 4 6 7 3 5 4
30433 125/300 Write block 3784 len 24576 data: 5 5 6 5 6 7
30434 126/300 Write block 260 len 12288 data: 7 8 7
30435 127/300 Write block 2252 len 32768 data: 6 5 5 7 6 6 5 7
30436 128/300 Read block 1569 len 36864
30437 129/300 Flush
30438 130/300 Write block 1030 len 24576 data: 4 4 5 6 5 3
30439 131/300 Read block 1329 len 36864
30440 132/300 Read block 1430 len 12288
30441 133/300 Read block 1282 len 24576
30442 134/300 Write block 911 len 36864 data: 3 3 3 3 3 3 3 3 3
30443 135/300 Write block 3701 len 24576 data: 5 5 5 5 4 3
30444 136/300 Write block 4929 len 36864 data: 7 7 6 7 5 5 4 5 8
30445 137/300 Write block 2478 len 12288 data: 5 5 4
30446 138/300 Read block 564 len 8192
30447 139/300 Write block 1091 len 20480 data: 6 8 6 7 6
30448 140/300 Read block 2280 len 12288
30449 141/300 Write block 4496 len 40960 data: 3 4 3 3 3 3 3 5 5 5
30450 142/300 Read block 989 len 24576
30451 143/300 Read block 3325 len 36864
30452 144/300 Read block 2877 len 4096
30453 145/300 Read block 3461 len 32768
30454 146/300 Read block 732 len 28672
30455 147/300 Write block 3180 len 36864 data: 5 4 4 3 5 5 5 6 6
30456 148/300 Write block 3108 len 20480 data: 4 3 3 3 4
30457 149/300 Write block 3310 len 40960 data: 9 9 9 8 10 9 9 10 8 5
30458 150/300 Flush
30459 151/300 Write block 4674 len 28672 data: 9 8 8 7 7 7 5
30460 152/300 Read block 4796 len 24576
30461 153/300 Write block 372 len 12288 data: 4 5 5
30462 154/300 Read block 4596 len 28672
30463 155/300 Write block 2042 len 40960 data: 3 2 3 3 4 5 4 4 4 5
30464 156/300 Read block 4556 len 40960
30465 157/300 Read block 2836 len 16384
30466 158/300 Write block 3382 len 28672 data: 3 3 3 2 3 4 4
30467 159/300 Flush
30468 160/300 Read block 4636 len 24576
30469 161/300 Write block 3432 len 4096 data: 8
30470 162/300 Write block 2895 len 20480 data: 3 2 2 2 3
30471 163/300 Write block 4481 len 12288 data: 7 7 6
30472 164/300 Flush
30473 165/300 Read block 770 len 16384
30474 166/300 Write block 895 len 12288 data: 6 6 4
30475 167/300 Write block 606 len 32768 data: 6 5 5 7 8 6 7 6
30476 168/300 Read block 3548 len 24576
30477 169/300 Read block 404 len 16384
30478 170/300 Flush
30479 171/300 Read block 3990 len 20480
30480 172/300 Read block 1889 len 16384
30481 173/300 Write block 3318 len 12288 data: 9 6 4
30482 174/300 Read block 1667 len 28672
30483 175/300 Read block 2615 len 40960
30484 176/300 Write block 1746 len 8192 data: 6 6
30485 177/300 Flush
30486 178/300 Read block 3683 len 12288
30487 179/300 Read block 3737 len 8192
30488 180/300 Read block 4858 len 24576
30489 181/300 Read block 3814 len 12288
30490 182/300 Write block 3461 len 8192 data: 6 6
30491 183/300 Write block 4534 len 40960 data: 8 8 7 5 5 4 3 3 3 3
30492 184/300 Read block 4906 len 4096
30493 185/300 Read block 1495 len 24576
30494 186/300 Read block 3133 len 32768
30495 187/300 Read block 602 len 40960
30496 188/300 Write block 590 len 40960 data: 5 5 5 6 6 7 8 9 9 11
30497 189/300 Read block 1381 len 4096
30498 190/300 Write block 3180 len 4096 data: 6
30499 191/300 Read block 2224 len 36864
30500 192/300 Read block 4107 len 4096
30501 193/300 Write block 4371 len 20480 data: 6 4 3 4 5
30502 194/300 Write block 4947 len 12288 data: 2 3 4
30503 195/300 Write block 3462 len 16384 data: 7 6 6 9
30504 196/300 Write block 4548 len 32768 data: 8 7 7 6 8 5 5 4
30505 197/300 Flush
30506 198/300 Write block 3677 len 4096 data: 6
30507 199/300 Read block 2236 len 40960
30508 200/300 Write block 1572 len 12288 data: 2 3 3
30509 201/300 Read block 3636 len 28672
30510 202/300 Write block 55 len 28672 data: 5 5 5 4 4 6 7
30511 203/300 Write block 2002 len 16384 data: 4 5 5 4
30512 204/300 Write block 111 len 28672 data: 6 6 5 5 3 5 5
30513 205/300 Read block 543 len 28672
30514 206/300 Write block 4787 len 28672 data: 3 3 4 6 6 7 7
30515 207/300 Write block 2718 len 4096 data: 8
30516 208/300 Read block 3100 len 40960
30517 209/300 Write block 2508 len 8192 data: 4 4
30518 210/300 Write block 263 len 40960 data: 7 7 6 5 4 4 4 5 5 6
30519 211/300 Write block 177 len 28672 data: 2 3 4 5 5 4 4
30520 212/300 Write block 520 len 4096 data: 4
30521 213/300 Read block 872 len 32768
30522 214/300 Read block 4630 len 36864
30523 215/300 Read block 251 len 16384
30524 216/300 Write block 3818 len 40960 data: 5 7 8 8 7 7 6 5 4 5
30525 217/300 Write block 3925 len 4096 data: 8
30526 218/300 Flush
30527 219/300 Read block 3759 len 12288
30528 220/300 Read block 1883 len 28672
30529 221/300 Write block 586 len 16384 data: 3 3 4 4
30530 222/300 Flush
30531 223/300 Read block 4590 len 24576
30532 224/300 Read block 4603 len 36864
30533 225/300 Write block 2341 len 16384 data: 3 3 3 4
30534 226/300 Write block 4654 len 28672 data: 3 5 6 7 8 8 7
30535 227/300 Write block 0 len 40960 data: 2 2 3 3 5 4 4 6 7 9
30536 228/300 Write block 1760 len 4096 data: 3
30537 229/300 Read block 2310 len 4096
30538 230/300 Write block 938 len 12288 data: 4 4 2
30539 231/300 Read block 542 len 20480
30540 232/300 Flush
30541 233/300 Read block 2108 len 4096
30542 234/300 Write block 117 len 32768 data: 6 7 7 10 9 9 6 6
30543 235/300 Read block 2458 len 4096
30544 236/300 Write block 1468 len 12288 data: 8 6 5
30545 237/300 Read block 4553 len 32768
30546 238/300 Read block 115 len 16384
30547 239/300 Flush
30548 240/300 Write block 2202 len 28672 data: 3 4 5 4 5 8 8
30549 241/300 Read block 3032 len 28672
30550 242/300 Read block 433 len 8192
30551 243/300 Write block 3961 len 20480 data: 8 8 7 5 5
30552 244/300 Read block 4038 len 12288
30553 245/300 Write block 4623 len 8192 data: 4 5
30554 246/300 Write block 1684 len 8192 data: 6 5
30555 247/300 Flush
30556 248/300 Read block 3018 len 16384
30557 249/300 Read block 2679 len 28672
30558 250/300 Write block 3 len 24576 data: 4 6 5 5 7 8
30559 251/300 Write block 675 len 8192 data: 5 5
30560 252/300 Read block 3138 len 40960
30561 253/300 Write block 3818 len 12288 data: 6 8 9
30562 254/300 Read block 1821 len 16384
30563 255/300 Read block 2589 len 24576
30564 256/300 Write block 4827 len 16384 data: 8 8 7 9
30565 257/300 Write block 915 len 36864 data: 4 4 4 4 4 3 3 3 2
30566 258/300 Write block 1924 len 36864 data: 3 4 2 3 3 3 2 2 3
30567 259/300 Write block 2344 len 28672 data: 5 3 4 5 6 6 6
30568 260/300 Write block 669 len 24576 data: 8 8 8 9 5 5
30569 261/300 Read block 3863 len 12288
30570 262/300 Write block 3669 len 24576 data: 6 6 6 8 5 8
30571 263/300 Flush
30572 264/300 Write block 2208 len 24576 data: 9 8 9 8 8 8
30573 265/300 Read block 2759 len 28672
30574 266/300 Write block 4652 len 20480 data: 3 4 4 6 7
30575 267/300 Write block 1613 len 40960 data: 5 5 4 4 7 6 4 5 5 5
30576 268/300 Write block 34 len 32768 data: 6 6 7 8 7 7 6 6
30577 269/300 Write block 2278 len 24576 data: 6 6 5 5 6 6
30578 270/300 Read block 3037 len 40960
30579 271/300 Write block 164 len 40960 data: 8 6 7 5 5 5 2 4 4 3
30580 272/300 Write block 2218 len 4096 data: 5
30581 273/300 Flush
30582 274/300 Read block 2780 len 8192
30583 275/300 Write block 2617 len 8192 data: 4 5
30584 276/300 Read block 2067 len 28672
30585 277/300 Flush
30586 278/300 Write block 364 len 36864 data: 6 7 9 8 7 6 7 5 5
30587 279/300 Write block 1096 len 40960 data: 7 7 9 7 8 8 5 5 6 5
30588 280/300 Flush
30589 281/300 Write block 1277 len 28672 data: 7 6 5 6 6 6 4
30590 282/300 Write block 1621 len 40960 data: 6 6 6 6 5 6 6 6 8 8
30591 283/300 Write block 4660 len 8192 data: 8 7
30592 284/300 Flush
30593 285/300 Read block 90 len 20480
30594 286/300 Read block 1764 len 24576
30595 287/300 Write block 1644 len 4096 data: 4
30596 288/300 Write block 3353 len 12288 data: 5 5 4
30597 289/300 Write block 1384 len 16384 data: 5 5 5 5
30598 290/300 Read block 4668 len 12288
30599 291/300 Write block 492 len 4096 data: 4
30600 292/300 Read block 4581 len 4096
30601 293/300 Write block 4962 len 40960 data: 7 7 8 8 8 8 8 8 8 8
30602 294/300 Read block 426 len 16384
30603 295/300 Read block 2034 len 12288
30604 296/300 Read block 453 len 40960
30605 297/300 Write block 3138 len 4096 data: 9
30606 298/300 Read block 523 len 28672
30607 299/300 Read block 833 len 12288
30608 300/300 Write block 199 len 40960 data: 3 3 5 5 4 5 4 4 4 5
30609 301/300 Write block 720 len 20480 data: 5 5 6 6 5
30610 Main task has work to do, go find it
30611 got dsc Start(2)
30612 start 2
30613 Nov 28 04:56:45.340 INFO request completed, latency_us: 243, response_code: 204, uri: /start/cid/2, method: POST, req_id: 10ea58dd-ee7b-4ec1-81ca-e5794d07174b, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
30614 Replay: started 2, returned:Ok(())
30615 ----------------------------------------------------------------
30616 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
30617 States: Active Active Offline
30618 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
30619 8547 Acked Write 2 Done Done Sent false
30620 8548 Acked Write 9 Done Done Sent false
30621 8549 Acked Write 10 Done Done Sent false
30622 8550 Acked Flush 0 Done Done Sent false
30623 8551 Acked Write 10 Done Done Sent false
30624 8552 Acked Read 4 Done Done Sent false
30625 8553 Acked Read 2 Done Done Sent false
30626 8554 Acked Read 2 Done Done Sent false
30627 8555 Acked Read 10 Done Done Sent false
30628 8556 Acked Write 3 Done Done Sent false
30629 8557 Acked Read 4 Done Done Sent false
30630 8558 Acked Write 5 Done Done Sent false
30631 8559 Acked Read 9 Done Done Sent false
30632 8560 Acked Read 5 Done Done Sent false
30633 8561 Acked Write 2 Done Done Sent false
30634 8562 Acked Flush 0 Done Done Sent false
30635 8563 Acked Write 6 Done Done Sent false
30636 8564 Acked Flush 0 Done Done Sent false
30637 8565 Acked Write 1 Done Done Sent false
30638 8566 Acked Write 6 Done Done Sent false
30639 8567 Acked Read 7 Done Done Sent false
30640 8568 Acked Flush 0 Done Done Sent false
30641 8569 Acked Flush 0 Done Done Sent false
30642 8570 Acked Read 4 Done Done Sent false
30643 8571 Acked Flush 0 Done Done Sent false
30644 8572 Acked Write 10 Done Done Sent false
30645 8573 Acked Read 7 Done Done Sent false
30646 8574 Acked Write 2 Done Done Sent false
30647 8575 Acked Write 6 Done Done Sent false
30648 8576 Acked Write 1 Done Done Sent false
30649 8577 Acked Write 4 Done Done Sent false
30650 8578 Acked Read 2 Done Done Sent false
30651 8579 Acked Read 9 Done Done Sent false
30652 8580 Acked Write 5 Done Done Sent false
30653 8581 Acked Write 5 Done Done Sent false
30654 8582 Acked Write 4 Done Done Sent false
30655 8583 Acked Read 1 Done Done Sent false
30656 8584 Acked Read 10 Done Done Sent false
30657 8585 Acked Read 2 Done Done Sent false
30658 8586 Acked Write 8 Done Done Sent false
30659 8587 Acked Write 6 Done Done Sent false
30660 8588 Acked Read 9 Done Done Sent false
30661 8589 Acked Read 4 Done Done Sent false
30662 8590 Acked Read 3 Done Done Sent false
30663 8591 Acked Write 1 Done Done Sent false
30664 8592 Acked Read 2 Done Done Sent false
30665 8593 Acked Write 4 Done Done Sent false
30666 8594 Acked Write 1 Done Done Sent false
30667 8595 Acked Write 10 Done Done Sent false
30668 8596 Acked Write 2 Done Done Sent false
30669 8597 Acked Write 8 Done Done Sent false
30670 8598 Acked Read 9 Done Done Sent false
30671 8599 Acked Read 2 Done Done Sent false
30672 8600 Acked Write 7 Done Done Sent false
30673 8601 Acked Write 6 Done Done Sent false
30674 8602 Acked Write 8 Done Done Sent false
30675 8603 Acked Read 5 Done Done Sent false
30676 8604 Acked Write 7 Done Done Sent false
30677 8605 Acked Read 5 Done Done Sent false
30678 8606 Acked Write 9 Done Done Sent false
30679 8607 Acked Write 7 Done Done Sent false
30680 8608 Acked Write 5 Done Done Sent false
30681 8609 Acked Read 10 Done Done Sent false
30682 8610 Acked Write 1 Done Done Sent false
30683 8611 Acked Write 1 Done Done Sent false
30684 8612 Acked Read 8 Done Done Sent false
30685 8613 Acked Write 10 Done Done Sent false
30686 8614 Acked Write 9 Done Done Sent false
30687 8615 Acked Write 8 Done Done Sent false
30688 8616 Acked Write 2 Done Done Sent false
30689 8617 Acked Read 7 Done Done Sent false
30690 8618 Acked Flush 0 Done Done Sent false
30691 8619 Acked Read 6 Done Done Sent false
30692 8620 Acked Write 3 Done Done Sent false
30693 8621 Acked Write 6 Done Done Sent false
30694 8622 Acked Read 10 Done Done Sent false
30695 8623 Acked Read 1 Done Done Sent false
30696 8624 Acked Read 1 Done Done Sent false
30697 8625 Acked Read 5 Done Done Sent false
30698 8626 Acked Read 6 Done Done Sent false
30699 8627 Acked Read 7 Done Done Sent false
30700 8628 Acked Read 3 Done Done Sent false
30701 8629 Acked Read 3 Done Done Sent false
30702 8630 Acked Read 5 Done Done Sent false
30703 8631 Acked Read 8 Done Done Sent false
30704 8632 Acked Read 1 Done Done Sent false
30705 8633 Acked Read 5 Done Done Sent false
30706 8634 Acked Read 5 Done Done Sent false
30707 8635 Acked Write 3 Done Done Sent false
30708 8636 Acked Flush 0 Done Done Sent false
30709 8637 Acked Read 2 Done Done Sent false
30710 8638 Acked Read 8 Done Done Sent false
30711 8639 Acked Read 5 Done Done Sent false
30712 8640 Acked Read 9 Done Done Sent false
30713 8641 Acked Read 4 Done Done Sent false
30714 8642 Acked Read 9 Done Done Sent false
30715 8643 Acked Read 5 Done Done Sent false
30716 8644 Acked Write 9 Done Done Sent false
30717 8645 Acked Write 2 Done Done Sent false
30718 8646 Acked Read 7 Done Done Sent false
30719 8647 Acked Write 4 Done Done Sent false
30720 8648 Acked Read 8 Done Done Sent false
30721 8649 Acked Read 5 Done Done Sent false
30722 8650 Acked Read 2 Done Done Sent false
30723 8651 Acked Read 1 Done Done Sent false
30724 8652 Acked Write 2 Done Done Sent false
30725 8653 Acked Flush 0 Done Done Sent false
30726 8654 Acked Read 4 Done Done Sent false
30727 8655 Acked Write 3 Done Done Sent false
30728 8656 Acked Read 2 Done Done Sent false
30729 8657 Acked Read 10 Done Done Sent false
30730 8658 Acked Read 5 Done Done Sent false
30731 8659 Acked Read 2 Done Done Sent false
30732 8660 Acked Write 2 Done Done Sent false
30733 8661 Acked Write 5 Done Done Sent false
30734 8662 Acked Read 1 Done Done Sent false
30735 8663 Acked Write 8 Done Done Sent false
30736 8664 Acked Read 10 Done Done Sent false
30737 8665 Acked Read 2 Done Done Sent false
30738 8666 Acked Write 6 Done Done Sent false
30739 8667 Acked Write 9 Done Done Sent false
30740 8668 Acked Read 9 Done Done Sent false
30741 8669 Acked Write 1 Done Done Sent false
30742 8670 Acked Write 6 Done Done Sent false
30743 8671 Acked Write 6 Done Done Sent false
30744 8672 Acked Write 3 Done Done Sent false
30745 8673 Acked Write 8 Done Done Sent false
30746 8674 Acked Read 9 Done Done Sent false
30747 8675 Acked Flush 0 Done Done Sent false
30748 8676 Acked Write 6 Done Done Sent false
30749 8677 Acked Read 9 Done Done Sent false
30750 8678 Acked Read 3 Done Done Sent false
30751 8679 Acked Read 6 Done Done Sent false
30752 8680 Acked Write 9 Done Done Sent false
30753 8681 Acked Write 6 Done Done Sent false
30754 8682 Acked Write 9 Done Done Sent false
30755 8683 Acked Write 3 Done Done Sent false
30756 8684 Acked Read 2 Done Done Sent false
30757 8685 Acked Write 5 Done Done Sent false
30758 8686 Acked Read 3 Done Done Sent false
30759 8687 Acked Write 10 Done Done Sent false
30760 8688 Acked Read 6 Done Done Sent false
30761 8689 Acked Read 9 Done Done Sent false
30762 8690 Acked Read 1 Done Done Sent false
30763 8691 Acked Read 8 Done Done Sent false
30764 8692 Acked Read 7 Done Done Sent false
30765 8693 Acked Write 9 Done Done Sent false
30766 8694 Acked Write 5 Done Done Sent false
30767 8695 Acked Write 10 Done Done Sent false
30768 8696 Acked Flush 0 Done Done Sent false
30769 8697 Acked Write 7 Done Done Sent false
30770 8698 Acked Read 6 Done Done Sent false
30771 8699 Acked Write 3 Done Done Sent false
30772 8700 Acked Read 7 Done Done Sent false
30773 8701 Acked Write 10 Done Done Sent false
30774 8702 Acked Read 10 Done Done Sent false
30775 8703 Acked Read 4 Done Done Sent false
30776 8704 Acked Write 7 Done Done Sent false
30777 8705 Acked Flush 0 Done Done Sent false
30778 8706 Acked Read 6 Done Done Sent false
30779 8707 Acked Write 1 Done Done Sent false
30780 8708 Acked Write 5 Done Done Sent false
30781 8709 Acked Write 3 Done Done Sent false
30782 8710 Acked Flush 0 Done Done Sent false
30783 8711 Acked Read 4 Done Done Sent false
30784 8712 Acked Write 3 Done Done Sent false
30785 8713 Acked Write 8 Done Done Sent false
30786 8714 Acked Read 6 Done Done Sent false
30787 8715 Acked Read 4 Done Done Sent false
30788 8716 Acked Flush 0 Done Done Sent false
30789 8717 Acked Read 5 Done Done Sent false
30790 8718 Acked Read 4 Done Done Sent false
30791 8719 Acked Write 3 Done Done Sent false
30792 8720 Acked Read 7 Done Done Sent false
30793 8721 Acked Read 10 Done Done Sent false
30794 8722 Acked Write 2 Done Done Sent false
30795 8723 Acked Flush 0 Done Done Sent false
30796 8724 Acked Read 3 Done Done Sent false
30797 8725 Acked Read 2 Done Done Sent false
30798 8726 Acked Read 6 Done Done Sent false
30799 8727 Acked Read 3 Done Done Sent false
30800 8728 Acked Write 2 Done Done Sent false
30801 8729 Acked Write 10 Done Done Sent false
30802 8730 Acked Read 1 Done Done Sent false
30803 8731 Acked Read 6 Done Done Sent false
30804 8732 Acked Read 8 Done Done Sent false
30805 8733 Acked Read 10 Done Done Sent false
30806 8734 Acked Write 10 Done Done Sent false
30807 8735 Acked Read 1 Done Done Sent false
30808 8736 Acked Write 1 Done Done Sent false
30809 8737 Acked Read 9 Done Done Sent false
30810 8738 Acked Read 1 Done Done Sent false
30811 8739 Acked Write 5 Done Done Sent false
30812 8740 Acked Write 3 Done Done Sent false
30813 8741 Acked Write 4 Done Done Sent false
30814 8742 Acked Write 8 Done Done Sent false
30815 8743 Acked Flush 0 Done Done Sent false
30816 8744 Acked Write 1 Done Done Sent false
30817 8745 Acked Read 10 Done Done Sent false
30818 8746 Acked Write 3 Done Done Sent false
30819 8747 Acked Read 7 Done Done Sent false
30820 8748 Acked Write 7 Done Done Sent false
30821 8749 Acked Write 4 Done Done Sent false
30822 8750 Acked Write 7 Done Done Sent false
30823 8751 Acked Read 7 Done Done Sent false
30824 8752 Acked Write 7 Done Done Sent false
30825 8753 Acked Write 1 Done Done Sent false
30826 8754 Acked Read 10 Done Done Sent false
30827 8755 Acked Write 2 Done Done Sent false
30828 8756 Acked Write 10 Done Done Sent false
30829 8757 Acked Write 7 Done Done Sent false
30830 8758 Acked Write 1 Done Done Sent false
30831 8759 Acked Read 8 Done Done Sent false
30832 8760 Acked Read 9 Done Done Sent false
30833 8761 Acked Read 4 Done Done Sent false
30834 8762 Acked Write 10 Done Done Sent false
30835 8763 Acked Write 1 Done Done Sent false
30836 8764 Acked Flush 0 Done Done Sent false
30837 8765 Acked Read 3 Done Done Sent false
30838 8766 Acked Read 7 Done Done Sent false
30839 8767 Acked Write 4 Done Done Sent false
30840 8768 Acked Flush 0 Done Done Sent false
30841 8769 Acked Read 6 Done Done Sent false
30842 8770 Acked Read 9 Done Done Sent false
30843 8771 Acked Write 4 Done Done Sent false
30844 8772 Acked Write 7 Done Done Sent false
30845 8773 Acked Write 10 Done Done Sent false
30846 8774 Acked Write 1 Done Done Sent false
30847 8775 Acked Read 1 Done Done Sent false
30848 8776 Acked Write 3 Done Done Sent false
30849 8777 Acked Read 5 Done Done Sent false
30850 8778 Acked Flush 0 Done Done Sent false
30851 8779 Acked Read 1 Done Done Sent false
30852 8780 Acked Write 8 Done Done Sent false
30853 8781 Acked Read 1 Done Done Sent false
30854 8782 Acked Write 3 Done Done Sent false
30855 8783 Acked Read 8 Done Done Sent false
30856 8784 Acked Read 4 Done Done Sent false
30857 8785 Acked Flush 0 Done Done Sent false
30858 8786 Acked Write 7 Done Done Sent false
30859 8787 Acked Read 7 Done Done Sent false
30860 8788 Acked Read 2 Done Done Sent false
30861 8789 Acked Write 5 Done Done Sent false
30862 8790 Acked Read 3 Done Done Sent false
30863 8791 Acked Write 2 Done Done Sent false
30864 8792 Acked Write 2 Done Done Sent false
30865 8793 Acked Flush 0 Done Done Sent false
30866 8794 Acked Read 4 Done Done Sent false
30867 8795 Acked Read 7 Done Done Sent false
30868 8796 Acked Write 6 Done Done Sent false
30869 8797 Acked Write 2 Done Done Sent false
30870 8798 Acked Read 10 Done Done Sent false
30871 8799 Acked Write 3 Done Done Sent false
30872 8800 Acked Read 4 Done Done Sent false
30873 8801 Acked Read 6 Done Done Sent false
30874 8802 Acked Write 4 Done Done Sent false
30875 8803 Acked Write 9 Done Done Sent false
30876 8804 Acked Write 9 Done Done Sent false
30877 8805 Acked Write 7 Done Done Sent false
30878 8806 Acked Write 6 Done Done Sent false
30879 8807 Acked Read 3 Done Done Sent false
30880 8808 Acked Write 6 Done Done Sent false
30881 8809 Acked Flush 0 Done Done Sent false
30882 8810 Acked Write 6 Done Done Sent false
30883 8811 Acked Read 7 Done Done Sent false
30884 8812 Acked Write 5 Done Done Sent false
30885 8813 Acked Write 10 Done Done Sent false
30886 8814 Acked Write 8 Done Done Sent false
30887 8815 Acked Write 6 Done Done Sent false
30888 8816 Acked Read 10 Done Done Sent false
30889 8817 Acked Write 10 Done Done Sent false
30890 8818 Acked Write 1 Done Done Sent false
30891 8819 Acked Flush 0 Done Done Sent false
30892 8820 Acked Read 2 Done Done Sent false
30893 8821 Acked Write 2 Done Done Sent false
30894 8822 Acked Read 7 Done Done Sent false
30895 8823 Acked Flush 0 Done Done Sent false
30896 8824 Acked Write 9 Done Done Sent false
30897 8825 Acked Write 10 Done Done Sent false
30898 8826 Acked Flush 0 Done Done Sent false
30899 8827 Acked Write 7 Done Done Sent false
30900 8828 Acked Write 10 Done Done Sent false
30901 8829 Acked Write 2 Done Done Sent false
30902 8830 Acked Flush 0 Done Done Sent false
30903 8831 Acked Read 5 Done Done Sent false
30904 8832 Acked Read 6 Done Done Sent false
30905 8833 Acked Write 1 Done Done Sent false
30906 8834 Acked Write 3 Done Done Sent false
30907 8835 Acked Write 4 Done Done Sent false
30908 8836 Acked Read 3 Done Done Sent false
30909 8837 Acked Write 1 Done Done Sent false
30910 8838 Acked Read 1 Done Done Sent false
30911 8839 Acked Write 10 Done Done Sent false
30912 8840 Acked Read 4 Done Done Sent false
30913 8841 Acked Read 3 Done Done Sent false
30914 8842 Acked Read 10 Done Done Sent false
30915 8843 Acked Write 1 Done Done Sent false
30916 8844 Acked Read 7 Done Done Sent false
30917 8845 Acked Read 3 Done Done Sent false
30918 8846 Acked Write 10 Done Done Sent false
30919 8847 Acked Write 5 Sent Sent Sent false
30920 STATES DS:0 DS:1 DS:2 TOTAL
30921 Sent 1 1 301 303
30922 Done 300 300 0 600
30923 Skipped 0 0 0 0
30924 Error 0 0 0 0
30925 Last Flush: 8830 8830 8546
30926 Downstairs last five completed: 8546 8545 8544 8543 8542
30927 Upstairs last five completed: 8847 8846 8845 8844 8843
30928 CLIENT: Up:0 ds:301 act:2
30929 [8830] got start action
30930 [8830] I am going to restart
30931 Starting downstairs at port 8830
30932 Make output file at "/tmp/dsc/downstairs-8830.txt"
30933 [8830][2] reports Starting
30934 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1228)
30935 [8830][2] reports Running
30936 ----------------------------------------------------------------
30937 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
30938 States: Active Active Offline
30939 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
30940 8547 Acked Write 2 Done Done Sent false
30941 8548 Acked Write 9 Done Done Sent false
30942 8549 Acked Write 10 Done Done Sent false
30943 8550 Acked Flush 0 Done Done Sent false
30944 8551 Acked Write 10 Done Done Sent false
30945 8552 Acked Read 4 Done Done Sent false
30946 8553 Acked Read 2 Done Done Sent false
30947 8554 Acked Read 2 Done Done Sent false
30948 8555 Acked Read 10 Done Done Sent false
30949 8556 Acked Write 3 Done Done Sent false
30950 8557 Acked Read 4 Done Done Sent false
30951 8558 Acked Write 5 Done Done Sent false
30952 8559 Acked Read 9 Done Done Sent false
30953 8560 Acked Read 5 Done Done Sent false
30954 8561 Acked Write 2 Done Done Sent false
30955 8562 Acked Flush 0 Done Done Sent false
30956 8563 Acked Write 6 Done Done Sent false
30957 8564 Acked Flush 0 Done Done Sent false
30958 8565 Acked Write 1 Done Done Sent false
30959 8566 Acked Write 6 Done Done Sent false
30960 8567 Acked Read 7 Done Done Sent false
30961 8568 Acked Flush 0 Done Done Sent false
30962 8569 Acked Flush 0 Done Done Sent false
30963 8570 Acked Read 4 Done Done Sent false
30964 8571 Acked Flush 0 Done Done Sent false
30965 8572 Acked Write 10 Done Done Sent false
30966 8573 Acked Read 7 Done Done Sent false
30967 8574 Acked Write 2 Done Done Sent false
30968 8575 Acked Write 6 Done Done Sent false
30969 8576 Acked Write 1 Done Done Sent false
30970 8577 Acked Write 4 Done Done Sent false
30971 8578 Acked Read 2 Done Done Sent false
30972 8579 Acked Read 9 Done Done Sent false
30973 8580 Acked Write 5 Done Done Sent false
30974 8581 Acked Write 5 Done Done Sent false
30975 8582 Acked Write 4 Done Done Sent false
30976 8583 Acked Read 1 Done Done Sent false
30977 8584 Acked Read 10 Done Done Sent false
30978 8585 Acked Read 2 Done Done Sent false
30979 8586 Acked Write 8 Done Done Sent false
30980 8587 Acked Write 6 Done Done Sent false
30981 8588 Acked Read 9 Done Done Sent false
30982 8589 Acked Read 4 Done Done Sent false
30983 8590 Acked Read 3 Done Done Sent false
30984 8591 Acked Write 1 Done Done Sent false
30985 8592 Acked Read 2 Done Done Sent false
30986 8593 Acked Write 4 Done Done Sent false
30987 8594 Acked Write 1 Done Done Sent false
30988 8595 Acked Write 10 Done Done Sent false
30989 8596 Acked Write 2 Done Done Sent false
30990 8597 Acked Write 8 Done Done Sent false
30991 8598 Acked Read 9 Done Done Sent false
30992 8599 Acked Read 2 Done Done Sent false
30993 8600 Acked Write 7 Done Done Sent false
30994 8601 Acked Write 6 Done Done Sent false
30995 8602 Acked Write 8 Done Done Sent false
30996 8603 Acked Read 5 Done Done Sent false
30997 8604 Acked Write 7 Done Done Sent false
30998 8605 Acked Read 5 Done Done Sent false
30999 8606 Acked Write 9 Done Done Sent false
31000 8607 Acked Write 7 Done Done Sent false
31001 8608 Acked Write 5 Done Done Sent false
31002 8609 Acked Read 10 Done Done Sent false
31003 8610 Acked Write 1 Done Done Sent false
31004 8611 Acked Write 1 Done Done Sent false
31005 8612 Acked Read 8 Done Done Sent false
31006 8613 Acked Write 10 Done Done Sent false
31007 8614 Acked Write 9 Done Done Sent false
31008 8615 Acked Write 8 Done Done Sent false
31009 8616 Acked Write 2 Done Done Sent false
31010 8617 Acked Read 7 Done Done Sent false
31011 8618 Acked Flush 0 Done Done Sent false
31012 8619 Acked Read 6 Done Done Sent false
31013 8620 Acked Write 3 Done Done Sent false
31014 8621 Acked Write 6 Done Done Sent false
31015 8622 Acked Read 10 Done Done Sent false
31016 8623 Acked Read 1 Done Done Sent false
31017 8624 Acked Read 1 Done Done Sent false
31018 8625 Acked Read 5 Done Done Sent false
31019 8626 Acked Read 6 Done Done Sent false
31020 8627 Acked Read 7 Done Done Sent false
31021 8628 Acked Read 3 Done Done Sent false
31022 8629 Acked Read 3 Done Done Sent false
31023 8630 Acked Read 5 Done Done Sent false
31024 8631 Acked Read 8 Done Done Sent false
31025 8632 Acked Read 1 Done Done Sent false
31026 8633 Acked Read 5 Done Done Sent false
31027 8634 Acked Read 5 Done Done Sent false
31028 8635 Acked Write 3 Done Done Sent false
31029 8636 Acked Flush 0 Done Done Sent false
31030 8637 Acked Read 2 Done Done Sent false
31031 8638 Acked Read 8 Done Done Sent false
31032 8639 Acked Read 5 Done Done Sent false
31033 8640 Acked Read 9 Done Done Sent false
31034 8641 Acked Read 4 Done Done Sent false
31035 8642 Acked Read 9 Done Done Sent false
31036 8643 Acked Read 5 Done Done Sent false
31037 8644 Acked Write 9 Done Done Sent false
31038 8645 Acked Write 2 Done Done Sent false
31039 8646 Acked Read 7 Done Done Sent false
31040 8647 Acked Write 4 Done Done Sent false
31041 8648 Acked Read 8 Done Done Sent false
31042 8649 Acked Read 5 Done Done Sent false
31043 8650 Acked Read 2 Done Done Sent false
31044 8651 Acked Read 1 Done Done Sent false
31045 8652 Acked Write 2 Done Done Sent false
31046 8653 Acked Flush 0 Done Done Sent false
31047 8654 Acked Read 4 Done Done Sent false
31048 8655 Acked Write 3 Done Done Sent false
31049 8656 Acked Read 2 Done Done Sent false
31050 8657 Acked Read 10 Done Done Sent false
31051 8658 Acked Read 5 Done Done Sent false
31052 8659 Acked Read 2 Done Done Sent false
31053 8660 Acked Write 2 Done Done Sent false
31054 8661 Acked Write 5 Done Done Sent false
31055 8662 Acked Read 1 Done Done Sent false
31056 8663 Acked Write 8 Done Done Sent false
31057 8664 Acked Read 10 Done Done Sent false
31058 8665 Acked Read 2 Done Done Sent false
31059 8666 Acked Write 6 Done Done Sent false
31060 8667 Acked Write 9 Done Done Sent false
31061 8668 Acked Read 9 Done Done Sent false
31062 8669 Acked Write 1 Done Done Sent false
31063 8670 Acked Write 6 Done Done Sent false
31064 8671 Acked Write 6 Done Done Sent false
31065 8672 Acked Write 3 Done Done Sent false
31066 8673 Acked Write 8 Done Done Sent false
31067 8674 Acked Read 9 Done Done Sent false
31068 8675 Acked Flush 0 Done Done Sent false
31069 8676 Acked Write 6 Done Done Sent false
31070 8677 Acked Read 9 Done Done Sent false
31071 8678 Acked Read 3 Done Done Sent false
31072 8679 Acked Read 6 Done Done Sent false
31073 8680 Acked Write 9 Done Done Sent false
31074 8681 Acked Write 6 Done Done Sent false
31075 8682 Acked Write 9 Done Done Sent false
31076 8683 Acked Write 3 Done Done Sent false
31077 8684 Acked Read 2 Done Done Sent false
31078 8685 Acked Write 5 Done Done Sent false
31079 8686 Acked Read 3 Done Done Sent false
31080 8687 Acked Write 10 Done Done Sent false
31081 8688 Acked Read 6 Done Done Sent false
31082 8689 Acked Read 9 Done Done Sent false
31083 8690 Acked Read 1 Done Done Sent false
31084 8691 Acked Read 8 Done Done Sent false
31085 8692 Acked Read 7 Done Done Sent false
31086 8693 Acked Write 9 Done Done Sent false
31087 8694 Acked Write 5 Done Done Sent false
31088 8695 Acked Write 10 Done Done Sent false
31089 8696 Acked Flush 0 Done Done Sent false
31090 8697 Acked Write 7 Done Done Sent false
31091 8698 Acked Read 6 Done Done Sent false
31092 8699 Acked Write 3 Done Done Sent false
31093 8700 Acked Read 7 Done Done Sent false
31094 8701 Acked Write 10 Done Done Sent false
31095 8702 Acked Read 10 Done Done Sent false
31096 8703 Acked Read 4 Done Done Sent false
31097 8704 Acked Write 7 Done Done Sent false
31098 8705 Acked Flush 0 Done Done Sent false
31099 8706 Acked Read 6 Done Done Sent false
31100 8707 Acked Write 1 Done Done Sent false
31101 8708 Acked Write 5 Done Done Sent false
31102 8709 Acked Write 3 Done Done Sent false
31103 8710 Acked Flush 0 Done Done Sent false
31104 8711 Acked Read 4 Done Done Sent false
31105 8712 Acked Write 3 Done Done Sent false
31106 8713 Acked Write 8 Done Done Sent false
31107 8714 Acked Read 6 Done Done Sent false
31108 8715 Acked Read 4 Done Done Sent false
31109 8716 Acked Flush 0 Done Done Sent false
31110 8717 Acked Read 5 Done Done Sent false
31111 8718 Acked Read 4 Done Done Sent false
31112 8719 Acked Write 3 Done Done Sent false
31113 8720 Acked Read 7 Done Done Sent false
31114 8721 Acked Read 10 Done Done Sent false
31115 8722 Acked Write 2 Done Done Sent false
31116 8723 Acked Flush 0 Done Done Sent false
31117 8724 Acked Read 3 Done Done Sent false
31118 8725 Acked Read 2 Done Done Sent false
31119 8726 Acked Read 6 Done Done Sent false
31120 8727 Acked Read 3 Done Done Sent false
31121 8728 Acked Write 2 Done Done Sent false
31122 8729 Acked Write 10 Done Done Sent false
31123 8730 Acked Read 1 Done Done Sent false
31124 8731 Acked Read 6 Done Done Sent false
31125 8732 Acked Read 8 Done Done Sent false
31126 8733 Acked Read 10 Done Done Sent false
31127 8734 Acked Write 10 Done Done Sent false
31128 8735 Acked Read 1 Done Done Sent false
31129 8736 Acked Write 1 Done Done Sent false
31130 8737 Acked Read 9 Done Done Sent false
31131 8738 Acked Read 1 Done Done Sent false
31132 8739 Acked Write 5 Done Done Sent false
31133 8740 Acked Write 3 Done Done Sent false
31134 8741 Acked Write 4 Done Done Sent false
31135 8742 Acked Write 8 Done Done Sent false
31136 8743 Acked Flush 0 Done Done Sent false
31137 8744 Acked Write 1 Done Done Sent false
31138 8745 Acked Read 10 Done Done Sent false
31139 8746 Acked Write 3 Done Done Sent false
31140 8747 Acked Read 7 Done Done Sent false
31141 8748 Acked Write 7 Done Done Sent false
31142 8749 Acked Write 4 Done Done Sent false
31143 8750 Acked Write 7 Done Done Sent false
31144 8751 Acked Read 7 Done Done Sent false
31145 8752 Acked Write 7 Done Done Sent false
31146 8753 Acked Write 1 Done Done Sent false
31147 8754 Acked Read 10 Done Done Sent false
31148 8755 Acked Write 2 Done Done Sent false
31149 8756 Acked Write 10 Done Done Sent false
31150 8757 Acked Write 7 Done Done Sent false
31151 8758 Acked Write 1 Done Done Sent false
31152 8759 Acked Read 8 Done Done Sent false
31153 8760 Acked Read 9 Done Done Sent false
31154 8761 Acked Read 4 Done Done Sent false
31155 8762 Acked Write 10 Done Done Sent false
31156 8763 Acked Write 1 Done Done Sent false
31157 8764 Acked Flush 0 Done Done Sent false
31158 8765 Acked Read 3 Done Done Sent false
31159 8766 Acked Read 7 Done Done Sent false
31160 8767 Acked Write 4 Done Done Sent false
31161 8768 Acked Flush 0 Done Done Sent false
31162 8769 Acked Read 6 Done Done Sent false
31163 8770 Acked Read 9 Done Done Sent false
31164 8771 Acked Write 4 Done Done Sent false
31165 8772 Acked Write 7 Done Done Sent false
31166 8773 Acked Write 10 Done Done Sent false
31167 8774 Acked Write 1 Done Done Sent false
31168 8775 Acked Read 1 Done Done Sent false
31169 8776 Acked Write 3 Done Done Sent false
31170 8777 Acked Read 5 Done Done Sent false
31171 8778 Acked Flush 0 Done Done Sent false
31172 8779 Acked Read 1 Done Done Sent false
31173 8780 Acked Write 8 Done Done Sent false
31174 8781 Acked Read 1 Done Done Sent false
31175 8782 Acked Write 3 Done Done Sent false
31176 8783 Acked Read 8 Done Done Sent false
31177 8784 Acked Read 4 Done Done Sent false
31178 8785 Acked Flush 0 Done Done Sent false
31179 8786 Acked Write 7 Done Done Sent false
31180 8787 Acked Read 7 Done Done Sent false
31181 8788 Acked Read 2 Done Done Sent false
31182 8789 Acked Write 5 Done Done Sent false
31183 8790 Acked Read 3 Done Done Sent false
31184 8791 Acked Write 2 Done Done Sent false
31185 8792 Acked Write 2 Done Done Sent false
31186 8793 Acked Flush 0 Done Done Sent false
31187 8794 Acked Read 4 Done Done Sent false
31188 8795 Acked Read 7 Done Done Sent false
31189 8796 Acked Write 6 Done Done Sent false
31190 8797 Acked Write 2 Done Done Sent false
31191 8798 Acked Read 10 Done Done Sent false
31192 8799 Acked Write 3 Done Done Sent false
31193 8800 Acked Read 4 Done Done Sent false
31194 8801 Acked Read 6 Done Done Sent false
31195 8802 Acked Write 4 Done Done Sent false
31196 8803 Acked Write 9 Done Done Sent false
31197 8804 Acked Write 9 Done Done Sent false
31198 8805 Acked Write 7 Done Done Sent false
31199 8806 Acked Write 6 Done Done Sent false
31200 8807 Acked Read 3 Done Done Sent false
31201 8808 Acked Write 6 Done Done Sent false
31202 8809 Acked Flush 0 Done Done Sent false
31203 8810 Acked Write 6 Done Done Sent false
31204 8811 Acked Read 7 Done Done Sent false
31205 8812 Acked Write 5 Done Done Sent false
31206 8813 Acked Write 10 Done Done Sent false
31207 8814 Acked Write 8 Done Done Sent false
31208 8815 Acked Write 6 Done Done Sent false
31209 8816 Acked Read 10 Done Done Sent false
31210 8817 Acked Write 10 Done Done Sent false
31211 8818 Acked Write 1 Done Done Sent false
31212 8819 Acked Flush 0 Done Done Sent false
31213 8820 Acked Read 2 Done Done Sent false
31214 8821 Acked Write 2 Done Done Sent false
31215 8822 Acked Read 7 Done Done Sent false
31216 8823 Acked Flush 0 Done Done Sent false
31217 8824 Acked Write 9 Done Done Sent false
31218 8825 Acked Write 10 Done Done Sent false
31219 8826 Acked Flush 0 Done Done Sent false
31220 8827 Acked Write 7 Done Done Sent false
31221 8828 Acked Write 10 Done Done Sent false
31222 8829 Acked Write 2 Done Done Sent false
31223 8830 Acked Flush 0 Done Done Sent false
31224 8831 Acked Read 5 Done Done Sent false
31225 8832 Acked Read 6 Done Done Sent false
31226 8833 Acked Write 1 Done Done Sent false
31227 8834 Acked Write 3 Done Done Sent false
31228 8835 Acked Write 4 Done Done Sent false
31229 8836 Acked Read 3 Done Done Sent false
31230 8837 Acked Write 1 Done Done Sent false
31231 8838 Acked Read 1 Done Done Sent false
31232 8839 Acked Write 10 Done Done Sent false
31233 8840 Acked Read 4 Done Done Sent false
31234 8841 Acked Read 3 Done Done Sent false
31235 8842 Acked Read 10 Done Done Sent false
31236 8843 Acked Write 1 Done Done Sent false
31237 8844 Acked Read 7 Done Done Sent false
31238 8845 Acked Read 3 Done Done Sent false
31239 8846 Acked Write 10 Done Done Sent false
31240 8847 Acked Write 5 Done Done Sent false
31241 8848 Acked Flush 0 Done Done Sent false
31242 STATES DS:0 DS:1 DS:2 TOTAL
31243 Sent 0 0 302 302
31244 Done 302 302 0 604
31245 Skipped 0 0 0 0
31246 Error 0 0 0 0
31247 Last Flush: 8848 8848 8546
31248 Downstairs last five completed: 8546 8545 8544 8543 8542
31249 Upstairs last five completed: 8848 8847 8846 8845 8844
31250 CLIENT: Up:0 ds:302 act:2
312512024-11-28T04:56:51.117ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
312522024-11-28T04:56:51.117ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:58064) = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
312532024-11-28T04:56:51.118ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
312542024-11-28T04:56:51.119ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
312552024-11-28T04:56:51.119ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
312562024-11-28T04:56:51.119ZINFOcrucible: send last flush ID to this DS: 8546 = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
312572024-11-28T04:56:51.119ZINFOcrucible: Replied this last flush ID: 8546 = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
312582024-11-28T04:56:51.119ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
312592024-11-28T04:56:51.119ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
312602024-11-28T04:56:51.119ZINFOcrucible: [2] client re-new 302 jobs since flush 8546 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
312612024-11-28T04:56:51.119ZINFOcrucible: [2] Replayed 302 jobs since flush: 8546 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
31262 ----------------------------------------------------------------
31263 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
31264 Downstairs last five completed: 8848 8847 8846 8845 8844
31265 Upstairs last five completed: 8848 8847 8846 8845 8844
31266 CLIENT: Up:0 ds:0 act:3
31267 Replay: All jobs finished
31268 Main task has work to do, go find it
31269 got dsc Stop(2)
31270 stop 2
31271 [8830] Got stop action so:false kr:false
31272 Nov 28 04:56:53.346 INFO request completed, latency_us: 345, response_code: 204, uri: /stop/cid/2, method: POST, req_id: d7dcaaaf-47bd-471c-9867-56847bbb836c, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
312732024-11-28T04:56:53.346ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
312742024-11-28T04:56:53.346ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
312752024-11-28T04:56:53.346ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
312762024-11-28T04:56:53.347ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
31277 Nov 28 04:56:53.347 INFO request completed, latency_us: 168, response_code: 200, uri: {"msg":"[2] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-28T04:56:53.347037574Z","hostname":"w-01JDRHKEEFJ1228TFPSBEGXWHY","pid":1202,"client":"2","":"downstairs","session_id":"4a2415b1-6594-436f-82e5-b8cf9a74eeaf"}
31278 /state/cid/2, method: GET, req_id: a945b790-a638-4dde-a9d0-5c472f9d5666, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
312792024-11-28T04:56:53.347ZINFOcrucible: client task is exiting = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
31280 [8830] Exited with: ExitStatus(unix_wait_status(9))
31281 [8830][2] reports Exit
31282 Nov 28 04:56:57.349 INFO request completed, latency_us: 242, response_code: 200, uri: /state/cid/2, method: GET, req_id: 62d7512a-4065-4707-b90e-2564a929c4a3, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
31283 001/300 Read block 4097 len 16384
31284 002/300 Write block 3848 len 40960 data: 4 6 6 5 4 5 6 7 6 6
31285 003/300 Read block 1153 len 16384
31286 004/300 Write block 4882 len 32768 data: 5 6 5 5 5 5 5 6
31287 005/300 Write block 3141 len 12288 data: 8 4 4
31288 006/300 Write block 4770 len 36864 data: 6 6 7 8 8 8 7 6 6
31289 007/300 Flush
31290 008/300 Read block 1547 len 20480
31291 009/300 Read block 915 len 24576
31292 010/300 Read block 1710 len 12288
31293 011/300 Read block 1455 len 16384
31294 012/300 Flush
31295 013/300 Read block 2624 len 16384
31296 014/300 Read block 2060 len 16384
31297 015/300 Write block 268 len 28672 data: 5 5 6 6 7 6 6
31298 016/300 Read block 674 len 4096
31299 017/300 Write block 1448 len 12288 data: 2 2 2
31300 018/300 Read block 2156 len 12288
31301 019/300 Write block 1452 len 4096 data: 5
31302 020/300 Write block 2885 len 4096 data: 2
31303 021/300 Read block 3152 len 36864
31304 022/300 Write block 3524 len 40960 data: 3 4 4 6 5 3 3 3 3 4
31305 023/300 Write block 4482 len 36864 data: 8 7 6 5 5 5 4 4 4
31306 024/300 Read block 2940 len 36864
31307 025/300 Write block 1590 len 4096 data: 3
31308 026/300 Write block 278 len 8192 data: 7 7
31309 027/300 Read block 4665 len 4096
31310 028/300 Write block 2028 len 12288 data: 5 5 5
31311 029/300 Read block 707 len 8192
31312 030/300 Read block 3136 len 24576
31313 031/300 Read block 4755 len 12288
31314 032/300 Read block 2348 len 12288
31315 033/300 Write block 3572 len 8192 data: 6 5
31316 034/300 Write block 662 len 8192 data: 10 10
31317 035/300 Read block 876 len 32768
31318 036/300 Write block 1162 len 8192 data: 3 3
31319 037/300 Read block 3906 len 24576
31320 038/300 Write block 3843 len 36864 data: 4 4 5 5 4 5 7 7 6
31321 039/300 Write block 4281 len 32768 data: 6 7 7 4 4 3 4 5
31322 040/300 Write block 3364 len 4096 data: 4
31323 041/300 Read block 789 len 40960
31324 042/300 Write block 3912 len 12288 data: 4 3 2
31325 043/300 Write block 1483 len 16384 data: 4 3 4 3
31326 044/300 Read block 1142 len 16384
31327 045/300 Write block 2877 len 32768 data: 8 8 8 9 8 6 4 4
31328 046/300 Write block 3146 len 8192 data: 5 4
31329 047/300 Write block 3917 len 4096 data: 4
31330 048/300 Write block 3131 len 40960 data: 4 4 6 7 7 5 7 10 9 9
31331 049/300 Write block 1641 len 4096 data: 4
31332 050/300 Flush
31333 051/300 Write block 3231 len 8192 data: 4 4
31334 052/300 Read block 4873 len 24576
31335 053/300 Read block 900 len 4096
31336 054/300 Read block 3075 len 28672
31337 055/300 Write block 4531 len 24576 data: 8 9 6 9 9 8
31338 056/300 Flush
31339 057/300 Write block 325 len 32768 data: 8 6 6 5 5 5 5 4
31340 058/300 Read block 2739 len 16384
31341 059/300 Read block 316 len 40960
31342 060/300 Write block 4117 len 32768 data: 4 5 7 7 8 8 4 4
31343 061/300 Read block 1413 len 16384
31344 062/300 Read block 4853 len 28672
31345 063/300 Read block 3182 len 40960
31346 064/300 Read block 3262 len 32768
31347 065/300 Write block 4139 len 36864 data: 5 7 8 6 5 5 4 3 2
31348 066/300 Write block 4325 len 8192 data: 4 4
31349 067/300 Write block 352 len 12288 data: 5 5 6
31350 068/300 Write block 1519 len 24576 data: 4 4 3 4 5 6
31351 069/300 Write block 3387 len 4096 data: 5
31352 070/300 Write block 2489 len 4096 data: 2
31353 071/300 Read block 2588 len 28672
31354 072/300 Read block 1080 len 12288
31355 073/300 Flush
31356 074/300 Write block 609 len 36864 data: 8 9 7 8 7 6 7 8 8
31357 075/300 Read block 4767 len 32768
31358 076/300 Flush
31359 077/300 Read block 1516 len 40960
31360 078/300 Write block 1193 len 20480 data: 5 5 6 6 7
31361 079/300 Flush
31362 080/300 Read block 4217 len 8192
31363 081/300 Read block 3919 len 20480
31364 082/300 Write block 4580 len 28672 data: 5 5 7 7 7 5 4
31365 083/300 Read block 4651 len 12288
31366 084/300 Write block 623 len 8192 data: 5 6
31367 085/300 Read block 987 len 32768
31368 086/300 Write block 3987 len 36864 data: 4 4 4 5 5 4 4 4 5
31369 087/300 Write block 701 len 28672 data: 2 3 5 5 8 7 5
31370 088/300 Write block 4665 len 36864 data: 5 4 3 3 4 4 5 6 8
31371 089/300 Read block 3515 len 40960
31372 090/300 Read block 4751 len 16384
31373 091/300 Read block 1149 len 16384
31374 092/300 Write block 2070 len 40960 data: 8 8 8 7 6 6 7 6 6 7
31375 093/300 Read block 947 len 36864
31376 094/300 Read block 1855 len 32768
31377 095/300 Write block 2553 len 36864 data: 9 9 8 9 9 7 5 5 4
31378 096/300 Read block 764 len 40960
31379 097/300 Read block 2591 len 24576
31380 098/300 Read block 3682 len 24576
31381 099/300 Write block 2920 len 40960 data: 8 8 6 8 7 5 6 5 4 6
31382 100/300 Read block 3773 len 12288
31383 101/300 Write block 2566 len 24576 data: 6 8 8 8 8 9
31384 102/300 Write block 4655 len 20480 data: 7 8 8 9 9
31385 103/300 Read block 2154 len 8192
31386 104/300 Write block 2696 len 24576 data: 7 6 5 5 5 8
31387 105/300 Read block 2944 len 8192
31388 106/300 Write block 319 len 12288 data: 6 7 8
31389 107/300 Flush
31390 108/300 Read block 1091 len 32768
31391 109/300 Write block 4167 len 8192 data: 9 8
31392 110/300 Read block 2962 len 4096
31393 111/300 Read block 1164 len 20480
31394 112/300 Read block 2504 len 24576
31395 113/300 Read block 2085 len 36864
31396 114/300 Write block 4364 len 36864 data: 11 10 9 6 6 6 4 7 5
31397 115/300 Read block 4618 len 24576
31398 116/300 Read block 4114 len 36864
31399 117/300 Write block 2315 len 24576 data: 6 5 5 5 7 8
31400 118/300 Read block 3814 len 12288
31401 119/300 Read block 3181 len 12288
31402 120/300 Read block 4969 len 40960
31403 121/300 Flush
31404 122/300 Write block 586 len 20480 data: 4 4 5 5 6
31405 123/300 Write block 3125 len 28672 data: 4 4 4 4 5 4 5
31406 124/300 Write block 4791 len 32768 data: 7 8 8 6 7 7 9 10
31407 125/300 Read block 3719 len 28672
31408 126/300 Read block 4501 len 40960
31409 127/300 Read block 1323 len 16384
31410 128/300 Read block 4101 len 28672
31411 129/300 Read block 1678 len 4096
31412 130/300 Read block 1003 len 28672
31413 131/300 Read block 3068 len 40960
31414 132/300 Read block 4908 len 12288
31415 133/300 Read block 817 len 16384
31416 134/300 Write block 388 len 28672 data: 3 3 3 3 4 5 7
31417 135/300 Read block 2345 len 36864
31418 136/300 Write block 3812 len 24576 data: 7 6 6 7 7 6
31419 137/300 Write block 950 len 16384 data: 3 3 5 5
31420 138/300 Write block 2178 len 4096 data: 6
31421 139/300 Write block 966 len 16384 data: 7 8 7 8
31422 140/300 Read block 1284 len 32768
31423 141/300 Write block 2114 len 24576 data: 7 7 7 5 5 4
31424 142/300 Write block 1028 len 36864 data: 4 3 5 5 6 7 6 4 2
31425 143/300 Write block 3695 len 24576 data: 11 10 9 7 7 6
31426 144/300 Read block 905 len 12288
31427 145/300 Write block 2562 len 4096 data: 5
31428 146/300 Read block 3073 len 16384
31429 147/300 Write block 2128 len 12288 data: 6 6 4
31430 148/300 Read block 3858 len 36864
31431 149/300 Write block 1181 len 28672 data: 8 8 7 5 4 4 4
31432 150/300 Read block 3270 len 20480
31433 151/300 Read block 1568 len 32768
31434 152/300 Read block 3476 len 24576
31435 153/300 Read block 1098 len 8192
31436 154/300 Read block 2941 len 16384
31437 155/300 Read block 2344 len 36864
31438 156/300 Read block 361 len 8192
31439 157/300 Read block 712 len 40960
31440 158/300 Write block 2527 len 32768 data: 8 7 5 5 6 7 6 5
31441 159/300 Flush
31442 160/300 Read block 987 len 24576
31443 161/300 Read block 4657 len 20480
31444 162/300 Read block 3483 len 8192
31445 163/300 Read block 3755 len 40960
31446 164/300 Read block 3075 len 16384
31447 165/300 Read block 2458 len 28672
31448 166/300 Write block 2800 len 4096 data: 3
31449 167/300 Read block 1034 len 4096
31450 168/300 Read block 3074 len 8192
31451 169/300 Write block 4029 len 4096 data: 6
31452 170/300 Write block 1656 len 32768 data: 5 3 3 3 4 3 3 3
31453 171/300 Write block 3644 len 40960 data: 7 7 5 6 6 8 9 8 6 6
31454 172/300 Read block 1188 len 32768
31455 173/300 Write block 390 len 12288 data: 4 4 5
31456 174/300 Read block 2673 len 20480
31457 175/300 Read block 1176 len 28672
31458 176/300 Read block 3030 len 32768
31459 177/300 Write block 4494 len 8192 data: 6 4
31460 178/300 Write block 3001 len 24576 data: 4 5 5 6 8 8
31461 179/300 Flush
31462 180/300 Read block 4666 len 24576
31463 181/300 Read block 654 len 12288
31464 182/300 Read block 2023 len 36864
31465 183/300 Write block 2196 len 8192 data: 7 6
31466 184/300 Read block 2399 len 20480
31467 185/300 Read block 2777 len 20480
31468 186/300 Write block 4564 len 28672 data: 6 7 7 6 6 7 5
31469 187/300 Read block 989 len 36864
31470 188/300 Read block 2848 len 32768
31471 189/300 Flush
31472 190/300 Read block 3747 len 8192
31473 191/300 Write block 1708 len 36864 data: 14 15 14 11 10 7 5 4 5
31474 192/300 Write block 1437 len 40960 data: 5 4 4 5 4 4 3 3 3 2
31475 193/300 Write block 2423 len 28672 data: 7 6 6 6 5 4 4
31476 194/300 Write block 2824 len 40960 data: 2 3 3 3 3 6 6 6 6 4
31477 195/300 Read block 2368 len 8192
31478 196/300 Write block 2177 len 36864 data: 6 7 5 5 5 5 7 6 7
31479 197/300 Read block 3262 len 40960
31480 198/300 Write block 694 len 32768 data: 5 4 4 3 2 2 2 3
31481 199/300 Read block 2010 len 8192
31482 200/300 Read block 3970 len 12288
31483 201/300 Read block 3161 len 4096
31484 202/300 Read block 567 len 40960
31485 203/300 Read block 4732 len 4096
31486 204/300 Read block 4801 len 36864
31487 205/300 Write block 622 len 28672 data: 6 6 7 8 9 7 4
31488 206/300 Read block 112 len 8192
31489 207/300 Read block 3265 len 12288
31490 208/300 Write block 4734 len 20480 data: 4 3 4 5 6
31491 209/300 Read block 2574 len 24576
31492 210/300 Flush
31493 211/300 Flush
31494 212/300 Write block 7 len 16384 data: 8 9 10 9
31495 213/300 Write block 3262 len 12288 data: 4 5 5
31496 214/300 Read block 1675 len 32768
31497 215/300 Write block 4151 len 20480 data: 6 7 7 7 7
31498 216/300 Read block 1370 len 28672
31499 217/300 Read block 4 len 24576
31500 218/300 Read block 1209 len 36864
31501 219/300 Read block 3312 len 40960
31502 220/300 Read block 1755 len 28672
31503 221/300 Write block 2884 len 8192 data: 5 3
31504 222/300 Read block 4742 len 8192
31505 223/300 Read block 1670 len 12288
31506 224/300 Read block 4267 len 20480
31507 225/300 Read block 2234 len 28672
31508 226/300 Write block 2568 len 32768 data: 9 9 9 10 7 6 6 7
31509 227/300 Read block 2269 len 28672
31510 228/300 Write block 3634 len 32768 data: 6 6 6 8 6 6 6 6
31511 229/300 Read block 155 len 24576
31512 230/300 Write block 1792 len 28672 data: 4 4 6 6 6 5 3
31513 231/300 Write block 164 len 32768 data: 9 7 8 6 6 6 3 5
31514 232/300 Write block 546 len 24576 data: 6 7 6 6 5 6
31515 233/300 Write block 2804 len 28672 data: 5 7 6 4 5 6 6
31516 234/300 Write block 3309 len 40960 data: 8 10 10 10 9 11 10 10 11 10
31517 235/300 Read block 4193 len 28672
31518 236/300 Write block 3506 len 32768 data: 7 8 9 9 9 8 7 7
31519 237/300 Read block 4944 len 36864
31520 238/300 Write block 1327 len 24576 data: 4 4 6 6 6 5
31521 239/300 Flush
31522 240/300 Read block 759 len 32768
31523 241/300 Read block 4381 len 32768
31524 242/300 Read block 4677 len 40960
31525 243/300 Write block 3943 len 16384 data: 4 4 4 4
31526 244/300 Read block 4161 len 4096
31527 245/300 Write block 173 len 4096 data: 4
31528 246/300 Read block 668 len 36864
31529 247/300 Write block 3767 len 20480 data: 8 8 7 7 6
31530 248/300 Read block 2301 len 28672
31531 249/300 Write block 2796 len 4096 data: 3
31532 250/300 Flush
31533 251/300 Write block 1521 len 36864 data: 4 5 6 7 5 5 4 4 6
31534 252/300 Write block 1546 len 32768 data: 8 7 8 11 9 8 7 5
31535 253/300 Read block 2058 len 16384
31536 254/300 Read block 1940 len 36864
31537 255/300 Read block 2915 len 40960
31538 256/300 Write block 4217 len 16384 data: 5 5 5 5
31539 257/300 Write block 3767 len 36864 data: 9 9 8 8 7 7 7 5 5
31540 258/300 Write block 2383 len 40960 data: 10 10 8 7 6 6 6 6 6 6
31541 259/300 Write block 1185 len 16384 data: 5 5 5 4
31542 260/300 Read block 229 len 4096
31543 261/300 Flush
31544 262/300 Read block 1292 len 16384
31545 263/300 Write block 1819 len 12288 data: 7 7 6
31546 264/300 Flush
31547 265/300 Read block 2727 len 8192
31548 266/300 Flush
31549 267/300 Read block 4404 len 28672
31550 268/300 Read block 3849 len 16384
31551 269/300 Write block 187 len 20480 data: 4 6 7 8 9
31552 270/300 Read block 3975 len 32768
31553 271/300 Write block 843 len 40960 data: 4 5 6 7 7 9 9 8 8 9
31554 272/300 Flush
31555 273/300 Read block 3740 len 40960
31556 274/300 Write block 3843 len 32768 data: 5 5 6 6 5 6 8 8
31557 275/300 Read block 3252 len 20480
31558 276/300 Write block 3212 len 16384 data: 5 5 5 7
31559 277/300 Read block 3420 len 16384
31560 278/300 Write block 3358 len 8192 data: 6 6
31561 279/300 Write block 4276 len 12288 data: 2 2 3
31562 280/300 Read block 2313 len 40960
31563 281/300 Flush
31564 282/300 Read block 1438 len 8192
31565 283/300 Read block 3817 len 4096
31566 284/300 Read block 1449 len 20480
31567 285/300 Write block 2077 len 36864 data: 7 7 8 6 7 7 8 9 7
31568 286/300 Read block 257 len 28672
31569 287/300 Read block 1354 len 40960
31570 288/300 Read block 4139 len 20480
31571 289/300 Read block 1157 len 36864
31572 290/300 Flush
31573 291/300 Write block 1686 len 28672 data: 5 5 5 5 4 3 2
31574 292/300 Write block 4510 len 20480 data: 4 3 3 4 4
31575 293/300 Read block 4846 len 4096
31576 294/300 Read block 4001 len 12288
31577 295/300 Read block 557 len 8192
31578 296/300 Read block 3905 len 8192
31579 297/300 Write block 3565 len 28672 data: 6 4 5 5 5 4 4
31580 298/300 Write block 636 len 8192 data: 9 8
31581 299/300 Flush
31582 300/300 Read block 4327 len 28672
31583 301/300 Write block 2582 len 20480 data: 4 4 3 4 4
31584 Main task has work to do, go find it
31585 got dsc Start(2)
31586 start 2
31587 Nov 28 04:56:57.593 INFO request completed, latency_us: 228, response_code: 204, uri: /start/cid/2, method: POST, req_id: e89e11ea-35c9-4ab2-bce5-229389f0ffc8, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
31588 Replay: started 2, returned:Ok(())
31589 ----------------------------------------------------------------
31590 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
31591 States: Active Active Offline
31592 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
31593 8849 Acked Read 4 Done Done Sent false
31594 8850 Acked Write 10 Done Done Sent false
31595 8851 Acked Read 4 Done Done Sent false
31596 8852 Acked Write 8 Done Done Sent false
31597 8853 Acked Write 3 Done Done Sent false
31598 8854 Acked Write 9 Done Done Sent false
31599 8855 Acked Flush 0 Done Done Sent false
31600 8856 Acked Read 5 Done Done Sent false
31601 8857 Acked Read 6 Done Done Sent false
31602 8858 Acked Read 3 Done Done Sent false
31603 8859 Acked Read 4 Done Done Sent false
31604 8860 Acked Flush 0 Done Done Sent false
31605 8861 Acked Read 4 Done Done Sent false
31606 8862 Acked Read 4 Done Done Sent false
31607 8863 Acked Write 7 Done Done Sent false
31608 8864 Acked Read 1 Done Done Sent false
31609 8865 Acked Write 3 Done Done Sent false
31610 8866 Acked Read 3 Done Done Sent false
31611 8867 Acked Write 1 Done Done Sent false
31612 8868 Acked Write 1 Done Done Sent false
31613 8869 Acked Read 9 Done Done Sent false
31614 8870 Acked Write 10 Done Done Sent false
31615 8871 Acked Write 9 Done Done Sent false
31616 8872 Acked Read 9 Done Done Sent false
31617 8873 Acked Write 1 Done Done Sent false
31618 8874 Acked Write 2 Done Done Sent false
31619 8875 Acked Read 1 Done Done Sent false
31620 8876 Acked Write 3 Done Done Sent false
31621 8877 Acked Read 2 Done Done Sent false
31622 8878 Acked Read 6 Done Done Sent false
31623 8879 Acked Read 3 Done Done Sent false
31624 8880 Acked Read 3 Done Done Sent false
31625 8881 Acked Write 2 Done Done Sent false
31626 8882 Acked Write 2 Done Done Sent false
31627 8883 Acked Read 8 Done Done Sent false
31628 8884 Acked Write 2 Done Done Sent false
31629 8885 Acked Read 6 Done Done Sent false
31630 8886 Acked Write 9 Done Done Sent false
31631 8887 Acked Write 8 Done Done Sent false
31632 8888 Acked Write 1 Done Done Sent false
31633 8889 Acked Read 10 Done Done Sent false
31634 8890 Acked Write 3 Done Done Sent false
31635 8891 Acked Write 4 Done Done Sent false
31636 8892 Acked Read 4 Done Done Sent false
31637 8893 Acked Write 8 Done Done Sent false
31638 8894 Acked Write 2 Done Done Sent false
31639 8895 Acked Write 1 Done Done Sent false
31640 8896 Acked Write 10 Done Done Sent false
31641 8897 Acked Write 1 Done Done Sent false
31642 8898 Acked Flush 0 Done Done Sent false
31643 8899 Acked Write 2 Done Done Sent false
31644 8900 Acked Read 6 Done Done Sent false
31645 8901 Acked Read 1 Done Done Sent false
31646 8902 Acked Read 7 Done Done Sent false
31647 8903 Acked Write 6 Done Done Sent false
31648 8904 Acked Flush 0 Done Done Sent false
31649 8905 Acked Write 8 Done Done Sent false
31650 8906 Acked Read 4 Done Done Sent false
31651 8907 Acked Read 10 Done Done Sent false
31652 8908 Acked Write 8 Done Done Sent false
31653 8909 Acked Read 4 Done Done Sent false
31654 8910 Acked Read 7 Done Done Sent false
31655 8911 Acked Read 10 Done Done Sent false
31656 8912 Acked Read 8 Done Done Sent false
31657 8913 Acked Write 9 Done Done Sent false
31658 8914 Acked Write 2 Done Done Sent false
31659 8915 Acked Write 3 Done Done Sent false
31660 8916 Acked Write 6 Done Done Sent false
31661 8917 Acked Write 1 Done Done Sent false
31662 8918 Acked Write 1 Done Done Sent false
31663 8919 Acked Read 7 Done Done Sent false
31664 8920 Acked Read 3 Done Done Sent false
31665 8921 Acked Flush 0 Done Done Sent false
31666 8922 Acked Write 9 Done Done Sent false
31667 8923 Acked Read 8 Done Done Sent false
31668 8924 Acked Flush 0 Done Done Sent false
31669 8925 Acked Read 10 Done Done Sent false
31670 8926 Acked Write 5 Done Done Sent false
31671 8927 Acked Flush 0 Done Done Sent false
31672 8928 Acked Read 2 Done Done Sent false
31673 8929 Acked Read 5 Done Done Sent false
31674 8930 Acked Write 7 Done Done Sent false
31675 8931 Acked Read 3 Done Done Sent false
31676 8932 Acked Write 2 Done Done Sent false
31677 8933 Acked Read 8 Done Done Sent false
31678 8934 Acked Write 9 Done Done Sent false
31679 8935 Acked Write 7 Done Done Sent false
31680 8936 Acked Write 9 Done Done Sent false
31681 8937 Acked Read 10 Done Done Sent false
31682 8938 Acked Read 4 Done Done Sent false
31683 8939 Acked Read 4 Done Done Sent false
31684 8940 Acked Write 10 Done Done Sent false
31685 8941 Acked Read 9 Done Done Sent false
31686 8942 Acked Read 8 Done Done Sent false
31687 8943 Acked Write 9 Done Done Sent false
31688 8944 Acked Read 10 Done Done Sent false
31689 8945 Acked Read 6 Done Done Sent false
31690 8946 Acked Read 6 Done Done Sent false
31691 8947 Acked Write 10 Done Done Sent false
31692 8948 Acked Read 3 Done Done Sent false
31693 8949 Acked Write 6 Done Done Sent false
31694 8950 Acked Write 5 Done Done Sent false
31695 8951 Acked Read 2 Done Done Sent false
31696 8952 Acked Write 6 Done Done Sent false
31697 8953 Acked Read 2 Done Done Sent false
31698 8954 Acked Write 3 Done Done Sent false
31699 8955 Acked Flush 0 Done Done Sent false
31700 8956 Acked Read 8 Done Done Sent false
31701 8957 Acked Write 2 Done Done Sent false
31702 8958 Acked Read 1 Done Done Sent false
31703 8959 Acked Read 5 Done Done Sent false
31704 8960 Acked Read 6 Done Done Sent false
31705 8961 Acked Read 9 Done Done Sent false
31706 8962 Acked Write 9 Done Done Sent false
31707 8963 Acked Read 6 Done Done Sent false
31708 8964 Acked Read 9 Done Done Sent false
31709 8965 Acked Write 6 Done Done Sent false
31710 8966 Acked Read 3 Done Done Sent false
31711 8967 Acked Read 3 Done Done Sent false
31712 8968 Acked Read 10 Done Done Sent false
31713 8969 Acked Flush 0 Done Done Sent false
31714 8970 Acked Write 5 Done Done Sent false
31715 8971 Acked Write 7 Done Done Sent false
31716 8972 Acked Write 8 Done Done Sent false
31717 8973 Acked Read 7 Done Done Sent false
31718 8974 Acked Read 10 Done Done Sent false
31719 8975 Acked Read 4 Done Done Sent false
31720 8976 Acked Read 7 Done Done Sent false
31721 8977 Acked Read 1 Done Done Sent false
31722 8978 Acked Read 7 Done Done Sent false
31723 8979 Acked Read 10 Done Done Sent false
31724 8980 Acked Read 3 Done Done Sent false
31725 8981 Acked Read 4 Done Done Sent false
31726 8982 Acked Write 7 Done Done Sent false
31727 8983 Acked Read 9 Done Done Sent false
31728 8984 Acked Write 6 Done Done Sent false
31729 8985 Acked Write 4 Done Done Sent false
31730 8986 Acked Write 1 Done Done Sent false
31731 8987 Acked Write 4 Done Done Sent false
31732 8988 Acked Read 8 Done Done Sent false
31733 8989 Acked Write 6 Done Done Sent false
31734 8990 Acked Write 9 Done Done Sent false
31735 8991 Acked Write 6 Done Done Sent false
31736 8992 Acked Read 3 Done Done Sent false
31737 8993 Acked Write 1 Done Done Sent false
31738 8994 Acked Read 4 Done Done Sent false
31739 8995 Acked Write 3 Done Done Sent false
31740 8996 Acked Read 9 Done Done Sent false
31741 8997 Acked Write 7 Done Done Sent false
31742 8998 Acked Read 5 Done Done Sent false
31743 8999 Acked Read 8 Done Done Sent false
31744 9000 Acked Read 6 Done Done Sent false
31745 9001 Acked Read 2 Done Done Sent false
31746 9002 Acked Read 4 Done Done Sent false
31747 9003 Acked Read 9 Done Done Sent false
31748 9004 Acked Read 2 Done Done Sent false
31749 9005 Acked Read 10 Done Done Sent false
31750 9006 Acked Write 8 Done Done Sent false
31751 9007 Acked Flush 0 Done Done Sent false
31752 9008 Acked Read 6 Done Done Sent false
31753 9009 Acked Read 5 Done Done Sent false
31754 9010 Acked Read 2 Done Done Sent false
31755 9011 Acked Read 10 Done Done Sent false
31756 9012 Acked Read 4 Done Done Sent false
31757 9013 Acked Read 7 Done Done Sent false
31758 9014 Acked Write 1 Done Done Sent false
31759 9015 Acked Read 1 Done Done Sent false
31760 9016 Acked Read 2 Done Done Sent false
31761 9017 Acked Write 1 Done Done Sent false
31762 9018 Acked Write 8 Done Done Sent false
31763 9019 Acked Write 10 Done Done Sent false
31764 9020 Acked Read 8 Done Done Sent false
31765 9021 Acked Write 3 Done Done Sent false
31766 9022 Acked Read 5 Done Done Sent false
31767 9023 Acked Read 7 Done Done Sent false
31768 9024 Acked Read 8 Done Done Sent false
31769 9025 Acked Write 2 Done Done Sent false
31770 9026 Acked Write 6 Done Done Sent false
31771 9027 Acked Flush 0 Done Done Sent false
31772 9028 Acked Read 6 Done Done Sent false
31773 9029 Acked Read 3 Done Done Sent false
31774 9030 Acked Read 9 Done Done Sent false
31775 9031 Acked Write 2 Done Done Sent false
31776 9032 Acked Read 5 Done Done Sent false
31777 9033 Acked Read 5 Done Done Sent false
31778 9034 Acked Write 7 Done Done Sent false
31779 9035 Acked Read 9 Done Done Sent false
31780 9036 Acked Read 8 Done Done Sent false
31781 9037 Acked Flush 0 Done Done Sent false
31782 9038 Acked Read 2 Done Done Sent false
31783 9039 Acked Write 9 Done Done Sent false
31784 9040 Acked Write 10 Done Done Sent false
31785 9041 Acked Write 7 Done Done Sent false
31786 9042 Acked Write 10 Done Done Sent false
31787 9043 Acked Read 2 Done Done Sent false
31788 9044 Acked Write 9 Done Done Sent false
31789 9045 Acked Read 10 Done Done Sent false
31790 9046 Acked Write 8 Done Done Sent false
31791 9047 Acked Read 2 Done Done Sent false
31792 9048 Acked Read 3 Done Done Sent false
31793 9049 Acked Read 1 Done Done Sent false
31794 9050 Acked Read 10 Done Done Sent false
31795 9051 Acked Read 1 Done Done Sent false
31796 9052 Acked Read 9 Done Done Sent false
31797 9053 Acked Write 7 Done Done Sent false
31798 9054 Acked Read 2 Done Done Sent false
31799 9055 Acked Read 3 Done Done Sent false
31800 9056 Acked Write 5 Done Done Sent false
31801 9057 Acked Read 6 Done Done Sent false
31802 9058 Acked Flush 0 Done Done Sent false
31803 9059 Acked Flush 0 Done Done Sent false
31804 9060 Acked Write 4 Done Done Sent false
31805 9061 Acked Write 3 Done Done Sent false
31806 9062 Acked Read 8 Done Done Sent false
31807 9063 Acked Write 5 Done Done Sent false
31808 9064 Acked Read 7 Done Done Sent false
31809 9065 Acked Read 6 Done Done Sent false
31810 9066 Acked Read 9 Done Done Sent false
31811 9067 Acked Read 10 Done Done Sent false
31812 9068 Acked Read 7 Done Done Sent false
31813 9069 Acked Write 2 Done Done Sent false
31814 9070 Acked Read 2 Done Done Sent false
31815 9071 Acked Read 3 Done Done Sent false
31816 9072 Acked Read 5 Done Done Sent false
31817 9073 Acked Read 7 Done Done Sent false
31818 9074 Acked Write 8 Done Done Sent false
31819 9075 Acked Read 7 Done Done Sent false
31820 9076 Acked Write 8 Done Done Sent false
31821 9077 Acked Read 6 Done Done Sent false
31822 9078 Acked Write 7 Done Done Sent false
31823 9079 Acked Write 8 Done Done Sent false
31824 9080 Acked Write 6 Done Done Sent false
31825 9081 Acked Write 7 Done Done Sent false
31826 9082 Acked Write 10 Done Done Sent false
31827 9083 Acked Read 7 Done Done Sent false
31828 9084 Acked Write 8 Done Done Sent false
31829 9085 Acked Read 9 Done Done Sent false
31830 9086 Acked Write 6 Done Done Sent false
31831 9087 Acked Flush 0 Done Done Sent false
31832 9088 Acked Read 8 Done Done Sent false
31833 9089 Acked Read 8 Done Done Sent false
31834 9090 Acked Read 10 Done Done Sent false
31835 9091 Acked Write 4 Done Done Sent false
31836 9092 Acked Read 1 Done Done Sent false
31837 9093 Acked Write 1 Done Done Sent false
31838 9094 Acked Read 9 Done Done Sent false
31839 9095 Acked Write 5 Done Done Sent false
31840 9096 Acked Read 7 Done Done Sent false
31841 9097 Acked Write 1 Done Done Sent false
31842 9098 Acked Flush 0 Done Done Sent false
31843 9099 Acked Write 9 Done Done Sent false
31844 9100 Acked Write 8 Done Done Sent false
31845 9101 Acked Read 4 Done Done Sent false
31846 9102 Acked Read 9 Done Done Sent false
31847 9103 Acked Read 10 Done Done Sent false
31848 9104 Acked Write 4 Done Done Sent false
31849 9105 Acked Write 9 Done Done Sent false
31850 9106 Acked Write 10 Done Done Sent false
31851 9107 Acked Write 4 Done Done Sent false
31852 9108 Acked Read 1 Done Done Sent false
31853 9109 Acked Flush 0 Done Done Sent false
31854 9110 Acked Read 4 Done Done Sent false
31855 9111 Acked Write 3 Done Done Sent false
31856 9112 Acked Flush 0 Done Done Sent false
31857 9113 Acked Read 2 Done Done Sent false
31858 9114 Acked Flush 0 Done Done Sent false
31859 9115 Acked Read 7 Done Done Sent false
31860 9116 Acked Read 4 Done Done Sent false
31861 9117 Acked Write 5 Done Done Sent false
31862 9118 Acked Read 8 Done Done Sent false
31863 9119 Acked Write 10 Done Done Sent false
31864 9120 Acked Flush 0 Done Done Sent false
31865 9121 Acked Read 10 Done Done Sent false
31866 9122 Acked Write 8 Done Done Sent false
31867 9123 Acked Read 5 Done Done Sent false
31868 9124 Acked Write 4 Done Done Sent false
31869 9125 Acked Read 4 Done Done Sent false
31870 9126 Acked Write 2 Done Done Sent false
31871 9127 Acked Write 3 Done Done Sent false
31872 9128 Acked Read 10 Done Done Sent false
31873 9129 Acked Flush 0 Done Done Sent false
31874 9130 Acked Read 2 Done Done Sent false
31875 9131 Acked Read 1 Done Done Sent false
31876 9132 Acked Read 5 Done Done Sent false
31877 9133 Acked Write 9 Done Done Sent false
31878 9134 Acked Read 7 Done Done Sent false
31879 9135 Acked Read 10 Done Done Sent false
31880 9136 Acked Read 5 Done Done Sent false
31881 9137 Acked Read 9 Done Done Sent false
31882 9138 Acked Flush 0 Done Done Sent false
31883 9139 Acked Write 7 Done Done Sent false
31884 9140 Acked Write 5 Done Done Sent false
31885 9141 Acked Read 1 Done Done Sent false
31886 9142 Acked Read 3 Done Done Sent false
31887 9143 Acked Read 2 Done Done Sent false
31888 9144 Acked Read 2 Done Done Sent false
31889 9145 Acked Write 7 Done Done Sent false
31890 9146 Acked Write 2 Done Done Sent false
31891 9147 Acked Flush 0 Done Done Sent false
31892 9148 Acked Read 7 Done Done Sent false
31893 9149 Acked Write 5 Done Done Sent false
31894 STATES DS:0 DS:1 DS:2 TOTAL
31895 Sent 0 0 301 301
31896 Done 301 301 0 602
31897 Skipped 0 0 0 0
31898 Error 0 0 0 0
31899 Last Flush: 9147 9147 8848
31900 Downstairs last five completed: 8848 8847 8846 8845 8844
31901 Upstairs last five completed: 9149 9148 9147 9146 9145
31902 CLIENT: Up:0 ds:301 act:2
31903 [8830] got start action
31904 [8830] I am going to restart
31905 Starting downstairs at port 8830
31906 Make output file at "/tmp/dsc/downstairs-8830.txt"
31907 [8830][2] reports Starting
31908 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1229)
31909 [8830][2] reports Running
31910 ----------------------------------------------------------------
31911 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
31912 States: Active Active Offline
31913 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
31914 8849 Acked Read 4 Done Done Sent false
31915 8850 Acked Write 10 Done Done Sent false
31916 8851 Acked Read 4 Done Done Sent false
31917 8852 Acked Write 8 Done Done Sent false
31918 8853 Acked Write 3 Done Done Sent false
31919 8854 Acked Write 9 Done Done Sent false
31920 8855 Acked Flush 0 Done Done Sent false
31921 8856 Acked Read 5 Done Done Sent false
31922 8857 Acked Read 6 Done Done Sent false
31923 8858 Acked Read 3 Done Done Sent false
31924 8859 Acked Read 4 Done Done Sent false
31925 8860 Acked Flush 0 Done Done Sent false
31926 8861 Acked Read 4 Done Done Sent false
31927 8862 Acked Read 4 Done Done Sent false
31928 8863 Acked Write 7 Done Done Sent false
31929 8864 Acked Read 1 Done Done Sent false
31930 8865 Acked Write 3 Done Done Sent false
31931 8866 Acked Read 3 Done Done Sent false
31932 8867 Acked Write 1 Done Done Sent false
31933 8868 Acked Write 1 Done Done Sent false
31934 8869 Acked Read 9 Done Done Sent false
31935 8870 Acked Write 10 Done Done Sent false
31936 8871 Acked Write 9 Done Done Sent false
31937 8872 Acked Read 9 Done Done Sent false
31938 8873 Acked Write 1 Done Done Sent false
31939 8874 Acked Write 2 Done Done Sent false
31940 8875 Acked Read 1 Done Done Sent false
31941 8876 Acked Write 3 Done Done Sent false
31942 8877 Acked Read 2 Done Done Sent false
31943 8878 Acked Read 6 Done Done Sent false
31944 8879 Acked Read 3 Done Done Sent false
31945 8880 Acked Read 3 Done Done Sent false
31946 8881 Acked Write 2 Done Done Sent false
31947 8882 Acked Write 2 Done Done Sent false
31948 8883 Acked Read 8 Done Done Sent false
31949 8884 Acked Write 2 Done Done Sent false
31950 8885 Acked Read 6 Done Done Sent false
31951 8886 Acked Write 9 Done Done Sent false
31952 8887 Acked Write 8 Done Done Sent false
31953 8888 Acked Write 1 Done Done Sent false
31954 8889 Acked Read 10 Done Done Sent false
31955 8890 Acked Write 3 Done Done Sent false
31956 8891 Acked Write 4 Done Done Sent false
31957 8892 Acked Read 4 Done Done Sent false
31958 8893 Acked Write 8 Done Done Sent false
31959 8894 Acked Write 2 Done Done Sent false
31960 8895 Acked Write 1 Done Done Sent false
31961 8896 Acked Write 10 Done Done Sent false
31962 8897 Acked Write 1 Done Done Sent false
31963 8898 Acked Flush 0 Done Done Sent false
31964 8899 Acked Write 2 Done Done Sent false
31965 8900 Acked Read 6 Done Done Sent false
31966 8901 Acked Read 1 Done Done Sent false
31967 8902 Acked Read 7 Done Done Sent false
31968 8903 Acked Write 6 Done Done Sent false
31969 8904 Acked Flush 0 Done Done Sent false
31970 8905 Acked Write 8 Done Done Sent false
31971 8906 Acked Read 4 Done Done Sent false
31972 8907 Acked Read 10 Done Done Sent false
31973 8908 Acked Write 8 Done Done Sent false
31974 8909 Acked Read 4 Done Done Sent false
31975 8910 Acked Read 7 Done Done Sent false
31976 8911 Acked Read 10 Done Done Sent false
31977 8912 Acked Read 8 Done Done Sent false
31978 8913 Acked Write 9 Done Done Sent false
31979 8914 Acked Write 2 Done Done Sent false
31980 8915 Acked Write 3 Done Done Sent false
31981 8916 Acked Write 6 Done Done Sent false
31982 8917 Acked Write 1 Done Done Sent false
31983 8918 Acked Write 1 Done Done Sent false
31984 8919 Acked Read 7 Done Done Sent false
31985 8920 Acked Read 3 Done Done Sent false
31986 8921 Acked Flush 0 Done Done Sent false
31987 8922 Acked Write 9 Done Done Sent false
31988 8923 Acked Read 8 Done Done Sent false
31989 8924 Acked Flush 0 Done Done Sent false
31990 8925 Acked Read 10 Done Done Sent false
31991 8926 Acked Write 5 Done Done Sent false
31992 8927 Acked Flush 0 Done Done Sent false
31993 8928 Acked Read 2 Done Done Sent false
31994 8929 Acked Read 5 Done Done Sent false
31995 8930 Acked Write 7 Done Done Sent false
31996 8931 Acked Read 3 Done Done Sent false
31997 8932 Acked Write 2 Done Done Sent false
31998 8933 Acked Read 8 Done Done Sent false
31999 8934 Acked Write 9 Done Done Sent false
32000 8935 Acked Write 7 Done Done Sent false
32001 8936 Acked Write 9 Done Done Sent false
32002 8937 Acked Read 10 Done Done Sent false
32003 8938 Acked Read 4 Done Done Sent false
32004 8939 Acked Read 4 Done Done Sent false
32005 8940 Acked Write 10 Done Done Sent false
32006 8941 Acked Read 9 Done Done Sent false
32007 8942 Acked Read 8 Done Done Sent false
32008 8943 Acked Write 9 Done Done Sent false
32009 8944 Acked Read 10 Done Done Sent false
32010 8945 Acked Read 6 Done Done Sent false
32011 8946 Acked Read 6 Done Done Sent false
32012 8947 Acked Write 10 Done Done Sent false
32013 8948 Acked Read 3 Done Done Sent false
32014 8949 Acked Write 6 Done Done Sent false
32015 8950 Acked Write 5 Done Done Sent false
32016 8951 Acked Read 2 Done Done Sent false
32017 8952 Acked Write 6 Done Done Sent false
32018 8953 Acked Read 2 Done Done Sent false
32019 8954 Acked Write 3 Done Done Sent false
32020 8955 Acked Flush 0 Done Done Sent false
32021 8956 Acked Read 8 Done Done Sent false
32022 8957 Acked Write 2 Done Done Sent false
32023 8958 Acked Read 1 Done Done Sent false
32024 8959 Acked Read 5 Done Done Sent false
32025 8960 Acked Read 6 Done Done Sent false
32026 8961 Acked Read 9 Done Done Sent false
32027 8962 Acked Write 9 Done Done Sent false
32028 8963 Acked Read 6 Done Done Sent false
32029 8964 Acked Read 9 Done Done Sent false
32030 8965 Acked Write 6 Done Done Sent false
32031 8966 Acked Read 3 Done Done Sent false
32032 8967 Acked Read 3 Done Done Sent false
32033 8968 Acked Read 10 Done Done Sent false
32034 8969 Acked Flush 0 Done Done Sent false
32035 8970 Acked Write 5 Done Done Sent false
32036 8971 Acked Write 7 Done Done Sent false
32037 8972 Acked Write 8 Done Done Sent false
32038 8973 Acked Read 7 Done Done Sent false
32039 8974 Acked Read 10 Done Done Sent false
32040 8975 Acked Read 4 Done Done Sent false
32041 8976 Acked Read 7 Done Done Sent false
32042 8977 Acked Read 1 Done Done Sent false
32043 8978 Acked Read 7 Done Done Sent false
32044 8979 Acked Read 10 Done Done Sent false
32045 8980 Acked Read 3 Done Done Sent false
32046 8981 Acked Read 4 Done Done Sent false
32047 8982 Acked Write 7 Done Done Sent false
32048 8983 Acked Read 9 Done Done Sent false
32049 8984 Acked Write 6 Done Done Sent false
32050 8985 Acked Write 4 Done Done Sent false
32051 8986 Acked Write 1 Done Done Sent false
32052 8987 Acked Write 4 Done Done Sent false
32053 8988 Acked Read 8 Done Done Sent false
32054 8989 Acked Write 6 Done Done Sent false
32055 8990 Acked Write 9 Done Done Sent false
32056 8991 Acked Write 6 Done Done Sent false
32057 8992 Acked Read 3 Done Done Sent false
32058 8993 Acked Write 1 Done Done Sent false
32059 8994 Acked Read 4 Done Done Sent false
32060 8995 Acked Write 3 Done Done Sent false
32061 8996 Acked Read 9 Done Done Sent false
32062 8997 Acked Write 7 Done Done Sent false
32063 8998 Acked Read 5 Done Done Sent false
32064 8999 Acked Read 8 Done Done Sent false
32065 9000 Acked Read 6 Done Done Sent false
32066 9001 Acked Read 2 Done Done Sent false
32067 9002 Acked Read 4 Done Done Sent false
32068 9003 Acked Read 9 Done Done Sent false
32069 9004 Acked Read 2 Done Done Sent false
32070 9005 Acked Read 10 Done Done Sent false
32071 9006 Acked Write 8 Done Done Sent false
32072 9007 Acked Flush 0 Done Done Sent false
32073 9008 Acked Read 6 Done Done Sent false
32074 9009 Acked Read 5 Done Done Sent false
32075 9010 Acked Read 2 Done Done Sent false
32076 9011 Acked Read 10 Done Done Sent false
32077 9012 Acked Read 4 Done Done Sent false
32078 9013 Acked Read 7 Done Done Sent false
32079 9014 Acked Write 1 Done Done Sent false
32080 9015 Acked Read 1 Done Done Sent false
32081 9016 Acked Read 2 Done Done Sent false
32082 9017 Acked Write 1 Done Done Sent false
32083 9018 Acked Write 8 Done Done Sent false
32084 9019 Acked Write 10 Done Done Sent false
32085 9020 Acked Read 8 Done Done Sent false
32086 9021 Acked Write 3 Done Done Sent false
32087 9022 Acked Read 5 Done Done Sent false
32088 9023 Acked Read 7 Done Done Sent false
32089 9024 Acked Read 8 Done Done Sent false
32090 9025 Acked Write 2 Done Done Sent false
32091 9026 Acked Write 6 Done Done Sent false
32092 9027 Acked Flush 0 Done Done Sent false
32093 9028 Acked Read 6 Done Done Sent false
32094 9029 Acked Read 3 Done Done Sent false
32095 9030 Acked Read 9 Done Done Sent false
32096 9031 Acked Write 2 Done Done Sent false
32097 9032 Acked Read 5 Done Done Sent false
32098 9033 Acked Read 5 Done Done Sent false
32099 9034 Acked Write 7 Done Done Sent false
32100 9035 Acked Read 9 Done Done Sent false
32101 9036 Acked Read 8 Done Done Sent false
32102 9037 Acked Flush 0 Done Done Sent false
32103 9038 Acked Read 2 Done Done Sent false
32104 9039 Acked Write 9 Done Done Sent false
32105 9040 Acked Write 10 Done Done Sent false
32106 9041 Acked Write 7 Done Done Sent false
32107 9042 Acked Write 10 Done Done Sent false
32108 9043 Acked Read 2 Done Done Sent false
32109 9044 Acked Write 9 Done Done Sent false
32110 9045 Acked Read 10 Done Done Sent false
32111 9046 Acked Write 8 Done Done Sent false
32112 9047 Acked Read 2 Done Done Sent false
32113 9048 Acked Read 3 Done Done Sent false
32114 9049 Acked Read 1 Done Done Sent false
32115 9050 Acked Read 10 Done Done Sent false
32116 9051 Acked Read 1 Done Done Sent false
32117 9052 Acked Read 9 Done Done Sent false
32118 9053 Acked Write 7 Done Done Sent false
32119 9054 Acked Read 2 Done Done Sent false
32120 9055 Acked Read 3 Done Done Sent false
32121 9056 Acked Write 5 Done Done Sent false
32122 9057 Acked Read 6 Done Done Sent false
32123 9058 Acked Flush 0 Done Done Sent false
32124 9059 Acked Flush 0 Done Done Sent false
32125 9060 Acked Write 4 Done Done Sent false
32126 9061 Acked Write 3 Done Done Sent false
32127 9062 Acked Read 8 Done Done Sent false
32128 9063 Acked Write 5 Done Done Sent false
32129 9064 Acked Read 7 Done Done Sent false
32130 9065 Acked Read 6 Done Done Sent false
32131 9066 Acked Read 9 Done Done Sent false
32132 9067 Acked Read 10 Done Done Sent false
32133 9068 Acked Read 7 Done Done Sent false
32134 9069 Acked Write 2 Done Done Sent false
32135 9070 Acked Read 2 Done Done Sent false
32136 9071 Acked Read 3 Done Done Sent false
32137 9072 Acked Read 5 Done Done Sent false
32138 9073 Acked Read 7 Done Done Sent false
32139 9074 Acked Write 8 Done Done Sent false
32140 9075 Acked Read 7 Done Done Sent false
32141 9076 Acked Write 8 Done Done Sent false
32142 9077 Acked Read 6 Done Done Sent false
32143 9078 Acked Write 7 Done Done Sent false
32144 9079 Acked Write 8 Done Done Sent false
32145 9080 Acked Write 6 Done Done Sent false
32146 9081 Acked Write 7 Done Done Sent false
32147 9082 Acked Write 10 Done Done Sent false
32148 9083 Acked Read 7 Done Done Sent false
32149 9084 Acked Write 8 Done Done Sent false
32150 9085 Acked Read 9 Done Done Sent false
32151 9086 Acked Write 6 Done Done Sent false
32152 9087 Acked Flush 0 Done Done Sent false
32153 9088 Acked Read 8 Done Done Sent false
32154 9089 Acked Read 8 Done Done Sent false
32155 9090 Acked Read 10 Done Done Sent false
32156 9091 Acked Write 4 Done Done Sent false
32157 9092 Acked Read 1 Done Done Sent false
32158 9093 Acked Write 1 Done Done Sent false
32159 9094 Acked Read 9 Done Done Sent false
32160 9095 Acked Write 5 Done Done Sent false
32161 9096 Acked Read 7 Done Done Sent false
32162 9097 Acked Write 1 Done Done Sent false
32163 9098 Acked Flush 0 Done Done Sent false
32164 9099 Acked Write 9 Done Done Sent false
32165 9100 Acked Write 8 Done Done Sent false
32166 9101 Acked Read 4 Done Done Sent false
32167 9102 Acked Read 9 Done Done Sent false
32168 9103 Acked Read 10 Done Done Sent false
32169 9104 Acked Write 4 Done Done Sent false
32170 9105 Acked Write 9 Done Done Sent false
32171 9106 Acked Write 10 Done Done Sent false
32172 9107 Acked Write 4 Done Done Sent false
32173 9108 Acked Read 1 Done Done Sent false
32174 9109 Acked Flush 0 Done Done Sent false
32175 9110 Acked Read 4 Done Done Sent false
32176 9111 Acked Write 3 Done Done Sent false
32177 9112 Acked Flush 0 Done Done Sent false
32178 9113 Acked Read 2 Done Done Sent false
32179 9114 Acked Flush 0 Done Done Sent false
32180 9115 Acked Read 7 Done Done Sent false
32181 9116 Acked Read 4 Done Done Sent false
32182 9117 Acked Write 5 Done Done Sent false
32183 9118 Acked Read 8 Done Done Sent false
32184 9119 Acked Write 10 Done Done Sent false
32185 9120 Acked Flush 0 Done Done Sent false
32186 9121 Acked Read 10 Done Done Sent false
32187 9122 Acked Write 8 Done Done Sent false
32188 9123 Acked Read 5 Done Done Sent false
32189 9124 Acked Write 4 Done Done Sent false
32190 9125 Acked Read 4 Done Done Sent false
32191 9126 Acked Write 2 Done Done Sent false
32192 9127 Acked Write 3 Done Done Sent false
32193 9128 Acked Read 10 Done Done Sent false
32194 9129 Acked Flush 0 Done Done Sent false
32195 9130 Acked Read 2 Done Done Sent false
32196 9131 Acked Read 1 Done Done Sent false
32197 9132 Acked Read 5 Done Done Sent false
32198 9133 Acked Write 9 Done Done Sent false
32199 9134 Acked Read 7 Done Done Sent false
32200 9135 Acked Read 10 Done Done Sent false
32201 9136 Acked Read 5 Done Done Sent false
32202 9137 Acked Read 9 Done Done Sent false
32203 9138 Acked Flush 0 Done Done Sent false
32204 9139 Acked Write 7 Done Done Sent false
32205 9140 Acked Write 5 Done Done Sent false
32206 9141 Acked Read 1 Done Done Sent false
32207 9142 Acked Read 3 Done Done Sent false
32208 9143 Acked Read 2 Done Done Sent false
32209 9144 Acked Read 2 Done Done Sent false
32210 9145 Acked Write 7 Done Done Sent false
32211 9146 Acked Write 2 Done Done Sent false
32212 9147 Acked Flush 0 Done Done Sent false
32213 9148 Acked Read 7 Done Done Sent false
32214 9149 Acked Write 5 Done Done Sent false
32215 9150 Acked Flush 0 Done Done Sent false
32216 STATES DS:0 DS:1 DS:2 TOTAL
32217 Sent 0 0 302 302
32218 Done 302 302 0 604
32219 Skipped 0 0 0 0
32220 Error 0 0 0 0
32221 Last Flush: 9150 9150 8848
32222 Downstairs last five completed: 8848 8847 8846 8845 8844
32223 Upstairs last five completed: 9150 9149 9148 9147 9146
32224 CLIENT: Up:0 ds:302 act:2
322252024-11-28T04:57:03.347ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
322262024-11-28T04:57:03.347ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:49509) = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
322272024-11-28T04:57:03.348ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
322282024-11-28T04:57:03.349ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
322292024-11-28T04:57:03.349ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
322302024-11-28T04:57:03.349ZINFOcrucible: send last flush ID to this DS: 8848 = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
322312024-11-28T04:57:03.349ZINFOcrucible: Replied this last flush ID: 8848 = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
322322024-11-28T04:57:03.349ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
322332024-11-28T04:57:03.349ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
322342024-11-28T04:57:03.349ZINFOcrucible: [2] client re-new 302 jobs since flush 8848 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
322352024-11-28T04:57:03.349ZINFOcrucible: [2] Replayed 302 jobs since flush: 8848 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
32236 ----------------------------------------------------------------
32237 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
32238 Downstairs last five completed: 9150 9149 9148 9147 9146
32239 Upstairs last five completed: 9150 9149 9148 9147 9146
32240 CLIENT: Up:0 ds:0 act:3
32241 Replay: All jobs finished
32242 Main task has work to do, go find it
32243 got dsc Stop(2)
32244 stop 2
32245 Nov 28 04:57:05.597 [8830] Got stop action so:false kr:false
32246 INFO request completed, latency_us: 353, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 5061af7e-f30a-42ac-b51b-749130ce7f9e, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
322472024-11-28T04:57:05.598ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
322482024-11-28T04:57:05.598ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
322492024-11-28T04:57:05.598ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
32250 Nov 28 04:57:05.598 INFO request completed, latency_us: 138, response_code: {"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-28T04:57:05.598275264Z","hostname":"w-01JDRHKEEFJ1228TFPSBEGXWHY","pid":1202,"client":"2","":"downstairs","session_id":"4a2415b1-6594-436f-82e5-b8cf9a74eeaf"}
32251 200, uri: /state/cid/2, method: GET, req_id:{"msg":"[2] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-28T04:57:05.59830295Z","hostname":"w-01JDRHKEEFJ1228TFPSBEGXWHY","pid":1202,"client":"2","":"downstairs","session_id":"4a2415b1-6594-436f-82e5-b8cf9a74eeaf"}
32252 e0df98f7-758f-4596-bb10-bc5df110e5fb, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2024-11-28T04:57:05.598327237Z","hostname":"w-01JDRHKEEFJ1228TFPSBEGXWHY","pid":1202,"":"io task","client":"2","":"downstairs","session_id":"4a2415b1-6594-436f-82e5-b8cf9a74eeaf"}
32253 1:9998
32254 [8830] Exited with: ExitStatus(unix_wait_status(9))
32255 [8830][2] reports Exit
32256 Nov 28 04:57:09.600 INFO request completed, latency_us: 206, response_code: 200, uri: /state/cid/2, method: GET, req_id: 8c78cc2a-105d-41e0-8165-5cbf3e62f594, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
32257 001/300 Read block 3620 len 12288
32258 002/300 Write block 3197 len 8192 data: 3 5
32259 003/300 Write block 4761 len 20480 data: 6 6 5 5 4
32260 004/300 Read block 121 len 16384
32261 005/300 Write block 3840 len 36864 data: 5 4 4 6 6 7 7 6 7
32262 006/300 Write block 220 len 16384 data: 5 5 5 4
32263 007/300 Write block 589 len 36864 data: 6 7 6 6 7 7 8 9 10
32264 008/300 Read block 3326 len 28672
32265 009/300 Read block 773 len 40960
32266 010/300 Read block 2147 len 36864
32267 011/300 Flush
32268 012/300 Flush
32269 013/300 Write block 1433 len 40960 data: 4 5 5 7 6 5 5 6 5 5
32270 014/300 Read block 3329 len 16384
32271 015/300 Read block 2646 len 32768
32272 016/300 Read block 3775 len 12288
32273 017/300 Flush
32274 018/300 Write block 2160 len 28672 data: 5 7 7 7 5 7 7
32275 019/300 Write block 1183 len 12288 data: 8 6 6
32276 020/300 Read block 2218 len 20480
32277 021/300 Write block 1373 len 40960 data: 5 6 5 5 3 3 3 3 4 5
32278 022/300 Flush
32279 023/300 Write block 2593 len 28672 data: 3 4 4 5 4 5 6
32280 024/300 Write block 352 len 40960 data: 6 6 7 6 6 5 3 3 3 3
32281 025/300 Write block 4562 len 4096 data: 6
32282 026/300 Write block 1902 len 12288 data: 6 6 5
32283 027/300 Read block 3290 len 8192
32284 028/300 Read block 263 len 16384
32285 029/300 Flush
32286 030/300 Read block 205 len 12288
32287 031/300 Read block 2060 len 28672
32288 032/300 Read block 4075 len 20480
32289 033/300 Read block 1010 len 32768
32290 034/300 Read block 1993 len 20480
32291 035/300 Write block 447 len 24576 data: 6 5 5 3 4 3
32292 036/300 Write block 2331 len 4096 data: 8
32293 037/300 Write block 2268 len 40960 data: 4 3 3 4 7 6 6 6 5 5
32294 038/300 Write block 3276 len 12288 data: 5 5 6
32295 039/300 Read block 4989 len 36864
32296 040/300 Read block 4811 len 24576
32297 041/300 Flush
32298 042/300 Read block 3775 len 40960
32299 043/300 Read block 4119 len 40960
32300 044/300 Write block 1375 len 28672 data: 6 6 4 4 4 4 5
32301 045/300 Write block 4291 len 24576 data: 3 3 3 3 3 4
32302 046/300 Read block 189 len 28672
32303 047/300 Read block 804 len 8192
32304 048/300 Write block 507 len 12288 data: 10 10 8
32305 049/300 Read block 1755 len 12288
32306 050/300 Write block 693 len 20480 data: 5 6 5 5 4
32307 051/300 Write block 3233 len 40960 data: 3 4 4 6 8 8 10 8 8 7
32308 052/300 Read block 2724 len 12288
32309 053/300 Write block 1968 len 24576 data: 7 7 7 7 8 9
32310 054/300 Write block 1776 len 28672 data: 8 7 6 6 5 6 5
32311 055/300 Flush
32312 056/300 Read block 1319 len 4096
32313 057/300 Read block 602 len 36864
32314 058/300 Write block 3583 len 28672 data: 6 6 6 5 6 6 5
32315 059/300 Write block 1084 len 20480 data: 6 5 5 6 7
32316 060/300 Write block 3919 len 4096 data: 3
32317 061/300 Write block 2250 len 20480 data: 8 6 7 6 6
32318 062/300 Read block 3719 len 8192
32319 063/300 Write block 3715 len 4096 data: 2
32320 064/300 Write block 1176 len 36864 data: 5 5 5 6 6 9 9 9 7
32321 065/300 Read block 3579 len 12288
32322 066/300 Read block 2977 len 8192
32323 067/300 Write block 678 len 32768 data: 3 3 3 3 3 3 5 6
32324 068/300 Read block 2377 len 40960
32325 069/300 Write block 3586 len 32768 data: 6 7 7 6 5 4 5 6
32326 070/300 Read block 2591 len 12288
32327 071/300 Read block 3490 len 28672
32328 072/300 Flush
32329 073/300 Write block 862 len 28672 data: 7 7 7 8 7 7 6
32330 074/300 Read block 14 len 24576
32331 075/300 Flush
32332 076/300 Read block 2708 len 8192
32333 077/300 Read block 1773 len 12288
32334 078/300 Flush
32335 079/300 Write block 3398 len 20480 data: 3 3 3 3 3
32336 080/300 Write block 3005 len 8192 data: 9 9
32337 081/300 Read block 4499 len 4096
32338 082/300 Write block 1584 len 40960 data: 6 5 3 3 3 3 4 3 3 3
32339 083/300 Read block 4598 len 36864
32340 084/300 Read block 3452 len 20480
32341 085/300 Read block 4811 len 24576
32342 086/300 Write block 3845 len 28672 data: 8 8 7 8 9 9 7
32343 087/300 Read block 468 len 16384
32344 088/300 Write block 3505 len 36864 data: 6 8 9 10 10 10 9 8 8
32345 089/300 Read block 1506 len 24576
32346 090/300 Read block 2717 len 28672
32347 091/300 Write block 1130 len 20480 data: 2 2 4 4 5
32348 092/300 Read block 453 len 8192
32349 093/300 Write block 513 len 32768 data: 8 7 7 4 6 5 4 5
32350 094/300 Flush
32351 095/300 Write block 4113 len 32768 data: 5 5 7 6 5 6 8 8
32352 096/300 Read block 1113 len 20480
32353 097/300 Read block 674 len 28672
32354 098/300 Flush
32355 099/300 Read block 3670 len 12288
32356 100/300 Flush
32357 101/300 Write block 702 len 16384 data: 4 6 6 9
32358 102/300 Read block 930 len 4096
32359 103/300 Write block 4991 len 32768 data: 6 7 6 6 4 4 5 5
32360 104/300 Write block 3618 len 24576 data: 7 7 8 8 9 8
32361 105/300 Write block 382 len 36864 data: 11 11 9 6 5 4 4 4 5
32362 106/300 Write block 4457 len 20480 data: 9 9 8 6 6
32363 107/300 Read block 505 len 8192
32364 108/300 Write block 2783 len 40960 data: 7 10 9 8 9 11 11 12 8 8
32365 109/300 Read block 1025 len 12288
32366 110/300 Read block 899 len 36864
32367 111/300 Read block 4944 len 20480
32368 112/300 Read block 665 len 32768
32369 113/300 Read block 661 len 36864
32370 114/300 Read block 2512 len 32768
32371 115/300 Read block 2880 len 28672
32372 116/300 Read block 3240 len 20480
32373 117/300 Read block 2318 len 36864
32374 118/300 Write block 934 len 32768 data: 3 3 3 2 5 5 3 3
32375 119/300 Write block 3378 len 4096 data: 6
32376 120/300 Read block 880 len 28672
32377 121/300 Read block 576 len 28672
32378 122/300 Read block 813 len 12288
32379 123/300 Read block 2725 len 4096
32380 124/300 Read block 2637 len 24576
32381 125/300 Read block 1076 len 32768
32382 126/300 Read block 1979 len 16384
32383 127/300 Read block 55 len 28672
32384 128/300 Read block 1908 len 8192
32385 129/300 Read block 3368 len 20480
32386 130/300 Read block 1800 len 24576
32387 131/300 Read block 4990 len 16384
32388 132/300 Read block 4858 len 20480
32389 133/300 Write block 2972 len 16384 data: 5 5 6 7
32390 134/300 Read block 1004 len 20480
32391 135/300 Write block 2466 len 16384 data: 4 5 5 7
32392 136/300 Flush
32393 137/300 Read block 1510 len 40960
32394 138/300 Write block 2787 len 40960 data: 10 12 12 13 9 9 7 6 4 4
32395 139/300 Write block 1699 len 16384 data: 6 6 6 8
32396 140/300 Read block 1316 len 20480
32397 141/300 Write block 3548 len 28672 data: 4 5 5 6 6 6 6
32398 142/300 Flush
32399 143/300 Read block 943 len 32768
32400 144/300 Read block 3952 len 36864
32401 145/300 Write block 4553 len 36864 data: 6 6 5 3 4 3 3 4 5
32402 146/300 Write block 1287 len 40960 data: 6 6 6 4 4 4 3 4 4 4
32403 147/300 Write block 1710 len 32768 data: 15 12 11 8 6 5 6 5
32404 148/300 Read block 2934 len 32768
32405 149/300 Write block 336 len 36864 data: 8 9 10 8 8 9 8 6 4
32406 150/300 Write block 885 len 36864 data: 6 5 5 4 3 5 6 7 4
32407 151/300 Read block 354 len 32768
32408 152/300 Write block 3087 len 28672 data: 5 6 8 9 9 8 6
32409 153/300 Write block 2631 len 16384 data: 3 6 4 4
32410 154/300 Read block 2991 len 12288
32411 155/300 Write block 1307 len 8192 data: 5 5
32412 156/300 Write block 1174 len 20480 data: 3 4 6 6 6
32413 157/300 Read block 1807 len 12288
32414 158/300 Read block 249 len 28672
32415 159/300 Write block 1964 len 36864 data: 7 7 6 7 8 8 8 8 9
32416 160/300 Write block 4876 len 24576 data: 2 2 2 3 3 4
32417 161/300 Flush
32418 162/300 Write block 1487 len 8192 data: 3 4
32419 163/300 Read block 886 len 20480
32420 164/300 Flush
32421 165/300 Read block 1585 len 8192
32422 166/300 Write block 3975 len 4096 data: 6
32423 167/300 Flush
32424 168/300 Write block 2407 len 40960 data: 4 5 5 6 5 6 6 6 4 3
32425 169/300 Write block 1780 len 40960 data: 6 7 6 6 7 6 7 6 5 7
32426 170/300 Write block 3730 len 28672 data: 4 4 6 6 6 6 5
32427 171/300 Read block 2211 len 12288
32428 172/300 Read block 618 len 36864
32429 173/300 Write block 1396 len 36864 data: 6 5 6 8 5 6 7 7 5
32430 174/300 Read block 1586 len 40960
32431 175/300 Write block 510 len 36864 data: 9 8 8 9 8 8 5 7 6
32432 176/300 Read block 306 len 36864
32433 177/300 Read block 4290 len 36864
32434 178/300 Read block 4807 len 20480
32435 179/300 Write block 223 len 12288 data: 5 3 2
32436 180/300 Read block 1381 len 8192
32437 181/300 Read block 1284 len 32768
32438 182/300 Write block 483 len 8192 data: 4 3
32439 183/300 Write block 2664 len 8192 data: 7 7
32440 184/300 Write block 4867 len 36864 data: 6 5 6 8 8 6 5 4 4
32441 185/300 Write block 2199 len 12288 data: 5 3 3
32442 186/300 Read block 18 len 16384
32443 187/300 Write block 4369 len 36864 data: 7 5 8 6 4 5 6 7 8
32444 188/300 Read block 4762 len 28672
32445 189/300 Write block 604 len 32768 data: 9 9 7 6 6 9 10 8
32446 190/300 Write block 818 len 8192 data: 6 6
32447 191/300 Read block 288 len 24576
32448 192/300 Write block 32 len 40960 data: 7 7 7 7 8 9 8 8 7 7
32449 193/300 Read block 3365 len 12288
32450 194/300 Flush
32451 195/300 Write block 564 len 16384 data: 9 8 9 6
32452 196/300 Read block 748 len 36864
32453 197/300 Flush
32454 198/300 Read block 536 len 40960
32455 199/300 Read block 4478 len 8192
32456 200/300 Write block 3311 len 20480 data: 11 11 10 12 11
32457 201/300 Write block 2657 len 16384 data: 5 5 7 7
32458 202/300 Read block 3169 len 4096
32459 203/300 Flush
32460 204/300 Read block 4556 len 24576
32461 205/300 Write block 4135 len 4096 data: 4
32462 206/300 Read block 2295 len 8192
32463 207/300 Write block 4246 len 24576 data: 10 10 8 8 8 5
32464 208/300 Read block 404 len 4096
32465 209/300 Write block 3984 len 24576 data: 3 3 5 5 5 5
32466 210/300 Write block 4048 len 28672 data: 2 3 3 3 4 4 6
32467 211/300 Read block 4744 len 40960
32468 212/300 Read block 3106 len 36864
32469 213/300 Read block 2817 len 20480
32470 214/300 Write block 123 len 36864 data: 7 7 6 8 6 5 5 4 4
32471 215/300 Write block 1267 len 32768 data: 8 6 6 6 7 10 10 9
32472 216/300 Flush
32473 217/300 Read block 562 len 20480
32474 218/300 Read block 91 len 40960
32475 219/300 Read block 4328 len 40960
32476 220/300 Read block 2090 len 16384
32477 221/300 Read block 4465 len 40960
32478 222/300 Read block 1473 len 4096
32479 223/300 Read block 301 len 8192
32480 224/300 Read block 2593 len 28672
32481 225/300 Write block 3571 len 16384 data: 5 7 6 4
32482 226/300 Write block 4436 len 32768 data: 6 6 6 6 5 5 5 4
32483 227/300 Flush
32484 228/300 Read block 3465 len 36864
32485 229/300 Write block 4518 len 8192 data: 6 6
32486 230/300 Flush
32487 231/300 Write block 1049 len 8192 data: 9 7
32488 232/300 Read block 230 len 12288
32489 233/300 Read block 2301 len 36864
32490 234/300 Write block 194 len 8192 data: 6 6
32491 235/300 Read block 2811 len 28672
32492 236/300 Write block 572 len 12288 data: 7 6 5
32493 237/300 Write block 4500 len 4096 data: 4
32494 238/300 Write block 4048 len 40960 data: 3 4 4 4 5 5 7 5 4 4
32495 239/300 Read block 4074 len 24576
32496 240/300 Read block 4027 len 32768
32497 241/300 Write block 4180 len 20480 data: 7 6 7 8 7
32498 242/300 Write block 4035 len 36864 data: 4 4 3 3 3 3 3 4 5
32499 243/300 Read block 3868 len 32768
32500 244/300 Write block 967 len 12288 data: 9 8 9
32501 245/300 Flush
32502 246/300 Read block 1115 len 32768
32503 247/300 Read block 487 len 28672
32504 248/300 Write block 441 len 36864 data: 6 6 6 5 4 6 7 6 6
32505 249/300 Write block 2529 len 24576 data: 6 6 7 8 7 6
32506 250/300 Read block 3551 len 36864
32507 251/300 Read block 2728 len 12288
32508 252/300 Read block 724 len 32768
32509 253/300 Read block 956 len 20480
32510 254/300 Write block 3761 len 20480 data: 5 4 5 6 7
32511 255/300 Read block 2530 len 16384
32512 256/300 Write block 3201 len 12288 data: 5 4 5
32513 257/300 Read block 328 len 36864
32514 258/300 Write block 2702 len 32768 data: 8 8 7 8 9 6 5 5
32515 259/300 Read block 710 len 16384
32516 260/300 Read block 4297 len 12288
32517 261/300 Read block 3820 len 36864
32518 262/300 Read block 2632 len 4096
32519 263/300 Write block 662 len 16384 data: 11 11 8 7
32520 264/300 Flush
32521 265/300 Flush
32522 266/300 Write block 1509 len 32768 data: 4 4 3 3 2 2 3 4
32523 267/300 Write block 4966 len 36864 data: 9 9 9 9 9 9 6 6 5
32524 268/300 Read block 3124 len 28672
32525 269/300 Read block 2460 len 24576
32526 270/300 Read block 2734 len 24576
32527 271/300 Write block 1704 len 24576 data: 10 13 13 13 15 16
32528 272/300 Read block 827 len 36864
32529 273/300 Read block 2170 len 20480
32530 274/300 Read block 113 len 12288
32531 275/300 Write block 2962 len 4096 data: 7
32532 276/300 Read block 1843 len 16384
32533 277/300 Read block 80 len 4096
32534 278/300 Write block 4451 len 24576 data: 10 9 8 11 11 10
32535 279/300 Write block 34 len 40960 data: 8 8 9 10 9 9 8 8 7 6
32536 280/300 Read block 2899 len 8192
32537 281/300 Read block 1978 len 32768
32538 282/300 Read block 2575 len 36864
32539 283/300 Write block 3481 len 36864 data: 6 5 5 5 6 5 6 7 4
32540 284/300 Write block 4820 len 32768 data: 5 5 5 6 7 7 9 9
32541 285/300 Read block 1814 len 4096
32542 286/300 Read block 2389 len 16384
32543 287/300 Flush
32544 288/300 Flush
32545 289/300 Write block 425 len 28672 data: 4 4 2 2 3 3 3
32546 290/300 Read block 3504 len 12288
32547 291/300 Write block 1387 len 4096 data: 6
32548 292/300 Read block 583 len 28672
32549 293/300 Read block 3617 len 8192
32550 294/300 Read block 2432 len 40960
32551 295/300 Read block 2799 len 8192
32552 296/300 Read block 2122 len 28672
32553 297/300 Read block 2496 len 4096
32554 298/300 Read block 2127 len 24576
32555 299/300 Read block 270 len 8192
32556 300/300 Read block 447 len 16384
32557 301/300 Write block 1655 len 28672 data: 4 6 4 4 4 5 4
32558 Main task has work to do, go find it
32559 got dsc Start(2)
32560 start 2
32561 Nov 28 04:57:09.845 INFO request completed, latency_us: 258, response_code: 204, uri: /start/cid/2, method: POST, req_id: 23e759dd-f293-47f8-bbd3-6ef8bde386d2, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
32562 Replay: started 2, returned:Ok(())
32563 ----------------------------------------------------------------
32564 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
32565 States: Active Active Offline
32566 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
32567 9151 Acked Read 3 Done Done Sent false
32568 9152 Acked Write 2 Done Done Sent false
32569 9153 Acked Write 5 Done Done Sent false
32570 9154 Acked Read 4 Done Done Sent false
32571 9155 Acked Write 9 Done Done Sent false
32572 9156 Acked Write 4 Done Done Sent false
32573 9157 Acked Write 9 Done Done Sent false
32574 9158 Acked Read 7 Done Done Sent false
32575 9159 Acked Read 10 Done Done Sent false
32576 9160 Acked Read 9 Done Done Sent false
32577 9161 Acked Flush 0 Done Done Sent false
32578 9162 Acked Flush 0 Done Done Sent false
32579 9163 Acked Write 10 Done Done Sent false
32580 9164 Acked Read 4 Done Done Sent false
32581 9165 Acked Read 8 Done Done Sent false
32582 9166 Acked Read 3 Done Done Sent false
32583 9167 Acked Flush 0 Done Done Sent false
32584 9168 Acked Write 7 Done Done Sent false
32585 9169 Acked Write 3 Done Done Sent false
32586 9170 Acked Read 5 Done Done Sent false
32587 9171 Acked Write 10 Done Done Sent false
32588 9172 Acked Flush 0 Done Done Sent false
32589 9173 Acked Write 7 Done Done Sent false
32590 9174 Acked Write 10 Done Done Sent false
32591 9175 Acked Write 1 Done Done Sent false
32592 9176 Acked Write 3 Done Done Sent false
32593 9177 Acked Read 2 Done Done Sent false
32594 9178 Acked Read 4 Done Done Sent false
32595 9179 Acked Flush 0 Done Done Sent false
32596 9180 Acked Read 3 Done Done Sent false
32597 9181 Acked Read 7 Done Done Sent false
32598 9182 Acked Read 5 Done Done Sent false
32599 9183 Acked Read 8 Done Done Sent false
32600 9184 Acked Read 5 Done Done Sent false
32601 9185 Acked Write 6 Done Done Sent false
32602 9186 Acked Write 1 Done Done Sent false
32603 9187 Acked Write 10 Done Done Sent false
32604 9188 Acked Write 3 Done Done Sent false
32605 9189 Acked Read 9 Done Done Sent false
32606 9190 Acked Read 6 Done Done Sent false
32607 9191 Acked Flush 0 Done Done Sent false
32608 9192 Acked Read 10 Done Done Sent false
32609 9193 Acked Read 10 Done Done Sent false
32610 9194 Acked Write 7 Done Done Sent false
32611 9195 Acked Write 6 Done Done Sent false
32612 9196 Acked Read 7 Done Done Sent false
32613 9197 Acked Read 2 Done Done Sent false
32614 9198 Acked Write 3 Done Done Sent false
32615 9199 Acked Read 3 Done Done Sent false
32616 9200 Acked Write 5 Done Done Sent false
32617 9201 Acked Write 10 Done Done Sent false
32618 9202 Acked Read 3 Done Done Sent false
32619 9203 Acked Write 6 Done Done Sent false
32620 9204 Acked Write 7 Done Done Sent false
32621 9205 Acked Flush 0 Done Done Sent false
32622 9206 Acked Read 1 Done Done Sent false
32623 9207 Acked Read 9 Done Done Sent false
32624 9208 Acked Write 7 Done Done Sent false
32625 9209 Acked Write 5 Done Done Sent false
32626 9210 Acked Write 1 Done Done Sent false
32627 9211 Acked Write 5 Done Done Sent false
32628 9212 Acked Read 2 Done Done Sent false
32629 9213 Acked Write 1 Done Done Sent false
32630 9214 Acked Write 9 Done Done Sent false
32631 9215 Acked Read 3 Done Done Sent false
32632 9216 Acked Read 2 Done Done Sent false
32633 9217 Acked Write 8 Done Done Sent false
32634 9218 Acked Read 10 Done Done Sent false
32635 9219 Acked Write 8 Done Done Sent false
32636 9220 Acked Read 3 Done Done Sent false
32637 9221 Acked Read 7 Done Done Sent false
32638 9222 Acked Flush 0 Done Done Sent false
32639 9223 Acked Write 7 Done Done Sent false
32640 9224 Acked Read 6 Done Done Sent false
32641 9225 Acked Flush 0 Done Done Sent false
32642 9226 Acked Read 2 Done Done Sent false
32643 9227 Acked Read 3 Done Done Sent false
32644 9228 Acked Flush 0 Done Done Sent false
32645 9229 Acked Write 5 Done Done Sent false
32646 9230 Acked Write 2 Done Done Sent false
32647 9231 Acked Read 1 Done Done Sent false
32648 9232 Acked Write 10 Done Done Sent false
32649 9233 Acked Read 9 Done Done Sent false
32650 9234 Acked Read 5 Done Done Sent false
32651 9235 Acked Read 6 Done Done Sent false
32652 9236 Acked Write 7 Done Done Sent false
32653 9237 Acked Read 4 Done Done Sent false
32654 9238 Acked Write 9 Done Done Sent false
32655 9239 Acked Read 6 Done Done Sent false
32656 9240 Acked Read 7 Done Done Sent false
32657 9241 Acked Write 5 Done Done Sent false
32658 9242 Acked Read 2 Done Done Sent false
32659 9243 Acked Write 8 Done Done Sent false
32660 9244 Acked Flush 0 Done Done Sent false
32661 9245 Acked Write 8 Done Done Sent false
32662 9246 Acked Read 5 Done Done Sent false
32663 9247 Acked Read 7 Done Done Sent false
32664 9248 Acked Flush 0 Done Done Sent false
32665 9249 Acked Read 3 Done Done Sent false
32666 9250 Acked Flush 0 Done Done Sent false
32667 9251 Acked Write 4 Done Done Sent false
32668 9252 Acked Read 1 Done Done Sent false
32669 9253 Acked Write 8 Done Done Sent false
32670 9254 Acked Write 6 Done Done Sent false
32671 9255 Acked Write 9 Done Done Sent false
32672 9256 Acked Write 5 Done Done Sent false
32673 9257 Acked Read 2 Done Done Sent false
32674 9258 Acked Write 10 Done Done Sent false
32675 9259 Acked Read 3 Done Done Sent false
32676 9260 Acked Read 9 Done Done Sent false
32677 9261 Acked Read 5 Done Done Sent false
32678 9262 Acked Read 8 Done Done Sent false
32679 9263 Acked Read 9 Done Done Sent false
32680 9264 Acked Read 8 Done Done Sent false
32681 9265 Acked Read 7 Done Done Sent false
32682 9266 Acked Read 5 Done Done Sent false
32683 9267 Acked Read 9 Done Done Sent false
32684 9268 Acked Write 8 Done Done Sent false
32685 9269 Acked Write 1 Done Done Sent false
32686 9270 Acked Read 7 Done Done Sent false
32687 9271 Acked Read 7 Done Done Sent false
32688 9272 Acked Read 3 Done Done Sent false
32689 9273 Acked Read 1 Done Done Sent false
32690 9274 Acked Read 6 Done Done Sent false
32691 9275 Acked Read 8 Done Done Sent false
32692 9276 Acked Read 4 Done Done Sent false
32693 9277 Acked Read 7 Done Done Sent false
32694 9278 Acked Read 2 Done Done Sent false
32695 9279 Acked Read 5 Done Done Sent false
32696 9280 Acked Read 6 Done Done Sent false
32697 9281 Acked Read 4 Done Done Sent false
32698 9282 Acked Read 5 Done Done Sent false
32699 9283 Acked Write 4 Done Done Sent false
32700 9284 Acked Read 5 Done Done Sent false
32701 9285 Acked Write 4 Done Done Sent false
32702 9286 Acked Flush 0 Done Done Sent false
32703 9287 Acked Read 10 Done Done Sent false
32704 9288 Acked Write 10 Done Done Sent false
32705 9289 Acked Write 4 Done Done Sent false
32706 9290 Acked Read 5 Done Done Sent false
32707 9291 Acked Write 7 Done Done Sent false
32708 9292 Acked Flush 0 Done Done Sent false
32709 9293 Acked Read 8 Done Done Sent false
32710 9294 Acked Read 9 Done Done Sent false
32711 9295 Acked Write 9 Done Done Sent false
32712 9296 Acked Write 10 Done Done Sent false
32713 9297 Acked Write 8 Done Done Sent false
32714 9298 Acked Read 8 Done Done Sent false
32715 9299 Acked Write 9 Done Done Sent false
32716 9300 Acked Write 9 Done Done Sent false
32717 9301 Acked Read 8 Done Done Sent false
32718 9302 Acked Write 7 Done Done Sent false
32719 9303 Acked Write 4 Done Done Sent false
32720 9304 Acked Read 3 Done Done Sent false
32721 9305 Acked Write 2 Done Done Sent false
32722 9306 Acked Write 5 Done Done Sent false
32723 9307 Acked Read 3 Done Done Sent false
32724 9308 Acked Read 7 Done Done Sent false
32725 9309 Acked Write 9 Done Done Sent false
32726 9310 Acked Write 6 Done Done Sent false
32727 9311 Acked Flush 0 Done Done Sent false
32728 9312 Acked Write 2 Done Done Sent false
32729 9313 Acked Read 5 Done Done Sent false
32730 9314 Acked Flush 0 Done Done Sent false
32731 9315 Acked Read 2 Done Done Sent false
32732 9316 Acked Write 1 Done Done Sent false
32733 9317 Acked Flush 0 Done Done Sent false
32734 9318 Acked Write 10 Done Done Sent false
32735 9319 Acked Write 10 Done Done Sent false
32736 9320 Acked Write 7 Done Done Sent false
32737 9321 Acked Read 3 Done Done Sent false
32738 9322 Acked Read 9 Done Done Sent false
32739 9323 Acked Write 9 Done Done Sent false
32740 9324 Acked Read 10 Done Done Sent false
32741 9325 Acked Write 9 Done Done Sent false
32742 9326 Acked Read 9 Done Done Sent false
32743 9327 Acked Read 9 Done Done Sent false
32744 9328 Acked Read 5 Done Done Sent false
32745 9329 Acked Write 3 Done Done Sent false
32746 9330 Acked Read 2 Done Done Sent false
32747 9331 Acked Read 8 Done Done Sent false
32748 9332 Acked Write 2 Done Done Sent false
32749 9333 Acked Write 2 Done Done Sent false
32750 9334 Acked Write 9 Done Done Sent false
32751 9335 Acked Write 3 Done Done Sent false
32752 9336 Acked Read 4 Done Done Sent false
32753 9337 Acked Write 9 Done Done Sent false
32754 9338 Acked Read 7 Done Done Sent false
32755 9339 Acked Write 8 Done Done Sent false
32756 9340 Acked Write 2 Done Done Sent false
32757 9341 Acked Read 6 Done Done Sent false
32758 9342 Acked Write 10 Done Done Sent false
32759 9343 Acked Read 3 Done Done Sent false
32760 9344 Acked Flush 0 Done Done Sent false
32761 9345 Acked Write 4 Done Done Sent false
32762 9346 Acked Read 9 Done Done Sent false
32763 9347 Acked Flush 0 Done Done Sent false
32764 9348 Acked Read 10 Done Done Sent false
32765 9349 Acked Read 2 Done Done Sent false
32766 9350 Acked Write 5 Done Done Sent false
32767 9351 Acked Write 4 Done Done Sent false
32768 9352 Acked Read 1 Done Done Sent false
32769 9353 Acked Flush 0 Done Done Sent false
32770 9354 Acked Read 6 Done Done Sent false
32771 9355 Acked Write 1 Done Done Sent false
32772 9356 Acked Read 2 Done Done Sent false
32773 9357 Acked Write 6 Done Done Sent false
32774 9358 Acked Read 1 Done Done Sent false
32775 9359 Acked Write 6 Done Done Sent false
32776 9360 Acked Write 7 Done Done Sent false
32777 9361 Acked Read 10 Done Done Sent false
32778 9362 Acked Read 9 Done Done Sent false
32779 9363 Acked Read 5 Done Done Sent false
32780 9364 Acked Write 9 Done Done Sent false
32781 9365 Acked Write 8 Done Done Sent false
32782 9366 Acked Flush 0 Done Done Sent false
32783 9367 Acked Read 5 Done Done Sent false
32784 9368 Acked Read 10 Done Done Sent false
32785 9369 Acked Read 10 Done Done Sent false
32786 9370 Acked Read 4 Done Done Sent false
32787 9371 Acked Read 10 Done Done Sent false
32788 9372 Acked Read 1 Done Done Sent false
32789 9373 Acked Read 2 Done Done Sent false
32790 9374 Acked Read 7 Done Done Sent false
32791 9375 Acked Write 4 Done Done Sent false
32792 9376 Acked Write 8 Done Done Sent false
32793 9377 Acked Flush 0 Done Done Sent false
32794 9378 Acked Read 9 Done Done Sent false
32795 9379 Acked Write 2 Done Done Sent false
32796 9380 Acked Flush 0 Done Done Sent false
32797 9381 Acked Write 2 Done Done Sent false
32798 9382 Acked Read 3 Done Done Sent false
32799 9383 Acked Read 9 Done Done Sent false
32800 9384 Acked Write 2 Done Done Sent false
32801 9385 Acked Read 7 Done Done Sent false
32802 9386 Acked Write 3 Done Done Sent false
32803 9387 Acked Write 1 Done Done Sent false
32804 9388 Acked Write 10 Done Done Sent false
32805 9389 Acked Read 6 Done Done Sent false
32806 9390 Acked Read 8 Done Done Sent false
32807 9391 Acked Write 5 Done Done Sent false
32808 9392 Acked Write 9 Done Done Sent false
32809 9393 Acked Read 8 Done Done Sent false
32810 9394 Acked Write 3 Done Done Sent false
32811 9395 Acked Flush 0 Done Done Sent false
32812 9396 Acked Read 8 Done Done Sent false
32813 9397 Acked Read 7 Done Done Sent false
32814 9398 Acked Write 9 Done Done Sent false
32815 9399 Acked Write 6 Done Done Sent false
32816 9400 Acked Read 9 Done Done Sent false
32817 9401 Acked Read 3 Done Done Sent false
32818 9402 Acked Read 8 Done Done Sent false
32819 9403 Acked Read 5 Done Done Sent false
32820 9404 Acked Write 5 Done Done Sent false
32821 9405 Acked Read 4 Done Done Sent false
32822 9406 Acked Write 3 Done Done Sent false
32823 9407 Acked Read 9 Done Done Sent false
32824 9408 Acked Write 8 Done Done Sent false
32825 9409 Acked Read 4 Done Done Sent false
32826 9410 Acked Read 3 Done Done Sent false
32827 9411 Acked Read 9 Done Done Sent false
32828 9412 Acked Read 1 Done Done Sent false
32829 9413 Acked Write 4 Done Done Sent false
32830 9414 Acked Flush 0 Done Done Sent false
32831 9415 Acked Flush 0 Done Done Sent false
32832 9416 Acked Write 8 Done Done Sent false
32833 9417 Acked Write 9 Done Done Sent false
32834 9418 Acked Read 7 Done Done Sent false
32835 9419 Acked Read 6 Done Done Sent false
32836 9420 Acked Read 6 Done Done Sent false
32837 9421 Acked Write 6 Done Done Sent false
32838 9422 Acked Read 9 Done Done Sent false
32839 9423 Acked Read 5 Done Done Sent false
32840 9424 Acked Read 3 Done Done Sent false
32841 9425 Acked Write 1 Done Done Sent false
32842 9426 Acked Read 4 Done Done Sent false
32843 9427 Acked Read 1 Done Done Sent false
32844 9428 Acked Write 6 Done Done Sent false
32845 9429 Acked Write 10 Done Done Sent false
32846 9430 Acked Read 2 Done Done Sent false
32847 9431 Acked Read 8 Done Done Sent false
32848 9432 Acked Read 9 Done Done Sent false
32849 9433 Acked Write 9 Done Done Sent false
32850 9434 Acked Write 8 Done Done Sent false
32851 9435 Acked Read 1 Done Done Sent false
32852 9436 Acked Read 4 Done Done Sent false
32853 9437 Acked Flush 0 Done Done Sent false
32854 9438 Acked Flush 0 Done Done Sent false
32855 9439 Acked Write 7 Done Done Sent false
32856 9440 Acked Read 3 Done Done Sent false
32857 9441 Acked Write 1 Done Done Sent false
32858 9442 Acked Read 7 Done Done Sent false
32859 9443 Acked Read 2 Done Done Sent false
32860 9444 Acked Read 10 Done Done Sent false
32861 9445 Acked Read 2 Done Done Sent false
32862 9446 Acked Read 7 Done Done Sent false
32863 9447 Acked Read 1 Done Done Sent false
32864 9448 Acked Read 6 Done Done Sent false
32865 9449 Acked Read 2 Done Done Sent false
32866 9450 Acked Read 4 Done Done Sent false
32867 9451 Acked Write 7 Done Done Sent false
32868 STATES DS:0 DS:1 DS:2 TOTAL
32869 Sent 0 0 301 301
32870 Done 301 301 0 602
32871 Skipped 0 0 0 0
32872 Error 0 0 0 0
32873 Last Flush: 9438 9438 9150
32874 Downstairs last five completed: 9150 9149 9148 9147 9146
32875 Upstairs last five completed: 9451 9450 9449 9448 9447
32876 CLIENT: Up:0 ds:301 act:2
32877 ----------------------------------------------------------------
32878 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
32879 States: Active Active Offline
32880 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
32881 9151 Acked Read 3 Done Done Sent false
32882 9152 Acked Write 2 Done Done Sent false
32883 9153 Acked Write 5 Done Done Sent false
32884 9154 Acked Read 4 Done Done Sent false
32885 9155 Acked Write 9 Done Done Sent false
32886 9156 Acked Write 4 Done Done Sent false
32887 9157 Acked Write 9 Done Done Sent false
32888 9158 Acked Read 7 Done Done Sent false
32889 9159 Acked Read 10 Done Done Sent false
32890 9160 Acked Read 9 Done Done Sent false
32891 9161 Acked Flush 0 Done Done Sent false
32892 9162 Acked Flush 0 Done Done Sent false
32893 9163 Acked Write 10 Done Done Sent false
32894 9164 Acked Read 4 Done Done Sent false
32895 9165 Acked Read 8 Done Done Sent false
32896 9166 Acked Read 3 Done Done Sent false
32897 9167 Acked Flush 0 Done Done Sent false
32898 9168 Acked Write 7 Done Done Sent false
32899 9169 Acked Write 3 Done Done Sent false
32900 9170 Acked Read 5 Done Done Sent false
32901 9171 Acked Write 10 Done Done Sent false
32902 9172 Acked Flush 0 Done Done Sent false
32903 9173 Acked Write 7 Done Done Sent false
32904 9174 Acked Write 10 Done Done Sent false
32905 9175 Acked Write 1 Done Done Sent false
32906 9176 Acked Write 3 Done Done Sent false
32907 9177 Acked Read 2 Done Done Sent false
32908 9178 Acked Read 4 Done Done Sent false
32909 9179 Acked Flush 0 Done Done Sent false
32910 9180 Acked Read 3 Done Done Sent false
32911 9181 Acked Read 7 Done Done Sent false
32912 9182 Acked Read 5 Done Done Sent false
32913 9183 Acked Read 8 Done Done Sent false
32914 9184 Acked Read 5 Done Done Sent false
32915 9185 Acked Write 6 Done Done Sent false
32916 9186 Acked Write 1 Done Done Sent false
32917 9187 Acked Write 10 Done Done Sent false
32918 9188 Acked Write 3 Done Done Sent false
32919 9189 Acked Read 9 Done Done Sent false
32920 9190 Acked Read 6 Done Done Sent false
32921 9191 Acked Flush 0 Done Done Sent false
32922 9192 Acked Read 10 Done Done Sent false
32923 9193 Acked Read 10 Done Done Sent false
32924 9194 Acked Write 7 Done Done Sent false
32925 9195 Acked Write 6 Done Done Sent false
32926 9196 Acked Read 7 Done Done Sent false
32927 9197 Acked Read 2 Done Done Sent false
32928 9198 Acked Write 3 Done Done Sent false
32929 9199 Acked Read 3 Done Done Sent false
32930 9200 Acked Write 5 Done Done Sent false
32931 9201 Acked Write 10 Done Done Sent false
32932 9202 Acked Read 3 Done Done Sent false
32933 9203 Acked Write 6 Done Done Sent false
32934 9204 Acked Write 7 Done Done Sent false
32935 9205 Acked Flush 0 Done Done Sent false
32936 9206 Acked Read 1 Done Done Sent false
32937 9207 Acked Read 9 Done Done Sent false
32938 9208 Acked Write 7 Done Done Sent false
32939 9209 Acked Write 5 Done Done Sent false
32940 9210 Acked Write 1 Done Done Sent false
32941 9211 Acked Write 5 Done Done Sent false
32942 9212 Acked Read 2 Done Done Sent false
32943 9213 Acked Write 1 Done Done Sent false
32944 9214 Acked Write 9 Done Done Sent false
32945 9215 Acked Read 3 Done Done Sent false
32946 9216 Acked Read 2 Done Done Sent false
32947 9217 Acked Write 8 Done Done Sent false
32948 9218 Acked Read 10 Done Done Sent false
32949 9219 Acked Write 8 Done Done Sent false
32950 9220 Acked Read 3 Done Done Sent false
32951 9221 Acked Read 7 Done Done Sent false
32952 9222 Acked Flush 0 Done Done Sent false
32953 9223 Acked Write 7 Done Done Sent false
32954 9224 Acked Read 6 Done Done Sent false
32955 9225 Acked Flush 0 Done Done Sent false
32956 9226 Acked Read 2 Done Done Sent false
32957 9227 Acked Read 3 Done Done Sent false
32958 9228 Acked Flush 0 Done Done Sent false
32959 9229 Acked Write 5 Done Done Sent false
32960 9230 Acked Write 2 Done Done Sent false
32961 9231 Acked Read 1 Done Done Sent false
32962 9232 Acked Write 10 Done Done Sent false
32963 9233 Acked Read 9 Done Done Sent false
32964 9234 Acked Read 5 Done Done Sent false
32965 9235 Acked Read 6 Done Done Sent false
32966 9236 Acked Write 7 Done Done Sent false
32967 9237 Acked Read 4 Done Done Sent false
32968 9238 Acked Write 9 Done Done Sent false
32969 9239 Acked Read 6 Done Done Sent false
32970 9240 Acked Read 7 Done Done Sent false
32971 9241 Acked Write 5 Done Done Sent false
32972 9242 Acked Read 2 Done Done Sent false
32973 9243 Acked Write 8 Done Done Sent false
32974 9244 Acked Flush 0 Done Done Sent false
32975 9245 Acked Write 8 Done Done Sent false
32976 9246 Acked Read 5 Done Done Sent false
32977 9247 Acked Read 7 Done Done Sent false
32978 9248 Acked Flush 0 Done Done Sent false
32979 9249 Acked Read 3 Done Done Sent false
32980 9250 Acked Flush 0 Done Done Sent false
32981 9251 Acked Write 4 Done Done Sent false
32982 9252 Acked Read 1 Done Done Sent false
32983 9253 Acked Write 8 Done Done Sent false
32984 9254 Acked Write 6 Done Done Sent false
32985 9255 Acked Write 9 Done Done Sent false
32986 9256 Acked Write 5 Done Done Sent false
32987 9257 Acked Read 2 Done Done Sent false
32988 9258 Acked Write 10 Done Done Sent false
32989 9259 Acked Read 3 Done Done Sent false
32990 9260 Acked Read 9 Done Done Sent false
32991 9261 Acked Read 5 Done Done Sent false
32992 9262 Acked Read 8 Done Done Sent false
32993 9263 Acked Read 9 Done Done Sent false
32994 9264 Acked Read 8 Done Done Sent false
32995 9265 Acked Read 7 Done Done Sent false
32996 9266 Acked Read 5 Done Done Sent false
32997 9267 Acked Read 9 Done Done Sent false
32998 9268 Acked Write 8 Done Done Sent false
32999 9269 Acked Write 1 Done Done Sent false
33000 9270 Acked Read 7 Done Done Sent false
33001 9271 Acked Read 7 Done Done Sent false
33002 9272 Acked Read 3 Done Done Sent false
33003 9273 Acked Read 1 Done Done Sent false
33004 9274 Acked Read 6 Done Done Sent false
33005 9275 Acked Read 8 Done Done Sent false
33006 9276 Acked Read 4 Done Done Sent false
33007 9277 Acked Read 7 Done Done Sent false
33008 9278 Acked Read 2 Done Done Sent false
33009 9279 Acked Read 5 Done Done Sent false
33010 9280 Acked Read 6 Done Done Sent false
33011 9281 Acked Read 4 Done Done Sent false
33012 9282 Acked Read 5 Done Done Sent false
33013 9283 Acked Write 4 Done Done Sent false
33014 9284 Acked Read 5 Done Done Sent false
33015 9285 Acked Write 4 Done Done Sent false
33016 9286 Acked Flush 0 Done Done Sent false
33017 9287 Acked Read 10 Done Done Sent false
33018 9288 Acked Write 10 Done Done Sent false
33019 9289 Acked Write 4 Done Done Sent false
33020 9290 Acked Read 5 Done Done Sent false
33021 9291 Acked Write 7 Done Done Sent false
33022 9292 Acked Flush 0 Done Done Sent false
33023 9293 Acked Read 8 Done Done Sent false
33024 9294 Acked Read 9 Done Done Sent false
33025 9295 Acked Write 9 Done Done Sent false
33026 9296 Acked Write 10 Done Done Sent false
33027 9297 Acked Write 8 Done Done Sent false
33028 9298 Acked Read 8 Done Done Sent false
33029 9299 Acked Write 9 Done Done Sent false
33030 9300 Acked Write 9 Done Done Sent false
33031 9301 Acked Read 8 Done Done Sent false
33032 9302 Acked Write 7 Done Done Sent false
33033 9303 Acked Write 4 Done Done Sent false
33034 9304 Acked Read 3 Done Done Sent false
33035 9305 Acked Write 2 Done Done Sent false
33036 9306 Acked Write 5 Done Done Sent false
33037 9307 Acked Read 3 Done Done Sent false
33038 9308 Acked Read 7 Done Done Sent false
33039 9309 Acked Write 9 Done Done Sent false
33040 9310 Acked Write 6 Done Done Sent false
33041 9311 Acked Flush 0 Done Done Sent false
33042 9312 Acked Write 2 Done Done Sent false
33043 9313 Acked Read 5 Done Done Sent false
33044 9314 Acked Flush 0 Done Done Sent false
33045 9315 Acked Read 2 Done Done Sent false
33046 9316 Acked Write 1 Done Done Sent false
33047 9317 Acked Flush 0 Done Done Sent false
33048 9318 Acked Write 10 Done Done Sent false
33049 9319 Acked Write 10 Done Done Sent false
33050 9320 Acked Write 7 Done Done Sent false
33051 9321 Acked Read 3 Done Done Sent false
33052 9322 Acked Read 9 Done Done Sent false
33053 9323 Acked Write 9 Done Done Sent false
33054 9324 Acked Read 10 Done Done Sent false
33055 9325 Acked Write 9 Done Done Sent false
33056 9326 Acked Read 9 Done Done Sent false
33057 9327 Acked Read 9 Done Done Sent false
33058 9328 Acked Read 5 Done Done Sent false
33059 9329 Acked Write 3 Done Done Sent false
33060 9330 Acked Read 2 Done Done Sent false
33061 9331 Acked Read 8 Done Done Sent false
33062 9332 Acked Write 2 Done Done Sent false
33063 9333 Acked Write 2 Done Done Sent false
33064 9334 Acked Write 9 Done Done Sent false
33065 9335 Acked Write 3 Done Done Sent false
33066 9336 Acked Read 4 Done Done Sent false
33067 9337 Acked Write 9 Done Done Sent false
33068 9338 Acked Read 7 Done Done Sent false
33069 9339 Acked Write 8 Done Done Sent false
33070 9340 Acked Write 2 Done Done Sent false
33071 9341 Acked Read 6 Done Done Sent false
33072 9342 Acked Write 10 Done Done Sent false
33073 9343 Acked Read 3 Done Done Sent false
33074 9344 Acked Flush 0 Done Done Sent false
33075 9345 Acked Write 4 Done Done Sent false
33076 9346 Acked Read 9 Done Done Sent false
33077 9347 Acked Flush 0 Done Done Sent false
33078 9348 Acked Read 10 Done Done Sent false
33079 9349 Acked Read 2 Done Done Sent false
33080 9350 Acked Write 5 Done Done Sent false
33081 9351 Acked Write 4 Done Done Sent false
33082 9352 Acked Read 1 Done Done Sent false
33083 9353 Acked Flush 0 Done Done Sent false
33084 9354 Acked Read 6 Done Done Sent false
33085 9355 Acked Write 1 Done Done Sent false
33086 9356 Acked Read 2 Done Done Sent false
33087 9357 Acked Write 6 Done Done Sent false
33088 9358 Acked Read 1 Done Done Sent false
33089 9359 Acked Write 6 Done Done Sent false
33090 9360 Acked Write 7 Done Done Sent false
33091 9361 Acked Read 10 Done Done Sent false
33092 9362 Acked Read 9 Done Done Sent false
33093 9363 Acked Read 5 Done Done Sent false
33094 9364 Acked Write 9 Done Done Sent false
33095 9365 Acked Write 8 Done Done Sent false
33096 9366 Acked Flush 0 Done Done Sent false
33097 9367 Acked Read 5 Done Done Sent false
33098 9368 Acked Read 10 Done Done Sent false
33099 9369 Acked Read 10 Done Done Sent false
33100 9370 Acked Read 4 Done Done Sent false
33101 9371 Acked Read 10 Done Done Sent false
33102 9372 Acked Read 1 Done Done Sent false
33103 9373 Acked Read 2 Done Done Sent false
33104 9374 Acked Read 7 Done Done Sent false
33105 9375 Acked Write 4 Done Done Sent false
33106 9376 Acked Write 8 Done Done Sent false
33107 9377 Acked Flush 0 Done Done Sent false
33108 9378 Acked Read 9 Done Done Sent false
33109 9379 Acked Write 2 Done Done Sent false
33110 9380 Acked Flush 0 Done Done Sent false
33111 9381 Acked Write 2 Done Done Sent false
33112 9382 Acked Read 3 Done Done Sent false
33113 9383 Acked Read 9 Done Done Sent false
33114 9384 Acked Write 2 Done Done Sent false
33115 9385 Acked Read 7 Done Done Sent false
33116 9386 Acked Write 3 Done Done Sent false
33117 9387 Acked Write 1 Done Done Sent false
33118 9388 Acked Write 10 Done Done Sent false
33119 9389 Acked Read 6 Done Done Sent false
33120 9390 Acked Read 8 Done Done Sent false
33121 9391 Acked Write 5 Done Done Sent false
33122 9392 Acked Write 9 Done Done Sent false
33123 9393 Acked Read 8 Done Done Sent false
33124 9394 Acked Write 3 Done Done Sent false
33125 9395 Acked Flush 0 Done Done Sent false
33126 9396 Acked Read 8 Done Done Sent false
33127 9397 Acked Read 7 Done Done Sent false
33128 9398 Acked Write 9 Done Done Sent false
33129 9399 Acked Write 6 Done Done Sent false
33130 9400 Acked Read 9 Done Done Sent false
33131 9401 Acked Read 3 Done Done Sent false
33132 9402 Acked Read 8 Done Done Sent false
33133 9403 Acked Read 5 Done Done Sent false
33134 9404 Acked Write 5 Done Done Sent false
33135 9405 Acked Read 4 Done Done Sent false
33136 9406 Acked Write 3 Done Done Sent false
33137 9407 Acked Read 9 Done Done Sent false
33138 9408 Acked Write 8 Done Done Sent false
33139 9409 Acked Read 4 Done Done Sent false
33140 9410 Acked Read 3 Done Done Sent false
33141 9411 Acked Read 9 Done Done Sent false
33142 9412 Acked Read 1 Done Done Sent false
33143 9413 Acked Write 4 Done Done Sent false
33144 9414 Acked Flush 0 Done Done Sent false
33145 9415 Acked Flush 0 Done Done Sent false
33146 9416 Acked Write 8 Done Done Sent false
33147 9417 Acked Write 9 Done Done Sent false
33148 9418 Acked Read 7 Done Done Sent false
33149 9419 Acked Read 6 Done Done Sent false
33150 9420 Acked Read 6 Done Done Sent false
33151 9421 Acked Write 6 Done Done Sent false
33152 9422 Acked Read 9 Done Done Sent false
33153 9423 Acked Read 5 Done Done Sent false
33154 9424 Acked Read 3 Done Done Sent false
33155 9425 Acked Write 1 Done Done Sent false
33156 9426 Acked Read 4 Done Done Sent false
33157 9427 Acked Read 1 Done Done Sent false
33158 9428 Acked Write 6 Done Done Sent false
33159 9429 Acked Write 10 Done Done Sent false
33160 9430 Acked Read 2 Done Done Sent false
33161 9431 Acked Read 8 Done Done Sent false
33162 9432 Acked Read 9 Done Done Sent false
33163 9433 Acked Write 9 Done Done Sent false
33164 9434 Acked Write 8 Done Done Sent false
33165 9435 Acked Read 1 Done Done Sent false
33166 9436 Acked Read 4 Done Done Sent false
33167 9437 Acked Flush 0 Done Done Sent false
33168 9438 Acked Flush 0 Done Done Sent false
33169 9439 Acked Write 7 Done Done Sent false
33170 9440 Acked Read 3 Done Done Sent false
33171 9441 Acked Write 1 Done Done Sent false
33172 9442 Acked Read 7 Done Done Sent false
33173 9443 Acked Read 2 Done Done Sent false
33174 9444 Acked Read 10 Done Done Sent false
33175 9445 Acked Read 2 Done Done Sent false
33176 9446 Acked Read 7 Done Done Sent false
33177 9447 Acked Read 1 Done Done Sent false
33178 9448 Acked Read 6 Done Done Sent false
33179 9449 Acked Read 2 Done Done Sent false
33180 9450 Acked Read 4 Done Done Sent false
33181 9451 Acked Write 7 Done Done Sent false
33182 9452 Acked Flush 0 Done Done Sent false
33183 STATES DS:0 DS:1 DS:2 TOTAL
33184 Sent 0 0 302 302
33185 Done 302 302 0 604
33186 Skipped 0 0 0 0
33187 Error 0 0 0 0
33188 Last Flush: 9452 9452 9150
33189 Downstairs last five completed: 9150 9149 9148 9147 9146
33190 Upstairs last five completed: 9452 9451 9450 9449 9448
33191 CLIENT: Up:0 ds:302 act:2
331922024-11-28T04:57:15.600ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
331932024-11-28T04:57:15.600ZWARNcrucible: ds_connection connect to 127.0.0.1:8830 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
331942024-11-28T04:57:15.600ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
331952024-11-28T04:57:15.600ZWARNcrucible: downstairs task for 2 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
331962024-11-28T04:57:15.600ZINFOcrucible: client task is exiting = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
33197 [8830] got start action
33198 [8830] I am going to restart
33199 Starting downstairs at port 8830
33200 Make output file at "/tmp/dsc/downstairs-8830.txt"
33201 [8830][2] reports Starting
33202 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1230)
33203 [8830][2] reports Running
33204 ----------------------------------------------------------------
33205 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
33206 States: Active Active Offline
33207 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
33208 9151 Acked Read 3 Done Done Sent false
33209 9152 Acked Write 2 Done Done Sent false
33210 9153 Acked Write 5 Done Done Sent false
33211 9154 Acked Read 4 Done Done Sent false
33212 9155 Acked Write 9 Done Done Sent false
33213 9156 Acked Write 4 Done Done Sent false
33214 9157 Acked Write 9 Done Done Sent false
33215 9158 Acked Read 7 Done Done Sent false
33216 9159 Acked Read 10 Done Done Sent false
33217 9160 Acked Read 9 Done Done Sent false
33218 9161 Acked Flush 0 Done Done Sent false
33219 9162 Acked Flush 0 Done Done Sent false
33220 9163 Acked Write 10 Done Done Sent false
33221 9164 Acked Read 4 Done Done Sent false
33222 9165 Acked Read 8 Done Done Sent false
33223 9166 Acked Read 3 Done Done Sent false
33224 9167 Acked Flush 0 Done Done Sent false
33225 9168 Acked Write 7 Done Done Sent false
33226 9169 Acked Write 3 Done Done Sent false
33227 9170 Acked Read 5 Done Done Sent false
33228 9171 Acked Write 10 Done Done Sent false
33229 9172 Acked Flush 0 Done Done Sent false
33230 9173 Acked Write 7 Done Done Sent false
33231 9174 Acked Write 10 Done Done Sent false
33232 9175 Acked Write 1 Done Done Sent false
33233 9176 Acked Write 3 Done Done Sent false
33234 9177 Acked Read 2 Done Done Sent false
33235 9178 Acked Read 4 Done Done Sent false
33236 9179 Acked Flush 0 Done Done Sent false
33237 9180 Acked Read 3 Done Done Sent false
33238 9181 Acked Read 7 Done Done Sent false
33239 9182 Acked Read 5 Done Done Sent false
33240 9183 Acked Read 8 Done Done Sent false
33241 9184 Acked Read 5 Done Done Sent false
33242 9185 Acked Write 6 Done Done Sent false
33243 9186 Acked Write 1 Done Done Sent false
33244 9187 Acked Write 10 Done Done Sent false
33245 9188 Acked Write 3 Done Done Sent false
33246 9189 Acked Read 9 Done Done Sent false
33247 9190 Acked Read 6 Done Done Sent false
33248 9191 Acked Flush 0 Done Done Sent false
33249 9192 Acked Read 10 Done Done Sent false
33250 9193 Acked Read 10 Done Done Sent false
33251 9194 Acked Write 7 Done Done Sent false
33252 9195 Acked Write 6 Done Done Sent false
33253 9196 Acked Read 7 Done Done Sent false
33254 9197 Acked Read 2 Done Done Sent false
33255 9198 Acked Write 3 Done Done Sent false
33256 9199 Acked Read 3 Done Done Sent false
33257 9200 Acked Write 5 Done Done Sent false
33258 9201 Acked Write 10 Done Done Sent false
33259 9202 Acked Read 3 Done Done Sent false
33260 9203 Acked Write 6 Done Done Sent false
33261 9204 Acked Write 7 Done Done Sent false
33262 9205 Acked Flush 0 Done Done Sent false
33263 9206 Acked Read 1 Done Done Sent false
33264 9207 Acked Read 9 Done Done Sent false
33265 9208 Acked Write 7 Done Done Sent false
33266 9209 Acked Write 5 Done Done Sent false
33267 9210 Acked Write 1 Done Done Sent false
33268 9211 Acked Write 5 Done Done Sent false
33269 9212 Acked Read 2 Done Done Sent false
33270 9213 Acked Write 1 Done Done Sent false
33271 9214 Acked Write 9 Done Done Sent false
33272 9215 Acked Read 3 Done Done Sent false
33273 9216 Acked Read 2 Done Done Sent false
33274 9217 Acked Write 8 Done Done Sent false
33275 9218 Acked Read 10 Done Done Sent false
33276 9219 Acked Write 8 Done Done Sent false
33277 9220 Acked Read 3 Done Done Sent false
33278 9221 Acked Read 7 Done Done Sent false
33279 9222 Acked Flush 0 Done Done Sent false
33280 9223 Acked Write 7 Done Done Sent false
33281 9224 Acked Read 6 Done Done Sent false
33282 9225 Acked Flush 0 Done Done Sent false
33283 9226 Acked Read 2 Done Done Sent false
33284 9227 Acked Read 3 Done Done Sent false
33285 9228 Acked Flush 0 Done Done Sent false
33286 9229 Acked Write 5 Done Done Sent false
33287 9230 Acked Write 2 Done Done Sent false
33288 9231 Acked Read 1 Done Done Sent false
33289 9232 Acked Write 10 Done Done Sent false
33290 9233 Acked Read 9 Done Done Sent false
33291 9234 Acked Read 5 Done Done Sent false
33292 9235 Acked Read 6 Done Done Sent false
33293 9236 Acked Write 7 Done Done Sent false
33294 9237 Acked Read 4 Done Done Sent false
33295 9238 Acked Write 9 Done Done Sent false
33296 9239 Acked Read 6 Done Done Sent false
33297 9240 Acked Read 7 Done Done Sent false
33298 9241 Acked Write 5 Done Done Sent false
33299 9242 Acked Read 2 Done Done Sent false
33300 9243 Acked Write 8 Done Done Sent false
33301 9244 Acked Flush 0 Done Done Sent false
33302 9245 Acked Write 8 Done Done Sent false
33303 9246 Acked Read 5 Done Done Sent false
33304 9247 Acked Read 7 Done Done Sent false
33305 9248 Acked Flush 0 Done Done Sent false
33306 9249 Acked Read 3 Done Done Sent false
33307 9250 Acked Flush 0 Done Done Sent false
33308 9251 Acked Write 4 Done Done Sent false
33309 9252 Acked Read 1 Done Done Sent false
33310 9253 Acked Write 8 Done Done Sent false
33311 9254 Acked Write 6 Done Done Sent false
33312 9255 Acked Write 9 Done Done Sent false
33313 9256 Acked Write 5 Done Done Sent false
33314 9257 Acked Read 2 Done Done Sent false
33315 9258 Acked Write 10 Done Done Sent false
33316 9259 Acked Read 3 Done Done Sent false
33317 9260 Acked Read 9 Done Done Sent false
33318 9261 Acked Read 5 Done Done Sent false
33319 9262 Acked Read 8 Done Done Sent false
33320 9263 Acked Read 9 Done Done Sent false
33321 9264 Acked Read 8 Done Done Sent false
33322 9265 Acked Read 7 Done Done Sent false
33323 9266 Acked Read 5 Done Done Sent false
33324 9267 Acked Read 9 Done Done Sent false
33325 9268 Acked Write 8 Done Done Sent false
33326 9269 Acked Write 1 Done Done Sent false
33327 9270 Acked Read 7 Done Done Sent false
33328 9271 Acked Read 7 Done Done Sent false
33329 9272 Acked Read 3 Done Done Sent false
33330 9273 Acked Read 1 Done Done Sent false
33331 9274 Acked Read 6 Done Done Sent false
33332 9275 Acked Read 8 Done Done Sent false
33333 9276 Acked Read 4 Done Done Sent false
33334 9277 Acked Read 7 Done Done Sent false
33335 9278 Acked Read 2 Done Done Sent false
33336 9279 Acked Read 5 Done Done Sent false
33337 9280 Acked Read 6 Done Done Sent false
33338 9281 Acked Read 4 Done Done Sent false
33339 9282 Acked Read 5 Done Done Sent false
33340 9283 Acked Write 4 Done Done Sent false
33341 9284 Acked Read 5 Done Done Sent false
33342 9285 Acked Write 4 Done Done Sent false
33343 9286 Acked Flush 0 Done Done Sent false
33344 9287 Acked Read 10 Done Done Sent false
33345 9288 Acked Write 10 Done Done Sent false
33346 9289 Acked Write 4 Done Done Sent false
33347 9290 Acked Read 5 Done Done Sent false
33348 9291 Acked Write 7 Done Done Sent false
33349 9292 Acked Flush 0 Done Done Sent false
33350 9293 Acked Read 8 Done Done Sent false
33351 9294 Acked Read 9 Done Done Sent false
33352 9295 Acked Write 9 Done Done Sent false
33353 9296 Acked Write 10 Done Done Sent false
33354 9297 Acked Write 8 Done Done Sent false
33355 9298 Acked Read 8 Done Done Sent false
33356 9299 Acked Write 9 Done Done Sent false
33357 9300 Acked Write 9 Done Done Sent false
33358 9301 Acked Read 8 Done Done Sent false
33359 9302 Acked Write 7 Done Done Sent false
33360 9303 Acked Write 4 Done Done Sent false
33361 9304 Acked Read 3 Done Done Sent false
33362 9305 Acked Write 2 Done Done Sent false
33363 9306 Acked Write 5 Done Done Sent false
33364 9307 Acked Read 3 Done Done Sent false
33365 9308 Acked Read 7 Done Done Sent false
33366 9309 Acked Write 9 Done Done Sent false
33367 9310 Acked Write 6 Done Done Sent false
33368 9311 Acked Flush 0 Done Done Sent false
33369 9312 Acked Write 2 Done Done Sent false
33370 9313 Acked Read 5 Done Done Sent false
33371 9314 Acked Flush 0 Done Done Sent false
33372 9315 Acked Read 2 Done Done Sent false
33373 9316 Acked Write 1 Done Done Sent false
33374 9317 Acked Flush 0 Done Done Sent false
33375 9318 Acked Write 10 Done Done Sent false
33376 9319 Acked Write 10 Done Done Sent false
33377 9320 Acked Write 7 Done Done Sent false
33378 9321 Acked Read 3 Done Done Sent false
33379 9322 Acked Read 9 Done Done Sent false
33380 9323 Acked Write 9 Done Done Sent false
33381 9324 Acked Read 10 Done Done Sent false
33382 9325 Acked Write 9 Done Done Sent false
33383 9326 Acked Read 9 Done Done Sent false
33384 9327 Acked Read 9 Done Done Sent false
33385 9328 Acked Read 5 Done Done Sent false
33386 9329 Acked Write 3 Done Done Sent false
33387 9330 Acked Read 2 Done Done Sent false
33388 9331 Acked Read 8 Done Done Sent false
33389 9332 Acked Write 2 Done Done Sent false
33390 9333 Acked Write 2 Done Done Sent false
33391 9334 Acked Write 9 Done Done Sent false
33392 9335 Acked Write 3 Done Done Sent false
33393 9336 Acked Read 4 Done Done Sent false
33394 9337 Acked Write 9 Done Done Sent false
33395 9338 Acked Read 7 Done Done Sent false
33396 9339 Acked Write 8 Done Done Sent false
33397 9340 Acked Write 2 Done Done Sent false
33398 9341 Acked Read 6 Done Done Sent false
33399 9342 Acked Write 10 Done Done Sent false
33400 9343 Acked Read 3 Done Done Sent false
33401 9344 Acked Flush 0 Done Done Sent false
33402 9345 Acked Write 4 Done Done Sent false
33403 9346 Acked Read 9 Done Done Sent false
33404 9347 Acked Flush 0 Done Done Sent false
33405 9348 Acked Read 10 Done Done Sent false
33406 9349 Acked Read 2 Done Done Sent false
33407 9350 Acked Write 5 Done Done Sent false
33408 9351 Acked Write 4 Done Done Sent false
33409 9352 Acked Read 1 Done Done Sent false
33410 9353 Acked Flush 0 Done Done Sent false
33411 9354 Acked Read 6 Done Done Sent false
33412 9355 Acked Write 1 Done Done Sent false
33413 9356 Acked Read 2 Done Done Sent false
33414 9357 Acked Write 6 Done Done Sent false
33415 9358 Acked Read 1 Done Done Sent false
33416 9359 Acked Write 6 Done Done Sent false
33417 9360 Acked Write 7 Done Done Sent false
33418 9361 Acked Read 10 Done Done Sent false
33419 9362 Acked Read 9 Done Done Sent false
33420 9363 Acked Read 5 Done Done Sent false
33421 9364 Acked Write 9 Done Done Sent false
33422 9365 Acked Write 8 Done Done Sent false
33423 9366 Acked Flush 0 Done Done Sent false
33424 9367 Acked Read 5 Done Done Sent false
33425 9368 Acked Read 10 Done Done Sent false
33426 9369 Acked Read 10 Done Done Sent false
33427 9370 Acked Read 4 Done Done Sent false
33428 9371 Acked Read 10 Done Done Sent false
33429 9372 Acked Read 1 Done Done Sent false
33430 9373 Acked Read 2 Done Done Sent false
33431 9374 Acked Read 7 Done Done Sent false
33432 9375 Acked Write 4 Done Done Sent false
33433 9376 Acked Write 8 Done Done Sent false
33434 9377 Acked Flush 0 Done Done Sent false
33435 9378 Acked Read 9 Done Done Sent false
33436 9379 Acked Write 2 Done Done Sent false
33437 9380 Acked Flush 0 Done Done Sent false
33438 9381 Acked Write 2 Done Done Sent false
33439 9382 Acked Read 3 Done Done Sent false
33440 9383 Acked Read 9 Done Done Sent false
33441 9384 Acked Write 2 Done Done Sent false
33442 9385 Acked Read 7 Done Done Sent false
33443 9386 Acked Write 3 Done Done Sent false
33444 9387 Acked Write 1 Done Done Sent false
33445 9388 Acked Write 10 Done Done Sent false
33446 9389 Acked Read 6 Done Done Sent false
33447 9390 Acked Read 8 Done Done Sent false
33448 9391 Acked Write 5 Done Done Sent false
33449 9392 Acked Write 9 Done Done Sent false
33450 9393 Acked Read 8 Done Done Sent false
33451 9394 Acked Write 3 Done Done Sent false
33452 9395 Acked Flush 0 Done Done Sent false
33453 9396 Acked Read 8 Done Done Sent false
33454 9397 Acked Read 7 Done Done Sent false
33455 9398 Acked Write 9 Done Done Sent false
33456 9399 Acked Write 6 Done Done Sent false
33457 9400 Acked Read 9 Done Done Sent false
33458 9401 Acked Read 3 Done Done Sent false
33459 9402 Acked Read 8 Done Done Sent false
33460 9403 Acked Read 5 Done Done Sent false
33461 9404 Acked Write 5 Done Done Sent false
33462 9405 Acked Read 4 Done Done Sent false
33463 9406 Acked Write 3 Done Done Sent false
33464 9407 Acked Read 9 Done Done Sent false
33465 9408 Acked Write 8 Done Done Sent false
33466 9409 Acked Read 4 Done Done Sent false
33467 9410 Acked Read 3 Done Done Sent false
33468 9411 Acked Read 9 Done Done Sent false
33469 9412 Acked Read 1 Done Done Sent false
33470 9413 Acked Write 4 Done Done Sent false
33471 9414 Acked Flush 0 Done Done Sent false
33472 9415 Acked Flush 0 Done Done Sent false
33473 9416 Acked Write 8 Done Done Sent false
33474 9417 Acked Write 9 Done Done Sent false
33475 9418 Acked Read 7 Done Done Sent false
33476 9419 Acked Read 6 Done Done Sent false
33477 9420 Acked Read 6 Done Done Sent false
33478 9421 Acked Write 6 Done Done Sent false
33479 9422 Acked Read 9 Done Done Sent false
33480 9423 Acked Read 5 Done Done Sent false
33481 9424 Acked Read 3 Done Done Sent false
33482 9425 Acked Write 1 Done Done Sent false
33483 9426 Acked Read 4 Done Done Sent false
33484 9427 Acked Read 1 Done Done Sent false
33485 9428 Acked Write 6 Done Done Sent false
33486 9429 Acked Write 10 Done Done Sent false
33487 9430 Acked Read 2 Done Done Sent false
33488 9431 Acked Read 8 Done Done Sent false
33489 9432 Acked Read 9 Done Done Sent false
33490 9433 Acked Write 9 Done Done Sent false
33491 9434 Acked Write 8 Done Done Sent false
33492 9435 Acked Read 1 Done Done Sent false
33493 9436 Acked Read 4 Done Done Sent false
33494 9437 Acked Flush 0 Done Done Sent false
33495 9438 Acked Flush 0 Done Done Sent false
33496 9439 Acked Write 7 Done Done Sent false
33497 9440 Acked Read 3 Done Done Sent false
33498 9441 Acked Write 1 Done Done Sent false
33499 9442 Acked Read 7 Done Done Sent false
33500 9443 Acked Read 2 Done Done Sent false
33501 9444 Acked Read 10 Done Done Sent false
33502 9445 Acked Read 2 Done Done Sent false
33503 9446 Acked Read 7 Done Done Sent false
33504 9447 Acked Read 1 Done Done Sent false
33505 9448 Acked Read 6 Done Done Sent false
33506 9449 Acked Read 2 Done Done Sent false
33507 9450 Acked Read 4 Done Done Sent false
33508 9451 Acked Write 7 Done Done Sent false
33509 9452 Acked Flush 0 Done Done Sent false
33510 STATES DS:0 DS:1 DS:2 TOTAL
33511 Sent 0 0 302 302
33512 Done 302 302 0 604
33513 Skipped 0 0 0 0
33514 Error 0 0 0 0
33515 Last Flush: 9452 9452 9150
33516 Downstairs last five completed: 9150 9149 9148 9147 9146
33517 Upstairs last five completed: 9452 9451 9450 9449 9448
33518 CLIENT: Up:0 ds:302 act:2
33519 ----------------------------------------------------------------
33520 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
33521 States: Active Active Offline
33522 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
33523 9151 Acked Read 3 Done Done Sent false
33524 9152 Acked Write 2 Done Done Sent false
33525 9153 Acked Write 5 Done Done Sent false
33526 9154 Acked Read 4 Done Done Sent false
33527 9155 Acked Write 9 Done Done Sent false
33528 9156 Acked Write 4 Done Done Sent false
33529 9157 Acked Write 9 Done Done Sent false
33530 9158 Acked Read 7 Done Done Sent false
33531 9159 Acked Read 10 Done Done Sent false
33532 9160 Acked Read 9 Done Done Sent false
33533 9161 Acked Flush 0 Done Done Sent false
33534 9162 Acked Flush 0 Done Done Sent false
33535 9163 Acked Write 10 Done Done Sent false
33536 9164 Acked Read 4 Done Done Sent false
33537 9165 Acked Read 8 Done Done Sent false
33538 9166 Acked Read 3 Done Done Sent false
33539 9167 Acked Flush 0 Done Done Sent false
33540 9168 Acked Write 7 Done Done Sent false
33541 9169 Acked Write 3 Done Done Sent false
33542 9170 Acked Read 5 Done Done Sent false
33543 9171 Acked Write 10 Done Done Sent false
33544 9172 Acked Flush 0 Done Done Sent false
33545 9173 Acked Write 7 Done Done Sent false
33546 9174 Acked Write 10 Done Done Sent false
33547 9175 Acked Write 1 Done Done Sent false
33548 9176 Acked Write 3 Done Done Sent false
33549 9177 Acked Read 2 Done Done Sent false
33550 9178 Acked Read 4 Done Done Sent false
33551 9179 Acked Flush 0 Done Done Sent false
33552 9180 Acked Read 3 Done Done Sent false
33553 9181 Acked Read 7 Done Done Sent false
33554 9182 Acked Read 5 Done Done Sent false
33555 9183 Acked Read 8 Done Done Sent false
33556 9184 Acked Read 5 Done Done Sent false
33557 9185 Acked Write 6 Done Done Sent false
33558 9186 Acked Write 1 Done Done Sent false
33559 9187 Acked Write 10 Done Done Sent false
33560 9188 Acked Write 3 Done Done Sent false
33561 9189 Acked Read 9 Done Done Sent false
33562 9190 Acked Read 6 Done Done Sent false
33563 9191 Acked Flush 0 Done Done Sent false
33564 9192 Acked Read 10 Done Done Sent false
33565 9193 Acked Read 10 Done Done Sent false
33566 9194 Acked Write 7 Done Done Sent false
33567 9195 Acked Write 6 Done Done Sent false
33568 9196 Acked Read 7 Done Done Sent false
33569 9197 Acked Read 2 Done Done Sent false
33570 9198 Acked Write 3 Done Done Sent false
33571 9199 Acked Read 3 Done Done Sent false
33572 9200 Acked Write 5 Done Done Sent false
33573 9201 Acked Write 10 Done Done Sent false
33574 9202 Acked Read 3 Done Done Sent false
33575 9203 Acked Write 6 Done Done Sent false
33576 9204 Acked Write 7 Done Done Sent false
33577 9205 Acked Flush 0 Done Done Sent false
33578 9206 Acked Read 1 Done Done Sent false
33579 9207 Acked Read 9 Done Done Sent false
33580 9208 Acked Write 7 Done Done Sent false
33581 9209 Acked Write 5 Done Done Sent false
33582 9210 Acked Write 1 Done Done Sent false
33583 9211 Acked Write 5 Done Done Sent false
33584 9212 Acked Read 2 Done Done Sent false
33585 9213 Acked Write 1 Done Done Sent false
33586 9214 Acked Write 9 Done Done Sent false
33587 9215 Acked Read 3 Done Done Sent false
33588 9216 Acked Read 2 Done Done Sent false
33589 9217 Acked Write 8 Done Done Sent false
33590 9218 Acked Read 10 Done Done Sent false
33591 9219 Acked Write 8 Done Done Sent false
33592 9220 Acked Read 3 Done Done Sent false
33593 9221 Acked Read 7 Done Done Sent false
33594 9222 Acked Flush 0 Done Done Sent false
33595 9223 Acked Write 7 Done Done Sent false
33596 9224 Acked Read 6 Done Done Sent false
33597 9225 Acked Flush 0 Done Done Sent false
33598 9226 Acked Read 2 Done Done Sent false
33599 9227 Acked Read 3 Done Done Sent false
33600 9228 Acked Flush 0 Done Done Sent false
33601 9229 Acked Write 5 Done Done Sent false
33602 9230 Acked Write 2 Done Done Sent false
33603 9231 Acked Read 1 Done Done Sent false
33604 9232 Acked Write 10 Done Done Sent false
33605 9233 Acked Read 9 Done Done Sent false
33606 9234 Acked Read 5 Done Done Sent false
33607 9235 Acked Read 6 Done Done Sent false
33608 9236 Acked Write 7 Done Done Sent false
33609 9237 Acked Read 4 Done Done Sent false
33610 9238 Acked Write 9 Done Done Sent false
33611 9239 Acked Read 6 Done Done Sent false
33612 9240 Acked Read 7 Done Done Sent false
33613 9241 Acked Write 5 Done Done Sent false
33614 9242 Acked Read 2 Done Done Sent false
33615 9243 Acked Write 8 Done Done Sent false
33616 9244 Acked Flush 0 Done Done Sent false
33617 9245 Acked Write 8 Done Done Sent false
33618 9246 Acked Read 5 Done Done Sent false
33619 9247 Acked Read 7 Done Done Sent false
33620 9248 Acked Flush 0 Done Done Sent false
33621 9249 Acked Read 3 Done Done Sent false
33622 9250 Acked Flush 0 Done Done Sent false
33623 9251 Acked Write 4 Done Done Sent false
33624 9252 Acked Read 1 Done Done Sent false
33625 9253 Acked Write 8 Done Done Sent false
33626 9254 Acked Write 6 Done Done Sent false
33627 9255 Acked Write 9 Done Done Sent false
33628 9256 Acked Write 5 Done Done Sent false
33629 9257 Acked Read 2 Done Done Sent false
33630 9258 Acked Write 10 Done Done Sent false
33631 9259 Acked Read 3 Done Done Sent false
33632 9260 Acked Read 9 Done Done Sent false
33633 9261 Acked Read 5 Done Done Sent false
33634 9262 Acked Read 8 Done Done Sent false
33635 9263 Acked Read 9 Done Done Sent false
33636 9264 Acked Read 8 Done Done Sent false
33637 9265 Acked Read 7 Done Done Sent false
33638 9266 Acked Read 5 Done Done Sent false
33639 9267 Acked Read 9 Done Done Sent false
33640 9268 Acked Write 8 Done Done Sent false
33641 9269 Acked Write 1 Done Done Sent false
33642 9270 Acked Read 7 Done Done Sent false
33643 9271 Acked Read 7 Done Done Sent false
33644 9272 Acked Read 3 Done Done Sent false
33645 9273 Acked Read 1 Done Done Sent false
33646 9274 Acked Read 6 Done Done Sent false
33647 9275 Acked Read 8 Done Done Sent false
33648 9276 Acked Read 4 Done Done Sent false
33649 9277 Acked Read 7 Done Done Sent false
33650 9278 Acked Read 2 Done Done Sent false
33651 9279 Acked Read 5 Done Done Sent false
33652 9280 Acked Read 6 Done Done Sent false
33653 9281 Acked Read 4 Done Done Sent false
33654 9282 Acked Read 5 Done Done Sent false
33655 9283 Acked Write 4 Done Done Sent false
33656 9284 Acked Read 5 Done Done Sent false
33657 9285 Acked Write 4 Done Done Sent false
33658 9286 Acked Flush 0 Done Done Sent false
33659 9287 Acked Read 10 Done Done Sent false
33660 9288 Acked Write 10 Done Done Sent false
33661 9289 Acked Write 4 Done Done Sent false
33662 9290 Acked Read 5 Done Done Sent false
33663 9291 Acked Write 7 Done Done Sent false
33664 9292 Acked Flush 0 Done Done Sent false
33665 9293 Acked Read 8 Done Done Sent false
33666 9294 Acked Read 9 Done Done Sent false
33667 9295 Acked Write 9 Done Done Sent false
33668 9296 Acked Write 10 Done Done Sent false
33669 9297 Acked Write 8 Done Done Sent false
33670 9298 Acked Read 8 Done Done Sent false
33671 9299 Acked Write 9 Done Done Sent false
33672 9300 Acked Write 9 Done Done Sent false
33673 9301 Acked Read 8 Done Done Sent false
33674 9302 Acked Write 7 Done Done Sent false
33675 9303 Acked Write 4 Done Done Sent false
33676 9304 Acked Read 3 Done Done Sent false
33677 9305 Acked Write 2 Done Done Sent false
33678 9306 Acked Write 5 Done Done Sent false
33679 9307 Acked Read 3 Done Done Sent false
33680 9308 Acked Read 7 Done Done Sent false
33681 9309 Acked Write 9 Done Done Sent false
33682 9310 Acked Write 6 Done Done Sent false
33683 9311 Acked Flush 0 Done Done Sent false
33684 9312 Acked Write 2 Done Done Sent false
33685 9313 Acked Read 5 Done Done Sent false
33686 9314 Acked Flush 0 Done Done Sent false
33687 9315 Acked Read 2 Done Done Sent false
33688 9316 Acked Write 1 Done Done Sent false
33689 9317 Acked Flush 0 Done Done Sent false
33690 9318 Acked Write 10 Done Done Sent false
33691 9319 Acked Write 10 Done Done Sent false
33692 9320 Acked Write 7 Done Done Sent false
33693 9321 Acked Read 3 Done Done Sent false
33694 9322 Acked Read 9 Done Done Sent false
33695 9323 Acked Write 9 Done Done Sent false
33696 9324 Acked Read 10 Done Done Sent false
33697 9325 Acked Write 9 Done Done Sent false
33698 9326 Acked Read 9 Done Done Sent false
33699 9327 Acked Read 9 Done Done Sent false
33700 9328 Acked Read 5 Done Done Sent false
33701 9329 Acked Write 3 Done Done Sent false
33702 9330 Acked Read 2 Done Done Sent false
33703 9331 Acked Read 8 Done Done Sent false
33704 9332 Acked Write 2 Done Done Sent false
33705 9333 Acked Write 2 Done Done Sent false
33706 9334 Acked Write 9 Done Done Sent false
33707 9335 Acked Write 3 Done Done Sent false
33708 9336 Acked Read 4 Done Done Sent false
33709 9337 Acked Write 9 Done Done Sent false
33710 9338 Acked Read 7 Done Done Sent false
33711 9339 Acked Write 8 Done Done Sent false
33712 9340 Acked Write 2 Done Done Sent false
33713 9341 Acked Read 6 Done Done Sent false
33714 9342 Acked Write 10 Done Done Sent false
33715 9343 Acked Read 3 Done Done Sent false
33716 9344 Acked Flush 0 Done Done Sent false
33717 9345 Acked Write 4 Done Done Sent false
33718 9346 Acked Read 9 Done Done Sent false
33719 9347 Acked Flush 0 Done Done Sent false
33720 9348 Acked Read 10 Done Done Sent false
33721 9349 Acked Read 2 Done Done Sent false
33722 9350 Acked Write 5 Done Done Sent false
33723 9351 Acked Write 4 Done Done Sent false
33724 9352 Acked Read 1 Done Done Sent false
33725 9353 Acked Flush 0 Done Done Sent false
33726 9354 Acked Read 6 Done Done Sent false
33727 9355 Acked Write 1 Done Done Sent false
33728 9356 Acked Read 2 Done Done Sent false
33729 9357 Acked Write 6 Done Done Sent false
33730 9358 Acked Read 1 Done Done Sent false
33731 9359 Acked Write 6 Done Done Sent false
33732 9360 Acked Write 7 Done Done Sent false
33733 9361 Acked Read 10 Done Done Sent false
33734 9362 Acked Read 9 Done Done Sent false
33735 9363 Acked Read 5 Done Done Sent false
33736 9364 Acked Write 9 Done Done Sent false
33737 9365 Acked Write 8 Done Done Sent false
33738 9366 Acked Flush 0 Done Done Sent false
33739 9367 Acked Read 5 Done Done Sent false
33740 9368 Acked Read 10 Done Done Sent false
33741 9369 Acked Read 10 Done Done Sent false
33742 9370 Acked Read 4 Done Done Sent false
33743 9371 Acked Read 10 Done Done Sent false
33744 9372 Acked Read 1 Done Done Sent false
33745 9373 Acked Read 2 Done Done Sent false
33746 9374 Acked Read 7 Done Done Sent false
33747 9375 Acked Write 4 Done Done Sent false
33748 9376 Acked Write 8 Done Done Sent false
33749 9377 Acked Flush 0 Done Done Sent false
33750 9378 Acked Read 9 Done Done Sent false
33751 9379 Acked Write 2 Done Done Sent false
33752 9380 Acked Flush 0 Done Done Sent false
33753 9381 Acked Write 2 Done Done Sent false
33754 9382 Acked Read 3 Done Done Sent false
33755 9383 Acked Read 9 Done Done Sent false
33756 9384 Acked Write 2 Done Done Sent false
33757 9385 Acked Read 7 Done Done Sent false
33758 9386 Acked Write 3 Done Done Sent false
33759 9387 Acked Write 1 Done Done Sent false
33760 9388 Acked Write 10 Done Done Sent false
33761 9389 Acked Read 6 Done Done Sent false
33762 9390 Acked Read 8 Done Done Sent false
33763 9391 Acked Write 5 Done Done Sent false
33764 9392 Acked Write 9 Done Done Sent false
33765 9393 Acked Read 8 Done Done Sent false
33766 9394 Acked Write 3 Done Done Sent false
33767 9395 Acked Flush 0 Done Done Sent false
33768 9396 Acked Read 8 Done Done Sent false
33769 9397 Acked Read 7 Done Done Sent false
33770 9398 Acked Write 9 Done Done Sent false
33771 9399 Acked Write 6 Done Done Sent false
33772 9400 Acked Read 9 Done Done Sent false
33773 9401 Acked Read 3 Done Done Sent false
33774 9402 Acked Read 8 Done Done Sent false
33775 9403 Acked Read 5 Done Done Sent false
33776 9404 Acked Write 5 Done Done Sent false
33777 9405 Acked Read 4 Done Done Sent false
33778 9406 Acked Write 3 Done Done Sent false
33779 9407 Acked Read 9 Done Done Sent false
33780 9408 Acked Write 8 Done Done Sent false
33781 9409 Acked Read 4 Done Done Sent false
33782 9410 Acked Read 3 Done Done Sent false
33783 9411 Acked Read 9 Done Done Sent false
33784 9412 Acked Read 1 Done Done Sent false
33785 9413 Acked Write 4 Done Done Sent false
33786 9414 Acked Flush 0 Done Done Sent false
33787 9415 Acked Flush 0 Done Done Sent false
33788 9416 Acked Write 8 Done Done Sent false
33789 9417 Acked Write 9 Done Done Sent false
33790 9418 Acked Read 7 Done Done Sent false
33791 9419 Acked Read 6 Done Done Sent false
33792 9420 Acked Read 6 Done Done Sent false
33793 9421 Acked Write 6 Done Done Sent false
33794 9422 Acked Read 9 Done Done Sent false
33795 9423 Acked Read 5 Done Done Sent false
33796 9424 Acked Read 3 Done Done Sent false
33797 9425 Acked Write 1 Done Done Sent false
33798 9426 Acked Read 4 Done Done Sent false
33799 9427 Acked Read 1 Done Done Sent false
33800 9428 Acked Write 6 Done Done Sent false
33801 9429 Acked Write 10 Done Done Sent false
33802 9430 Acked Read 2 Done Done Sent false
33803 9431 Acked Read 8 Done Done Sent false
33804 9432 Acked Read 9 Done Done Sent false
33805 9433 Acked Write 9 Done Done Sent false
33806 9434 Acked Write 8 Done Done Sent false
33807 9435 Acked Read 1 Done Done Sent false
33808 9436 Acked Read 4 Done Done Sent false
33809 9437 Acked Flush 0 Done Done Sent false
33810 9438 Acked Flush 0 Done Done Sent false
33811 9439 Acked Write 7 Done Done Sent false
33812 9440 Acked Read 3 Done Done Sent false
33813 9441 Acked Write 1 Done Done Sent false
33814 9442 Acked Read 7 Done Done Sent false
33815 9443 Acked Read 2 Done Done Sent false
33816 9444 Acked Read 10 Done Done Sent false
33817 9445 Acked Read 2 Done Done Sent false
33818 9446 Acked Read 7 Done Done Sent false
33819 9447 Acked Read 1 Done Done Sent false
33820 9448 Acked Read 6 Done Done Sent false
33821 9449 Acked Read 2 Done Done Sent false
33822 9450 Acked Read 4 Done Done Sent false
33823 9451 Acked Write 7 Done Done Sent false
33824 9452 Acked Flush 0 Done Done Sent false
33825 STATES DS:0 DS:1 DS:2 TOTAL
33826 Sent 0 0 302 302
33827 Done 302 302 0 604
33828 Skipped 0 0 0 0
33829 Error 0 0 0 0
33830 Last Flush: 9452 9452 9150
33831 Downstairs last five completed: 9150 9149 9148 9147 9146
33832 Upstairs last five completed: 9452 9451 9450 9449 9448
33833 CLIENT: Up:0 ds:302 act:2
338342024-11-28T04:57:25.601ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
338352024-11-28T04:57:25.601ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:57248) = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
338362024-11-28T04:57:25.602ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
338372024-11-28T04:57:25.603ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
338382024-11-28T04:57:25.603ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
338392024-11-28T04:57:25.603ZINFOcrucible: send last flush ID to this DS: 9150 = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
338402024-11-28T04:57:25.603ZINFOcrucible: Replied this last flush ID: 9150 = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
338412024-11-28T04:57:25.603ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
338422024-11-28T04:57:25.603ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
338432024-11-28T04:57:25.603ZINFOcrucible: [2] client re-new 302 jobs since flush 9150 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
338442024-11-28T04:57:25.603ZINFOcrucible: [2] Replayed 302 jobs since flush: 9150 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
33845 ----------------------------------------------------------------
33846 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
33847 Downstairs last five completed: 9452 9451 9450 9449 9448
33848 Upstairs last five completed: 9452 9451 9450 9449 9448
33849 CLIENT: Up:0 ds:0 act:3
33850 Replay: All jobs finished
33851 Main task has work to do, go find it
33852 got dsc Stop(2)
33853 stop 2
33854 Nov 28 04:57:25.854[8830] Got stop action so:false kr:false
33855 INFO request completed, latency_us: 346, response_code: 204, uri: /stop/cid/2, method: POST, req_id: f03fb5a8-45d7-433f-8e7e-5924a273bdc3, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
338562024-11-28T04:57:25.854ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
338572024-11-28T04:57:25.854ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
338582024-11-28T04:57:25.854ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
33859 Nov 28 04:57:25.854 INFO request completed, {"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-28T04:57:25.854798162Z","hostname":"w-01JDRHKEEFJ1228TFPSBEGXWHY","pid":1202,"client":"2","":"downstairs","session_id":"4a2415b1-6594-436f-82e5-b8cf9a74eeaf"}
33860 latency_us: 109, response_code{"msg":"[2] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-28T04:57:25.854822578Z","hostname":"w-01JDRHKEEFJ1228TFPSBEGXWHY","pid":1202,"client":"2","":"downstairs","session_id":"4a2415b1-6594-436f-82e5-b8cf9a74eeaf"}
33861 : 200, uri: /state/cid/2, method: GET, req_id: 61c25e04-fc73-4b5a-8517-27354fd1f988, remote_addr: 127.0.0{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2024-11-28T04:57:25.854865232Z","hostname":"w-01JDRHKEEFJ1228TFPSBEGXWHY","pid":1202,"":"io task","client":"2","":"downstairs","session_id":"4a2415b1-6594-436f-82e5-b8cf9a74eeaf"}
33862 .1:60418, local_addr: 127.0.0.1:9998
33863 [8830] Exited with: ExitStatus(unix_wait_status(9))
33864 [8830][2] reports Exit
33865 Nov 28 04:57:29.856 INFO request completed, latency_us: 224, response_code: 200, uri: /state/cid/2, method: GET, req_id: d04fd3c7-3874-41a6-b7e4-e391e930eecf, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
33866 001/300 Read block 4579 len 36864
33867 002/300 Write block 3904 len 4096 data: 8
33868 003/300 Read block 2684 len 32768
33869 004/300 Read block 4950 len 12288
33870 005/300 Write block 2043 len 28672 data: 3 4 4 5 6 5 5
33871 006/300 Flush
33872 007/300 Read block 967 len 12288
33873 008/300 Read block 1902 len 12288
33874 009/300 Read block 2097 len 32768
33875 010/300 Read block 3690 len 40960
33876 011/300 Write block 893 len 12288 data: 5 6 7
33877 012/300 Flush
33878 013/300 Read block 2795 len 28672
33879 014/300 Write block 4637 len 40960 data: 5 6 5 4 7 7 8 5 4 6
33880 015/300 Write block 3773 len 24576 data: 8 6 6 6 4 5
33881 016/300 Write block 2256 len 36864 data: 7 7 6 8 8 8 7 6 5
33882 017/300 Read block 424 len 40960
33883 018/300 Write block 1163 len 4096 data: 4
33884 019/300 Write block 1456 len 24576 data: 3 3 4 6 6 7
33885 020/300 Write block 2298 len 4096 data: 7
33886 021/300 Read block 761 len 28672
33887 022/300 Read block 3522 len 24576
33888 023/300 Write block 1815 len 24576 data: 6 6 6 6 8 8
33889 024/300 Read block 1756 len 40960
33890 025/300 Read block 2445 len 40960
33891 026/300 Flush
33892 027/300 Read block 332 len 12288
33893 028/300 Write block 2097 len 16384 data: 7 6 5 6
33894 029/300 Read block 4690 len 4096
33895 030/300 Read block 1068 len 40960
33896 031/300 Flush
33897 032/300 Write block 1670 len 40960 data: 5 5 7 6 7 7 6 6 6 6
33898 033/300 Write block 2727 len 4096 data: 3
33899 034/300 Read block 1979 len 24576
33900 035/300 Write block 1090 len 28672 data: 7 7 9 7 8 7 8
33901 036/300 Write block 304 len 32768 data: 6 6 7 7 6 5 6 6
33902 037/300 Read block 4044 len 20480
33903 038/300 Read block 1474 len 16384
33904 039/300 Read block 4109 len 16384
33905 040/300 Read block 3451 len 16384
33906 041/300 Write block 4729 len 4096 data: 5
33907 042/300 Read block 103 len 36864
33908 043/300 Write block 4068 len 16384 data: 9 7 7 8
33909 044/300 Write block 2828 len 32768 data: 4 7 7 7 7 5 4 3
33910 045/300 Write block 32 len 28672 data: 8 8 9 9 10 11 10
33911 046/300 Write block 324 len 4096 data: 8
33912 047/300 Read block 1938 len 12288
33913 048/300 Read block 4372 len 32768
33914 049/300 Read block 889 len 16384
33915 050/300 Write block 3818 len 32768 data: 7 9 10 9 8 8 7 6
33916 051/300 Write block 4088 len 36864 data: 7 8 10 11 12 11 10 9 9
33917 052/300 Write block 590 len 4096 data: 8
33918 053/300 Write block 1644 len 8192 data: 5 3
33919 054/300 Write block 4646 len 8192 data: 7 5
33920 055/300 Read block 3212 len 32768
33921 056/300 Flush
33922 057/300 Write block 3881 len 28672 data: 4 4 6 5 4 5 6
33923 058/300 Write block 2410 len 16384 data: 7 6 7 7
33924 059/300 Read block 105 len 4096
33925 060/300 Read block 1214 len 28672
33926 061/300 Read block 4603 len 8192
33927 062/300 Flush
33928 063/300 Read block 944 len 32768
33929 064/300 Write block 4100 len 20480 data: 3 2 4 4 5
33930 065/300 Read block 4506 len 20480
33931 066/300 Write block 4982 len 16384 data: 6 6 6 6
33932 067/300 Write block 2477 len 32768 data: 5 6 6 5 4 4 5 5
33933 068/300 Read block 827 len 32768
33934 069/300 Write block 13 len 32768 data: 7 9 9 9 9 7 5 6
33935 070/300 Write block 3993 len 40960 data: 5 5 6 5 6 4 4 4 3 4
33936 071/300 Read block 3915 len 24576
33937 072/300 Flush
33938 073/300 Flush
33939 074/300 Read block 2048 len 4096
33940 075/300 Flush
33941 076/300 Flush
33942 077/300 Read block 4392 len 28672
33943 078/300 Read block 4705 len 24576
33944 079/300 Read block 519 len 8192
33945 080/300 Read block 1239 len 36864
33946 081/300 Read block 4628 len 20480
33947 082/300 Flush
33948 083/300 Read block 3548 len 20480
33949 084/300 Write block 4258 len 12288 data: 3 4 5
33950 085/300 Write block 413 len 4096 data: 8
33951 086/300 Write block 4556 len 40960 data: 4 5 4 4 5 6 7 5 7 8
33952 087/300 Read block 4626 len 12288
33953 088/300 Flush
33954 089/300 Read block 3592 len 20480
33955 090/300 Read block 1692 len 8192
33956 091/300 Read block 4300 len 24576
33957 092/300 Write block 2022 len 16384 data: 6 7 7 5
33958 093/300 Read block 966 len 12288
33959 094/300 Write block 4045 len 40960 data: 4 4 3 4 5 5 5 6 6 8
33960 095/300 Read block 4151 len 12288
33961 096/300 Write block 2943 len 4096 data: 6
33962 097/300 Read block 4245 len 32768
33963 098/300 Write block 4616 len 4096 data: 2
33964 099/300 Write block 4984 len 16384 data: 7 7 6 7
33965 100/300 Write block 247 len 16384 data: 6 5 5 5
33966 101/300 Read block 1739 len 24576
33967 102/300 Read block 3081 len 24576
33968 103/300 Read block 1138 len 24576
33969 104/300 Write block 4383 len 28672 data: 5 6 5 5 7 7 7
33970 105/300 Write block 4134 len 40960 data: 5 5 5 5 6 6 8 9 7 6
33971 106/300 Read block 1185 len 12288
33972 107/300 Read block 4762 len 20480
33973 108/300 Write block 2049 len 40960 data: 6 5 6 6 6 7 7 7 6 6
33974 109/300 Write block 2278 len 24576 data: 7 7 6 6 7 7
33975 110/300 Read block 1483 len 8192
33976 111/300 Read block 410 len 28672
33977 112/300 Write block 2153 len 28672 data: 2 4 4 4 6 7 5
33978 113/300 Read block 4751 len 16384
33979 114/300 Read block 2325 len 28672
33980 115/300 Read block 1473 len 28672
33981 116/300 Read block 4169 len 16384
33982 117/300 Write block 2767 len 4096 data: 3
33983 118/300 Flush
33984 119/300 Write block 3770 len 32768 data: 9 8 8 9 7 7 7 5
33985 120/300 Write block 2785 len 28672 data: 10 9 11 13 13 14 10
33986 121/300 Read block 2382 len 28672
33987 122/300 Read block 999 len 20480
33988 123/300 Write block 1813 len 24576 data: 4 5 7 7 7 7
33989 124/300 Read block 1948 len 20480
33990 125/300 Read block 4023 len 24576
33991 126/300 Read block 334 len 20480
33992 127/300 Write block 4869 len 8192 data: 7 9
33993 128/300 Read block 1801 len 36864
33994 129/300 Read block 2839 len 28672
33995 130/300 Read block 3685 len 4096
33996 131/300 Write block 2889 len 16384 data: 4 5 5 5
33997 132/300 Write block 2203 len 36864 data: 5 6 5 6 9 10 9 10 9
33998 133/300 Flush
33999 134/300 Write block 1259 len 20480 data: 4 5 7 7 7
34000 135/300 Write block 129 len 36864 data: 6 5 5 5 5 5 7 8 8
34001 136/300 Read block 1606 len 28672
34002 137/300 Write block 2884 len 12288 data: 6 4 3
34003 138/300 Write block 2769 len 16384 data: 5 6 7 7
34004 139/300 Read block 2963 len 8192
34005 140/300 Read block 1084 len 24576
34006 141/300 Write block 2332 len 20480 data: 8 9 10 8 8
34007 142/300 Read block 743 len 16384
34008 143/300 Read block 4372 len 24576
34009 144/300 Write block 654 len 32768 data: 6 7 8 9 7 10 9 9
34010 145/300 Read block 856 len 20480
34011 146/300 Read block 1989 len 12288
34012 147/300 Read block 2002 len 28672
34013 148/300 Read block 4255 len 24576
34014 149/300 Write block 449 len 20480 data: 7 4 5 4 3
34015 150/300 Write block 378 len 16384 data: 9 10 10 10
34016 151/300 Read block 1587 len 36864
34017 152/300 Read block 2959 len 36864
34018 153/300 Read block 2423 len 40960
34019 154/300 Write block 4960 len 36864 data: 7 7 8 8 9 9 10 10 10
34020 155/300 Read block 550 len 36864
34021 156/300 Read block 2656 len 20480
34022 157/300 Read block 298 len 20480
34023 158/300 Read block 2302 len 8192
34024 159/300 Write block 3845 len 12288 data: 9 9 8
34025 160/300 Read block 3901 len 28672
34026 161/300 Read block 608 len 16384
34027 162/300 Write block 4314 len 36864 data: 5 5 5 4 4 4 5 5 6
34028 163/300 Read block 995 len 16384
34029 164/300 Flush
34030 165/300 Write block 4745 len 28672 data: 3 2 3 3 5 4 4
34031 166/300 Read block 1078 len 8192
34032 167/300 Write block 4958 len 20480 data: 7 8 8 8 9
34033 168/300 Read block 3150 len 28672
34034 169/300 Write block 139 len 32768 data: 7 8 7 7 7 6 3 3
34035 170/300 Write block 399 len 4096 data: 5
34036 171/300 Write block 635 len 40960 data: 8 10 9 8 8 6 6 6 7 7
34037 172/300 Read block 2522 len 4096
34038 173/300 Write block 3922 len 40960 data: 5 6 5 9 6 6 5 5 7 7
34039 174/300 Read block 4700 len 28672
34040 175/300 Read block 4828 len 20480
34041 176/300 Write block 2460 len 32768 data: 5 4 5 6 5 5 5 6
34042 177/300 Read block 321 len 40960
34043 178/300 Read block 2539 len 36864
34044 179/300 Write block 2371 len 16384 data: 6 6 9 8
34045 180/300 Write block 3309 len 28672 data: 9 11 12 12 11 13 12
34046 181/300 Read block 2168 len 12288
34047 182/300 Read block 2453 len 40960
34048 183/300 Flush
34049 184/300 Read block 3015 len 36864
34050 185/300 Write block 279 len 16384 data: 8 7 5 4
34051 186/300 Write block 977 len 16384 data: 5 6 5 5
34052 187/300 Write block 3149 len 12288 data: 3 6 5
34053 188/300 Write block 2763 len 28672 data: 4 4 4 4 4 5 6
34054 189/300 Read block 516 len 12288
34055 190/300 Write block 4811 len 32768 data: 3 3 3 2 3 4 4 3
34056 191/300 Read block 123 len 16384
34057 192/300 Write block 773 len 4096 data: 8
34058 193/300 Write block 2839 len 12288 data: 3 3 3
34059 194/300 Write block 3224 len 4096 data: 6
34060 195/300 Read block 132 len 16384
34061 196/300 Read block 678 len 40960
34062 197/300 Read block 2240 len 4096
34063 198/300 Write block 3095 len 16384 data: 5 4 4 3
34064 199/300 Read block 1355 len 4096
34065 200/300 Read block 4056 len 40960
34066 201/300 Read block 3187 len 24576
34067 202/300 Read block 443 len 16384
34068 203/300 Read block 3106 len 4096
34069 204/300 Read block 1400 len 20480
34070 205/300 Read block 826 len 20480
34071 206/300 Write block 3327 len 16384 data: 5 6 7 7
34072 207/300 Read block 1592 len 8192
34073 208/300 Write block 1853 len 40960 data: 6 6 6 6 6 6 5 6 7 5
34074 209/300 Flush
34075 210/300 Read block 507 len 12288
34076 211/300 Read block 4816 len 36864
34077 212/300 Write block 1336 len 12288 data: 8 8 7
34078 213/300 Write block 3739 len 4096 data: 5
34079 214/300 Write block 4028 len 32768 data: 7 7 6 6 6 8 6 5
34080 215/300 Read block 791 len 32768
34081 216/300 Read block 657 len 24576
34082 217/300 Read block 3994 len 40960
34083 218/300 Read block 2709 len 8192
34084 219/300 Write block 4280 len 24576 data: 5 7 8 8 5 5
34085 220/300 Read block 3180 len 16384
34086 221/300 Read block 4894 len 4096
34087 222/300 Read block 265 len 12288
34088 223/300 Write block 2565 len 36864 data: 6 7 9 10 10 10 11 8 7
34089 224/300 Read block 41 len 32768
34090 225/300 Read block 2323 len 28672
34091 226/300 Write block 3113 len 28672 data: 3 3 4 5 6 5 6
34092 227/300 Read block 3878 len 40960
34093 228/300 Read block 3322 len 12288
34094 229/300 Read block 1498 len 40960
34095 230/300 Write block 4078 len 40960 data: 11 13 11 12 8 7 9 6 6 7
34096 231/300 Read block 4046 len 20480
34097 232/300 Write block 1197 len 28672 data: 8 7 5 5 6 5 4
34098 233/300 Read block 2850 len 20480
34099 234/300 Read block 1679 len 4096
34100 235/300 Write block 844 len 36864 data: 6 7 8 8 10 10 9 9 10
34101 236/300 Write block 1543 len 24576 data: 6 6 9 9 8 9
34102 237/300 Read block 983 len 28672
34103 238/300 Read block 291 len 16384
34104 239/300 Read block 3860 len 36864
34105 240/300 Read block 4314 len 36864
34106 241/300 Write block 2400 len 12288 data: 5 3 4
34107 242/300 Read block 3099 len 8192
34108 243/300 Read block 5 len 4096
34109 244/300 Flush
34110 245/300 Read block 3135 len 24576
34111 246/300 Write block 1875 len 4096 data: 4
34112 247/300 Read block 4806 len 40960
34113 248/300 Write block 2494 len 28672 data: 4 4 4 4 5 5 5
34114 249/300 Read block 815 len 40960
34115 250/300 Write block 1079 len 24576 data: 6 5 6 7 7 7
34116 251/300 Write block 2992 len 12288 data: 4 2 3
34117 252/300 Flush
34118 253/300 Flush
34119 254/300 Read block 3511 len 32768
34120 255/300 Write block 1813 len 20480 data: 5 6 8 8 8
34121 256/300 Read block 4606 len 4096
34122 257/300 Write block 2331 len 16384 data: 9 9 10 11
34123 258/300 Write block 729 len 16384 data: 5 6 6 6
34124 259/300 Flush
34125 260/300 Read block 4823 len 40960
34126 261/300 Read block 760 len 36864
34127 262/300 Read block 948 len 16384
34128 263/300 Flush
34129 264/300 Write block 4376 len 4096 data: 8
34130 265/300 Flush
34131 266/300 Flush
34132 267/300 Read block 3754 len 12288
34133 268/300 Write block 477 len 8192 data: 6 4
34134 269/300 Write block 2823 len 16384 data: 2 3 4 4
34135 270/300 Read block 4894 len 32768
34136 271/300 Read block 1050 len 32768
34137 272/300 Write block 2101 len 36864 data: 5 6 6 5 6 4 4 4 5
34138 273/300 Write block 2287 len 4096 data: 6
34139 274/300 Read block 8 len 24576
34140 275/300 Read block 4068 len 32768
34141 276/300 Read block 2718 len 8192
34142 277/300 Read block 3494 len 16384
34143 278/300 Read block 1033 len 32768
34144 279/300 Write block 4351 len 12288 data: 6 7 7
34145 280/300 Read block 1774 len 4096
34146 281/300 Write block 2955 len 24576 data: 4 4 5 6 7 8
34147 282/300 Read block 4568 len 24576
34148 283/300 Write block 4209 len 24576 data: 5 6 4 4 5 2
34149 284/300 Read block 1569 len 24576
34150 285/300 Write block 4468 len 28672 data: 6 5 7 7 7 7 7
34151 286/300 Read block 4527 len 28672
34152 287/300 Read block 446 len 12288
34153 288/300 Read block 1763 len 24576
34154 289/300 Write block 2494 len 4096 data: 5
34155 290/300 Write block 1589 len 4096 data: 4
34156 291/300 Write block 1249 len 4096 data: 4
34157 292/300 Read block 849 len 24576
34158 293/300 Read block 2357 len 4096
34159 294/300 Flush
34160 295/300 Read block 1976 len 28672
34161 296/300 Write block 3033 len 8192 data: 8 8
34162 297/300 Read block 4794 len 8192
34163 298/300 Write block 2471 len 36864 data: 9 8 7 7 8 9 6 7 7
34164 299/300 Read block 4387 len 4096
34165 300/300 Read block 2099 len 8192
34166 301/300 Write block 1038 len 32768 data: 3 3 4 4 4 4 4 5
34167 Main task has work to do, go find it
34168 got dsc Start(2)
34169 start 2
34170 Nov 28 04:57:30.098 INFO request completed, latency_us: 220, response_code: 204, uri: /start/cid/2, method: POST, req_id: 68183765-af18-4a0b-90d7-84e02fa3702d, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
34171 Replay: started 2, returned:Ok(())
34172 ----------------------------------------------------------------
34173 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
34174 States: Active Active Offline
34175 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
34176 9453 Acked Read 9 Done Done Sent false
34177 9454 Acked Write 1 Done Done Sent false
34178 9455 Acked Read 8 Done Done Sent false
34179 9456 Acked Read 3 Done Done Sent false
34180 9457 Acked Write 7 Done Done Sent false
34181 9458 Acked Flush 0 Done Done Sent false
34182 9459 Acked Read 3 Done Done Sent false
34183 9460 Acked Read 3 Done Done Sent false
34184 9461 Acked Read 8 Done Done Sent false
34185 9462 Acked Read 10 Done Done Sent false
34186 9463 Acked Write 3 Done Done Sent false
34187 9464 Acked Flush 0 Done Done Sent false
34188 9465 Acked Read 7 Done Done Sent false
34189 9466 Acked Write 10 Done Done Sent false
34190 9467 Acked Write 6 Done Done Sent false
34191 9468 Acked Write 9 Done Done Sent false
34192 9469 Acked Read 10 Done Done Sent false
34193 9470 Acked Write 1 Done Done Sent false
34194 9471 Acked Write 6 Done Done Sent false
34195 9472 Acked Write 1 Done Done Sent false
34196 9473 Acked Read 7 Done Done Sent false
34197 9474 Acked Read 6 Done Done Sent false
34198 9475 Acked Write 6 Done Done Sent false
34199 9476 Acked Read 10 Done Done Sent false
34200 9477 Acked Read 10 Done Done Sent false
34201 9478 Acked Flush 0 Done Done Sent false
34202 9479 Acked Read 3 Done Done Sent false
34203 9480 Acked Write 4 Done Done Sent false
34204 9481 Acked Read 1 Done Done Sent false
34205 9482 Acked Read 10 Done Done Sent false
34206 9483 Acked Flush 0 Done Done Sent false
34207 9484 Acked Write 10 Done Done Sent false
34208 9485 Acked Write 1 Done Done Sent false
34209 9486 Acked Read 6 Done Done Sent false
34210 9487 Acked Write 7 Done Done Sent false
34211 9488 Acked Write 8 Done Done Sent false
34212 9489 Acked Read 5 Done Done Sent false
34213 9490 Acked Read 4 Done Done Sent false
34214 9491 Acked Read 4 Done Done Sent false
34215 9492 Acked Read 4 Done Done Sent false
34216 9493 Acked Write 1 Done Done Sent false
34217 9494 Acked Read 9 Done Done Sent false
34218 9495 Acked Write 4 Done Done Sent false
34219 9496 Acked Write 8 Done Done Sent false
34220 9497 Acked Write 7 Done Done Sent false
34221 9498 Acked Write 1 Done Done Sent false
34222 9499 Acked Read 3 Done Done Sent false
34223 9500 Acked Read 8 Done Done Sent false
34224 9501 Acked Read 4 Done Done Sent false
34225 9502 Acked Write 8 Done Done Sent false
34226 9503 Acked Write 9 Done Done Sent false
34227 9504 Acked Write 1 Done Done Sent false
34228 9505 Acked Write 2 Done Done Sent false
34229 9506 Acked Write 2 Done Done Sent false
34230 9507 Acked Read 8 Done Done Sent false
34231 9508 Acked Flush 0 Done Done Sent false
34232 9509 Acked Write 7 Done Done Sent false
34233 9510 Acked Write 4 Done Done Sent false
34234 9511 Acked Read 1 Done Done Sent false
34235 9512 Acked Read 7 Done Done Sent false
34236 9513 Acked Read 2 Done Done Sent false
34237 9514 Acked Flush 0 Done Done Sent false
34238 9515 Acked Read 8 Done Done Sent false
34239 9516 Acked Write 5 Done Done Sent false
34240 9517 Acked Read 5 Done Done Sent false
34241 9518 Acked Write 4 Done Done Sent false
34242 9519 Acked Write 8 Done Done Sent false
34243 9520 Acked Read 8 Done Done Sent false
34244 9521 Acked Write 8 Done Done Sent false
34245 9522 Acked Write 10 Done Done Sent false
34246 9523 Acked Read 6 Done Done Sent false
34247 9524 Acked Flush 0 Done Done Sent false
34248 9525 Acked Flush 0 Done Done Sent false
34249 9526 Acked Read 1 Done Done Sent false
34250 9527 Acked Flush 0 Done Done Sent false
34251 9528 Acked Flush 0 Done Done Sent false
34252 9529 Acked Read 7 Done Done Sent false
34253 9530 Acked Read 6 Done Done Sent false
34254 9531 Acked Read 2 Done Done Sent false
34255 9532 Acked Read 9 Done Done Sent false
34256 9533 Acked Read 5 Done Done Sent false
34257 9534 Acked Flush 0 Done Done Sent false
34258 9535 Acked Read 5 Done Done Sent false
34259 9536 Acked Write 3 Done Done Sent false
34260 9537 Acked Write 1 Done Done Sent false
34261 9538 Acked Write 10 Done Done Sent false
34262 9539 Acked Read 3 Done Done Sent false
34263 9540 Acked Flush 0 Done Done Sent false
34264 9541 Acked Read 5 Done Done Sent false
34265 9542 Acked Read 2 Done Done Sent false
34266 9543 Acked Read 6 Done Done Sent false
34267 9544 Acked Write 4 Done Done Sent false
34268 9545 Acked Read 3 Done Done Sent false
34269 9546 Acked Write 10 Done Done Sent false
34270 9547 Acked Read 3 Done Done Sent false
34271 9548 Acked Write 1 Done Done Sent false
34272 9549 Acked Read 8 Done Done Sent false
34273 9550 Acked Write 1 Done Done Sent false
34274 9551 Acked Write 4 Done Done Sent false
34275 9552 Acked Write 4 Done Done Sent false
34276 9553 Acked Read 6 Done Done Sent false
34277 9554 Acked Read 6 Done Done Sent false
34278 9555 Acked Read 6 Done Done Sent false
34279 9556 Acked Write 7 Done Done Sent false
34280 9557 Acked Write 10 Done Done Sent false
34281 9558 Acked Read 3 Done Done Sent false
34282 9559 Acked Read 5 Done Done Sent false
34283 9560 Acked Write 10 Done Done Sent false
34284 9561 Acked Write 6 Done Done Sent false
34285 9562 Acked Read 2 Done Done Sent false
34286 9563 Acked Read 7 Done Done Sent false
34287 9564 Acked Write 7 Done Done Sent false
34288 9565 Acked Read 4 Done Done Sent false
34289 9566 Acked Read 7 Done Done Sent false
34290 9567 Acked Read 7 Done Done Sent false
34291 9568 Acked Read 4 Done Done Sent false
34292 9569 Acked Write 1 Done Done Sent false
34293 9570 Acked Flush 0 Done Done Sent false
34294 9571 Acked Write 8 Done Done Sent false
34295 9572 Acked Write 7 Done Done Sent false
34296 9573 Acked Read 7 Done Done Sent false
34297 9574 Acked Read 5 Done Done Sent false
34298 9575 Acked Write 6 Done Done Sent false
34299 9576 Acked Read 5 Done Done Sent false
34300 9577 Acked Read 6 Done Done Sent false
34301 9578 Acked Read 5 Done Done Sent false
34302 9579 Acked Write 2 Done Done Sent false
34303 9580 Acked Read 9 Done Done Sent false
34304 9581 Acked Read 7 Done Done Sent false
34305 9582 Acked Read 1 Done Done Sent false
34306 9583 Acked Write 4 Done Done Sent false
34307 9584 Acked Write 9 Done Done Sent false
34308 9585 Acked Flush 0 Done Done Sent false
34309 9586 Acked Write 5 Done Done Sent false
34310 9587 Acked Write 9 Done Done Sent false
34311 9588 Acked Read 7 Done Done Sent false
34312 9589 Acked Write 3 Done Done Sent false
34313 9590 Acked Write 4 Done Done Sent false
34314 9591 Acked Read 2 Done Done Sent false
34315 9592 Acked Read 6 Done Done Sent false
34316 9593 Acked Write 5 Done Done Sent false
34317 9594 Acked Read 4 Done Done Sent false
34318 9595 Acked Read 6 Done Done Sent false
34319 9596 Acked Write 8 Done Done Sent false
34320 9597 Acked Read 5 Done Done Sent false
34321 9598 Acked Read 3 Done Done Sent false
34322 9599 Acked Read 7 Done Done Sent false
34323 9600 Acked Read 6 Done Done Sent false
34324 9601 Acked Write 5 Done Done Sent false
34325 9602 Acked Write 4 Done Done Sent false
34326 9603 Acked Read 9 Done Done Sent false
34327 9604 Acked Read 9 Done Done Sent false
34328 9605 Acked Read 10 Done Done Sent false
34329 9606 Acked Write 9 Done Done Sent false
34330 9607 Acked Read 9 Done Done Sent false
34331 9608 Acked Read 5 Done Done Sent false
34332 9609 Acked Read 5 Done Done Sent false
34333 9610 Acked Read 2 Done Done Sent false
34334 9611 Acked Write 3 Done Done Sent false
34335 9612 Acked Read 7 Done Done Sent false
34336 9613 Acked Read 4 Done Done Sent false
34337 9614 Acked Write 9 Done Done Sent false
34338 9615 Acked Read 4 Done Done Sent false
34339 9616 Acked Flush 0 Done Done Sent false
34340 9617 Acked Write 7 Done Done Sent false
34341 9618 Acked Read 2 Done Done Sent false
34342 9619 Acked Write 5 Done Done Sent false
34343 9620 Acked Read 7 Done Done Sent false
34344 9621 Acked Write 8 Done Done Sent false
34345 9622 Acked Write 1 Done Done Sent false
34346 9623 Acked Write 10 Done Done Sent false
34347 9624 Acked Read 1 Done Done Sent false
34348 9625 Acked Write 10 Done Done Sent false
34349 9626 Acked Read 7 Done Done Sent false
34350 9627 Acked Read 5 Done Done Sent false
34351 9628 Acked Write 8 Done Done Sent false
34352 9629 Acked Read 10 Done Done Sent false
34353 9630 Acked Read 9 Done Done Sent false
34354 9631 Acked Write 4 Done Done Sent false
34355 9632 Acked Write 7 Done Done Sent false
34356 9633 Acked Read 3 Done Done Sent false
34357 9634 Acked Read 10 Done Done Sent false
34358 9635 Acked Flush 0 Done Done Sent false
34359 9636 Acked Read 9 Done Done Sent false
34360 9637 Acked Write 4 Done Done Sent false
34361 9638 Acked Write 4 Done Done Sent false
34362 9639 Acked Write 3 Done Done Sent false
34363 9640 Acked Write 7 Done Done Sent false
34364 9641 Acked Read 3 Done Done Sent false
34365 9642 Acked Write 8 Done Done Sent false
34366 9643 Acked Read 4 Done Done Sent false
34367 9644 Acked Write 1 Done Done Sent false
34368 9645 Acked Write 3 Done Done Sent false
34369 9646 Acked Write 1 Done Done Sent false
34370 9647 Acked Read 4 Done Done Sent false
34371 9648 Acked Read 10 Done Done Sent false
34372 9649 Acked Read 1 Done Done Sent false
34373 9650 Acked Write 4 Done Done Sent false
34374 9651 Acked Read 1 Done Done Sent false
34375 9652 Acked Read 10 Done Done Sent false
34376 9653 Acked Read 6 Done Done Sent false
34377 9654 Acked Read 4 Done Done Sent false
34378 9655 Acked Read 1 Done Done Sent false
34379 9656 Acked Read 5 Done Done Sent false
34380 9657 Acked Read 5 Done Done Sent false
34381 9658 Acked Write 4 Done Done Sent false
34382 9659 Acked Read 2 Done Done Sent false
34383 9660 Acked Write 10 Done Done Sent false
34384 9661 Acked Flush 0 Done Done Sent false
34385 9662 Acked Read 3 Done Done Sent false
34386 9663 Acked Read 9 Done Done Sent false
34387 9664 Acked Write 3 Done Done Sent false
34388 9665 Acked Write 1 Done Done Sent false
34389 9666 Acked Write 8 Done Done Sent false
34390 9667 Acked Read 8 Done Done Sent false
34391 9668 Acked Read 6 Done Done Sent false
34392 9669 Acked Read 10 Done Done Sent false
34393 9670 Acked Read 2 Done Done Sent false
34394 9671 Acked Write 6 Done Done Sent false
34395 9672 Acked Read 4 Done Done Sent false
34396 9673 Acked Read 1 Done Done Sent false
34397 9674 Acked Read 3 Done Done Sent false
34398 9675 Acked Write 9 Done Done Sent false
34399 9676 Acked Read 8 Done Done Sent false
34400 9677 Acked Read 7 Done Done Sent false
34401 9678 Acked Write 7 Done Done Sent false
34402 9679 Acked Read 10 Done Done Sent false
34403 9680 Acked Read 3 Done Done Sent false
34404 9681 Acked Read 10 Done Done Sent false
34405 9682 Acked Write 10 Done Done Sent false
34406 9683 Acked Read 5 Done Done Sent false
34407 9684 Acked Write 7 Done Done Sent false
34408 9685 Acked Read 5 Done Done Sent false
34409 9686 Acked Read 1 Done Done Sent false
34410 9687 Acked Write 9 Done Done Sent false
34411 9688 Acked Write 6 Done Done Sent false
34412 9689 Acked Read 7 Done Done Sent false
34413 9690 Acked Read 4 Done Done Sent false
34414 9691 Acked Read 9 Done Done Sent false
34415 9692 Acked Read 9 Done Done Sent false
34416 9693 Acked Write 3 Done Done Sent false
34417 9694 Acked Read 2 Done Done Sent false
34418 9695 Acked Read 1 Done Done Sent false
34419 9696 Acked Flush 0 Done Done Sent false
34420 9697 Acked Read 6 Done Done Sent false
34421 9698 Acked Write 1 Done Done Sent false
34422 9699 Acked Read 10 Done Done Sent false
34423 9700 Acked Write 7 Done Done Sent false
34424 9701 Acked Read 10 Done Done Sent false
34425 9702 Acked Write 6 Done Done Sent false
34426 9703 Acked Write 3 Done Done Sent false
34427 9704 Acked Flush 0 Done Done Sent false
34428 9705 Acked Flush 0 Done Done Sent false
34429 9706 Acked Read 8 Done Done Sent false
34430 9707 Acked Write 5 Done Done Sent false
34431 9708 Acked Read 1 Done Done Sent false
34432 9709 Acked Write 4 Done Done Sent false
34433 9710 Acked Write 4 Done Done Sent false
34434 9711 Acked Flush 0 Done Done Sent false
34435 9712 Acked Read 10 Done Done Sent false
34436 9713 Acked Read 9 Done Done Sent false
34437 9714 Acked Read 4 Done Done Sent false
34438 9715 Acked Flush 0 Done Done Sent false
34439 9716 Acked Write 1 Done Done Sent false
34440 9717 Acked Flush 0 Done Done Sent false
34441 9718 Acked Flush 0 Done Done Sent false
34442 9719 Acked Read 3 Done Done Sent false
34443 9720 Acked Write 2 Done Done Sent false
34444 9721 Acked Write 4 Done Done Sent false
34445 9722 Acked Read 8 Done Done Sent false
34446 9723 Acked Read 8 Done Done Sent false
34447 9724 Acked Write 9 Done Done Sent false
34448 9725 Acked Write 1 Done Done Sent false
34449 9726 Acked Read 6 Done Done Sent false
34450 9727 Acked Read 8 Done Done Sent false
34451 9728 Acked Read 2 Done Done Sent false
34452 9729 Acked Read 4 Done Done Sent false
34453 9730 Acked Read 8 Done Done Sent false
34454 9731 Acked Write 3 Done Done Sent false
34455 9732 Acked Read 1 Done Done Sent false
34456 9733 Acked Write 6 Done Done Sent false
34457 9734 Acked Read 6 Done Done Sent false
34458 9735 Acked Write 6 Done Done Sent false
34459 9736 Acked Read 6 Done Done Sent false
34460 9737 Acked Write 7 Done Done Sent false
34461 9738 Acked Read 7 Done Done Sent false
34462 9739 Acked Read 3 Done Done Sent false
34463 9740 Acked Read 6 Done Done Sent false
34464 9741 Acked Write 1 Done Done Sent false
34465 9742 Acked Write 1 Done Done Sent false
34466 9743 Acked Write 1 Done Done Sent false
34467 9744 Acked Read 6 Done Done Sent false
34468 9745 Acked Read 1 Done Done Sent false
34469 9746 Acked Flush 0 Done Done Sent false
34470 9747 Acked Read 7 Done Done Sent false
34471 9748 Acked Write 2 Done Done Sent false
34472 9749 Acked Read 2 Done Done Sent false
34473 9750 Acked Write 9 Done Done Sent false
34474 9751 Acked Read 1 Done Done Sent false
34475 9752 Acked Read 2 Done Done Sent false
34476 9753 Acked Write 8 Done Done Sent false
34477 STATES DS:0 DS:1 DS:2 TOTAL
34478 Sent 0 0 301 301
34479 Done 301 301 0 602
34480 Skipped 0 0 0 0
34481 Error 0 0 0 0
34482 Last Flush: 9746 9746 9452
34483 Downstairs last five completed: 9452 9451 9450 9449 9448
34484 Upstairs last five completed: 9753 9752 9751 9750 9749
34485 CLIENT: Up:0 ds:301 act:2
34486 ----------------------------------------------------------------
34487 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
34488 States: Active Active Offline
34489 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
34490 9453 Acked Read 9 Done Done Sent false
34491 9454 Acked Write 1 Done Done Sent false
34492 9455 Acked Read 8 Done Done Sent false
34493 9456 Acked Read 3 Done Done Sent false
34494 9457 Acked Write 7 Done Done Sent false
34495 9458 Acked Flush 0 Done Done Sent false
34496 9459 Acked Read 3 Done Done Sent false
34497 9460 Acked Read 3 Done Done Sent false
34498 9461 Acked Read 8 Done Done Sent false
34499 9462 Acked Read 10 Done Done Sent false
34500 9463 Acked Write 3 Done Done Sent false
34501 9464 Acked Flush 0 Done Done Sent false
34502 9465 Acked Read 7 Done Done Sent false
34503 9466 Acked Write 10 Done Done Sent false
34504 9467 Acked Write 6 Done Done Sent false
34505 9468 Acked Write 9 Done Done Sent false
34506 9469 Acked Read 10 Done Done Sent false
34507 9470 Acked Write 1 Done Done Sent false
34508 9471 Acked Write 6 Done Done Sent false
34509 9472 Acked Write 1 Done Done Sent false
34510 9473 Acked Read 7 Done Done Sent false
34511 9474 Acked Read 6 Done Done Sent false
34512 9475 Acked Write 6 Done Done Sent false
34513 9476 Acked Read 10 Done Done Sent false
34514 9477 Acked Read 10 Done Done Sent false
34515 9478 Acked Flush 0 Done Done Sent false
34516 9479 Acked Read 3 Done Done Sent false
34517 9480 Acked Write 4 Done Done Sent false
34518 9481 Acked Read 1 Done Done Sent false
34519 9482 Acked Read 10 Done Done Sent false
34520 9483 Acked Flush 0 Done Done Sent false
34521 9484 Acked Write 10 Done Done Sent false
34522 9485 Acked Write 1 Done Done Sent false
34523 9486 Acked Read 6 Done Done Sent false
34524 9487 Acked Write 7 Done Done Sent false
34525 9488 Acked Write 8 Done Done Sent false
34526 9489 Acked Read 5 Done Done Sent false
34527 9490 Acked Read 4 Done Done Sent false
34528 9491 Acked Read 4 Done Done Sent false
34529 9492 Acked Read 4 Done Done Sent false
34530 9493 Acked Write 1 Done Done Sent false
34531 9494 Acked Read 9 Done Done Sent false
34532 9495 Acked Write 4 Done Done Sent false
34533 9496 Acked Write 8 Done Done Sent false
34534 9497 Acked Write 7 Done Done Sent false
34535 9498 Acked Write 1 Done Done Sent false
34536 9499 Acked Read 3 Done Done Sent false
34537 9500 Acked Read 8 Done Done Sent false
34538 9501 Acked Read 4 Done Done Sent false
34539 9502 Acked Write 8 Done Done Sent false
34540 9503 Acked Write 9 Done Done Sent false
34541 9504 Acked Write 1 Done Done Sent false
34542 9505 Acked Write 2 Done Done Sent false
34543 9506 Acked Write 2 Done Done Sent false
34544 9507 Acked Read 8 Done Done Sent false
34545 9508 Acked Flush 0 Done Done Sent false
34546 9509 Acked Write 7 Done Done Sent false
34547 9510 Acked Write 4 Done Done Sent false
34548 9511 Acked Read 1 Done Done Sent false
34549 9512 Acked Read 7 Done Done Sent false
34550 9513 Acked Read 2 Done Done Sent false
34551 9514 Acked Flush 0 Done Done Sent false
34552 9515 Acked Read 8 Done Done Sent false
34553 9516 Acked Write 5 Done Done Sent false
34554 9517 Acked Read 5 Done Done Sent false
34555 9518 Acked Write 4 Done Done Sent false
34556 9519 Acked Write 8 Done Done Sent false
34557 9520 Acked Read 8 Done Done Sent false
34558 9521 Acked Write 8 Done Done Sent false
34559 9522 Acked Write 10 Done Done Sent false
34560 9523 Acked Read 6 Done Done Sent false
34561 9524 Acked Flush 0 Done Done Sent false
34562 9525 Acked Flush 0 Done Done Sent false
34563 9526 Acked Read 1 Done Done Sent false
34564 9527 Acked Flush 0 Done Done Sent false
34565 9528 Acked Flush 0 Done Done Sent false
34566 9529 Acked Read 7 Done Done Sent false
34567 9530 Acked Read 6 Done Done Sent false
34568 9531 Acked Read 2 Done Done Sent false
34569 9532 Acked Read 9 Done Done Sent false
34570 9533 Acked Read 5 Done Done Sent false
34571 9534 Acked Flush 0 Done Done Sent false
34572 9535 Acked Read 5 Done Done Sent false
34573 9536 Acked Write 3 Done Done Sent false
34574 9537 Acked Write 1 Done Done Sent false
34575 9538 Acked Write 10 Done Done Sent false
34576 9539 Acked Read 3 Done Done Sent false
34577 9540 Acked Flush 0 Done Done Sent false
34578 9541 Acked Read 5 Done Done Sent false
34579 9542 Acked Read 2 Done Done Sent false
34580 9543 Acked Read 6 Done Done Sent false
34581 9544 Acked Write 4 Done Done Sent false
34582 9545 Acked Read 3 Done Done Sent false
34583 9546 Acked Write 10 Done Done Sent false
34584 9547 Acked Read 3 Done Done Sent false
34585 9548 Acked Write 1 Done Done Sent false
34586 9549 Acked Read 8 Done Done Sent false
34587 9550 Acked Write 1 Done Done Sent false
34588 9551 Acked Write 4 Done Done Sent false
34589 9552 Acked Write 4 Done Done Sent false
34590 9553 Acked Read 6 Done Done Sent false
34591 9554 Acked Read 6 Done Done Sent false
34592 9555 Acked Read 6 Done Done Sent false
34593 9556 Acked Write 7 Done Done Sent false
34594 9557 Acked Write 10 Done Done Sent false
34595 9558 Acked Read 3 Done Done Sent false
34596 9559 Acked Read 5 Done Done Sent false
34597 9560 Acked Write 10 Done Done Sent false
34598 9561 Acked Write 6 Done Done Sent false
34599 9562 Acked Read 2 Done Done Sent false
34600 9563 Acked Read 7 Done Done Sent false
34601 9564 Acked Write 7 Done Done Sent false
34602 9565 Acked Read 4 Done Done Sent false
34603 9566 Acked Read 7 Done Done Sent false
34604 9567 Acked Read 7 Done Done Sent false
34605 9568 Acked Read 4 Done Done Sent false
34606 9569 Acked Write 1 Done Done Sent false
34607 9570 Acked Flush 0 Done Done Sent false
34608 9571 Acked Write 8 Done Done Sent false
34609 9572 Acked Write 7 Done Done Sent false
34610 9573 Acked Read 7 Done Done Sent false
34611 9574 Acked Read 5 Done Done Sent false
34612 9575 Acked Write 6 Done Done Sent false
34613 9576 Acked Read 5 Done Done Sent false
34614 9577 Acked Read 6 Done Done Sent false
34615 9578 Acked Read 5 Done Done Sent false
34616 9579 Acked Write 2 Done Done Sent false
34617 9580 Acked Read 9 Done Done Sent false
34618 9581 Acked Read 7 Done Done Sent false
34619 9582 Acked Read 1 Done Done Sent false
34620 9583 Acked Write 4 Done Done Sent false
34621 9584 Acked Write 9 Done Done Sent false
34622 9585 Acked Flush 0 Done Done Sent false
34623 9586 Acked Write 5 Done Done Sent false
34624 9587 Acked Write 9 Done Done Sent false
34625 9588 Acked Read 7 Done Done Sent false
34626 9589 Acked Write 3 Done Done Sent false
34627 9590 Acked Write 4 Done Done Sent false
34628 9591 Acked Read 2 Done Done Sent false
34629 9592 Acked Read 6 Done Done Sent false
34630 9593 Acked Write 5 Done Done Sent false
34631 9594 Acked Read 4 Done Done Sent false
34632 9595 Acked Read 6 Done Done Sent false
34633 9596 Acked Write 8 Done Done Sent false
34634 9597 Acked Read 5 Done Done Sent false
34635 9598 Acked Read 3 Done Done Sent false
34636 9599 Acked Read 7 Done Done Sent false
34637 9600 Acked Read 6 Done Done Sent false
34638 9601 Acked Write 5 Done Done Sent false
34639 9602 Acked Write 4 Done Done Sent false
34640 9603 Acked Read 9 Done Done Sent false
34641 9604 Acked Read 9 Done Done Sent false
34642 9605 Acked Read 10 Done Done Sent false
34643 9606 Acked Write 9 Done Done Sent false
34644 9607 Acked Read 9 Done Done Sent false
34645 9608 Acked Read 5 Done Done Sent false
34646 9609 Acked Read 5 Done Done Sent false
34647 9610 Acked Read 2 Done Done Sent false
34648 9611 Acked Write 3 Done Done Sent false
34649 9612 Acked Read 7 Done Done Sent false
34650 9613 Acked Read 4 Done Done Sent false
34651 9614 Acked Write 9 Done Done Sent false
34652 9615 Acked Read 4 Done Done Sent false
34653 9616 Acked Flush 0 Done Done Sent false
34654 9617 Acked Write 7 Done Done Sent false
34655 9618 Acked Read 2 Done Done Sent false
34656 9619 Acked Write 5 Done Done Sent false
34657 9620 Acked Read 7 Done Done Sent false
34658 9621 Acked Write 8 Done Done Sent false
34659 9622 Acked Write 1 Done Done Sent false
34660 9623 Acked Write 10 Done Done Sent false
34661 9624 Acked Read 1 Done Done Sent false
34662 9625 Acked Write 10 Done Done Sent false
34663 9626 Acked Read 7 Done Done Sent false
34664 9627 Acked Read 5 Done Done Sent false
34665 9628 Acked Write 8 Done Done Sent false
34666 9629 Acked Read 10 Done Done Sent false
34667 9630 Acked Read 9 Done Done Sent false
34668 9631 Acked Write 4 Done Done Sent false
34669 9632 Acked Write 7 Done Done Sent false
34670 9633 Acked Read 3 Done Done Sent false
34671 9634 Acked Read 10 Done Done Sent false
34672 9635 Acked Flush 0 Done Done Sent false
34673 9636 Acked Read 9 Done Done Sent false
34674 9637 Acked Write 4 Done Done Sent false
34675 9638 Acked Write 4 Done Done Sent false
34676 9639 Acked Write 3 Done Done Sent false
34677 9640 Acked Write 7 Done Done Sent false
34678 9641 Acked Read 3 Done Done Sent false
34679 9642 Acked Write 8 Done Done Sent false
34680 9643 Acked Read 4 Done Done Sent false
34681 9644 Acked Write 1 Done Done Sent false
34682 9645 Acked Write 3 Done Done Sent false
34683 9646 Acked Write 1 Done Done Sent false
34684 9647 Acked Read 4 Done Done Sent false
34685 9648 Acked Read 10 Done Done Sent false
34686 9649 Acked Read 1 Done Done Sent false
34687 9650 Acked Write 4 Done Done Sent false
34688 9651 Acked Read 1 Done Done Sent false
34689 9652 Acked Read 10 Done Done Sent false
34690 9653 Acked Read 6 Done Done Sent false
34691 9654 Acked Read 4 Done Done Sent false
34692 9655 Acked Read 1 Done Done Sent false
34693 9656 Acked Read 5 Done Done Sent false
34694 9657 Acked Read 5 Done Done Sent false
34695 9658 Acked Write 4 Done Done Sent false
34696 9659 Acked Read 2 Done Done Sent false
34697 9660 Acked Write 10 Done Done Sent false
34698 9661 Acked Flush 0 Done Done Sent false
34699 9662 Acked Read 3 Done Done Sent false
34700 9663 Acked Read 9 Done Done Sent false
34701 9664 Acked Write 3 Done Done Sent false
34702 9665 Acked Write 1 Done Done Sent false
34703 9666 Acked Write 8 Done Done Sent false
34704 9667 Acked Read 8 Done Done Sent false
34705 9668 Acked Read 6 Done Done Sent false
34706 9669 Acked Read 10 Done Done Sent false
34707 9670 Acked Read 2 Done Done Sent false
34708 9671 Acked Write 6 Done Done Sent false
34709 9672 Acked Read 4 Done Done Sent false
34710 9673 Acked Read 1 Done Done Sent false
34711 9674 Acked Read 3 Done Done Sent false
34712 9675 Acked Write 9 Done Done Sent false
34713 9676 Acked Read 8 Done Done Sent false
34714 9677 Acked Read 7 Done Done Sent false
34715 9678 Acked Write 7 Done Done Sent false
34716 9679 Acked Read 10 Done Done Sent false
34717 9680 Acked Read 3 Done Done Sent false
34718 9681 Acked Read 10 Done Done Sent false
34719 9682 Acked Write 10 Done Done Sent false
34720 9683 Acked Read 5 Done Done Sent false
34721 9684 Acked Write 7 Done Done Sent false
34722 9685 Acked Read 5 Done Done Sent false
34723 9686 Acked Read 1 Done Done Sent false
34724 9687 Acked Write 9 Done Done Sent false
34725 9688 Acked Write 6 Done Done Sent false
34726 9689 Acked Read 7 Done Done Sent false
34727 9690 Acked Read 4 Done Done Sent false
34728 9691 Acked Read 9 Done Done Sent false
34729 9692 Acked Read 9 Done Done Sent false
34730 9693 Acked Write 3 Done Done Sent false
34731 9694 Acked Read 2 Done Done Sent false
34732 9695 Acked Read 1 Done Done Sent false
34733 9696 Acked Flush 0 Done Done Sent false
34734 9697 Acked Read 6 Done Done Sent false
34735 9698 Acked Write 1 Done Done Sent false
34736 9699 Acked Read 10 Done Done Sent false
34737 9700 Acked Write 7 Done Done Sent false
34738 9701 Acked Read 10 Done Done Sent false
34739 9702 Acked Write 6 Done Done Sent false
34740 9703 Acked Write 3 Done Done Sent false
34741 9704 Acked Flush 0 Done Done Sent false
34742 9705 Acked Flush 0 Done Done Sent false
34743 9706 Acked Read 8 Done Done Sent false
34744 9707 Acked Write 5 Done Done Sent false
34745 9708 Acked Read 1 Done Done Sent false
34746 9709 Acked Write 4 Done Done Sent false
34747 9710 Acked Write 4 Done Done Sent false
34748 9711 Acked Flush 0 Done Done Sent false
34749 9712 Acked Read 10 Done Done Sent false
34750 9713 Acked Read 9 Done Done Sent false
34751 9714 Acked Read 4 Done Done Sent false
34752 9715 Acked Flush 0 Done Done Sent false
34753 9716 Acked Write 1 Done Done Sent false
34754 9717 Acked Flush 0 Done Done Sent false
34755 9718 Acked Flush 0 Done Done Sent false
34756 9719 Acked Read 3 Done Done Sent false
34757 9720 Acked Write 2 Done Done Sent false
34758 9721 Acked Write 4 Done Done Sent false
34759 9722 Acked Read 8 Done Done Sent false
34760 9723 Acked Read 8 Done Done Sent false
34761 9724 Acked Write 9 Done Done Sent false
34762 9725 Acked Write 1 Done Done Sent false
34763 9726 Acked Read 6 Done Done Sent false
34764 9727 Acked Read 8 Done Done Sent false
34765 9728 Acked Read 2 Done Done Sent false
34766 9729 Acked Read 4 Done Done Sent false
34767 9730 Acked Read 8 Done Done Sent false
34768 9731 Acked Write 3 Done Done Sent false
34769 9732 Acked Read 1 Done Done Sent false
34770 9733 Acked Write 6 Done Done Sent false
34771 9734 Acked Read 6 Done Done Sent false
34772 9735 Acked Write 6 Done Done Sent false
34773 9736 Acked Read 6 Done Done Sent false
34774 9737 Acked Write 7 Done Done Sent false
34775 9738 Acked Read 7 Done Done Sent false
34776 9739 Acked Read 3 Done Done Sent false
34777 9740 Acked Read 6 Done Done Sent false
34778 9741 Acked Write 1 Done Done Sent false
34779 9742 Acked Write 1 Done Done Sent false
34780 9743 Acked Write 1 Done Done Sent false
34781 9744 Acked Read 6 Done Done Sent false
34782 9745 Acked Read 1 Done Done Sent false
34783 9746 Acked Flush 0 Done Done Sent false
34784 9747 Acked Read 7 Done Done Sent false
34785 9748 Acked Write 2 Done Done Sent false
34786 9749 Acked Read 2 Done Done Sent false
34787 9750 Acked Write 9 Done Done Sent false
34788 9751 Acked Read 1 Done Done Sent false
34789 9752 Acked Read 2 Done Done Sent false
34790 9753 Acked Write 8 Done Done Sent false
34791 9754 Acked Flush 0 Done Done Sent false
34792 STATES DS:0 DS:1 DS:2 TOTAL
34793 Sent 0 0 302 302
34794 Done 302 302 0 604
34795 Skipped 0 0 0 0
34796 Error 0 0 0 0
34797 Last Flush: 9754 9754 9452
34798 Downstairs last five completed: 9452 9451 9450 9449 9448
34799 Upstairs last five completed: 9754 9753 9752 9751 9750
34800 CLIENT: Up:0 ds:302 act:2
348012024-11-28T04:57:35.856ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
348022024-11-28T04:57:35.856ZWARNcrucible: ds_connection connect to 127.0.0.1:8830 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
348032024-11-28T04:57:35.856ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
348042024-11-28T04:57:35.856ZWARNcrucible: downstairs task for 2 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
348052024-11-28T04:57:35.856ZINFOcrucible: client task is exiting = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
34806 ----------------------------------------------------------------
34807 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
34808 States: Active Active Offline
34809 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
34810 9453 Acked Read 9 Done Done Sent false
34811 9454 Acked Write 1 Done Done Sent false
34812 9455 Acked Read 8 Done Done Sent false
34813 9456 Acked Read 3 Done Done Sent false
34814 9457 Acked Write 7 Done Done Sent false
34815 9458 Acked Flush 0 Done Done Sent false
34816 9459 Acked Read 3 Done Done Sent false
34817 9460 Acked Read 3 Done Done Sent false
34818 9461 Acked Read 8 Done Done Sent false
34819 9462 Acked Read 10 Done Done Sent false
34820 9463 Acked Write 3 Done Done Sent false
34821 9464 Acked Flush 0 Done Done Sent false
34822 9465 Acked Read 7 Done Done Sent false
34823 9466 Acked Write 10 Done Done Sent false
34824 9467 Acked Write 6 Done Done Sent false
34825 9468 Acked Write 9 Done Done Sent false
34826 9469 Acked Read 10 Done Done Sent false
34827 9470 Acked Write 1 Done Done Sent false
34828 9471 Acked Write 6 Done Done Sent false
34829 9472 Acked Write 1 Done Done Sent false
34830 9473 Acked Read 7 Done Done Sent false
34831 9474 Acked Read 6 Done Done Sent false
34832 9475 Acked Write 6 Done Done Sent false
34833 9476 Acked Read 10 Done Done Sent false
34834 9477 Acked Read 10 Done Done Sent false
34835 9478 Acked Flush 0 Done Done Sent false
34836 9479 Acked Read 3 Done Done Sent false
34837 9480 Acked Write 4 Done Done Sent false
34838 9481 Acked Read 1 Done Done Sent false
34839 9482 Acked Read 10 Done Done Sent false
34840 9483 Acked Flush 0 Done Done Sent false
34841 9484 Acked Write 10 Done Done Sent false
34842 9485 Acked Write 1 Done Done Sent false
34843 9486 Acked Read 6 Done Done Sent false
34844 9487 Acked Write 7 Done Done Sent false
34845 9488 Acked Write 8 Done Done Sent false
34846 9489 Acked Read 5 Done Done Sent false
34847 9490 Acked Read 4 Done Done Sent false
34848 9491 Acked Read 4 Done Done Sent false
34849 9492 Acked Read 4 Done Done Sent false
34850 9493 Acked Write 1 Done Done Sent false
34851 9494 Acked Read 9 Done Done Sent false
34852 9495 Acked Write 4 Done Done Sent false
34853 9496 Acked Write 8 Done Done Sent false
34854 9497 Acked Write 7 Done Done Sent false
34855 9498 Acked Write 1 Done Done Sent false
34856 9499 Acked Read 3 Done Done Sent false
34857 9500 Acked Read 8 Done Done Sent false
34858 9501 Acked Read 4 Done Done Sent false
34859 9502 Acked Write 8 Done Done Sent false
34860 9503 Acked Write 9 Done Done Sent false
34861 9504 Acked Write 1 Done Done Sent false
34862 9505 Acked Write 2 Done Done Sent false
34863 9506 Acked Write 2 Done Done Sent false
34864 9507 Acked Read 8 Done Done Sent false
34865 9508 Acked Flush 0 Done Done Sent false
34866 9509 Acked Write 7 Done Done Sent false
34867 9510 Acked Write 4 Done Done Sent false
34868 9511 Acked Read 1 Done Done Sent false
34869 9512 Acked Read 7 Done Done Sent false
34870 9513 Acked Read 2 Done Done Sent false
34871 9514 Acked Flush 0 Done Done Sent false
34872 9515 Acked Read 8 Done Done Sent false
34873 9516 Acked Write 5 Done Done Sent false
34874 9517 Acked Read 5 Done Done Sent false
34875 9518 Acked Write 4 Done Done Sent false
34876 9519 Acked Write 8 Done Done Sent false
34877 9520 Acked Read 8 Done Done Sent false
34878 9521 Acked Write 8 Done Done Sent false
34879 9522 Acked Write 10 Done Done Sent false
34880 9523 Acked Read 6 Done Done Sent false
34881 9524 Acked Flush 0 Done Done Sent false
34882 9525 Acked Flush 0 Done Done Sent false
34883 9526 Acked Read 1 Done Done Sent false
34884 9527 Acked Flush 0 Done Done Sent false
34885 9528 Acked Flush 0 Done Done Sent false
34886 9529 Acked Read 7 Done Done Sent false
34887 9530 Acked Read 6 Done Done Sent false
34888 9531 Acked Read 2 Done Done Sent false
34889 9532 Acked Read 9 Done Done Sent false
34890 9533 Acked Read 5 Done Done Sent false
34891 9534 Acked Flush 0 Done Done Sent false
34892 9535 Acked Read 5 Done Done Sent false
34893 9536 Acked Write 3 Done Done Sent false
34894 9537 Acked Write 1 Done Done Sent false
34895 9538 Acked Write 10 Done Done Sent false
34896 9539 Acked Read 3 Done Done Sent false
34897 9540 Acked Flush 0 Done Done Sent false
34898 9541 Acked Read 5 Done Done Sent false
34899 9542 Acked Read 2 Done Done Sent false
34900 9543 Acked Read 6 Done Done Sent false
34901 9544 Acked Write 4 Done Done Sent false
34902 9545 Acked Read 3 Done Done Sent false
34903 9546 Acked Write 10 Done Done Sent false
34904 9547 Acked Read 3 Done Done Sent false
34905 9548 Acked Write 1 Done Done Sent false
34906 9549 Acked Read 8 Done Done Sent false
34907 9550 Acked Write 1 Done Done Sent false
34908 9551 Acked Write 4 Done Done Sent false
34909 9552 Acked Write 4 Done Done Sent false
34910 9553 Acked Read 6 Done Done Sent false
34911 9554 Acked Read 6 Done Done Sent false
34912 9555 Acked Read 6 Done Done Sent false
34913 9556 Acked Write 7 Done Done Sent false
34914 9557 Acked Write 10 Done Done Sent false
34915 9558 Acked Read 3 Done Done Sent false
34916 9559 Acked Read 5 Done Done Sent false
34917 9560 Acked Write 10 Done Done Sent false
34918 9561 Acked Write 6 Done Done Sent false
34919 9562 Acked Read 2 Done Done Sent false
34920 9563 Acked Read 7 Done Done Sent false
34921 9564 Acked Write 7 Done Done Sent false
34922 9565 Acked Read 4 Done Done Sent false
34923 9566 Acked Read 7 Done Done Sent false
34924 9567 Acked Read 7 Done Done Sent false
34925 9568 Acked Read 4 Done Done Sent false
34926 9569 Acked Write 1 Done Done Sent false
34927 9570 Acked Flush 0 Done Done Sent false
34928 9571 Acked Write 8 Done Done Sent false
34929 9572 Acked Write 7 Done Done Sent false
34930 9573 Acked Read 7 Done Done Sent false
34931 9574 Acked Read 5 Done Done Sent false
34932 9575 Acked Write 6 Done Done Sent false
34933 9576 Acked Read 5 Done Done Sent false
34934 9577 Acked Read 6 Done Done Sent false
34935 9578 Acked Read 5 Done Done Sent false
34936 9579 Acked Write 2 Done Done Sent false
34937 9580 Acked Read 9 Done Done Sent false
34938 9581 Acked Read 7 Done Done Sent false
34939 9582 Acked Read 1 Done Done Sent false
34940 9583 Acked Write 4 Done Done Sent false
34941 9584 Acked Write 9 Done Done Sent false
34942 9585 Acked Flush 0 Done Done Sent false
34943 9586 Acked Write 5 Done Done Sent false
34944 9587 Acked Write 9 Done Done Sent false
34945 9588 Acked Read 7 Done Done Sent false
34946 9589 Acked Write 3 Done Done Sent false
34947 9590 Acked Write 4 Done Done Sent false
34948 9591 Acked Read 2 Done Done Sent false
34949 9592 Acked Read 6 Done Done Sent false
34950 9593 Acked Write 5 Done Done Sent false
34951 9594 Acked Read 4 Done Done Sent false
34952 9595 Acked Read 6 Done Done Sent false
34953 9596 Acked Write 8 Done Done Sent false
34954 9597 Acked Read 5 Done Done Sent false
34955 9598 Acked Read 3 Done Done Sent false
34956 9599 Acked Read 7 Done Done Sent false
34957 9600 Acked Read 6 Done Done Sent false
34958 9601 Acked Write 5 Done Done Sent false
34959 9602 Acked Write 4 Done Done Sent false
34960 9603 Acked Read 9 Done Done Sent false
34961 9604 Acked Read 9 Done Done Sent false
34962 9605 Acked Read 10 Done Done Sent false
34963 9606 Acked Write 9 Done Done Sent false
34964 9607 Acked Read 9 Done Done Sent false
34965 9608 Acked Read 5 Done Done Sent false
34966 9609 Acked Read 5 Done Done Sent false
34967 9610 Acked Read 2 Done Done Sent false
34968 9611 Acked Write 3 Done Done Sent false
34969 9612 Acked Read 7 Done Done Sent false
34970 9613 Acked Read 4 Done Done Sent false
34971 9614 Acked Write 9 Done Done Sent false
34972 9615 Acked Read 4 Done Done Sent false
34973 9616 Acked Flush 0 Done Done Sent false
34974 9617 Acked Write 7 Done Done Sent false
34975 9618 Acked Read 2 Done Done Sent false
34976 9619 Acked Write 5 Done Done Sent false
34977 9620 Acked Read 7 Done Done Sent false
34978 9621 Acked Write 8 Done Done Sent false
34979 9622 Acked Write 1 Done Done Sent false
34980 9623 Acked Write 10 Done Done Sent false
34981 9624 Acked Read 1 Done Done Sent false
34982 9625 Acked Write 10 Done Done Sent false
34983 9626 Acked Read 7 Done Done Sent false
34984 9627 Acked Read 5 Done Done Sent false
34985 9628 Acked Write 8 Done Done Sent false
34986 9629 Acked Read 10 Done Done Sent false
34987 9630 Acked Read 9 Done Done Sent false
34988 9631 Acked Write 4 Done Done Sent false
34989 9632 Acked Write 7 Done Done Sent false
34990 9633 Acked Read 3 Done Done Sent false
34991 9634 Acked Read 10 Done Done Sent false
34992 9635 Acked Flush 0 Done Done Sent false
34993 9636 Acked Read 9 Done Done Sent false
34994 9637 Acked Write 4 Done Done Sent false
34995 9638 Acked Write 4 Done Done Sent false
34996 9639 Acked Write 3 Done Done Sent false
34997 9640 Acked Write 7 Done Done Sent false
34998 9641 Acked Read 3 Done Done Sent false
34999 9642 Acked Write 8 Done Done Sent false
35000 9643 Acked Read 4 Done Done Sent false
35001 9644 Acked Write 1 Done Done Sent false
35002 9645 Acked Write 3 Done Done Sent false
35003 9646 Acked Write 1 Done Done Sent false
35004 9647 Acked Read 4 Done Done Sent false
35005 9648 Acked Read 10 Done Done Sent false
35006 9649 Acked Read 1 Done Done Sent false
35007 9650 Acked Write 4 Done Done Sent false
35008 9651 Acked Read 1 Done Done Sent false
35009 9652 Acked Read 10 Done Done Sent false
35010 9653 Acked Read 6 Done Done Sent false
35011 9654 Acked Read 4 Done Done Sent false
35012 9655 Acked Read 1 Done Done Sent false
35013 9656 Acked Read 5 Done Done Sent false
35014 9657 Acked Read 5 Done Done Sent false
35015 9658 Acked Write 4 Done Done Sent false
35016 9659 Acked Read 2 Done Done Sent false
35017 9660 Acked Write 10 Done Done Sent false
35018 9661 Acked Flush 0 Done Done Sent false
35019 9662 Acked Read 3 Done Done Sent false
35020 9663 Acked Read 9 Done Done Sent false
35021 9664 Acked Write 3 Done Done Sent false
35022 9665 Acked Write 1 Done Done Sent false
35023 9666 Acked Write 8 Done Done Sent false
35024 9667 Acked Read 8 Done Done Sent false
35025 9668 Acked Read 6 Done Done Sent false
35026 9669 Acked Read 10 Done Done Sent false
35027 9670 Acked Read 2 Done Done Sent false
35028 9671 Acked Write 6 Done Done Sent false
35029 9672 Acked Read 4 Done Done Sent false
35030 9673 Acked Read 1 Done Done Sent false
35031 9674 Acked Read 3 Done Done Sent false
35032 9675 Acked Write 9 Done Done Sent false
35033 9676 Acked Read 8 Done Done Sent false
35034 9677 Acked Read 7 Done Done Sent false
35035 9678 Acked Write 7 Done Done Sent false
35036 9679 Acked Read 10 Done Done Sent false
35037 9680 Acked Read 3 Done Done Sent false
35038 9681 Acked Read 10 Done Done Sent false
35039 9682 Acked Write 10 Done Done Sent false
35040 9683 Acked Read 5 Done Done Sent false
35041 9684 Acked Write 7 Done Done Sent false
35042 9685 Acked Read 5 Done Done Sent false
35043 9686 Acked Read 1 Done Done Sent false
35044 9687 Acked Write 9 Done Done Sent false
35045 9688 Acked Write 6 Done Done Sent false
35046 9689 Acked Read 7 Done Done Sent false
35047 9690 Acked Read 4 Done Done Sent false
35048 9691 Acked Read 9 Done Done Sent false
35049 9692 Acked Read 9 Done Done Sent false
35050 9693 Acked Write 3 Done Done Sent false
35051 9694 Acked Read 2 Done Done Sent false
35052 9695 Acked Read 1 Done Done Sent false
35053 9696 Acked Flush 0 Done Done Sent false
35054 9697 Acked Read 6 Done Done Sent false
35055 9698 Acked Write 1 Done Done Sent false
35056 9699 Acked Read 10 Done Done Sent false
35057 9700 Acked Write 7 Done Done Sent false
35058 9701 Acked Read 10 Done Done Sent false
35059 9702 Acked Write 6 Done Done Sent false
35060 9703 Acked Write 3 Done Done Sent false
35061 9704 Acked Flush 0 Done Done Sent false
35062 9705 Acked Flush 0 Done Done Sent false
35063 9706 Acked Read 8 Done Done Sent false
35064 9707 Acked Write 5 Done Done Sent false
35065 9708 Acked Read 1 Done Done Sent false
35066 9709 Acked Write 4 Done Done Sent false
35067 9710 Acked Write 4 Done Done Sent false
35068 9711 Acked Flush 0 Done Done Sent false
35069 9712 Acked Read 10 Done Done Sent false
35070 9713 Acked Read 9 Done Done Sent false
35071 9714 Acked Read 4 Done Done Sent false
35072 9715 Acked Flush 0 Done Done Sent false
35073 9716 Acked Write 1 Done Done Sent false
35074 9717 Acked Flush 0 Done Done Sent false
35075 9718 Acked Flush 0 Done Done Sent false
35076 9719 Acked Read 3 Done Done Sent false
35077 9720 Acked Write 2 Done Done Sent false
35078 9721 Acked Write 4 Done Done Sent false
35079 9722 Acked Read 8 Done Done Sent false
35080 9723 Acked Read 8 Done Done Sent false
35081 9724 Acked Write 9 Done Done Sent false
35082 9725 Acked Write 1 Done Done Sent false
35083 9726 Acked Read 6 Done Done Sent false
35084 9727 Acked Read 8 Done Done Sent false
35085 9728 Acked Read 2 Done Done Sent false
35086 9729 Acked Read 4 Done Done Sent false
35087 9730 Acked Read 8 Done Done Sent false
35088 9731 Acked Write 3 Done Done Sent false
35089 9732 Acked Read 1 Done Done Sent false
35090 9733 Acked Write 6 Done Done Sent false
35091 9734 Acked Read 6 Done Done Sent false
35092 9735 Acked Write 6 Done Done Sent false
35093 9736 Acked Read 6 Done Done Sent false
35094 9737 Acked Write 7 Done Done Sent false
35095 9738 Acked Read 7 Done Done Sent false
35096 9739 Acked Read 3 Done Done Sent false
35097 9740 Acked Read 6 Done Done Sent false
35098 9741 Acked Write 1 Done Done Sent false
35099 9742 Acked Write 1 Done Done Sent false
35100 9743 Acked Write 1 Done Done Sent false
35101 9744 Acked Read 6 Done Done Sent false
35102 9745 Acked Read 1 Done Done Sent false
35103 9746 Acked Flush 0 Done Done Sent false
35104 9747 Acked Read 7 Done Done Sent false
35105 9748 Acked Write 2 Done Done Sent false
35106 9749 Acked Read 2 Done Done Sent false
35107 9750 Acked Write 9 Done Done Sent false
35108 9751 Acked Read 1 Done Done Sent false
35109 9752 Acked Read 2 Done Done Sent false
35110 9753 Acked Write 8 Done Done Sent false
35111 9754 Acked Flush 0 Done Done Sent false
35112 STATES DS:0 DS:1 DS:2 TOTAL
35113 Sent 0 0 302 302
35114 Done 302 302 0 604
35115 Skipped 0 0 0 0
35116 Error 0 0 0 0
35117 Last Flush: 9754 9754 9452
35118 Downstairs last five completed: 9452 9451 9450 9449 9448
35119 Upstairs last five completed: 9754 9753 9752 9751 9750
35120 CLIENT: Up:0 ds:302 act:2
35121 ----------------------------------------------------------------
35122 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
35123 States: Active Active Offline
35124 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
35125 9453 Acked Read 9 Done Done Sent false
35126 9454 Acked Write 1 Done Done Sent false
35127 9455 Acked Read 8 Done Done Sent false
35128 9456 Acked Read 3 Done Done Sent false
35129 9457 Acked Write 7 Done Done Sent false
35130 9458 Acked Flush 0 Done Done Sent false
35131 9459 Acked Read 3 Done Done Sent false
35132 9460 Acked Read 3 Done Done Sent false
35133 9461 Acked Read 8 Done Done Sent false
35134 9462 Acked Read 10 Done Done Sent false
35135 9463 Acked Write 3 Done Done Sent false
35136 9464 Acked Flush 0 Done Done Sent false
35137 9465 Acked Read 7 Done Done Sent false
35138 9466 Acked Write 10 Done Done Sent false
35139 9467 Acked Write 6 Done Done Sent false
35140 9468 Acked Write 9 Done Done Sent false
35141 9469 Acked Read 10 Done Done Sent false
35142 9470 Acked Write 1 Done Done Sent false
35143 9471 Acked Write 6 Done Done Sent false
35144 9472 Acked Write 1 Done Done Sent false
35145 9473 Acked Read 7 Done Done Sent false
35146 9474 Acked Read 6 Done Done Sent false
35147 9475 Acked Write 6 Done Done Sent false
35148 9476 Acked Read 10 Done Done Sent false
35149 9477 Acked Read 10 Done Done Sent false
35150 9478 Acked Flush 0 Done Done Sent false
35151 9479 Acked Read 3 Done Done Sent false
35152 9480 Acked Write 4 Done Done Sent false
35153 9481 Acked Read 1 Done Done Sent false
35154 9482 Acked Read 10 Done Done Sent false
35155 9483 Acked Flush 0 Done Done Sent false
35156 9484 Acked Write 10 Done Done Sent false
35157 9485 Acked Write 1 Done Done Sent false
35158 9486 Acked Read 6 Done Done Sent false
35159 9487 Acked Write 7 Done Done Sent false
35160 9488 Acked Write 8 Done Done Sent false
35161 9489 Acked Read 5 Done Done Sent false
35162 9490 Acked Read 4 Done Done Sent false
35163 9491 Acked Read 4 Done Done Sent false
35164 9492 Acked Read 4 Done Done Sent false
35165 9493 Acked Write 1 Done Done Sent false
35166 9494 Acked Read 9 Done Done Sent false
35167 9495 Acked Write 4 Done Done Sent false
35168 9496 Acked Write 8 Done Done Sent false
35169 9497 Acked Write 7 Done Done Sent false
35170 9498 Acked Write 1 Done Done Sent false
35171 9499 Acked Read 3 Done Done Sent false
35172 9500 Acked Read 8 Done Done Sent false
35173 9501 Acked Read 4 Done Done Sent false
35174 9502 Acked Write 8 Done Done Sent false
35175 9503 Acked Write 9 Done Done Sent false
35176 9504 Acked Write 1 Done Done Sent false
35177 9505 Acked Write 2 Done Done Sent false
35178 9506 Acked Write 2 Done Done Sent false
35179 9507 Acked Read 8 Done Done Sent false
35180 9508 Acked Flush 0 Done Done Sent false
35181 9509 Acked Write 7 Done Done Sent false
35182 9510 Acked Write 4 Done Done Sent false
35183 9511 Acked Read 1 Done Done Sent false
35184 9512 Acked Read 7 Done Done Sent false
35185 9513 Acked Read 2 Done Done Sent false
35186 9514 Acked Flush 0 Done Done Sent false
35187 9515 Acked Read 8 Done Done Sent false
35188 9516 Acked Write 5 Done Done Sent false
35189 9517 Acked Read 5 Done Done Sent false
35190 9518 Acked Write 4 Done Done Sent false
35191 9519 Acked Write 8 Done Done Sent false
35192 9520 Acked Read 8 Done Done Sent false
35193 9521 Acked Write 8 Done Done Sent false
35194 9522 Acked Write 10 Done Done Sent false
35195 9523 Acked Read 6 Done Done Sent false
35196 9524 Acked Flush 0 Done Done Sent false
35197 9525 Acked Flush 0 Done Done Sent false
35198 9526 Acked Read 1 Done Done Sent false
35199 9527 Acked Flush 0 Done Done Sent false
35200 9528 Acked Flush 0 Done Done Sent false
35201 9529 Acked Read 7 Done Done Sent false
35202 9530 Acked Read 6 Done Done Sent false
35203 9531 Acked Read 2 Done Done Sent false
35204 9532 Acked Read 9 Done Done Sent false
35205 9533 Acked Read 5 Done Done Sent false
35206 9534 Acked Flush 0 Done Done Sent false
35207 9535 Acked Read 5 Done Done Sent false
35208 9536 Acked Write 3 Done Done Sent false
35209 9537 Acked Write 1 Done Done Sent false
35210 9538 Acked Write 10 Done Done Sent false
35211 9539 Acked Read 3 Done Done Sent false
35212 9540 Acked Flush 0 Done Done Sent false
35213 9541 Acked Read 5 Done Done Sent false
35214 9542 Acked Read 2 Done Done Sent false
35215 9543 Acked Read 6 Done Done Sent false
35216 9544 Acked Write 4 Done Done Sent false
35217 9545 Acked Read 3 Done Done Sent false
35218 9546 Acked Write 10 Done Done Sent false
35219 9547 Acked Read 3 Done Done Sent false
35220 9548 Acked Write 1 Done Done Sent false
35221 9549 Acked Read 8 Done Done Sent false
35222 9550 Acked Write 1 Done Done Sent false
35223 9551 Acked Write 4 Done Done Sent false
35224 9552 Acked Write 4 Done Done Sent false
35225 9553 Acked Read 6 Done Done Sent false
35226 9554 Acked Read 6 Done Done Sent false
35227 9555 Acked Read 6 Done Done Sent false
35228 9556 Acked Write 7 Done Done Sent false
35229 9557 Acked Write 10 Done Done Sent false
35230 9558 Acked Read 3 Done Done Sent false
35231 9559 Acked Read 5 Done Done Sent false
35232 9560 Acked Write 10 Done Done Sent false
35233 9561 Acked Write 6 Done Done Sent false
35234 9562 Acked Read 2 Done Done Sent false
35235 9563 Acked Read 7 Done Done Sent false
35236 9564 Acked Write 7 Done Done Sent false
35237 9565 Acked Read 4 Done Done Sent false
35238 9566 Acked Read 7 Done Done Sent false
35239 9567 Acked Read 7 Done Done Sent false
35240 9568 Acked Read 4 Done Done Sent false
35241 9569 Acked Write 1 Done Done Sent false
35242 9570 Acked Flush 0 Done Done Sent false
35243 9571 Acked Write 8 Done Done Sent false
35244 9572 Acked Write 7 Done Done Sent false
35245 9573 Acked Read 7 Done Done Sent false
35246 9574 Acked Read 5 Done Done Sent false
35247 9575 Acked Write 6 Done Done Sent false
35248 9576 Acked Read 5 Done Done Sent false
35249 9577 Acked Read 6 Done Done Sent false
35250 9578 Acked Read 5 Done Done Sent false
35251 9579 Acked Write 2 Done Done Sent false
35252 9580 Acked Read 9 Done Done Sent false
35253 9581 Acked Read 7 Done Done Sent false
35254 9582 Acked Read 1 Done Done Sent false
35255 9583 Acked Write 4 Done Done Sent false
35256 9584 Acked Write 9 Done Done Sent false
35257 9585 Acked Flush 0 Done Done Sent false
35258 9586 Acked Write 5 Done Done Sent false
35259 9587 Acked Write 9 Done Done Sent false
35260 9588 Acked Read 7 Done Done Sent false
35261 9589 Acked Write 3 Done Done Sent false
35262 9590 Acked Write 4 Done Done Sent false
35263 9591 Acked Read 2 Done Done Sent false
35264 9592 Acked Read 6 Done Done Sent false
35265 9593 Acked Write 5 Done Done Sent false
35266 9594 Acked Read 4 Done Done Sent false
35267 9595 Acked Read 6 Done Done Sent false
35268 9596 Acked Write 8 Done Done Sent false
35269 9597 Acked Read 5 Done Done Sent false
35270 9598 Acked Read 3 Done Done Sent false
35271 9599 Acked Read 7 Done Done Sent false
35272 9600 Acked Read 6 Done Done Sent false
35273 9601 Acked Write 5 Done Done Sent false
35274 9602 Acked Write 4 Done Done Sent false
35275 9603 Acked Read 9 Done Done Sent false
35276 9604 Acked Read 9 Done Done Sent false
35277 9605 Acked Read 10 Done Done Sent false
35278 9606 Acked Write 9 Done Done Sent false
35279 9607 Acked Read 9 Done Done Sent false
35280 9608 Acked Read 5 Done Done Sent false
35281 9609 Acked Read 5 Done Done Sent false
35282 9610 Acked Read 2 Done Done Sent false
35283 9611 Acked Write 3 Done Done Sent false
35284 9612 Acked Read 7 Done Done Sent false
35285 9613 Acked Read 4 Done Done Sent false
35286 9614 Acked Write 9 Done Done Sent false
35287 9615 Acked Read 4 Done Done Sent false
35288 9616 Acked Flush 0 Done Done Sent false
35289 9617 Acked Write 7 Done Done Sent false
35290 9618 Acked Read 2 Done Done Sent false
35291 9619 Acked Write 5 Done Done Sent false
35292 9620 Acked Read 7 Done Done Sent false
35293 9621 Acked Write 8 Done Done Sent false
35294 9622 Acked Write 1 Done Done Sent false
35295 9623 Acked Write 10 Done Done Sent false
35296 9624 Acked Read 1 Done Done Sent false
35297 9625 Acked Write 10 Done Done Sent false
35298 9626 Acked Read 7 Done Done Sent false
35299 9627 Acked Read 5 Done Done Sent false
35300 9628 Acked Write 8 Done Done Sent false
35301 9629 Acked Read 10 Done Done Sent false
35302 9630 Acked Read 9 Done Done Sent false
35303 9631 Acked Write 4 Done Done Sent false
35304 9632 Acked Write 7 Done Done Sent false
35305 9633 Acked Read 3 Done Done Sent false
35306 9634 Acked Read 10 Done Done Sent false
35307 9635 Acked Flush 0 Done Done Sent false
35308 9636 Acked Read 9 Done Done Sent false
35309 9637 Acked Write 4 Done Done Sent false
35310 9638 Acked Write 4 Done Done Sent false
35311 9639 Acked Write 3 Done Done Sent false
35312 9640 Acked Write 7 Done Done Sent false
35313 9641 Acked Read 3 Done Done Sent false
35314 9642 Acked Write 8 Done Done Sent false
35315 9643 Acked Read 4 Done Done Sent false
35316 9644 Acked Write 1 Done Done Sent false
35317 9645 Acked Write 3 Done Done Sent false
35318 9646 Acked Write 1 Done Done Sent false
35319 9647 Acked Read 4 Done Done Sent false
35320 9648 Acked Read 10 Done Done Sent false
35321 9649 Acked Read 1 Done Done Sent false
35322 9650 Acked Write 4 Done Done Sent false
35323 9651 Acked Read 1 Done Done Sent false
35324 9652 Acked Read 10 Done Done Sent false
35325 9653 Acked Read 6 Done Done Sent false
35326 9654 Acked Read 4 Done Done Sent false
35327 9655 Acked Read 1 Done Done Sent false
35328 9656 Acked Read 5 Done Done Sent false
35329 9657 Acked Read 5 Done Done Sent false
35330 9658 Acked Write 4 Done Done Sent false
35331 9659 Acked Read 2 Done Done Sent false
35332 9660 Acked Write 10 Done Done Sent false
35333 9661 Acked Flush 0 Done Done Sent false
35334 9662 Acked Read 3 Done Done Sent false
35335 9663 Acked Read 9 Done Done Sent false
35336 9664 Acked Write 3 Done Done Sent false
35337 9665 Acked Write 1 Done Done Sent false
35338 9666 Acked Write 8 Done Done Sent false
35339 9667 Acked Read 8 Done Done Sent false
35340 9668 Acked Read 6 Done Done Sent false
35341 9669 Acked Read 10 Done Done Sent false
35342 9670 Acked Read 2 Done Done Sent false
35343 9671 Acked Write 6 Done Done Sent false
35344 9672 Acked Read 4 Done Done Sent false
35345 9673 Acked Read 1 Done Done Sent false
35346 9674 Acked Read 3 Done Done Sent false
35347 9675 Acked Write 9 Done Done Sent false
35348 9676 Acked Read 8 Done Done Sent false
35349 9677 Acked Read 7 Done Done Sent false
35350 9678 Acked Write 7 Done Done Sent false
35351 9679 Acked Read 10 Done Done Sent false
35352 9680 Acked Read 3 Done Done Sent false
35353 9681 Acked Read 10 Done Done Sent false
35354 9682 Acked Write 10 Done Done Sent false
35355 9683 Acked Read 5 Done Done Sent false
35356 9684 Acked Write 7 Done Done Sent false
35357 9685 Acked Read 5 Done Done Sent false
35358 9686 Acked Read 1 Done Done Sent false
35359 9687 Acked Write 9 Done Done Sent false
35360 9688 Acked Write 6 Done Done Sent false
35361 9689 Acked Read 7 Done Done Sent false
35362 9690 Acked Read 4 Done Done Sent false
35363 9691 Acked Read 9 Done Done Sent false
35364 9692 Acked Read 9 Done Done Sent false
35365 9693 Acked Write 3 Done Done Sent false
35366 9694 Acked Read 2 Done Done Sent false
35367 9695 Acked Read 1 Done Done Sent false
35368 9696 Acked Flush 0 Done Done Sent false
35369 9697 Acked Read 6 Done Done Sent false
35370 9698 Acked Write 1 Done Done Sent false
35371 9699 Acked Read 10 Done Done Sent false
35372 9700 Acked Write 7 Done Done Sent false
35373 9701 Acked Read 10 Done Done Sent false
35374 9702 Acked Write 6 Done Done Sent false
35375 9703 Acked Write 3 Done Done Sent false
35376 9704 Acked Flush 0 Done Done Sent false
35377 9705 Acked Flush 0 Done Done Sent false
35378 9706 Acked Read 8 Done Done Sent false
35379 9707 Acked Write 5 Done Done Sent false
35380 9708 Acked Read 1 Done Done Sent false
35381 9709 Acked Write 4 Done Done Sent false
35382 9710 Acked Write 4 Done Done Sent false
35383 9711 Acked Flush 0 Done Done Sent false
35384 9712 Acked Read 10 Done Done Sent false
35385 9713 Acked Read 9 Done Done Sent false
35386 9714 Acked Read 4 Done Done Sent false
35387 9715 Acked Flush 0 Done Done Sent false
35388 9716 Acked Write 1 Done Done Sent false
35389 9717 Acked Flush 0 Done Done Sent false
35390 9718 Acked Flush 0 Done Done Sent false
35391 9719 Acked Read 3 Done Done Sent false
35392 9720 Acked Write 2 Done Done Sent false
35393 9721 Acked Write 4 Done Done Sent false
35394 9722 Acked Read 8 Done Done Sent false
35395 9723 Acked Read 8 Done Done Sent false
35396 9724 Acked Write 9 Done Done Sent false
35397 9725 Acked Write 1 Done Done Sent false
35398 9726 Acked Read 6 Done Done Sent false
35399 9727 Acked Read 8 Done Done Sent false
35400 9728 Acked Read 2 Done Done Sent false
35401 9729 Acked Read 4 Done Done Sent false
35402 9730 Acked Read 8 Done Done Sent false
35403 9731 Acked Write 3 Done Done Sent false
35404 9732 Acked Read 1 Done Done Sent false
35405 9733 Acked Write 6 Done Done Sent false
35406 9734 Acked Read 6 Done Done Sent false
35407 9735 Acked Write 6 Done Done Sent false
35408 9736 Acked Read 6 Done Done Sent false
35409 9737 Acked Write 7 Done Done Sent false
35410 9738 Acked Read 7 Done Done Sent false
35411 9739 Acked Read 3 Done Done Sent false
35412 9740 Acked Read 6 Done Done Sent false
35413 9741 Acked Write 1 Done Done Sent false
35414 9742 Acked Write 1 Done Done Sent false
35415 9743 Acked Write 1 Done Done Sent false
35416 9744 Acked Read 6 Done Done Sent false
35417 9745 Acked Read 1 Done Done Sent false
35418 9746 Acked Flush 0 Done Done Sent false
35419 9747 Acked Read 7 Done Done Sent false
35420 9748 Acked Write 2 Done Done Sent false
35421 9749 Acked Read 2 Done Done Sent false
35422 9750 Acked Write 9 Done Done Sent false
35423 9751 Acked Read 1 Done Done Sent false
35424 9752 Acked Read 2 Done Done Sent false
35425 9753 Acked Write 8 Done Done Sent false
35426 9754 Acked Flush 0 Done Done Sent false
35427 STATES DS:0 DS:1 DS:2 TOTAL
35428 Sent 0 0 302 302
35429 Done 302 302 0 604
35430 Skipped 0 0 0 0
35431 Error 0 0 0 0
35432 Last Flush: 9754 9754 9452
35433 Downstairs last five completed: 9452 9451 9450 9449 9448
35434 Upstairs last five completed: 9754 9753 9752 9751 9750
35435 CLIENT: Up:0 ds:302 act:2
35436 [8830] got start action
35437 [8830] I am going to restart
35438 Starting downstairs at port 8830
35439 Make output file at "/tmp/dsc/downstairs-8830.txt"
35440 [8830][2] reports Starting
35441 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1231)
35442 [8830][2] reports Running
354432024-11-28T04:57:45.857ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
354442024-11-28T04:57:45.857ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:64241) = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
354452024-11-28T04:57:45.858ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
354462024-11-28T04:57:45.859ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
354472024-11-28T04:57:45.859ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
354482024-11-28T04:57:45.859ZINFOcrucible: send last flush ID to this DS: 9452 = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
354492024-11-28T04:57:45.859ZINFOcrucible: Replied this last flush ID: 9452 = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
354502024-11-28T04:57:45.859ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
354512024-11-28T04:57:45.859ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
354522024-11-28T04:57:45.859ZINFOcrucible: [2] client re-new 302 jobs since flush 9452 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
354532024-11-28T04:57:45.859ZINFOcrucible: [2] Replayed 302 jobs since flush: 9452 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
35454 ----------------------------------------------------------------
35455 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
35456 Downstairs last five completed: 9754 9753 9752 9751 9750
35457 Upstairs last five completed: 9754 9753 9752 9751 9750
35458 CLIENT: Up:0 ds:0 act:3
35459 Replay: All jobs finished
35460 Main task has work to do, go find it
35461 got dsc Stop(2)
35462 stop 2
35463 [8830] Got stop action so:false kr:false
35464 Nov 28 04:57:46.106 INFO request completed, latency_us: 339, response_code: 204, uri: /stop/cid/2, method: POST, req_id: eed5c19f-2d73-4e63-ad7a-43e415c9eea5, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
35465 Nov 28 04:57:46.107 INFO request completed, latency_us: 119, response_code: 200, uri: /state/cid/2, method: GET, req_id: 3c519b73-ab4a-4c49-91ce-63bdbd7c26ca, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
354662024-11-28T04:57:46.107ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
354672024-11-28T04:57:46.107ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
354682024-11-28T04:57:46.107ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
354692024-11-28T04:57:46.107ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
354702024-11-28T04:57:46.107ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
354712024-11-28T04:57:46.107ZINFOcrucible: client task is exiting = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
35472 [8830] Exited with: ExitStatus(unix_wait_status(9))
35473 [8830][2] reports Exit
35474 Nov 28 04:57:50.109 INFO request completed, latency_us: 212, response_code: 200, uri: /state/cid/2, method: GET, req_id: d9be9cf5-8946-41a8-91ef-5d5725f022e4, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
35475 001/300 Read block 3573 len 20480
35476 002/300 Write block 1497 len 20480 data: 12 9 9 9 9
35477 003/300 Flush
35478 004/300 Write block 4450 len 20480 data: 11 11 10 9 12
35479 005/300 Flush
35480 006/300 Write block 2532 len 20480 data: 9 8 7 6 5
35481 007/300 Write block 1000 len 28672 data: 7 7 5 5 4 3 3
35482 008/300 Read block 3029 len 12288
35483 009/300 Read block 2608 len 36864
35484 010/300 Write block 872 len 36864 data: 7 7 6 5 6 4 5 6 6
35485 011/300 Flush
35486 012/300 Flush
35487 013/300 Write block 4525 len 36864 data: 9 8 8 5 8 7 9 10 7
35488 014/300 Write block 4880 len 32768 data: 4 5 6 7 6 6 6 6
35489 015/300 Read block 3931 len 20480
35490 016/300 Read block 396 len 4096
35491 017/300 Read block 2887 len 40960
35492 018/300 Write block 360 len 4096 data: 4
35493 019/300 Write block 2483 len 28672 data: 6 6 6 5 4 4 3
35494 020/300 Write block 2476 len 12288 data: 10 7 8
35495 021/300 Read block 3371 len 4096
35496 022/300 Read block 2137 len 28672
35497 023/300 Write block 4264 len 32768 data: 5 5 8 7 8 8 7 5
35498 024/300 Read block 3797 len 36864
35499 025/300 Write block 4305 len 24576 data: 5 5 4 3 4 5
35500 026/300 Flush
35501 027/300 Read block 3424 len 24576
35502 028/300 Read block 2331 len 40960
35503 029/300 Read block 2508 len 28672
35504 030/300 Read block 3463 len 20480
35505 031/300 Write block 38 len 20480 data: 11 10 9 9 8
35506 032/300 Read block 337 len 32768
35507 033/300 Read block 3372 len 12288
35508 034/300 Write block 2254 len 12288 data: 7 8 8
35509 035/300 Write block 4759 len 16384 data: 5 5 7 7
35510 036/300 Write block 3587 len 16384 data: 8 8 7 6
35511 037/300 Write block 2724 len 8192 data: 4 3
35512 038/300 Write block 1048 len 16384 data: 8 10 8 8
35513 039/300 Read block 2354 len 40960
35514 040/300 Write block 3547 len 36864 data: 3 5 6 6 7 7 7 7 6
35515 041/300 Read block 2800 len 4096
35516 042/300 Flush
35517 043/300 Read block 41 len 28672
35518 044/300 Write block 4247 len 32768 data: 11 9 9 9 6 6 6 4
35519 045/300 Read block 4344 len 4096
35520 046/300 Read block 912 len 24576
35521 047/300 Write block 2763 len 8192 data: 5 5
35522 048/300 Write block 364 len 12288 data: 7 8 10
35523 049/300 Write block 3583 len 20480 data: 7 7 7 7 9
35524 050/300 Write block 1629 len 28672 data: 9 9 8 8 6 4 3
35525 051/300 Read block 3640 len 12288
35526 052/300 Read block 3772 len 8192
35527 053/300 Write block 245 len 8192 data: 6 6
35528 054/300 Flush
35529 055/300 Write block 2428 len 12288 data: 5 5 3
35530 056/300 Write block 80 len 4096 data: 3
35531 057/300 Write block 4905 len 8192 data: 3 4
35532 058/300 Write block 3721 len 24576 data: 4 5 5 5 4 6
35533 059/300 Flush
35534 060/300 Write block 4932 len 16384 data: 8 6 6 5
35535 061/300 Read block 2425 len 32768
35536 062/300 Write block 800 len 8192 data: 7 7
35537 063/300 Read block 691 len 40960
35538 064/300 Read block 1236 len 24576
35539 065/300 Flush
35540 066/300 Read block 4038 len 16384
35541 067/300 Read block 4266 len 36864
35542 068/300 Read block 907 len 36864
35543 069/300 Flush
35544 070/300 Read block 129 len 16384
35545 071/300 Write block 2341 len 36864 data: 4 4 4 6 4 5 6 7 7
35546 072/300 Read block 3016 len 36864
35547 073/300 Flush
35548 074/300 Flush
35549 075/300 Read block 2002 len 16384
35550 076/300 Read block 2213 len 28672
35551 077/300 Read block 2050 len 20480
35552 078/300 Read block 1608 len 8192
35553 079/300 Flush
35554 080/300 Write block 2484 len 28672 data: 7 7 6 5 5 4 2
35555 081/300 Write block 3952 len 40960 data: 8 8 7 7 8 6 6 6 7 9
35556 082/300 Write block 2306 len 40960 data: 2 3 4 5 5 5 6 6 6 7
35557 083/300 Read block 804 len 4096
35558 084/300 Write block 1076 len 24576 data: 8 8 5 7 6 7
35559 085/300 Write block 4639 len 8192 data: 6 5
35560 086/300 Write block 464 len 4096 data: 5
35561 087/300 Write block 3365 len 28672 data: 3 3 3 4 4 5 5
35562 088/300 Read block 25 len 4096
35563 089/300 Read block 4138 len 16384
35564 090/300 Write block 1561 len 36864 data: 3 3 3 3 4 4 3 3 3
35565 091/300 Write block 466 len 8192 data: 7 6
35566 092/300 Read block 4862 len 8192
35567 093/300 Write block 3345 len 4096 data: 7
35568 094/300 Write block 2518 len 12288 data: 5 3 5
35569 095/300 Read block 2116 len 36864
35570 096/300 Write block 2027 len 36864 data: 4 6 6 6 4 5 6 7 8
35571 097/300 Read block 4842 len 12288
35572 098/300 Write block 1534 len 32768 data: 6 7 6 7 7 8 9 7
35573 099/300 Write block 4980 len 8192 data: 6 6
35574 100/300 Write block 2321 len 32768 data: 7 5 5 5 7 6 6 5
35575 101/300 Read block 2551 len 36864
35576 102/300 Read block 473 len 20480
35577 103/300 Read block 2835 len 24576
35578 104/300 Write block 295 len 20480 data: 5 5 5 4 3
35579 105/300 Write block 327 len 24576 data: 7 6 6 6 6 5
35580 106/300 Write block 3908 len 4096 data: 5
35581 107/300 Read block 4285 len 28672
35582 108/300 Read block 1068 len 8192
35583 109/300 Read block 185 len 20480
35584 110/300 Write block 3178 len 8192 data: 7 6
35585 111/300 Read block 3590 len 28672
35586 112/300 Read block 3609 len 4096
35587 113/300 Read block 4721 len 4096
35588 114/300 Read block 1021 len 40960
35589 115/300 Read block 2837 len 12288
35590 116/300 Read block 1643 len 36864
35591 117/300 Read block 4079 len 32768
35592 118/300 Flush
35593 119/300 Write block 77 len 8192 data: 3 4
35594 120/300 Read block 1766 len 16384
35595 121/300 Write block 753 len 8192 data: 5 5
35596 122/300 Write block 3193 len 28672 data: 6 5 5 3 4 6 4
35597 123/300 Write block 656 len 24576 data: 9 10 8 11 10 10
35598 124/300 Write block 696 len 24576 data: 6 5 3 3 3 4
35599 125/300 Write block 3979 len 28672 data: 3 4 4 3 3 4 4
35600 126/300 Read block 4832 len 8192
35601 127/300 Read block 2130 len 40960
35602 128/300 Read block 1942 len 28672
35603 129/300 Read block 1201 len 36864
35604 130/300 Write block 211 len 28672 data: 7 6 6 4 4 4 5
35605 131/300 Write block 4014 len 28672 data: 3 4 3 3 4 5 5
35606 132/300 Read block 3395 len 28672
35607 133/300 Write block 839 len 16384 data: 3 4 4 4
35608 134/300 Read block 68 len 24576
35609 135/300 Write block 4706 len 8192 data: 6 6
35610 136/300 Write block 4369 len 16384 data: 8 6 9 7
35611 137/300 Read block 2673 len 4096
35612 138/300 Flush
35613 139/300 Flush
35614 140/300 Write block 206 len 20480 data: 5 5 6 5 6
35615 141/300 Read block 4539 len 32768
35616 142/300 Write block 3497 len 20480 data: 5 6 6 4 4
35617 143/300 Read block 4276 len 40960
35618 144/300 Write block 3575 len 20480 data: 4 4 4 4 6
35619 145/300 Read block 1000 len 20480
35620 146/300 Write block 15 len 32768 data: 10 10 10 8 6 7 5 6
35621 147/300 Write block 1049 len 32768 data: 11 9 9 8 9 6 7 5
35622 148/300 Read block 2441 len 28672
35623 149/300 Write block 2104 len 12288 data: 6 7 5
35624 150/300 Write block 4514 len 20480 data: 5 6 5 5 7
35625 151/300 Read block 403 len 36864
35626 152/300 Flush
35627 153/300 Read block 973 len 8192
35628 154/300 Read block 4574 len 28672
35629 155/300 Write block 816 len 36864 data: 9 9 7 7 5 5 4 4 4
35630 156/300 Read block 1111 len 28672
35631 157/300 Read block 3611 len 20480
35632 158/300 Read block 913 len 8192
35633 159/300 Write block 3385 len 40960 data: 3 4 6 5 4 4 4 4 4 4
35634 160/300 Write block 2922 len 8192 data: 7 9
35635 161/300 Write block 4149 len 20480 data: 4 5 7 8 8
35636 162/300 Write block 4053 len 12288 data: 7 9 6
35637 163/300 Write block 1600 len 36864 data: 4 4 5 4 3 5 5 6 6
35638 164/300 Write block 2624 len 12288 data: 5 5 4
35639 165/300 Write block 398 len 28672 data: 7 6 5 5 3 3 3
35640 166/300 Read block 2713 len 8192
35641 167/300 Read block 1001 len 8192
35642 168/300 Write block 2094 len 20480 data: 5 5 6 8 7
35643 169/300 Write block 4052 len 32768 data: 7 8 10 7 5 5 4 9
35644 170/300 Write block 4721 len 4096 data: 4
35645 171/300 Write block 215 len 24576 data: 5 5 6 5 5 6
35646 172/300 Write block 1585 len 20480 data: 6 4 4 4 5
35647 173/300 Read block 444 len 12288
35648 174/300 Read block 4946 len 16384
35649 175/300 Write block 2993 len 20480 data: 3 4 3 3 3
35650 176/300 Write block 4637 len 28672 data: 6 7 7 6 8 8 9
35651 177/300 Read block 1616 len 12288
35652 178/300 Read block 3506 len 40960
35653 179/300 Write block 1794 len 8192 data: 7 7
35654 180/300 Read block 2956 len 20480
35655 181/300 Read block 4051 len 32768
35656 182/300 Write block 2779 len 12288 data: 10 10 9
35657 183/300 Write block 3642 len 16384 data: 6 6 8 8
35658 184/300 Write block 3378 len 20480 data: 7 5 3 3 4
35659 185/300 Read block 3171 len 24576
35660 186/300 Write block 4726 len 16384 data: 7 6 5 6
35661 187/300 Read block 3776 len 16384
35662 188/300 Write block 4444 len 24576 data: 6 9 10 9 8 10
35663 189/300 Write block 1888 len 8192 data: 6 5
35664 190/300 Read block 2298 len 32768
35665 191/300 Read block 4021 len 16384
35666 192/300 Write block 544 len 28672 data: 8 7 7 8 7 7 6
35667 193/300 Read block 4331 len 16384
35668 194/300 Read block 3034 len 20480
35669 195/300 Read block 3567 len 12288
35670 196/300 Read block 1815 len 12288
35671 197/300 Read block 4497 len 36864
35672 198/300 Write block 3423 len 4096 data: 4
35673 199/300 Write block 4099 len 4096 data: 5
35674 200/300 Read block 2001 len 24576
35675 201/300 Flush
35676 202/300 Write block 253 len 24576 data: 4 3 4 7 8 9
35677 203/300 Read block 2821 len 32768
35678 204/300 Write block 2328 len 8192 data: 6 5
35679 205/300 Read block 564 len 32768
35680 206/300 Read block 4849 len 20480
35681 207/300 Read block 1244 len 4096
35682 208/300 Flush
35683 209/300 Read block 144 len 4096
35684 210/300 Read block 74 len 40960
35685 211/300 Write block 3424 len 32768 data: 6 8 9 10 11 9 7 6
35686 212/300 Write block 4006 len 16384 data: 3 3 3 3
35687 213/300 Write block 3844 len 16384 data: 7 10 10 9
35688 214/300 Read block 1172 len 32768
35689 215/300 Write block 495 len 12288 data: 3 4 4
35690 216/300 Read block 943 len 4096
35691 217/300 Write block 4909 len 28672 data: 4 5 4 4 4 6 6
35692 218/300 Read block 1204 len 24576
35693 219/300 Write block 2116 len 40960 data: 8 6 6 5 3 3 4 7 9 7
35694 220/300 Write block 3670 len 12288 data: 7 7 9
35695 221/300 Write block 3678 len 20480 data: 4 4 5 5 4
35696 222/300 Read block 1821 len 24576
35697 223/300 Write block 2792 len 36864 data: 10 8 7 5 5 3 3 3 4
35698 224/300 Write block 4956 len 36864 data: 5 6 8 9 9 9 10 9 10
35699 225/300 Read block 801 len 24576
35700 226/300 Read block 1716 len 20480
35701 227/300 Flush
35702 228/300 Write block 292 len 28672 data: 5 7 6 6 6 6 5
35703 229/300 Write block 1568 len 8192 data: 4 4
35704 230/300 Write block 3600 len 36864 data: 8 9 6 6 6 5 5 6 7
35705 231/300 Read block 1437 len 40960
35706 232/300 Read block 3700 len 16384
35707 233/300 Write block 601 len 28672 data: 11 13 10 10 10 8 7
35708 234/300 Write block 3818 len 20480 data: 8 10 11 10 9
35709 235/300 Write block 2818 len 32768 data: 4 4 3 2 3 3 4 5
35710 236/300 Flush
35711 237/300 Read block 855 len 4096
35712 238/300 Read block 3469 len 12288
35713 239/300 Read block 4729 len 32768
35714 240/300 Write block 2694 len 8192 data: 5 5
35715 241/300 Read block 1305 len 12288
35716 242/300 Write block 2602 len 12288 data: 4 4 4
35717 243/300 Read block 1562 len 16384
35718 244/300 Write block 4830 len 20480 data: 10 7 6 6 6
35719 245/300 Read block 3391 len 40960
35720 246/300 Read block 2794 len 12288
35721 247/300 Read block 3689 len 16384
35722 248/300 Read block 1901 len 36864
35723 249/300 Read block 566 len 12288
35724 250/300 Write block 2957 len 12288 data: 6 7 8
35725 251/300 Read block 2238 len 40960
35726 252/300 Write block 2771 len 32768 data: 8 8 9 8 9 9 8 8
35727 253/300 Write block 1447 len 20480 data: 2 3 3 3 2
35728 254/300 Read block 3543 len 12288
35729 255/300 Write block 4759 len 4096 data: 6
35730 256/300 Read block 2343 len 16384
35731 257/300 Flush
35732 258/300 Write block 659 len 24576 data: 12 11 11 12 12 9
35733 259/300 Read block 1279 len 40960
35734 260/300 Write block 1351 len 4096 data: 5
35735 261/300 Flush
35736 262/300 Read block 3046 len 4096
35737 263/300 Read block 4164 len 24576
35738 264/300 Flush
35739 265/300 Read block 457 len 12288
35740 266/300 Write block 469 len 4096 data: 11
35741 267/300 Write block 4021 len 8192 data: 5 4
35742 268/300 Read block 2785 len 24576
35743 269/300 Flush
35744 270/300 Write block 1043 len 12288 data: 5 5 6
35745 271/300 Write block 1817 len 12288 data: 9 8 9
35746 272/300 Write block 2850 len 36864 data: 6 7 7 6 3 4 4 3 3
35747 273/300 Read block 2125 len 16384
35748 274/300 Write block 1565 len 4096 data: 5
35749 275/300 Read block 3701 len 12288
35750 276/300 Write block 1531 len 36864 data: 5 6 5 7 8 7 8 8 9
35751 277/300 Read block 4751 len 8192
35752 278/300 Read block 2309 len 16384
35753 279/300 Read block 880 len 28672
35754 280/300 Write block 4731 len 4096 data: 3
35755 281/300 Write block 2974 len 8192 data: 7 8
35756 282/300 Read block 1730 len 12288
35757 283/300 Read block 2316 len 36864
35758 284/300 Read block 3402 len 16384
35759 285/300 Read block 2597 len 12288
35760 286/300 Write block 2793 len 24576 data: 9 8 6 6 4 4
35761 287/300 Read block 4865 len 36864
35762 288/300 Read block 2903 len 24576
35763 289/300 Write block 439 len 12288 data: 8 8 7
35764 290/300 Write block 843 len 16384 data: 5 7 8 9
35765 291/300 Write block 3364 len 20480 data: 5 4 4 4 5
35766 292/300 Read block 1845 len 8192
35767 293/300 Read block 2549 len 20480
35768 294/300 Read block 4119 len 28672
35769 295/300 Write block 3717 len 36864 data: 3 3 4 4 5 6 6 6 5
35770 296/300 Write block 2967 len 4096 data: 5
35771 297/300 Write block 4065 len 16384 data: 9 8 8 10
35772 298/300 Read block 3819 len 32768
35773 299/300 Write block 1730 len 16384 data: 3 4 4 3
35774 300/300 Write block 2272 len 16384 data: 8 7 7 7
35775 301/300 Read block 2370 len 32768
35776 Main task has work to do, go find it
35777 got dsc Start(2)
35778 start 2
35779 Nov 28 04:57:50.330 INFO request completed, latency_us: 214, response_code: 204, uri: /start/cid/2, method: POST, req_id: 5a0d5a23-0550-4dbc-99c0-94fd55b5d622, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
35780 Replay: started 2, returned:Ok(())
35781 ----------------------------------------------------------------
35782 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
35783 States: Active Active Offline
35784 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
35785 9755 Acked Read 5 Done Done Sent false
35786 9756 Acked Write 5 Done Done Sent false
35787 9757 Acked Flush 0 Done Done Sent false
35788 9758 Acked Write 5 Done Done Sent false
35789 9759 Acked Flush 0 Done Done Sent false
35790 9760 Acked Write 5 Done Done Sent false
35791 9761 Acked Write 7 Done Done Sent false
35792 9762 Acked Read 3 Done Done Sent false
35793 9763 Acked Read 9 Done Done Sent false
35794 9764 Acked Write 9 Done Done Sent false
35795 9765 Acked Flush 0 Done Done Sent false
35796 9766 Acked Flush 0 Done Done Sent false
35797 9767 Acked Write 9 Done Done Sent false
35798 9768 Acked Write 8 Done Done Sent false
35799 9769 Acked Read 5 Done Done Sent false
35800 9770 Acked Read 1 Done Done Sent false
35801 9771 Acked Read 10 Done Done Sent false
35802 9772 Acked Write 1 Done Done Sent false
35803 9773 Acked Write 7 Done Done Sent false
35804 9774 Acked Write 3 Done Done Sent false
35805 9775 Acked Read 1 Done Done Sent false
35806 9776 Acked Read 7 Done Done Sent false
35807 9777 Acked Write 8 Done Done Sent false
35808 9778 Acked Read 9 Done Done Sent false
35809 9779 Acked Write 6 Done Done Sent false
35810 9780 Acked Flush 0 Done Done Sent false
35811 9781 Acked Read 6 Done Done Sent false
35812 9782 Acked Read 10 Done Done Sent false
35813 9783 Acked Read 7 Done Done Sent false
35814 9784 Acked Read 5 Done Done Sent false
35815 9785 Acked Write 5 Done Done Sent false
35816 9786 Acked Read 8 Done Done Sent false
35817 9787 Acked Read 3 Done Done Sent false
35818 9788 Acked Write 3 Done Done Sent false
35819 9789 Acked Write 4 Done Done Sent false
35820 9790 Acked Write 4 Done Done Sent false
35821 9791 Acked Write 2 Done Done Sent false
35822 9792 Acked Write 4 Done Done Sent false
35823 9793 Acked Read 10 Done Done Sent false
35824 9794 Acked Write 9 Done Done Sent false
35825 9795 Acked Read 1 Done Done Sent false
35826 9796 Acked Flush 0 Done Done Sent false
35827 9797 Acked Read 7 Done Done Sent false
35828 9798 Acked Write 8 Done Done Sent false
35829 9799 Acked Read 1 Done Done Sent false
35830 9800 Acked Read 6 Done Done Sent false
35831 9801 Acked Write 2 Done Done Sent false
35832 9802 Acked Write 3 Done Done Sent false
35833 9803 Acked Write 5 Done Done Sent false
35834 9804 Acked Write 7 Done Done Sent false
35835 9805 Acked Read 3 Done Done Sent false
35836 9806 Acked Read 2 Done Done Sent false
35837 9807 Acked Write 2 Done Done Sent false
35838 9808 Acked Flush 0 Done Done Sent false
35839 9809 Acked Write 3 Done Done Sent false
35840 9810 Acked Write 1 Done Done Sent false
35841 9811 Acked Write 2 Done Done Sent false
35842 9812 Acked Write 6 Done Done Sent false
35843 9813 Acked Flush 0 Done Done Sent false
35844 9814 Acked Write 4 Done Done Sent false
35845 9815 Acked Read 8 Done Done Sent false
35846 9816 Acked Write 2 Done Done Sent false
35847 9817 Acked Read 10 Done Done Sent false
35848 9818 Acked Read 6 Done Done Sent false
35849 9819 Acked Flush 0 Done Done Sent false
35850 9820 Acked Read 4 Done Done Sent false
35851 9821 Acked Read 9 Done Done Sent false
35852 9822 Acked Read 9 Done Done Sent false
35853 9823 Acked Flush 0 Done Done Sent false
35854 9824 Acked Read 4 Done Done Sent false
35855 9825 Acked Write 9 Done Done Sent false
35856 9826 Acked Read 9 Done Done Sent false
35857 9827 Acked Flush 0 Done Done Sent false
35858 9828 Acked Flush 0 Done Done Sent false
35859 9829 Acked Read 4 Done Done Sent false
35860 9830 Acked Read 7 Done Done Sent false
35861 9831 Acked Read 5 Done Done Sent false
35862 9832 Acked Read 2 Done Done Sent false
35863 9833 Acked Flush 0 Done Done Sent false
35864 9834 Acked Write 7 Done Done Sent false
35865 9835 Acked Write 10 Done Done Sent false
35866 9836 Acked Write 10 Done Done Sent false
35867 9837 Acked Read 1 Done Done Sent false
35868 9838 Acked Write 6 Done Done Sent false
35869 9839 Acked Write 2 Done Done Sent false
35870 9840 Acked Write 1 Done Done Sent false
35871 9841 Acked Write 7 Done Done Sent false
35872 9842 Acked Read 1 Done Done Sent false
35873 9843 Acked Read 4 Done Done Sent false
35874 9844 Acked Write 9 Done Done Sent false
35875 9845 Acked Write 2 Done Done Sent false
35876 9846 Acked Read 2 Done Done Sent false
35877 9847 Acked Write 1 Done Done Sent false
35878 9848 Acked Write 3 Done Done Sent false
35879 9849 Acked Read 9 Done Done Sent false
35880 9850 Acked Write 9 Done Done Sent false
35881 9851 Acked Read 3 Done Done Sent false
35882 9852 Acked Write 8 Done Done Sent false
35883 9853 Acked Write 2 Done Done Sent false
35884 9854 Acked Write 8 Done Done Sent false
35885 9855 Acked Read 9 Done Done Sent false
35886 9856 Acked Read 5 Done Done Sent false
35887 9857 Acked Read 6 Done Done Sent false
35888 9858 Acked Write 5 Done Done Sent false
35889 9859 Acked Write 6 Done Done Sent false
35890 9860 Acked Write 1 Done Done Sent false
35891 9861 Acked Read 7 Done Done Sent false
35892 9862 Acked Read 2 Done Done Sent false
35893 9863 Acked Read 5 Done Done Sent false
35894 9864 Acked Write 2 Done Done Sent false
35895 9865 Acked Read 7 Done Done Sent false
35896 9866 Acked Read 1 Done Done Sent false
35897 9867 Acked Read 1 Done Done Sent false
35898 9868 Acked Read 10 Done Done Sent false
35899 9869 Acked Read 3 Done Done Sent false
35900 9870 Acked Read 9 Done Done Sent false
35901 9871 Acked Read 8 Done Done Sent false
35902 9872 Acked Flush 0 Done Done Sent false
35903 9873 Acked Write 2 Done Done Sent false
35904 9874 Acked Read 4 Done Done Sent false
35905 9875 Acked Write 2 Done Done Sent false
35906 9876 Acked Write 7 Done Done Sent false
35907 9877 Acked Write 6 Done Done Sent false
35908 9878 Acked Write 6 Done Done Sent false
35909 9879 Acked Write 7 Done Done Sent false
35910 9880 Acked Read 2 Done Done Sent false
35911 9881 Acked Read 10 Done Done Sent false
35912 9882 Acked Read 7 Done Done Sent false
35913 9883 Acked Read 9 Done Done Sent false
35914 9884 Acked Write 7 Done Done Sent false
35915 9885 Acked Write 7 Done Done Sent false
35916 9886 Acked Read 7 Done Done Sent false
35917 9887 Acked Write 4 Done Done Sent false
35918 9888 Acked Read 6 Done Done Sent false
35919 9889 Acked Write 2 Done Done Sent false
35920 9890 Acked Write 4 Done Done Sent false
35921 9891 Acked Read 1 Done Done Sent false
35922 9892 Acked Flush 0 Done Done Sent false
35923 9893 Acked Flush 0 Done Done Sent false
35924 9894 Acked Write 5 Done Done Sent false
35925 9895 Acked Read 8 Done Done Sent false
35926 9896 Acked Write 5 Done Done Sent false
35927 9897 Acked Read 10 Done Done Sent false
35928 9898 Acked Write 5 Done Done Sent false
35929 9899 Acked Read 5 Done Done Sent false
35930 9900 Acked Write 8 Done Done Sent false
35931 9901 Acked Write 8 Done Done Sent false
35932 9902 Acked Read 7 Done Done Sent false
35933 9903 Acked Write 3 Done Done Sent false
35934 9904 Acked Write 5 Done Done Sent false
35935 9905 Acked Read 9 Done Done Sent false
35936 9906 Acked Flush 0 Done Done Sent false
35937 9907 Acked Read 2 Done Done Sent false
35938 9908 Acked Read 7 Done Done Sent false
35939 9909 Acked Write 9 Done Done Sent false
35940 9910 Acked Read 7 Done Done Sent false
35941 9911 Acked Read 5 Done Done Sent false
35942 9912 Acked Read 2 Done Done Sent false
35943 9913 Acked Write 10 Done Done Sent false
35944 9914 Acked Write 2 Done Done Sent false
35945 9915 Acked Write 5 Done Done Sent false
35946 9916 Acked Write 3 Done Done Sent false
35947 9917 Acked Write 9 Done Done Sent false
35948 9918 Acked Write 3 Done Done Sent false
35949 9919 Acked Write 7 Done Done Sent false
35950 9920 Acked Read 2 Done Done Sent false
35951 9921 Acked Read 2 Done Done Sent false
35952 9922 Acked Write 5 Done Done Sent false
35953 9923 Acked Write 8 Done Done Sent false
35954 9924 Acked Write 1 Done Done Sent false
35955 9925 Acked Write 6 Done Done Sent false
35956 9926 Acked Write 5 Done Done Sent false
35957 9927 Acked Read 3 Done Done Sent false
35958 9928 Acked Read 4 Done Done Sent false
35959 9929 Acked Write 5 Done Done Sent false
35960 9930 Acked Write 7 Done Done Sent false
35961 9931 Acked Read 3 Done Done Sent false
35962 9932 Acked Read 10 Done Done Sent false
35963 9933 Acked Write 2 Done Done Sent false
35964 9934 Acked Read 5 Done Done Sent false
35965 9935 Acked Read 8 Done Done Sent false
35966 9936 Acked Write 3 Done Done Sent false
35967 9937 Acked Write 4 Done Done Sent false
35968 9938 Acked Write 5 Done Done Sent false
35969 9939 Acked Read 6 Done Done Sent false
35970 9940 Acked Write 4 Done Done Sent false
35971 9941 Acked Read 4 Done Done Sent false
35972 9942 Acked Write 6 Done Done Sent false
35973 9943 Acked Write 2 Done Done Sent false
35974 9944 Acked Read 8 Done Done Sent false
35975 9945 Acked Read 4 Done Done Sent false
35976 9946 Acked Write 7 Done Done Sent false
35977 9947 Acked Read 4 Done Done Sent false
35978 9948 Acked Read 5 Done Done Sent false
35979 9949 Acked Read 3 Done Done Sent false
35980 9950 Acked Read 3 Done Done Sent false
35981 9951 Acked Read 9 Done Done Sent false
35982 9952 Acked Write 1 Done Done Sent false
35983 9953 Acked Write 1 Done Done Sent false
35984 9954 Acked Read 6 Done Done Sent false
35985 9955 Acked Flush 0 Done Done Sent false
35986 9956 Acked Write 6 Done Done Sent false
35987 9957 Acked Read 8 Done Done Sent false
35988 9958 Acked Write 2 Done Done Sent false
35989 9959 Acked Read 8 Done Done Sent false
35990 9960 Acked Read 5 Done Done Sent false
35991 9961 Acked Read 1 Done Done Sent false
35992 9962 Acked Flush 0 Done Done Sent false
35993 9963 Acked Read 1 Done Done Sent false
35994 9964 Acked Read 10 Done Done Sent false
35995 9965 Acked Write 8 Done Done Sent false
35996 9966 Acked Write 4 Done Done Sent false
35997 9967 Acked Write 4 Done Done Sent false
35998 9968 Acked Read 8 Done Done Sent false
35999 9969 Acked Write 3 Done Done Sent false
36000 9970 Acked Read 1 Done Done Sent false
36001 9971 Acked Write 7 Done Done Sent false
36002 9972 Acked Read 6 Done Done Sent false
36003 9973 Acked Write 10 Done Done Sent false
36004 9974 Acked Write 3 Done Done Sent false
36005 9975 Acked Write 5 Done Done Sent false
36006 9976 Acked Read 6 Done Done Sent false
36007 9977 Acked Write 9 Done Done Sent false
36008 9978 Acked Write 9 Done Done Sent false
36009 9979 Acked Read 6 Done Done Sent false
36010 9980 Acked Read 5 Done Done Sent false
36011 9981 Acked Flush 0 Done Done Sent false
36012 9982 Acked Write 7 Done Done Sent false
36013 9983 Acked Write 2 Done Done Sent false
36014 9984 Acked Write 9 Done Done Sent false
36015 9985 Acked Read 10 Done Done Sent false
36016 9986 Acked Read 4 Done Done Sent false
36017 9987 Acked Write 7 Done Done Sent false
36018 9988 Acked Write 5 Done Done Sent false
36019 9989 Acked Write 8 Done Done Sent false
36020 9990 Acked Flush 0 Done Done Sent false
36021 9991 Acked Read 1 Done Done Sent false
36022 9992 Acked Read 3 Done Done Sent false
36023 9993 Acked Read 8 Done Done Sent false
36024 9994 Acked Write 2 Done Done Sent false
36025 9995 Acked Read 3 Done Done Sent false
36026 9996 Acked Write 3 Done Done Sent false
36027 9997 Acked Read 4 Done Done Sent false
36028 9998 Acked Write 5 Done Done Sent false
36029 9999 Acked Read 10 Done Done Sent false
36030 10000 Acked Read 3 Done Done Sent false
36031 10001 Acked Read 4 Done Done Sent false
36032 10002 Acked Read 9 Done Done Sent false
36033 10003 Acked Read 3 Done Done Sent false
36034 10004 Acked Write 3 Done Done Sent false
36035 10005 Acked Read 10 Done Done Sent false
36036 10006 Acked Write 8 Done Done Sent false
36037 10007 Acked Write 5 Done Done Sent false
36038 10008 Acked Read 3 Done Done Sent false
36039 10009 Acked Write 1 Done Done Sent false
36040 10010 Acked Read 4 Done Done Sent false
36041 10011 Acked Flush 0 Done Done Sent false
36042 10012 Acked Write 6 Done Done Sent false
36043 10013 Acked Read 10 Done Done Sent false
36044 10014 Acked Write 1 Done Done Sent false
36045 10015 Acked Flush 0 Done Done Sent false
36046 10016 Acked Read 1 Done Done Sent false
36047 10017 Acked Read 6 Done Done Sent false
36048 10018 Acked Flush 0 Done Done Sent false
36049 10019 Acked Read 3 Done Done Sent false
36050 10020 Acked Write 1 Done Done Sent false
36051 10021 Acked Write 2 Done Done Sent false
36052 10022 Acked Read 6 Done Done Sent false
36053 10023 Acked Flush 0 Done Done Sent false
36054 10024 Acked Write 3 Done Done Sent false
36055 10025 Acked Write 3 Done Done Sent false
36056 10026 Acked Write 9 Done Done Sent false
36057 10027 Acked Read 4 Done Done Sent false
36058 10028 Acked Write 1 Done Done Sent false
36059 10029 Acked Read 3 Done Done Sent false
36060 10030 Acked Write 9 Done Done Sent false
36061 10031 Acked Read 2 Done Done Sent false
36062 10032 Acked Read 4 Done Done Sent false
36063 10033 Acked Read 7 Done Done Sent false
36064 10034 Acked Write 1 Done Done Sent false
36065 10035 Acked Write 2 Done Done Sent false
36066 10036 Acked Read 3 Done Done Sent false
36067 10037 Acked Read 9 Done Done Sent false
36068 10038 Acked Read 4 Done Done Sent false
36069 10039 Acked Read 3 Done Done Sent false
36070 10040 Acked Write 6 Done Done Sent false
36071 10041 Acked Read 9 Done Done Sent false
36072 10042 Acked Read 6 Done Done Sent false
36073 10043 Acked Write 3 Done Done Sent false
36074 10044 Acked Write 4 Done Done Sent false
36075 10045 Acked Write 5 Done Done Sent false
36076 10046 Acked Read 2 Done Done Sent false
36077 10047 Acked Read 5 Done Done Sent false
36078 10048 Acked Read 7 Done Done Sent false
36079 10049 Acked Write 9 Done Done Sent false
36080 10050 Acked Write 1 Done Done Sent false
36081 10051 Acked Write 4 Done Done Sent false
36082 10052 Acked Read 8 Done Done Sent false
36083 10053 Acked Write 4 Done Done Sent false
36084 10054 Acked Write 4 Done Done Sent false
36085 10055 Acked Read 8 Done Done Sent false
36086 STATES DS:0 DS:1 DS:2 TOTAL
36087 Sent 0 0 301 301
36088 Done 301 301 0 602
36089 Skipped 0 0 0 0
36090 Error 0 0 0 0
36091 Last Flush: 10023 10023 9754
36092 Downstairs last five completed: 9754 9753 9752 9751 9750
36093 Upstairs last five completed: 10055 10054 10053 10052 10051
36094 CLIENT: Up:0 ds:301 act:2
36095 [8830] got start action
36096 [8830] I am going to restart
36097 Starting downstairs at port 8830
36098 Make output file at "/tmp/dsc/downstairs-8830.txt"
36099 [8830][2] reports Starting
36100 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1232)
36101 [8830][2] reports Running
36102 ----------------------------------------------------------------
36103 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
36104 States: Active Active Offline
36105 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
36106 9755 Acked Read 5 Done Done Sent false
36107 9756 Acked Write 5 Done Done Sent false
36108 9757 Acked Flush 0 Done Done Sent false
36109 9758 Acked Write 5 Done Done Sent false
36110 9759 Acked Flush 0 Done Done Sent false
36111 9760 Acked Write 5 Done Done Sent false
36112 9761 Acked Write 7 Done Done Sent false
36113 9762 Acked Read 3 Done Done Sent false
36114 9763 Acked Read 9 Done Done Sent false
36115 9764 Acked Write 9 Done Done Sent false
36116 9765 Acked Flush 0 Done Done Sent false
36117 9766 Acked Flush 0 Done Done Sent false
36118 9767 Acked Write 9 Done Done Sent false
36119 9768 Acked Write 8 Done Done Sent false
36120 9769 Acked Read 5 Done Done Sent false
36121 9770 Acked Read 1 Done Done Sent false
36122 9771 Acked Read 10 Done Done Sent false
36123 9772 Acked Write 1 Done Done Sent false
36124 9773 Acked Write 7 Done Done Sent false
36125 9774 Acked Write 3 Done Done Sent false
36126 9775 Acked Read 1 Done Done Sent false
36127 9776 Acked Read 7 Done Done Sent false
36128 9777 Acked Write 8 Done Done Sent false
36129 9778 Acked Read 9 Done Done Sent false
36130 9779 Acked Write 6 Done Done Sent false
36131 9780 Acked Flush 0 Done Done Sent false
36132 9781 Acked Read 6 Done Done Sent false
36133 9782 Acked Read 10 Done Done Sent false
36134 9783 Acked Read 7 Done Done Sent false
36135 9784 Acked Read 5 Done Done Sent false
36136 9785 Acked Write 5 Done Done Sent false
36137 9786 Acked Read 8 Done Done Sent false
36138 9787 Acked Read 3 Done Done Sent false
36139 9788 Acked Write 3 Done Done Sent false
36140 9789 Acked Write 4 Done Done Sent false
36141 9790 Acked Write 4 Done Done Sent false
36142 9791 Acked Write 2 Done Done Sent false
36143 9792 Acked Write 4 Done Done Sent false
36144 9793 Acked Read 10 Done Done Sent false
36145 9794 Acked Write 9 Done Done Sent false
36146 9795 Acked Read 1 Done Done Sent false
36147 9796 Acked Flush 0 Done Done Sent false
36148 9797 Acked Read 7 Done Done Sent false
36149 9798 Acked Write 8 Done Done Sent false
36150 9799 Acked Read 1 Done Done Sent false
36151 9800 Acked Read 6 Done Done Sent false
36152 9801 Acked Write 2 Done Done Sent false
36153 9802 Acked Write 3 Done Done Sent false
36154 9803 Acked Write 5 Done Done Sent false
36155 9804 Acked Write 7 Done Done Sent false
36156 9805 Acked Read 3 Done Done Sent false
36157 9806 Acked Read 2 Done Done Sent false
36158 9807 Acked Write 2 Done Done Sent false
36159 9808 Acked Flush 0 Done Done Sent false
36160 9809 Acked Write 3 Done Done Sent false
36161 9810 Acked Write 1 Done Done Sent false
36162 9811 Acked Write 2 Done Done Sent false
36163 9812 Acked Write 6 Done Done Sent false
36164 9813 Acked Flush 0 Done Done Sent false
36165 9814 Acked Write 4 Done Done Sent false
36166 9815 Acked Read 8 Done Done Sent false
36167 9816 Acked Write 2 Done Done Sent false
36168 9817 Acked Read 10 Done Done Sent false
36169 9818 Acked Read 6 Done Done Sent false
36170 9819 Acked Flush 0 Done Done Sent false
36171 9820 Acked Read 4 Done Done Sent false
36172 9821 Acked Read 9 Done Done Sent false
36173 9822 Acked Read 9 Done Done Sent false
36174 9823 Acked Flush 0 Done Done Sent false
36175 9824 Acked Read 4 Done Done Sent false
36176 9825 Acked Write 9 Done Done Sent false
36177 9826 Acked Read 9 Done Done Sent false
36178 9827 Acked Flush 0 Done Done Sent false
36179 9828 Acked Flush 0 Done Done Sent false
36180 9829 Acked Read 4 Done Done Sent false
36181 9830 Acked Read 7 Done Done Sent false
36182 9831 Acked Read 5 Done Done Sent false
36183 9832 Acked Read 2 Done Done Sent false
36184 9833 Acked Flush 0 Done Done Sent false
36185 9834 Acked Write 7 Done Done Sent false
36186 9835 Acked Write 10 Done Done Sent false
36187 9836 Acked Write 10 Done Done Sent false
36188 9837 Acked Read 1 Done Done Sent false
36189 9838 Acked Write 6 Done Done Sent false
36190 9839 Acked Write 2 Done Done Sent false
36191 9840 Acked Write 1 Done Done Sent false
36192 9841 Acked Write 7 Done Done Sent false
36193 9842 Acked Read 1 Done Done Sent false
36194 9843 Acked Read 4 Done Done Sent false
36195 9844 Acked Write 9 Done Done Sent false
36196 9845 Acked Write 2 Done Done Sent false
36197 9846 Acked Read 2 Done Done Sent false
36198 9847 Acked Write 1 Done Done Sent false
36199 9848 Acked Write 3 Done Done Sent false
36200 9849 Acked Read 9 Done Done Sent false
36201 9850 Acked Write 9 Done Done Sent false
36202 9851 Acked Read 3 Done Done Sent false
36203 9852 Acked Write 8 Done Done Sent false
36204 9853 Acked Write 2 Done Done Sent false
36205 9854 Acked Write 8 Done Done Sent false
36206 9855 Acked Read 9 Done Done Sent false
36207 9856 Acked Read 5 Done Done Sent false
36208 9857 Acked Read 6 Done Done Sent false
36209 9858 Acked Write 5 Done Done Sent false
36210 9859 Acked Write 6 Done Done Sent false
36211 9860 Acked Write 1 Done Done Sent false
36212 9861 Acked Read 7 Done Done Sent false
36213 9862 Acked Read 2 Done Done Sent false
36214 9863 Acked Read 5 Done Done Sent false
36215 9864 Acked Write 2 Done Done Sent false
36216 9865 Acked Read 7 Done Done Sent false
36217 9866 Acked Read 1 Done Done Sent false
36218 9867 Acked Read 1 Done Done Sent false
36219 9868 Acked Read 10 Done Done Sent false
36220 9869 Acked Read 3 Done Done Sent false
36221 9870 Acked Read 9 Done Done Sent false
36222 9871 Acked Read 8 Done Done Sent false
36223 9872 Acked Flush 0 Done Done Sent false
36224 9873 Acked Write 2 Done Done Sent false
36225 9874 Acked Read 4 Done Done Sent false
36226 9875 Acked Write 2 Done Done Sent false
36227 9876 Acked Write 7 Done Done Sent false
36228 9877 Acked Write 6 Done Done Sent false
36229 9878 Acked Write 6 Done Done Sent false
36230 9879 Acked Write 7 Done Done Sent false
36231 9880 Acked Read 2 Done Done Sent false
36232 9881 Acked Read 10 Done Done Sent false
36233 9882 Acked Read 7 Done Done Sent false
36234 9883 Acked Read 9 Done Done Sent false
36235 9884 Acked Write 7 Done Done Sent false
36236 9885 Acked Write 7 Done Done Sent false
36237 9886 Acked Read 7 Done Done Sent false
36238 9887 Acked Write 4 Done Done Sent false
36239 9888 Acked Read 6 Done Done Sent false
36240 9889 Acked Write 2 Done Done Sent false
36241 9890 Acked Write 4 Done Done Sent false
36242 9891 Acked Read 1 Done Done Sent false
36243 9892 Acked Flush 0 Done Done Sent false
36244 9893 Acked Flush 0 Done Done Sent false
36245 9894 Acked Write 5 Done Done Sent false
36246 9895 Acked Read 8 Done Done Sent false
36247 9896 Acked Write 5 Done Done Sent false
36248 9897 Acked Read 10 Done Done Sent false
36249 9898 Acked Write 5 Done Done Sent false
36250 9899 Acked Read 5 Done Done Sent false
36251 9900 Acked Write 8 Done Done Sent false
36252 9901 Acked Write 8 Done Done Sent false
36253 9902 Acked Read 7 Done Done Sent false
36254 9903 Acked Write 3 Done Done Sent false
36255 9904 Acked Write 5 Done Done Sent false
36256 9905 Acked Read 9 Done Done Sent false
36257 9906 Acked Flush 0 Done Done Sent false
36258 9907 Acked Read 2 Done Done Sent false
36259 9908 Acked Read 7 Done Done Sent false
36260 9909 Acked Write 9 Done Done Sent false
36261 9910 Acked Read 7 Done Done Sent false
36262 9911 Acked Read 5 Done Done Sent false
36263 9912 Acked Read 2 Done Done Sent false
36264 9913 Acked Write 10 Done Done Sent false
36265 9914 Acked Write 2 Done Done Sent false
36266 9915 Acked Write 5 Done Done Sent false
36267 9916 Acked Write 3 Done Done Sent false
36268 9917 Acked Write 9 Done Done Sent false
36269 9918 Acked Write 3 Done Done Sent false
36270 9919 Acked Write 7 Done Done Sent false
36271 9920 Acked Read 2 Done Done Sent false
36272 9921 Acked Read 2 Done Done Sent false
36273 9922 Acked Write 5 Done Done Sent false
36274 9923 Acked Write 8 Done Done Sent false
36275 9924 Acked Write 1 Done Done Sent false
36276 9925 Acked Write 6 Done Done Sent false
36277 9926 Acked Write 5 Done Done Sent false
36278 9927 Acked Read 3 Done Done Sent false
36279 9928 Acked Read 4 Done Done Sent false
36280 9929 Acked Write 5 Done Done Sent false
36281 9930 Acked Write 7 Done Done Sent false
36282 9931 Acked Read 3 Done Done Sent false
36283 9932 Acked Read 10 Done Done Sent false
36284 9933 Acked Write 2 Done Done Sent false
36285 9934 Acked Read 5 Done Done Sent false
36286 9935 Acked Read 8 Done Done Sent false
36287 9936 Acked Write 3 Done Done Sent false
36288 9937 Acked Write 4 Done Done Sent false
36289 9938 Acked Write 5 Done Done Sent false
36290 9939 Acked Read 6 Done Done Sent false
36291 9940 Acked Write 4 Done Done Sent false
36292 9941 Acked Read 4 Done Done Sent false
36293 9942 Acked Write 6 Done Done Sent false
36294 9943 Acked Write 2 Done Done Sent false
36295 9944 Acked Read 8 Done Done Sent false
36296 9945 Acked Read 4 Done Done Sent false
36297 9946 Acked Write 7 Done Done Sent false
36298 9947 Acked Read 4 Done Done Sent false
36299 9948 Acked Read 5 Done Done Sent false
36300 9949 Acked Read 3 Done Done Sent false
36301 9950 Acked Read 3 Done Done Sent false
36302 9951 Acked Read 9 Done Done Sent false
36303 9952 Acked Write 1 Done Done Sent false
36304 9953 Acked Write 1 Done Done Sent false
36305 9954 Acked Read 6 Done Done Sent false
36306 9955 Acked Flush 0 Done Done Sent false
36307 9956 Acked Write 6 Done Done Sent false
36308 9957 Acked Read 8 Done Done Sent false
36309 9958 Acked Write 2 Done Done Sent false
36310 9959 Acked Read 8 Done Done Sent false
36311 9960 Acked Read 5 Done Done Sent false
36312 9961 Acked Read 1 Done Done Sent false
36313 9962 Acked Flush 0 Done Done Sent false
36314 9963 Acked Read 1 Done Done Sent false
36315 9964 Acked Read 10 Done Done Sent false
36316 9965 Acked Write 8 Done Done Sent false
36317 9966 Acked Write 4 Done Done Sent false
36318 9967 Acked Write 4 Done Done Sent false
36319 9968 Acked Read 8 Done Done Sent false
36320 9969 Acked Write 3 Done Done Sent false
36321 9970 Acked Read 1 Done Done Sent false
36322 9971 Acked Write 7 Done Done Sent false
36323 9972 Acked Read 6 Done Done Sent false
36324 9973 Acked Write 10 Done Done Sent false
36325 9974 Acked Write 3 Done Done Sent false
36326 9975 Acked Write 5 Done Done Sent false
36327 9976 Acked Read 6 Done Done Sent false
36328 9977 Acked Write 9 Done Done Sent false
36329 9978 Acked Write 9 Done Done Sent false
36330 9979 Acked Read 6 Done Done Sent false
36331 9980 Acked Read 5 Done Done Sent false
36332 9981 Acked Flush 0 Done Done Sent false
36333 9982 Acked Write 7 Done Done Sent false
36334 9983 Acked Write 2 Done Done Sent false
36335 9984 Acked Write 9 Done Done Sent false
36336 9985 Acked Read 10 Done Done Sent false
36337 9986 Acked Read 4 Done Done Sent false
36338 9987 Acked Write 7 Done Done Sent false
36339 9988 Acked Write 5 Done Done Sent false
36340 9989 Acked Write 8 Done Done Sent false
36341 9990 Acked Flush 0 Done Done Sent false
36342 9991 Acked Read 1 Done Done Sent false
36343 9992 Acked Read 3 Done Done Sent false
36344 9993 Acked Read 8 Done Done Sent false
36345 9994 Acked Write 2 Done Done Sent false
36346 9995 Acked Read 3 Done Done Sent false
36347 9996 Acked Write 3 Done Done Sent false
36348 9997 Acked Read 4 Done Done Sent false
36349 9998 Acked Write 5 Done Done Sent false
36350 9999 Acked Read 10 Done Done Sent false
36351 10000 Acked Read 3 Done Done Sent false
36352 10001 Acked Read 4 Done Done Sent false
36353 10002 Acked Read 9 Done Done Sent false
36354 10003 Acked Read 3 Done Done Sent false
36355 10004 Acked Write 3 Done Done Sent false
36356 10005 Acked Read 10 Done Done Sent false
36357 10006 Acked Write 8 Done Done Sent false
36358 10007 Acked Write 5 Done Done Sent false
36359 10008 Acked Read 3 Done Done Sent false
36360 10009 Acked Write 1 Done Done Sent false
36361 10010 Acked Read 4 Done Done Sent false
36362 10011 Acked Flush 0 Done Done Sent false
36363 10012 Acked Write 6 Done Done Sent false
36364 10013 Acked Read 10 Done Done Sent false
36365 10014 Acked Write 1 Done Done Sent false
36366 10015 Acked Flush 0 Done Done Sent false
36367 10016 Acked Read 1 Done Done Sent false
36368 10017 Acked Read 6 Done Done Sent false
36369 10018 Acked Flush 0 Done Done Sent false
36370 10019 Acked Read 3 Done Done Sent false
36371 10020 Acked Write 1 Done Done Sent false
36372 10021 Acked Write 2 Done Done Sent false
36373 10022 Acked Read 6 Done Done Sent false
36374 10023 Acked Flush 0 Done Done Sent false
36375 10024 Acked Write 3 Done Done Sent false
36376 10025 Acked Write 3 Done Done Sent false
36377 10026 Acked Write 9 Done Done Sent false
36378 10027 Acked Read 4 Done Done Sent false
36379 10028 Acked Write 1 Done Done Sent false
36380 10029 Acked Read 3 Done Done Sent false
36381 10030 Acked Write 9 Done Done Sent false
36382 10031 Acked Read 2 Done Done Sent false
36383 10032 Acked Read 4 Done Done Sent false
36384 10033 Acked Read 7 Done Done Sent false
36385 10034 Acked Write 1 Done Done Sent false
36386 10035 Acked Write 2 Done Done Sent false
36387 10036 Acked Read 3 Done Done Sent false
36388 10037 Acked Read 9 Done Done Sent false
36389 10038 Acked Read 4 Done Done Sent false
36390 10039 Acked Read 3 Done Done Sent false
36391 10040 Acked Write 6 Done Done Sent false
36392 10041 Acked Read 9 Done Done Sent false
36393 10042 Acked Read 6 Done Done Sent false
36394 10043 Acked Write 3 Done Done Sent false
36395 10044 Acked Write 4 Done Done Sent false
36396 10045 Acked Write 5 Done Done Sent false
36397 10046 Acked Read 2 Done Done Sent false
36398 10047 Acked Read 5 Done Done Sent false
36399 10048 Acked Read 7 Done Done Sent false
36400 10049 Acked Write 9 Done Done Sent false
36401 10050 Acked Write 1 Done Done Sent false
36402 10051 Acked Write 4 Done Done Sent false
36403 10052 Acked Read 8 Done Done Sent false
36404 10053 Acked Write 4 Done Done Sent false
36405 10054 Acked Write 4 Done Done Sent false
36406 10055 Acked Read 8 Done Done Sent false
36407 10056 Acked Flush 0 Done Done Sent false
36408 STATES DS:0 DS:1 DS:2 TOTAL
36409 Sent 0 0 302 302
36410 Done 302 302 0 604
36411 Skipped 0 0 0 0
36412 Error 0 0 0 0
36413 Last Flush: 10056 10056 9754
36414 Downstairs last five completed: 9754 9753 9752 9751 9750
36415 Upstairs last five completed: 10056 10055 10054 10053 10052
36416 CLIENT: Up:0 ds:302 act:2
364172024-11-28T04:57:56.109ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
364182024-11-28T04:57:56.109ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:61322) = io task client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
364192024-11-28T04:57:56.110ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
364202024-11-28T04:57:56.110ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
364212024-11-28T04:57:56.111ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
364222024-11-28T04:57:56.111ZINFOcrucible: send last flush ID to this DS: 9754 = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
364232024-11-28T04:57:56.111ZINFOcrucible: Replied this last flush ID: 9754 = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
364242024-11-28T04:57:56.111ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
364252024-11-28T04:57:56.111ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
364262024-11-28T04:57:56.111ZINFOcrucible: [2] client re-new 302 jobs since flush 9754 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
364272024-11-28T04:57:56.111ZINFOcrucible: [2] Replayed 302 jobs since flush: 9754 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
36428 ----------------------------------------------------------------
36429 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
36430 Downstairs last five completed: 10056 10055 10054 10053 10052
36431 Upstairs last five completed: 10056 10055 10054 10053 10052
36432 CLIENT: Up:0 ds:0 act:3
36433 Replay: All jobs finished
36434 Main task has work to do, go find it
36435 got dsc Stop(1)
36436 stop 1
36437 Nov 28 04:57:58.335[8820] Got stop action so:false kr:false
36438 INFO request completed, latency_us: 338, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 65305ebd-6466-4647-bab0-6e4de6328c9e, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
364392024-11-28T04:57:58.336ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
364402024-11-28T04:57:58.336ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
36441 Nov 28 04:57:58.336 INFO request completed, latency_us: 115, response_code{"msg":"downstairs task for 1 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2024-11-28T04:57:58.336507025Z","hostname":"w-01JDRHKEEFJ1228TFPSBEGXWHY","pid":1202,"session_id":"4a2415b1-6594-436f-82e5-b8cf9a74eeaf"}
36442 : 200, uri: /state/cid/1, method: GET, req_id{"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-28T04:57:58.336526892Z","hostname":"w-01JDRHKEEFJ1228TFPSBEGXWHY","pid":1202,"client":"1","":"downstairs","session_id":"4a2415b1-6594-436f-82e5-b8cf9a74eeaf"}
36443 : b0677962-72bf-4ce6-937c-266f5c784160, remote_addr: 127{"msg":"[1] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-28T04:57:58.336560587Z","hostname":"w-01JDRHKEEFJ1228TFPSBEGXWHY","pid":1202,"client":"1","":"downstairs","session_id":"4a2415b1-6594-436f-82e5-b8cf9a74eeaf"}
36444 .0.0.1:60418, local_addr: 127.0.{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2024-11-28T04:57:58.336594723Z","hostname":"w-01JDRHKEEFJ1228TFPSBEGXWHY","pid":1202,"":"io task","client":"1","":"downstairs","session_id":"4a2415b1-6594-436f-82e5-b8cf9a74eeaf"}
36445 0.1:9998
36446 [8820] Exited with: ExitStatus(unix_wait_status(9))
36447 [8820][1] reports Exit
36448 Nov 28 04:58:02.338 INFO request completed, latency_us: 218, response_code: 200, uri: /state/cid/1, method: GET, req_id: ece3052f-6f6d-4991-8487-49a47c8e9c15, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
36449 001/300 Read block 2150 len 28672
36450 002/300 Write block 618 len 24576 data: 7 6 5 6 7 7
36451 003/300 Read block 1103 len 8192
36452 004/300 Read block 3789 len 4096
36453 005/300 Write block 38 len 40960 data: 12 11 10 10 9 7 6 6 6 6
36454 006/300 Read block 1456 len 28672
36455 007/300 Read block 2884 len 16384
36456 008/300 Write block 4356 len 36864 data: 6 4 6 6 7 8 8 10 12
36457 009/300 Write block 4770 len 40960 data: 7 7 8 9 9 9 8 7 7 7
36458 010/300 Write block 120 len 4096 data: 11
36459 011/300 Read block 4514 len 28672
36460 012/300 Write block 4839 len 8192 data: 4 3
36461 013/300 Read block 1207 len 8192
36462 014/300 Read block 284 len 8192
36463 015/300 Read block 4744 len 24576
36464 016/300 Write block 112 len 32768 data: 7 6 6 4 6 7 8 8
36465 017/300 Flush
36466 018/300 Read block 1067 len 40960
36467 019/300 Write block 1519 len 32768 data: 5 5 5 6 7 8 6 6
36468 020/300 Read block 960 len 16384
36469 021/300 Read block 4273 len 24576
36470 022/300 Read block 306 len 4096
36471 023/300 Write block 4672 len 40960 data: 7 9 10 9 9 8 8 8 6 5
36472 024/300 Write block 1360 len 24576 data: 6 8 7 6 7 7
36473 025/300 Read block 3851 len 36864
36474 026/300 Write block 1924 len 24576 data: 4 5 3 4 4 4
36475 027/300 Read block 3952 len 12288
36476 028/300 Flush
36477 029/300 Write block 4820 len 24576 data: 6 6 6 7 8 8
36478 030/300 Write block 138 len 28672 data: 7 8 9 8 8 8 7
36479 031/300 Read block 82 len 4096
36480 032/300 Read block 2032 len 28672
36481 033/300 Read block 2552 len 32768
36482 034/300 Write block 2391 len 40960 data: 7 7 7 8 8 7 8 7 6 6
36483 035/300 Flush
36484 036/300 Flush
36485 037/300 Write block 2427 len 28672 data: 6 6 6 4 5 5 5
36486 038/300 Write block 3380 len 40960 data: 4 4 5 4 4 4 5 7 6 5
36487 039/300 Read block 1330 len 4096
36488 040/300 Write block 273 len 20480 data: 7 7 7 7 5
36489 041/300 Write block 3399 len 24576 data: 4 4 4 4 6 9
36490 042/300 Read block 3191 len 16384
36491 043/300 Write block 473 len 8192 data: 9 9
36492 044/300 Write block 2674 len 12288 data: 6 6 6
36493 045/300 Read block 4229 len 36864
36494 046/300 Read block 4877 len 20480
36495 047/300 Flush
36496 048/300 Write block 218 len 32768 data: 6 6 7 6 6 6 4 3
36497 049/300 Write block 2887 len 36864 data: 4 4 5 6 6 6 4 3 4
36498 050/300 Read block 3833 len 12288
36499 051/300 Read block 2437 len 40960
36500 052/300 Read block 973 len 40960
36501 053/300 Read block 195 len 24576
36502 054/300 Write block 394 len 12288 data: 8 7 7
36503 055/300 Read block 2028 len 8192
36504 056/300 Read block 173 len 32768
36505 057/300 Write block 2444 len 12288 data: 3 4 3
36506 058/300 Write block 2238 len 16384 data: 4 5 4 6
36507 059/300 Write block 1781 len 36864 data: 8 7 7 8 7 8 7 6 8
36508 060/300 Write block 4410 len 36864 data: 4 4 4 4 4 3 5 5 7
36509 061/300 Read block 88 len 40960
36510 062/300 Read block 2066 len 16384
36511 063/300 Read block 2444 len 4096
36512 064/300 Write block 2062 len 40960 data: 7 6 6 8 7 6 6 6 9 9
36513 065/300 Read block 612 len 32768
36514 066/300 Write block 2216 len 12288 data: 4 4 6
36515 067/300 Read block 2289 len 36864
36516 068/300 Write block 4790 len 4096 data: 7
36517 069/300 Write block 1621 len 16384 data: 7 7 7 7
36518 070/300 Write block 3892 len 32768 data: 5 6 4 4 4 4 4 5
36519 071/300 Read block 3232 len 4096
36520 072/300 Read block 3654 len 20480
36521 073/300 Write block 4295 len 28672 data: 4 5 4 4 4 5 5
36522 074/300 Flush
36523 075/300 Write block 2099 len 28672 data: 6 7 6 7 7 7 8
36524 076/300 Write block 1958 len 32768 data: 6 6 6 7 6 5 8 8
36525 077/300 Read block 4017 len 40960
36526 078/300 Write block 727 len 28672 data: 7 6 6 7 7 7 4
36527 079/300 Write block 4935 len 24576 data: 6 6 9 8 8 6
36528 080/300 Write block 3002 len 16384 data: 6 6 7 10
36529 081/300 Read block 1633 len 16384
36530 082/300 Write block 2658 len 20480 data: 6 8 8 8 8
36531 083/300 Read block 688 len 4096
36532 084/300 Read block 534 len 28672
36533 085/300 Flush
36534 086/300 Write block 3158 len 4096 data: 7
36535 087/300 Write block 3645 len 24576 data: 9 6 7 7 9 10
36536 088/300 Read block 1293 len 40960
36537 089/300 Read block 4839 len 40960
36538 090/300 Write block 2098 len 12288 data: 8 7 8
36539 091/300 Write block 2466 len 40960 data: 6 7 6 8 7 10 9 8 8 9
36540 092/300 Read block 1923 len 16384
36541 093/300 Read block 3494 len 24576
36542 094/300 Read block 4593 len 28672
36543 095/300 Read block 4945 len 24576
36544 096/300 Read block 2076 len 12288
36545 097/300 Write block 2173 len 12288 data: 9 7 6
36546 098/300 Read block 3876 len 4096
36547 099/300 Read block 350 len 24576
36548 100/300 Read block 2087 len 24576
36549 101/300 Write block 4565 len 28672 data: 9 8 7 7 8 6 5
36550 102/300 Read block 2607 len 40960
36551 103/300 Read block 1790 len 32768
36552 104/300 Read block 2732 len 24576
36553 105/300 Write block 851 len 40960 data: 10 11 6 6 5 5 5 7 8 6
36554 106/300 Flush
36555 107/300 Read block 4377 len 16384
36556 108/300 Read block 377 len 40960
36557 109/300 Write block 3267 len 24576 data: 4 4 3 3 4 4
36558 110/300 Write block 825 len 8192 data: 5 8
36559 111/300 Flush
36560 112/300 Write block 709 len 32768 data: 3 3 4 5 5 5 5 5
36561 113/300 Read block 2998 len 32768
36562 114/300 Flush
36563 115/300 Write block 4440 len 12288 data: 6 6 6
36564 116/300 Read block 1039 len 24576
36565 117/300 Read block 373 len 8192
36566 118/300 Read block 1472 len 4096
36567 119/300 Read block 3292 len 32768
36568 120/300 Write block 277 len 16384 data: 6 8 9 8
36569 121/300 Read block 3353 len 20480
36570 122/300 Read block 3065 len 36864
36571 123/300 Write block 2492 len 20480 data: 4 4 6 5 5
36572 124/300 Write block 3393 len 40960 data: 5 5 3 4 4 4 5 5 5 5
36573 125/300 Read block 347 len 12288
36574 126/300 Write block 131 len 4096 data: 6
36575 127/300 Flush
36576 128/300 Flush
36577 129/300 Write block 3750 len 16384 data: 3 3 3 4
36578 130/300 Flush
36579 131/300 Read block 968 len 20480
36580 132/300 Read block 3475 len 16384
36581 133/300 Read block 3356 len 8192
36582 134/300 Write block 4758 len 40960 data: 4 7 6 8 8 6 6 5 5 6
36583 135/300 Read block 393 len 32768
36584 136/300 Write block 3719 len 28672 data: 5 5 6 7 7 7 6
36585 137/300 Read block 3218 len 12288
36586 138/300 Read block 3265 len 36864
36587 139/300 Write block 1678 len 40960 data: 7 7 4 3 5 5 7 6 6 6
36588 140/300 Read block 4754 len 32768
36589 141/300 Read block 2051 len 8192
36590 142/300 Read block 4914 len 24576
36591 143/300 Write block 17 len 8192 data: 11 9
36592 144/300 Read block 2918 len 24576
36593 145/300 Write block 4960 len 36864 data: 10 10 11 10 11 10 11 11 11
36594 146/300 Write block 2446 len 4096 data: 4
36595 147/300 Read block 4809 len 16384
36596 148/300 Write block 1665 len 40960 data: 4 4 5 5 5 6 6 8 7 8
36597 149/300 Flush
36598 150/300 Write block 1523 len 4096 data: 8
36599 151/300 Read block 142 len 12288
36600 152/300 Write block 4916 len 8192 data: 5 5
36601 153/300 Write block 1783 len 16384 data: 8 9 8 9
36602 154/300 Write block 4874 len 24576 data: 5 5 3 3 3 4
36603 155/300 Write block 1416 len 24576 data: 6 5 5 6 6 5
36604 156/300 Write block 2433 len 8192 data: 6 5
36605 157/300 Read block 3511 len 32768
36606 158/300 Write block 3861 len 40960 data: 10 10 7 8 7 8 7 5 5 6
36607 159/300 Read block 4977 len 12288
36608 160/300 Write block 194 len 28672 data: 7 7 6 6 5 4 4
36609 161/300 Read block 557 len 24576
36610 162/300 Write block 3438 len 20480 data: 5 6 5 5 6
36611 163/300 Write block 3477 len 12288 data: 8 7 7
36612 164/300 Read block 4680 len 4096
36613 165/300 Write block 2692 len 20480 data: 5 4 6 6 8
36614 166/300 Flush
36615 167/300 Read block 2868 len 36864
36616 168/300 Write block 2769 len 8192 data: 7 7
36617 169/300 Write block 3480 len 8192 data: 8 7
36618 170/300 Read block 4415 len 20480
36619 171/300 Write block 2002 len 32768 data: 5 6 6 5 3 3 4 4
36620 172/300 Read block 4918 len 8192
36621 173/300 Write block 3931 len 36864 data: 8 6 8 7 6 6 7 6 4
36622 174/300 Write block 1049 len 40960 data: 12 10 10 9 10 7 8 6 6 5
36623 175/300 Write block 34 len 20480 data: 10 10 11 12 13
36624 176/300 Write block 496 len 4096 data: 5
36625 177/300 Read block 1056 len 24576
36626 178/300 Read block 4100 len 8192
36627 179/300 Read block 2788 len 12288
36628 180/300 Read block 626 len 20480
36629 181/300 Write block 161 len 8192 data: 5 6
36630 182/300 Read block 4227 len 8192
36631 183/300 Flush
36632 184/300 Write block 724 len 36864 data: 6 6 6 8 7 7 8 8 8
36633 185/300 Read block 4548 len 36864
36634 186/300 Read block 2644 len 28672
36635 187/300 Read block 1835 len 16384
36636 188/300 Read block 1502 len 32768
36637 189/300 Write block 3087 len 40960 data: 6 7 9 10 10 9 7 6 6 5
36638 190/300 Write block 590 len 16384 data: 9 7 7 8
36639 191/300 Write block 2266 len 8192 data: 4 3
36640 192/300 Read block 4349 len 24576
36641 193/300 Write block 1220 len 16384 data: 8 9 9 8
36642 194/300 Read block 2515 len 36864
36643 195/300 Write block 502 len 8192 data: 6 6
36644 196/300 Read block 4937 len 28672
36645 197/300 Write block 4130 len 24576 data: 6 5 5 5 6 6
36646 198/300 Read block 2570 len 36864
36647 199/300 Write block 722 len 4096 data: 7
36648 200/300 Write block 3783 len 8192 data: 6 6
36649 201/300 Read block 4583 len 20480
36650 202/300 Flush
36651 203/300 Write block 1137 len 40960 data: 5 5 5 5 4 3 3 3 4 4
36652 204/300 Read block 718 len 4096
36653 205/300 Read block 19 len 36864
36654 206/300 Write block 2309 len 24576 data: 6 6 6 7 7 7
36655 207/300 Read block 1293 len 28672
36656 208/300 Write block 4387 len 16384 data: 8 8 8 8
36657 209/300 Write block 2377 len 36864 data: 8 10 10 11 12 9 11 11 9
36658 210/300 Read block 1619 len 8192
36659 211/300 Write block 614 len 32768 data: 7 8 9 9 8 7 6 7
36660 212/300 Read block 4797 len 12288
36661 213/300 Read block 745 len 32768
36662 214/300 Flush
36663 215/300 Read block 969 len 8192
36664 216/300 Read block 3452 len 20480
36665 217/300 Read block 1413 len 24576
36666 218/300 Write block 3568 len 12288 data: 6 6 5
36667 219/300 Read block 3025 len 16384
36668 220/300 Read block 4416 len 40960
36669 221/300 Read block 4171 len 4096
36670 222/300 Read block 196 len 32768
36671 223/300 Write block 2846 len 8192 data: 7 7
36672 224/300 Read block 2360 len 24576
36673 225/300 Read block 3598 len 20480
36674 226/300 Write block 3883 len 40960 data: 7 6 5 6 7 6 7 7 7 6
36675 227/300 Write block 4363 len 40960 data: 11 13 11 10 7 7 9 7 10 8
36676 228/300 Write block 3803 len 20480 data: 9 8 10 8 7
36677 229/300 Flush
36678 230/300 Read block 2376 len 32768
36679 231/300 Write block 4812 len 36864 data: 4 4 3 4 5 5 4 3 7
36680 232/300 Write block 870 len 12288 data: 7 7 8
36681 233/300 Read block 1422 len 16384
36682 234/300 Flush
36683 235/300 Read block 2068 len 28672
36684 236/300 Flush
36685 237/300 Write block 1468 len 24576 data: 9 7 6 5 5 3
36686 238/300 Read block 2161 len 32768
36687 239/300 Write block 1706 len 8192 data: 14 14
36688 240/300 Write block 1507 len 8192 data: 3 4
36689 241/300 Read block 3843 len 12288
36690 242/300 Write block 4089 len 20480 data: 9 11 12 13 12
36691 243/300 Read block 3774 len 28672
36692 244/300 Read block 4593 len 20480
36693 245/300 Read block 1857 len 36864
36694 246/300 Read block 4250 len 32768
36695 247/300 Read block 4698 len 28672
36696 248/300 Write block 4372 len 16384 data: 9 5 6 7
36697 249/300 Write block 744 len 4096 data: 5
36698 250/300 Write block 3937 len 12288 data: 8 7 5
36699 251/300 Write block 2088 len 36864 data: 5 5 5 4 6 7 6 6 7
36700 252/300 Read block 3031 len 36864
36701 253/300 Read block 4688 len 12288
36702 254/300 Read block 4608 len 40960
36703 255/300 Write block 3269 len 36864 data: 4 4 5 5 4 4 4 6 6
36704 256/300 Read block 400 len 28672
36705 257/300 Write block 1923 len 8192 data: 5 5
36706 258/300 Write block 3183 len 4096 data: 4
36707 259/300 Write block 3137 len 24576 data: 8 11 10 10 9 5
36708 260/300 Write block 3531 len 20480 data: 4 4 5 5 5
36709 261/300 Write block 2873 len 32768 data: 5 6 5 6 9 9 9 10
36710 262/300 Flush
36711 263/300 Write block 1735 len 16384 data: 4 4 5 4
36712 264/300 Read block 2215 len 12288
36713 265/300 Write block 2637 len 8192 data: 4 4
36714 266/300 Read block 2327 len 8192
36715 267/300 Read block 4453 len 4096
36716 268/300 Read block 4739 len 40960
36717 269/300 Read block 4275 len 24576
36718 270/300 Write block 478 len 8192 data: 5 4
36719 271/300 Read block 3490 len 8192
36720 272/300 Write block 2913 len 40960 data: 5 5 6 7 6 5 7 9 9 8
36721 273/300 Write block 4777 len 20480 data: 8 8 8 6 6
36722 274/300 Read block 4191 len 36864
36723 275/300 Read block 1839 len 28672
36724 276/300 Write block 2538 len 40960 data: 6 5 6 6 6 4 4 3 3 3
36725 277/300 Read block 60 len 32768
36726 278/300 Write block 1292 len 12288 data: 5 4 5
36727 279/300 Write block 3975 len 32768 data: 7 5 4 4 4 5 5 4
36728 280/300 Read block 760 len 8192
36729 281/300 Write block 4053 len 12288 data: 9 11 8
36730 282/300 Write block 4194 len 8192 data: 5 6
36731 283/300 Read block 539 len 40960
36732 284/300 Flush
36733 285/300 Read block 2421 len 40960
36734 286/300 Flush
36735 287/300 Write block 1694 len 20480 data: 4 3 5 4 5
36736 288/300 Read block 3523 len 12288
36737 289/300 Write block 114 len 40960 data: 7 5 7 8 9 9 12 10 10 8
36738 290/300 Read block 231 len 24576
36739 291/300 Write block 1931 len 16384 data: 3 4 4 4
36740 292/300 Read block 2026 len 40960
36741 293/300 Read block 596 len 12288
36742 294/300 Flush
36743 295/300 Write block 1240 len 36864 data: 5 5 6 7 6 6 5 4 4
36744 296/300 Flush
36745 297/300 Read block 3292 len 4096
36746 298/300 Flush
36747 299/300 Write block 4391 len 32768 data: 6 6 5 3 3 3 3 3
36748 300/300 Write block 4704 len 20480 data: 8 7 7 7 6
36749 301/300 Read block 4697 len 16384
36750 Main task has work to do, go find it
36751 got dsc Start(1)
36752 start 1
36753 Nov 28 04:58:02.573 INFO request completed, latency_us: 231, response_code: 204, uri: /start/cid/1, method: POST, req_id: ffeb9597-9bd1-4493-91b9-64e7b37720bd, remote_addr: 127.0.0.1:60418, local_addr: 127.0.0.1:9998
36754 Replay: started 1, returned:Ok(())
36755 ----------------------------------------------------------------
36756 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
36757 States: Active Offline Active
36758 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
36759 10057 Acked Read 7 Done Sent Done false
36760 10058 Acked Write 6 Done Sent Done false
36761 10059 Acked Read 2 Done Sent Done false
36762 10060 Acked Read 1 Done Sent Done false
36763 10061 Acked Write 10 Done Sent Done false
36764 10062 Acked Read 7 Done Sent Done false
36765 10063 Acked Read 4 Done Sent Done false
36766 10064 Acked Write 9 Done Sent Done false
36767 10065 Acked Write 10 Done Sent Done false
36768 10066 Acked Write 1 Done Sent Done false
36769 10067 Acked Read 7 Done Sent Done false
36770 10068 Acked Write 2 Done Sent Done false
36771 10069 Acked Read 2 Done Sent Done false
36772 10070 Acked Read 2 Done Sent Done false
36773 10071 Acked Read 6 Done Sent Done false
36774 10072 Acked Write 8 Done Sent Done false
36775 10073 Acked Flush 0 Done Sent Done false
36776 10074 Acked Read 10 Done Sent Done false
36777 10075 Acked Write 8 Done Sent Done false
36778 10076 Acked Read 4 Done Sent Done false
36779 10077 Acked Read 6 Done Sent Done false
36780 10078 Acked Read 1 Done Sent Done false
36781 10079 Acked Write 10 Done Sent Done false
36782 10080 Acked Write 6 Done Sent Done false
36783 10081 Acked Read 9 Done Sent Done false
36784 10082 Acked Write 6 Done Sent Done false
36785 10083 Acked Read 3 Done Sent Done false
36786 10084 Acked Flush 0 Done Sent Done false
36787 10085 Acked Write 6 Done Sent Done false
36788 10086 Acked Write 7 Done Sent Done false
36789 10087 Acked Read 1 Done Sent Done false
36790 10088 Acked Read 7 Done Sent Done false
36791 10089 Acked Read 8 Done Sent Done false
36792 10090 Acked Write 10 Done Sent Done false
36793 10091 Acked Flush 0 Done Sent Done false
36794 10092 Acked Flush 0 Done Sent Done false
36795 10093 Acked Write 7 Done Sent Done false
36796 10094 Acked Write 10 Done Sent Done false
36797 10095 Acked Read 1 Done Sent Done false
36798 10096 Acked Write 5 Done Sent Done false
36799 10097 Acked Write 6 Done Sent Done false
36800 10098 Acked Read 4 Done Sent Done false
36801 10099 Acked Write 2 Done Sent Done false
36802 10100 Acked Write 3 Done Sent Done false
36803 10101 Acked Read 9 Done Sent Done false
36804 10102 Acked Read 5 Done Sent Done false
36805 10103 Acked Flush 0 Done Sent Done false
36806 10104 Acked Write 8 Done Sent Done false
36807 10105 Acked Write 9 Done Sent Done false
36808 10106 Acked Read 3 Done Sent Done false
36809 10107 Acked Read 10 Done Sent Done false
36810 10108 Acked Read 10 Done Sent Done false
36811 10109 Acked Read 6 Done Sent Done false
36812 10110 Acked Write 3 Done Sent Done false
36813 10111 Acked Read 2 Done Sent Done false
36814 10112 Acked Read 8 Done Sent Done false
36815 10113 Acked Write 3 Done Sent Done false
36816 10114 Acked Write 4 Done Sent Done false
36817 10115 Acked Write 9 Done Sent Done false
36818 10116 Acked Write 9 Done Sent Done false
36819 10117 Acked Read 10 Done Sent Done false
36820 10118 Acked Read 4 Done Sent Done false
36821 10119 Acked Read 1 Done Sent Done false
36822 10120 Acked Write 10 Done Sent Done false
36823 10121 Acked Read 8 Done Sent Done false
36824 10122 Acked Write 3 Done Sent Done false
36825 10123 Acked Read 9 Done Sent Done false
36826 10124 Acked Write 1 Done Sent Done false
36827 10125 Acked Write 4 Done Sent Done false
36828 10126 Acked Write 8 Done Sent Done false
36829 10127 Acked Read 1 Done Sent Done false
36830 10128 Acked Read 5 Done Sent Done false
36831 10129 Acked Write 7 Done Sent Done false
36832 10130 Acked Flush 0 Done Sent Done false
36833 10131 Acked Write 7 Done Sent Done false
36834 10132 Acked Write 8 Done Sent Done false
36835 10133 Acked Read 10 Done Sent Done false
36836 10134 Acked Write 7 Done Sent Done false
36837 10135 Acked Write 6 Done Sent Done false
36838 10136 Acked Write 4 Done Sent Done false
36839 10137 Acked Read 4 Done Sent Done false
36840 10138 Acked Write 5 Done Sent Done false
36841 10139 Acked Read 1 Done Sent Done false
36842 10140 Acked Read 7 Done Sent Done false
36843 10141 Acked Flush 0 Done Sent Done false
36844 10142 Acked Write 1 Done Sent Done false
36845 10143 Acked Write 6 Done Sent Done false
36846 10144 Acked Read 10 Done Sent Done false
36847 10145 Acked Read 10 Done Sent Done false
36848 10146 Acked Write 3 Done Sent Done false
36849 10147 Acked Write 10 Done Sent Done false
36850 10148 Acked Read 4 Done Sent Done false
36851 10149 Acked Read 6 Done Sent Done false
36852 10150 Acked Read 7 Done Sent Done false
36853 10151 Acked Read 6 Done Sent Done false
36854 10152 Acked Read 3 Done Sent Done false
36855 10153 Acked Write 3 Done Sent Done false
36856 10154 Acked Read 1 Done Sent Done false
36857 10155 Acked Read 6 Done Sent Done false
36858 10156 Acked Read 6 Done Sent Done false
36859 10157 Acked Write 7 Done Sent Done false
36860 10158 Acked Read 10 Done Sent Done false
36861 10159 Acked Read 8 Done Sent Done false
36862 10160 Acked Read 6 Done Sent Done false
36863 10161 Acked Write 10 Done Sent Done false
36864 10162 Acked Flush 0 Done Sent Done false
36865 10163 Acked Read 4 Done Sent Done false
36866 10164 Acked Read 10 Done Sent Done false
36867 10165 Acked Write 6 Done Sent Done false
36868 10166 Acked Write 2 Done Sent Done false
36869 10167 Acked Flush 0 Done Sent Done false
36870 10168 Acked Write 8 Done Sent Done false
36871 10169 Acked Read 8 Done Sent Done false
36872 10170 Acked Flush 0 Done Sent Done false
36873 10171 Acked Write 3 Done Sent Done false
36874 10172 Acked Read 6 Done Sent Done false
36875 10173 Acked Read 2 Done Sent Done false
36876 10174 Acked Read 1 Done Sent Done false
36877 10175 Acked Read 8 Done Sent Done false
36878 10176 Acked Write 4 Done Sent Done false
36879 10177 Acked Read 5 Done Sent Done false
36880 10178 Acked Read 9 Done Sent Done false
36881 10179 Acked Write 5 Done Sent Done false
36882 10180 Acked Write 10 Done Sent Done false
36883 10181 Acked Read 3 Done Sent Done false
36884 10182 Acked Write 1 Done Sent Done false
36885 10183 Acked Flush 0 Done Sent Done false
36886 10184 Acked Flush 0 Done Sent Done false
36887 10185 Acked Write 4 Done Sent Done false
36888 10186 Acked Flush 0 Done Sent Done false
36889 10187 Acked Read 5 Done Sent Done false
36890 10188 Acked Read 4 Done Sent Done false
36891 10189 Acked Read 2 Done Sent Done false
36892 10190 Acked Write 10 Done Sent Done false
36893 10191 Acked Read 8 Done Sent Done false
36894 10192 Acked Write 7 Done Sent Done false
36895 10193 Acked Read 3 Done Sent Done false
36896 10194 Acked Read 9 Done Sent Done false
36897 10195 Acked Write 10 Done Sent Done false
36898 10196 Acked Read 8 Done Sent Done false
36899 10197 Acked Read 2 Done Sent Done false
36900 10198 Acked Read 6 Done Sent Done false
36901 10199 Acked Write 2 Done Sent Done false
36902 10200 Acked Read 6 Done Sent Done false
36903 10201 Acked Write 9 Done Sent Done false
36904 10202 Acked Write 1 Done Sent Done false
36905 10203 Acked Read 4 Done Sent Done false
36906 10204 Acked Write 10 Done Sent Done false
36907 10205 Acked Flush 0 Done Sent Done false
36908 10206 Acked Write 1 Done Sent Done false
36909 10207 Acked Read 3 Done Sent Done false
36910 10208 Acked Write 2 Done Sent Done false
36911 10209 Acked Write 4 Done Sent Done false
36912 10210 Acked Write 6 Done Sent Done false
36913 10211 Acked Write 6 Done Sent Done false
36914 10212 Acked Write 2 Done Sent Done false
36915 10213 Acked Read 8 Done Sent Done false
36916 10214 Acked Write 10 Done Sent Done false
36917 10215 Acked Read 3 Done Sent Done false
36918 10216 Acked Write 7 Done Sent Done false
36919 10217 Acked Read 6 Done Sent Done false
36920 10218 Acked Write 5 Done Sent Done false
36921 10219 Acked Write 3 Done Sent Done false
36922 10220 Acked Read 1 Done Sent Done false
36923 10221 Acked Write 5 Done Sent Done false
36924 10222 Acked Flush 0 Done Sent Done false
36925 10223 Acked Read 9 Done Sent Done false
36926 10224 Acked Write 2 Done Sent Done false
36927 10225 Acked Write 2 Done Sent Done false
36928 10226 Acked Read 5 Done Sent Done false
36929 10227 Acked Write 8 Done Sent Done false
36930 10228 Acked Read 2 Done Sent Done false
36931 10229 Acked Write 9 Done Sent Done false
36932 10230 Acked Write 10 Done Sent Done false
36933 10231 Acked Write 5 Done Sent Done false
36934 10232 Acked Write 1 Done Sent Done false
36935 10233 Acked Read 6 Done Sent Done false
36936 10234 Acked Read 2 Done Sent Done false
36937 10235 Acked Read 3 Done Sent Done false
36938 10236 Acked Read 5 Done Sent Done false
36939 10237 Acked Write 2 Done Sent Done false
36940 10238 Acked Read 2 Done Sent Done false
36941 10239 Acked Flush 0 Done Sent Done false
36942 10240 Acked Write 9 Done Sent Done false
36943 10241 Acked Read 9 Done Sent Done false
36944 10242 Acked Read 7 Done Sent Done false
36945 10243 Acked Read 4 Done Sent Done false
36946 10244 Acked Read 8 Done Sent Done false
36947 10245 Acked Write 10 Done Sent Done false
36948 10246 Acked Write 4 Done Sent Done false
36949 10247 Acked Write 2 Done Sent Done false
36950 10248 Acked Read 6 Done Sent Done false
36951 10249 Acked Write 4 Done Sent Done false
36952 10250 Acked Read 9 Done Sent Done false
36953 10251 Acked Write 2 Done Sent Done false
36954 10252 Acked Read 7 Done Sent Done false
36955 10253 Acked Write 6 Done Sent Done false
36956 10254 Acked Read 9 Done Sent Done false
36957 10255 Acked Write 1 Done Sent Done false
36958 10256 Acked Write 2 Done Sent Done false
36959 10257 Acked Read 5 Done Sent Done false
36960 10258 Acked Flush 0 Done Sent Done false
36961 10259 Acked Write 10 Done Sent Done false
36962 10260 Acked Read 1 Done Sent Done false
36963 10261 Acked Read 9 Done Sent Done false
36964 10262 Acked Write 6 Done Sent Done false
36965 10263 Acked Read 7 Done Sent Done false
36966 10264 Acked Write 4 Done Sent Done false
36967 10265 Acked Write 9 Done Sent Done false
36968 10266 Acked Read 2 Done Sent Done false
36969 10267 Acked Write 8 Done Sent Done false
36970 10268 Acked Read 3 Done Sent Done false
36971 10269 Acked Read 8 Done Sent Done false
36972 10270 Acked Flush 0 Done Sent Done false
36973 10271 Acked Read 2 Done Sent Done false
36974 10272 Acked Read 5 Done Sent Done false
36975 10273 Acked Read 6 Done Sent Done false
36976 10274 Acked Write 3 Done Sent Done false
36977 10275 Acked Read 4 Done Sent Done false
36978 10276 Acked Read 10 Done Sent Done false
36979 10277 Acked Read 1 Done Sent Done false
36980 10278 Acked Read 8 Done Sent Done false
36981 10279 Acked Write 2 Done Sent Done false
36982 10280 Acked Read 6 Done Sent Done false
36983 10281 Acked Read 5 Done Sent Done false
36984 10282 Acked Write 10 Done Sent Done false
36985 10283 Acked Write 10 Done Sent Done false
36986 10284 Acked Write 5 Done Sent Done false
36987 10285 Acked Flush 0 Done Sent Done false
36988 10286 Acked Read 8 Done Sent Done false
36989 10287 Acked Write 9 Done Sent Done false
36990 10288 Acked Write 3 Done Sent Done false
36991 10289 Acked Read 4 Done Sent Done false
36992 10290 Acked Flush 0 Done Sent Done false
36993 10291 Acked Read 7 Done Sent Done false
36994 10292 Acked Flush 0 Done Sent Done false
36995 10293 Acked Write 6 Done Sent Done false
36996 10294 Acked Read 8 Done Sent Done false
36997 10295 Acked Write 2 Done Sent Done false
36998 10296 Acked Write 2 Done Sent Done false
36999 10297 Acked Read 3 Done Sent Done false
37000 10298 Acked Write 5 Done Sent Done false
37001 10299 Acked Read 7 Done Sent Done false
37002 10300 Acked Read 5 Done Sent Done false
37003 10301 Acked Read 9 Done Sent Done false
37004 10302 Acked Read 8 Done Sent Done false
37005 10303 Acked Read 7 Done Sent Done false
37006 10304 Acked Write 4 Done Sent Done false
37007 10305 Acked Write 1 Done Sent Done false
37008 10306 Acked Write 3 Done Sent Done false
37009 10307 Acked Write 9 Done Sent Done false
37010 10308 Acked Read 9 Done Sent Done false
37011 10309 Acked Read 3 Done Sent Done false
37012 10310 Acked Read 10 Done Sent Done false
37013 10311 Acked Write 9 Done Sent Done false
37014 10312 Acked Read 7 Done Sent Done false
37015 10313 Acked Write 2 Done Sent Done false
37016 10314 Acked Write 1 Done Sent Done false
37017 10315 Acked Write 6 Done Sent Done false
37018 10316 Acked Write 5 Done Sent Done false
37019 10317 Acked Write 8 Done Sent Done false
37020 10318 Acked Flush 0 Done Sent Done false
37021 10319 Acked Write 4 Done Sent Done false
37022 10320 Acked Read 3 Done Sent Done false
37023 10321 Acked Write 2 Done Sent Done false
37024 10322 Acked Read 2 Done Sent Done false
37025 10323 Acked Read 1 Done Sent Done false
37026 10324 Acked Read 10 Done Sent Done false
37027 10325 Acked Read 6 Done Sent Done false
37028 10326 Acked Write 2 Done Sent Done false
37029 10327 Acked Read 2 Done Sent Done false
37030 10328 Acked Write 10 Done Sent Done false
37031 10329 Acked Write 5 Done Sent Done false
37032 10330 Acked Read 9 Done Sent Done false
37033 10331 Acked Read 7 Done Sent Done false
37034 10332 Acked Write 10 Done Sent Done false
37035 10333 Acked Read 8 Done Sent Done false
37036 10334 Acked Write 3 Done Sent Done false
37037 10335 Acked Write 8 Done Sent Done false
37038 10336 Acked Read 2 Done Sent Done false
37039 10337 Acked Write 3 Done Sent Done false
37040 10338 Acked Write 2 Done Sent Done false
37041 10339 Acked Read 10 Done Sent Done false
37042 10340 Acked Flush 0 Done Sent Done false
37043 10341 Acked Read 10 Done Sent Done false
37044 10342 Acked Flush 0 Done Sent Done false
37045 10343 Acked Write 5 Done Sent Done false
37046 10344 Acked Read 3 Done Sent Done false
37047 10345 Acked Write 10 Done Sent Done false
37048 10346 Acked Read 6 Done Sent Done false
37049 10347 Acked Write 4 Done Sent Done false
37050 10348 Acked Read 10 Done Sent Done false
37051 10349 Acked Read 3 Done Sent Done false
37052 10350 Acked Flush 0 Done Sent Done false
37053 10351 Acked Write 9 Done Sent Done false
37054 10352 Acked Flush 0 Done Sent Done false
37055 10353 Acked Read 1 Done Sent Done false
37056 10354 Acked Flush 0 Done Sent Done false
37057 10355 Acked Write 8 Done Sent Done false
37058 10356 Acked Write 5 Done Sent Done false
37059 10357 Acked Read 4 Done Sent Done false
37060 STATES DS:0 DS:1 DS:2 TOTAL
37061 Sent 0 301 0 301
37062 Done 301 0 301 602
37063 Skipped 0 0 0 0
37064 Error 0 0 0 0
37065 Last Flush: 10354 10056 10354
37066 Downstairs last five completed: 10056 10055 10054 10053 10052
37067 Upstairs last five completed: 10357 10356 10355 10354 10353
37068 CLIENT: Up:0 ds:301 act:2
37069 ----------------------------------------------------------------
37070 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
37071 States: Active Offline Active
37072 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
37073 10057 Acked Read 7 Done Sent Done false
37074 10058 Acked Write 6 Done Sent Done false
37075 10059 Acked Read 2 Done Sent Done false
37076 10060 Acked Read 1 Done Sent Done false
37077 10061 Acked Write 10 Done Sent Done false
37078 10062 Acked Read 7 Done Sent Done false
37079 10063 Acked Read 4 Done Sent Done false
37080 10064 Acked Write 9 Done Sent Done false
37081 10065 Acked Write 10 Done Sent Done false
37082 10066 Acked Write 1 Done Sent Done false
37083 10067 Acked Read 7 Done Sent Done false
37084 10068 Acked Write 2 Done Sent Done false
37085 10069 Acked Read 2 Done Sent Done false
37086 10070 Acked Read 2 Done Sent Done false
37087 10071 Acked Read 6 Done Sent Done false
37088 10072 Acked Write 8 Done Sent Done false
37089 10073 Acked Flush 0 Done Sent Done false
37090 10074 Acked Read 10 Done Sent Done false
37091 10075 Acked Write 8 Done Sent Done false
37092 10076 Acked Read 4 Done Sent Done false
37093 10077 Acked Read 6 Done Sent Done false
37094 10078 Acked Read 1 Done Sent Done false
37095 10079 Acked Write 10 Done Sent Done false
37096 10080 Acked Write 6 Done Sent Done false
37097 10081 Acked Read 9 Done Sent Done false
37098 10082 Acked Write 6 Done Sent Done false
37099 10083 Acked Read 3 Done Sent Done false
37100 10084 Acked Flush 0 Done Sent Done false
37101 10085 Acked Write 6 Done Sent Done false
37102 10086 Acked Write 7 Done Sent Done false
37103 10087 Acked Read 1 Done Sent Done false
37104 10088 Acked Read 7 Done Sent Done false
37105 10089 Acked Read 8 Done Sent Done false
37106 10090 Acked Write 10 Done Sent Done false
37107 10091 Acked Flush 0 Done Sent Done false
37108 10092 Acked Flush 0 Done Sent Done false
37109 10093 Acked Write 7 Done Sent Done false
37110 10094 Acked Write 10 Done Sent Done false
37111 10095 Acked Read 1 Done Sent Done false
37112 10096 Acked Write 5 Done Sent Done false
37113 10097 Acked Write 6 Done Sent Done false
37114 10098 Acked Read 4 Done Sent Done false
37115 10099 Acked Write 2 Done Sent Done false
37116 10100 Acked Write 3 Done Sent Done false
37117 10101 Acked Read 9 Done Sent Done false
37118 10102 Acked Read 5 Done Sent Done false
37119 10103 Acked Flush 0 Done Sent Done false
37120 10104 Acked Write 8 Done Sent Done false
37121 10105 Acked Write 9 Done Sent Done false
37122 10106 Acked Read 3 Done Sent Done false
37123 10107 Acked Read 10 Done Sent Done false
37124 10108 Acked Read 10 Done Sent Done false
37125 10109 Acked Read 6 Done Sent Done false
37126 10110 Acked Write 3 Done Sent Done false
37127 10111 Acked Read 2 Done Sent Done false
37128 10112 Acked Read 8 Done Sent Done false
37129 10113 Acked Write 3 Done Sent Done false
37130 10114 Acked Write 4 Done Sent Done false
37131 10115 Acked Write 9 Done Sent Done false
37132 10116 Acked Write 9 Done Sent Done false
37133 10117 Acked Read 10 Done Sent Done false
37134 10118 Acked Read 4 Done Sent Done false
37135 10119 Acked Read 1 Done Sent Done false
37136 10120 Acked Write 10 Done Sent Done false
37137 10121 Acked Read 8 Done Sent Done false
37138 10122 Acked Write 3 Done Sent Done false
37139 10123 Acked Read 9 Done Sent Done false
37140 10124 Acked Write 1 Done Sent Done false
37141 10125 Acked Write 4 Done Sent Done false
37142 10126 Acked Write 8 Done Sent Done false
37143 10127 Acked Read 1 Done Sent Done false
37144 10128 Acked Read 5 Done Sent Done false
37145 10129 Acked Write 7 Done Sent Done false
37146 10130 Acked Flush 0 Done Sent Done false
37147 10131 Acked Write 7 Done Sent Done false
37148 10132 Acked Write 8 Done Sent Done false
37149 10133 Acked Read 10 Done Sent Done false
37150 10134 Acked Write 7 Done Sent Done false
37151 10135 Acked Write 6 Done Sent Done false
37152 10136 Acked Write 4 Done Sent Done false
37153 10137 Acked Read 4 Done Sent Done false
37154 10138 Acked Write 5 Done Sent Done false
37155 10139 Acked Read 1 Done Sent Done false
37156 10140 Acked Read 7 Done Sent Done false
37157 10141 Acked Flush 0 Done Sent Done false
37158 10142 Acked Write 1 Done Sent Done false
37159 10143 Acked Write 6 Done Sent Done false
37160 10144 Acked Read 10 Done Sent Done false
37161 10145 Acked Read 10 Done Sent Done false
37162 10146 Acked Write 3 Done Sent Done false
37163 10147 Acked Write 10 Done Sent Done false
37164 10148 Acked Read 4 Done Sent Done false
37165 10149 Acked Read 6 Done Sent Done false
37166 10150 Acked Read 7 Done Sent Done false
37167 10151 Acked Read 6 Done Sent Done false
37168 10152 Acked Read 3 Done Sent Done false
37169 10153 Acked Write 3 Done Sent Done false
37170 10154 Acked Read 1 Done Sent Done false
37171 10155 Acked Read 6 Done Sent Done false
37172 10156 Acked Read 6 Done Sent Done false
37173 10157 Acked Write 7 Done Sent Done false
37174 10158 Acked Read 10 Done Sent Done false
37175 10159 Acked Read 8 Done Sent Done false
37176 10160 Acked Read 6 Done Sent Done false
37177 10161 Acked Write 10 Done Sent Done false
37178 10162 Acked Flush 0 Done Sent Done false
37179 10163 Acked Read 4 Done Sent Done false
37180 10164 Acked Read 10 Done Sent Done false
37181 10165 Acked Write 6 Done Sent Done false
37182 10166 Acked Write 2 Done Sent Done false
37183 10167 Acked Flush 0 Done Sent Done false
37184 10168 Acked Write 8 Done Sent Done false
37185 10169 Acked Read 8 Done Sent Done false
37186 10170 Acked Flush 0 Done Sent Done false
37187 10171 Acked Write 3 Done Sent Done false
37188 10172 Acked Read 6 Done Sent Done false
37189 10173 Acked Read 2 Done Sent Done false
37190 10174 Acked Read 1 Done Sent Done false
37191 10175 Acked Read 8 Done Sent Done false
37192 10176 Acked Write 4 Done Sent Done false
37193 10177 Acked Read 5 Done Sent Done false
37194 10178 Acked Read 9 Done Sent Done false
37195 10179 Acked Write 5 Done Sent Done false
37196 10180 Acked Write 10 Done Sent Done false
37197 10181 Acked Read 3 Done Sent Done false
37198 10182 Acked Write 1 Done Sent Done false
37199 10183 Acked Flush 0 Done Sent Done false
37200 10184 Acked Flush 0 Done Sent Done false
37201 10185 Acked Write 4 Done Sent Done false
37202 10186 Acked Flush 0 Done Sent Done false
37203 10187 Acked Read 5 Done Sent Done false
37204 10188 Acked Read 4 Done Sent Done false
37205 10189 Acked Read 2 Done Sent Done false
37206 10190 Acked Write 10 Done Sent Done false
37207 10191 Acked Read 8 Done Sent Done false
37208 10192 Acked Write 7 Done Sent Done false
37209 10193 Acked Read 3 Done Sent Done false
37210 10194 Acked Read 9 Done Sent Done false
37211 10195 Acked Write 10 Done Sent Done false
37212 10196 Acked Read 8 Done Sent Done false
37213 10197 Acked Read 2 Done Sent Done false
37214 10198 Acked Read 6 Done Sent Done false
37215 10199 Acked Write 2 Done Sent Done false
37216 10200 Acked Read 6 Done Sent Done false
37217 10201 Acked Write 9 Done Sent Done false
37218 10202 Acked Write 1 Done Sent Done false
37219 10203 Acked Read 4 Done Sent Done false
37220 10204 Acked Write 10 Done Sent Done false
37221 10205 Acked Flush 0 Done Sent Done false
37222 10206 Acked Write 1 Done Sent Done false
37223 10207 Acked Read 3 Done Sent Done false
37224 10208 Acked Write 2 Done Sent Done false
37225 10209 Acked Write 4 Done Sent Done false
37226 10210 Acked Write 6 Done Sent Done false
37227 10211 Acked Write 6 Done Sent Done false
37228 10212 Acked Write 2 Done Sent Done false
37229 10213 Acked Read 8 Done Sent Done false
37230 10214 Acked Write 10 Done Sent Done false
37231 10215 Acked Read 3 Done Sent Done false
37232 10216 Acked Write 7 Done Sent Done false
37233 10217 Acked Read 6 Done Sent Done false
37234 10218 Acked Write 5 Done Sent Done false
37235 10219 Acked Write 3 Done Sent Done false
37236 10220 Acked Read 1 Done Sent Done false
37237 10221 Acked Write 5 Done Sent Done false
37238 10222 Acked Flush 0 Done Sent Done false
37239 10223 Acked Read 9 Done Sent Done false
37240 10224 Acked Write 2 Done Sent Done false
37241 10225 Acked Write 2 Done Sent Done false
37242 10226 Acked Read 5 Done Sent Done false
37243 10227 Acked Write 8 Done Sent Done false
37244 10228 Acked Read 2 Done Sent Done false
37245 10229 Acked Write 9 Done Sent Done false
37246 10230 Acked Write 10 Done Sent Done false
37247 10231 Acked Write 5 Done Sent Done false
37248 10232 Acked Write 1 Done Sent Done false
37249 10233 Acked Read 6 Done Sent Done false
37250 10234 Acked Read 2 Done Sent Done false
37251 10235 Acked Read 3 Done Sent Done false
37252 10236 Acked Read 5 Done Sent Done false
37253 10237 Acked Write 2 Done Sent Done false
37254 10238 Acked Read 2 Done Sent Done false
37255 10239 Acked Flush 0 Done Sent Done false
37256 10240 Acked Write 9 Done Sent Done false
37257 10241 Acked Read 9 Done Sent Done false
37258 10242 Acked Read 7 Done Sent Done false
37259 10243 Acked Read 4 Done Sent Done false
37260 10244 Acked Read 8 Done Sent Done false
37261 10245 Acked Write 10 Done Sent Done false
37262 10246 Acked Write 4 Done Sent Done false
37263 10247 Acked Write 2 Done Sent Done false
37264 10248 Acked Read 6 Done Sent Done false
37265 10249 Acked Write 4 Done Sent Done false
37266 10250 Acked Read 9 Done Sent Done false
37267 10251 Acked Write 2 Done Sent Done false
37268 10252 Acked Read 7 Done Sent Done false
37269 10253 Acked Write 6 Done Sent Done false
37270 10254 Acked Read 9 Done Sent Done false
37271 10255 Acked Write 1 Done Sent Done false
37272 10256 Acked Write 2 Done Sent Done false
37273 10257 Acked Read 5 Done Sent Done false
37274 10258 Acked Flush 0 Done Sent Done false
37275 10259 Acked Write 10 Done Sent Done false
37276 10260 Acked Read 1 Done Sent Done false
37277 10261 Acked Read 9 Done Sent Done false
37278 10262 Acked Write 6 Done Sent Done false
37279 10263 Acked Read 7 Done Sent Done false
37280 10264 Acked Write 4 Done Sent Done false
37281 10265 Acked Write 9 Done Sent Done false
37282 10266 Acked Read 2 Done Sent Done false
37283 10267 Acked Write 8 Done Sent Done false
37284 10268 Acked Read 3 Done Sent Done false
37285 10269 Acked Read 8 Done Sent Done false
37286 10270 Acked Flush 0 Done Sent Done false
37287 10271 Acked Read 2 Done Sent Done false
37288 10272 Acked Read 5 Done Sent Done false
37289 10273 Acked Read 6 Done Sent Done false
37290 10274 Acked Write 3 Done Sent Done false
37291 10275 Acked Read 4 Done Sent Done false
37292 10276 Acked Read 10 Done Sent Done false
37293 10277 Acked Read 1 Done Sent Done false
37294 10278 Acked Read 8 Done Sent Done false
37295 10279 Acked Write 2 Done Sent Done false
37296 10280 Acked Read 6 Done Sent Done false
37297 10281 Acked Read 5 Done Sent Done false
37298 10282 Acked Write 10 Done Sent Done false
37299 10283 Acked Write 10 Done Sent Done false
37300 10284 Acked Write 5 Done Sent Done false
37301 10285 Acked Flush 0 Done Sent Done false
37302 10286 Acked Read 8 Done Sent Done false
37303 10287 Acked Write 9 Done Sent Done false
37304 10288 Acked Write 3 Done Sent Done false
37305 10289 Acked Read 4 Done Sent Done false
37306 10290 Acked Flush 0 Done Sent Done false
37307 10291 Acked Read 7 Done Sent Done false
37308 10292 Acked Flush 0 Done Sent Done false
37309 10293 Acked Write 6 Done Sent Done false
37310 10294 Acked Read 8 Done Sent Done false
37311 10295 Acked Write 2 Done Sent Done false
37312 10296 Acked Write 2 Done Sent Done false
37313 10297 Acked Read 3 Done Sent Done false
37314 10298 Acked Write 5 Done Sent Done false
37315 10299 Acked Read 7 Done Sent Done false
37316 10300 Acked Read 5 Done Sent Done false
37317 10301 Acked Read 9 Done Sent Done false
37318 10302 Acked Read 8 Done Sent Done false
37319 10303 Acked Read 7 Done Sent Done false
37320 10304 Acked Write 4 Done Sent Done false
37321 10305 Acked Write 1 Done Sent Done false
37322 10306 Acked Write 3 Done Sent Done false
37323 10307 Acked Write 9 Done Sent Done false
37324 10308 Acked Read 9 Done Sent Done false
37325 10309 Acked Read 3 Done Sent Done false
37326 10310 Acked Read 10 Done Sent Done false
37327 10311 Acked Write 9 Done Sent Done false
37328 10312 Acked Read 7 Done Sent Done false
37329 10313 Acked Write 2 Done Sent Done false
37330 10314 Acked Write 1 Done Sent Done false
37331 10315 Acked Write 6 Done Sent Done false
37332 10316 Acked Write 5 Done Sent Done false
37333 10317 Acked Write 8 Done Sent Done false
37334 10318 Acked Flush 0 Done Sent Done false
37335 10319 Acked Write 4 Done Sent Done false
37336 10320 Acked Read 3 Done Sent Done false
37337 10321 Acked Write 2 Done Sent Done false
37338 10322 Acked Read 2 Done Sent Done false
37339 10323 Acked Read 1 Done Sent Done false
37340 10324 Acked Read 10 Done Sent Done false
37341 10325 Acked Read 6 Done Sent Done false
37342 10326 Acked Write 2 Done Sent Done false
37343 10327 Acked Read 2 Done Sent Done false
37344 10328 Acked Write 10 Done Sent Done false
37345 10329 Acked Write 5 Done Sent Done false
37346 10330 Acked Read 9 Done Sent Done false
37347 10331 Acked Read 7 Done Sent Done false
37348 10332 Acked Write 10 Done Sent Done false
37349 10333 Acked Read 8 Done Sent Done false
37350 10334 Acked Write 3 Done Sent Done false
37351 10335 Acked Write 8 Done Sent Done false
37352 10336 Acked Read 2 Done Sent Done false
37353 10337 Acked Write 3 Done Sent Done false
37354 10338 Acked Write 2 Done Sent Done false
37355 10339 Acked Read 10 Done Sent Done false
37356 10340 Acked Flush 0 Done Sent Done false
37357 10341 Acked Read 10 Done Sent Done false
37358 10342 Acked Flush 0 Done Sent Done false
37359 10343 Acked Write 5 Done Sent Done false
37360 10344 Acked Read 3 Done Sent Done false
37361 10345 Acked Write 10 Done Sent Done false
37362 10346 Acked Read 6 Done Sent Done false
37363 10347 Acked Write 4 Done Sent Done false
37364 10348 Acked Read 10 Done Sent Done false
37365 10349 Acked Read 3 Done Sent Done false
37366 10350 Acked Flush 0 Done Sent Done false
37367 10351 Acked Write 9 Done Sent Done false
37368 10352 Acked Flush 0 Done Sent Done false
37369 10353 Acked Read 1 Done Sent Done false
37370 10354 Acked Flush 0 Done Sent Done false
37371 10355 Acked Write 8 Done Sent Done false
37372 10356 Acked Write 5 Done Sent Done false
37373 10357 Acked Read 4 Done Sent Done false
37374 10358 Acked Flush 0 Done Sent Done false
37375 STATES DS:0 DS:1 DS:2 TOTAL
37376 Sent 0 302 0 302
37377 Done 302 0 302 604
37378 Skipped 0 0 0 0
37379 Error 0 0 0 0
37380 Last Flush: 10358 10056 10358
37381 Downstairs last five completed: 10056 10055 10054 10053 10052
37382 Upstairs last five completed: 10358 10357 10356 10355 10354
37383 CLIENT: Up:0 ds:302 act:2
373842024-11-28T04:58:08.338ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
373852024-11-28T04:58:08.338ZWARNcrucible: ds_connection connect to 127.0.0.1:8820 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
373862024-11-28T04:58:08.338ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
373872024-11-28T04:58:08.338ZWARNcrucible: downstairs task for 1 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
373882024-11-28T04:58:08.338ZINFOcrucible: client task is exiting = io task client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
37389 [8820] got start action
37390 [8820] I am going to restart
37391 Starting downstairs at port 8820
37392 Make output file at "/tmp/dsc/downstairs-8820.txt"
37393 [8820][1] reports Starting
37394 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1233)
37395 [8820][1] reports Running
37396 ----------------------------------------------------------------
37397 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
37398 States: Active Offline Active
37399 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
37400 10057 Acked Read 7 Done Sent Done false
37401 10058 Acked Write 6 Done Sent Done false
37402 10059 Acked Read 2 Done Sent Done false
37403 10060 Acked Read 1 Done Sent Done false
37404 10061 Acked Write 10 Done Sent Done false
37405 10062 Acked Read 7 Done Sent Done false
37406 10063 Acked Read 4 Done Sent Done false
37407 10064 Acked Write 9 Done Sent Done false
37408 10065 Acked Write 10 Done Sent Done false
37409 10066 Acked Write 1 Done Sent Done false
37410 10067 Acked Read 7 Done Sent Done false
37411 10068 Acked Write 2 Done Sent Done false
37412 10069 Acked Read 2 Done Sent Done false
37413 10070 Acked Read 2 Done Sent Done false
37414 10071 Acked Read 6 Done Sent Done false
37415 10072 Acked Write 8 Done Sent Done false
37416 10073 Acked Flush 0 Done Sent Done false
37417 10074 Acked Read 10 Done Sent Done false
37418 10075 Acked Write 8 Done Sent Done false
37419 10076 Acked Read 4 Done Sent Done false
37420 10077 Acked Read 6 Done Sent Done false
37421 10078 Acked Read 1 Done Sent Done false
37422 10079 Acked Write 10 Done Sent Done false
37423 10080 Acked Write 6 Done Sent Done false
37424 10081 Acked Read 9 Done Sent Done false
37425 10082 Acked Write 6 Done Sent Done false
37426 10083 Acked Read 3 Done Sent Done false
37427 10084 Acked Flush 0 Done Sent Done false
37428 10085 Acked Write 6 Done Sent Done false
37429 10086 Acked Write 7 Done Sent Done false
37430 10087 Acked Read 1 Done Sent Done false
37431 10088 Acked Read 7 Done Sent Done false
37432 10089 Acked Read 8 Done Sent Done false
37433 10090 Acked Write 10 Done Sent Done false
37434 10091 Acked Flush 0 Done Sent Done false
37435 10092 Acked Flush 0 Done Sent Done false
37436 10093 Acked Write 7 Done Sent Done false
37437 10094 Acked Write 10 Done Sent Done false
37438 10095 Acked Read 1 Done Sent Done false
37439 10096 Acked Write 5 Done Sent Done false
37440 10097 Acked Write 6 Done Sent Done false
37441 10098 Acked Read 4 Done Sent Done false
37442 10099 Acked Write 2 Done Sent Done false
37443 10100 Acked Write 3 Done Sent Done false
37444 10101 Acked Read 9 Done Sent Done false
37445 10102 Acked Read 5 Done Sent Done false
37446 10103 Acked Flush 0 Done Sent Done false
37447 10104 Acked Write 8 Done Sent Done false
37448 10105 Acked Write 9 Done Sent Done false
37449 10106 Acked Read 3 Done Sent Done false
37450 10107 Acked Read 10 Done Sent Done false
37451 10108 Acked Read 10 Done Sent Done false
37452 10109 Acked Read 6 Done Sent Done false
37453 10110 Acked Write 3 Done Sent Done false
37454 10111 Acked Read 2 Done Sent Done false
37455 10112 Acked Read 8 Done Sent Done false
37456 10113 Acked Write 3 Done Sent Done false
37457 10114 Acked Write 4 Done Sent Done false
37458 10115 Acked Write 9 Done Sent Done false
37459 10116 Acked Write 9 Done Sent Done false
37460 10117 Acked Read 10 Done Sent Done false
37461 10118 Acked Read 4 Done Sent Done false
37462 10119 Acked Read 1 Done Sent Done false
37463 10120 Acked Write 10 Done Sent Done false
37464 10121 Acked Read 8 Done Sent Done false
37465 10122 Acked Write 3 Done Sent Done false
37466 10123 Acked Read 9 Done Sent Done false
37467 10124 Acked Write 1 Done Sent Done false
37468 10125 Acked Write 4 Done Sent Done false
37469 10126 Acked Write 8 Done Sent Done false
37470 10127 Acked Read 1 Done Sent Done false
37471 10128 Acked Read 5 Done Sent Done false
37472 10129 Acked Write 7 Done Sent Done false
37473 10130 Acked Flush 0 Done Sent Done false
37474 10131 Acked Write 7 Done Sent Done false
37475 10132 Acked Write 8 Done Sent Done false
37476 10133 Acked Read 10 Done Sent Done false
37477 10134 Acked Write 7 Done Sent Done false
37478 10135 Acked Write 6 Done Sent Done false
37479 10136 Acked Write 4 Done Sent Done false
37480 10137 Acked Read 4 Done Sent Done false
37481 10138 Acked Write 5 Done Sent Done false
37482 10139 Acked Read 1 Done Sent Done false
37483 10140 Acked Read 7 Done Sent Done false
37484 10141 Acked Flush 0 Done Sent Done false
37485 10142 Acked Write 1 Done Sent Done false
37486 10143 Acked Write 6 Done Sent Done false
37487 10144 Acked Read 10 Done Sent Done false
37488 10145 Acked Read 10 Done Sent Done false
37489 10146 Acked Write 3 Done Sent Done false
37490 10147 Acked Write 10 Done Sent Done false
37491 10148 Acked Read 4 Done Sent Done false
37492 10149 Acked Read 6 Done Sent Done false
37493 10150 Acked Read 7 Done Sent Done false
37494 10151 Acked Read 6 Done Sent Done false
37495 10152 Acked Read 3 Done Sent Done false
37496 10153 Acked Write 3 Done Sent Done false
37497 10154 Acked Read 1 Done Sent Done false
37498 10155 Acked Read 6 Done Sent Done false
37499 10156 Acked Read 6 Done Sent Done false
37500 10157 Acked Write 7 Done Sent Done false
37501 10158 Acked Read 10 Done Sent Done false
37502 10159 Acked Read 8 Done Sent Done false
37503 10160 Acked Read 6 Done Sent Done false
37504 10161 Acked Write 10 Done Sent Done false
37505 10162 Acked Flush 0 Done Sent Done false
37506 10163 Acked Read 4 Done Sent Done false
37507 10164 Acked Read 10 Done Sent Done false
37508 10165 Acked Write 6 Done Sent Done false
37509 10166 Acked Write 2 Done Sent Done false
37510 10167 Acked Flush 0 Done Sent Done false
37511 10168 Acked Write 8 Done Sent Done false
37512 10169 Acked Read 8 Done Sent Done false
37513 10170 Acked Flush 0 Done Sent Done false
37514 10171 Acked Write 3 Done Sent Done false
37515 10172 Acked Read 6 Done Sent Done false
37516 10173 Acked Read 2 Done Sent Done false
37517 10174 Acked Read 1 Done Sent Done false
37518 10175 Acked Read 8 Done Sent Done false
37519 10176 Acked Write 4 Done Sent Done false
37520 10177 Acked Read 5 Done Sent Done false
37521 10178 Acked Read 9 Done Sent Done false
37522 10179 Acked Write 5 Done Sent Done false
37523 10180 Acked Write 10 Done Sent Done false
37524 10181 Acked Read 3 Done Sent Done false
37525 10182 Acked Write 1 Done Sent Done false
37526 10183 Acked Flush 0 Done Sent Done false
37527 10184 Acked Flush 0 Done Sent Done false
37528 10185 Acked Write 4 Done Sent Done false
37529 10186 Acked Flush 0 Done Sent Done false
37530 10187 Acked Read 5 Done Sent Done false
37531 10188 Acked Read 4 Done Sent Done false
37532 10189 Acked Read 2 Done Sent Done false
37533 10190 Acked Write 10 Done Sent Done false
37534 10191 Acked Read 8 Done Sent Done false
37535 10192 Acked Write 7 Done Sent Done false
37536 10193 Acked Read 3 Done Sent Done false
37537 10194 Acked Read 9 Done Sent Done false
37538 10195 Acked Write 10 Done Sent Done false
37539 10196 Acked Read 8 Done Sent Done false
37540 10197 Acked Read 2 Done Sent Done false
37541 10198 Acked Read 6 Done Sent Done false
37542 10199 Acked Write 2 Done Sent Done false
37543 10200 Acked Read 6 Done Sent Done false
37544 10201 Acked Write 9 Done Sent Done false
37545 10202 Acked Write 1 Done Sent Done false
37546 10203 Acked Read 4 Done Sent Done false
37547 10204 Acked Write 10 Done Sent Done false
37548 10205 Acked Flush 0 Done Sent Done false
37549 10206 Acked Write 1 Done Sent Done false
37550 10207 Acked Read 3 Done Sent Done false
37551 10208 Acked Write 2 Done Sent Done false
37552 10209 Acked Write 4 Done Sent Done false
37553 10210 Acked Write 6 Done Sent Done false
37554 10211 Acked Write 6 Done Sent Done false
37555 10212 Acked Write 2 Done Sent Done false
37556 10213 Acked Read 8 Done Sent Done false
37557 10214 Acked Write 10 Done Sent Done false
37558 10215 Acked Read 3 Done Sent Done false
37559 10216 Acked Write 7 Done Sent Done false
37560 10217 Acked Read 6 Done Sent Done false
37561 10218 Acked Write 5 Done Sent Done false
37562 10219 Acked Write 3 Done Sent Done false
37563 10220 Acked Read 1 Done Sent Done false
37564 10221 Acked Write 5 Done Sent Done false
37565 10222 Acked Flush 0 Done Sent Done false
37566 10223 Acked Read 9 Done Sent Done false
37567 10224 Acked Write 2 Done Sent Done false
37568 10225 Acked Write 2 Done Sent Done false
37569 10226 Acked Read 5 Done Sent Done false
37570 10227 Acked Write 8 Done Sent Done false
37571 10228 Acked Read 2 Done Sent Done false
37572 10229 Acked Write 9 Done Sent Done false
37573 10230 Acked Write 10 Done Sent Done false
37574 10231 Acked Write 5 Done Sent Done false
37575 10232 Acked Write 1 Done Sent Done false
37576 10233 Acked Read 6 Done Sent Done false
37577 10234 Acked Read 2 Done Sent Done false
37578 10235 Acked Read 3 Done Sent Done false
37579 10236 Acked Read 5 Done Sent Done false
37580 10237 Acked Write 2 Done Sent Done false
37581 10238 Acked Read 2 Done Sent Done false
37582 10239 Acked Flush 0 Done Sent Done false
37583 10240 Acked Write 9 Done Sent Done false
37584 10241 Acked Read 9 Done Sent Done false
37585 10242 Acked Read 7 Done Sent Done false
37586 10243 Acked Read 4 Done Sent Done false
37587 10244 Acked Read 8 Done Sent Done false
37588 10245 Acked Write 10 Done Sent Done false
37589 10246 Acked Write 4 Done Sent Done false
37590 10247 Acked Write 2 Done Sent Done false
37591 10248 Acked Read 6 Done Sent Done false
37592 10249 Acked Write 4 Done Sent Done false
37593 10250 Acked Read 9 Done Sent Done false
37594 10251 Acked Write 2 Done Sent Done false
37595 10252 Acked Read 7 Done Sent Done false
37596 10253 Acked Write 6 Done Sent Done false
37597 10254 Acked Read 9 Done Sent Done false
37598 10255 Acked Write 1 Done Sent Done false
37599 10256 Acked Write 2 Done Sent Done false
37600 10257 Acked Read 5 Done Sent Done false
37601 10258 Acked Flush 0 Done Sent Done false
37602 10259 Acked Write 10 Done Sent Done false
37603 10260 Acked Read 1 Done Sent Done false
37604 10261 Acked Read 9 Done Sent Done false
37605 10262 Acked Write 6 Done Sent Done false
37606 10263 Acked Read 7 Done Sent Done false
37607 10264 Acked Write 4 Done Sent Done false
37608 10265 Acked Write 9 Done Sent Done false
37609 10266 Acked Read 2 Done Sent Done false
37610 10267 Acked Write 8 Done Sent Done false
37611 10268 Acked Read 3 Done Sent Done false
37612 10269 Acked Read 8 Done Sent Done false
37613 10270 Acked Flush 0 Done Sent Done false
37614 10271 Acked Read 2 Done Sent Done false
37615 10272 Acked Read 5 Done Sent Done false
37616 10273 Acked Read 6 Done Sent Done false
37617 10274 Acked Write 3 Done Sent Done false
37618 10275 Acked Read 4 Done Sent Done false
37619 10276 Acked Read 10 Done Sent Done false
37620 10277 Acked Read 1 Done Sent Done false
37621 10278 Acked Read 8 Done Sent Done false
37622 10279 Acked Write 2 Done Sent Done false
37623 10280 Acked Read 6 Done Sent Done false
37624 10281 Acked Read 5 Done Sent Done false
37625 10282 Acked Write 10 Done Sent Done false
37626 10283 Acked Write 10 Done Sent Done false
37627 10284 Acked Write 5 Done Sent Done false
37628 10285 Acked Flush 0 Done Sent Done false
37629 10286 Acked Read 8 Done Sent Done false
37630 10287 Acked Write 9 Done Sent Done false
37631 10288 Acked Write 3 Done Sent Done false
37632 10289 Acked Read 4 Done Sent Done false
37633 10290 Acked Flush 0 Done Sent Done false
37634 10291 Acked Read 7 Done Sent Done false
37635 10292 Acked Flush 0 Done Sent Done false
37636 10293 Acked Write 6 Done Sent Done false
37637 10294 Acked Read 8 Done Sent Done false
37638 10295 Acked Write 2 Done Sent Done false
37639 10296 Acked Write 2 Done Sent Done false
37640 10297 Acked Read 3 Done Sent Done false
37641 10298 Acked Write 5 Done Sent Done false
37642 10299 Acked Read 7 Done Sent Done false
37643 10300 Acked Read 5 Done Sent Done false
37644 10301 Acked Read 9 Done Sent Done false
37645 10302 Acked Read 8 Done Sent Done false
37646 10303 Acked Read 7 Done Sent Done false
37647 10304 Acked Write 4 Done Sent Done false
37648 10305 Acked Write 1 Done Sent Done false
37649 10306 Acked Write 3 Done Sent Done false
37650 10307 Acked Write 9 Done Sent Done false
37651 10308 Acked Read 9 Done Sent Done false
37652 10309 Acked Read 3 Done Sent Done false
37653 10310 Acked Read 10 Done Sent Done false
37654 10311 Acked Write 9 Done Sent Done false
37655 10312 Acked Read 7 Done Sent Done false
37656 10313 Acked Write 2 Done Sent Done false
37657 10314 Acked Write 1 Done Sent Done false
37658 10315 Acked Write 6 Done Sent Done false
37659 10316 Acked Write 5 Done Sent Done false
37660 10317 Acked Write 8 Done Sent Done false
37661 10318 Acked Flush 0 Done Sent Done false
37662 10319 Acked Write 4 Done Sent Done false
37663 10320 Acked Read 3 Done Sent Done false
37664 10321 Acked Write 2 Done Sent Done false
37665 10322 Acked Read 2 Done Sent Done false
37666 10323 Acked Read 1 Done Sent Done false
37667 10324 Acked Read 10 Done Sent Done false
37668 10325 Acked Read 6 Done Sent Done false
37669 10326 Acked Write 2 Done Sent Done false
37670 10327 Acked Read 2 Done Sent Done false
37671 10328 Acked Write 10 Done Sent Done false
37672 10329 Acked Write 5 Done Sent Done false
37673 10330 Acked Read 9 Done Sent Done false
37674 10331 Acked Read 7 Done Sent Done false
37675 10332 Acked Write 10 Done Sent Done false
37676 10333 Acked Read 8 Done Sent Done false
37677 10334 Acked Write 3 Done Sent Done false
37678 10335 Acked Write 8 Done Sent Done false
37679 10336 Acked Read 2 Done Sent Done false
37680 10337 Acked Write 3 Done Sent Done false
37681 10338 Acked Write 2 Done Sent Done false
37682 10339 Acked Read 10 Done Sent Done false
37683 10340 Acked Flush 0 Done Sent Done false
37684 10341 Acked Read 10 Done Sent Done false
37685 10342 Acked Flush 0 Done Sent Done false
37686 10343 Acked Write 5 Done Sent Done false
37687 10344 Acked Read 3 Done Sent Done false
37688 10345 Acked Write 10 Done Sent Done false
37689 10346 Acked Read 6 Done Sent Done false
37690 10347 Acked Write 4 Done Sent Done false
37691 10348 Acked Read 10 Done Sent Done false
37692 10349 Acked Read 3 Done Sent Done false
37693 10350 Acked Flush 0 Done Sent Done false
37694 10351 Acked Write 9 Done Sent Done false
37695 10352 Acked Flush 0 Done Sent Done false
37696 10353 Acked Read 1 Done Sent Done false
37697 10354 Acked Flush 0 Done Sent Done false
37698 10355 Acked Write 8 Done Sent Done false
37699 10356 Acked Write 5 Done Sent Done false
37700 10357 Acked Read 4 Done Sent Done false
37701 10358 Acked Flush 0 Done Sent Done false
37702 STATES DS:0 DS:1 DS:2 TOTAL
37703 Sent 0 302 0 302
37704 Done 302 0 302 604
37705 Skipped 0 0 0 0
37706 Error 0 0 0 0
37707 Last Flush: 10358 10056 10358
37708 Downstairs last five completed: 10056 10055 10054 10053 10052
37709 Upstairs last five completed: 10358 10357 10356 10355 10354
37710 CLIENT: Up:0 ds:302 act:2
37711 ----------------------------------------------------------------
37712 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
37713 States: Active Offline Active
37714 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
37715 10057 Acked Read 7 Done Sent Done false
37716 10058 Acked Write 6 Done Sent Done false
37717 10059 Acked Read 2 Done Sent Done false
37718 10060 Acked Read 1 Done Sent Done false
37719 10061 Acked Write 10 Done Sent Done false
37720 10062 Acked Read 7 Done Sent Done false
37721 10063 Acked Read 4 Done Sent Done false
37722 10064 Acked Write 9 Done Sent Done false
37723 10065 Acked Write 10 Done Sent Done false
37724 10066 Acked Write 1 Done Sent Done false
37725 10067 Acked Read 7 Done Sent Done false
37726 10068 Acked Write 2 Done Sent Done false
37727 10069 Acked Read 2 Done Sent Done false
37728 10070 Acked Read 2 Done Sent Done false
37729 10071 Acked Read 6 Done Sent Done false
37730 10072 Acked Write 8 Done Sent Done false
37731 10073 Acked Flush 0 Done Sent Done false
37732 10074 Acked Read 10 Done Sent Done false
37733 10075 Acked Write 8 Done Sent Done false
37734 10076 Acked Read 4 Done Sent Done false
37735 10077 Acked Read 6 Done Sent Done false
37736 10078 Acked Read 1 Done Sent Done false
37737 10079 Acked Write 10 Done Sent Done false
37738 10080 Acked Write 6 Done Sent Done false
37739 10081 Acked Read 9 Done Sent Done false
37740 10082 Acked Write 6 Done Sent Done false
37741 10083 Acked Read 3 Done Sent Done false
37742 10084 Acked Flush 0 Done Sent Done false
37743 10085 Acked Write 6 Done Sent Done false
37744 10086 Acked Write 7 Done Sent Done false
37745 10087 Acked Read 1 Done Sent Done false
37746 10088 Acked Read 7 Done Sent Done false
37747 10089 Acked Read 8 Done Sent Done false
37748 10090 Acked Write 10 Done Sent Done false
37749 10091 Acked Flush 0 Done Sent Done false
37750 10092 Acked Flush 0 Done Sent Done false
37751 10093 Acked Write 7 Done Sent Done false
37752 10094 Acked Write 10 Done Sent Done false
37753 10095 Acked Read 1 Done Sent Done false
37754 10096 Acked Write 5 Done Sent Done false
37755 10097 Acked Write 6 Done Sent Done false
37756 10098 Acked Read 4 Done Sent Done false
37757 10099 Acked Write 2 Done Sent Done false
37758 10100 Acked Write 3 Done Sent Done false
37759 10101 Acked Read 9 Done Sent Done false
37760 10102 Acked Read 5 Done Sent Done false
37761 10103 Acked Flush 0 Done Sent Done false
37762 10104 Acked Write 8 Done Sent Done false
37763 10105 Acked Write 9 Done Sent Done false
37764 10106 Acked Read 3 Done Sent Done false
37765 10107 Acked Read 10 Done Sent Done false
37766 10108 Acked Read 10 Done Sent Done false
37767 10109 Acked Read 6 Done Sent Done false
37768 10110 Acked Write 3 Done Sent Done false
37769 10111 Acked Read 2 Done Sent Done false
37770 10112 Acked Read 8 Done Sent Done false
37771 10113 Acked Write 3 Done Sent Done false
37772 10114 Acked Write 4 Done Sent Done false
37773 10115 Acked Write 9 Done Sent Done false
37774 10116 Acked Write 9 Done Sent Done false
37775 10117 Acked Read 10 Done Sent Done false
37776 10118 Acked Read 4 Done Sent Done false
37777 10119 Acked Read 1 Done Sent Done false
37778 10120 Acked Write 10 Done Sent Done false
37779 10121 Acked Read 8 Done Sent Done false
37780 10122 Acked Write 3 Done Sent Done false
37781 10123 Acked Read 9 Done Sent Done false
37782 10124 Acked Write 1 Done Sent Done false
37783 10125 Acked Write 4 Done Sent Done false
37784 10126 Acked Write 8 Done Sent Done false
37785 10127 Acked Read 1 Done Sent Done false
37786 10128 Acked Read 5 Done Sent Done false
37787 10129 Acked Write 7 Done Sent Done false
37788 10130 Acked Flush 0 Done Sent Done false
37789 10131 Acked Write 7 Done Sent Done false
37790 10132 Acked Write 8 Done Sent Done false
37791 10133 Acked Read 10 Done Sent Done false
37792 10134 Acked Write 7 Done Sent Done false
37793 10135 Acked Write 6 Done Sent Done false
37794 10136 Acked Write 4 Done Sent Done false
37795 10137 Acked Read 4 Done Sent Done false
37796 10138 Acked Write 5 Done Sent Done false
37797 10139 Acked Read 1 Done Sent Done false
37798 10140 Acked Read 7 Done Sent Done false
37799 10141 Acked Flush 0 Done Sent Done false
37800 10142 Acked Write 1 Done Sent Done false
37801 10143 Acked Write 6 Done Sent Done false
37802 10144 Acked Read 10 Done Sent Done false
37803 10145 Acked Read 10 Done Sent Done false
37804 10146 Acked Write 3 Done Sent Done false
37805 10147 Acked Write 10 Done Sent Done false
37806 10148 Acked Read 4 Done Sent Done false
37807 10149 Acked Read 6 Done Sent Done false
37808 10150 Acked Read 7 Done Sent Done false
37809 10151 Acked Read 6 Done Sent Done false
37810 10152 Acked Read 3 Done Sent Done false
37811 10153 Acked Write 3 Done Sent Done false
37812 10154 Acked Read 1 Done Sent Done false
37813 10155 Acked Read 6 Done Sent Done false
37814 10156 Acked Read 6 Done Sent Done false
37815 10157 Acked Write 7 Done Sent Done false
37816 10158 Acked Read 10 Done Sent Done false
37817 10159 Acked Read 8 Done Sent Done false
37818 10160 Acked Read 6 Done Sent Done false
37819 10161 Acked Write 10 Done Sent Done false
37820 10162 Acked Flush 0 Done Sent Done false
37821 10163 Acked Read 4 Done Sent Done false
37822 10164 Acked Read 10 Done Sent Done false
37823 10165 Acked Write 6 Done Sent Done false
37824 10166 Acked Write 2 Done Sent Done false
37825 10167 Acked Flush 0 Done Sent Done false
37826 10168 Acked Write 8 Done Sent Done false
37827 10169 Acked Read 8 Done Sent Done false
37828 10170 Acked Flush 0 Done Sent Done false
37829 10171 Acked Write 3 Done Sent Done false
37830 10172 Acked Read 6 Done Sent Done false
37831 10173 Acked Read 2 Done Sent Done false
37832 10174 Acked Read 1 Done Sent Done false
37833 10175 Acked Read 8 Done Sent Done false
37834 10176 Acked Write 4 Done Sent Done false
37835 10177 Acked Read 5 Done Sent Done false
37836 10178 Acked Read 9 Done Sent Done false
37837 10179 Acked Write 5 Done Sent Done false
37838 10180 Acked Write 10 Done Sent Done false
37839 10181 Acked Read 3 Done Sent Done false
37840 10182 Acked Write 1 Done Sent Done false
37841 10183 Acked Flush 0 Done Sent Done false
37842 10184 Acked Flush 0 Done Sent Done false
37843 10185 Acked Write 4 Done Sent Done false
37844 10186 Acked Flush 0 Done Sent Done false
37845 10187 Acked Read 5 Done Sent Done false
37846 10188 Acked Read 4 Done Sent Done false
37847 10189 Acked Read 2 Done Sent Done false
37848 10190 Acked Write 10 Done Sent Done false
37849 10191 Acked Read 8 Done Sent Done false
37850 10192 Acked Write 7 Done Sent Done false
37851 10193 Acked Read 3 Done Sent Done false
37852 10194 Acked Read 9 Done Sent Done false
37853 10195 Acked Write 10 Done Sent Done false
37854 10196 Acked Read 8 Done Sent Done false
37855 10197 Acked Read 2 Done Sent Done false
37856 10198 Acked Read 6 Done Sent Done false
37857 10199 Acked Write 2 Done Sent Done false
37858 10200 Acked Read 6 Done Sent Done false
37859 10201 Acked Write 9 Done Sent Done false
37860 10202 Acked Write 1 Done Sent Done false
37861 10203 Acked Read 4 Done Sent Done false
37862 10204 Acked Write 10 Done Sent Done false
37863 10205 Acked Flush 0 Done Sent Done false
37864 10206 Acked Write 1 Done Sent Done false
37865 10207 Acked Read 3 Done Sent Done false
37866 10208 Acked Write 2 Done Sent Done false
37867 10209 Acked Write 4 Done Sent Done false
37868 10210 Acked Write 6 Done Sent Done false
37869 10211 Acked Write 6 Done Sent Done false
37870 10212 Acked Write 2 Done Sent Done false
37871 10213 Acked Read 8 Done Sent Done false
37872 10214 Acked Write 10 Done Sent Done false
37873 10215 Acked Read 3 Done Sent Done false
37874 10216 Acked Write 7 Done Sent Done false
37875 10217 Acked Read 6 Done Sent Done false
37876 10218 Acked Write 5 Done Sent Done false
37877 10219 Acked Write 3 Done Sent Done false
37878 10220 Acked Read 1 Done Sent Done false
37879 10221 Acked Write 5 Done Sent Done false
37880 10222 Acked Flush 0 Done Sent Done false
37881 10223 Acked Read 9 Done Sent Done false
37882 10224 Acked Write 2 Done Sent Done false
37883 10225 Acked Write 2 Done Sent Done false
37884 10226 Acked Read 5 Done Sent Done false
37885 10227 Acked Write 8 Done Sent Done false
37886 10228 Acked Read 2 Done Sent Done false
37887 10229 Acked Write 9 Done Sent Done false
37888 10230 Acked Write 10 Done Sent Done false
37889 10231 Acked Write 5 Done Sent Done false
37890 10232 Acked Write 1 Done Sent Done false
37891 10233 Acked Read 6 Done Sent Done false
37892 10234 Acked Read 2 Done Sent Done false
37893 10235 Acked Read 3 Done Sent Done false
37894 10236 Acked Read 5 Done Sent Done false
37895 10237 Acked Write 2 Done Sent Done false
37896 10238 Acked Read 2 Done Sent Done false
37897 10239 Acked Flush 0 Done Sent Done false
37898 10240 Acked Write 9 Done Sent Done false
37899 10241 Acked Read 9 Done Sent Done false
37900 10242 Acked Read 7 Done Sent Done false
37901 10243 Acked Read 4 Done Sent Done false
37902 10244 Acked Read 8 Done Sent Done false
37903 10245 Acked Write 10 Done Sent Done false
37904 10246 Acked Write 4 Done Sent Done false
37905 10247 Acked Write 2 Done Sent Done false
37906 10248 Acked Read 6 Done Sent Done false
37907 10249 Acked Write 4 Done Sent Done false
37908 10250 Acked Read 9 Done Sent Done false
37909 10251 Acked Write 2 Done Sent Done false
37910 10252 Acked Read 7 Done Sent Done false
37911 10253 Acked Write 6 Done Sent Done false
37912 10254 Acked Read 9 Done Sent Done false
37913 10255 Acked Write 1 Done Sent Done false
37914 10256 Acked Write 2 Done Sent Done false
37915 10257 Acked Read 5 Done Sent Done false
37916 10258 Acked Flush 0 Done Sent Done false
37917 10259 Acked Write 10 Done Sent Done false
37918 10260 Acked Read 1 Done Sent Done false
37919 10261 Acked Read 9 Done Sent Done false
37920 10262 Acked Write 6 Done Sent Done false
37921 10263 Acked Read 7 Done Sent Done false
37922 10264 Acked Write 4 Done Sent Done false
37923 10265 Acked Write 9 Done Sent Done false
37924 10266 Acked Read 2 Done Sent Done false
37925 10267 Acked Write 8 Done Sent Done false
37926 10268 Acked Read 3 Done Sent Done false
37927 10269 Acked Read 8 Done Sent Done false
37928 10270 Acked Flush 0 Done Sent Done false
37929 10271 Acked Read 2 Done Sent Done false
37930 10272 Acked Read 5 Done Sent Done false
37931 10273 Acked Read 6 Done Sent Done false
37932 10274 Acked Write 3 Done Sent Done false
37933 10275 Acked Read 4 Done Sent Done false
37934 10276 Acked Read 10 Done Sent Done false
37935 10277 Acked Read 1 Done Sent Done false
37936 10278 Acked Read 8 Done Sent Done false
37937 10279 Acked Write 2 Done Sent Done false
37938 10280 Acked Read 6 Done Sent Done false
37939 10281 Acked Read 5 Done Sent Done false
37940 10282 Acked Write 10 Done Sent Done false
37941 10283 Acked Write 10 Done Sent Done false
37942 10284 Acked Write 5 Done Sent Done false
37943 10285 Acked Flush 0 Done Sent Done false
37944 10286 Acked Read 8 Done Sent Done false
37945 10287 Acked Write 9 Done Sent Done false
37946 10288 Acked Write 3 Done Sent Done false
37947 10289 Acked Read 4 Done Sent Done false
37948 10290 Acked Flush 0 Done Sent Done false
37949 10291 Acked Read 7 Done Sent Done false
37950 10292 Acked Flush 0 Done Sent Done false
37951 10293 Acked Write 6 Done Sent Done false
37952 10294 Acked Read 8 Done Sent Done false
37953 10295 Acked Write 2 Done Sent Done false
37954 10296 Acked Write 2 Done Sent Done false
37955 10297 Acked Read 3 Done Sent Done false
37956 10298 Acked Write 5 Done Sent Done false
37957 10299 Acked Read 7 Done Sent Done false
37958 10300 Acked Read 5 Done Sent Done false
37959 10301 Acked Read 9 Done Sent Done false
37960 10302 Acked Read 8 Done Sent Done false
37961 10303 Acked Read 7 Done Sent Done false
37962 10304 Acked Write 4 Done Sent Done false
37963 10305 Acked Write 1 Done Sent Done false
37964 10306 Acked Write 3 Done Sent Done false
37965 10307 Acked Write 9 Done Sent Done false
37966 10308 Acked Read 9 Done Sent Done false
37967 10309 Acked Read 3 Done Sent Done false
37968 10310 Acked Read 10 Done Sent Done false
37969 10311 Acked Write 9 Done Sent Done false
37970 10312 Acked Read 7 Done Sent Done false
37971 10313 Acked Write 2 Done Sent Done false
37972 10314 Acked Write 1 Done Sent Done false
37973 10315 Acked Write 6 Done Sent Done false
37974 10316 Acked Write 5 Done Sent Done false
37975 10317 Acked Write 8 Done Sent Done false
37976 10318 Acked Flush 0 Done Sent Done false
37977 10319 Acked Write 4 Done Sent Done false
37978 10320 Acked Read 3 Done Sent Done false
37979 10321 Acked Write 2 Done Sent Done false
37980 10322 Acked Read 2 Done Sent Done false
37981 10323 Acked Read 1 Done Sent Done false
37982 10324 Acked Read 10 Done Sent Done false
37983 10325 Acked Read 6 Done Sent Done false
37984 10326 Acked Write 2 Done Sent Done false
37985 10327 Acked Read 2 Done Sent Done false
37986 10328 Acked Write 10 Done Sent Done false
37987 10329 Acked Write 5 Done Sent Done false
37988 10330 Acked Read 9 Done Sent Done false
37989 10331 Acked Read 7 Done Sent Done false
37990 10332 Acked Write 10 Done Sent Done false
37991 10333 Acked Read 8 Done Sent Done false
37992 10334 Acked Write 3 Done Sent Done false
37993 10335 Acked Write 8 Done Sent Done false
37994 10336 Acked Read 2 Done Sent Done false
37995 10337 Acked Write 3 Done Sent Done false
37996 10338 Acked Write 2 Done Sent Done false
37997 10339 Acked Read 10 Done Sent Done false
37998 10340 Acked Flush 0 Done Sent Done false
37999 10341 Acked Read 10 Done Sent Done false
38000 10342 Acked Flush 0 Done Sent Done false
38001 10343 Acked Write 5 Done Sent Done false
38002 10344 Acked Read 3 Done Sent Done false
38003 10345 Acked Write 10 Done Sent Done false
38004 10346 Acked Read 6 Done Sent Done false
38005 10347 Acked Write 4 Done Sent Done false
38006 10348 Acked Read 10 Done Sent Done false
38007 10349 Acked Read 3 Done Sent Done false
38008 10350 Acked Flush 0 Done Sent Done false
38009 10351 Acked Write 9 Done Sent Done false
38010 10352 Acked Flush 0 Done Sent Done false
38011 10353 Acked Read 1 Done Sent Done false
38012 10354 Acked Flush 0 Done Sent Done false
38013 10355 Acked Write 8 Done Sent Done false
38014 10356 Acked Write 5 Done Sent Done false
38015 10357 Acked Read 4 Done Sent Done false
38016 10358 Acked Flush 0 Done Sent Done false
38017 STATES DS:0 DS:1 DS:2 TOTAL
38018 Sent 0 302 0 302
38019 Done 302 0 302 604
38020 Skipped 0 0 0 0
38021 Error 0 0 0 0
38022 Last Flush: 10358 10056 10358
38023 Downstairs last five completed: 10056 10055 10054 10053 10052
38024 Upstairs last five completed: 10358 10357 10356 10355 10354
38025 CLIENT: Up:0 ds:302 act:2
380262024-11-28T04:58:18.340ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
380272024-11-28T04:58:18.340ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:59595) = io task client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
380282024-11-28T04:58:18.341ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
380292024-11-28T04:58:18.341ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
380302024-11-28T04:58:18.341ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
380312024-11-28T04:58:18.341ZINFOcrucible: send last flush ID to this DS: 10056 = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
380322024-11-28T04:58:18.341ZINFOcrucible: Replied this last flush ID: 10056 = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
380332024-11-28T04:58:18.341ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
380342024-11-28T04:58:18.341ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
380352024-11-28T04:58:18.342ZINFOcrucible: [1] client re-new 302 jobs since flush 10056 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
380362024-11-28T04:58:18.342ZINFOcrucible: [1] Replayed 302 jobs since flush: 10056 = downstairs session_id = 4a2415b1-6594-436f-82e5-b8cf9a74eeaf
38037 ----------------------------------------------------------------
38038 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
38039 Downstairs last five completed: 10358 10357 10356 10355 10354
38040 Upstairs last five completed: 10358 10357 10356 10355 10354
38041 CLIENT: Up:0 ds:0 act:3
38042 Replay: All jobs finished
38043 Test replay has completed
38044 Wrote out file "/tmp/test_replay_verify.log"
38045 CLIENT: Tests done. All submitted work has been ACK'd
38046 ----------------------------------------------------------------
38047 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
38048 Downstairs last five completed: 10358 10357 10356 10355 10354
38049 Upstairs last five completed: 10358 10357 10356 10355 10354
38050 CLIENT: Up:0 ds:0 act:3
38051 CLIENT: All jobs finished, all DS active.
380522024-11-28T04:58:18.584ZWARNcrucible: Guest handle has been dropped
38053 Replays:30 time: 7:56
38054 Do final verify
38055 Nov 28 04:58:18.657 INFO accepted connection, remote_addr: 127.0.0.1:60535, local_addr: 127.0.0.1:9998
38056 Nov 28 04:58:18.658 INFO request completed, latency_us: 195, response_code: 200, uri: /regioninfo, method: GET, req_id: 55c09d85-86b2-40c2-954c-b6e9d8a8088e, remote_addr: 127.0.0.1:60535, local_addr: 127.0.0.1:9998
380572024-11-28T04:58:18.658ZINFOcrucible: Use this region info from dsc: RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 } task = crutest
38058 Nov 28 04:58:18.659 INFO request completed, latency_us: 114, response_code: 200, uri: /regioncount, method: GET, req_id: 894493fe-7a98-4710-a57c-5f64329b612a, remote_addr: 127.0.0.1:60535, local_addr: 127.0.0.1:9998
380592024-11-28T04:58:18.659ZINFOcrucible: dsc has 3 regions. This means 1 sub_volumes task = crutest
38060 Nov 28 04:58:18.659 INFO request completed, latency_us: 118, response_code: 200, uri: /port/cid/0, method: GET, req_id: 9be6db2e-f282-46fe-a528-261efb476e8d, remote_addr: 127.0.0.1:60535, local_addr: 127.0.0.1:9998
38061 Nov 28 04:58:18.660 INFO request completed, latency_us: 95, response_code: 200, uri: /port/cid/1, method: GET, req_id: c974d0a7-4a10-49b7-8b5f-a051886267c1, remote_addr: 127.0.0.1:60535, local_addr: 127.0.0.1:9998
38062 Nov 28 04:58:18.660 INFO request completed, latency_us: 91, response_code: 200, uri: /port/cid/2, method: GET, req_id: 574e3697-3b8a-477e-86ea-751dfb5e0838, remote_addr: 127.0.0.1:60535, local_addr: 127.0.0.1:9998
380632024-11-28T04:58:18.661ZINFOcrucible: SV 0 has targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830] task = crutest
380642024-11-28T04:58:18.661ZINFOcrucible: Upstairs starts
380652024-11-28T04:58:18.661ZINFOcrucible: Crucible Version: BuildInfo { version: "0.0.1", git_sha: "ccd99baabd70fa07cf893ca85ae393437a35a59a", git_commit_timestamp: "2024-11-28T04:33:57.000000000Z", git_branch: "renovate/opentelemetry-0.x", rustc_semver: "1.82.0", rustc_channel: "stable", rustc_host_triple: "x86_64-unknown-illumos", rustc_commit_sha: "f6e511eec7342f59a25f7c0534f1dbea00d01b14", cargo_triple: "x86_64-unknown-illumos", debug: true, opt_level: 0, }
380662024-11-28T04:58:18.661ZINFOcrucible: Upstairs <-> Downstairs Message Version: 12
380672024-11-28T04:58:18.661ZINFOcrucible: Using region definition RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 00000000-0000-0000-0000-000000000000, encrypted: false, database_read_version: 1, database_write_version: 1 }
380682024-11-28T04:58:18.661ZINFOcrucible: Crucible 1b8a1832-9ed9-4e2f-a4a1-385bae0b75e3 has session id: a1266168-33d3-4792-8acb-380d3b8c499e session_id = a1266168-33d3-4792-8acb-380d3b8c499e
380692024-11-28T04:58:18.661ZINFOcrucible: Upstairs opts: Upstairs UUID: 1b8a1832-9ed9-4e2f-a4a1-385bae0b75e3, Targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830], lossy: false, flush_timeout: None, key populated: false, cert_pem populated: false, key_pem populated: false, root_cert_pem populated: false, Control: None, read_only: false session_id = a1266168-33d3-4792-8acb-380d3b8c499e
380702024-11-28T04:58:18.661ZINFOcrucible: Crucible stats registered with UUID: 1b8a1832-9ed9-4e2f-a4a1-385bae0b75e3 session_id = a1266168-33d3-4792-8acb-380d3b8c499e
380712024-11-28T04:58:18.695ZINFOcrucible: Downstairs in volume = 3 task = crutest
380722024-11-28T04:58:18.695ZINFOcrucible: The guest has requested activation with gen:3
380732024-11-28T04:58:18.695ZINFOcrucible: 1b8a1832-9ed9-4e2f-a4a1-385bae0b75e3 active request set session_id = a1266168-33d3-4792-8acb-380d3b8c499e
380742024-11-28T04:58:18.695ZINFOcrucible: sending connect oneshot to client = downstairs client = 0 session_id = a1266168-33d3-4792-8acb-380d3b8c499e
380752024-11-28T04:58:18.695ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 0 session_id = a1266168-33d3-4792-8acb-380d3b8c499e
380762024-11-28T04:58:18.695ZINFOcrucible: sending connect oneshot to client = downstairs client = 1 session_id = a1266168-33d3-4792-8acb-380d3b8c499e
380772024-11-28T04:58:18.695ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 1 session_id = a1266168-33d3-4792-8acb-380d3b8c499e
380782024-11-28T04:58:18.695ZINFOcrucible: sending connect oneshot to client = downstairs client = 2 session_id = a1266168-33d3-4792-8acb-380d3b8c499e
380792024-11-28T04:58:18.695ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 2 session_id = a1266168-33d3-4792-8acb-380d3b8c499e
380802024-11-28T04:58:18.695ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = a1266168-33d3-4792-8acb-380d3b8c499e
380812024-11-28T04:58:18.695ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = a1266168-33d3-4792-8acb-380d3b8c499e
380822024-11-28T04:58:18.695ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = a1266168-33d3-4792-8acb-380d3b8c499e
380832024-11-28T04:58:18.695ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:39078) = io task client = 0 session_id = a1266168-33d3-4792-8acb-380d3b8c499e
380842024-11-28T04:58:18.695ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:62034) = io task client = 2 session_id = a1266168-33d3-4792-8acb-380d3b8c499e
380852024-11-28T04:58:18.696ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:44001) = io task client = 1 session_id = a1266168-33d3-4792-8acb-380d3b8c499e
380862024-11-28T04:58:18.696ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 1 session_id = a1266168-33d3-4792-8acb-380d3b8c499e
380872024-11-28T04:58:18.696ZINFOcrucible: [1] Transition from New to WaitActive = downstairs client = 1 session_id = a1266168-33d3-4792-8acb-380d3b8c499e
380882024-11-28T04:58:18.696ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 2 session_id = a1266168-33d3-4792-8acb-380d3b8c499e
380892024-11-28T04:58:18.696ZINFOcrucible: [2] Transition from New to WaitActive = downstairs client = 2 session_id = a1266168-33d3-4792-8acb-380d3b8c499e
380902024-11-28T04:58:18.696ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 0 session_id = a1266168-33d3-4792-8acb-380d3b8c499e
380912024-11-28T04:58:18.696ZINFOcrucible: [0] Transition from New to WaitActive = downstairs client = 0 session_id = a1266168-33d3-4792-8acb-380d3b8c499e
380922024-11-28T04:58:18.697ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = a1266168-33d3-4792-8acb-380d3b8c499e
380932024-11-28T04:58:18.697ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = a1266168-33d3-4792-8acb-380d3b8c499e
380942024-11-28T04:58:18.697ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = a1266168-33d3-4792-8acb-380d3b8c499e
380952024-11-28T04:58:18.698ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 1 session_id = a1266168-33d3-4792-8acb-380d3b8c499e
380962024-11-28T04:58:18.698ZINFOcrucible: [1] Transition from WaitActive to WaitQuorum = downstairs client = 1 session_id = a1266168-33d3-4792-8acb-380d3b8c499e
380972024-11-28T04:58:18.698ZINFOcrucible: Waiting for more clients to be ready session_id = a1266168-33d3-4792-8acb-380d3b8c499e
380982024-11-28T04:58:18.698ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 2 session_id = a1266168-33d3-4792-8acb-380d3b8c499e
380992024-11-28T04:58:18.698ZINFOcrucible: [2] Transition from WaitActive to WaitQuorum = downstairs client = 2 session_id = a1266168-33d3-4792-8acb-380d3b8c499e
381002024-11-28T04:58:18.698ZINFOcrucible: Waiting for more clients to be ready session_id = a1266168-33d3-4792-8acb-380d3b8c499e
381012024-11-28T04:58:18.698ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 0 session_id = a1266168-33d3-4792-8acb-380d3b8c499e
381022024-11-28T04:58:18.698ZINFOcrucible: [0] Transition from WaitActive to WaitQuorum = downstairs client = 0 session_id = a1266168-33d3-4792-8acb-380d3b8c499e
381032024-11-28T04:58:18.698ZINFOcrucible: [0]R flush_numbers[0..12]: [919, 928, 918, 909, 926, 920, 921, 925, 923, 868, 919, 921] = downstairs session_id = a1266168-33d3-4792-8acb-380d3b8c499e
381042024-11-28T04:58:18.698ZINFOcrucible: [0]R generation[0..12]: [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2] = downstairs session_id = a1266168-33d3-4792-8acb-380d3b8c499e
381052024-11-28T04:58:18.698ZINFOcrucible: [0]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = a1266168-33d3-4792-8acb-380d3b8c499e
381062024-11-28T04:58:18.698ZINFOcrucible: [1]R flush_numbers[0..12]: [919, 928, 918, 909, 926, 920, 921, 925, 923, 868, 919, 921] = downstairs session_id = a1266168-33d3-4792-8acb-380d3b8c499e
381072024-11-28T04:58:18.698ZINFOcrucible: [1]R generation[0..12]: [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2] = downstairs session_id = a1266168-33d3-4792-8acb-380d3b8c499e
381082024-11-28T04:58:18.698ZINFOcrucible: [1]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = a1266168-33d3-4792-8acb-380d3b8c499e
381092024-11-28T04:58:18.698ZINFOcrucible: [2]R flush_numbers[0..12]: [919, 928, 918, 909, 926, 920, 921, 925, 923, 868, 919, 921] = downstairs session_id = a1266168-33d3-4792-8acb-380d3b8c499e
381102024-11-28T04:58:18.698ZINFOcrucible: [2]R generation[0..12]: [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2] = downstairs session_id = a1266168-33d3-4792-8acb-380d3b8c499e
381112024-11-28T04:58:18.698ZINFOcrucible: [2]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = a1266168-33d3-4792-8acb-380d3b8c499e
381122024-11-28T04:58:18.698ZINFOcrucible: Max found gen is 3 = downstairs session_id = a1266168-33d3-4792-8acb-380d3b8c499e
381132024-11-28T04:58:18.698ZINFOcrucible: Generation requested: 3 >= found:3 = downstairs session_id = a1266168-33d3-4792-8acb-380d3b8c499e
38114 {"msg":"Next flush: 932","v":0,"name":"crucible","level"Wait for a query_work_queue command to finish before sending IO
38115 :30,"time":"2024-11-28T04:58:18.698501309Z","hostname":"w-01JDRHKEEFJ1228TFPSBEGXWHY","pid":1238,"":"downstairs","":"downstairs","session_id":"a1266168-33d3-4792-8acb-380d3b8c499e"}
381162024-11-28T04:58:18.698ZINFOcrucible: All extents match = downstairs session_id = a1266168-33d3-4792-8acb-380d3b8c499e
381172024-11-28T04:58:18.698ZINFOcrucible: No downstairs reconciliation required session_id = a1266168-33d3-4792-8acb-380d3b8c499e
381182024-11-28T04:58:18.698ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 0 session_id = a1266168-33d3-4792-8acb-380d3b8c499e
381192024-11-28T04:58:18.698ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 1 session_id = a1266168-33d3-4792-8acb-380d3b8c499e
381202024-11-28T04:58:18.698ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 2 session_id = a1266168-33d3-4792-8acb-380d3b8c499e
381212024-11-28T04:58:18.698ZINFOcrucible: All required reconciliation work is completed session_id = a1266168-33d3-4792-8acb-380d3b8c499e
381222024-11-28T04:58:18.698ZINFOcrucible: Set Downstairs and Upstairs active after reconciliation session_id = a1266168-33d3-4792-8acb-380d3b8c499e
381232024-11-28T04:58:18.698ZINFOcrucible: 1b8a1832-9ed9-4e2f-a4a1-385bae0b75e3 is now active with session: a1266168-33d3-4792-8acb-380d3b8c499e session_id = a1266168-33d3-4792-8acb-380d3b8c499e
381242024-11-28T04:58:18.698ZINFOcrucible: Set Active after no reconciliation session_id = a1266168-33d3-4792-8acb-380d3b8c499e
381252024-11-28T04:58:18.698ZINFOcrucible: The guest has finished waiting for activation with:3
38126 Disk: sv:1 bs:4096 ts:20480000 tb:5000 max_io:256 or 1048576
38127 Loading write count information from file "/tmp/test_replay_verify.log"
38128 Read and Verify all blocks (0..5000 range:false)
38129 Wrote out file "/tmp/test_replay_verify.log"
38130 Verify test completed
38131 Wrote out file "/tmp/test_replay_verify.log"
38132 CLIENT: Tests done. All submitted work has been ACK'd
38133 ----------------------------------------------------------------
38134 Crucible gen:3 GIO:true work queues: Upstairs:0 downstairs:20
38135 States: Active Active Active
38136 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
38137 1000 Acked Read 256 Done Done Done false
38138 1001 Acked Read 256 Done Done Done false
38139 1002 Acked Read 256 Done Done Done false
38140 1003 Acked Read 256 Done Done Done false
38141 1004 Acked Read 256 Done Done Done false
38142 1005 Acked Read 256 Done Done Done false
38143 1006 Acked Read 256 Done Done Done false
38144 1007 Acked Read 256 Done Done Done false
38145 1008 Acked Read 256 Done Done Done false
38146 1009 Acked Read 256 Done Done Done false
38147 1010 Acked Read 256 Done Done Done false
38148 1011 Acked Read 256 Done Done Done false
38149 1012 Acked Read 256 Done Done Done false
38150 1013 Acked Read 256 Done Done Done false
38151 1014 Acked Read 256 Done Done Done false
38152 1015 Acked Read 256 Done Done Done false
38153 1016 Acked Read 256 Done Done Done false
38154 1017 Acked Read 256 Done Done Done false
38155 1018 Acked Read 256 Done Done Done false
38156 1019 Acked Read 136 Done Done Done false
38157 STATES DS:0 DS:1 DS:2 TOTAL
38158 Sent 0 0 0 0
38159 Done 20 20 20 60
38160 Skipped 0 0 0 0
38161 Error 0 0 0 0
38162 Last Flush: 0 0 0
38163 Downstairs last five completed:
38164 Upstairs last five completed: 1019 1018 1017 1016 1015
38165 CLIENT: Up:0 ds:20 act:3
38166 ----------------------------------------------------------------
38167 Crucible gen:3 GIO:true work queues: Upstairs:0 downstairs:0
38168 Downstairs last five completed: 1020 1019 1018 1017 1016
38169 Upstairs last five completed: 1020 1019 1018 1017 1016
38170 CLIENT: Up:0 ds:0 act:3
38171 CLIENT: All crucible jobs finished, exiting program
381722024-11-28T04:58:23.104ZWARNcrucible: Guest handle has been dropped
38173 Nov 28 04:58:23.171 INFO accepted connection, remote_addr: 127.0.0.1:51448, local_addr: 127.0.0.1:9998
38174 Main task has work to do, go find it
38175 got dsc Shutdown
38176 Shutdown
38177 Nov 28 04:58:23.172 INFO request completed, latency_us: 187, response_code: 204, Shut it down
38178 uri: [8820] Disable keep_running
38179 /shutdown, method: POST, req_id:[8820] Got stop action so:false kr:false
38180 026c3083-07b3-4e0f-9f5c-35102c9e17ed, remote_addr:[8810] Disable keep_running
38181 [8810] Got stop action so:false kr:false
38182 127[8830] Disable keep_running
38183 .[8830] Got stop action so:false kr:false
38184 0.0.1:51448, local_addr: 127.0.0.1:9998
38185 [8820] Exited with: ExitStatus(unix_wait_status(9))
38186 [8820][1] reports Exit
38187 [8830] Exited with: ExitStatus(unix_wait_status(9))
38188 [8830][2] reports Exit
38189 [8810] Exited with: ExitStatus(unix_wait_status(9))
38190 [8810][0] reports Exit
38191 November 28, 2024 at 04:58:28 AM UTC Test ends with 0