Max level shown:
1 starting November 26, 2024 at 10:04:26 PM UTC
2 Creating 3 downstairs regions
3 Creating region directory at: /var/tmp/dsc/region
4 Creating dsc directory at: /tmp/dsc
5 Downstairs region 0 created at /var/tmp/dsc/region/8810 in 0.2601624
6 Downstairs region 1 created at /var/tmp/dsc/region/8820 in 0.020770008
7 Downstairs region 2 created at /var/tmp/dsc/region/8830 in 0.019356916
8 Created 3 regions
9 Starting 3 downstairs
10 Using existing output directory "/tmp/dsc"
11 Update our region info with: Some(RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 })
12 start ds: 8810
13 start access at:127.0.0.1:9998
14 start ds: 8820
15 start ds: 8830
16 Starting downstairs at port 8810
17 Starting downstairs at port 8820
18 Make output file at "/tmp/dsc/downstairs-8810.txt"
19 Make output file at "/tmp/dsc/downstairs-8820.txt"
20 Starting downstairs at port 8830
21 Make output file at "/tmp/dsc/downstairs-8830.txt"
22 [8810][0] initial start wait reports Starting
23 [8820][1] initial start wait reports Starting
24 [8830][2] initial start wait reports Starting
25 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1193)
26 [8810][0] initial start wait reports Running
27 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1194)
28 [8820][1] initial start wait reports Running
29 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1195)
30 [8830][2] initial start wait reports Running
31 All downstairs are running
32 Nov 26 22:04:26.584 INFO listening, local_addr: 127.0.0.1:9998
33 Control access at:127.0.0.1:9998
34 Running initial fill
35 Nov 26 22:04:31.651 INFO accepted connection, remote_addr: 127.0.0.1:57766, local_addr: 127.0.0.1:9998
36 Nov 26 22:04:31.654 INFO request completed, latency_us: 694, response_code: 200, uri: /regioninfo, method: GET, req_id: 6800e03d-7a2e-40fd-b8f4-8b45d2ec03ae, remote_addr: 127.0.0.1:57766, local_addr: 127.0.0.1:9998
37 Nov 26 22:04:31.656 INFO request completed, latency_us: 107, response_code: 200, uri: /regioncount, method: GET, req_id: 309fecab-5bfd-4f8c-8683-b29992f8619b, remote_addr: 127.0.0.1:57766, local_addr: 127.0.0.1:9998
382024-11-26T22:04:31.656ZINFOcrucible: Use this region info from dsc: RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 } task = crutest
392024-11-26T22:04:31.656ZINFOcrucible: dsc has 3 regions. This means 1 sub_volumes task = crutest
40 Nov 26 22:04:31.657 INFO request completed, latency_us: 168, response_code: 200, uri: /port/cid/0, method: GET, req_id: a519a7d0-bb53-4b04-af60-5b22aa13a879, remote_addr: 127.0.0.1:57766, local_addr: 127.0.0.1:9998
41 Nov 26 22:04:31.657 INFO request completed, latency_us: 92, response_code: 200, uri: /port/cid/1, method: GET, req_id: 614ec61a-4083-420a-8c1d-6f876cfbea8a, remote_addr: 127.0.0.1:57766, local_addr: 127.0.0.1:9998
42 Nov 26 22:04:31.658 INFO request completed, latency_us: 104, response_code: 200, uri: /port/cid/2, method: GET, req_id: f525ca2a-da1c-4ca5-8736-15bd2e2f5255, remote_addr: 127.0.0.1:57766, local_addr: 127.0.0.1:9998
432024-11-26T22:04:31.658ZINFOcrucible: SV 0 has targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830] task = crutest
442024-11-26T22:04:31.658ZINFOcrucible: Upstairs starts
452024-11-26T22:04:31.658ZINFOcrucible: Crucible Version: BuildInfo { version: "0.0.1", git_sha: "6bdc5ae0b2033c80bbfa3a59474c7c52737638e4", git_commit_timestamp: "2024-11-26T21:48:03.000000000Z", git_branch: "mkeeter/defer-flush-and-barrier", rustc_semver: "1.82.0", rustc_channel: "stable", rustc_host_triple: "x86_64-unknown-illumos", rustc_commit_sha: "f6e511eec7342f59a25f7c0534f1dbea00d01b14", cargo_triple: "x86_64-unknown-illumos", debug: true, opt_level: 0, }
462024-11-26T22:04:31.658ZINFOcrucible: Upstairs <-> Downstairs Message Version: 12
472024-11-26T22:04:31.659ZINFOcrucible: Using region definition RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 00000000-0000-0000-0000-000000000000, encrypted: false, database_read_version: 1, database_write_version: 1 }
482024-11-26T22:04:31.659ZINFOcrucible: Crucible b5a3a708-f3db-44c3-ac40-f538101ad0e5 has session id: 840aaa6a-bbd8-4cab-98fc-5717a0010afe session_id = 840aaa6a-bbd8-4cab-98fc-5717a0010afe
492024-11-26T22:04:31.659ZINFOcrucible: Upstairs opts: Upstairs UUID: b5a3a708-f3db-44c3-ac40-f538101ad0e5, Targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830], lossy: false, flush_timeout: None, key populated: false, cert_pem populated: false, key_pem populated: false, root_cert_pem populated: false, Control: None, read_only: false session_id = 840aaa6a-bbd8-4cab-98fc-5717a0010afe
502024-11-26T22:04:31.659ZINFOcrucible: Crucible stats registered with UUID: b5a3a708-f3db-44c3-ac40-f538101ad0e5 session_id = 840aaa6a-bbd8-4cab-98fc-5717a0010afe
512024-11-26T22:04:31.694ZINFOcrucible: Downstairs in volume = 3 task = crutest
522024-11-26T22:04:31.694ZINFOcrucible: The guest has requested activation with gen:1
532024-11-26T22:04:31.694ZINFOcrucible: b5a3a708-f3db-44c3-ac40-f538101ad0e5 active request set session_id = 840aaa6a-bbd8-4cab-98fc-5717a0010afe
542024-11-26T22:04:31.694ZINFOcrucible: sending connect oneshot to client = downstairs client = 0 session_id = 840aaa6a-bbd8-4cab-98fc-5717a0010afe
552024-11-26T22:04:31.694ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 0 session_id = 840aaa6a-bbd8-4cab-98fc-5717a0010afe
562024-11-26T22:04:31.694ZINFOcrucible: sending connect oneshot to client = downstairs client = 1 session_id = 840aaa6a-bbd8-4cab-98fc-5717a0010afe
572024-11-26T22:04:31.694ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 1 session_id = 840aaa6a-bbd8-4cab-98fc-5717a0010afe
582024-11-26T22:04:31.694ZINFOcrucible: sending connect oneshot to client = downstairs client = 2 session_id = 840aaa6a-bbd8-4cab-98fc-5717a0010afe
592024-11-26T22:04:31.694ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 2 session_id = 840aaa6a-bbd8-4cab-98fc-5717a0010afe
602024-11-26T22:04:31.694ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 840aaa6a-bbd8-4cab-98fc-5717a0010afe
612024-11-26T22:04:31.694ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 840aaa6a-bbd8-4cab-98fc-5717a0010afe
622024-11-26T22:04:31.694ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 840aaa6a-bbd8-4cab-98fc-5717a0010afe
632024-11-26T22:04:31.694ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:55665) = io task client = 2 session_id = 840aaa6a-bbd8-4cab-98fc-5717a0010afe
642024-11-26T22:04:31.695ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:46008) = io task client = 0 session_id = 840aaa6a-bbd8-4cab-98fc-5717a0010afe
652024-11-26T22:04:31.695ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:59392) = io task client = 1 session_id = 840aaa6a-bbd8-4cab-98fc-5717a0010afe
662024-11-26T22:04:31.696ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 1 session_id = 840aaa6a-bbd8-4cab-98fc-5717a0010afe
672024-11-26T22:04:31.696ZINFOcrucible: [1] Transition from New to WaitActive = downstairs client = 1 session_id = 840aaa6a-bbd8-4cab-98fc-5717a0010afe
682024-11-26T22:04:31.696ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 2 session_id = 840aaa6a-bbd8-4cab-98fc-5717a0010afe
692024-11-26T22:04:31.696ZINFOcrucible: [2] Transition from New to WaitActive = downstairs client = 2 session_id = 840aaa6a-bbd8-4cab-98fc-5717a0010afe
702024-11-26T22:04:31.697ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 0 session_id = 840aaa6a-bbd8-4cab-98fc-5717a0010afe
712024-11-26T22:04:31.697ZINFOcrucible: [0] Transition from New to WaitActive = downstairs client = 0 session_id = 840aaa6a-bbd8-4cab-98fc-5717a0010afe
722024-11-26T22:04:31.697ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 840aaa6a-bbd8-4cab-98fc-5717a0010afe
732024-11-26T22:04:31.697ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 840aaa6a-bbd8-4cab-98fc-5717a0010afe
742024-11-26T22:04:31.697ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 840aaa6a-bbd8-4cab-98fc-5717a0010afe
752024-11-26T22:04:31.698ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 1 session_id = 840aaa6a-bbd8-4cab-98fc-5717a0010afe
762024-11-26T22:04:31.698ZINFOcrucible: [1] Transition from WaitActive to WaitQuorum = downstairs client = 1 session_id = 840aaa6a-bbd8-4cab-98fc-5717a0010afe
772024-11-26T22:04:31.698ZINFOcrucible: Waiting for more clients to be ready session_id = 840aaa6a-bbd8-4cab-98fc-5717a0010afe
782024-11-26T22:04:31.698ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 0 session_id = 840aaa6a-bbd8-4cab-98fc-5717a0010afe
792024-11-26T22:04:31.698ZINFOcrucible: [0] Transition from WaitActive to WaitQuorum = downstairs client = 0 session_id = 840aaa6a-bbd8-4cab-98fc-5717a0010afe
802024-11-26T22:04:31.698ZINFOcrucible: Waiting for more clients to be ready session_id = 840aaa6a-bbd8-4cab-98fc-5717a0010afe
812024-11-26T22:04:31.698ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 2 session_id = 840aaa6a-bbd8-4cab-98fc-5717a0010afe
822024-11-26T22:04:31.698ZINFOcrucible: [2] Transition from WaitActive to WaitQuorum = downstairs client = 2 session_id = 840aaa6a-bbd8-4cab-98fc-5717a0010afe
832024-11-26T22:04:31.699ZINFOcrucible: [0]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 840aaa6a-bbd8-4cab-98fc-5717a0010afe
842024-11-26T22:04:31.699ZINFOcrucible: [0]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 840aaa6a-bbd8-4cab-98fc-5717a0010afe
852024-11-26T22:04:31.699ZINFOcrucible: [0]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 840aaa6a-bbd8-4cab-98fc-5717a0010afe
862024-11-26T22:04:31.699ZINFOcrucible: [1]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 840aaa6a-bbd8-4cab-98fc-5717a0010afe
872024-11-26T22:04:31.699ZINFOcrucible: [1]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 840aaa6a-bbd8-4cab-98fc-5717a0010afe
882024-11-26T22:04:31.699ZINFOcrucible: [1]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 840aaa6a-bbd8-4cab-98fc-5717a0010afe
892024-11-26T22:04:31.699ZINFOcrucible: [2]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 840aaa6a-bbd8-4cab-98fc-5717a0010afe
902024-11-26T22:04:31.699ZINFOcrucible: [2]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 840aaa6a-bbd8-4cab-98fc-5717a0010afe
912024-11-26T22:04:31.699ZINFOcrucible: [2]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 840aaa6a-bbd8-4cab-98fc-5717a0010afe
922024-11-26T22:04:31.699ZINFOcrucible: Max found gen is 1 = downstairs session_id = 840aaa6a-bbd8-4cab-98fc-5717a0010afe
932024-11-26T22:04:31.699ZINFOcrucible: Generation requested: 1 >= found:1 = downstairs session_id = 840aaa6a-bbd8-4cab-98fc-5717a0010afe
942024-11-26T22:04:31.699ZINFOcrucible: Next flush: 1 = downstairs session_id = 840aaa6a-bbd8-4cab-98fc-5717a0010afe
952024-11-26T22:04:31.699ZINFOcrucible: All extents match = downstairs session_id = 840aaa6a-bbd8-4cab-98fc-5717a0010afe
962024-11-26T22:04:31.699ZINFOcrucible: No downstairs reconciliation required session_id = 840aaa6a-bbd8-4cab-98fc-5717a0010afe
972024-11-26T22:04:31.699ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 0 session_id = 840aaa6a-bbd8-4cab-98fc-5717a0010afe
982024-11-26T22:04:31.699ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 1 session_id = 840aaa6a-bbd8-4cab-98fc-5717a0010afe
992024-11-26T22:04:31.699ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 2 session_id = 840aaa6a-bbd8-4cab-98fc-5717a0010afe
1002024-11-26T22:04:31.699ZINFOcrucible: All required reconciliation work is completed session_id = 840aaa6a-bbd8-4cab-98fc-5717a0010afe
1012024-11-26T22:04:31.699ZINFOcrucible: Set Downstairs and Upstairs active after reconciliation session_id = 840aaa6a-bbd8-4cab-98fc-5717a0010afe
1022024-11-26T22:04:31.699ZINFOcrucible: b5a3a708-f3db-44c3-ac40-f538101ad0e5 is now active with session: 840aaa6a-bbd8-4cab-98fc-5717a0010afe session_id = 840aaa6a-bbd8-4cab-98fc-5717a0010afe
103 {"msg":"Set Active after no reconciliation","v":0,"name":"crucible","level":30Activate successful
104 ,"Wait for a query_work_queue command to finish before sending IO
105 time":"2024-11-26T22:04:31.699449008Z","hostname":"w-01JDN7ZQ17ABQG15KHKN3F2D26","pid":1199,"session_id":"840aaa6a-bbd8-4cab-98fc-5717a0010afe"}
1062024-11-26T22:04:31.699ZINFOcrucible: The guest has finished waiting for activation with:1
107 Disk: sv:1 bs:4096 ts:20480000 tb:5000 max_io:256 or 1048576
108 Fill test
109 Read and Verify all blocks (0..5000 range:false)
110 Wrote out file "/tmp/test_replay_verify.log"
111 CLIENT: Tests done. All submitted work has been ACK'd
112 ----------------------------------------------------------------
113 Crucible gen:1 GIO:true work queues: Upstairs:0 downstairs:20
114 States: Active Active Active
115 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
116 1021 Acked Read 256 Done Done Done false
117 1022 Acked Read 256 Done Done Done false
118 1023 Acked Read 256 Done Done Done false
119 1024 Acked Read 256 Done Done Done false
120 1025 Acked Read 256 Done Done Done false
121 1026 Acked Read 256 Done Done Done false
122 1027 Acked Read 256 Done Done Done false
123 1028 Acked Read 256 Done Done Done false
124 1029 Acked Read 256 Done Done Done false
125 1030 Acked Read 256 Done Done Done false
126 1031 Acked Read 256 Done Done Done false
127 1032 Acked Read 256 Done Done Done false
128 1033 Acked Read 256 Done Done Done false
129 1034 Acked Read 256 Done Done Done false
130 1035 Acked Read 256 Done Done Done false
131 1036 Acked Read 256 Done Done Done false
132 1037 Acked Read 256 Done Done Done false
133 1038 Acked Read 256 Done Done Done false
134 1039 Acked Read 136 Done Done Done false
135 1040 Acked Read 256 Done Done Done false
136 STATES DS:0 DS:1 DS:2 TOTAL
137 Sent 0 0 0 0
138 Done 20 20 20 60
139 Skipped 0 0 0 0
140 Error 0 0 0 0
141 Last Flush: 1020 1020 1020
142 Downstairs last five completed: 1020 1019 1018 1017 1016
143 Upstairs last five completed: 1040 1039 1038 1037 1036
144 CLIENT: Up:0 ds:20 act:3
145 ----------------------------------------------------------------
146 Crucible gen:1 GIO:true work queues: Upstairs:0 downstairs:0
147 Downstairs last five completed: 1041 1040 1039 1038 1037
148 Upstairs last five completed: 1041 1040 1039 1038 1037
149 CLIENT: Up:0 ds:0 act:3
150 CLIENT: All crucible jobs finished, exiting program
1512024-11-26T22:04:36.258ZWARNcrucible: Guest handle has been dropped
152 Replay loop starts now November 26, 2024 at 10:04:36 PM UTC
153 Nov 26 22:04:36.332 INFO accepted connection, remote_addr: 127.0.0.1:43684, local_addr: 127.0.0.1:9998
154 Nov 26 22:04:36.333 INFO request completed, latency_us: 161, response_code: 200, uri: /regioninfo, method: GET, req_id: c697f622-04a0-4635-b99a-14d112030b25, remote_addr: 127.0.0.1:43684, local_addr: 127.0.0.1:9998
1552024-11-26T22:04:36.333ZINFOcrucible: Use this region info from dsc: RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 } task = crutest
156 Nov 26 22:04:36.334 INFO request completed, latency_us: 110, response_code: 200, uri: /regioncount, method: GET, req_id: 793ca8e2-c07e-49f2-9b60-f208e19a9226, remote_addr: 127.0.0.1:43684, local_addr: 127.0.0.1:9998
1572024-11-26T22:04:36.334ZINFOcrucible: dsc has 3 regions. This means 1 sub_volumes task = crutest
158 Nov 26 22:04:36.334 INFO request completed, latency_us: 114, response_code: 200, uri: /port/cid/0, method: GET, req_id: 15d0351b-d373-489c-9a21-32ab018a0079, remote_addr: 127.0.0.1:43684, local_addr: 127.0.0.1:9998
159 Nov 26 22:04:36.335 INFO request completed, latency_us: 99, response_code: 200, uri: /port/cid/1, method: GET, req_id: 624a86ad-c9e9-4ca9-8757-8d18ee9b2601, remote_addr: 127.0.0.1:43684, local_addr: 127.0.0.1:9998
160 Nov 26 22:04:36.335 INFO request completed, latency_us: 96, response_code: 200, uri: /port/cid/2, method: GET, req_id: 12a0cdad-b98d-47ed-9006-10b7caf6c95a, remote_addr: 127.0.0.1:43684, local_addr: 127.0.0.1:9998
1612024-11-26T22:04:36.336ZINFOcrucible: SV 0 has targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830] task = crutest
1622024-11-26T22:04:36.336ZINFOcrucible: Upstairs starts
1632024-11-26T22:04:36.336ZINFOcrucible: Crucible Version: BuildInfo { version: "0.0.1", git_sha: "6bdc5ae0b2033c80bbfa3a59474c7c52737638e4", git_commit_timestamp: "2024-11-26T21:48:03.000000000Z", git_branch: "mkeeter/defer-flush-and-barrier", rustc_semver: "1.82.0", rustc_channel: "stable", rustc_host_triple: "x86_64-unknown-illumos", rustc_commit_sha: "f6e511eec7342f59a25f7c0534f1dbea00d01b14", cargo_triple: "x86_64-unknown-illumos", debug: true, opt_level: 0, }
1642024-11-26T22:04:36.336ZINFOcrucible: Upstairs <-> Downstairs Message Version: 12
1652024-11-26T22:04:36.336ZINFOcrucible: Using region definition RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 00000000-0000-0000-0000-000000000000, encrypted: false, database_read_version: 1, database_write_version: 1 }
1662024-11-26T22:04:36.336ZINFOcrucible: Crucible f8ff4dff-20dd-48b9-bfe1-620687331e85 has session id: 86a36c17-3718-484c-87fd-2b8a1dc2ab18 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
1672024-11-26T22:04:36.336ZINFOcrucible: Upstairs opts: Upstairs UUID: f8ff4dff-20dd-48b9-bfe1-620687331e85, Targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830], lossy: false, flush_timeout: None, key populated: false, cert_pem populated: false, key_pem populated: false, root_cert_pem populated: false, Control: None, read_only: false session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
1682024-11-26T22:04:36.336ZINFOcrucible: Crucible stats registered with UUID: f8ff4dff-20dd-48b9-bfe1-620687331e85 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
1692024-11-26T22:04:36.370ZINFOcrucible: Downstairs in volume = 3 task = crutest
1702024-11-26T22:04:36.370ZINFOcrucible: The guest has requested activation with gen:2
1712024-11-26T22:04:36.370ZINFOcrucible: f8ff4dff-20dd-48b9-bfe1-620687331e85 active request set session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
1722024-11-26T22:04:36.370ZINFOcrucible: sending connect oneshot to client = downstairs client = 0 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
1732024-11-26T22:04:36.370ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 0 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
1742024-11-26T22:04:36.370ZINFOcrucible: sending connect oneshot to client = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
1752024-11-26T22:04:36.370ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
1762024-11-26T22:04:36.370ZINFOcrucible: sending connect oneshot to client = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
1772024-11-26T22:04:36.370ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
1782024-11-26T22:04:36.370ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
1792024-11-26T22:04:36.370ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
1802024-11-26T22:04:36.370ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
1812024-11-26T22:04:36.370ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:40611) = io task client = 0 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
1822024-11-26T22:04:36.371ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:36198) = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
1832024-11-26T22:04:36.371ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:61232) = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
1842024-11-26T22:04:36.371ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 0 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
1852024-11-26T22:04:36.371ZINFOcrucible: [0] Transition from New to WaitActive = downstairs client = 0 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
1862024-11-26T22:04:36.371ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
1872024-11-26T22:04:36.371ZINFOcrucible: [1] Transition from New to WaitActive = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
1882024-11-26T22:04:36.371ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
1892024-11-26T22:04:36.371ZINFOcrucible: [2] Transition from New to WaitActive = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
1902024-11-26T22:04:36.372ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
1912024-11-26T22:04:36.372ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
1922024-11-26T22:04:36.372ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
1932024-11-26T22:04:36.373ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
1942024-11-26T22:04:36.373ZINFOcrucible: [1] Transition from WaitActive to WaitQuorum = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
1952024-11-26T22:04:36.373ZINFOcrucible: Waiting for more clients to be ready session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
1962024-11-26T22:04:36.373ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
1972024-11-26T22:04:36.373ZINFOcrucible: [2] Transition from WaitActive to WaitQuorum = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
1982024-11-26T22:04:36.373ZINFOcrucible: Waiting for more clients to be ready session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
1992024-11-26T22:04:36.373ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 0 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
2002024-11-26T22:04:36.373ZINFOcrucible: [0] Transition from WaitActive to WaitQuorum = downstairs client = 0 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
2012024-11-26T22:04:36.373ZINFOcrucible: [0]R flush_numbers[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
2022024-11-26T22:04:36.373ZINFOcrucible: [0]R generation[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
2032024-11-26T22:04:36.373ZINFOcrucible: [0]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
2042024-11-26T22:04:36.373ZINFOcrucible: [1]R flush_numbers[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
2052024-11-26T22:04:36.373ZINFOcrucible: [1]R generation[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
2062024-11-26T22:04:36.373ZINFOcrucible: [1]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
2072024-11-26T22:04:36.373ZINFOcrucible: [2]R flush_numbers[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
2082024-11-26T22:04:36.373ZINFOcrucible: [2]R generation[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
2092024-11-26T22:04:36.373ZINFOcrucible: [2]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
2102024-11-26T22:04:36.373ZINFOcrucible: Max found gen is 2 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
2112024-11-26T22:04:36.373ZINFOcrucible: Generation requested: 2 >= found:2 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
2122024-11-26T22:04:36.373ZINFOcrucible: Next flush: 2 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
213 {"msg":"All extents match","v":0,"name":"crucible","level":30Activate successful
214 ,Wait for a query_work_queue command to finish before sending IO
215 "time":"2024-11-26T22:04:36.373412519Z","hostname":"w-01JDN7ZQ17ABQG15KHKN3F2D26","pid":1203,"":"downstairs","":"downstairs","session_id":"86a36c17-3718-484c-87fd-2b8a1dc2ab18"}
2162024-11-26T22:04:36.373ZINFOcrucible: No downstairs reconciliation required session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
2172024-11-26T22:04:36.373ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 0 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
2182024-11-26T22:04:36.373ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
2192024-11-26T22:04:36.373ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
2202024-11-26T22:04:36.373ZINFOcrucible: All required reconciliation work is completed session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
2212024-11-26T22:04:36.373ZINFOcrucible: Set Downstairs and Upstairs active after reconciliation session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
2222024-11-26T22:04:36.373ZINFOcrucible: f8ff4dff-20dd-48b9-bfe1-620687331e85 is now active with session: 86a36c17-3718-484c-87fd-2b8a1dc2ab18 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
2232024-11-26T22:04:36.373ZINFOcrucible: Set Active after no reconciliation session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
2242024-11-26T22:04:36.373ZINFOcrucible: The guest has finished waiting for activation with:2
225 Disk: sv:1 bs:4096 ts:20480000 tb:5000 max_io:256 or 1048576
226 Loading write count information from file "/tmp/test_replay_verify.log"
227 Nov 26 22:04:36.406 INFO accepted connection, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
228 Main task has work to do, go find it
229 got dsc Stop(1)
230 stop 1
231 Nov 26 22:04:36.407 INFO request completed, latency_us: 176, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 02cb7126-a3e5-4b1b-b6aa-02f4b5c63a80, remote_addr[8820] Got stop action so:false kr:false
232 : 127.0.0.1:36829, local_addr: 127.0.0.1:9998
2332024-11-26T22:04:36.407ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
2342024-11-26T22:04:36.407ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
2352024-11-26T22:04:36.408ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
2362024-11-26T22:04:36.408ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
2372024-11-26T22:04:36.408ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
2382024-11-26T22:04:36.408ZINFOcrucible: client task is exiting = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
239 Nov 26 22:04:36.408 INFO request completed, latency_us: 136, response_code: 200, uri: /state/cid/1, method: GET, req_id: f7a21e69-07aa-47d3-8560-417f7e37a93b, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
240 [8820] Exited with: ExitStatus(unix_wait_status(9))
241 [8820][1] reports Exit
242 Nov 26 22:04:40.410 INFO request completed, latency_us: 206, response_code: 200, uri: /state/cid/1, method: GET, req_id: c424957d-edad-44f4-9588-710c03781ee4, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
243 001/300 Read block 3221 len 8192
244 002/300 Flush
245 003/300 Write block 2804 len 4096 data: 2
246 004/300 Read block 1944 len 32768
247 005/300 Flush
248 006/300 Read block 2581 len 12288
249 007/300 Write block 462 len 20480 data: 2 2 2 2 2
250 008/300 Write block 56 len 24576 data: 2 2 2 2 2 2
251 009/300 Write block 4434 len 24576 data: 2 2 2 2 2 2
252 010/300 Read block 1392 len 32768
253 011/300 Read block 2424 len 16384
254 012/300 Write block 3796 len 32768 data: 2 2 2 2 2 2 2 2
255 013/300 Write block 403 len 12288 data: 2 2 2
256 014/300 Write block 4568 len 36864 data: 2 2 2 2 2 2 2 2 2
257 015/300 Flush
258 016/300 Read block 1304 len 16384
259 017/300 Write block 722 len 16384 data: 2 2 2 2
260 018/300 Write block 2960 len 12288 data: 2 2 2
261 019/300 Read block 2421 len 32768
262 020/300 Read block 2235 len 24576
263 021/300 Write block 4322 len 28672 data: 2 2 2 2 2 2 2
264 022/300 Write block 1815 len 8192 data: 2 2
265 023/300 Read block 3584 len 4096
266 024/300 Write block 3378 len 32768 data: 2 2 2 2 2 2 2 2
267 025/300 Read block 1906 len 24576
268 026/300 Read block 2647 len 24576
269 027/300 Write block 10 len 4096 data: 2
270 028/300 Write block 114 len 20480 data: 2 2 2 2 2
271 029/300 Read block 2046 len 12288
272 030/300 Read block 4431 len 12288
273 031/300 Write block 2196 len 20480 data: 2 2 2 2 2
274 032/300 Write block 1779 len 8192 data: 2 2
275 033/300 Write block 2320 len 20480 data: 2 2 2 2 2
276 034/300 Write block 571 len 8192 data: 2 2
277 035/300 Write block 2556 len 40960 data: 2 2 2 2 2 2 2 2 2 2
278 036/300 Write block 2100 len 24576 data: 2 2 2 2 2 2
279 037/300 Read block 4124 len 16384
280 038/300 Read block 1490 len 24576
281 039/300 Read block 629 len 24576
282 040/300 Read block 2536 len 4096
283 041/300 Read block 3521 len 40960
284 042/300 Write block 3698 len 16384 data: 2 2 2 2
285 043/300 Read block 1566 len 36864
286 044/300 Write block 4611 len 24576 data: 2 2 2 2 2 2
287 045/300 Write block 4388 len 8192 data: 2 2
288 046/300 Read block 4359 len 4096
289 047/300 Write block 3461 len 12288 data: 2 2 2
290 048/300 Read block 4672 len 12288
291 049/300 Write block 2340 len 8192 data: 2 2
292 050/300 Read block 1295 len 12288
293 051/300 Write block 74 len 24576 data: 2 2 2 2 2 2
294 052/300 Read block 2554 len 12288
295 053/300 Write block 3391 len 20480 data: 2 2 2 2 2
296 054/300 Read block 1632 len 16384
297 055/300 Flush
298 056/300 Read block 1456 len 28672
299 057/300 Write block 3176 len 28672 data: 2 2 2 2 2 2 2
300 058/300 Read block 3428 len 20480
301 059/300 Write block 1031 len 20480 data: 2 2 2 2 2
302 060/300 Write block 461 len 8192 data: 2 3
303 061/300 Write block 2330 len 4096 data: 2
304 062/300 Read block 2772 len 12288
305 063/300 Read block 789 len 24576
306 064/300 Read block 718 len 32768
307 065/300 Read block 2160 len 32768
308 066/300 Write block 4856 len 36864 data: 2 2 2 2 2 2 2 2 2
309 067/300 Flush
310 068/300 Read block 766 len 8192
311 069/300 Read block 401 len 16384
312 070/300 Read block 837 len 32768
313 071/300 Write block 1264 len 28672 data: 2 2 2 2 2 2 2
314 072/300 Read block 1855 len 8192
315 073/300 Flush
316 074/300 Read block 1920 len 20480
317 075/300 Read block 2662 len 20480
318 076/300 Write block 1021 len 4096 data: 2
319 077/300 Flush
320 078/300 Read block 2857 len 24576
321 079/300 Read block 2285 len 20480
322 080/300 Read block 3503 len 40960
323 081/300 Read block 2895 len 36864
324 082/300 Write block 2667 len 8192 data: 2 2
325 083/300 Write block 1075 len 40960 data: 2 2 2 2 2 2 2 2 2 2
326 084/300 Read block 1280 len 16384
327 085/300 Write block 1662 len 16384 data: 2 2 2 2
328 086/300 Write block 1649 len 12288 data: 2 2 2
329 087/300 Read block 4522 len 32768
330 088/300 Read block 3302 len 36864
331 089/300 Read block 4942 len 24576
332 090/300 Read block 2990 len 4096
333 091/300 Read block 2906 len 40960
334 092/300 Flush
335 093/300 Read block 1751 len 24576
336 094/300 Write block 1052 len 20480 data: 2 2 2 2 2
337 095/300 Read block 3923 len 4096
338 096/300 Flush
339 097/300 Read block 4519 len 40960
340 098/300 Write block 4731 len 12288 data: 2 2 2
341 099/300 Write block 1790 len 16384 data: 2 2 2 2
342 100/300 Read block 1799 len 40960
343 101/300 Read block 1815 len 8192
344 102/300 Write block 2702 len 32768 data: 2 2 2 2 2 2 2 2
345 103/300 Read block 1364 len 4096
346 104/300 Write block 289 len 32768 data: 2 2 2 2 2 2 2 2
347 105/300 Read block 4163 len 40960
348 106/300 Read block 4709 len 36864
349 107/300 Read block 3164 len 4096
350 108/300 Read block 1810 len 28672
351 109/300 Write block 3261 len 12288 data: 2 2 2
352 110/300 Write block 1880 len 24576 data: 2 2 2 2 2 2
353 111/300 Write block 3827 len 20480 data: 2 2 2 2 2
354 112/300 Write block 2638 len 28672 data: 2 2 2 2 2 2 2
355 113/300 Read block 1421 len 24576
356 114/300 Read block 852 len 28672
357 115/300 Write block 3213 len 32768 data: 2 2 2 2 2 2 2 2
358 116/300 Read block 2404 len 40960
359 117/300 Read block 3085 len 8192
360 118/300 Write block 3394 len 20480 data: 3 3 2 2 2
361 119/300 Read block 4038 len 36864
362 120/300 Read block 1800 len 12288
363 121/300 Write block 4216 len 12288 data: 2 2 2
364 122/300 Write block 3624 len 40960 data: 2 2 2 2 2 2 2 2 2 2
365 123/300 Write block 2763 len 8192 data: 2 2
366 124/300 Read block 1480 len 4096
367 125/300 Write block 101 len 16384 data: 2 2 2 2
368 126/300 Write block 4789 len 28672 data: 2 2 2 2 2 2 2
369 127/300 Flush
370 128/300 Flush
371 129/300 Flush
372 130/300 Write block 4267 len 4096 data: 2
373 131/300 Read block 79 len 16384
374 132/300 Write block 745 len 8192 data: 2 2
375 133/300 Write block 1599 len 28672 data: 2 2 2 2 2 2 2
376 134/300 Read block 714 len 32768
377 135/300 Read block 2233 len 24576
378 136/300 Write block 496 len 4096 data: 2
379 137/300 Read block 2463 len 40960
380 138/300 Read block 1191 len 32768
381 139/300 Write block 4644 len 40960 data: 2 2 2 2 2 2 2 2 2 2
382 140/300 Read block 1769 len 12288
383 141/300 Read block 4077 len 12288
384 142/300 Write block 2685 len 20480 data: 2 2 2 2 2
385 143/300 Read block 2120 len 12288
386 144/300 Read block 1605 len 4096
387 145/300 Write block 2636 len 16384 data: 2 2 3 3
388 146/300 Flush
389 147/300 Read block 149 len 20480
390 148/300 Write block 3827 len 40960 data: 3 3 3 3 3 2 2 2 2 2
391 149/300 Write block 4515 len 8192 data: 2 2
392 150/300 Write block 2500 len 4096 data: 2
393 151/300 Read block 3618 len 4096
394 152/300 Write block 4537 len 12288 data: 2 2 2
395 153/300 Write block 4827 len 40960 data: 2 2 2 2 2 2 2 2 2 2
396 154/300 Flush
397 155/300 Write block 2998 len 16384 data: 2 2 2 2
398 156/300 Read block 3682 len 12288
399 157/300 Write block 3589 len 40960 data: 2 2 2 2 2 2 2 2 2 2
400 158/300 Write block 2411 len 16384 data: 2 2 2 2
401 159/300 Read block 762 len 12288
402 160/300 Read block 3500 len 24576
403 161/300 Read block 2951 len 36864
404 162/300 Read block 3997 len 36864
405 163/300 Read block 295 len 24576
406 164/300 Write block 4925 len 24576 data: 2 2 2 2 2 2
407 165/300 Read block 3506 len 32768
408 166/300 Write block 3971 len 8192 data: 2 2
409 167/300 Read block 1137 len 4096
410 168/300 Flush
411 169/300 Read block 1280 len 20480
412 170/300 Read block 278 len 40960
413 171/300 Read block 1259 len 32768
414 172/300 Read block 1422 len 8192
415 173/300 Read block 4777 len 8192
416 174/300 Write block 1667 len 24576 data: 2 2 2 2 2 2
417 175/300 Write block 1201 len 20480 data: 2 2 2 2 2
418 176/300 Write block 1903 len 12288 data: 2 2 2
419 177/300 Read block 301 len 24576
420 178/300 Read block 1877 len 8192
421 179/300 Write block 1937 len 36864 data: 2 2 2 2 2 2 2 2 2
422 180/300 Write block 3054 len 4096 data: 2
423 181/300 Write block 4271 len 40960 data: 2 2 2 2 2 2 2 2 2 2
424 182/300 Write block 2567 len 28672 data: 2 2 2 2 2 2 2
425 183/300 Write block 2822 len 8192 data: 2 2
426 184/300 Write block 4715 len 32768 data: 2 2 2 2 2 2 2 2
427 185/300 Read block 3115 len 36864
428 186/300 Write block 2589 len 32768 data: 2 2 2 2 2 2 2 2
429 187/300 Write block 4372 len 12288 data: 2 2 2
430 188/300 Read block 240 len 16384
431 189/300 Write block 4608 len 36864 data: 2 2 2 3 3 3 3 3 3
432 190/300 Write block 4068 len 32768 data: 2 2 2 2 2 2 2 2
433 191/300 Read block 3249 len 16384
434 192/300 Read block 2467 len 8192
435 193/300 Read block 3874 len 28672
436 194/300 Write block 1850 len 36864 data: 2 2 2 2 2 2 2 2 2
437 195/300 Read block 1802 len 24576
438 196/300 Read block 1346 len 36864
439 197/300 Write block 1140 len 12288 data: 2 2 2
440 198/300 Read block 1215 len 28672
441 199/300 Read block 1280 len 12288
442 200/300 Flush
443 201/300 Write block 3593 len 28672 data: 3 3 3 3 3 3 2
444 202/300 Flush
445 203/300 Write block 4801 len 24576 data: 2 2 2 2 2 2
446 204/300 Read block 4898 len 32768
447 205/300 Read block 4718 len 4096
448 206/300 Read block 175 len 32768
449 207/300 Read block 3138 len 4096
450 208/300 Write block 1533 len 40960 data: 2 2 2 2 2 2 2 2 2 2
451 209/300 Read block 4141 len 20480
452 210/300 Write block 2259 len 12288 data: 2 2 2
453 211/300 Flush
454 212/300 Read block 3863 len 24576
455 213/300 Read block 2777 len 24576
456 214/300 Read block 4406 len 12288
457 215/300 Read block 3251 len 28672
458 216/300 Write block 3777 len 36864 data: 2 2 2 2 2 2 2 2 2
459 217/300 Write block 3287 len 8192 data: 2 2
460 218/300 Flush
461 219/300 Write block 1847 len 28672 data: 2 2 2 3 3 3 3
462 220/300 Read block 2827 len 40960
463 221/300 Read block 1210 len 36864
464 222/300 Read block 3274 len 12288
465 223/300 Read block 1844 len 40960
466 224/300 Read block 1071 len 4096
467 225/300 Flush
468 226/300 Read block 4091 len 12288
469 227/300 Read block 4324 len 8192
470 228/300 Write block 3647 len 32768 data: 2 2 2 2 2 2 2 2
471 229/300 Flush
472 230/300 Read block 322 len 28672
473 231/300 Read block 719 len 24576
474 232/300 Read block 2036 len 40960
475 233/300 Flush
476 234/300 Flush
477 235/300 Read block 2396 len 36864
478 236/300 Flush
479 237/300 Flush
480 238/300 Write block 1766 len 8192 data: 2 2
481 239/300 Write block 1343 len 4096 data: 2
482 240/300 Write block 2160 len 12288 data: 2 2 2
483 241/300 Read block 280 len 24576
484 242/300 Flush
485 243/300 Write block 4434 len 8192 data: 3 3
486 244/300 Flush
487 245/300 Read block 4654 len 36864
488 246/300 Read block 232 len 12288
489 247/300 Read block 160 len 16384
490 248/300 Flush
491 249/300 Write block 2080 len 32768 data: 2 2 2 2 2 2 2 2
492 250/300 Read block 640 len 40960
493 251/300 Read block 2456 len 16384
494 252/300 Read block 3891 len 24576
495 253/300 Read block 1742 len 40960
496 254/300 Read block 3669 len 8192
497 255/300 Write block 2314 len 16384 data: 2 2 2 2
498 256/300 Flush
499 257/300 Read block 2754 len 8192
500 258/300 Write block 1715 len 12288 data: 2 2 2
501 259/300 Write block 2838 len 24576 data: 2 2 2 2 2 2
502 260/300 Read block 1106 len 24576
503 261/300 Flush
504 262/300 Read block 1737 len 4096
505 263/300 Read block 2211 len 8192
506 264/300 Read block 2876 len 4096
507 265/300 Write block 2396 len 40960 data: 2 2 2 2 2 2 2 2 2 2
508 266/300 Read block 3268 len 24576
509 267/300 Read block 4210 len 40960
510 268/300 Write block 567 len 40960 data: 2 2 2 2 3 3 2 2 2 2
511 269/300 Read block 4075 len 32768
512 270/300 Read block 4374 len 28672
513 271/300 Write block 1420 len 4096 data: 2
514 272/300 Write block 1209 len 28672 data: 2 2 2 2 2 2 2
515 273/300 Write block 3664 len 12288 data: 2 2 2
516 274/300 Flush
517 275/300 Write block 2516 len 32768 data: 2 2 2 2 2 2 2 2
518 276/300 Read block 3122 len 8192
519 277/300 Read block 3474 len 40960
520 278/300 Write block 2246 len 36864 data: 2 2 2 2 2 2 2 2 2
521 279/300 Write block 3473 len 36864 data: 2 2 2 2 2 2 2 2 2
522 280/300 Write block 860 len 36864 data: 2 2 2 2 2 2 2 2 2
523 281/300 Read block 3790 len 8192
524 282/300 Read block 1588 len 8192
525 283/300 Read block 4674 len 24576
526 284/300 Read block 793 len 32768
527 285/300 Read block 4137 len 12288
528 286/300 Read block 3414 len 8192
529 287/300 Read block 3267 len 12288
530 288/300 Flush
531 289/300 Write block 281 len 24576 data: 2 2 2 2 2 2
532 290/300 Write block 1209 len 40960 data: 3 3 3 3 3 3 3 2 2 2
533 291/300 Read block 4168 len 36864
534 292/300 Write block 985 len 24576 data: 2 2 2 2 2 2
535 293/300 Write block 1673 len 8192 data: 2 2
536 294/300 Read block 223 len 32768
537 295/300 Read block 2381 len 36864
538 296/300 Read block 1139 len 4096
539 297/300 Read block 4170 len 12288
540 298/300 Read block 944 len 20480
541 299/300 Read block 3773 len 4096
542 300/300 Read block 1337 len 24576
543 301/300 Read block 3396 len 32768
544 Main task has work to do, go find it
545 got dsc Start(1)
546 start 1
547 Nov 26 22:04:40.645 INFO request completed, latency_us: 213, response_code: 204, uri: /start/cid/1, method: POST, req_id: 1fb9e48d-2c46-4b70-ae13-9d8cd0f09496, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
548 Replay: started 1, returned:Ok(())
549 ----------------------------------------------------------------
550 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
551 States: Active Offline Active
552 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
553 1000 Acked Read 2 Done Sent Done false
554 1001 Acked Flush 0 Done Sent Done false
555 1002 Acked Write 1 Done Sent Done false
556 1003 Acked Read 8 Done Sent Done false
557 1004 Acked Flush 0 Done Sent Done false
558 1005 Acked Read 3 Done Sent Done false
559 1006 Acked Write 5 Done Sent Done false
560 1007 Acked Write 6 Done Sent Done false
561 1008 Acked Write 6 Done Sent Done false
562 1009 Acked Read 8 Done Sent Done false
563 1010 Acked Read 4 Done Sent Done false
564 1011 Acked Write 8 Done Sent Done false
565 1012 Acked Write 3 Done Sent Done false
566 1013 Acked Write 9 Done Sent Done false
567 1014 Acked Flush 0 Done Sent Done false
568 1015 Acked Read 4 Done Sent Done false
569 1016 Acked Write 4 Done Sent Done false
570 1017 Acked Write 3 Done Sent Done false
571 1018 Acked Read 8 Done Sent Done false
572 1019 Acked Read 6 Done Sent Done false
573 1020 Acked Write 7 Done Sent Done false
574 1021 Acked Write 2 Done Sent Done false
575 1022 Acked Read 1 Done Sent Done false
576 1023 Acked Write 8 Done Sent Done false
577 1024 Acked Read 6 Done Sent Done false
578 1025 Acked Read 6 Done Sent Done false
579 1026 Acked Write 1 Done Sent Done false
580 1027 Acked Write 5 Done Sent Done false
581 1028 Acked Read 3 Done Sent Done false
582 1029 Acked Read 3 Done Sent Done false
583 1030 Acked Write 5 Done Sent Done false
584 1031 Acked Write 2 Done Sent Done false
585 1032 Acked Write 5 Done Sent Done false
586 1033 Acked Write 2 Done Sent Done false
587 1034 Acked Write 10 Done Sent Done false
588 1035 Acked Write 6 Done Sent Done false
589 1036 Acked Read 4 Done Sent Done false
590 1037 Acked Read 6 Done Sent Done false
591 1038 Acked Read 6 Done Sent Done false
592 1039 Acked Read 1 Done Sent Done false
593 1040 Acked Read 10 Done Sent Done false
594 1041 Acked Write 4 Done Sent Done false
595 1042 Acked Read 9 Done Sent Done false
596 1043 Acked Write 6 Done Sent Done false
597 1044 Acked Write 2 Done Sent Done false
598 1045 Acked Read 1 Done Sent Done false
599 1046 Acked Write 3 Done Sent Done false
600 1047 Acked Read 3 Done Sent Done false
601 1048 Acked Write 2 Done Sent Done false
602 1049 Acked Read 3 Done Sent Done false
603 1050 Acked Write 6 Done Sent Done false
604 1051 Acked Read 3 Done Sent Done false
605 1052 Acked Write 5 Done Sent Done false
606 1053 Acked Read 4 Done Sent Done false
607 1054 Acked Flush 0 Done Sent Done false
608 1055 Acked Read 7 Done Sent Done false
609 1056 Acked Write 7 Done Sent Done false
610 1057 Acked Read 5 Done Sent Done false
611 1058 Acked Write 5 Done Sent Done false
612 1059 Acked Write 2 Done Sent Done false
613 1060 Acked Write 1 Done Sent Done false
614 1061 Acked Read 3 Done Sent Done false
615 1062 Acked Read 6 Done Sent Done false
616 1063 Acked Read 8 Done Sent Done false
617 1064 Acked Read 8 Done Sent Done false
618 1065 Acked Write 9 Done Sent Done false
619 1066 Acked Flush 0 Done Sent Done false
620 1067 Acked Read 2 Done Sent Done false
621 1068 Acked Read 4 Done Sent Done false
622 1069 Acked Read 8 Done Sent Done false
623 1070 Acked Write 7 Done Sent Done false
624 1071 Acked Read 2 Done Sent Done false
625 1072 Acked Flush 0 Done Sent Done false
626 1073 Acked Read 5 Done Sent Done false
627 1074 Acked Read 5 Done Sent Done false
628 1075 Acked Write 1 Done Sent Done false
629 1076 Acked Flush 0 Done Sent Done false
630 1077 Acked Read 6 Done Sent Done false
631 1078 Acked Read 5 Done Sent Done false
632 1079 Acked Read 10 Done Sent Done false
633 1080 Acked Read 9 Done Sent Done false
634 1081 Acked Write 2 Done Sent Done false
635 1082 Acked Write 10 Done Sent Done false
636 1083 Acked Read 4 Done Sent Done false
637 1084 Acked Write 4 Done Sent Done false
638 1085 Acked Write 3 Done Sent Done false
639 1086 Acked Read 8 Done Sent Done false
640 1087 Acked Read 9 Done Sent Done false
641 1088 Acked Read 6 Done Sent Done false
642 1089 Acked Read 1 Done Sent Done false
643 1090 Acked Read 10 Done Sent Done false
644 1091 Acked Flush 0 Done Sent Done false
645 1092 Acked Read 6 Done Sent Done false
646 1093 Acked Write 5 Done Sent Done false
647 1094 Acked Read 1 Done Sent Done false
648 1095 Acked Flush 0 Done Sent Done false
649 1096 Acked Read 10 Done Sent Done false
650 1097 Acked Write 3 Done Sent Done false
651 1098 Acked Write 4 Done Sent Done false
652 1099 Acked Read 10 Done Sent Done false
653 1100 Acked Read 2 Done Sent Done false
654 1101 Acked Write 8 Done Sent Done false
655 1102 Acked Read 1 Done Sent Done false
656 1103 Acked Write 8 Done Sent Done false
657 1104 Acked Read 10 Done Sent Done false
658 1105 Acked Read 9 Done Sent Done false
659 1106 Acked Read 1 Done Sent Done false
660 1107 Acked Read 7 Done Sent Done false
661 1108 Acked Write 3 Done Sent Done false
662 1109 Acked Write 6 Done Sent Done false
663 1110 Acked Write 5 Done Sent Done false
664 1111 Acked Write 7 Done Sent Done false
665 1112 Acked Read 6 Done Sent Done false
666 1113 Acked Read 7 Done Sent Done false
667 1114 Acked Write 8 Done Sent Done false
668 1115 Acked Read 10 Done Sent Done false
669 1116 Acked Read 2 Done Sent Done false
670 1117 Acked Write 5 Done Sent Done false
671 1118 Acked Read 9 Done Sent Done false
672 1119 Acked Read 3 Done Sent Done false
673 1120 Acked Write 3 Done Sent Done false
674 1121 Acked Write 10 Done Sent Done false
675 1122 Acked Write 2 Done Sent Done false
676 1123 Acked Read 1 Done Sent Done false
677 1124 Acked Write 4 Done Sent Done false
678 1125 Acked Write 7 Done Sent Done false
679 1126 Acked Flush 0 Done Sent Done false
680 1127 Acked Flush 0 Done Sent Done false
681 1128 Acked Flush 0 Done Sent Done false
682 1129 Acked Write 1 Done Sent Done false
683 1130 Acked Read 4 Done Sent Done false
684 1131 Acked Write 2 Done Sent Done false
685 1132 Acked Write 7 Done Sent Done false
686 1133 Acked Read 8 Done Sent Done false
687 1134 Acked Read 6 Done Sent Done false
688 1135 Acked Write 1 Done Sent Done false
689 1136 Acked Read 10 Done Sent Done false
690 1137 Acked Read 8 Done Sent Done false
691 1138 Acked Write 10 Done Sent Done false
692 1139 Acked Read 3 Done Sent Done false
693 1140 Acked Read 3 Done Sent Done false
694 1141 Acked Write 5 Done Sent Done false
695 1142 Acked Read 3 Done Sent Done false
696 1143 Acked Read 1 Done Sent Done false
697 1144 Acked Write 4 Done Sent Done false
698 1145 Acked Flush 0 Done Sent Done false
699 1146 Acked Read 5 Done Sent Done false
700 1147 Acked Write 10 Done Sent Done false
701 1148 Acked Write 2 Done Sent Done false
702 1149 Acked Write 1 Done Sent Done false
703 1150 Acked Read 1 Done Sent Done false
704 1151 Acked Write 3 Done Sent Done false
705 1152 Acked Write 10 Done Sent Done false
706 1153 Acked Flush 0 Done Sent Done false
707 1154 Acked Write 4 Done Sent Done false
708 1155 Acked Read 3 Done Sent Done false
709 1156 Acked Write 10 Done Sent Done false
710 1157 Acked Write 4 Done Sent Done false
711 1158 Acked Read 3 Done Sent Done false
712 1159 Acked Read 6 Done Sent Done false
713 1160 Acked Read 9 Done Sent Done false
714 1161 Acked Read 9 Done Sent Done false
715 1162 Acked Read 6 Done Sent Done false
716 1163 Acked Write 6 Done Sent Done false
717 1164 Acked Read 8 Done Sent Done false
718 1165 Acked Write 2 Done Sent Done false
719 1166 Acked Read 1 Done Sent Done false
720 1167 Acked Flush 0 Done Sent Done false
721 1168 Acked Read 5 Done Sent Done false
722 1169 Acked Read 10 Done Sent Done false
723 1170 Acked Read 8 Done Sent Done false
724 1171 Acked Read 2 Done Sent Done false
725 1172 Acked Read 2 Done Sent Done false
726 1173 Acked Write 6 Done Sent Done false
727 1174 Acked Write 5 Done Sent Done false
728 1175 Acked Write 3 Done Sent Done false
729 1176 Acked Read 6 Done Sent Done false
730 1177 Acked Read 2 Done Sent Done false
731 1178 Acked Write 9 Done Sent Done false
732 1179 Acked Write 1 Done Sent Done false
733 1180 Acked Write 10 Done Sent Done false
734 1181 Acked Write 7 Done Sent Done false
735 1182 Acked Write 2 Done Sent Done false
736 1183 Acked Write 8 Done Sent Done false
737 1184 Acked Read 9 Done Sent Done false
738 1185 Acked Write 8 Done Sent Done false
739 1186 Acked Write 3 Done Sent Done false
740 1187 Acked Read 4 Done Sent Done false
741 1188 Acked Write 9 Done Sent Done false
742 1189 Acked Write 8 Done Sent Done false
743 1190 Acked Read 4 Done Sent Done false
744 1191 Acked Read 2 Done Sent Done false
745 1192 Acked Read 7 Done Sent Done false
746 1193 Acked Write 9 Done Sent Done false
747 1194 Acked Read 6 Done Sent Done false
748 1195 Acked Read 9 Done Sent Done false
749 1196 Acked Write 3 Done Sent Done false
750 1197 Acked Read 7 Done Sent Done false
751 1198 Acked Read 3 Done Sent Done false
752 1199 Acked Flush 0 Done Sent Done false
753 1200 Acked Write 7 Done Sent Done false
754 1201 Acked Flush 0 Done Sent Done false
755 1202 Acked Write 6 Done Sent Done false
756 1203 Acked Read 8 Done Sent Done false
757 1204 Acked Read 1 Done Sent Done false
758 1205 Acked Read 8 Done Sent Done false
759 1206 Acked Read 1 Done Sent Done false
760 1207 Acked Write 10 Done Sent Done false
761 1208 Acked Read 5 Done Sent Done false
762 1209 Acked Write 3 Done Sent Done false
763 1210 Acked Flush 0 Done Sent Done false
764 1211 Acked Read 6 Done Sent Done false
765 1212 Acked Read 6 Done Sent Done false
766 1213 Acked Read 3 Done Sent Done false
767 1214 Acked Read 7 Done Sent Done false
768 1215 Acked Write 9 Done Sent Done false
769 1216 Acked Write 2 Done Sent Done false
770 1217 Acked Flush 0 Done Sent Done false
771 1218 Acked Write 7 Done Sent Done false
772 1219 Acked Read 10 Done Sent Done false
773 1220 Acked Read 9 Done Sent Done false
774 1221 Acked Read 3 Done Sent Done false
775 1222 Acked Read 10 Done Sent Done false
776 1223 Acked Read 1 Done Sent Done false
777 1224 Acked Flush 0 Done Sent Done false
778 1225 Acked Read 3 Done Sent Done false
779 1226 Acked Read 2 Done Sent Done false
780 1227 Acked Write 8 Done Sent Done false
781 1228 Acked Flush 0 Done Sent Done false
782 1229 Acked Read 7 Done Sent Done false
783 1230 Acked Read 6 Done Sent Done false
784 1231 Acked Read 10 Done Sent Done false
785 1232 Acked Flush 0 Done Sent Done false
786 1233 Acked Flush 0 Done Sent Done false
787 1234 Acked Read 9 Done Sent Done false
788 1235 Acked Flush 0 Done Sent Done false
789 1236 Acked Flush 0 Done Sent Done false
790 1237 Acked Write 2 Done Sent Done false
791 1238 Acked Write 1 Done Sent Done false
792 1239 Acked Write 3 Done Sent Done false
793 1240 Acked Read 6 Done Sent Done false
794 1241 Acked Flush 0 Done Sent Done false
795 1242 Acked Write 2 Done Sent Done false
796 1243 Acked Flush 0 Done Sent Done false
797 1244 Acked Read 9 Done Sent Done false
798 1245 Acked Read 3 Done Sent Done false
799 1246 Acked Read 4 Done Sent Done false
800 1247 Acked Flush 0 Done Sent Done false
801 1248 Acked Write 8 Done Sent Done false
802 1249 Acked Read 10 Done Sent Done false
803 1250 Acked Read 4 Done Sent Done false
804 1251 Acked Read 6 Done Sent Done false
805 1252 Acked Read 10 Done Sent Done false
806 1253 Acked Read 2 Done Sent Done false
807 1254 Acked Write 4 Done Sent Done false
808 1255 Acked Flush 0 Done Sent Done false
809 1256 Acked Read 2 Done Sent Done false
810 1257 Acked Write 3 Done Sent Done false
811 1258 Acked Write 6 Done Sent Done false
812 1259 Acked Read 6 Done Sent Done false
813 1260 Acked Flush 0 Done Sent Done false
814 1261 Acked Read 1 Done Sent Done false
815 1262 Acked Read 2 Done Sent Done false
816 1263 Acked Read 1 Done Sent Done false
817 1264 Acked Write 10 Done Sent Done false
818 1265 Acked Read 6 Done Sent Done false
819 1266 Acked Read 10 Done Sent Done false
820 1267 Acked Write 10 Done Sent Done false
821 1268 Acked Read 8 Done Sent Done false
822 1269 Acked Read 7 Done Sent Done false
823 1270 Acked Write 1 Done Sent Done false
824 1271 Acked Write 7 Done Sent Done false
825 1272 Acked Write 3 Done Sent Done false
826 1273 Acked Flush 0 Done Sent Done false
827 1274 Acked Write 8 Done Sent Done false
828 1275 Acked Read 2 Done Sent Done false
829 1276 Acked Read 10 Done Sent Done false
830 1277 Acked Write 9 Done Sent Done false
831 1278 Acked Write 9 Done Sent Done false
832 1279 Acked Write 9 Done Sent Done false
833 1280 Acked Read 2 Done Sent Done false
834 1281 Acked Read 2 Done Sent Done false
835 1282 Acked Read 6 Done Sent Done false
836 1283 Acked Read 8 Done Sent Done false
837 1284 Acked Read 3 Done Sent Done false
838 1285 Acked Read 2 Done Sent Done false
839 1286 Acked Read 3 Done Sent Done false
840 1287 Acked Flush 0 Done Sent Done false
841 1288 Acked Write 6 Done Sent Done false
842 1289 Acked Write 10 Done Sent Done false
843 1290 Acked Read 9 Done Sent Done false
844 1291 Acked Write 6 Done Sent Done false
845 1292 Acked Write 2 Done Sent Done false
846 1293 Acked Read 8 Done Sent Done false
847 1294 Acked Read 9 Done Sent Done false
848 1295 Acked Read 1 Done Sent Done false
849 1296 Acked Read 3 Done Sent Done false
850 1297 Acked Read 5 Done Sent Done false
851 1298 Acked Read 1 Done Sent Done false
852 1299 Acked Read 6 Done Sent Done false
853 1300 Acked Read 8 Done Sent Done false
854 STATES DS:0 DS:1 DS:2 TOTAL
855 Sent 0 301 0 301
856 Done 301 0 301 602
857 Skipped 0 0 0 0
858 Error 0 0 0 0
859 Last Flush: 1287 0 1287
860 Downstairs last five completed:
861 Upstairs last five completed: 1300 1299 1298 1297 1296
862 CLIENT: Up:0 ds:301 act:2
863 [8820] got start action
864 [8820] I am going to restart
865 Starting downstairs at port 8820
866 Make output file at "/tmp/dsc/downstairs-8820.txt"
867 [8820][1] reports Starting
868 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1204)
869 [8820][1] reports Running
870 ----------------------------------------------------------------
871 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
872 States: Active Offline Active
873 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
874 1000 Acked Read 2 Done Sent Done false
875 1001 Acked Flush 0 Done Sent Done false
876 1002 Acked Write 1 Done Sent Done false
877 1003 Acked Read 8 Done Sent Done false
878 1004 Acked Flush 0 Done Sent Done false
879 1005 Acked Read 3 Done Sent Done false
880 1006 Acked Write 5 Done Sent Done false
881 1007 Acked Write 6 Done Sent Done false
882 1008 Acked Write 6 Done Sent Done false
883 1009 Acked Read 8 Done Sent Done false
884 1010 Acked Read 4 Done Sent Done false
885 1011 Acked Write 8 Done Sent Done false
886 1012 Acked Write 3 Done Sent Done false
887 1013 Acked Write 9 Done Sent Done false
888 1014 Acked Flush 0 Done Sent Done false
889 1015 Acked Read 4 Done Sent Done false
890 1016 Acked Write 4 Done Sent Done false
891 1017 Acked Write 3 Done Sent Done false
892 1018 Acked Read 8 Done Sent Done false
893 1019 Acked Read 6 Done Sent Done false
894 1020 Acked Write 7 Done Sent Done false
895 1021 Acked Write 2 Done Sent Done false
896 1022 Acked Read 1 Done Sent Done false
897 1023 Acked Write 8 Done Sent Done false
898 1024 Acked Read 6 Done Sent Done false
899 1025 Acked Read 6 Done Sent Done false
900 1026 Acked Write 1 Done Sent Done false
901 1027 Acked Write 5 Done Sent Done false
902 1028 Acked Read 3 Done Sent Done false
903 1029 Acked Read 3 Done Sent Done false
904 1030 Acked Write 5 Done Sent Done false
905 1031 Acked Write 2 Done Sent Done false
906 1032 Acked Write 5 Done Sent Done false
907 1033 Acked Write 2 Done Sent Done false
908 1034 Acked Write 10 Done Sent Done false
909 1035 Acked Write 6 Done Sent Done false
910 1036 Acked Read 4 Done Sent Done false
911 1037 Acked Read 6 Done Sent Done false
912 1038 Acked Read 6 Done Sent Done false
913 1039 Acked Read 1 Done Sent Done false
914 1040 Acked Read 10 Done Sent Done false
915 1041 Acked Write 4 Done Sent Done false
916 1042 Acked Read 9 Done Sent Done false
917 1043 Acked Write 6 Done Sent Done false
918 1044 Acked Write 2 Done Sent Done false
919 1045 Acked Read 1 Done Sent Done false
920 1046 Acked Write 3 Done Sent Done false
921 1047 Acked Read 3 Done Sent Done false
922 1048 Acked Write 2 Done Sent Done false
923 1049 Acked Read 3 Done Sent Done false
924 1050 Acked Write 6 Done Sent Done false
925 1051 Acked Read 3 Done Sent Done false
926 1052 Acked Write 5 Done Sent Done false
927 1053 Acked Read 4 Done Sent Done false
928 1054 Acked Flush 0 Done Sent Done false
929 1055 Acked Read 7 Done Sent Done false
930 1056 Acked Write 7 Done Sent Done false
931 1057 Acked Read 5 Done Sent Done false
932 1058 Acked Write 5 Done Sent Done false
933 1059 Acked Write 2 Done Sent Done false
934 1060 Acked Write 1 Done Sent Done false
935 1061 Acked Read 3 Done Sent Done false
936 1062 Acked Read 6 Done Sent Done false
937 1063 Acked Read 8 Done Sent Done false
938 1064 Acked Read 8 Done Sent Done false
939 1065 Acked Write 9 Done Sent Done false
940 1066 Acked Flush 0 Done Sent Done false
941 1067 Acked Read 2 Done Sent Done false
942 1068 Acked Read 4 Done Sent Done false
943 1069 Acked Read 8 Done Sent Done false
944 1070 Acked Write 7 Done Sent Done false
945 1071 Acked Read 2 Done Sent Done false
946 1072 Acked Flush 0 Done Sent Done false
947 1073 Acked Read 5 Done Sent Done false
948 1074 Acked Read 5 Done Sent Done false
949 1075 Acked Write 1 Done Sent Done false
950 1076 Acked Flush 0 Done Sent Done false
951 1077 Acked Read 6 Done Sent Done false
952 1078 Acked Read 5 Done Sent Done false
953 1079 Acked Read 10 Done Sent Done false
954 1080 Acked Read 9 Done Sent Done false
955 1081 Acked Write 2 Done Sent Done false
956 1082 Acked Write 10 Done Sent Done false
957 1083 Acked Read 4 Done Sent Done false
958 1084 Acked Write 4 Done Sent Done false
959 1085 Acked Write 3 Done Sent Done false
960 1086 Acked Read 8 Done Sent Done false
961 1087 Acked Read 9 Done Sent Done false
962 1088 Acked Read 6 Done Sent Done false
963 1089 Acked Read 1 Done Sent Done false
964 1090 Acked Read 10 Done Sent Done false
965 1091 Acked Flush 0 Done Sent Done false
966 1092 Acked Read 6 Done Sent Done false
967 1093 Acked Write 5 Done Sent Done false
968 1094 Acked Read 1 Done Sent Done false
969 1095 Acked Flush 0 Done Sent Done false
970 1096 Acked Read 10 Done Sent Done false
971 1097 Acked Write 3 Done Sent Done false
972 1098 Acked Write 4 Done Sent Done false
973 1099 Acked Read 10 Done Sent Done false
974 1100 Acked Read 2 Done Sent Done false
975 1101 Acked Write 8 Done Sent Done false
976 1102 Acked Read 1 Done Sent Done false
977 1103 Acked Write 8 Done Sent Done false
978 1104 Acked Read 10 Done Sent Done false
979 1105 Acked Read 9 Done Sent Done false
980 1106 Acked Read 1 Done Sent Done false
981 1107 Acked Read 7 Done Sent Done false
982 1108 Acked Write 3 Done Sent Done false
983 1109 Acked Write 6 Done Sent Done false
984 1110 Acked Write 5 Done Sent Done false
985 1111 Acked Write 7 Done Sent Done false
986 1112 Acked Read 6 Done Sent Done false
987 1113 Acked Read 7 Done Sent Done false
988 1114 Acked Write 8 Done Sent Done false
989 1115 Acked Read 10 Done Sent Done false
990 1116 Acked Read 2 Done Sent Done false
991 1117 Acked Write 5 Done Sent Done false
992 1118 Acked Read 9 Done Sent Done false
993 1119 Acked Read 3 Done Sent Done false
994 1120 Acked Write 3 Done Sent Done false
995 1121 Acked Write 10 Done Sent Done false
996 1122 Acked Write 2 Done Sent Done false
997 1123 Acked Read 1 Done Sent Done false
998 1124 Acked Write 4 Done Sent Done false
999 1125 Acked Write 7 Done Sent Done false
1000 1126 Acked Flush 0 Done Sent Done false
1001 1127 Acked Flush 0 Done Sent Done false
1002 1128 Acked Flush 0 Done Sent Done false
1003 1129 Acked Write 1 Done Sent Done false
1004 1130 Acked Read 4 Done Sent Done false
1005 1131 Acked Write 2 Done Sent Done false
1006 1132 Acked Write 7 Done Sent Done false
1007 1133 Acked Read 8 Done Sent Done false
1008 1134 Acked Read 6 Done Sent Done false
1009 1135 Acked Write 1 Done Sent Done false
1010 1136 Acked Read 10 Done Sent Done false
1011 1137 Acked Read 8 Done Sent Done false
1012 1138 Acked Write 10 Done Sent Done false
1013 1139 Acked Read 3 Done Sent Done false
1014 1140 Acked Read 3 Done Sent Done false
1015 1141 Acked Write 5 Done Sent Done false
1016 1142 Acked Read 3 Done Sent Done false
1017 1143 Acked Read 1 Done Sent Done false
1018 1144 Acked Write 4 Done Sent Done false
1019 1145 Acked Flush 0 Done Sent Done false
1020 1146 Acked Read 5 Done Sent Done false
1021 1147 Acked Write 10 Done Sent Done false
1022 1148 Acked Write 2 Done Sent Done false
1023 1149 Acked Write 1 Done Sent Done false
1024 1150 Acked Read 1 Done Sent Done false
1025 1151 Acked Write 3 Done Sent Done false
1026 1152 Acked Write 10 Done Sent Done false
1027 1153 Acked Flush 0 Done Sent Done false
1028 1154 Acked Write 4 Done Sent Done false
1029 1155 Acked Read 3 Done Sent Done false
1030 1156 Acked Write 10 Done Sent Done false
1031 1157 Acked Write 4 Done Sent Done false
1032 1158 Acked Read 3 Done Sent Done false
1033 1159 Acked Read 6 Done Sent Done false
1034 1160 Acked Read 9 Done Sent Done false
1035 1161 Acked Read 9 Done Sent Done false
1036 1162 Acked Read 6 Done Sent Done false
1037 1163 Acked Write 6 Done Sent Done false
1038 1164 Acked Read 8 Done Sent Done false
1039 1165 Acked Write 2 Done Sent Done false
1040 1166 Acked Read 1 Done Sent Done false
1041 1167 Acked Flush 0 Done Sent Done false
1042 1168 Acked Read 5 Done Sent Done false
1043 1169 Acked Read 10 Done Sent Done false
1044 1170 Acked Read 8 Done Sent Done false
1045 1171 Acked Read 2 Done Sent Done false
1046 1172 Acked Read 2 Done Sent Done false
1047 1173 Acked Write 6 Done Sent Done false
1048 1174 Acked Write 5 Done Sent Done false
1049 1175 Acked Write 3 Done Sent Done false
1050 1176 Acked Read 6 Done Sent Done false
1051 1177 Acked Read 2 Done Sent Done false
1052 1178 Acked Write 9 Done Sent Done false
1053 1179 Acked Write 1 Done Sent Done false
1054 1180 Acked Write 10 Done Sent Done false
1055 1181 Acked Write 7 Done Sent Done false
1056 1182 Acked Write 2 Done Sent Done false
1057 1183 Acked Write 8 Done Sent Done false
1058 1184 Acked Read 9 Done Sent Done false
1059 1185 Acked Write 8 Done Sent Done false
1060 1186 Acked Write 3 Done Sent Done false
1061 1187 Acked Read 4 Done Sent Done false
1062 1188 Acked Write 9 Done Sent Done false
1063 1189 Acked Write 8 Done Sent Done false
1064 1190 Acked Read 4 Done Sent Done false
1065 1191 Acked Read 2 Done Sent Done false
1066 1192 Acked Read 7 Done Sent Done false
1067 1193 Acked Write 9 Done Sent Done false
1068 1194 Acked Read 6 Done Sent Done false
1069 1195 Acked Read 9 Done Sent Done false
1070 1196 Acked Write 3 Done Sent Done false
1071 1197 Acked Read 7 Done Sent Done false
1072 1198 Acked Read 3 Done Sent Done false
1073 1199 Acked Flush 0 Done Sent Done false
1074 1200 Acked Write 7 Done Sent Done false
1075 1201 Acked Flush 0 Done Sent Done false
1076 1202 Acked Write 6 Done Sent Done false
1077 1203 Acked Read 8 Done Sent Done false
1078 1204 Acked Read 1 Done Sent Done false
1079 1205 Acked Read 8 Done Sent Done false
1080 1206 Acked Read 1 Done Sent Done false
1081 1207 Acked Write 10 Done Sent Done false
1082 1208 Acked Read 5 Done Sent Done false
1083 1209 Acked Write 3 Done Sent Done false
1084 1210 Acked Flush 0 Done Sent Done false
1085 1211 Acked Read 6 Done Sent Done false
1086 1212 Acked Read 6 Done Sent Done false
1087 1213 Acked Read 3 Done Sent Done false
1088 1214 Acked Read 7 Done Sent Done false
1089 1215 Acked Write 9 Done Sent Done false
1090 1216 Acked Write 2 Done Sent Done false
1091 1217 Acked Flush 0 Done Sent Done false
1092 1218 Acked Write 7 Done Sent Done false
1093 1219 Acked Read 10 Done Sent Done false
1094 1220 Acked Read 9 Done Sent Done false
1095 1221 Acked Read 3 Done Sent Done false
1096 1222 Acked Read 10 Done Sent Done false
1097 1223 Acked Read 1 Done Sent Done false
1098 1224 Acked Flush 0 Done Sent Done false
1099 1225 Acked Read 3 Done Sent Done false
1100 1226 Acked Read 2 Done Sent Done false
1101 1227 Acked Write 8 Done Sent Done false
1102 1228 Acked Flush 0 Done Sent Done false
1103 1229 Acked Read 7 Done Sent Done false
1104 1230 Acked Read 6 Done Sent Done false
1105 1231 Acked Read 10 Done Sent Done false
1106 1232 Acked Flush 0 Done Sent Done false
1107 1233 Acked Flush 0 Done Sent Done false
1108 1234 Acked Read 9 Done Sent Done false
1109 1235 Acked Flush 0 Done Sent Done false
1110 1236 Acked Flush 0 Done Sent Done false
1111 1237 Acked Write 2 Done Sent Done false
1112 1238 Acked Write 1 Done Sent Done false
1113 1239 Acked Write 3 Done Sent Done false
1114 1240 Acked Read 6 Done Sent Done false
1115 1241 Acked Flush 0 Done Sent Done false
1116 1242 Acked Write 2 Done Sent Done false
1117 1243 Acked Flush 0 Done Sent Done false
1118 1244 Acked Read 9 Done Sent Done false
1119 1245 Acked Read 3 Done Sent Done false
1120 1246 Acked Read 4 Done Sent Done false
1121 1247 Acked Flush 0 Done Sent Done false
1122 1248 Acked Write 8 Done Sent Done false
1123 1249 Acked Read 10 Done Sent Done false
1124 1250 Acked Read 4 Done Sent Done false
1125 1251 Acked Read 6 Done Sent Done false
1126 1252 Acked Read 10 Done Sent Done false
1127 1253 Acked Read 2 Done Sent Done false
1128 1254 Acked Write 4 Done Sent Done false
1129 1255 Acked Flush 0 Done Sent Done false
1130 1256 Acked Read 2 Done Sent Done false
1131 1257 Acked Write 3 Done Sent Done false
1132 1258 Acked Write 6 Done Sent Done false
1133 1259 Acked Read 6 Done Sent Done false
1134 1260 Acked Flush 0 Done Sent Done false
1135 1261 Acked Read 1 Done Sent Done false
1136 1262 Acked Read 2 Done Sent Done false
1137 1263 Acked Read 1 Done Sent Done false
1138 1264 Acked Write 10 Done Sent Done false
1139 1265 Acked Read 6 Done Sent Done false
1140 1266 Acked Read 10 Done Sent Done false
1141 1267 Acked Write 10 Done Sent Done false
1142 1268 Acked Read 8 Done Sent Done false
1143 1269 Acked Read 7 Done Sent Done false
1144 1270 Acked Write 1 Done Sent Done false
1145 1271 Acked Write 7 Done Sent Done false
1146 1272 Acked Write 3 Done Sent Done false
1147 1273 Acked Flush 0 Done Sent Done false
1148 1274 Acked Write 8 Done Sent Done false
1149 1275 Acked Read 2 Done Sent Done false
1150 1276 Acked Read 10 Done Sent Done false
1151 1277 Acked Write 9 Done Sent Done false
1152 1278 Acked Write 9 Done Sent Done false
1153 1279 Acked Write 9 Done Sent Done false
1154 1280 Acked Read 2 Done Sent Done false
1155 1281 Acked Read 2 Done Sent Done false
1156 1282 Acked Read 6 Done Sent Done false
1157 1283 Acked Read 8 Done Sent Done false
1158 1284 Acked Read 3 Done Sent Done false
1159 1285 Acked Read 2 Done Sent Done false
1160 1286 Acked Read 3 Done Sent Done false
1161 1287 Acked Flush 0 Done Sent Done false
1162 1288 Acked Write 6 Done Sent Done false
1163 1289 Acked Write 10 Done Sent Done false
1164 1290 Acked Read 9 Done Sent Done false
1165 1291 Acked Write 6 Done Sent Done false
1166 1292 Acked Write 2 Done Sent Done false
1167 1293 Acked Read 8 Done Sent Done false
1168 1294 Acked Read 9 Done Sent Done false
1169 1295 Acked Read 1 Done Sent Done false
1170 1296 Acked Read 3 Done Sent Done false
1171 1297 Acked Read 5 Done Sent Done false
1172 1298 Acked Read 1 Done Sent Done false
1173 1299 Acked Read 6 Done Sent Done false
1174 1300 Acked Read 8 Done Sent Done false
1175 1301 Acked Flush 0 Done Sent Done false
1176 STATES DS:0 DS:1 DS:2 TOTAL
1177 Sent 0 302 0 302
1178 Done 302 0 302 604
1179 Skipped 0 0 0 0
1180 Error 0 0 0 0
1181 Last Flush: 1301 0 1301
1182 Downstairs last five completed:
1183 Upstairs last five completed: 1301 1300 1299 1298 1297
1184 CLIENT: Up:0 ds:302 act:2
11852024-11-26T22:04:46.409ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
11862024-11-26T22:04:46.409ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:38864) = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
11872024-11-26T22:04:46.410ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
11882024-11-26T22:04:46.411ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
11892024-11-26T22:04:46.411ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
11902024-11-26T22:04:46.411ZINFOcrucible: send last flush ID to this DS: 0 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
11912024-11-26T22:04:46.411ZINFOcrucible: Replied this last flush ID: 0 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
11922024-11-26T22:04:46.411ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
11932024-11-26T22:04:46.411ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
11942024-11-26T22:04:46.411ZINFOcrucible: [1] client re-new 302 jobs since flush 0 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
11952024-11-26T22:04:46.411ZINFOcrucible: [1] Replayed 302 jobs since flush: 0 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
1196 ----------------------------------------------------------------
1197 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
1198 Downstairs last five completed: 1301 1300 1299 1298 1297
1199 Upstairs last five completed: 1301 1300 1299 1298 1297
1200 CLIENT: Up:0 ds:0 act:3
1201 Replay: All jobs finished
1202 Main task has work to do, go find it
1203 got dsc Stop(1)
1204 stop 1
1205 Nov 26 22:04:48.650 INFO request completed, latency_us: 322, response_code[8820] Got stop action so:false kr:false
1206 : 204, uri: /stop/cid/1, method: POST, req_id: 4ad6aed7-8a91-46bc-b6b9-da947e710432, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
1207 Nov 26 22:04:48.650 INFO request completed, latency_us: 107, response_code:{"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2024-11-26T22:04:48.650724194Z","hostname":"w-01JDN7ZQ17ABQG15KHKN3F2D26","pid":1203,"":"io task","client":"1","":"downstairs","session_id":"86a36c17-3718-484c-87fd-2b8a1dc2ab18"}
1208 200, uri: /state/cid/1, method: GET{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2024-11-26T22:04:48.650824461Z","hostname":"w-01JDN7ZQ17ABQG15KHKN3F2D26","pid":1203,"":"io task","client":"1","":"downstairs","session_id":"86a36c17-3718-484c-87fd-2b8a1dc2ab18"}
1209 , {"msg":"downstairs task for 1 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2024-11-26T22:04:48.650862627Z","hostname":"w-01JDN7ZQ17ABQG15KHKN3F2D26","pid":1203,"session_id":"86a36c17-3718-484c-87fd-2b8a1dc2ab18"}
1210 req_id: 5659cb3b-bc8c-4654-adf0-c6085bdf988b, remote_addr: 127.0.0.1:36829, local_addr: 127.{"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-26T22:04:48.650894983Z","hostname":"w-01JDN7ZQ17ABQG15KHKN3F2D26","pid":1203,"client":"1","":"downstairs","session_id":"86a36c17-3718-484c-87fd-2b8a1dc2ab18"}
1211 0{"msg":"[1] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-26T22:04:48.650929139Z","hostname":"w-01JDN7ZQ17ABQG15KHKN3F2D26","pid":1203,"client":"1","":"downstairs","session_id":"86a36c17-3718-484c-87fd-2b8a1dc2ab18"}
1212 .0.1:9998
12132024-11-26T22:04:48.651ZINFOcrucible: client task is exiting = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
1214 [8820] Exited with: ExitStatus(unix_wait_status(9))
1215 [8820][1] reports Exit
1216 Nov 26 22:04:52.653 INFO request completed, latency_us: 220, response_code: 200, uri: /state/cid/1, method: GET, req_id: 4cbce2bd-b09d-4b82-9a36-f9d480c1274c, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
1217 001/300 Write block 2068 len 28672 data: 2 2 2 2 2 2 2
1218 002/300 Read block 360 len 12288
1219 003/300 Read block 1966 len 36864
1220 004/300 Read block 50 len 12288
1221 005/300 Write block 61 len 16384 data: 3 2 2 2
1222 006/300 Write block 432 len 20480 data: 2 2 2 2 2
1223 007/300 Write block 3315 len 8192 data: 2 2
1224 008/300 Read block 4950 len 32768
1225 009/300 Read block 136 len 8192
1226 010/300 Read block 3856 len 4096
1227 011/300 Write block 1999 len 36864 data: 2 2 2 2 2 2 2 2 2
1228 012/300 Write block 1477 len 32768 data: 2 2 2 2 2 2 2 2
1229 013/300 Read block 4750 len 20480
1230 014/300 Read block 400 len 20480
1231 015/300 Read block 921 len 16384
1232 016/300 Read block 1390 len 36864
1233 017/300 Flush
1234 018/300 Read block 4063 len 20480
1235 019/300 Read block 3131 len 12288
1236 020/300 Write block 874 len 28672 data: 2 2 2 2 2 2 2
1237 021/300 Read block 4628 len 4096
1238 022/300 Read block 331 len 8192
1239 023/300 Write block 1168 len 16384 data: 2 2 2 2
1240 024/300 Write block 1924 len 28672 data: 2 2 2 2 2 2 2
1241 025/300 Read block 4519 len 36864
1242 026/300 Read block 4781 len 28672
1243 027/300 Flush
1244 028/300 Read block 3596 len 12288
1245 029/300 Read block 2441 len 28672
1246 030/300 Read block 2963 len 20480
1247 031/300 Write block 1073 len 32768 data: 2 2 3 3 3 3 3 3
1248 032/300 Read block 1326 len 28672
1249 033/300 Read block 2876 len 16384
1250 034/300 Flush
1251 035/300 Write block 1277 len 36864 data: 2 2 2 2 2 2 2 2 2
1252 036/300 Read block 4654 len 20480
1253 037/300 Write block 4703 len 8192 data: 2 2
1254 038/300 Read block 1322 len 36864
1255 039/300 Read block 2803 len 36864
1256 040/300 Read block 3603 len 28672
1257 041/300 Read block 3814 len 36864
1258 042/300 Write block 1179 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1259 043/300 Flush
1260 044/300 Write block 4454 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1261 045/300 Flush
1262 046/300 Write block 2528 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1263 047/300 Flush
1264 048/300 Write block 2024 len 20480 data: 2 2 2 2 2
1265 049/300 Write block 4230 len 20480 data: 2 2 2 2 2
1266 050/300 Read block 1516 len 36864
1267 051/300 Read block 158 len 16384
1268 052/300 Write block 2911 len 32768 data: 2 2 2 2 2 2 2 2
1269 053/300 Read block 895 len 4096
1270 054/300 Write block 3160 len 36864 data: 2 2 2 2 2 2 2 2 2
1271 055/300 Read block 4804 len 12288
1272 056/300 Write block 1955 len 8192 data: 2 2
1273 057/300 Read block 1434 len 20480
1274 058/300 Read block 1785 len 24576
1275 059/300 Write block 4436 len 32768 data: 3 3 3 3 2 2 2 2
1276 060/300 Read block 2483 len 36864
1277 061/300 Write block 3560 len 8192 data: 2 2
1278 062/300 Read block 770 len 24576
1279 063/300 Write block 4860 len 16384 data: 3 3 3 3
1280 064/300 Write block 3907 len 8192 data: 2 2
1281 065/300 Write block 4802 len 12288 data: 3 3 3
1282 066/300 Write block 3005 len 28672 data: 2 2 2 2 2 2 2
1283 067/300 Flush
1284 068/300 Read block 510 len 16384
1285 069/300 Write block 2540 len 12288 data: 2 2 2
1286 070/300 Read block 991 len 20480
1287 071/300 Write block 3793 len 16384 data: 2 2 2 3
1288 072/300 Read block 2369 len 16384
1289 073/300 Read block 401 len 8192
1290 074/300 Write block 0 len 36864 data: 2 2 2 2 2 2 2 2 2
1291 075/300 Read block 356 len 12288
1292 076/300 Read block 4541 len 8192
1293 077/300 Read block 578 len 24576
1294 078/300 Write block 261 len 36864 data: 2 2 2 2 2 2 2 2 2
1295 079/300 Write block 1117 len 28672 data: 2 2 2 2 2 2 2
1296 080/300 Write block 234 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1297 081/300 Flush
1298 082/300 Write block 233 len 4096 data: 2
1299 083/300 Read block 2346 len 16384
1300 084/300 Read block 3044 len 20480
1301 085/300 Flush
1302 086/300 Write block 4236 len 24576 data: 2 2 2 2 2 2
1303 087/300 Read block 3740 len 32768
1304 088/300 Write block 3034 len 12288 data: 2 2 2
1305 089/300 Write block 2183 len 28672 data: 2 2 2 2 2 2 2
1306 090/300 Write block 4393 len 8192 data: 2 2
1307 091/300 Write block 4688 len 32768 data: 2 2 2 2 2 2 2 2
1308 092/300 Read block 4677 len 8192
1309 093/300 Read block 2851 len 36864
1310 094/300 Read block 10 len 16384
1311 095/300 Read block 4222 len 20480
1312 096/300 Read block 3870 len 24576
1313 097/300 Read block 3755 len 4096
1314 098/300 Read block 1763 len 28672
1315 099/300 Read block 890 len 24576
1316 100/300 Read block 4915 len 36864
1317 101/300 Read block 2957 len 16384
1318 102/300 Write block 2921 len 12288 data: 2 2 2
1319 103/300 Write block 1777 len 16384 data: 2 2 3 3
1320 104/300 Read block 3639 len 20480
1321 105/300 Read block 3437 len 20480
1322 106/300 Read block 2485 len 36864
1323 107/300 Read block 757 len 16384
1324 108/300 Read block 2757 len 36864
1325 109/300 Write block 3810 len 28672 data: 2 2 2 2 2 2 2
1326 110/300 Read block 511 len 32768
1327 111/300 Read block 4179 len 40960
1328 112/300 Read block 3719 len 16384
1329 113/300 Write block 2985 len 24576 data: 2 2 2 2 2 2
1330 114/300 Write block 3672 len 24576 data: 2 2 2 2 2 2
1331 115/300 Read block 1977 len 8192
1332 116/300 Write block 3653 len 4096 data: 3
1333 117/300 Read block 250 len 28672
1334 118/300 Write block 2556 len 40960 data: 3 3 3 3 3 3 3 3 3 3
1335 119/300 Read block 670 len 16384
1336 120/300 Read block 3757 len 40960
1337 121/300 Write block 3010 len 36864 data: 3 3 2 2 2 2 2 2 2
1338 122/300 Flush
1339 123/300 Flush
1340 124/300 Read block 390 len 28672
1341 125/300 Read block 3200 len 36864
1342 126/300 Write block 1624 len 8192 data: 2 2
1343 127/300 Read block 210 len 40960
1344 128/300 Write block 2224 len 36864 data: 2 2 2 2 2 2 2 2 2
1345 129/300 Read block 1724 len 12288
1346 130/300 Read block 2540 len 8192
1347 131/300 Read block 1903 len 4096
1348 132/300 Write block 3395 len 32768 data: 4 3 3 3 2 2 2 2
1349 133/300 Flush
1350 134/300 Write block 4059 len 40960 data: 2 2 2 2 2 2 2 2 2 3
1351 135/300 Read block 2344 len 32768
1352 136/300 Read block 859 len 36864
1353 137/300 Read block 2988 len 24576
1354 138/300 Read block 1755 len 4096
1355 139/300 Read block 4198 len 40960
1356 140/300 Read block 1913 len 28672
1357 141/300 Write block 4465 len 24576 data: 2 2 2 2 2 2
1358 142/300 Write block 1054 len 28672 data: 3 3 3 2 2 2 2
1359 143/300 Write block 3726 len 36864 data: 2 2 2 2 2 2 2 2 2
1360 144/300 Read block 4842 len 16384
1361 145/300 Read block 4489 len 28672
1362 146/300 Read block 4955 len 40960
1363 147/300 Write block 2246 len 12288 data: 3 3 3
1364 148/300 Read block 658 len 8192
1365 149/300 Write block 3991 len 12288 data: 2 2 2
1366 150/300 Read block 4064 len 16384
1367 151/300 Read block 537 len 12288
1368 152/300 Flush
1369 153/300 Write block 4043 len 20480 data: 2 2 2 2 2
1370 154/300 Write block 4002 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1371 155/300 Read block 655 len 8192
1372 156/300 Read block 249 len 12288
1373 157/300 Read block 2555 len 36864
1374 158/300 Read block 602 len 32768
1375 159/300 Write block 4598 len 32768 data: 2 2 2 2 2 2 2 2
1376 160/300 Read block 647 len 8192
1377 161/300 Write block 1516 len 4096 data: 2
1378 162/300 Write block 1886 len 20480 data: 2 2 2 2 2
1379 163/300 Write block 4343 len 8192 data: 2 2
1380 164/300 Write block 3226 len 36864 data: 2 2 2 2 2 2 2 2 2
1381 165/300 Read block 4885 len 32768
1382 166/300 Read block 4691 len 24576
1383 167/300 Write block 2709 len 4096 data: 3
1384 168/300 Read block 3691 len 32768
1385 169/300 Read block 3509 len 12288
1386 170/300 Flush
1387 171/300 Write block 2433 len 16384 data: 2 2 2 2
1388 172/300 Read block 1928 len 4096
1389 173/300 Write block 3373 len 16384 data: 2 2 2 2
1390 174/300 Flush
1391 175/300 Read block 2355 len 4096
1392 176/300 Read block 2130 len 40960
1393 177/300 Write block 407 len 32768 data: 2 2 2 2 2 2 2 2
1394 178/300 Write block 3344 len 20480 data: 2 2 2 2 2
1395 179/300 Read block 4207 len 8192
1396 180/300 Read block 4281 len 28672
1397 181/300 Read block 4728 len 12288
1398 182/300 Read block 855 len 36864
1399 183/300 Write block 1472 len 32768 data: 2 2 2 2 2 3 3 3
1400 184/300 Read block 658 len 32768
1401 185/300 Write block 1396 len 12288 data: 2 2 2
1402 186/300 Write block 1418 len 4096 data: 2
1403 187/300 Write block 4448 len 4096 data: 2
1404 188/300 Write block 2734 len 16384 data: 2 2 2 2
1405 189/300 Read block 3913 len 36864
1406 190/300 Read block 2076 len 16384
1407 191/300 Read block 2368 len 36864
1408 192/300 Read block 2276 len 40960
1409 193/300 Read block 4720 len 40960
1410 194/300 Read block 1136 len 36864
1411 195/300 Read block 2296 len 32768
1412 196/300 Read block 4178 len 28672
1413 197/300 Read block 345 len 24576
1414 198/300 Read block 2558 len 8192
1415 199/300 Write block 2738 len 16384 data: 2 2 2 2
1416 200/300 Read block 1902 len 12288
1417 201/300 Read block 2414 len 16384
1418 202/300 Write block 2467 len 32768 data: 2 2 2 2 2 2 2 2
1419 203/300 Read block 2081 len 12288
1420 204/300 Write block 2173 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1421 205/300 Write block 1761 len 12288 data: 2 2 2
1422 206/300 Write block 3924 len 12288 data: 2 2 2
1423 207/300 Write block 3551 len 16384 data: 2 2 2 2
1424 208/300 Read block 2316 len 8192
1425 209/300 Write block 3180 len 12288 data: 3 3 3
1426 210/300 Write block 2378 len 12288 data: 2 2 2
1427 211/300 Read block 3517 len 36864
1428 212/300 Read block 1391 len 36864
1429 213/300 Write block 983 len 12288 data: 2 2 3
1430 214/300 Write block 3707 len 20480 data: 2 2 2 2 2
1431 215/300 Write block 1350 len 28672 data: 2 2 2 2 2 2 2
1432 216/300 Read block 4306 len 16384
1433 217/300 Flush
1434 218/300 Read block 3021 len 28672
1435 219/300 Read block 1253 len 40960
1436 220/300 Read block 4207 len 20480
1437 221/300 Write block 1414 len 24576 data: 2 2 2 2 3 2
1438 222/300 Write block 3592 len 12288 data: 3 4 4
1439 223/300 Read block 1436 len 4096
1440 224/300 Flush
1441 225/300 Read block 1126 len 24576
1442 226/300 Read block 4051 len 4096
1443 227/300 Read block 3627 len 16384
1444 228/300 Write block 2087 len 16384 data: 3 2 2 2
1445 229/300 Read block 235 len 24576
1446 230/300 Read block 3255 len 4096
1447 231/300 Read block 703 len 36864
1448 232/300 Write block 479 len 20480 data: 2 2 2 2 2
1449 233/300 Read block 1419 len 24576
1450 234/300 Read block 2134 len 36864
1451 235/300 Read block 2178 len 32768
1452 236/300 Write block 3540 len 28672 data: 2 2 2 2 2 2 2
1453 237/300 Write block 2034 len 28672 data: 2 2 2 2 2 2 2
1454 238/300 Write block 4937 len 12288 data: 2 2 2
1455 239/300 Read block 512 len 36864
1456 240/300 Read block 871 len 4096
1457 241/300 Read block 4297 len 28672
1458 242/300 Flush
1459 243/300 Read block 3772 len 4096
1460 244/300 Write block 575 len 4096 data: 3
1461 245/300 Write block 3066 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1462 246/300 Flush
1463 247/300 Write block 3715 len 24576 data: 2 2 2 2 2 2
1464 248/300 Flush
1465 249/300 Write block 1550 len 28672 data: 2 2 2 2 2 2 2
1466 250/300 Read block 3577 len 4096
1467 251/300 Write block 3272 len 20480 data: 2 2 2 2 2
1468 252/300 Flush
1469 253/300 Read block 1086 len 24576
1470 254/300 Write block 3749 len 16384 data: 2 2 2 2
1471 255/300 Read block 187 len 16384
1472 256/300 Read block 1941 len 24576
1473 257/300 Read block 3038 len 16384
1474 258/300 Write block 4808 len 12288 data: 2 2 2
1475 259/300 Read block 4113 len 24576
1476 260/300 Read block 865 len 16384
1477 261/300 Read block 3656 len 12288
1478 262/300 Read block 4868 len 32768
1479 263/300 Write block 2611 len 4096 data: 2
1480 264/300 Write block 4977 len 32768 data: 2 2 2 2 2 2 2 2
1481 265/300 Write block 1528 len 36864 data: 2 2 2 2 2 3 3 3 3
1482 266/300 Read block 4389 len 24576
1483 267/300 Write block 799 len 4096 data: 2
1484 268/300 Read block 2120 len 8192
1485 269/300 Write block 929 len 8192 data: 2 2
1486 270/300 Write block 2296 len 32768 data: 2 2 2 2 2 2 2 2
1487 271/300 Write block 2790 len 36864 data: 2 2 2 2 2 2 2 2 2
1488 272/300 Read block 4608 len 12288
1489 273/300 Read block 3056 len 32768
1490 274/300 Read block 4013 len 40960
1491 275/300 Read block 1971 len 20480
1492 276/300 Read block 44 len 24576
1493 277/300 Write block 1644 len 24576 data: 2 2 2 2 2 3
1494 278/300 Read block 3493 len 36864
1495 279/300 Write block 4965 len 24576 data: 2 2 2 2 2 2
1496 280/300 Read block 1261 len 36864
1497 281/300 Flush
1498 282/300 Read block 2830 len 32768
1499 283/300 Read block 2240 len 8192
1500 284/300 Write block 2630 len 24576 data: 2 2 2 2 2 2
1501 285/300 Read block 1034 len 36864
1502 286/300 Read block 3262 len 24576
1503 287/300 Write block 509 len 12288 data: 2 2 2
1504 288/300 Write block 3966 len 20480 data: 2 2 2 2 2
1505 289/300 Flush
1506 290/300 Write block 3944 len 4096 data: 2
1507 291/300 Read block 4250 len 40960
1508 292/300 Write block 180 len 24576 data: 2 2 2 2 2 2
1509 293/300 Read block 116 len 32768
1510 294/300 Read block 2238 len 40960
1511 295/300 Read block 815 len 24576
1512 296/300 Read block 2551 len 16384
1513 297/300 Write block 4535 len 12288 data: 2 2 3
1514 298/300 Write block 2230 len 8192 data: 3 3
1515 299/300 Write block 2441 len 20480 data: 2 2 2 2 2
1516 300/300 Read block 1113 len 12288
1517 301/300 Read block 2484 len 4096
1518 Main task has work to do, go find it
1519 got dsc Start(1)
1520 start 1
1521 Nov 26 22:04:52.897 INFO request completed, latency_us: 193, response_code: 204, uri: /start/cid/1, method: POST, req_id: 17cd64de-b036-4ec1-a84f-da64c97c543a, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
1522 Replay: started 1, returned:Ok(())
1523 ----------------------------------------------------------------
1524 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
1525 States: Active Offline Active
1526 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
1527 1302 Acked Write 7 Done Sent Done false
1528 1303 Acked Read 3 Done Sent Done false
1529 1304 Acked Read 9 Done Sent Done false
1530 1305 Acked Read 3 Done Sent Done false
1531 1306 Acked Write 4 Done Sent Done false
1532 1307 Acked Write 5 Done Sent Done false
1533 1308 Acked Write 2 Done Sent Done false
1534 1309 Acked Read 8 Done Sent Done false
1535 1310 Acked Read 2 Done Sent Done false
1536 1311 Acked Read 1 Done Sent Done false
1537 1312 Acked Write 9 Done Sent Done false
1538 1313 Acked Write 8 Done Sent Done false
1539 1314 Acked Read 5 Done Sent Done false
1540 1315 Acked Read 5 Done Sent Done false
1541 1316 Acked Read 4 Done Sent Done false
1542 1317 Acked Read 9 Done Sent Done false
1543 1318 Acked Flush 0 Done Sent Done false
1544 1319 Acked Read 5 Done Sent Done false
1545 1320 Acked Read 3 Done Sent Done false
1546 1321 Acked Write 7 Done Sent Done false
1547 1322 Acked Read 1 Done Sent Done false
1548 1323 Acked Read 2 Done Sent Done false
1549 1324 Acked Write 4 Done Sent Done false
1550 1325 Acked Write 7 Done Sent Done false
1551 1326 Acked Read 9 Done Sent Done false
1552 1327 Acked Read 7 Done Sent Done false
1553 1328 Acked Flush 0 Done Sent Done false
1554 1329 Acked Read 3 Done Sent Done false
1555 1330 Acked Read 7 Done Sent Done false
1556 1331 Acked Read 5 Done Sent Done false
1557 1332 Acked Write 8 Done Sent Done false
1558 1333 Acked Read 7 Done Sent Done false
1559 1334 Acked Read 4 Done Sent Done false
1560 1335 Acked Flush 0 Done Sent Done false
1561 1336 Acked Write 9 Done Sent Done false
1562 1337 Acked Read 5 Done Sent Done false
1563 1338 Acked Write 2 Done Sent Done false
1564 1339 Acked Read 9 Done Sent Done false
1565 1340 Acked Read 9 Done Sent Done false
1566 1341 Acked Read 7 Done Sent Done false
1567 1342 Acked Read 9 Done Sent Done false
1568 1343 Acked Write 10 Done Sent Done false
1569 1344 Acked Flush 0 Done Sent Done false
1570 1345 Acked Write 10 Done Sent Done false
1571 1346 Acked Flush 0 Done Sent Done false
1572 1347 Acked Write 10 Done Sent Done false
1573 1348 Acked Flush 0 Done Sent Done false
1574 1349 Acked Write 5 Done Sent Done false
1575 1350 Acked Write 5 Done Sent Done false
1576 1351 Acked Read 9 Done Sent Done false
1577 1352 Acked Read 4 Done Sent Done false
1578 1353 Acked Write 8 Done Sent Done false
1579 1354 Acked Read 1 Done Sent Done false
1580 1355 Acked Write 9 Done Sent Done false
1581 1356 Acked Read 3 Done Sent Done false
1582 1357 Acked Write 2 Done Sent Done false
1583 1358 Acked Read 5 Done Sent Done false
1584 1359 Acked Read 6 Done Sent Done false
1585 1360 Acked Write 8 Done Sent Done false
1586 1361 Acked Read 9 Done Sent Done false
1587 1362 Acked Write 2 Done Sent Done false
1588 1363 Acked Read 6 Done Sent Done false
1589 1364 Acked Write 4 Done Sent Done false
1590 1365 Acked Write 2 Done Sent Done false
1591 1366 Acked Write 3 Done Sent Done false
1592 1367 Acked Write 7 Done Sent Done false
1593 1368 Acked Flush 0 Done Sent Done false
1594 1369 Acked Read 4 Done Sent Done false
1595 1370 Acked Write 3 Done Sent Done false
1596 1371 Acked Read 5 Done Sent Done false
1597 1372 Acked Write 4 Done Sent Done false
1598 1373 Acked Read 4 Done Sent Done false
1599 1374 Acked Read 2 Done Sent Done false
1600 1375 Acked Write 9 Done Sent Done false
1601 1376 Acked Read 3 Done Sent Done false
1602 1377 Acked Read 2 Done Sent Done false
1603 1378 Acked Read 6 Done Sent Done false
1604 1379 Acked Write 9 Done Sent Done false
1605 1380 Acked Write 7 Done Sent Done false
1606 1381 Acked Write 10 Done Sent Done false
1607 1382 Acked Flush 0 Done Sent Done false
1608 1383 Acked Write 1 Done Sent Done false
1609 1384 Acked Read 4 Done Sent Done false
1610 1385 Acked Read 5 Done Sent Done false
1611 1386 Acked Flush 0 Done Sent Done false
1612 1387 Acked Write 6 Done Sent Done false
1613 1388 Acked Read 8 Done Sent Done false
1614 1389 Acked Write 3 Done Sent Done false
1615 1390 Acked Write 7 Done Sent Done false
1616 1391 Acked Write 2 Done Sent Done false
1617 1392 Acked Write 8 Done Sent Done false
1618 1393 Acked Read 2 Done Sent Done false
1619 1394 Acked Read 9 Done Sent Done false
1620 1395 Acked Read 4 Done Sent Done false
1621 1396 Acked Read 5 Done Sent Done false
1622 1397 Acked Read 6 Done Sent Done false
1623 1398 Acked Read 1 Done Sent Done false
1624 1399 Acked Read 7 Done Sent Done false
1625 1400 Acked Read 6 Done Sent Done false
1626 1401 Acked Read 9 Done Sent Done false
1627 1402 Acked Read 4 Done Sent Done false
1628 1403 Acked Write 3 Done Sent Done false
1629 1404 Acked Write 4 Done Sent Done false
1630 1405 Acked Read 5 Done Sent Done false
1631 1406 Acked Read 5 Done Sent Done false
1632 1407 Acked Read 9 Done Sent Done false
1633 1408 Acked Read 4 Done Sent Done false
1634 1409 Acked Read 9 Done Sent Done false
1635 1410 Acked Write 7 Done Sent Done false
1636 1411 Acked Read 8 Done Sent Done false
1637 1412 Acked Read 10 Done Sent Done false
1638 1413 Acked Read 4 Done Sent Done false
1639 1414 Acked Write 6 Done Sent Done false
1640 1415 Acked Write 6 Done Sent Done false
1641 1416 Acked Read 2 Done Sent Done false
1642 1417 Acked Write 1 Done Sent Done false
1643 1418 Acked Read 7 Done Sent Done false
1644 1419 Acked Write 10 Done Sent Done false
1645 1420 Acked Read 4 Done Sent Done false
1646 1421 Acked Read 10 Done Sent Done false
1647 1422 Acked Write 9 Done Sent Done false
1648 1423 Acked Flush 0 Done Sent Done false
1649 1424 Acked Flush 0 Done Sent Done false
1650 1425 Acked Read 7 Done Sent Done false
1651 1426 Acked Read 9 Done Sent Done false
1652 1427 Acked Write 2 Done Sent Done false
1653 1428 Acked Read 10 Done Sent Done false
1654 1429 Acked Write 9 Done Sent Done false
1655 1430 Acked Read 3 Done Sent Done false
1656 1431 Acked Read 2 Done Sent Done false
1657 1432 Acked Read 1 Done Sent Done false
1658 1433 Acked Write 8 Done Sent Done false
1659 1434 Acked Flush 0 Done Sent Done false
1660 1435 Acked Write 10 Done Sent Done false
1661 1436 Acked Read 8 Done Sent Done false
1662 1437 Acked Read 9 Done Sent Done false
1663 1438 Acked Read 6 Done Sent Done false
1664 1439 Acked Read 1 Done Sent Done false
1665 1440 Acked Read 10 Done Sent Done false
1666 1441 Acked Read 7 Done Sent Done false
1667 1442 Acked Write 6 Done Sent Done false
1668 1443 Acked Write 7 Done Sent Done false
1669 1444 Acked Write 9 Done Sent Done false
1670 1445 Acked Read 4 Done Sent Done false
1671 1446 Acked Read 7 Done Sent Done false
1672 1447 Acked Read 10 Done Sent Done false
1673 1448 Acked Write 3 Done Sent Done false
1674 1449 Acked Read 2 Done Sent Done false
1675 1450 Acked Write 3 Done Sent Done false
1676 1451 Acked Read 4 Done Sent Done false
1677 1452 Acked Read 3 Done Sent Done false
1678 1453 Acked Flush 0 Done Sent Done false
1679 1454 Acked Write 5 Done Sent Done false
1680 1455 Acked Write 10 Done Sent Done false
1681 1456 Acked Read 2 Done Sent Done false
1682 1457 Acked Read 3 Done Sent Done false
1683 1458 Acked Read 9 Done Sent Done false
1684 1459 Acked Read 8 Done Sent Done false
1685 1460 Acked Write 8 Done Sent Done false
1686 1461 Acked Read 2 Done Sent Done false
1687 1462 Acked Write 1 Done Sent Done false
1688 1463 Acked Write 5 Done Sent Done false
1689 1464 Acked Write 2 Done Sent Done false
1690 1465 Acked Write 9 Done Sent Done false
1691 1466 Acked Read 8 Done Sent Done false
1692 1467 Acked Read 6 Done Sent Done false
1693 1468 Acked Write 1 Done Sent Done false
1694 1469 Acked Read 8 Done Sent Done false
1695 1470 Acked Read 3 Done Sent Done false
1696 1471 Acked Flush 0 Done Sent Done false
1697 1472 Acked Write 4 Done Sent Done false
1698 1473 Acked Read 1 Done Sent Done false
1699 1474 Acked Write 4 Done Sent Done false
1700 1475 Acked Flush 0 Done Sent Done false
1701 1476 Acked Read 1 Done Sent Done false
1702 1477 Acked Read 10 Done Sent Done false
1703 1478 Acked Write 8 Done Sent Done false
1704 1479 Acked Write 5 Done Sent Done false
1705 1480 Acked Read 2 Done Sent Done false
1706 1481 Acked Read 7 Done Sent Done false
1707 1482 Acked Read 3 Done Sent Done false
1708 1483 Acked Read 9 Done Sent Done false
1709 1484 Acked Write 8 Done Sent Done false
1710 1485 Acked Read 8 Done Sent Done false
1711 1486 Acked Write 3 Done Sent Done false
1712 1487 Acked Write 1 Done Sent Done false
1713 1488 Acked Write 1 Done Sent Done false
1714 1489 Acked Write 4 Done Sent Done false
1715 1490 Acked Read 9 Done Sent Done false
1716 1491 Acked Read 4 Done Sent Done false
1717 1492 Acked Read 9 Done Sent Done false
1718 1493 Acked Read 10 Done Sent Done false
1719 1494 Acked Read 10 Done Sent Done false
1720 1495 Acked Read 9 Done Sent Done false
1721 1496 Acked Read 8 Done Sent Done false
1722 1497 Acked Read 7 Done Sent Done false
1723 1498 Acked Read 6 Done Sent Done false
1724 1499 Acked Read 2 Done Sent Done false
1725 1500 Acked Write 4 Done Sent Done false
1726 1501 Acked Read 3 Done Sent Done false
1727 1502 Acked Read 4 Done Sent Done false
1728 1503 Acked Write 8 Done Sent Done false
1729 1504 Acked Read 3 Done Sent Done false
1730 1505 Acked Write 10 Done Sent Done false
1731 1506 Acked Write 3 Done Sent Done false
1732 1507 Acked Write 3 Done Sent Done false
1733 1508 Acked Write 4 Done Sent Done false
1734 1509 Acked Read 2 Done Sent Done false
1735 1510 Acked Write 3 Done Sent Done false
1736 1511 Acked Write 3 Done Sent Done false
1737 1512 Acked Read 9 Done Sent Done false
1738 1513 Acked Read 9 Done Sent Done false
1739 1514 Acked Write 3 Done Sent Done false
1740 1515 Acked Write 5 Done Sent Done false
1741 1516 Acked Write 7 Done Sent Done false
1742 1517 Acked Read 4 Done Sent Done false
1743 1518 Acked Flush 0 Done Sent Done false
1744 1519 Acked Read 7 Done Sent Done false
1745 1520 Acked Read 10 Done Sent Done false
1746 1521 Acked Read 5 Done Sent Done false
1747 1522 Acked Write 6 Done Sent Done false
1748 1523 Acked Write 3 Done Sent Done false
1749 1524 Acked Read 1 Done Sent Done false
1750 1525 Acked Flush 0 Done Sent Done false
1751 1526 Acked Read 6 Done Sent Done false
1752 1527 Acked Read 1 Done Sent Done false
1753 1528 Acked Read 4 Done Sent Done false
1754 1529 Acked Write 4 Done Sent Done false
1755 1530 Acked Read 6 Done Sent Done false
1756 1531 Acked Read 1 Done Sent Done false
1757 1532 Acked Read 9 Done Sent Done false
1758 1533 Acked Write 5 Done Sent Done false
1759 1534 Acked Read 6 Done Sent Done false
1760 1535 Acked Read 9 Done Sent Done false
1761 1536 Acked Read 8 Done Sent Done false
1762 1537 Acked Write 7 Done Sent Done false
1763 1538 Acked Write 7 Done Sent Done false
1764 1539 Acked Write 3 Done Sent Done false
1765 1540 Acked Read 9 Done Sent Done false
1766 1541 Acked Read 1 Done Sent Done false
1767 1542 Acked Read 7 Done Sent Done false
1768 1543 Acked Flush 0 Done Sent Done false
1769 1544 Acked Read 1 Done Sent Done false
1770 1545 Acked Write 1 Done Sent Done false
1771 1546 Acked Write 10 Done Sent Done false
1772 1547 Acked Flush 0 Done Sent Done false
1773 1548 Acked Write 6 Done Sent Done false
1774 1549 Acked Flush 0 Done Sent Done false
1775 1550 Acked Write 7 Done Sent Done false
1776 1551 Acked Read 1 Done Sent Done false
1777 1552 Acked Write 5 Done Sent Done false
1778 1553 Acked Flush 0 Done Sent Done false
1779 1554 Acked Read 6 Done Sent Done false
1780 1555 Acked Write 4 Done Sent Done false
1781 1556 Acked Read 4 Done Sent Done false
1782 1557 Acked Read 6 Done Sent Done false
1783 1558 Acked Read 4 Done Sent Done false
1784 1559 Acked Write 3 Done Sent Done false
1785 1560 Acked Read 6 Done Sent Done false
1786 1561 Acked Read 4 Done Sent Done false
1787 1562 Acked Read 3 Done Sent Done false
1788 1563 Acked Read 8 Done Sent Done false
1789 1564 Acked Write 1 Done Sent Done false
1790 1565 Acked Write 8 Done Sent Done false
1791 1566 Acked Write 9 Done Sent Done false
1792 1567 Acked Read 6 Done Sent Done false
1793 1568 Acked Write 1 Done Sent Done false
1794 1569 Acked Read 2 Done Sent Done false
1795 1570 Acked Write 2 Done Sent Done false
1796 1571 Acked Write 8 Done Sent Done false
1797 1572 Acked Write 9 Done Sent Done false
1798 1573 Acked Read 3 Done Sent Done false
1799 1574 Acked Read 8 Done Sent Done false
1800 1575 Acked Read 10 Done Sent Done false
1801 1576 Acked Read 5 Done Sent Done false
1802 1577 Acked Read 6 Done Sent Done false
1803 1578 Acked Write 6 Done Sent Done false
1804 1579 Acked Read 9 Done Sent Done false
1805 1580 Acked Write 6 Done Sent Done false
1806 1581 Acked Read 9 Done Sent Done false
1807 1582 Acked Flush 0 Done Sent Done false
1808 1583 Acked Read 8 Done Sent Done false
1809 1584 Acked Read 2 Done Sent Done false
1810 1585 Acked Write 6 Done Sent Done false
1811 1586 Acked Read 9 Done Sent Done false
1812 1587 Acked Read 6 Done Sent Done false
1813 1588 Acked Write 3 Done Sent Done false
1814 1589 Acked Write 5 Done Sent Done false
1815 1590 Acked Flush 0 Done Sent Done false
1816 1591 Acked Write 1 Done Sent Done false
1817 1592 Acked Read 10 Done Sent Done false
1818 1593 Acked Write 6 Done Sent Done false
1819 1594 Acked Read 8 Done Sent Done false
1820 1595 Acked Read 10 Done Sent Done false
1821 1596 Acked Read 6 Done Sent Done false
1822 1597 Acked Read 4 Done Sent Done false
1823 1598 Acked Write 3 Done Sent Done false
1824 1599 Acked Write 2 Done Sent Done false
1825 1600 Acked Write 5 Done Sent Done false
1826 1601 Acked Read 3 Done Sent Done false
1827 1602 Acked Read 1 Done Sent Done false
1828 STATES DS:0 DS:1 DS:2 TOTAL
1829 Sent 0 301 0 301
1830 Done 301 0 301 602
1831 Skipped 0 0 0 0
1832 Error 0 0 0 0
1833 Last Flush: 1590 1301 1590
1834 Downstairs last five completed: 1301 1300 1299 1298 1297
1835 Upstairs last five completed: 1602 1601 1600 1599 1598
1836 CLIENT: Up:0 ds:301 act:2
1837 [8820] got start action
1838 [8820] I am going to restart
1839 Starting downstairs at port 8820
1840 Make output file at "/tmp/dsc/downstairs-8820.txt"
1841 [8820][1] reports Starting
1842 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1205)
1843 [8820][1] reports Running
1844 ----------------------------------------------------------------
1845 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
1846 States: Active Offline Active
1847 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
1848 1302 Acked Write 7 Done Sent Done false
1849 1303 Acked Read 3 Done Sent Done false
1850 1304 Acked Read 9 Done Sent Done false
1851 1305 Acked Read 3 Done Sent Done false
1852 1306 Acked Write 4 Done Sent Done false
1853 1307 Acked Write 5 Done Sent Done false
1854 1308 Acked Write 2 Done Sent Done false
1855 1309 Acked Read 8 Done Sent Done false
1856 1310 Acked Read 2 Done Sent Done false
1857 1311 Acked Read 1 Done Sent Done false
1858 1312 Acked Write 9 Done Sent Done false
1859 1313 Acked Write 8 Done Sent Done false
1860 1314 Acked Read 5 Done Sent Done false
1861 1315 Acked Read 5 Done Sent Done false
1862 1316 Acked Read 4 Done Sent Done false
1863 1317 Acked Read 9 Done Sent Done false
1864 1318 Acked Flush 0 Done Sent Done false
1865 1319 Acked Read 5 Done Sent Done false
1866 1320 Acked Read 3 Done Sent Done false
1867 1321 Acked Write 7 Done Sent Done false
1868 1322 Acked Read 1 Done Sent Done false
1869 1323 Acked Read 2 Done Sent Done false
1870 1324 Acked Write 4 Done Sent Done false
1871 1325 Acked Write 7 Done Sent Done false
1872 1326 Acked Read 9 Done Sent Done false
1873 1327 Acked Read 7 Done Sent Done false
1874 1328 Acked Flush 0 Done Sent Done false
1875 1329 Acked Read 3 Done Sent Done false
1876 1330 Acked Read 7 Done Sent Done false
1877 1331 Acked Read 5 Done Sent Done false
1878 1332 Acked Write 8 Done Sent Done false
1879 1333 Acked Read 7 Done Sent Done false
1880 1334 Acked Read 4 Done Sent Done false
1881 1335 Acked Flush 0 Done Sent Done false
1882 1336 Acked Write 9 Done Sent Done false
1883 1337 Acked Read 5 Done Sent Done false
1884 1338 Acked Write 2 Done Sent Done false
1885 1339 Acked Read 9 Done Sent Done false
1886 1340 Acked Read 9 Done Sent Done false
1887 1341 Acked Read 7 Done Sent Done false
1888 1342 Acked Read 9 Done Sent Done false
1889 1343 Acked Write 10 Done Sent Done false
1890 1344 Acked Flush 0 Done Sent Done false
1891 1345 Acked Write 10 Done Sent Done false
1892 1346 Acked Flush 0 Done Sent Done false
1893 1347 Acked Write 10 Done Sent Done false
1894 1348 Acked Flush 0 Done Sent Done false
1895 1349 Acked Write 5 Done Sent Done false
1896 1350 Acked Write 5 Done Sent Done false
1897 1351 Acked Read 9 Done Sent Done false
1898 1352 Acked Read 4 Done Sent Done false
1899 1353 Acked Write 8 Done Sent Done false
1900 1354 Acked Read 1 Done Sent Done false
1901 1355 Acked Write 9 Done Sent Done false
1902 1356 Acked Read 3 Done Sent Done false
1903 1357 Acked Write 2 Done Sent Done false
1904 1358 Acked Read 5 Done Sent Done false
1905 1359 Acked Read 6 Done Sent Done false
1906 1360 Acked Write 8 Done Sent Done false
1907 1361 Acked Read 9 Done Sent Done false
1908 1362 Acked Write 2 Done Sent Done false
1909 1363 Acked Read 6 Done Sent Done false
1910 1364 Acked Write 4 Done Sent Done false
1911 1365 Acked Write 2 Done Sent Done false
1912 1366 Acked Write 3 Done Sent Done false
1913 1367 Acked Write 7 Done Sent Done false
1914 1368 Acked Flush 0 Done Sent Done false
1915 1369 Acked Read 4 Done Sent Done false
1916 1370 Acked Write 3 Done Sent Done false
1917 1371 Acked Read 5 Done Sent Done false
1918 1372 Acked Write 4 Done Sent Done false
1919 1373 Acked Read 4 Done Sent Done false
1920 1374 Acked Read 2 Done Sent Done false
1921 1375 Acked Write 9 Done Sent Done false
1922 1376 Acked Read 3 Done Sent Done false
1923 1377 Acked Read 2 Done Sent Done false
1924 1378 Acked Read 6 Done Sent Done false
1925 1379 Acked Write 9 Done Sent Done false
1926 1380 Acked Write 7 Done Sent Done false
1927 1381 Acked Write 10 Done Sent Done false
1928 1382 Acked Flush 0 Done Sent Done false
1929 1383 Acked Write 1 Done Sent Done false
1930 1384 Acked Read 4 Done Sent Done false
1931 1385 Acked Read 5 Done Sent Done false
1932 1386 Acked Flush 0 Done Sent Done false
1933 1387 Acked Write 6 Done Sent Done false
1934 1388 Acked Read 8 Done Sent Done false
1935 1389 Acked Write 3 Done Sent Done false
1936 1390 Acked Write 7 Done Sent Done false
1937 1391 Acked Write 2 Done Sent Done false
1938 1392 Acked Write 8 Done Sent Done false
1939 1393 Acked Read 2 Done Sent Done false
1940 1394 Acked Read 9 Done Sent Done false
1941 1395 Acked Read 4 Done Sent Done false
1942 1396 Acked Read 5 Done Sent Done false
1943 1397 Acked Read 6 Done Sent Done false
1944 1398 Acked Read 1 Done Sent Done false
1945 1399 Acked Read 7 Done Sent Done false
1946 1400 Acked Read 6 Done Sent Done false
1947 1401 Acked Read 9 Done Sent Done false
1948 1402 Acked Read 4 Done Sent Done false
1949 1403 Acked Write 3 Done Sent Done false
1950 1404 Acked Write 4 Done Sent Done false
1951 1405 Acked Read 5 Done Sent Done false
1952 1406 Acked Read 5 Done Sent Done false
1953 1407 Acked Read 9 Done Sent Done false
1954 1408 Acked Read 4 Done Sent Done false
1955 1409 Acked Read 9 Done Sent Done false
1956 1410 Acked Write 7 Done Sent Done false
1957 1411 Acked Read 8 Done Sent Done false
1958 1412 Acked Read 10 Done Sent Done false
1959 1413 Acked Read 4 Done Sent Done false
1960 1414 Acked Write 6 Done Sent Done false
1961 1415 Acked Write 6 Done Sent Done false
1962 1416 Acked Read 2 Done Sent Done false
1963 1417 Acked Write 1 Done Sent Done false
1964 1418 Acked Read 7 Done Sent Done false
1965 1419 Acked Write 10 Done Sent Done false
1966 1420 Acked Read 4 Done Sent Done false
1967 1421 Acked Read 10 Done Sent Done false
1968 1422 Acked Write 9 Done Sent Done false
1969 1423 Acked Flush 0 Done Sent Done false
1970 1424 Acked Flush 0 Done Sent Done false
1971 1425 Acked Read 7 Done Sent Done false
1972 1426 Acked Read 9 Done Sent Done false
1973 1427 Acked Write 2 Done Sent Done false
1974 1428 Acked Read 10 Done Sent Done false
1975 1429 Acked Write 9 Done Sent Done false
1976 1430 Acked Read 3 Done Sent Done false
1977 1431 Acked Read 2 Done Sent Done false
1978 1432 Acked Read 1 Done Sent Done false
1979 1433 Acked Write 8 Done Sent Done false
1980 1434 Acked Flush 0 Done Sent Done false
1981 1435 Acked Write 10 Done Sent Done false
1982 1436 Acked Read 8 Done Sent Done false
1983 1437 Acked Read 9 Done Sent Done false
1984 1438 Acked Read 6 Done Sent Done false
1985 1439 Acked Read 1 Done Sent Done false
1986 1440 Acked Read 10 Done Sent Done false
1987 1441 Acked Read 7 Done Sent Done false
1988 1442 Acked Write 6 Done Sent Done false
1989 1443 Acked Write 7 Done Sent Done false
1990 1444 Acked Write 9 Done Sent Done false
1991 1445 Acked Read 4 Done Sent Done false
1992 1446 Acked Read 7 Done Sent Done false
1993 1447 Acked Read 10 Done Sent Done false
1994 1448 Acked Write 3 Done Sent Done false
1995 1449 Acked Read 2 Done Sent Done false
1996 1450 Acked Write 3 Done Sent Done false
1997 1451 Acked Read 4 Done Sent Done false
1998 1452 Acked Read 3 Done Sent Done false
1999 1453 Acked Flush 0 Done Sent Done false
2000 1454 Acked Write 5 Done Sent Done false
2001 1455 Acked Write 10 Done Sent Done false
2002 1456 Acked Read 2 Done Sent Done false
2003 1457 Acked Read 3 Done Sent Done false
2004 1458 Acked Read 9 Done Sent Done false
2005 1459 Acked Read 8 Done Sent Done false
2006 1460 Acked Write 8 Done Sent Done false
2007 1461 Acked Read 2 Done Sent Done false
2008 1462 Acked Write 1 Done Sent Done false
2009 1463 Acked Write 5 Done Sent Done false
2010 1464 Acked Write 2 Done Sent Done false
2011 1465 Acked Write 9 Done Sent Done false
2012 1466 Acked Read 8 Done Sent Done false
2013 1467 Acked Read 6 Done Sent Done false
2014 1468 Acked Write 1 Done Sent Done false
2015 1469 Acked Read 8 Done Sent Done false
2016 1470 Acked Read 3 Done Sent Done false
2017 1471 Acked Flush 0 Done Sent Done false
2018 1472 Acked Write 4 Done Sent Done false
2019 1473 Acked Read 1 Done Sent Done false
2020 1474 Acked Write 4 Done Sent Done false
2021 1475 Acked Flush 0 Done Sent Done false
2022 1476 Acked Read 1 Done Sent Done false
2023 1477 Acked Read 10 Done Sent Done false
2024 1478 Acked Write 8 Done Sent Done false
2025 1479 Acked Write 5 Done Sent Done false
2026 1480 Acked Read 2 Done Sent Done false
2027 1481 Acked Read 7 Done Sent Done false
2028 1482 Acked Read 3 Done Sent Done false
2029 1483 Acked Read 9 Done Sent Done false
2030 1484 Acked Write 8 Done Sent Done false
2031 1485 Acked Read 8 Done Sent Done false
2032 1486 Acked Write 3 Done Sent Done false
2033 1487 Acked Write 1 Done Sent Done false
2034 1488 Acked Write 1 Done Sent Done false
2035 1489 Acked Write 4 Done Sent Done false
2036 1490 Acked Read 9 Done Sent Done false
2037 1491 Acked Read 4 Done Sent Done false
2038 1492 Acked Read 9 Done Sent Done false
2039 1493 Acked Read 10 Done Sent Done false
2040 1494 Acked Read 10 Done Sent Done false
2041 1495 Acked Read 9 Done Sent Done false
2042 1496 Acked Read 8 Done Sent Done false
2043 1497 Acked Read 7 Done Sent Done false
2044 1498 Acked Read 6 Done Sent Done false
2045 1499 Acked Read 2 Done Sent Done false
2046 1500 Acked Write 4 Done Sent Done false
2047 1501 Acked Read 3 Done Sent Done false
2048 1502 Acked Read 4 Done Sent Done false
2049 1503 Acked Write 8 Done Sent Done false
2050 1504 Acked Read 3 Done Sent Done false
2051 1505 Acked Write 10 Done Sent Done false
2052 1506 Acked Write 3 Done Sent Done false
2053 1507 Acked Write 3 Done Sent Done false
2054 1508 Acked Write 4 Done Sent Done false
2055 1509 Acked Read 2 Done Sent Done false
2056 1510 Acked Write 3 Done Sent Done false
2057 1511 Acked Write 3 Done Sent Done false
2058 1512 Acked Read 9 Done Sent Done false
2059 1513 Acked Read 9 Done Sent Done false
2060 1514 Acked Write 3 Done Sent Done false
2061 1515 Acked Write 5 Done Sent Done false
2062 1516 Acked Write 7 Done Sent Done false
2063 1517 Acked Read 4 Done Sent Done false
2064 1518 Acked Flush 0 Done Sent Done false
2065 1519 Acked Read 7 Done Sent Done false
2066 1520 Acked Read 10 Done Sent Done false
2067 1521 Acked Read 5 Done Sent Done false
2068 1522 Acked Write 6 Done Sent Done false
2069 1523 Acked Write 3 Done Sent Done false
2070 1524 Acked Read 1 Done Sent Done false
2071 1525 Acked Flush 0 Done Sent Done false
2072 1526 Acked Read 6 Done Sent Done false
2073 1527 Acked Read 1 Done Sent Done false
2074 1528 Acked Read 4 Done Sent Done false
2075 1529 Acked Write 4 Done Sent Done false
2076 1530 Acked Read 6 Done Sent Done false
2077 1531 Acked Read 1 Done Sent Done false
2078 1532 Acked Read 9 Done Sent Done false
2079 1533 Acked Write 5 Done Sent Done false
2080 1534 Acked Read 6 Done Sent Done false
2081 1535 Acked Read 9 Done Sent Done false
2082 1536 Acked Read 8 Done Sent Done false
2083 1537 Acked Write 7 Done Sent Done false
2084 1538 Acked Write 7 Done Sent Done false
2085 1539 Acked Write 3 Done Sent Done false
2086 1540 Acked Read 9 Done Sent Done false
2087 1541 Acked Read 1 Done Sent Done false
2088 1542 Acked Read 7 Done Sent Done false
2089 1543 Acked Flush 0 Done Sent Done false
2090 1544 Acked Read 1 Done Sent Done false
2091 1545 Acked Write 1 Done Sent Done false
2092 1546 Acked Write 10 Done Sent Done false
2093 1547 Acked Flush 0 Done Sent Done false
2094 1548 Acked Write 6 Done Sent Done false
2095 1549 Acked Flush 0 Done Sent Done false
2096 1550 Acked Write 7 Done Sent Done false
2097 1551 Acked Read 1 Done Sent Done false
2098 1552 Acked Write 5 Done Sent Done false
2099 1553 Acked Flush 0 Done Sent Done false
2100 1554 Acked Read 6 Done Sent Done false
2101 1555 Acked Write 4 Done Sent Done false
2102 1556 Acked Read 4 Done Sent Done false
2103 1557 Acked Read 6 Done Sent Done false
2104 1558 Acked Read 4 Done Sent Done false
2105 1559 Acked Write 3 Done Sent Done false
2106 1560 Acked Read 6 Done Sent Done false
2107 1561 Acked Read 4 Done Sent Done false
2108 1562 Acked Read 3 Done Sent Done false
2109 1563 Acked Read 8 Done Sent Done false
2110 1564 Acked Write 1 Done Sent Done false
2111 1565 Acked Write 8 Done Sent Done false
2112 1566 Acked Write 9 Done Sent Done false
2113 1567 Acked Read 6 Done Sent Done false
2114 1568 Acked Write 1 Done Sent Done false
2115 1569 Acked Read 2 Done Sent Done false
2116 1570 Acked Write 2 Done Sent Done false
2117 1571 Acked Write 8 Done Sent Done false
2118 1572 Acked Write 9 Done Sent Done false
2119 1573 Acked Read 3 Done Sent Done false
2120 1574 Acked Read 8 Done Sent Done false
2121 1575 Acked Read 10 Done Sent Done false
2122 1576 Acked Read 5 Done Sent Done false
2123 1577 Acked Read 6 Done Sent Done false
2124 1578 Acked Write 6 Done Sent Done false
2125 1579 Acked Read 9 Done Sent Done false
2126 1580 Acked Write 6 Done Sent Done false
2127 1581 Acked Read 9 Done Sent Done false
2128 1582 Acked Flush 0 Done Sent Done false
2129 1583 Acked Read 8 Done Sent Done false
2130 1584 Acked Read 2 Done Sent Done false
2131 1585 Acked Write 6 Done Sent Done false
2132 1586 Acked Read 9 Done Sent Done false
2133 1587 Acked Read 6 Done Sent Done false
2134 1588 Acked Write 3 Done Sent Done false
2135 1589 Acked Write 5 Done Sent Done false
2136 1590 Acked Flush 0 Done Sent Done false
2137 1591 Acked Write 1 Done Sent Done false
2138 1592 Acked Read 10 Done Sent Done false
2139 1593 Acked Write 6 Done Sent Done false
2140 1594 Acked Read 8 Done Sent Done false
2141 1595 Acked Read 10 Done Sent Done false
2142 1596 Acked Read 6 Done Sent Done false
2143 1597 Acked Read 4 Done Sent Done false
2144 1598 Acked Write 3 Done Sent Done false
2145 1599 Acked Write 2 Done Sent Done false
2146 1600 Acked Write 5 Done Sent Done false
2147 1601 Acked Read 3 Done Sent Done false
2148 1602 Acked Read 1 Done Sent Done false
2149 1603 Acked Flush 0 Done Sent Done false
2150 STATES DS:0 DS:1 DS:2 TOTAL
2151 Sent 0 302 0 302
2152 Done 302 0 302 604
2153 Skipped 0 0 0 0
2154 Error 0 0 0 0
2155 Last Flush: 1603 1301 1603
2156 Downstairs last five completed: 1301 1300 1299 1298 1297
2157 Upstairs last five completed: 1603 1602 1601 1600 1599
2158 CLIENT: Up:0 ds:302 act:2
21592024-11-26T22:04:58.652ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
21602024-11-26T22:04:58.652ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:64523) = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
21612024-11-26T22:04:58.653ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
21622024-11-26T22:04:58.653ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
21632024-11-26T22:04:58.653ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
21642024-11-26T22:04:58.653ZINFOcrucible: send last flush ID to this DS: 1301 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
21652024-11-26T22:04:58.653ZINFOcrucible: Replied this last flush ID: 1301 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
21662024-11-26T22:04:58.654ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
21672024-11-26T22:04:58.654ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
21682024-11-26T22:04:58.654ZINFOcrucible: [1] client re-new 302 jobs since flush 1301 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
21692024-11-26T22:04:58.654ZINFOcrucible: [1] Replayed 302 jobs since flush: 1301 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
2170 ----------------------------------------------------------------
2171 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
2172 Downstairs last five completed: 1603 1602 1601 1600 1599
2173 Upstairs last five completed: 1603 1602 1601 1600 1599
2174 CLIENT: Up:0 ds:0 act:3
2175 Replay: All jobs finished
2176 Main task has work to do, go find it
2177 got dsc Stop(1)
2178 stop 1
2179 Nov 26 22:05:00.901 INFO request completed, latency_us: 352, response_code: 204, uri: /stop/cid/1, method: POST, req_id: bc8c21fb-27a9-4f4f-b993-6ed2ab49ad4f, remote_addr: 127.0.0.1:36829, local_addr:[8820] Got stop action so:false kr:false
2180 127.0.0.1:9998
2181 Nov 26 22:05:00.902 INFO request completed, latency_us: 123, response_code: 200, uri: /state/cid/1, method: GET, req_id: b1f35866-e7d3-49f4-a817-1174849cb0a9, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
21822024-11-26T22:05:00.902ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
21832024-11-26T22:05:00.902ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
21842024-11-26T22:05:00.902ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
21852024-11-26T22:05:00.902ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
21862024-11-26T22:05:00.902ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
21872024-11-26T22:05:00.902ZINFOcrucible: client task is exiting = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
2188 [8820] Exited with: ExitStatus(unix_wait_status(9))
2189 [8820][1] reports Exit
2190 Nov 26 22:05:04.904 INFO request completed, latency_us: 236, response_code: 200, uri: /state/cid/1, method: GET, req_id: 82b10c5d-33dd-4e83-b0d1-6e13754da113, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
2191 001/300 Read block 1060 len 8192
2192 002/300 Write block 2165 len 24576 data: 2 2 2 2 2 2
2193 003/300 Write block 908 len 36864 data: 2 2 2 2 2 2 2 2 2
2194 004/300 Read block 3808 len 20480
2195 005/300 Read block 4858 len 28672
2196 006/300 Write block 3653 len 8192 data: 4 3
2197 007/300 Write block 3247 len 4096 data: 2
2198 008/300 Write block 1774 len 36864 data: 2 2 2 3 3 4 4 2 2
2199 009/300 Write block 3989 len 40960 data: 2 2 3 3 3 2 2 2 2 2
2200 010/300 Write block 4587 len 12288 data: 2 2 2
2201 011/300 Read block 2068 len 12288
2202 012/300 Read block 3256 len 32768
2203 013/300 Read block 2182 len 20480
2204 014/300 Write block 870 len 12288 data: 2 2 2
2205 015/300 Read block 1259 len 8192
2206 016/300 Read block 4809 len 16384
2207 017/300 Write block 2191 len 24576 data: 2 2 2 2 2 3
2208 018/300 Read block 2934 len 16384
2209 019/300 Read block 4386 len 16384
2210 020/300 Write block 2182 len 16384 data: 3 3 3 3
2211 021/300 Read block 1470 len 4096
2212 022/300 Write block 3315 len 24576 data: 3 3 2 2 2 2
2213 023/300 Read block 2273 len 8192
2214 024/300 Write block 1144 len 32768 data: 2 2 2 2 2 2 2 2
2215 025/300 Write block 1861 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2216 026/300 Flush
2217 027/300 Write block 4575 len 4096 data: 3
2218 028/300 Read block 4438 len 8192
2219 029/300 Read block 3722 len 8192
2220 030/300 Read block 1476 len 32768
2221 031/300 Write block 3550 len 28672 data: 2 3 3 3 3 2 2
2222 032/300 Write block 488 len 40960 data: 2 2 2 2 2 2 2 2 3 2
2223 033/300 Read block 1971 len 36864
2224 034/300 Read block 1829 len 12288
2225 035/300 Flush
2226 036/300 Read block 2718 len 24576
2227 037/300 Write block 4313 len 16384 data: 2 2 2 2
2228 038/300 Read block 3871 len 20480
2229 039/300 Read block 4012 len 12288
2230 040/300 Read block 40 len 4096
2231 041/300 Read block 4534 len 36864
2232 042/300 Write block 3059 len 16384 data: 2 2 2 2
2233 043/300 Write block 1540 len 20480 data: 3 3 3 2 2
2234 044/300 Read block 3664 len 40960
2235 045/300 Write block 35 len 32768 data: 2 2 2 2 2 2 2 2
2236 046/300 Read block 4983 len 4096
2237 047/300 Read block 1671 len 40960
2238 048/300 Read block 3949 len 36864
2239 049/300 Write block 971 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2240 050/300 Read block 3162 len 28672
2241 051/300 Write block 4338 len 20480 data: 2 2 2 2 2
2242 052/300 Read block 3485 len 24576
2243 053/300 Read block 1498 len 4096
2244 054/300 Write block 363 len 20480 data: 2 2 2 2 2
2245 055/300 Flush
2246 056/300 Read block 3921 len 36864
2247 057/300 Read block 2813 len 16384
2248 058/300 Read block 452 len 36864
2249 059/300 Read block 1308 len 4096
2250 060/300 Write block 4932 len 12288 data: 2 2 2
2251 061/300 Flush
2252 062/300 Flush
2253 063/300 Read block 3563 len 24576
2254 064/300 Write block 291 len 28672 data: 3 3 3 3 3 3 2
2255 065/300 Read block 1178 len 40960
2256 066/300 Write block 4485 len 36864 data: 2 2 2 2 2 2 2 2 2
2257 067/300 Write block 2478 len 36864 data: 2 2 2 2 2 2 2 2 2
2258 068/300 Read block 1618 len 20480
2259 069/300 Read block 3071 len 8192
2260 070/300 Write block 3872 len 16384 data: 2 2 2 2
2261 071/300 Read block 3507 len 40960
2262 072/300 Write block 1572 len 36864 data: 2 2 2 2 2 2 2 2 2
2263 073/300 Read block 1753 len 20480
2264 074/300 Write block 1181 len 4096 data: 3
2265 075/300 Write block 2758 len 16384 data: 2 2 2 2
2266 076/300 Write block 4246 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2267 077/300 Read block 2440 len 8192
2268 078/300 Read block 1694 len 16384
2269 079/300 Write block 3926 len 8192 data: 3 2
2270 080/300 Write block 2390 len 8192 data: 2 2
2271 081/300 Flush
2272 082/300 Read block 1448 len 20480
2273 083/300 Read block 685 len 12288
2274 084/300 Read block 4354 len 36864
2275 085/300 Read block 3076 len 24576
2276 086/300 Read block 4995 len 16384
2277 087/300 Read block 267 len 36864
2278 088/300 Read block 3836 len 4096
2279 089/300 Read block 641 len 8192
2280 090/300 Read block 860 len 40960
2281 091/300 Read block 4737 len 12288
2282 092/300 Read block 1857 len 28672
2283 093/300 Write block 1488 len 16384 data: 2 2 2 2
2284 094/300 Read block 4798 len 8192
2285 095/300 Read block 2635 len 8192
2286 096/300 Read block 2967 len 40960
2287 097/300 Read block 3043 len 36864
2288 098/300 Write block 3182 len 12288 data: 4 2 2
2289 099/300 Read block 1401 len 20480
2290 100/300 Write block 1520 len 8192 data: 2 2
2291 101/300 Flush
2292 102/300 Read block 3136 len 16384
2293 103/300 Write block 958 len 12288 data: 2 2 2
2294 104/300 Flush
2295 105/300 Read block 2895 len 24576
2296 106/300 Read block 4470 len 28672
2297 107/300 Flush
2298 108/300 Write block 4960 len 28672 data: 2 2 2 2 2 3 3
2299 109/300 Read block 4023 len 20480
2300 110/300 Write block 4242 len 12288 data: 2 2 2
2301 111/300 Read block 3241 len 16384
2302 112/300 Write block 3244 len 24576 data: 2 2 2 3 2 2
2303 113/300 Write block 509 len 36864 data: 3 3 3 2 2 2 2 2 2
2304 114/300 Read block 1374 len 4096
2305 115/300 Read block 2632 len 24576
2306 116/300 Write block 4360 len 12288 data: 2 2 2
2307 117/300 Write block 4689 len 28672 data: 3 3 3 3 3 3 3
2308 118/300 Read block 2267 len 12288
2309 119/300 Read block 2450 len 20480
2310 120/300 Write block 355 len 36864 data: 2 2 2 2 2 2 2 2 3
2311 121/300 Read block 2284 len 32768
2312 122/300 Write block 2903 len 28672 data: 2 2 2 2 2 2 2
2313 123/300 Read block 2426 len 12288
2314 124/300 Write block 4333 len 36864 data: 2 2 2 2 2 3 3 3 3
2315 125/300 Read block 2079 len 4096
2316 126/300 Read block 234 len 32768
2317 127/300 Write block 1318 len 12288 data: 2 2 2
2318 128/300 Read block 1917 len 24576
2319 129/300 Write block 3349 len 12288 data: 2 2 2
2320 130/300 Write block 3763 len 8192 data: 2 2
2321 131/300 Read block 3550 len 12288
2322 132/300 Read block 4643 len 20480
2323 133/300 Read block 3574 len 8192
2324 134/300 Write block 3678 len 24576 data: 2 2 2 2 2 2
2325 135/300 Read block 3095 len 8192
2326 136/300 Read block 3553 len 12288
2327 137/300 Write block 1537 len 20480 data: 3 3 3 4 4
2328 138/300 Write block 1846 len 28672 data: 2 3 3 3 4 4 4
2329 139/300 Read block 1315 len 32768
2330 140/300 Read block 3673 len 32768
2331 141/300 Flush
2332 142/300 Write block 46 len 36864 data: 2 2 2 2 2 2 2 2 2
2333 143/300 Write block 1490 len 20480 data: 3 3 2 2 2
2334 144/300 Read block 3682 len 20480
2335 145/300 Flush
2336 146/300 Read block 4150 len 16384
2337 147/300 Write block 781 len 32768 data: 2 2 2 2 2 2 2 2
2338 148/300 Read block 1818 len 28672
2339 149/300 Read block 3609 len 24576
2340 150/300 Write block 910 len 40960 data: 3 3 3 3 3 3 3 2 2 2
2341 151/300 Write block 4922 len 36864 data: 2 2 2 3 3 3 3 3 3
2342 152/300 Write block 1499 len 24576 data: 2 2 2 2 2 2
2343 153/300 Read block 482 len 28672
2344 154/300 Write block 288 len 32768 data: 2 3 3 4 4 4 4 4
2345 155/300 Write block 3667 len 32768 data: 2 2 2 2 2 3 3 3
2346 156/300 Read block 1110 len 16384
2347 157/300 Read block 2860 len 40960
2348 158/300 Write block 3491 len 24576 data: 2 2 2 2 2 2
2349 159/300 Read block 3340 len 32768
2350 160/300 Write block 1095 len 4096 data: 2
2351 161/300 Write block 3175 len 24576 data: 2 3 3 3 3 4
2352 162/300 Read block 3701 len 16384
2353 163/300 Write block 4467 len 8192 data: 3 3
2354 164/300 Flush
2355 165/300 Write block 154 len 32768 data: 2 2 2 2 2 2 2 2
2356 166/300 Write block 4813 len 28672 data: 2 2 2 2 2 2 2
2357 167/300 Write block 1566 len 16384 data: 2 2 2 2
2358 168/300 Write block 560 len 28672 data: 2 2 2 2 2 2 2
2359 169/300 Read block 2196 len 20480
2360 170/300 Flush
2361 171/300 Read block 4273 len 36864
2362 172/300 Write block 3569 len 24576 data: 2 2 2 2 2 2
2363 173/300 Write block 4806 len 28672 data: 3 2 3 3 3 2 2
2364 174/300 Write block 4415 len 12288 data: 2 2 2
2365 175/300 Read block 3618 len 32768
2366 176/300 Read block 1807 len 40960
2367 177/300 Read block 3408 len 36864
2368 178/300 Flush
2369 179/300 Read block 2664 len 36864
2370 180/300 Flush
2371 181/300 Write block 2081 len 12288 data: 3 3 3
2372 182/300 Read block 3390 len 8192
2373 183/300 Read block 173 len 32768
2374 184/300 Read block 2052 len 16384
2375 185/300 Write block 2846 len 12288 data: 2 2 2
2376 186/300 Read block 1160 len 24576
2377 187/300 Write block 2054 len 4096 data: 2
2378 188/300 Read block 692 len 16384
2379 189/300 Write block 1471 len 24576 data: 2 3 3 3 3 3
2380 190/300 Read block 2477 len 12288
2381 191/300 Write block 2165 len 8192 data: 3 3
2382 192/300 Write block 758 len 32768 data: 2 2 2 2 2 2 2 2
2383 193/300 Read block 3140 len 4096
2384 194/300 Write block 1855 len 16384 data: 3 3 3 3
2385 195/300 Write block 3990 len 32768 data: 3 4 4 4 3 3 3 3
2386 196/300 Read block 111 len 8192
2387 197/300 Write block 3256 len 20480 data: 2 2 2 2 2
2388 198/300 Write block 302 len 4096 data: 2
2389 199/300 Flush
2390 200/300 Read block 3684 len 32768
2391 201/300 Read block 2781 len 40960
2392 202/300 Write block 1932 len 32768 data: 2 2 2 2 2 3 3 3
2393 203/300 Write block 2894 len 40960 data: 2 2 2 2 2 2 2 2 2 3
2394 204/300 Read block 1877 len 4096
2395 205/300 Write block 4489 len 40960 data: 3 3 3 3 3 2 2 2 2 2
2396 206/300 Write block 4527 len 32768 data: 2 2 2 2 2 2 2 2
2397 207/300 Flush
2398 208/300 Write block 2373 len 12288 data: 2 2 2
2399 209/300 Write block 2135 len 32768 data: 2 2 2 2 2 2 2 2
2400 210/300 Read block 1611 len 40960
2401 211/300 Write block 3242 len 24576 data: 2 2 3 3 3 4
2402 212/300 Read block 201 len 32768
2403 213/300 Write block 1858 len 8192 data: 4 2
2404 214/300 Read block 2372 len 24576
2405 215/300 Write block 337 len 36864 data: 2 2 2 2 2 2 2 2 2
2406 216/300 Flush
2407 217/300 Flush
2408 218/300 Read block 1306 len 4096
2409 219/300 Read block 3109 len 16384
2410 220/300 Read block 4855 len 8192
2411 221/300 Read block 796 len 28672
2412 222/300 Write block 1163 len 8192 data: 2 2
2413 223/300 Read block 3281 len 36864
2414 224/300 Write block 65 len 20480 data: 2 2 2 2 2
2415 225/300 Read block 1135 len 28672
2416 226/300 Write block 766 len 24576 data: 2 2 2 2 2 2
2417 227/300 Read block 3780 len 24576
2418 228/300 Read block 1681 len 12288
2419 229/300 Read block 3508 len 8192
2420 230/300 Flush
2421 231/300 Write block 3404 len 24576 data: 2 2 2 2 2 2
2422 232/300 Flush
2423 233/300 Read block 4022 len 36864
2424 234/300 Read block 104 len 16384
2425 235/300 Flush
2426 236/300 Flush
2427 237/300 Write block 2654 len 12288 data: 2 2 2
2428 238/300 Write block 4475 len 12288 data: 2 2 2
2429 239/300 Flush
2430 240/300 Flush
2431 241/300 Read block 3469 len 24576
2432 242/300 Write block 4903 len 20480 data: 2 2 2 2 2
2433 243/300 Read block 2080 len 36864
2434 244/300 Read block 542 len 32768
2435 245/300 Write block 4807 len 12288 data: 3 4 4
2436 246/300 Read block 3022 len 28672
2437 247/300 Write block 4062 len 12288 data: 3 3 3
2438 248/300 Write block 2284 len 24576 data: 2 2 2 2 2 2
2439 249/300 Flush
2440 250/300 Read block 2778 len 36864
2441 251/300 Read block 1194 len 8192
2442 252/300 Read block 2141 len 32768
2443 253/300 Read block 1519 len 20480
2444 254/300 Write block 4313 len 32768 data: 3 3 3 3 2 2 2 2
2445 255/300 Read block 683 len 16384
2446 256/300 Write block 3676 len 8192 data: 3 3
2447 257/300 Read block 4043 len 4096
2448 258/300 Read block 2175 len 20480
2449 259/300 Write block 20 len 24576 data: 2 2 2 2 2 2
2450 260/300 Write block 3671 len 32768 data: 3 4 4 4 3 4 4 3
2451 261/300 Write block 147 len 4096 data: 2
2452 262/300 Write block 1523 len 40960 data: 2 2 2 2 2 3 3 3 3 3
2453 263/300 Read block 4738 len 40960
2454 264/300 Write block 2771 len 36864 data: 2 2 2 2 2 2 2 2 2
2455 265/300 Write block 622 len 12288 data: 2 2 2
2456 266/300 Write block 3087 len 16384 data: 2 2 2 2
2457 267/300 Read block 1571 len 32768
2458 268/300 Write block 2443 len 16384 data: 3 3 3 2
2459 269/300 Flush
2460 270/300 Write block 246 len 32768 data: 2 2 2 2 2 2 2 2
2461 271/300 Read block 303 len 32768
2462 272/300 Read block 3706 len 40960
2463 273/300 Read block 3650 len 40960
2464 274/300 Read block 2914 len 4096
2465 275/300 Read block 4314 len 24576
2466 276/300 Write block 2299 len 20480 data: 3 3 3 3 3
2467 277/300 Write block 755 len 4096 data: 2
2468 278/300 Read block 2328 len 36864
2469 279/300 Write block 3709 len 4096 data: 3
2470 280/300 Read block 654 len 12288
2471 281/300 Read block 86 len 8192
2472 282/300 Write block 1937 len 28672 data: 4 4 4 3 3 3 3
2473 283/300 Write block 1553 len 24576 data: 3 3 3 3 2 2
2474 284/300 Write block 1891 len 12288 data: 2 2 2
2475 285/300 Write block 1625 len 20480 data: 3 2 2 2 2
2476 286/300 Read block 455 len 4096
2477 287/300 Read block 325 len 4096
2478 288/300 Write block 2543 len 16384 data: 2 2 2 2
2479 289/300 Write block 4025 len 24576 data: 2 2 2 2 2 2
2480 290/300 Read block 3373 len 12288
2481 291/300 Write block 28 len 4096 data: 2
2482 292/300 Read block 3612 len 8192
2483 293/300 Read block 1420 len 40960
2484 294/300 Write block 4862 len 12288 data: 4 4 3
2485 295/300 Write block 926 len 4096 data: 2
2486 296/300 Flush
2487 297/300 Read block 211 len 20480
2488 298/300 Flush
2489 299/300 Write block 4380 len 36864 data: 2 2 2 2 2 2 2 2 3
2490 300/300 Write block 4536 len 20480 data: 3 4 3 3 2
2491 301/300 Read block 2150 len 16384
2492 Main task has work to do, go find it
2493 got dsc Start(1)
2494 start 1
2495 Nov 26 22:05:05.134 INFO request completed, latency_us: 204, response_code: 204, uri: /start/cid/1, method: POST, req_id: a57b80fc-705c-4e24-8ee2-4e4ec1bc5c06, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
2496 Replay: started 1, returned:Ok(())
2497 ----------------------------------------------------------------
2498 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
2499 States: Active Offline Active
2500 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
2501 1604 Acked Read 2 Done Sent Done false
2502 1605 Acked Write 6 Done Sent Done false
2503 1606 Acked Write 9 Done Sent Done false
2504 1607 Acked Read 5 Done Sent Done false
2505 1608 Acked Read 7 Done Sent Done false
2506 1609 Acked Write 2 Done Sent Done false
2507 1610 Acked Write 1 Done Sent Done false
2508 1611 Acked Write 9 Done Sent Done false
2509 1612 Acked Write 10 Done Sent Done false
2510 1613 Acked Write 3 Done Sent Done false
2511 1614 Acked Read 3 Done Sent Done false
2512 1615 Acked Read 8 Done Sent Done false
2513 1616 Acked Read 5 Done Sent Done false
2514 1617 Acked Write 3 Done Sent Done false
2515 1618 Acked Read 2 Done Sent Done false
2516 1619 Acked Read 4 Done Sent Done false
2517 1620 Acked Write 6 Done Sent Done false
2518 1621 Acked Read 4 Done Sent Done false
2519 1622 Acked Read 4 Done Sent Done false
2520 1623 Acked Write 4 Done Sent Done false
2521 1624 Acked Read 1 Done Sent Done false
2522 1625 Acked Write 6 Done Sent Done false
2523 1626 Acked Read 2 Done Sent Done false
2524 1627 Acked Write 8 Done Sent Done false
2525 1628 Acked Write 10 Done Sent Done false
2526 1629 Acked Flush 0 Done Sent Done false
2527 1630 Acked Write 1 Done Sent Done false
2528 1631 Acked Read 2 Done Sent Done false
2529 1632 Acked Read 2 Done Sent Done false
2530 1633 Acked Read 8 Done Sent Done false
2531 1634 Acked Write 7 Done Sent Done false
2532 1635 Acked Write 10 Done Sent Done false
2533 1636 Acked Read 9 Done Sent Done false
2534 1637 Acked Read 3 Done Sent Done false
2535 1638 Acked Flush 0 Done Sent Done false
2536 1639 Acked Read 6 Done Sent Done false
2537 1640 Acked Write 4 Done Sent Done false
2538 1641 Acked Read 5 Done Sent Done false
2539 1642 Acked Read 3 Done Sent Done false
2540 1643 Acked Read 1 Done Sent Done false
2541 1644 Acked Read 9 Done Sent Done false
2542 1645 Acked Write 4 Done Sent Done false
2543 1646 Acked Write 5 Done Sent Done false
2544 1647 Acked Read 10 Done Sent Done false
2545 1648 Acked Write 8 Done Sent Done false
2546 1649 Acked Read 1 Done Sent Done false
2547 1650 Acked Read 10 Done Sent Done false
2548 1651 Acked Read 9 Done Sent Done false
2549 1652 Acked Write 10 Done Sent Done false
2550 1653 Acked Read 7 Done Sent Done false
2551 1654 Acked Write 5 Done Sent Done false
2552 1655 Acked Read 6 Done Sent Done false
2553 1656 Acked Read 1 Done Sent Done false
2554 1657 Acked Write 5 Done Sent Done false
2555 1658 Acked Flush 0 Done Sent Done false
2556 1659 Acked Read 9 Done Sent Done false
2557 1660 Acked Read 4 Done Sent Done false
2558 1661 Acked Read 9 Done Sent Done false
2559 1662 Acked Read 1 Done Sent Done false
2560 1663 Acked Write 3 Done Sent Done false
2561 1664 Acked Flush 0 Done Sent Done false
2562 1665 Acked Flush 0 Done Sent Done false
2563 1666 Acked Read 6 Done Sent Done false
2564 1667 Acked Write 7 Done Sent Done false
2565 1668 Acked Read 10 Done Sent Done false
2566 1669 Acked Write 9 Done Sent Done false
2567 1670 Acked Write 9 Done Sent Done false
2568 1671 Acked Read 5 Done Sent Done false
2569 1672 Acked Read 2 Done Sent Done false
2570 1673 Acked Write 4 Done Sent Done false
2571 1674 Acked Read 10 Done Sent Done false
2572 1675 Acked Write 9 Done Sent Done false
2573 1676 Acked Read 5 Done Sent Done false
2574 1677 Acked Write 1 Done Sent Done false
2575 1678 Acked Write 4 Done Sent Done false
2576 1679 Acked Write 10 Done Sent Done false
2577 1680 Acked Read 2 Done Sent Done false
2578 1681 Acked Read 4 Done Sent Done false
2579 1682 Acked Write 2 Done Sent Done false
2580 1683 Acked Write 2 Done Sent Done false
2581 1684 Acked Flush 0 Done Sent Done false
2582 1685 Acked Read 5 Done Sent Done false
2583 1686 Acked Read 3 Done Sent Done false
2584 1687 Acked Read 9 Done Sent Done false
2585 1688 Acked Read 6 Done Sent Done false
2586 1689 Acked Read 4 Done Sent Done false
2587 1690 Acked Read 9 Done Sent Done false
2588 1691 Acked Read 1 Done Sent Done false
2589 1692 Acked Read 2 Done Sent Done false
2590 1693 Acked Read 10 Done Sent Done false
2591 1694 Acked Read 3 Done Sent Done false
2592 1695 Acked Read 7 Done Sent Done false
2593 1696 Acked Write 4 Done Sent Done false
2594 1697 Acked Read 2 Done Sent Done false
2595 1698 Acked Read 2 Done Sent Done false
2596 1699 Acked Read 10 Done Sent Done false
2597 1700 Acked Read 9 Done Sent Done false
2598 1701 Acked Write 3 Done Sent Done false
2599 1702 Acked Read 5 Done Sent Done false
2600 1703 Acked Write 2 Done Sent Done false
2601 1704 Acked Flush 0 Done Sent Done false
2602 1705 Acked Read 4 Done Sent Done false
2603 1706 Acked Write 3 Done Sent Done false
2604 1707 Acked Flush 0 Done Sent Done false
2605 1708 Acked Read 6 Done Sent Done false
2606 1709 Acked Read 7 Done Sent Done false
2607 1710 Acked Flush 0 Done Sent Done false
2608 1711 Acked Write 7 Done Sent Done false
2609 1712 Acked Read 5 Done Sent Done false
2610 1713 Acked Write 3 Done Sent Done false
2611 1714 Acked Read 4 Done Sent Done false
2612 1715 Acked Write 6 Done Sent Done false
2613 1716 Acked Write 9 Done Sent Done false
2614 1717 Acked Read 1 Done Sent Done false
2615 1718 Acked Read 6 Done Sent Done false
2616 1719 Acked Write 3 Done Sent Done false
2617 1720 Acked Write 7 Done Sent Done false
2618 1721 Acked Read 3 Done Sent Done false
2619 1722 Acked Read 5 Done Sent Done false
2620 1723 Acked Write 9 Done Sent Done false
2621 1724 Acked Read 8 Done Sent Done false
2622 1725 Acked Write 7 Done Sent Done false
2623 1726 Acked Read 3 Done Sent Done false
2624 1727 Acked Write 9 Done Sent Done false
2625 1728 Acked Read 1 Done Sent Done false
2626 1729 Acked Read 8 Done Sent Done false
2627 1730 Acked Write 3 Done Sent Done false
2628 1731 Acked Read 6 Done Sent Done false
2629 1732 Acked Write 3 Done Sent Done false
2630 1733 Acked Write 2 Done Sent Done false
2631 1734 Acked Read 3 Done Sent Done false
2632 1735 Acked Read 5 Done Sent Done false
2633 1736 Acked Read 2 Done Sent Done false
2634 1737 Acked Write 6 Done Sent Done false
2635 1738 Acked Read 2 Done Sent Done false
2636 1739 Acked Read 3 Done Sent Done false
2637 1740 Acked Write 5 Done Sent Done false
2638 1741 Acked Write 7 Done Sent Done false
2639 1742 Acked Read 8 Done Sent Done false
2640 1743 Acked Read 8 Done Sent Done false
2641 1744 Acked Flush 0 Done Sent Done false
2642 1745 Acked Write 9 Done Sent Done false
2643 1746 Acked Write 5 Done Sent Done false
2644 1747 Acked Read 5 Done Sent Done false
2645 1748 Acked Flush 0 Done Sent Done false
2646 1749 Acked Read 4 Done Sent Done false
2647 1750 Acked Write 8 Done Sent Done false
2648 1751 Acked Read 7 Done Sent Done false
2649 1752 Acked Read 6 Done Sent Done false
2650 1753 Acked Write 10 Done Sent Done false
2651 1754 Acked Write 9 Done Sent Done false
2652 1755 Acked Write 6 Done Sent Done false
2653 1756 Acked Read 7 Done Sent Done false
2654 1757 Acked Write 8 Done Sent Done false
2655 1758 Acked Write 8 Done Sent Done false
2656 1759 Acked Read 4 Done Sent Done false
2657 1760 Acked Read 10 Done Sent Done false
2658 1761 Acked Write 6 Done Sent Done false
2659 1762 Acked Read 8 Done Sent Done false
2660 1763 Acked Write 1 Done Sent Done false
2661 1764 Acked Write 6 Done Sent Done false
2662 1765 Acked Read 4 Done Sent Done false
2663 1766 Acked Write 2 Done Sent Done false
2664 1767 Acked Flush 0 Done Sent Done false
2665 1768 Acked Write 8 Done Sent Done false
2666 1769 Acked Write 7 Done Sent Done false
2667 1770 Acked Write 4 Done Sent Done false
2668 1771 Acked Write 7 Done Sent Done false
2669 1772 Acked Read 5 Done Sent Done false
2670 1773 Acked Flush 0 Done Sent Done false
2671 1774 Acked Read 9 Done Sent Done false
2672 1775 Acked Write 6 Done Sent Done false
2673 1776 Acked Write 7 Done Sent Done false
2674 1777 Acked Write 3 Done Sent Done false
2675 1778 Acked Read 8 Done Sent Done false
2676 1779 Acked Read 10 Done Sent Done false
2677 1780 Acked Read 9 Done Sent Done false
2678 1781 Acked Flush 0 Done Sent Done false
2679 1782 Acked Read 9 Done Sent Done false
2680 1783 Acked Flush 0 Done Sent Done false
2681 1784 Acked Write 3 Done Sent Done false
2682 1785 Acked Read 2 Done Sent Done false
2683 1786 Acked Read 8 Done Sent Done false
2684 1787 Acked Read 4 Done Sent Done false
2685 1788 Acked Write 3 Done Sent Done false
2686 1789 Acked Read 6 Done Sent Done false
2687 1790 Acked Write 1 Done Sent Done false
2688 1791 Acked Read 4 Done Sent Done false
2689 1792 Acked Write 6 Done Sent Done false
2690 1793 Acked Read 3 Done Sent Done false
2691 1794 Acked Write 2 Done Sent Done false
2692 1795 Acked Write 8 Done Sent Done false
2693 1796 Acked Read 1 Done Sent Done false
2694 1797 Acked Write 4 Done Sent Done false
2695 1798 Acked Write 8 Done Sent Done false
2696 1799 Acked Read 2 Done Sent Done false
2697 1800 Acked Write 5 Done Sent Done false
2698 1801 Acked Write 1 Done Sent Done false
2699 1802 Acked Flush 0 Done Sent Done false
2700 1803 Acked Read 8 Done Sent Done false
2701 1804 Acked Read 10 Done Sent Done false
2702 1805 Acked Write 8 Done Sent Done false
2703 1806 Acked Write 10 Done Sent Done false
2704 1807 Acked Read 1 Done Sent Done false
2705 1808 Acked Write 10 Done Sent Done false
2706 1809 Acked Write 8 Done Sent Done false
2707 1810 Acked Flush 0 Done Sent Done false
2708 1811 Acked Write 3 Done Sent Done false
2709 1812 Acked Write 8 Done Sent Done false
2710 1813 Acked Read 10 Done Sent Done false
2711 1814 Acked Write 6 Done Sent Done false
2712 1815 Acked Read 8 Done Sent Done false
2713 1816 Acked Write 2 Done Sent Done false
2714 1817 Acked Read 6 Done Sent Done false
2715 1818 Acked Write 9 Done Sent Done false
2716 1819 Acked Flush 0 Done Sent Done false
2717 1820 Acked Flush 0 Done Sent Done false
2718 1821 Acked Read 1 Done Sent Done false
2719 1822 Acked Read 4 Done Sent Done false
2720 1823 Acked Read 2 Done Sent Done false
2721 1824 Acked Read 7 Done Sent Done false
2722 1825 Acked Write 2 Done Sent Done false
2723 1826 Acked Read 9 Done Sent Done false
2724 1827 Acked Write 5 Done Sent Done false
2725 1828 Acked Read 7 Done Sent Done false
2726 1829 Acked Write 6 Done Sent Done false
2727 1830 Acked Read 6 Done Sent Done false
2728 1831 Acked Read 3 Done Sent Done false
2729 1832 Acked Read 2 Done Sent Done false
2730 1833 Acked Flush 0 Done Sent Done false
2731 1834 Acked Write 6 Done Sent Done false
2732 1835 Acked Flush 0 Done Sent Done false
2733 1836 Acked Read 9 Done Sent Done false
2734 1837 Acked Read 4 Done Sent Done false
2735 1838 Acked Flush 0 Done Sent Done false
2736 1839 Acked Flush 0 Done Sent Done false
2737 1840 Acked Write 3 Done Sent Done false
2738 1841 Acked Write 3 Done Sent Done false
2739 1842 Acked Flush 0 Done Sent Done false
2740 1843 Acked Flush 0 Done Sent Done false
2741 1844 Acked Read 6 Done Sent Done false
2742 1845 Acked Write 5 Done Sent Done false
2743 1846 Acked Read 9 Done Sent Done false
2744 1847 Acked Read 8 Done Sent Done false
2745 1848 Acked Write 3 Done Sent Done false
2746 1849 Acked Read 7 Done Sent Done false
2747 1850 Acked Write 3 Done Sent Done false
2748 1851 Acked Write 6 Done Sent Done false
2749 1852 Acked Flush 0 Done Sent Done false
2750 1853 Acked Read 9 Done Sent Done false
2751 1854 Acked Read 2 Done Sent Done false
2752 1855 Acked Read 8 Done Sent Done false
2753 1856 Acked Read 5 Done Sent Done false
2754 1857 Acked Write 8 Done Sent Done false
2755 1858 Acked Read 4 Done Sent Done false
2756 1859 Acked Write 2 Done Sent Done false
2757 1860 Acked Read 1 Done Sent Done false
2758 1861 Acked Read 5 Done Sent Done false
2759 1862 Acked Write 6 Done Sent Done false
2760 1863 Acked Write 8 Done Sent Done false
2761 1864 Acked Write 1 Done Sent Done false
2762 1865 Acked Write 10 Done Sent Done false
2763 1866 Acked Read 10 Done Sent Done false
2764 1867 Acked Write 9 Done Sent Done false
2765 1868 Acked Write 3 Done Sent Done false
2766 1869 Acked Write 4 Done Sent Done false
2767 1870 Acked Read 8 Done Sent Done false
2768 1871 Acked Write 4 Done Sent Done false
2769 1872 Acked Flush 0 Done Sent Done false
2770 1873 Acked Write 8 Done Sent Done false
2771 1874 Acked Read 8 Done Sent Done false
2772 1875 Acked Read 10 Done Sent Done false
2773 1876 Acked Read 10 Done Sent Done false
2774 1877 Acked Read 1 Done Sent Done false
2775 1878 Acked Read 6 Done Sent Done false
2776 1879 Acked Write 5 Done Sent Done false
2777 1880 Acked Write 1 Done Sent Done false
2778 1881 Acked Read 9 Done Sent Done false
2779 1882 Acked Write 1 Done Sent Done false
2780 1883 Acked Read 3 Done Sent Done false
2781 1884 Acked Read 2 Done Sent Done false
2782 1885 Acked Write 7 Done Sent Done false
2783 1886 Acked Write 6 Done Sent Done false
2784 1887 Acked Write 3 Done Sent Done false
2785 1888 Acked Write 5 Done Sent Done false
2786 1889 Acked Read 1 Done Sent Done false
2787 1890 Acked Read 1 Done Sent Done false
2788 1891 Acked Write 4 Done Sent Done false
2789 1892 Acked Write 6 Done Sent Done false
2790 1893 Acked Read 3 Done Sent Done false
2791 1894 Acked Write 1 Done Sent Done false
2792 1895 Acked Read 2 Done Sent Done false
2793 1896 Acked Read 10 Done Sent Done false
2794 1897 Acked Write 3 Done Sent Done false
2795 1898 Acked Write 1 Done Sent Done false
2796 1899 Acked Flush 0 Done Sent Done false
2797 1900 Acked Read 5 Done Sent Done false
2798 1901 Acked Flush 0 Done Sent Done false
2799 1902 Acked Write 9 Done Sent Done false
2800 1903 Acked Write 5 Done Sent Done false
2801 1904 Acked Read 4 Done Sent Done false
2802 STATES DS:0 DS:1 DS:2 TOTAL
2803 Sent 0 301 0 301
2804 Done 301 0 301 602
2805 Skipped 0 0 0 0
2806 Error 0 0 0 0
2807 Last Flush: 1901 1603 1901
2808 Downstairs last five completed: 1603 1602 1601 1600 1599
2809 Upstairs last five completed: 1904 1903 1902 1901 1900
2810 CLIENT: Up:0 ds:301 act:2
2811 [8820] got start action
2812 [8820] I am going to restart
2813 Starting downstairs at port 8820
2814 Make output file at "/tmp/dsc/downstairs-8820.txt"
2815 [8820][1] reports Starting
2816 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1206)
2817 [8820][1] reports Running
2818 ----------------------------------------------------------------
2819 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
2820 States: Active Offline Active
2821 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
2822 1604 Acked Read 2 Done Sent Done false
2823 1605 Acked Write 6 Done Sent Done false
2824 1606 Acked Write 9 Done Sent Done false
2825 1607 Acked Read 5 Done Sent Done false
2826 1608 Acked Read 7 Done Sent Done false
2827 1609 Acked Write 2 Done Sent Done false
2828 1610 Acked Write 1 Done Sent Done false
2829 1611 Acked Write 9 Done Sent Done false
2830 1612 Acked Write 10 Done Sent Done false
2831 1613 Acked Write 3 Done Sent Done false
2832 1614 Acked Read 3 Done Sent Done false
2833 1615 Acked Read 8 Done Sent Done false
2834 1616 Acked Read 5 Done Sent Done false
2835 1617 Acked Write 3 Done Sent Done false
2836 1618 Acked Read 2 Done Sent Done false
2837 1619 Acked Read 4 Done Sent Done false
2838 1620 Acked Write 6 Done Sent Done false
2839 1621 Acked Read 4 Done Sent Done false
2840 1622 Acked Read 4 Done Sent Done false
2841 1623 Acked Write 4 Done Sent Done false
2842 1624 Acked Read 1 Done Sent Done false
2843 1625 Acked Write 6 Done Sent Done false
2844 1626 Acked Read 2 Done Sent Done false
2845 1627 Acked Write 8 Done Sent Done false
2846 1628 Acked Write 10 Done Sent Done false
2847 1629 Acked Flush 0 Done Sent Done false
2848 1630 Acked Write 1 Done Sent Done false
2849 1631 Acked Read 2 Done Sent Done false
2850 1632 Acked Read 2 Done Sent Done false
2851 1633 Acked Read 8 Done Sent Done false
2852 1634 Acked Write 7 Done Sent Done false
2853 1635 Acked Write 10 Done Sent Done false
2854 1636 Acked Read 9 Done Sent Done false
2855 1637 Acked Read 3 Done Sent Done false
2856 1638 Acked Flush 0 Done Sent Done false
2857 1639 Acked Read 6 Done Sent Done false
2858 1640 Acked Write 4 Done Sent Done false
2859 1641 Acked Read 5 Done Sent Done false
2860 1642 Acked Read 3 Done Sent Done false
2861 1643 Acked Read 1 Done Sent Done false
2862 1644 Acked Read 9 Done Sent Done false
2863 1645 Acked Write 4 Done Sent Done false
2864 1646 Acked Write 5 Done Sent Done false
2865 1647 Acked Read 10 Done Sent Done false
2866 1648 Acked Write 8 Done Sent Done false
2867 1649 Acked Read 1 Done Sent Done false
2868 1650 Acked Read 10 Done Sent Done false
2869 1651 Acked Read 9 Done Sent Done false
2870 1652 Acked Write 10 Done Sent Done false
2871 1653 Acked Read 7 Done Sent Done false
2872 1654 Acked Write 5 Done Sent Done false
2873 1655 Acked Read 6 Done Sent Done false
2874 1656 Acked Read 1 Done Sent Done false
2875 1657 Acked Write 5 Done Sent Done false
2876 1658 Acked Flush 0 Done Sent Done false
2877 1659 Acked Read 9 Done Sent Done false
2878 1660 Acked Read 4 Done Sent Done false
2879 1661 Acked Read 9 Done Sent Done false
2880 1662 Acked Read 1 Done Sent Done false
2881 1663 Acked Write 3 Done Sent Done false
2882 1664 Acked Flush 0 Done Sent Done false
2883 1665 Acked Flush 0 Done Sent Done false
2884 1666 Acked Read 6 Done Sent Done false
2885 1667 Acked Write 7 Done Sent Done false
2886 1668 Acked Read 10 Done Sent Done false
2887 1669 Acked Write 9 Done Sent Done false
2888 1670 Acked Write 9 Done Sent Done false
2889 1671 Acked Read 5 Done Sent Done false
2890 1672 Acked Read 2 Done Sent Done false
2891 1673 Acked Write 4 Done Sent Done false
2892 1674 Acked Read 10 Done Sent Done false
2893 1675 Acked Write 9 Done Sent Done false
2894 1676 Acked Read 5 Done Sent Done false
2895 1677 Acked Write 1 Done Sent Done false
2896 1678 Acked Write 4 Done Sent Done false
2897 1679 Acked Write 10 Done Sent Done false
2898 1680 Acked Read 2 Done Sent Done false
2899 1681 Acked Read 4 Done Sent Done false
2900 1682 Acked Write 2 Done Sent Done false
2901 1683 Acked Write 2 Done Sent Done false
2902 1684 Acked Flush 0 Done Sent Done false
2903 1685 Acked Read 5 Done Sent Done false
2904 1686 Acked Read 3 Done Sent Done false
2905 1687 Acked Read 9 Done Sent Done false
2906 1688 Acked Read 6 Done Sent Done false
2907 1689 Acked Read 4 Done Sent Done false
2908 1690 Acked Read 9 Done Sent Done false
2909 1691 Acked Read 1 Done Sent Done false
2910 1692 Acked Read 2 Done Sent Done false
2911 1693 Acked Read 10 Done Sent Done false
2912 1694 Acked Read 3 Done Sent Done false
2913 1695 Acked Read 7 Done Sent Done false
2914 1696 Acked Write 4 Done Sent Done false
2915 1697 Acked Read 2 Done Sent Done false
2916 1698 Acked Read 2 Done Sent Done false
2917 1699 Acked Read 10 Done Sent Done false
2918 1700 Acked Read 9 Done Sent Done false
2919 1701 Acked Write 3 Done Sent Done false
2920 1702 Acked Read 5 Done Sent Done false
2921 1703 Acked Write 2 Done Sent Done false
2922 1704 Acked Flush 0 Done Sent Done false
2923 1705 Acked Read 4 Done Sent Done false
2924 1706 Acked Write 3 Done Sent Done false
2925 1707 Acked Flush 0 Done Sent Done false
2926 1708 Acked Read 6 Done Sent Done false
2927 1709 Acked Read 7 Done Sent Done false
2928 1710 Acked Flush 0 Done Sent Done false
2929 1711 Acked Write 7 Done Sent Done false
2930 1712 Acked Read 5 Done Sent Done false
2931 1713 Acked Write 3 Done Sent Done false
2932 1714 Acked Read 4 Done Sent Done false
2933 1715 Acked Write 6 Done Sent Done false
2934 1716 Acked Write 9 Done Sent Done false
2935 1717 Acked Read 1 Done Sent Done false
2936 1718 Acked Read 6 Done Sent Done false
2937 1719 Acked Write 3 Done Sent Done false
2938 1720 Acked Write 7 Done Sent Done false
2939 1721 Acked Read 3 Done Sent Done false
2940 1722 Acked Read 5 Done Sent Done false
2941 1723 Acked Write 9 Done Sent Done false
2942 1724 Acked Read 8 Done Sent Done false
2943 1725 Acked Write 7 Done Sent Done false
2944 1726 Acked Read 3 Done Sent Done false
2945 1727 Acked Write 9 Done Sent Done false
2946 1728 Acked Read 1 Done Sent Done false
2947 1729 Acked Read 8 Done Sent Done false
2948 1730 Acked Write 3 Done Sent Done false
2949 1731 Acked Read 6 Done Sent Done false
2950 1732 Acked Write 3 Done Sent Done false
2951 1733 Acked Write 2 Done Sent Done false
2952 1734 Acked Read 3 Done Sent Done false
2953 1735 Acked Read 5 Done Sent Done false
2954 1736 Acked Read 2 Done Sent Done false
2955 1737 Acked Write 6 Done Sent Done false
2956 1738 Acked Read 2 Done Sent Done false
2957 1739 Acked Read 3 Done Sent Done false
2958 1740 Acked Write 5 Done Sent Done false
2959 1741 Acked Write 7 Done Sent Done false
2960 1742 Acked Read 8 Done Sent Done false
2961 1743 Acked Read 8 Done Sent Done false
2962 1744 Acked Flush 0 Done Sent Done false
2963 1745 Acked Write 9 Done Sent Done false
2964 1746 Acked Write 5 Done Sent Done false
2965 1747 Acked Read 5 Done Sent Done false
2966 1748 Acked Flush 0 Done Sent Done false
2967 1749 Acked Read 4 Done Sent Done false
2968 1750 Acked Write 8 Done Sent Done false
2969 1751 Acked Read 7 Done Sent Done false
2970 1752 Acked Read 6 Done Sent Done false
2971 1753 Acked Write 10 Done Sent Done false
2972 1754 Acked Write 9 Done Sent Done false
2973 1755 Acked Write 6 Done Sent Done false
2974 1756 Acked Read 7 Done Sent Done false
2975 1757 Acked Write 8 Done Sent Done false
2976 1758 Acked Write 8 Done Sent Done false
2977 1759 Acked Read 4 Done Sent Done false
2978 1760 Acked Read 10 Done Sent Done false
2979 1761 Acked Write 6 Done Sent Done false
2980 1762 Acked Read 8 Done Sent Done false
2981 1763 Acked Write 1 Done Sent Done false
2982 1764 Acked Write 6 Done Sent Done false
2983 1765 Acked Read 4 Done Sent Done false
2984 1766 Acked Write 2 Done Sent Done false
2985 1767 Acked Flush 0 Done Sent Done false
2986 1768 Acked Write 8 Done Sent Done false
2987 1769 Acked Write 7 Done Sent Done false
2988 1770 Acked Write 4 Done Sent Done false
2989 1771 Acked Write 7 Done Sent Done false
2990 1772 Acked Read 5 Done Sent Done false
2991 1773 Acked Flush 0 Done Sent Done false
2992 1774 Acked Read 9 Done Sent Done false
2993 1775 Acked Write 6 Done Sent Done false
2994 1776 Acked Write 7 Done Sent Done false
2995 1777 Acked Write 3 Done Sent Done false
2996 1778 Acked Read 8 Done Sent Done false
2997 1779 Acked Read 10 Done Sent Done false
2998 1780 Acked Read 9 Done Sent Done false
2999 1781 Acked Flush 0 Done Sent Done false
3000 1782 Acked Read 9 Done Sent Done false
3001 1783 Acked Flush 0 Done Sent Done false
3002 1784 Acked Write 3 Done Sent Done false
3003 1785 Acked Read 2 Done Sent Done false
3004 1786 Acked Read 8 Done Sent Done false
3005 1787 Acked Read 4 Done Sent Done false
3006 1788 Acked Write 3 Done Sent Done false
3007 1789 Acked Read 6 Done Sent Done false
3008 1790 Acked Write 1 Done Sent Done false
3009 1791 Acked Read 4 Done Sent Done false
3010 1792 Acked Write 6 Done Sent Done false
3011 1793 Acked Read 3 Done Sent Done false
3012 1794 Acked Write 2 Done Sent Done false
3013 1795 Acked Write 8 Done Sent Done false
3014 1796 Acked Read 1 Done Sent Done false
3015 1797 Acked Write 4 Done Sent Done false
3016 1798 Acked Write 8 Done Sent Done false
3017 1799 Acked Read 2 Done Sent Done false
3018 1800 Acked Write 5 Done Sent Done false
3019 1801 Acked Write 1 Done Sent Done false
3020 1802 Acked Flush 0 Done Sent Done false
3021 1803 Acked Read 8 Done Sent Done false
3022 1804 Acked Read 10 Done Sent Done false
3023 1805 Acked Write 8 Done Sent Done false
3024 1806 Acked Write 10 Done Sent Done false
3025 1807 Acked Read 1 Done Sent Done false
3026 1808 Acked Write 10 Done Sent Done false
3027 1809 Acked Write 8 Done Sent Done false
3028 1810 Acked Flush 0 Done Sent Done false
3029 1811 Acked Write 3 Done Sent Done false
3030 1812 Acked Write 8 Done Sent Done false
3031 1813 Acked Read 10 Done Sent Done false
3032 1814 Acked Write 6 Done Sent Done false
3033 1815 Acked Read 8 Done Sent Done false
3034 1816 Acked Write 2 Done Sent Done false
3035 1817 Acked Read 6 Done Sent Done false
3036 1818 Acked Write 9 Done Sent Done false
3037 1819 Acked Flush 0 Done Sent Done false
3038 1820 Acked Flush 0 Done Sent Done false
3039 1821 Acked Read 1 Done Sent Done false
3040 1822 Acked Read 4 Done Sent Done false
3041 1823 Acked Read 2 Done Sent Done false
3042 1824 Acked Read 7 Done Sent Done false
3043 1825 Acked Write 2 Done Sent Done false
3044 1826 Acked Read 9 Done Sent Done false
3045 1827 Acked Write 5 Done Sent Done false
3046 1828 Acked Read 7 Done Sent Done false
3047 1829 Acked Write 6 Done Sent Done false
3048 1830 Acked Read 6 Done Sent Done false
3049 1831 Acked Read 3 Done Sent Done false
3050 1832 Acked Read 2 Done Sent Done false
3051 1833 Acked Flush 0 Done Sent Done false
3052 1834 Acked Write 6 Done Sent Done false
3053 1835 Acked Flush 0 Done Sent Done false
3054 1836 Acked Read 9 Done Sent Done false
3055 1837 Acked Read 4 Done Sent Done false
3056 1838 Acked Flush 0 Done Sent Done false
3057 1839 Acked Flush 0 Done Sent Done false
3058 1840 Acked Write 3 Done Sent Done false
3059 1841 Acked Write 3 Done Sent Done false
3060 1842 Acked Flush 0 Done Sent Done false
3061 1843 Acked Flush 0 Done Sent Done false
3062 1844 Acked Read 6 Done Sent Done false
3063 1845 Acked Write 5 Done Sent Done false
3064 1846 Acked Read 9 Done Sent Done false
3065 1847 Acked Read 8 Done Sent Done false
3066 1848 Acked Write 3 Done Sent Done false
3067 1849 Acked Read 7 Done Sent Done false
3068 1850 Acked Write 3 Done Sent Done false
3069 1851 Acked Write 6 Done Sent Done false
3070 1852 Acked Flush 0 Done Sent Done false
3071 1853 Acked Read 9 Done Sent Done false
3072 1854 Acked Read 2 Done Sent Done false
3073 1855 Acked Read 8 Done Sent Done false
3074 1856 Acked Read 5 Done Sent Done false
3075 1857 Acked Write 8 Done Sent Done false
3076 1858 Acked Read 4 Done Sent Done false
3077 1859 Acked Write 2 Done Sent Done false
3078 1860 Acked Read 1 Done Sent Done false
3079 1861 Acked Read 5 Done Sent Done false
3080 1862 Acked Write 6 Done Sent Done false
3081 1863 Acked Write 8 Done Sent Done false
3082 1864 Acked Write 1 Done Sent Done false
3083 1865 Acked Write 10 Done Sent Done false
3084 1866 Acked Read 10 Done Sent Done false
3085 1867 Acked Write 9 Done Sent Done false
3086 1868 Acked Write 3 Done Sent Done false
3087 1869 Acked Write 4 Done Sent Done false
3088 1870 Acked Read 8 Done Sent Done false
3089 1871 Acked Write 4 Done Sent Done false
3090 1872 Acked Flush 0 Done Sent Done false
3091 1873 Acked Write 8 Done Sent Done false
3092 1874 Acked Read 8 Done Sent Done false
3093 1875 Acked Read 10 Done Sent Done false
3094 1876 Acked Read 10 Done Sent Done false
3095 1877 Acked Read 1 Done Sent Done false
3096 1878 Acked Read 6 Done Sent Done false
3097 1879 Acked Write 5 Done Sent Done false
3098 1880 Acked Write 1 Done Sent Done false
3099 1881 Acked Read 9 Done Sent Done false
3100 1882 Acked Write 1 Done Sent Done false
3101 1883 Acked Read 3 Done Sent Done false
3102 1884 Acked Read 2 Done Sent Done false
3103 1885 Acked Write 7 Done Sent Done false
3104 1886 Acked Write 6 Done Sent Done false
3105 1887 Acked Write 3 Done Sent Done false
3106 1888 Acked Write 5 Done Sent Done false
3107 1889 Acked Read 1 Done Sent Done false
3108 1890 Acked Read 1 Done Sent Done false
3109 1891 Acked Write 4 Done Sent Done false
3110 1892 Acked Write 6 Done Sent Done false
3111 1893 Acked Read 3 Done Sent Done false
3112 1894 Acked Write 1 Done Sent Done false
3113 1895 Acked Read 2 Done Sent Done false
3114 1896 Acked Read 10 Done Sent Done false
3115 1897 Acked Write 3 Done Sent Done false
3116 1898 Acked Write 1 Done Sent Done false
3117 1899 Acked Flush 0 Done Sent Done false
3118 1900 Acked Read 5 Done Sent Done false
3119 1901 Acked Flush 0 Done Sent Done false
3120 1902 Acked Write 9 Done Sent Done false
3121 1903 Acked Write 5 Done Sent Done false
3122 1904 Acked Read 4 Done Sent Done false
3123 1905 Acked Flush 0 Done Sent Done false
3124 STATES DS:0 DS:1 DS:2 TOTAL
3125 Sent 0 302 0 302
3126 Done 302 0 302 604
3127 Skipped 0 0 0 0
3128 Error 0 0 0 0
3129 Last Flush: 1905 1603 1905
3130 Downstairs last five completed: 1603 1602 1601 1600 1599
3131 Upstairs last five completed: 1905 1904 1903 1902 1901
3132 CLIENT: Up:0 ds:302 act:2
31332024-11-26T22:05:10.902ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
31342024-11-26T22:05:10.903ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:36700) = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
31352024-11-26T22:05:10.904ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
31362024-11-26T22:05:10.904ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
31372024-11-26T22:05:10.904ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
31382024-11-26T22:05:10.904ZINFOcrucible: send last flush ID to this DS: 1603 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
31392024-11-26T22:05:10.904ZINFOcrucible: Replied this last flush ID: 1603 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
31402024-11-26T22:05:10.904ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
31412024-11-26T22:05:10.904ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
31422024-11-26T22:05:10.904ZINFOcrucible: [1] client re-new 302 jobs since flush 1603 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
31432024-11-26T22:05:10.905ZINFOcrucible: [1] Replayed 302 jobs since flush: 1603 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
3144 ----------------------------------------------------------------
3145 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
3146 Downstairs last five completed: 1905 1904 1903 1902 1901
3147 Upstairs last five completed: 1905 1904 1903 1902 1901
3148 CLIENT: Up:0 ds:0 act:3
3149 Replay: All jobs finished
3150 Main task has work to do, go find it
3151 got dsc Stop(2)
3152 stop 2
3153 Nov 26 22:05:13.139 INFO request completed, latency_us: 379, response_code: [8830] Got stop action so:false kr:false
3154 204, uri: /stop/cid/2, method: POST, req_id: 223fc128-25ee-4832-bd4e-a8533f300992, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
31552024-11-26T22:05:13.140ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
3156 Nov 26 22:05:13.140 {"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2024-11-26T22:05:13.140265042Z","hostname":"w-01JDN7ZQ17ABQG15KHKN3F2D26","pid":1203,"":"io task","client":"2","":"downstairs","session_id":"86a36c17-3718-484c-87fd-2b8a1dc2ab18"}
3157 INFO request completed, latency_us: 131, response_code: 200, uri: /state/cid/2, method: GET{"msg":"downstairs task for 2 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2024-11-26T22:05:13.140301907Z","hostname":"w-01JDN7ZQ17ABQG15KHKN3F2D26","pid":1203,"session_id":"86a36c17-3718-484c-87fd-2b8a1dc2ab18"}
3158 , req_id: 1988c898-f4f4-46c5-913b-f3e279811844, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.{"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-26T22:05:13.140333303Z","hostname":"w-01JDN7ZQ17ABQG15KHKN3F2D26","pid":1203,"client":"2","":"downstairs","session_id":"86a36c17-3718-484c-87fd-2b8a1dc2ab18"}
3159 1:9998
31602024-11-26T22:05:13.140ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
31612024-11-26T22:05:13.140ZINFOcrucible: client task is exiting = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
3162 [8830] Exited with: ExitStatus(unix_wait_status(9))
3163 [8830][2] reports Exit
3164 Nov 26 22:05:17.141 INFO request completed, latency_us: 211, response_code: 200, uri: /state/cid/2, method: GET, req_id: a58202f5-995b-4173-99bc-c434c3a8b838, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
3165 001/300 Read block 2734 len 28672
3166 002/300 Write block 2811 len 4096 data: 2
3167 003/300 Write block 1890 len 8192 data: 3 3
3168 004/300 Read block 1040 len 20480
3169 005/300 Flush
3170 006/300 Write block 485 len 28672 data: 2 2 2 3 3 3 3
3171 007/300 Write block 3782 len 28672 data: 3 3 3 3 2 2 2
3172 008/300 Write block 4053 len 32768 data: 2 2 2 2 2 2 3 3
3173 009/300 Read block 3064 len 32768
3174 010/300 Write block 864 len 28672 data: 3 3 3 3 3 2 3
3175 011/300 Write block 2901 len 32768 data: 3 3 4 3 3 3 3 3
3176 012/300 Read block 2875 len 28672
3177 013/300 Read block 4513 len 16384
3178 014/300 Read block 2765 len 32768
3179 015/300 Read block 3874 len 24576
3180 016/300 Read block 3486 len 36864
3181 017/300 Read block 1421 len 8192
3182 018/300 Flush
3183 019/300 Write block 4791 len 16384 data: 3 3 3 3
3184 020/300 Read block 723 len 8192
3185 021/300 Write block 4586 len 36864 data: 2 3 3 3 2 2 2 2 2
3186 022/300 Write block 1499 len 8192 data: 3 3
3187 023/300 Write block 481 len 8192 data: 3 3
3188 024/300 Read block 25 len 36864
3189 025/300 Read block 3749 len 20480
3190 026/300 Write block 639 len 24576 data: 2 2 2 2 2 2
3191 027/300 Read block 3412 len 40960
3192 028/300 Flush
3193 029/300 Write block 1683 len 40960 data: 2 2 2 2 2 2 2 2 2 2
3194 030/300 Write block 834 len 4096 data: 2
3195 031/300 Flush
3196 032/300 Read block 3686 len 8192
3197 033/300 Write block 2657 len 12288 data: 2 2 2
3198 034/300 Read block 736 len 8192
3199 035/300 Read block 414 len 4096
3200 036/300 Read block 2354 len 8192
3201 037/300 Write block 45 len 20480 data: 2 3 3 3 3
3202 038/300 Read block 1809 len 4096
3203 039/300 Read block 3303 len 8192
3204 040/300 Read block 3386 len 28672
3205 041/300 Read block 2684 len 32768
3206 042/300 Write block 4871 len 40960 data: 2 2 2 2 2 2 2 2 2 2
3207 043/300 Write block 379 len 24576 data: 2 2 2 2 2 2
3208 044/300 Flush
3209 045/300 Write block 4192 len 36864 data: 2 2 2 2 2 2 2 2 2
3210 046/300 Read block 3782 len 32768
3211 047/300 Write block 21 len 40960 data: 3 3 3 3 3 2 2 3 2 2
3212 048/300 Read block 3777 len 36864
3213 049/300 Write block 727 len 36864 data: 2 2 2 2 2 2 2 2 2
3214 050/300 Read block 1185 len 24576
3215 051/300 Read block 1196 len 40960
3216 052/300 Write block 3768 len 8192 data: 2 2
3217 053/300 Flush
3218 054/300 Write block 3730 len 8192 data: 3 3
3219 055/300 Read block 2954 len 12288
3220 056/300 Read block 4377 len 8192
3221 057/300 Write block 3676 len 40960 data: 5 5 4 3 3 3 3 3 2 2
3222 058/300 Flush
3223 059/300 Read block 359 len 40960
3224 060/300 Read block 1811 len 40960
3225 061/300 Write block 3037 len 16384 data: 2 2 2 2
3226 062/300 Read block 4321 len 4096
3227 063/300 Flush
3228 064/300 Read block 330 len 28672
3229 065/300 Read block 4672 len 40960
3230 066/300 Read block 4756 len 12288
3231 067/300 Read block 1100 len 36864
3232 068/300 Write block 4436 len 16384 data: 4 4 4 4
3233 069/300 Read block 675 len 32768
3234 070/300 Read block 2504 len 20480
3235 071/300 Write block 4593 len 20480 data: 3 3 2 2 2
3236 072/300 Read block 2584 len 8192
3237 073/300 Write block 1757 len 24576 data: 2 2 2 2 3 3
3238 074/300 Read block 4108 len 12288
3239 075/300 Read block 789 len 28672
3240 076/300 Write block 2806 len 32768 data: 2 2 2 2 2 3 2 2
3241 077/300 Read block 2570 len 8192
3242 078/300 Write block 2607 len 24576 data: 2 2 2 2 3 2
3243 079/300 Flush
3244 080/300 Write block 4405 len 32768 data: 2 2 2 2 2 2 2 2
3245 081/300 Write block 3934 len 4096 data: 2
3246 082/300 Write block 3140 len 32768 data: 2 2 2 2 2 2 2 2
3247 083/300 Read block 1714 len 36864
3248 084/300 Read block 1053 len 8192
3249 085/300 Read block 4155 len 4096
3250 086/300 Write block 1911 len 4096 data: 2
3251 087/300 Write block 1875 len 4096 data: 2
3252 088/300 Write block 482 len 36864 data: 4 3 2 3 3 3 4 4 4
3253 089/300 Write block 4452 len 8192 data: 2 2
3254 090/300 Write block 183 len 24576 data: 3 3 3 2 2 2
3255 091/300 Read block 3364 len 8192
3256 092/300 Flush
3257 093/300 Write block 4327 len 40960 data: 3 3 2 2 2 2 3 3 3 3
3258 094/300 Write block 4787 len 12288 data: 2 2 3
3259 095/300 Write block 3531 len 24576 data: 2 2 2 2 2 2
3260 096/300 Write block 4920 len 8192 data: 2 2
3261 097/300 Write block 3192 len 12288 data: 2 2 2
3262 098/300 Read block 2990 len 28672
3263 099/300 Flush
3264 100/300 Write block 540 len 36864 data: 2 2 2 2 2 2 2 2 2
3265 101/300 Read block 1248 len 28672
3266 102/300 Read block 3640 len 32768
3267 103/300 Read block 3728 len 16384
3268 104/300 Flush
3269 105/300 Read block 26 len 24576
3270 106/300 Flush
3271 107/300 Write block 3661 len 28672 data: 2 2 2 3 3 3 3
3272 108/300 Write block 965 len 20480 data: 2 2 2 2 2
3273 109/300 Write block 4361 len 40960 data: 3 3 2 2 2 2 2 2 2 2
3274 110/300 Read block 1253 len 32768
3275 111/300 Write block 275 len 36864 data: 2 2 2 2 2 2 3 3 3
3276 112/300 Read block 4817 len 8192
3277 113/300 Write block 188 len 28672 data: 3 2 2 2 2 2 2
3278 114/300 Read block 4394 len 40960
3279 115/300 Flush
3280 116/300 Write block 2618 len 12288 data: 2 2 2
3281 117/300 Read block 3745 len 36864
3282 118/300 Write block 4385 len 12288 data: 3 3 3
3283 119/300 Read block 4244 len 32768
3284 120/300 Write block 4223 len 20480 data: 2 2 2 2 2
3285 121/300 Read block 936 len 40960
3286 122/300 Read block 735 len 20480
3287 123/300 Flush
3288 124/300 Flush
3289 125/300 Read block 1400 len 40960
3290 126/300 Read block 4944 len 20480
3291 127/300 Read block 518 len 24576
3292 128/300 Write block 2904 len 4096 data: 4
3293 129/300 Write block 1525 len 24576 data: 3 3 3 4 4 4
3294 130/300 Read block 2484 len 12288
3295 131/300 Read block 2620 len 12288
3296 132/300 Read block 4585 len 36864
3297 133/300 Flush
3298 134/300 Write block 2735 len 4096 data: 3
3299 135/300 Flush
3300 136/300 Flush
3301 137/300 Read block 2544 len 24576
3302 138/300 Read block 2550 len 40960
3303 139/300 Read block 1380 len 16384
3304 140/300 Read block 709 len 28672
3305 141/300 Read block 4811 len 28672
3306 142/300 Flush
3307 143/300 Write block 190 len 24576 data: 3 3 3 3 3 2
3308 144/300 Flush
3309 145/300 Read block 691 len 32768
3310 146/300 Read block 4727 len 8192
3311 147/300 Write block 469 len 32768 data: 2 2 2 2 2 2 2 2
3312 148/300 Write block 3570 len 24576 data: 3 3 3 3 3 2
3313 149/300 Read block 4584 len 36864
3314 150/300 Write block 4290 len 16384 data: 2 2 2 2
3315 151/300 Read block 1676 len 16384
3316 152/300 Flush
3317 153/300 Read block 1968 len 40960
3318 154/300 Write block 623 len 40960 data: 3 3 2 2 2 2 2 2 2 2
3319 155/300 Write block 4105 len 32768 data: 2 2 2 2 2 2 2 2
3320 156/300 Read block 4235 len 4096
3321 157/300 Read block 2823 len 4096
3322 158/300 Write block 3396 len 16384 data: 4 4 4 3
3323 159/300 Read block 2419 len 24576
3324 160/300 Read block 2278 len 40960
3325 161/300 Flush
3326 162/300 Write block 512 len 16384 data: 3 3 3 3
3327 163/300 Read block 2247 len 4096
3328 164/300 Write block 3540 len 12288 data: 3 3 3
3329 165/300 Read block 3421 len 16384
3330 166/300 Read block 3606 len 4096
3331 167/300 Read block 4106 len 8192
3332 168/300 Read block 3817 len 8192
3333 169/300 Read block 1799 len 12288
3334 170/300 Flush
3335 171/300 Read block 4013 len 24576
3336 172/300 Flush
3337 173/300 Write block 12 len 12288 data: 2 2 2
3338 174/300 Read block 2392 len 16384
3339 175/300 Write block 122 len 36864 data: 2 2 2 2 2 2 2 2 2
3340 176/300 Write block 4433 len 4096 data: 2
3341 177/300 Read block 3239 len 12288
3342 178/300 Read block 4375 len 28672
3343 179/300 Write block 4444 len 36864 data: 2 2 2 2 3 2 2 2 3
3344 180/300 Read block 540 len 36864
3345 181/300 Read block 763 len 40960
3346 182/300 Write block 1616 len 20480 data: 2 2 2 2 2
3347 183/300 Flush
3348 184/300 Read block 3458 len 16384
3349 185/300 Read block 417 len 20480
3350 186/300 Flush
3351 187/300 Write block 2838 len 36864 data: 3 3 3 3 3 3 2 2 3
3352 188/300 Read block 3358 len 16384
3353 189/300 Read block 2874 len 20480
3354 190/300 Read block 1005 len 8192
3355 191/300 Write block 3947 len 40960 data: 2 2 2 2 2 2 2 2 2 2
3356 192/300 Write block 3716 len 16384 data: 3 3 3 3
3357 193/300 Write block 3755 len 4096 data: 2
3358 194/300 Read block 1063 len 36864
3359 195/300 Read block 4472 len 32768
3360 196/300 Read block 4165 len 12288
3361 197/300 Read block 1293 len 12288
3362 198/300 Read block 2201 len 16384
3363 199/300 Read block 176 len 16384
3364 200/300 Flush
3365 201/300 Read block 3034 len 32768
3366 202/300 Read block 2698 len 16384
3367 203/300 Write block 1092 len 28672 data: 2 2 2 3 2 2 2
3368 204/300 Write block 885 len 32768 data: 2 2 2 2 2 2 2 2
3369 205/300 Write block 4041 len 40960 data: 2 2 3 3 3 3 3 2 2 2
3370 206/300 Write block 3601 len 8192 data: 2 2
3371 207/300 Write block 941 len 40960 data: 2 2 2 2 2 2 2 2 2 2
3372 208/300 Read block 2089 len 28672
3373 209/300 Write block 2454 len 36864 data: 2 2 2 2 2 2 2 2 2
3374 210/300 Read block 4469 len 8192
3375 211/300 Write block 1944 len 20480 data: 3 3 2 2 2
3376 212/300 Write block 4230 len 24576 data: 3 3 3 3 3 2
3377 213/300 Write block 2032 len 32768 data: 2 2 3 3 3 3 3 3
3378 214/300 Write block 2455 len 4096 data: 3
3379 215/300 Write block 1828 len 24576 data: 2 2 2 2 2 2
3380 216/300 Read block 3812 len 20480
3381 217/300 Read block 1690 len 28672
3382 218/300 Read block 1982 len 40960
3383 219/300 Flush
3384 220/300 Read block 4725 len 12288
3385 221/300 Write block 2422 len 12288 data: 2 2 2
3386 222/300 Read block 4523 len 32768
3387 223/300 Write block 2500 len 32768 data: 3 2 2 2 2 2 2 2
3388 224/300 Write block 1821 len 8192 data: 2 2
3389 225/300 Write block 2648 len 4096 data: 2
3390 226/300 Write block 2480 len 36864 data: 3 3 3 3 3 3 3 2 2
3391 227/300 Read block 4835 len 16384
3392 228/300 Read block 2343 len 4096
3393 229/300 Write block 4356 len 20480 data: 2 2 2 2 3
3394 230/300 Write block 1133 len 4096 data: 2
3395 231/300 Flush
3396 232/300 Write block 3010 len 36864 data: 4 4 3 3 3 3 3 3 3
3397 233/300 Read block 4825 len 40960
3398 234/300 Read block 2458 len 16384
3399 235/300 Read block 4185 len 32768
3400 236/300 Write block 3784 len 4096 data: 4
3401 237/300 Flush
3402 238/300 Write block 2915 len 12288 data: 3 3 3
3403 239/300 Read block 4070 len 8192
3404 240/300 Read block 231 len 16384
3405 241/300 Write block 4989 len 36864 data: 2 2 2 2 2 2 2 2 2
3406 242/300 Read block 4391 len 32768
3407 243/300 Read block 617 len 36864
3408 244/300 Write block 2465 len 36864 data: 2 2 3 3 3 3 3 3 3
3409 245/300 Flush
3410 246/300 Read block 4901 len 24576
3411 247/300 Write block 4513 len 12288 data: 2 2 3
3412 248/300 Flush
3413 249/300 Flush
3414 250/300 Read block 2846 len 12288
3415 251/300 Write block 800 len 20480 data: 2 2 2 2 2
3416 252/300 Read block 2919 len 24576
3417 253/300 Flush
3418 254/300 Write block 4102 len 16384 data: 2 2 2 3
3419 255/300 Write block 1986 len 36864 data: 2 2 2 2 2 2 2 2 2
3420 256/300 Read block 3590 len 32768
3421 257/300 Write block 1343 len 24576 data: 3 2 2 2 2 2
3422 258/300 Read block 3067 len 40960
3423 259/300 Read block 1698 len 40960
3424 260/300 Write block 2758 len 16384 data: 3 3 3 3
3425 261/300 Read block 4812 len 20480
3426 262/300 Write block 4987 len 40960 data: 2 2 3 3 3 3 3 3 3 3
3427 263/300 Read block 2277 len 24576
3428 264/300 Write block 3118 len 20480 data: 2 2 2 2 2
3429 265/300 Write block 828 len 16384 data: 2 2 2 2
3430 266/300 Read block 4450 len 8192
3431 267/300 Read block 1332 len 40960
3432 268/300 Read block 2534 len 16384
3433 269/300 Write block 885 len 36864 data: 3 3 3 3 3 3 3 3 2
3434 270/300 Read block 2819 len 32768
3435 271/300 Read block 4625 len 12288
3436 272/300 Read block 3521 len 40960
3437 273/300 Write block 2658 len 32768 data: 3 3 2 2 2 2 2 2
3438 274/300 Write block 873 len 12288 data: 2 3 3
3439 275/300 Flush
3440 276/300 Write block 672 len 36864 data: 2 2 2 2 2 2 2 2 2
3441 277/300 Write block 516 len 40960 data: 3 3 2 2 2 2 2 2 2 2
3442 278/300 Write block 4185 len 8192 data: 2 2
3443 279/300 Read block 72 len 40960
3444 280/300 Read block 4472 len 28672
3445 281/300 Read block 1641 len 8192
3446 282/300 Write block 3609 len 28672 data: 2 2 2 2 2 2 2
3447 283/300 Write block 466 len 40960 data: 3 2 2 3 3 3 3 3 3 3
3448 284/300 Read block 3287 len 36864
3449 285/300 Read block 629 len 28672
3450 286/300 Write block 3682 len 28672 data: 4 4 3 3 2 2 2
3451 287/300 Read block 4456 len 28672
3452 288/300 Read block 1088 len 16384
3453 289/300 Read block 3882 len 20480
3454 290/300 Read block 425 len 12288
3455 291/300 Flush
3456 292/300 Read block 2137 len 36864
3457 293/300 Read block 1248 len 16384
3458 294/300 Read block 876 len 32768
3459 295/300 Write block 870 len 12288 data: 4 3 3
3460 296/300 Read block 468 len 4096
3461 297/300 Read block 1574 len 28672
3462 298/300 Write block 3312 len 4096 data: 2
3463 299/300 Write block 121 len 40960 data: 2 3 3 3 3 3 3 3 3 3
3464 300/300 Read block 4072 len 12288
3465 301/300 Read block 4855 len 20480
3466 Main task has work to do, go find it
3467 got dsc Start(2)
3468 start 2
3469 Nov 26 22:05:17.378 INFO request completed, latency_us: 206, response_code: 204, uri: /start/cid/2, method: POST, req_id: 3cb45a19-4764-4e6c-911b-92b09a618819, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
3470 Replay: started 2, returned:Ok(())
3471 ----------------------------------------------------------------
3472 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
3473 States: Active Active Offline
3474 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
3475 1906 Acked Read 7 Done Done Sent false
3476 1907 Acked Write 1 Done Done Sent false
3477 1908 Acked Write 2 Done Done Sent false
3478 1909 Acked Read 5 Done Done Sent false
3479 1910 Acked Flush 0 Done Done Sent false
3480 1911 Acked Write 7 Done Done Sent false
3481 1912 Acked Write 7 Done Done Sent false
3482 1913 Acked Write 8 Done Done Sent false
3483 1914 Acked Read 8 Done Done Sent false
3484 1915 Acked Write 7 Done Done Sent false
3485 1916 Acked Write 8 Done Done Sent false
3486 1917 Acked Read 7 Done Done Sent false
3487 1918 Acked Read 4 Done Done Sent false
3488 1919 Acked Read 8 Done Done Sent false
3489 1920 Acked Read 6 Done Done Sent false
3490 1921 Acked Read 9 Done Done Sent false
3491 1922 Acked Read 2 Done Done Sent false
3492 1923 Acked Flush 0 Done Done Sent false
3493 1924 Acked Write 4 Done Done Sent false
3494 1925 Acked Read 2 Done Done Sent false
3495 1926 Acked Write 9 Done Done Sent false
3496 1927 Acked Write 2 Done Done Sent false
3497 1928 Acked Write 2 Done Done Sent false
3498 1929 Acked Read 9 Done Done Sent false
3499 1930 Acked Read 5 Done Done Sent false
3500 1931 Acked Write 6 Done Done Sent false
3501 1932 Acked Read 10 Done Done Sent false
3502 1933 Acked Flush 0 Done Done Sent false
3503 1934 Acked Write 10 Done Done Sent false
3504 1935 Acked Write 1 Done Done Sent false
3505 1936 Acked Flush 0 Done Done Sent false
3506 1937 Acked Read 2 Done Done Sent false
3507 1938 Acked Write 3 Done Done Sent false
3508 1939 Acked Read 2 Done Done Sent false
3509 1940 Acked Read 1 Done Done Sent false
3510 1941 Acked Read 2 Done Done Sent false
3511 1942 Acked Write 5 Done Done Sent false
3512 1943 Acked Read 1 Done Done Sent false
3513 1944 Acked Read 2 Done Done Sent false
3514 1945 Acked Read 7 Done Done Sent false
3515 1946 Acked Read 8 Done Done Sent false
3516 1947 Acked Write 10 Done Done Sent false
3517 1948 Acked Write 6 Done Done Sent false
3518 1949 Acked Flush 0 Done Done Sent false
3519 1950 Acked Write 9 Done Done Sent false
3520 1951 Acked Read 8 Done Done Sent false
3521 1952 Acked Write 10 Done Done Sent false
3522 1953 Acked Read 9 Done Done Sent false
3523 1954 Acked Write 9 Done Done Sent false
3524 1955 Acked Read 6 Done Done Sent false
3525 1956 Acked Read 10 Done Done Sent false
3526 1957 Acked Write 2 Done Done Sent false
3527 1958 Acked Flush 0 Done Done Sent false
3528 1959 Acked Write 2 Done Done Sent false
3529 1960 Acked Read 3 Done Done Sent false
3530 1961 Acked Read 2 Done Done Sent false
3531 1962 Acked Write 10 Done Done Sent false
3532 1963 Acked Flush 0 Done Done Sent false
3533 1964 Acked Read 10 Done Done Sent false
3534 1965 Acked Read 10 Done Done Sent false
3535 1966 Acked Write 4 Done Done Sent false
3536 1967 Acked Read 1 Done Done Sent false
3537 1968 Acked Flush 0 Done Done Sent false
3538 1969 Acked Read 7 Done Done Sent false
3539 1970 Acked Read 10 Done Done Sent false
3540 1971 Acked Read 3 Done Done Sent false
3541 1972 Acked Read 9 Done Done Sent false
3542 1973 Acked Write 4 Done Done Sent false
3543 1974 Acked Read 8 Done Done Sent false
3544 1975 Acked Read 5 Done Done Sent false
3545 1976 Acked Write 5 Done Done Sent false
3546 1977 Acked Read 2 Done Done Sent false
3547 1978 Acked Write 6 Done Done Sent false
3548 1979 Acked Read 3 Done Done Sent false
3549 1980 Acked Read 7 Done Done Sent false
3550 1981 Acked Write 8 Done Done Sent false
3551 1982 Acked Read 2 Done Done Sent false
3552 1983 Acked Write 6 Done Done Sent false
3553 1984 Acked Flush 0 Done Done Sent false
3554 1985 Acked Write 8 Done Done Sent false
3555 1986 Acked Write 1 Done Done Sent false
3556 1987 Acked Write 8 Done Done Sent false
3557 1988 Acked Read 9 Done Done Sent false
3558 1989 Acked Read 2 Done Done Sent false
3559 1990 Acked Read 1 Done Done Sent false
3560 1991 Acked Write 1 Done Done Sent false
3561 1992 Acked Write 1 Done Done Sent false
3562 1993 Acked Write 9 Done Done Sent false
3563 1994 Acked Write 2 Done Done Sent false
3564 1995 Acked Write 6 Done Done Sent false
3565 1996 Acked Read 2 Done Done Sent false
3566 1997 Acked Flush 0 Done Done Sent false
3567 1998 Acked Write 10 Done Done Sent false
3568 1999 Acked Write 3 Done Done Sent false
3569 2000 Acked Write 6 Done Done Sent false
3570 2001 Acked Write 2 Done Done Sent false
3571 2002 Acked Write 3 Done Done Sent false
3572 2003 Acked Read 7 Done Done Sent false
3573 2004 Acked Flush 0 Done Done Sent false
3574 2005 Acked Write 9 Done Done Sent false
3575 2006 Acked Read 7 Done Done Sent false
3576 2007 Acked Read 8 Done Done Sent false
3577 2008 Acked Read 4 Done Done Sent false
3578 2009 Acked Flush 0 Done Done Sent false
3579 2010 Acked Read 6 Done Done Sent false
3580 2011 Acked Flush 0 Done Done Sent false
3581 2012 Acked Write 7 Done Done Sent false
3582 2013 Acked Write 5 Done Done Sent false
3583 2014 Acked Write 10 Done Done Sent false
3584 2015 Acked Read 8 Done Done Sent false
3585 2016 Acked Write 9 Done Done Sent false
3586 2017 Acked Read 2 Done Done Sent false
3587 2018 Acked Write 7 Done Done Sent false
3588 2019 Acked Read 10 Done Done Sent false
3589 2020 Acked Flush 0 Done Done Sent false
3590 2021 Acked Write 3 Done Done Sent false
3591 2022 Acked Read 9 Done Done Sent false
3592 2023 Acked Write 3 Done Done Sent false
3593 2024 Acked Read 8 Done Done Sent false
3594 2025 Acked Write 5 Done Done Sent false
3595 2026 Acked Read 10 Done Done Sent false
3596 2027 Acked Read 5 Done Done Sent false
3597 2028 Acked Flush 0 Done Done Sent false
3598 2029 Acked Flush 0 Done Done Sent false
3599 2030 Acked Read 10 Done Done Sent false
3600 2031 Acked Read 5 Done Done Sent false
3601 2032 Acked Read 6 Done Done Sent false
3602 2033 Acked Write 1 Done Done Sent false
3603 2034 Acked Write 6 Done Done Sent false
3604 2035 Acked Read 3 Done Done Sent false
3605 2036 Acked Read 3 Done Done Sent false
3606 2037 Acked Read 9 Done Done Sent false
3607 2038 Acked Flush 0 Done Done Sent false
3608 2039 Acked Write 1 Done Done Sent false
3609 2040 Acked Flush 0 Done Done Sent false
3610 2041 Acked Flush 0 Done Done Sent false
3611 2042 Acked Read 6 Done Done Sent false
3612 2043 Acked Read 10 Done Done Sent false
3613 2044 Acked Read 4 Done Done Sent false
3614 2045 Acked Read 7 Done Done Sent false
3615 2046 Acked Read 7 Done Done Sent false
3616 2047 Acked Flush 0 Done Done Sent false
3617 2048 Acked Write 6 Done Done Sent false
3618 2049 Acked Flush 0 Done Done Sent false
3619 2050 Acked Read 8 Done Done Sent false
3620 2051 Acked Read 2 Done Done Sent false
3621 2052 Acked Write 8 Done Done Sent false
3622 2053 Acked Write 6 Done Done Sent false
3623 2054 Acked Read 9 Done Done Sent false
3624 2055 Acked Write 4 Done Done Sent false
3625 2056 Acked Read 4 Done Done Sent false
3626 2057 Acked Flush 0 Done Done Sent false
3627 2058 Acked Read 10 Done Done Sent false
3628 2059 Acked Write 10 Done Done Sent false
3629 2060 Acked Write 8 Done Done Sent false
3630 2061 Acked Read 1 Done Done Sent false
3631 2062 Acked Read 1 Done Done Sent false
3632 2063 Acked Write 4 Done Done Sent false
3633 2064 Acked Read 6 Done Done Sent false
3634 2065 Acked Read 10 Done Done Sent false
3635 2066 Acked Flush 0 Done Done Sent false
3636 2067 Acked Write 4 Done Done Sent false
3637 2068 Acked Read 1 Done Done Sent false
3638 2069 Acked Write 3 Done Done Sent false
3639 2070 Acked Read 4 Done Done Sent false
3640 2071 Acked Read 1 Done Done Sent false
3641 2072 Acked Read 2 Done Done Sent false
3642 2073 Acked Read 2 Done Done Sent false
3643 2074 Acked Read 3 Done Done Sent false
3644 2075 Acked Flush 0 Done Done Sent false
3645 2076 Acked Read 6 Done Done Sent false
3646 2077 Acked Flush 0 Done Done Sent false
3647 2078 Acked Write 3 Done Done Sent false
3648 2079 Acked Read 4 Done Done Sent false
3649 2080 Acked Write 9 Done Done Sent false
3650 2081 Acked Write 1 Done Done Sent false
3651 2082 Acked Read 3 Done Done Sent false
3652 2083 Acked Read 7 Done Done Sent false
3653 2084 Acked Write 9 Done Done Sent false
3654 2085 Acked Read 9 Done Done Sent false
3655 2086 Acked Read 10 Done Done Sent false
3656 2087 Acked Write 5 Done Done Sent false
3657 2088 Acked Flush 0 Done Done Sent false
3658 2089 Acked Read 4 Done Done Sent false
3659 2090 Acked Read 5 Done Done Sent false
3660 2091 Acked Flush 0 Done Done Sent false
3661 2092 Acked Write 9 Done Done Sent false
3662 2093 Acked Read 4 Done Done Sent false
3663 2094 Acked Read 5 Done Done Sent false
3664 2095 Acked Read 2 Done Done Sent false
3665 2096 Acked Write 10 Done Done Sent false
3666 2097 Acked Write 4 Done Done Sent false
3667 2098 Acked Write 1 Done Done Sent false
3668 2099 Acked Read 9 Done Done Sent false
3669 2100 Acked Read 8 Done Done Sent false
3670 2101 Acked Read 3 Done Done Sent false
3671 2102 Acked Read 3 Done Done Sent false
3672 2103 Acked Read 4 Done Done Sent false
3673 2104 Acked Read 4 Done Done Sent false
3674 2105 Acked Flush 0 Done Done Sent false
3675 2106 Acked Read 8 Done Done Sent false
3676 2107 Acked Read 4 Done Done Sent false
3677 2108 Acked Write 7 Done Done Sent false
3678 2109 Acked Write 8 Done Done Sent false
3679 2110 Acked Write 10 Done Done Sent false
3680 2111 Acked Write 2 Done Done Sent false
3681 2112 Acked Write 10 Done Done Sent false
3682 2113 Acked Read 7 Done Done Sent false
3683 2114 Acked Write 9 Done Done Sent false
3684 2115 Acked Read 2 Done Done Sent false
3685 2116 Acked Write 5 Done Done Sent false
3686 2117 Acked Write 6 Done Done Sent false
3687 2118 Acked Write 8 Done Done Sent false
3688 2119 Acked Write 1 Done Done Sent false
3689 2120 Acked Write 6 Done Done Sent false
3690 2121 Acked Read 5 Done Done Sent false
3691 2122 Acked Read 7 Done Done Sent false
3692 2123 Acked Read 10 Done Done Sent false
3693 2124 Acked Flush 0 Done Done Sent false
3694 2125 Acked Read 3 Done Done Sent false
3695 2126 Acked Write 3 Done Done Sent false
3696 2127 Acked Read 8 Done Done Sent false
3697 2128 Acked Write 8 Done Done Sent false
3698 2129 Acked Write 2 Done Done Sent false
3699 2130 Acked Write 1 Done Done Sent false
3700 2131 Acked Write 9 Done Done Sent false
3701 2132 Acked Read 4 Done Done Sent false
3702 2133 Acked Read 1 Done Done Sent false
3703 2134 Acked Write 5 Done Done Sent false
3704 2135 Acked Write 1 Done Done Sent false
3705 2136 Acked Flush 0 Done Done Sent false
3706 2137 Acked Write 9 Done Done Sent false
3707 2138 Acked Read 10 Done Done Sent false
3708 2139 Acked Read 4 Done Done Sent false
3709 2140 Acked Read 8 Done Done Sent false
3710 2141 Acked Write 1 Done Done Sent false
3711 2142 Acked Flush 0 Done Done Sent false
3712 2143 Acked Write 3 Done Done Sent false
3713 2144 Acked Read 2 Done Done Sent false
3714 2145 Acked Read 4 Done Done Sent false
3715 2146 Acked Write 9 Done Done Sent false
3716 2147 Acked Read 8 Done Done Sent false
3717 2148 Acked Read 9 Done Done Sent false
3718 2149 Acked Write 9 Done Done Sent false
3719 2150 Acked Flush 0 Done Done Sent false
3720 2151 Acked Read 6 Done Done Sent false
3721 2152 Acked Write 3 Done Done Sent false
3722 2153 Acked Flush 0 Done Done Sent false
3723 2154 Acked Flush 0 Done Done Sent false
3724 2155 Acked Read 3 Done Done Sent false
3725 2156 Acked Write 5 Done Done Sent false
3726 2157 Acked Read 6 Done Done Sent false
3727 2158 Acked Flush 0 Done Done Sent false
3728 2159 Acked Write 4 Done Done Sent false
3729 2160 Acked Write 9 Done Done Sent false
3730 2161 Acked Read 8 Done Done Sent false
3731 2162 Acked Write 6 Done Done Sent false
3732 2163 Acked Read 10 Done Done Sent false
3733 2164 Acked Read 10 Done Done Sent false
3734 2165 Acked Write 4 Done Done Sent false
3735 2166 Acked Read 5 Done Done Sent false
3736 2167 Acked Write 10 Done Done Sent false
3737 2168 Acked Read 6 Done Done Sent false
3738 2169 Acked Write 5 Done Done Sent false
3739 2170 Acked Write 4 Done Done Sent false
3740 2171 Acked Read 2 Done Done Sent false
3741 2172 Acked Read 10 Done Done Sent false
3742 2173 Acked Read 4 Done Done Sent false
3743 2174 Acked Write 9 Done Done Sent false
3744 2175 Acked Read 8 Done Done Sent false
3745 2176 Acked Read 3 Done Done Sent false
3746 2177 Acked Read 10 Done Done Sent false
3747 2178 Acked Write 8 Done Done Sent false
3748 2179 Acked Write 3 Done Done Sent false
3749 2180 Acked Flush 0 Done Done Sent false
3750 2181 Acked Write 9 Done Done Sent false
3751 2182 Acked Write 10 Done Done Sent false
3752 2183 Acked Write 2 Done Done Sent false
3753 2184 Acked Read 10 Done Done Sent false
3754 2185 Acked Read 7 Done Done Sent false
3755 2186 Acked Read 2 Done Done Sent false
3756 2187 Acked Write 7 Done Done Sent false
3757 2188 Acked Write 10 Done Done Sent false
3758 2189 Acked Read 9 Done Done Sent false
3759 2190 Acked Read 7 Done Done Sent false
3760 2191 Acked Write 7 Done Done Sent false
3761 2192 Acked Read 7 Done Done Sent false
3762 2193 Acked Read 4 Done Done Sent false
3763 2194 Acked Read 5 Done Done Sent false
3764 2195 Acked Read 3 Done Done Sent false
3765 2196 Acked Flush 0 Done Done Sent false
3766 2197 Acked Read 9 Done Done Sent false
3767 2198 Acked Read 4 Done Done Sent false
3768 2199 Acked Read 8 Done Done Sent false
3769 2200 Acked Write 3 Done Done Sent false
3770 2201 Acked Read 1 Done Done Sent false
3771 2202 Acked Read 7 Done Done Sent false
3772 2203 Acked Write 1 Done Done Sent false
3773 2204 Acked Write 10 Done Done Sent false
3774 2205 Acked Read 3 Done Done Sent false
3775 2206 Acked Read 5 Done Done Sent false
3776 STATES DS:0 DS:1 DS:2 TOTAL
3777 Sent 0 0 301 301
3778 Done 301 301 0 602
3779 Skipped 0 0 0 0
3780 Error 0 0 0 0
3781 Last Flush: 2196 2196 1905
3782 Downstairs last five completed: 1905 1904 1903 1902 1901
3783 Upstairs last five completed: 2206 2205 2204 2203 2202
3784 CLIENT: Up:0 ds:301 act:2
3785 ----------------------------------------------------------------
3786 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
3787 States: Active Active Offline
3788 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
3789 1906 Acked Read 7 Done Done Sent false
3790 1907 Acked Write 1 Done Done Sent false
3791 1908 Acked Write 2 Done Done Sent false
3792 1909 Acked Read 5 Done Done Sent false
3793 1910 Acked Flush 0 Done Done Sent false
3794 1911 Acked Write 7 Done Done Sent false
3795 1912 Acked Write 7 Done Done Sent false
3796 1913 Acked Write 8 Done Done Sent false
3797 1914 Acked Read 8 Done Done Sent false
3798 1915 Acked Write 7 Done Done Sent false
3799 1916 Acked Write 8 Done Done Sent false
3800 1917 Acked Read 7 Done Done Sent false
3801 1918 Acked Read 4 Done Done Sent false
3802 1919 Acked Read 8 Done Done Sent false
3803 1920 Acked Read 6 Done Done Sent false
3804 1921 Acked Read 9 Done Done Sent false
3805 1922 Acked Read 2 Done Done Sent false
3806 1923 Acked Flush 0 Done Done Sent false
3807 1924 Acked Write 4 Done Done Sent false
3808 1925 Acked Read 2 Done Done Sent false
3809 1926 Acked Write 9 Done Done Sent false
3810 1927 Acked Write 2 Done Done Sent false
3811 1928 Acked Write 2 Done Done Sent false
3812 1929 Acked Read 9 Done Done Sent false
3813 1930 Acked Read 5 Done Done Sent false
3814 1931 Acked Write 6 Done Done Sent false
3815 1932 Acked Read 10 Done Done Sent false
3816 1933 Acked Flush 0 Done Done Sent false
3817 1934 Acked Write 10 Done Done Sent false
3818 1935 Acked Write 1 Done Done Sent false
3819 1936 Acked Flush 0 Done Done Sent false
3820 1937 Acked Read 2 Done Done Sent false
3821 1938 Acked Write 3 Done Done Sent false
3822 1939 Acked Read 2 Done Done Sent false
3823 1940 Acked Read 1 Done Done Sent false
3824 1941 Acked Read 2 Done Done Sent false
3825 1942 Acked Write 5 Done Done Sent false
3826 1943 Acked Read 1 Done Done Sent false
3827 1944 Acked Read 2 Done Done Sent false
3828 1945 Acked Read 7 Done Done Sent false
3829 1946 Acked Read 8 Done Done Sent false
3830 1947 Acked Write 10 Done Done Sent false
3831 1948 Acked Write 6 Done Done Sent false
3832 1949 Acked Flush 0 Done Done Sent false
3833 1950 Acked Write 9 Done Done Sent false
3834 1951 Acked Read 8 Done Done Sent false
3835 1952 Acked Write 10 Done Done Sent false
3836 1953 Acked Read 9 Done Done Sent false
3837 1954 Acked Write 9 Done Done Sent false
3838 1955 Acked Read 6 Done Done Sent false
3839 1956 Acked Read 10 Done Done Sent false
3840 1957 Acked Write 2 Done Done Sent false
3841 1958 Acked Flush 0 Done Done Sent false
3842 1959 Acked Write 2 Done Done Sent false
3843 1960 Acked Read 3 Done Done Sent false
3844 1961 Acked Read 2 Done Done Sent false
3845 1962 Acked Write 10 Done Done Sent false
3846 1963 Acked Flush 0 Done Done Sent false
3847 1964 Acked Read 10 Done Done Sent false
3848 1965 Acked Read 10 Done Done Sent false
3849 1966 Acked Write 4 Done Done Sent false
3850 1967 Acked Read 1 Done Done Sent false
3851 1968 Acked Flush 0 Done Done Sent false
3852 1969 Acked Read 7 Done Done Sent false
3853 1970 Acked Read 10 Done Done Sent false
3854 1971 Acked Read 3 Done Done Sent false
3855 1972 Acked Read 9 Done Done Sent false
3856 1973 Acked Write 4 Done Done Sent false
3857 1974 Acked Read 8 Done Done Sent false
3858 1975 Acked Read 5 Done Done Sent false
3859 1976 Acked Write 5 Done Done Sent false
3860 1977 Acked Read 2 Done Done Sent false
3861 1978 Acked Write 6 Done Done Sent false
3862 1979 Acked Read 3 Done Done Sent false
3863 1980 Acked Read 7 Done Done Sent false
3864 1981 Acked Write 8 Done Done Sent false
3865 1982 Acked Read 2 Done Done Sent false
3866 1983 Acked Write 6 Done Done Sent false
3867 1984 Acked Flush 0 Done Done Sent false
3868 1985 Acked Write 8 Done Done Sent false
3869 1986 Acked Write 1 Done Done Sent false
3870 1987 Acked Write 8 Done Done Sent false
3871 1988 Acked Read 9 Done Done Sent false
3872 1989 Acked Read 2 Done Done Sent false
3873 1990 Acked Read 1 Done Done Sent false
3874 1991 Acked Write 1 Done Done Sent false
3875 1992 Acked Write 1 Done Done Sent false
3876 1993 Acked Write 9 Done Done Sent false
3877 1994 Acked Write 2 Done Done Sent false
3878 1995 Acked Write 6 Done Done Sent false
3879 1996 Acked Read 2 Done Done Sent false
3880 1997 Acked Flush 0 Done Done Sent false
3881 1998 Acked Write 10 Done Done Sent false
3882 1999 Acked Write 3 Done Done Sent false
3883 2000 Acked Write 6 Done Done Sent false
3884 2001 Acked Write 2 Done Done Sent false
3885 2002 Acked Write 3 Done Done Sent false
3886 2003 Acked Read 7 Done Done Sent false
3887 2004 Acked Flush 0 Done Done Sent false
3888 2005 Acked Write 9 Done Done Sent false
3889 2006 Acked Read 7 Done Done Sent false
3890 2007 Acked Read 8 Done Done Sent false
3891 2008 Acked Read 4 Done Done Sent false
3892 2009 Acked Flush 0 Done Done Sent false
3893 2010 Acked Read 6 Done Done Sent false
3894 2011 Acked Flush 0 Done Done Sent false
3895 2012 Acked Write 7 Done Done Sent false
3896 2013 Acked Write 5 Done Done Sent false
3897 2014 Acked Write 10 Done Done Sent false
3898 2015 Acked Read 8 Done Done Sent false
3899 2016 Acked Write 9 Done Done Sent false
3900 2017 Acked Read 2 Done Done Sent false
3901 2018 Acked Write 7 Done Done Sent false
3902 2019 Acked Read 10 Done Done Sent false
3903 2020 Acked Flush 0 Done Done Sent false
3904 2021 Acked Write 3 Done Done Sent false
3905 2022 Acked Read 9 Done Done Sent false
3906 2023 Acked Write 3 Done Done Sent false
3907 2024 Acked Read 8 Done Done Sent false
3908 2025 Acked Write 5 Done Done Sent false
3909 2026 Acked Read 10 Done Done Sent false
3910 2027 Acked Read 5 Done Done Sent false
3911 2028 Acked Flush 0 Done Done Sent false
3912 2029 Acked Flush 0 Done Done Sent false
3913 2030 Acked Read 10 Done Done Sent false
3914 2031 Acked Read 5 Done Done Sent false
3915 2032 Acked Read 6 Done Done Sent false
3916 2033 Acked Write 1 Done Done Sent false
3917 2034 Acked Write 6 Done Done Sent false
3918 2035 Acked Read 3 Done Done Sent false
3919 2036 Acked Read 3 Done Done Sent false
3920 2037 Acked Read 9 Done Done Sent false
3921 2038 Acked Flush 0 Done Done Sent false
3922 2039 Acked Write 1 Done Done Sent false
3923 2040 Acked Flush 0 Done Done Sent false
3924 2041 Acked Flush 0 Done Done Sent false
3925 2042 Acked Read 6 Done Done Sent false
3926 2043 Acked Read 10 Done Done Sent false
3927 2044 Acked Read 4 Done Done Sent false
3928 2045 Acked Read 7 Done Done Sent false
3929 2046 Acked Read 7 Done Done Sent false
3930 2047 Acked Flush 0 Done Done Sent false
3931 2048 Acked Write 6 Done Done Sent false
3932 2049 Acked Flush 0 Done Done Sent false
3933 2050 Acked Read 8 Done Done Sent false
3934 2051 Acked Read 2 Done Done Sent false
3935 2052 Acked Write 8 Done Done Sent false
3936 2053 Acked Write 6 Done Done Sent false
3937 2054 Acked Read 9 Done Done Sent false
3938 2055 Acked Write 4 Done Done Sent false
3939 2056 Acked Read 4 Done Done Sent false
3940 2057 Acked Flush 0 Done Done Sent false
3941 2058 Acked Read 10 Done Done Sent false
3942 2059 Acked Write 10 Done Done Sent false
3943 2060 Acked Write 8 Done Done Sent false
3944 2061 Acked Read 1 Done Done Sent false
3945 2062 Acked Read 1 Done Done Sent false
3946 2063 Acked Write 4 Done Done Sent false
3947 2064 Acked Read 6 Done Done Sent false
3948 2065 Acked Read 10 Done Done Sent false
3949 2066 Acked Flush 0 Done Done Sent false
3950 2067 Acked Write 4 Done Done Sent false
3951 2068 Acked Read 1 Done Done Sent false
3952 2069 Acked Write 3 Done Done Sent false
3953 2070 Acked Read 4 Done Done Sent false
3954 2071 Acked Read 1 Done Done Sent false
3955 2072 Acked Read 2 Done Done Sent false
3956 2073 Acked Read 2 Done Done Sent false
3957 2074 Acked Read 3 Done Done Sent false
3958 2075 Acked Flush 0 Done Done Sent false
3959 2076 Acked Read 6 Done Done Sent false
3960 2077 Acked Flush 0 Done Done Sent false
3961 2078 Acked Write 3 Done Done Sent false
3962 2079 Acked Read 4 Done Done Sent false
3963 2080 Acked Write 9 Done Done Sent false
3964 2081 Acked Write 1 Done Done Sent false
3965 2082 Acked Read 3 Done Done Sent false
3966 2083 Acked Read 7 Done Done Sent false
3967 2084 Acked Write 9 Done Done Sent false
3968 2085 Acked Read 9 Done Done Sent false
3969 2086 Acked Read 10 Done Done Sent false
3970 2087 Acked Write 5 Done Done Sent false
3971 2088 Acked Flush 0 Done Done Sent false
3972 2089 Acked Read 4 Done Done Sent false
3973 2090 Acked Read 5 Done Done Sent false
3974 2091 Acked Flush 0 Done Done Sent false
3975 2092 Acked Write 9 Done Done Sent false
3976 2093 Acked Read 4 Done Done Sent false
3977 2094 Acked Read 5 Done Done Sent false
3978 2095 Acked Read 2 Done Done Sent false
3979 2096 Acked Write 10 Done Done Sent false
3980 2097 Acked Write 4 Done Done Sent false
3981 2098 Acked Write 1 Done Done Sent false
3982 2099 Acked Read 9 Done Done Sent false
3983 2100 Acked Read 8 Done Done Sent false
3984 2101 Acked Read 3 Done Done Sent false
3985 2102 Acked Read 3 Done Done Sent false
3986 2103 Acked Read 4 Done Done Sent false
3987 2104 Acked Read 4 Done Done Sent false
3988 2105 Acked Flush 0 Done Done Sent false
3989 2106 Acked Read 8 Done Done Sent false
3990 2107 Acked Read 4 Done Done Sent false
3991 2108 Acked Write 7 Done Done Sent false
3992 2109 Acked Write 8 Done Done Sent false
3993 2110 Acked Write 10 Done Done Sent false
3994 2111 Acked Write 2 Done Done Sent false
3995 2112 Acked Write 10 Done Done Sent false
3996 2113 Acked Read 7 Done Done Sent false
3997 2114 Acked Write 9 Done Done Sent false
3998 2115 Acked Read 2 Done Done Sent false
3999 2116 Acked Write 5 Done Done Sent false
4000 2117 Acked Write 6 Done Done Sent false
4001 2118 Acked Write 8 Done Done Sent false
4002 2119 Acked Write 1 Done Done Sent false
4003 2120 Acked Write 6 Done Done Sent false
4004 2121 Acked Read 5 Done Done Sent false
4005 2122 Acked Read 7 Done Done Sent false
4006 2123 Acked Read 10 Done Done Sent false
4007 2124 Acked Flush 0 Done Done Sent false
4008 2125 Acked Read 3 Done Done Sent false
4009 2126 Acked Write 3 Done Done Sent false
4010 2127 Acked Read 8 Done Done Sent false
4011 2128 Acked Write 8 Done Done Sent false
4012 2129 Acked Write 2 Done Done Sent false
4013 2130 Acked Write 1 Done Done Sent false
4014 2131 Acked Write 9 Done Done Sent false
4015 2132 Acked Read 4 Done Done Sent false
4016 2133 Acked Read 1 Done Done Sent false
4017 2134 Acked Write 5 Done Done Sent false
4018 2135 Acked Write 1 Done Done Sent false
4019 2136 Acked Flush 0 Done Done Sent false
4020 2137 Acked Write 9 Done Done Sent false
4021 2138 Acked Read 10 Done Done Sent false
4022 2139 Acked Read 4 Done Done Sent false
4023 2140 Acked Read 8 Done Done Sent false
4024 2141 Acked Write 1 Done Done Sent false
4025 2142 Acked Flush 0 Done Done Sent false
4026 2143 Acked Write 3 Done Done Sent false
4027 2144 Acked Read 2 Done Done Sent false
4028 2145 Acked Read 4 Done Done Sent false
4029 2146 Acked Write 9 Done Done Sent false
4030 2147 Acked Read 8 Done Done Sent false
4031 2148 Acked Read 9 Done Done Sent false
4032 2149 Acked Write 9 Done Done Sent false
4033 2150 Acked Flush 0 Done Done Sent false
4034 2151 Acked Read 6 Done Done Sent false
4035 2152 Acked Write 3 Done Done Sent false
4036 2153 Acked Flush 0 Done Done Sent false
4037 2154 Acked Flush 0 Done Done Sent false
4038 2155 Acked Read 3 Done Done Sent false
4039 2156 Acked Write 5 Done Done Sent false
4040 2157 Acked Read 6 Done Done Sent false
4041 2158 Acked Flush 0 Done Done Sent false
4042 2159 Acked Write 4 Done Done Sent false
4043 2160 Acked Write 9 Done Done Sent false
4044 2161 Acked Read 8 Done Done Sent false
4045 2162 Acked Write 6 Done Done Sent false
4046 2163 Acked Read 10 Done Done Sent false
4047 2164 Acked Read 10 Done Done Sent false
4048 2165 Acked Write 4 Done Done Sent false
4049 2166 Acked Read 5 Done Done Sent false
4050 2167 Acked Write 10 Done Done Sent false
4051 2168 Acked Read 6 Done Done Sent false
4052 2169 Acked Write 5 Done Done Sent false
4053 2170 Acked Write 4 Done Done Sent false
4054 2171 Acked Read 2 Done Done Sent false
4055 2172 Acked Read 10 Done Done Sent false
4056 2173 Acked Read 4 Done Done Sent false
4057 2174 Acked Write 9 Done Done Sent false
4058 2175 Acked Read 8 Done Done Sent false
4059 2176 Acked Read 3 Done Done Sent false
4060 2177 Acked Read 10 Done Done Sent false
4061 2178 Acked Write 8 Done Done Sent false
4062 2179 Acked Write 3 Done Done Sent false
4063 2180 Acked Flush 0 Done Done Sent false
4064 2181 Acked Write 9 Done Done Sent false
4065 2182 Acked Write 10 Done Done Sent false
4066 2183 Acked Write 2 Done Done Sent false
4067 2184 Acked Read 10 Done Done Sent false
4068 2185 Acked Read 7 Done Done Sent false
4069 2186 Acked Read 2 Done Done Sent false
4070 2187 Acked Write 7 Done Done Sent false
4071 2188 Acked Write 10 Done Done Sent false
4072 2189 Acked Read 9 Done Done Sent false
4073 2190 Acked Read 7 Done Done Sent false
4074 2191 Acked Write 7 Done Done Sent false
4075 2192 Acked Read 7 Done Done Sent false
4076 2193 Acked Read 4 Done Done Sent false
4077 2194 Acked Read 5 Done Done Sent false
4078 2195 Acked Read 3 Done Done Sent false
4079 2196 Acked Flush 0 Done Done Sent false
4080 2197 Acked Read 9 Done Done Sent false
4081 2198 Acked Read 4 Done Done Sent false
4082 2199 Acked Read 8 Done Done Sent false
4083 2200 Acked Write 3 Done Done Sent false
4084 2201 Acked Read 1 Done Done Sent false
4085 2202 Acked Read 7 Done Done Sent false
4086 2203 Acked Write 1 Done Done Sent false
4087 2204 Acked Write 10 Done Done Sent false
4088 2205 Acked Read 3 Done Done Sent false
4089 2206 Acked Read 5 Done Done Sent false
4090 2207 Acked Flush 0 Done Done Sent false
4091 STATES DS:0 DS:1 DS:2 TOTAL
4092 Sent 0 0 302 302
4093 Done 302 302 0 604
4094 Skipped 0 0 0 0
4095 Error 0 0 0 0
4096 Last Flush: 2207 2207 1905
4097 Downstairs last five completed: 1905 1904 1903 1902 1901
4098 Upstairs last five completed: 2207 2206 2205 2204 2203
4099 CLIENT: Up:0 ds:302 act:2
41002024-11-26T22:05:23.141ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
41012024-11-26T22:05:23.141ZWARNcrucible: ds_connection connect to 127.0.0.1:8830 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
41022024-11-26T22:05:23.141ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
41032024-11-26T22:05:23.141ZWARNcrucible: downstairs task for 2 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
41042024-11-26T22:05:23.141ZINFOcrucible: client task is exiting = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
4105 [8830] got start action
4106 [8830] I am going to restart
4107 Starting downstairs at port 8830
4108 Make output file at "/tmp/dsc/downstairs-8830.txt"
4109 [8830][2] reports Starting
4110 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1207)
4111 [8830][2] reports Running
4112 ----------------------------------------------------------------
4113 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
4114 States: Active Active Offline
4115 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
4116 1906 Acked Read 7 Done Done Sent false
4117 1907 Acked Write 1 Done Done Sent false
4118 1908 Acked Write 2 Done Done Sent false
4119 1909 Acked Read 5 Done Done Sent false
4120 1910 Acked Flush 0 Done Done Sent false
4121 1911 Acked Write 7 Done Done Sent false
4122 1912 Acked Write 7 Done Done Sent false
4123 1913 Acked Write 8 Done Done Sent false
4124 1914 Acked Read 8 Done Done Sent false
4125 1915 Acked Write 7 Done Done Sent false
4126 1916 Acked Write 8 Done Done Sent false
4127 1917 Acked Read 7 Done Done Sent false
4128 1918 Acked Read 4 Done Done Sent false
4129 1919 Acked Read 8 Done Done Sent false
4130 1920 Acked Read 6 Done Done Sent false
4131 1921 Acked Read 9 Done Done Sent false
4132 1922 Acked Read 2 Done Done Sent false
4133 1923 Acked Flush 0 Done Done Sent false
4134 1924 Acked Write 4 Done Done Sent false
4135 1925 Acked Read 2 Done Done Sent false
4136 1926 Acked Write 9 Done Done Sent false
4137 1927 Acked Write 2 Done Done Sent false
4138 1928 Acked Write 2 Done Done Sent false
4139 1929 Acked Read 9 Done Done Sent false
4140 1930 Acked Read 5 Done Done Sent false
4141 1931 Acked Write 6 Done Done Sent false
4142 1932 Acked Read 10 Done Done Sent false
4143 1933 Acked Flush 0 Done Done Sent false
4144 1934 Acked Write 10 Done Done Sent false
4145 1935 Acked Write 1 Done Done Sent false
4146 1936 Acked Flush 0 Done Done Sent false
4147 1937 Acked Read 2 Done Done Sent false
4148 1938 Acked Write 3 Done Done Sent false
4149 1939 Acked Read 2 Done Done Sent false
4150 1940 Acked Read 1 Done Done Sent false
4151 1941 Acked Read 2 Done Done Sent false
4152 1942 Acked Write 5 Done Done Sent false
4153 1943 Acked Read 1 Done Done Sent false
4154 1944 Acked Read 2 Done Done Sent false
4155 1945 Acked Read 7 Done Done Sent false
4156 1946 Acked Read 8 Done Done Sent false
4157 1947 Acked Write 10 Done Done Sent false
4158 1948 Acked Write 6 Done Done Sent false
4159 1949 Acked Flush 0 Done Done Sent false
4160 1950 Acked Write 9 Done Done Sent false
4161 1951 Acked Read 8 Done Done Sent false
4162 1952 Acked Write 10 Done Done Sent false
4163 1953 Acked Read 9 Done Done Sent false
4164 1954 Acked Write 9 Done Done Sent false
4165 1955 Acked Read 6 Done Done Sent false
4166 1956 Acked Read 10 Done Done Sent false
4167 1957 Acked Write 2 Done Done Sent false
4168 1958 Acked Flush 0 Done Done Sent false
4169 1959 Acked Write 2 Done Done Sent false
4170 1960 Acked Read 3 Done Done Sent false
4171 1961 Acked Read 2 Done Done Sent false
4172 1962 Acked Write 10 Done Done Sent false
4173 1963 Acked Flush 0 Done Done Sent false
4174 1964 Acked Read 10 Done Done Sent false
4175 1965 Acked Read 10 Done Done Sent false
4176 1966 Acked Write 4 Done Done Sent false
4177 1967 Acked Read 1 Done Done Sent false
4178 1968 Acked Flush 0 Done Done Sent false
4179 1969 Acked Read 7 Done Done Sent false
4180 1970 Acked Read 10 Done Done Sent false
4181 1971 Acked Read 3 Done Done Sent false
4182 1972 Acked Read 9 Done Done Sent false
4183 1973 Acked Write 4 Done Done Sent false
4184 1974 Acked Read 8 Done Done Sent false
4185 1975 Acked Read 5 Done Done Sent false
4186 1976 Acked Write 5 Done Done Sent false
4187 1977 Acked Read 2 Done Done Sent false
4188 1978 Acked Write 6 Done Done Sent false
4189 1979 Acked Read 3 Done Done Sent false
4190 1980 Acked Read 7 Done Done Sent false
4191 1981 Acked Write 8 Done Done Sent false
4192 1982 Acked Read 2 Done Done Sent false
4193 1983 Acked Write 6 Done Done Sent false
4194 1984 Acked Flush 0 Done Done Sent false
4195 1985 Acked Write 8 Done Done Sent false
4196 1986 Acked Write 1 Done Done Sent false
4197 1987 Acked Write 8 Done Done Sent false
4198 1988 Acked Read 9 Done Done Sent false
4199 1989 Acked Read 2 Done Done Sent false
4200 1990 Acked Read 1 Done Done Sent false
4201 1991 Acked Write 1 Done Done Sent false
4202 1992 Acked Write 1 Done Done Sent false
4203 1993 Acked Write 9 Done Done Sent false
4204 1994 Acked Write 2 Done Done Sent false
4205 1995 Acked Write 6 Done Done Sent false
4206 1996 Acked Read 2 Done Done Sent false
4207 1997 Acked Flush 0 Done Done Sent false
4208 1998 Acked Write 10 Done Done Sent false
4209 1999 Acked Write 3 Done Done Sent false
4210 2000 Acked Write 6 Done Done Sent false
4211 2001 Acked Write 2 Done Done Sent false
4212 2002 Acked Write 3 Done Done Sent false
4213 2003 Acked Read 7 Done Done Sent false
4214 2004 Acked Flush 0 Done Done Sent false
4215 2005 Acked Write 9 Done Done Sent false
4216 2006 Acked Read 7 Done Done Sent false
4217 2007 Acked Read 8 Done Done Sent false
4218 2008 Acked Read 4 Done Done Sent false
4219 2009 Acked Flush 0 Done Done Sent false
4220 2010 Acked Read 6 Done Done Sent false
4221 2011 Acked Flush 0 Done Done Sent false
4222 2012 Acked Write 7 Done Done Sent false
4223 2013 Acked Write 5 Done Done Sent false
4224 2014 Acked Write 10 Done Done Sent false
4225 2015 Acked Read 8 Done Done Sent false
4226 2016 Acked Write 9 Done Done Sent false
4227 2017 Acked Read 2 Done Done Sent false
4228 2018 Acked Write 7 Done Done Sent false
4229 2019 Acked Read 10 Done Done Sent false
4230 2020 Acked Flush 0 Done Done Sent false
4231 2021 Acked Write 3 Done Done Sent false
4232 2022 Acked Read 9 Done Done Sent false
4233 2023 Acked Write 3 Done Done Sent false
4234 2024 Acked Read 8 Done Done Sent false
4235 2025 Acked Write 5 Done Done Sent false
4236 2026 Acked Read 10 Done Done Sent false
4237 2027 Acked Read 5 Done Done Sent false
4238 2028 Acked Flush 0 Done Done Sent false
4239 2029 Acked Flush 0 Done Done Sent false
4240 2030 Acked Read 10 Done Done Sent false
4241 2031 Acked Read 5 Done Done Sent false
4242 2032 Acked Read 6 Done Done Sent false
4243 2033 Acked Write 1 Done Done Sent false
4244 2034 Acked Write 6 Done Done Sent false
4245 2035 Acked Read 3 Done Done Sent false
4246 2036 Acked Read 3 Done Done Sent false
4247 2037 Acked Read 9 Done Done Sent false
4248 2038 Acked Flush 0 Done Done Sent false
4249 2039 Acked Write 1 Done Done Sent false
4250 2040 Acked Flush 0 Done Done Sent false
4251 2041 Acked Flush 0 Done Done Sent false
4252 2042 Acked Read 6 Done Done Sent false
4253 2043 Acked Read 10 Done Done Sent false
4254 2044 Acked Read 4 Done Done Sent false
4255 2045 Acked Read 7 Done Done Sent false
4256 2046 Acked Read 7 Done Done Sent false
4257 2047 Acked Flush 0 Done Done Sent false
4258 2048 Acked Write 6 Done Done Sent false
4259 2049 Acked Flush 0 Done Done Sent false
4260 2050 Acked Read 8 Done Done Sent false
4261 2051 Acked Read 2 Done Done Sent false
4262 2052 Acked Write 8 Done Done Sent false
4263 2053 Acked Write 6 Done Done Sent false
4264 2054 Acked Read 9 Done Done Sent false
4265 2055 Acked Write 4 Done Done Sent false
4266 2056 Acked Read 4 Done Done Sent false
4267 2057 Acked Flush 0 Done Done Sent false
4268 2058 Acked Read 10 Done Done Sent false
4269 2059 Acked Write 10 Done Done Sent false
4270 2060 Acked Write 8 Done Done Sent false
4271 2061 Acked Read 1 Done Done Sent false
4272 2062 Acked Read 1 Done Done Sent false
4273 2063 Acked Write 4 Done Done Sent false
4274 2064 Acked Read 6 Done Done Sent false
4275 2065 Acked Read 10 Done Done Sent false
4276 2066 Acked Flush 0 Done Done Sent false
4277 2067 Acked Write 4 Done Done Sent false
4278 2068 Acked Read 1 Done Done Sent false
4279 2069 Acked Write 3 Done Done Sent false
4280 2070 Acked Read 4 Done Done Sent false
4281 2071 Acked Read 1 Done Done Sent false
4282 2072 Acked Read 2 Done Done Sent false
4283 2073 Acked Read 2 Done Done Sent false
4284 2074 Acked Read 3 Done Done Sent false
4285 2075 Acked Flush 0 Done Done Sent false
4286 2076 Acked Read 6 Done Done Sent false
4287 2077 Acked Flush 0 Done Done Sent false
4288 2078 Acked Write 3 Done Done Sent false
4289 2079 Acked Read 4 Done Done Sent false
4290 2080 Acked Write 9 Done Done Sent false
4291 2081 Acked Write 1 Done Done Sent false
4292 2082 Acked Read 3 Done Done Sent false
4293 2083 Acked Read 7 Done Done Sent false
4294 2084 Acked Write 9 Done Done Sent false
4295 2085 Acked Read 9 Done Done Sent false
4296 2086 Acked Read 10 Done Done Sent false
4297 2087 Acked Write 5 Done Done Sent false
4298 2088 Acked Flush 0 Done Done Sent false
4299 2089 Acked Read 4 Done Done Sent false
4300 2090 Acked Read 5 Done Done Sent false
4301 2091 Acked Flush 0 Done Done Sent false
4302 2092 Acked Write 9 Done Done Sent false
4303 2093 Acked Read 4 Done Done Sent false
4304 2094 Acked Read 5 Done Done Sent false
4305 2095 Acked Read 2 Done Done Sent false
4306 2096 Acked Write 10 Done Done Sent false
4307 2097 Acked Write 4 Done Done Sent false
4308 2098 Acked Write 1 Done Done Sent false
4309 2099 Acked Read 9 Done Done Sent false
4310 2100 Acked Read 8 Done Done Sent false
4311 2101 Acked Read 3 Done Done Sent false
4312 2102 Acked Read 3 Done Done Sent false
4313 2103 Acked Read 4 Done Done Sent false
4314 2104 Acked Read 4 Done Done Sent false
4315 2105 Acked Flush 0 Done Done Sent false
4316 2106 Acked Read 8 Done Done Sent false
4317 2107 Acked Read 4 Done Done Sent false
4318 2108 Acked Write 7 Done Done Sent false
4319 2109 Acked Write 8 Done Done Sent false
4320 2110 Acked Write 10 Done Done Sent false
4321 2111 Acked Write 2 Done Done Sent false
4322 2112 Acked Write 10 Done Done Sent false
4323 2113 Acked Read 7 Done Done Sent false
4324 2114 Acked Write 9 Done Done Sent false
4325 2115 Acked Read 2 Done Done Sent false
4326 2116 Acked Write 5 Done Done Sent false
4327 2117 Acked Write 6 Done Done Sent false
4328 2118 Acked Write 8 Done Done Sent false
4329 2119 Acked Write 1 Done Done Sent false
4330 2120 Acked Write 6 Done Done Sent false
4331 2121 Acked Read 5 Done Done Sent false
4332 2122 Acked Read 7 Done Done Sent false
4333 2123 Acked Read 10 Done Done Sent false
4334 2124 Acked Flush 0 Done Done Sent false
4335 2125 Acked Read 3 Done Done Sent false
4336 2126 Acked Write 3 Done Done Sent false
4337 2127 Acked Read 8 Done Done Sent false
4338 2128 Acked Write 8 Done Done Sent false
4339 2129 Acked Write 2 Done Done Sent false
4340 2130 Acked Write 1 Done Done Sent false
4341 2131 Acked Write 9 Done Done Sent false
4342 2132 Acked Read 4 Done Done Sent false
4343 2133 Acked Read 1 Done Done Sent false
4344 2134 Acked Write 5 Done Done Sent false
4345 2135 Acked Write 1 Done Done Sent false
4346 2136 Acked Flush 0 Done Done Sent false
4347 2137 Acked Write 9 Done Done Sent false
4348 2138 Acked Read 10 Done Done Sent false
4349 2139 Acked Read 4 Done Done Sent false
4350 2140 Acked Read 8 Done Done Sent false
4351 2141 Acked Write 1 Done Done Sent false
4352 2142 Acked Flush 0 Done Done Sent false
4353 2143 Acked Write 3 Done Done Sent false
4354 2144 Acked Read 2 Done Done Sent false
4355 2145 Acked Read 4 Done Done Sent false
4356 2146 Acked Write 9 Done Done Sent false
4357 2147 Acked Read 8 Done Done Sent false
4358 2148 Acked Read 9 Done Done Sent false
4359 2149 Acked Write 9 Done Done Sent false
4360 2150 Acked Flush 0 Done Done Sent false
4361 2151 Acked Read 6 Done Done Sent false
4362 2152 Acked Write 3 Done Done Sent false
4363 2153 Acked Flush 0 Done Done Sent false
4364 2154 Acked Flush 0 Done Done Sent false
4365 2155 Acked Read 3 Done Done Sent false
4366 2156 Acked Write 5 Done Done Sent false
4367 2157 Acked Read 6 Done Done Sent false
4368 2158 Acked Flush 0 Done Done Sent false
4369 2159 Acked Write 4 Done Done Sent false
4370 2160 Acked Write 9 Done Done Sent false
4371 2161 Acked Read 8 Done Done Sent false
4372 2162 Acked Write 6 Done Done Sent false
4373 2163 Acked Read 10 Done Done Sent false
4374 2164 Acked Read 10 Done Done Sent false
4375 2165 Acked Write 4 Done Done Sent false
4376 2166 Acked Read 5 Done Done Sent false
4377 2167 Acked Write 10 Done Done Sent false
4378 2168 Acked Read 6 Done Done Sent false
4379 2169 Acked Write 5 Done Done Sent false
4380 2170 Acked Write 4 Done Done Sent false
4381 2171 Acked Read 2 Done Done Sent false
4382 2172 Acked Read 10 Done Done Sent false
4383 2173 Acked Read 4 Done Done Sent false
4384 2174 Acked Write 9 Done Done Sent false
4385 2175 Acked Read 8 Done Done Sent false
4386 2176 Acked Read 3 Done Done Sent false
4387 2177 Acked Read 10 Done Done Sent false
4388 2178 Acked Write 8 Done Done Sent false
4389 2179 Acked Write 3 Done Done Sent false
4390 2180 Acked Flush 0 Done Done Sent false
4391 2181 Acked Write 9 Done Done Sent false
4392 2182 Acked Write 10 Done Done Sent false
4393 2183 Acked Write 2 Done Done Sent false
4394 2184 Acked Read 10 Done Done Sent false
4395 2185 Acked Read 7 Done Done Sent false
4396 2186 Acked Read 2 Done Done Sent false
4397 2187 Acked Write 7 Done Done Sent false
4398 2188 Acked Write 10 Done Done Sent false
4399 2189 Acked Read 9 Done Done Sent false
4400 2190 Acked Read 7 Done Done Sent false
4401 2191 Acked Write 7 Done Done Sent false
4402 2192 Acked Read 7 Done Done Sent false
4403 2193 Acked Read 4 Done Done Sent false
4404 2194 Acked Read 5 Done Done Sent false
4405 2195 Acked Read 3 Done Done Sent false
4406 2196 Acked Flush 0 Done Done Sent false
4407 2197 Acked Read 9 Done Done Sent false
4408 2198 Acked Read 4 Done Done Sent false
4409 2199 Acked Read 8 Done Done Sent false
4410 2200 Acked Write 3 Done Done Sent false
4411 2201 Acked Read 1 Done Done Sent false
4412 2202 Acked Read 7 Done Done Sent false
4413 2203 Acked Write 1 Done Done Sent false
4414 2204 Acked Write 10 Done Done Sent false
4415 2205 Acked Read 3 Done Done Sent false
4416 2206 Acked Read 5 Done Done Sent false
4417 2207 Acked Flush 0 Done Done Sent false
4418 STATES DS:0 DS:1 DS:2 TOTAL
4419 Sent 0 0 302 302
4420 Done 302 302 0 604
4421 Skipped 0 0 0 0
4422 Error 0 0 0 0
4423 Last Flush: 2207 2207 1905
4424 Downstairs last five completed: 1905 1904 1903 1902 1901
4425 Upstairs last five completed: 2207 2206 2205 2204 2203
4426 CLIENT: Up:0 ds:302 act:2
4427 ----------------------------------------------------------------
4428 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
4429 States: Active Active Offline
4430 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
4431 1906 Acked Read 7 Done Done Sent false
4432 1907 Acked Write 1 Done Done Sent false
4433 1908 Acked Write 2 Done Done Sent false
4434 1909 Acked Read 5 Done Done Sent false
4435 1910 Acked Flush 0 Done Done Sent false
4436 1911 Acked Write 7 Done Done Sent false
4437 1912 Acked Write 7 Done Done Sent false
4438 1913 Acked Write 8 Done Done Sent false
4439 1914 Acked Read 8 Done Done Sent false
4440 1915 Acked Write 7 Done Done Sent false
4441 1916 Acked Write 8 Done Done Sent false
4442 1917 Acked Read 7 Done Done Sent false
4443 1918 Acked Read 4 Done Done Sent false
4444 1919 Acked Read 8 Done Done Sent false
4445 1920 Acked Read 6 Done Done Sent false
4446 1921 Acked Read 9 Done Done Sent false
4447 1922 Acked Read 2 Done Done Sent false
4448 1923 Acked Flush 0 Done Done Sent false
4449 1924 Acked Write 4 Done Done Sent false
4450 1925 Acked Read 2 Done Done Sent false
4451 1926 Acked Write 9 Done Done Sent false
4452 1927 Acked Write 2 Done Done Sent false
4453 1928 Acked Write 2 Done Done Sent false
4454 1929 Acked Read 9 Done Done Sent false
4455 1930 Acked Read 5 Done Done Sent false
4456 1931 Acked Write 6 Done Done Sent false
4457 1932 Acked Read 10 Done Done Sent false
4458 1933 Acked Flush 0 Done Done Sent false
4459 1934 Acked Write 10 Done Done Sent false
4460 1935 Acked Write 1 Done Done Sent false
4461 1936 Acked Flush 0 Done Done Sent false
4462 1937 Acked Read 2 Done Done Sent false
4463 1938 Acked Write 3 Done Done Sent false
4464 1939 Acked Read 2 Done Done Sent false
4465 1940 Acked Read 1 Done Done Sent false
4466 1941 Acked Read 2 Done Done Sent false
4467 1942 Acked Write 5 Done Done Sent false
4468 1943 Acked Read 1 Done Done Sent false
4469 1944 Acked Read 2 Done Done Sent false
4470 1945 Acked Read 7 Done Done Sent false
4471 1946 Acked Read 8 Done Done Sent false
4472 1947 Acked Write 10 Done Done Sent false
4473 1948 Acked Write 6 Done Done Sent false
4474 1949 Acked Flush 0 Done Done Sent false
4475 1950 Acked Write 9 Done Done Sent false
4476 1951 Acked Read 8 Done Done Sent false
4477 1952 Acked Write 10 Done Done Sent false
4478 1953 Acked Read 9 Done Done Sent false
4479 1954 Acked Write 9 Done Done Sent false
4480 1955 Acked Read 6 Done Done Sent false
4481 1956 Acked Read 10 Done Done Sent false
4482 1957 Acked Write 2 Done Done Sent false
4483 1958 Acked Flush 0 Done Done Sent false
4484 1959 Acked Write 2 Done Done Sent false
4485 1960 Acked Read 3 Done Done Sent false
4486 1961 Acked Read 2 Done Done Sent false
4487 1962 Acked Write 10 Done Done Sent false
4488 1963 Acked Flush 0 Done Done Sent false
4489 1964 Acked Read 10 Done Done Sent false
4490 1965 Acked Read 10 Done Done Sent false
4491 1966 Acked Write 4 Done Done Sent false
4492 1967 Acked Read 1 Done Done Sent false
4493 1968 Acked Flush 0 Done Done Sent false
4494 1969 Acked Read 7 Done Done Sent false
4495 1970 Acked Read 10 Done Done Sent false
4496 1971 Acked Read 3 Done Done Sent false
4497 1972 Acked Read 9 Done Done Sent false
4498 1973 Acked Write 4 Done Done Sent false
4499 1974 Acked Read 8 Done Done Sent false
4500 1975 Acked Read 5 Done Done Sent false
4501 1976 Acked Write 5 Done Done Sent false
4502 1977 Acked Read 2 Done Done Sent false
4503 1978 Acked Write 6 Done Done Sent false
4504 1979 Acked Read 3 Done Done Sent false
4505 1980 Acked Read 7 Done Done Sent false
4506 1981 Acked Write 8 Done Done Sent false
4507 1982 Acked Read 2 Done Done Sent false
4508 1983 Acked Write 6 Done Done Sent false
4509 1984 Acked Flush 0 Done Done Sent false
4510 1985 Acked Write 8 Done Done Sent false
4511 1986 Acked Write 1 Done Done Sent false
4512 1987 Acked Write 8 Done Done Sent false
4513 1988 Acked Read 9 Done Done Sent false
4514 1989 Acked Read 2 Done Done Sent false
4515 1990 Acked Read 1 Done Done Sent false
4516 1991 Acked Write 1 Done Done Sent false
4517 1992 Acked Write 1 Done Done Sent false
4518 1993 Acked Write 9 Done Done Sent false
4519 1994 Acked Write 2 Done Done Sent false
4520 1995 Acked Write 6 Done Done Sent false
4521 1996 Acked Read 2 Done Done Sent false
4522 1997 Acked Flush 0 Done Done Sent false
4523 1998 Acked Write 10 Done Done Sent false
4524 1999 Acked Write 3 Done Done Sent false
4525 2000 Acked Write 6 Done Done Sent false
4526 2001 Acked Write 2 Done Done Sent false
4527 2002 Acked Write 3 Done Done Sent false
4528 2003 Acked Read 7 Done Done Sent false
4529 2004 Acked Flush 0 Done Done Sent false
4530 2005 Acked Write 9 Done Done Sent false
4531 2006 Acked Read 7 Done Done Sent false
4532 2007 Acked Read 8 Done Done Sent false
4533 2008 Acked Read 4 Done Done Sent false
4534 2009 Acked Flush 0 Done Done Sent false
4535 2010 Acked Read 6 Done Done Sent false
4536 2011 Acked Flush 0 Done Done Sent false
4537 2012 Acked Write 7 Done Done Sent false
4538 2013 Acked Write 5 Done Done Sent false
4539 2014 Acked Write 10 Done Done Sent false
4540 2015 Acked Read 8 Done Done Sent false
4541 2016 Acked Write 9 Done Done Sent false
4542 2017 Acked Read 2 Done Done Sent false
4543 2018 Acked Write 7 Done Done Sent false
4544 2019 Acked Read 10 Done Done Sent false
4545 2020 Acked Flush 0 Done Done Sent false
4546 2021 Acked Write 3 Done Done Sent false
4547 2022 Acked Read 9 Done Done Sent false
4548 2023 Acked Write 3 Done Done Sent false
4549 2024 Acked Read 8 Done Done Sent false
4550 2025 Acked Write 5 Done Done Sent false
4551 2026 Acked Read 10 Done Done Sent false
4552 2027 Acked Read 5 Done Done Sent false
4553 2028 Acked Flush 0 Done Done Sent false
4554 2029 Acked Flush 0 Done Done Sent false
4555 2030 Acked Read 10 Done Done Sent false
4556 2031 Acked Read 5 Done Done Sent false
4557 2032 Acked Read 6 Done Done Sent false
4558 2033 Acked Write 1 Done Done Sent false
4559 2034 Acked Write 6 Done Done Sent false
4560 2035 Acked Read 3 Done Done Sent false
4561 2036 Acked Read 3 Done Done Sent false
4562 2037 Acked Read 9 Done Done Sent false
4563 2038 Acked Flush 0 Done Done Sent false
4564 2039 Acked Write 1 Done Done Sent false
4565 2040 Acked Flush 0 Done Done Sent false
4566 2041 Acked Flush 0 Done Done Sent false
4567 2042 Acked Read 6 Done Done Sent false
4568 2043 Acked Read 10 Done Done Sent false
4569 2044 Acked Read 4 Done Done Sent false
4570 2045 Acked Read 7 Done Done Sent false
4571 2046 Acked Read 7 Done Done Sent false
4572 2047 Acked Flush 0 Done Done Sent false
4573 2048 Acked Write 6 Done Done Sent false
4574 2049 Acked Flush 0 Done Done Sent false
4575 2050 Acked Read 8 Done Done Sent false
4576 2051 Acked Read 2 Done Done Sent false
4577 2052 Acked Write 8 Done Done Sent false
4578 2053 Acked Write 6 Done Done Sent false
4579 2054 Acked Read 9 Done Done Sent false
4580 2055 Acked Write 4 Done Done Sent false
4581 2056 Acked Read 4 Done Done Sent false
4582 2057 Acked Flush 0 Done Done Sent false
4583 2058 Acked Read 10 Done Done Sent false
4584 2059 Acked Write 10 Done Done Sent false
4585 2060 Acked Write 8 Done Done Sent false
4586 2061 Acked Read 1 Done Done Sent false
4587 2062 Acked Read 1 Done Done Sent false
4588 2063 Acked Write 4 Done Done Sent false
4589 2064 Acked Read 6 Done Done Sent false
4590 2065 Acked Read 10 Done Done Sent false
4591 2066 Acked Flush 0 Done Done Sent false
4592 2067 Acked Write 4 Done Done Sent false
4593 2068 Acked Read 1 Done Done Sent false
4594 2069 Acked Write 3 Done Done Sent false
4595 2070 Acked Read 4 Done Done Sent false
4596 2071 Acked Read 1 Done Done Sent false
4597 2072 Acked Read 2 Done Done Sent false
4598 2073 Acked Read 2 Done Done Sent false
4599 2074 Acked Read 3 Done Done Sent false
4600 2075 Acked Flush 0 Done Done Sent false
4601 2076 Acked Read 6 Done Done Sent false
4602 2077 Acked Flush 0 Done Done Sent false
4603 2078 Acked Write 3 Done Done Sent false
4604 2079 Acked Read 4 Done Done Sent false
4605 2080 Acked Write 9 Done Done Sent false
4606 2081 Acked Write 1 Done Done Sent false
4607 2082 Acked Read 3 Done Done Sent false
4608 2083 Acked Read 7 Done Done Sent false
4609 2084 Acked Write 9 Done Done Sent false
4610 2085 Acked Read 9 Done Done Sent false
4611 2086 Acked Read 10 Done Done Sent false
4612 2087 Acked Write 5 Done Done Sent false
4613 2088 Acked Flush 0 Done Done Sent false
4614 2089 Acked Read 4 Done Done Sent false
4615 2090 Acked Read 5 Done Done Sent false
4616 2091 Acked Flush 0 Done Done Sent false
4617 2092 Acked Write 9 Done Done Sent false
4618 2093 Acked Read 4 Done Done Sent false
4619 2094 Acked Read 5 Done Done Sent false
4620 2095 Acked Read 2 Done Done Sent false
4621 2096 Acked Write 10 Done Done Sent false
4622 2097 Acked Write 4 Done Done Sent false
4623 2098 Acked Write 1 Done Done Sent false
4624 2099 Acked Read 9 Done Done Sent false
4625 2100 Acked Read 8 Done Done Sent false
4626 2101 Acked Read 3 Done Done Sent false
4627 2102 Acked Read 3 Done Done Sent false
4628 2103 Acked Read 4 Done Done Sent false
4629 2104 Acked Read 4 Done Done Sent false
4630 2105 Acked Flush 0 Done Done Sent false
4631 2106 Acked Read 8 Done Done Sent false
4632 2107 Acked Read 4 Done Done Sent false
4633 2108 Acked Write 7 Done Done Sent false
4634 2109 Acked Write 8 Done Done Sent false
4635 2110 Acked Write 10 Done Done Sent false
4636 2111 Acked Write 2 Done Done Sent false
4637 2112 Acked Write 10 Done Done Sent false
4638 2113 Acked Read 7 Done Done Sent false
4639 2114 Acked Write 9 Done Done Sent false
4640 2115 Acked Read 2 Done Done Sent false
4641 2116 Acked Write 5 Done Done Sent false
4642 2117 Acked Write 6 Done Done Sent false
4643 2118 Acked Write 8 Done Done Sent false
4644 2119 Acked Write 1 Done Done Sent false
4645 2120 Acked Write 6 Done Done Sent false
4646 2121 Acked Read 5 Done Done Sent false
4647 2122 Acked Read 7 Done Done Sent false
4648 2123 Acked Read 10 Done Done Sent false
4649 2124 Acked Flush 0 Done Done Sent false
4650 2125 Acked Read 3 Done Done Sent false
4651 2126 Acked Write 3 Done Done Sent false
4652 2127 Acked Read 8 Done Done Sent false
4653 2128 Acked Write 8 Done Done Sent false
4654 2129 Acked Write 2 Done Done Sent false
4655 2130 Acked Write 1 Done Done Sent false
4656 2131 Acked Write 9 Done Done Sent false
4657 2132 Acked Read 4 Done Done Sent false
4658 2133 Acked Read 1 Done Done Sent false
4659 2134 Acked Write 5 Done Done Sent false
4660 2135 Acked Write 1 Done Done Sent false
4661 2136 Acked Flush 0 Done Done Sent false
4662 2137 Acked Write 9 Done Done Sent false
4663 2138 Acked Read 10 Done Done Sent false
4664 2139 Acked Read 4 Done Done Sent false
4665 2140 Acked Read 8 Done Done Sent false
4666 2141 Acked Write 1 Done Done Sent false
4667 2142 Acked Flush 0 Done Done Sent false
4668 2143 Acked Write 3 Done Done Sent false
4669 2144 Acked Read 2 Done Done Sent false
4670 2145 Acked Read 4 Done Done Sent false
4671 2146 Acked Write 9 Done Done Sent false
4672 2147 Acked Read 8 Done Done Sent false
4673 2148 Acked Read 9 Done Done Sent false
4674 2149 Acked Write 9 Done Done Sent false
4675 2150 Acked Flush 0 Done Done Sent false
4676 2151 Acked Read 6 Done Done Sent false
4677 2152 Acked Write 3 Done Done Sent false
4678 2153 Acked Flush 0 Done Done Sent false
4679 2154 Acked Flush 0 Done Done Sent false
4680 2155 Acked Read 3 Done Done Sent false
4681 2156 Acked Write 5 Done Done Sent false
4682 2157 Acked Read 6 Done Done Sent false
4683 2158 Acked Flush 0 Done Done Sent false
4684 2159 Acked Write 4 Done Done Sent false
4685 2160 Acked Write 9 Done Done Sent false
4686 2161 Acked Read 8 Done Done Sent false
4687 2162 Acked Write 6 Done Done Sent false
4688 2163 Acked Read 10 Done Done Sent false
4689 2164 Acked Read 10 Done Done Sent false
4690 2165 Acked Write 4 Done Done Sent false
4691 2166 Acked Read 5 Done Done Sent false
4692 2167 Acked Write 10 Done Done Sent false
4693 2168 Acked Read 6 Done Done Sent false
4694 2169 Acked Write 5 Done Done Sent false
4695 2170 Acked Write 4 Done Done Sent false
4696 2171 Acked Read 2 Done Done Sent false
4697 2172 Acked Read 10 Done Done Sent false
4698 2173 Acked Read 4 Done Done Sent false
4699 2174 Acked Write 9 Done Done Sent false
4700 2175 Acked Read 8 Done Done Sent false
4701 2176 Acked Read 3 Done Done Sent false
4702 2177 Acked Read 10 Done Done Sent false
4703 2178 Acked Write 8 Done Done Sent false
4704 2179 Acked Write 3 Done Done Sent false
4705 2180 Acked Flush 0 Done Done Sent false
4706 2181 Acked Write 9 Done Done Sent false
4707 2182 Acked Write 10 Done Done Sent false
4708 2183 Acked Write 2 Done Done Sent false
4709 2184 Acked Read 10 Done Done Sent false
4710 2185 Acked Read 7 Done Done Sent false
4711 2186 Acked Read 2 Done Done Sent false
4712 2187 Acked Write 7 Done Done Sent false
4713 2188 Acked Write 10 Done Done Sent false
4714 2189 Acked Read 9 Done Done Sent false
4715 2190 Acked Read 7 Done Done Sent false
4716 2191 Acked Write 7 Done Done Sent false
4717 2192 Acked Read 7 Done Done Sent false
4718 2193 Acked Read 4 Done Done Sent false
4719 2194 Acked Read 5 Done Done Sent false
4720 2195 Acked Read 3 Done Done Sent false
4721 2196 Acked Flush 0 Done Done Sent false
4722 2197 Acked Read 9 Done Done Sent false
4723 2198 Acked Read 4 Done Done Sent false
4724 2199 Acked Read 8 Done Done Sent false
4725 2200 Acked Write 3 Done Done Sent false
4726 2201 Acked Read 1 Done Done Sent false
4727 2202 Acked Read 7 Done Done Sent false
4728 2203 Acked Write 1 Done Done Sent false
4729 2204 Acked Write 10 Done Done Sent false
4730 2205 Acked Read 3 Done Done Sent false
4731 2206 Acked Read 5 Done Done Sent false
4732 2207 Acked Flush 0 Done Done Sent false
4733 STATES DS:0 DS:1 DS:2 TOTAL
4734 Sent 0 0 302 302
4735 Done 302 302 0 604
4736 Skipped 0 0 0 0
4737 Error 0 0 0 0
4738 Last Flush: 2207 2207 1905
4739 Downstairs last five completed: 1905 1904 1903 1902 1901
4740 Upstairs last five completed: 2207 2206 2205 2204 2203
4741 CLIENT: Up:0 ds:302 act:2
47422024-11-26T22:05:33.142ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
47432024-11-26T22:05:33.142ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:40248) = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
47442024-11-26T22:05:33.143ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
47452024-11-26T22:05:33.143ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
47462024-11-26T22:05:33.143ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
47472024-11-26T22:05:33.143ZINFOcrucible: send last flush ID to this DS: 1905 = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
47482024-11-26T22:05:33.144ZINFOcrucible: Replied this last flush ID: 1905 = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
47492024-11-26T22:05:33.144ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
47502024-11-26T22:05:33.144ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
47512024-11-26T22:05:33.144ZINFOcrucible: [2] client re-new 302 jobs since flush 1905 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
47522024-11-26T22:05:33.144ZINFOcrucible: [2] Replayed 302 jobs since flush: 1905 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
4753 ----------------------------------------------------------------
4754 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
4755 Downstairs last five completed: 2207 2206 2205 2204 2203
4756 Upstairs last five completed: 2207 2206 2205 2204 2203
4757 CLIENT: Up:0 ds:0 act:3
4758 Replay: All jobs finished
4759 Main task has work to do, go find it
4760 got dsc Stop(0)
4761 stop 0
4762 Nov 26 22:05:33.386 INFO request completed, [8810] Got stop action so:false kr:false
4763 latency_us: 361, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 16c28b4e-07c3-4015-8f0d-1d2a74602001, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
47642024-11-26T22:05:33.386ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
47652024-11-26T22:05:33.386ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
47662024-11-26T22:05:33.386ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
47672024-11-26T22:05:33.386ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
47682024-11-26T22:05:33.386ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
47692024-11-26T22:05:33.387ZINFOcrucible: client task is exiting = io task client = 0 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
4770 Nov 26 22:05:33.387 INFO request completed, latency_us: 116, response_code: 200, uri: /state/cid/0, method: GET, req_id: 576ef44b-987c-468c-9b17-811a000d7280, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
4771 [8810] Exited with: ExitStatus(unix_wait_status(9))
4772 [8810][0] reports Exit
4773 Nov 26 22:05:37.389 INFO request completed, latency_us: 384, response_code: 200, uri: /state/cid/0, method: GET, req_id: 828f8678-5b89-4ac0-9276-f8588261fdfa, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
4774 001/300 Flush
4775 002/300 Write block 3401 len 36864 data: 3 3 2 3 3 3 3 3 3
4776 003/300 Read block 2354 len 40960
4777 004/300 Write block 4229 len 12288 data: 2 4 4
4778 005/300 Read block 4690 len 8192
4779 006/300 Write block 3114 len 4096 data: 2
4780 007/300 Write block 1721 len 28672 data: 2 2 2 2 2 2 2
4781 008/300 Read block 3731 len 12288
4782 009/300 Read block 418 len 16384
4783 010/300 Write block 1842 len 32768 data: 2 2 2 2 3 4 4 4
4784 011/300 Read block 3114 len 4096
4785 012/300 Read block 1519 len 20480
4786 013/300 Read block 831 len 32768
4787 014/300 Read block 4320 len 8192
4788 015/300 Flush
4789 016/300 Read block 281 len 36864
4790 017/300 Write block 1197 len 40960 data: 2 2 2 2 3 3 3 3 3 2
4791 018/300 Read block 2135 len 28672
4792 019/300 Read block 3031 len 28672
4793 020/300 Flush
4794 021/300 Write block 222 len 36864 data: 2 2 2 2 2 2 2 2 2
4795 022/300 Flush
4796 023/300 Write block 489 len 8192 data: 5 5
4797 024/300 Write block 3631 len 20480 data: 3 3 3 2 2
4798 025/300 Write block 3140 len 36864 data: 3 3 3 3 3 3 3 3 2
4799 026/300 Read block 3195 len 28672
4800 027/300 Write block 4750 len 4096 data: 2
4801 028/300 Read block 3323 len 40960
4802 029/300 Write block 1583 len 4096 data: 2
4803 030/300 Write block 3712 len 36864 data: 2 2 2 3 4 4 4 4 3
4804 031/300 Read block 2229 len 8192
4805 032/300 Write block 4691 len 32768 data: 4 4 4 4 4 2 2 2
4806 033/300 Read block 1262 len 20480
4807 034/300 Read block 987 len 8192
4808 035/300 Read block 1703 len 40960
4809 036/300 Write block 120 len 32768 data: 2 3 4 4 4 4 4 4
4810 037/300 Read block 4338 len 36864
4811 038/300 Write block 2266 len 28672 data: 2 2 2 2 2 2 2
4812 039/300 Read block 4076 len 12288
4813 040/300 Write block 2282 len 8192 data: 2 2
4814 041/300 Read block 1460 len 16384
4815 042/300 Flush
4816 043/300 Write block 4066 len 20480 data: 3 3 4 3 3
4817 044/300 Write block 3944 len 32768 data: 3 2 2 3 3 3 3 3
4818 045/300 Write block 361 len 36864 data: 3 3 4 3 3 3 3 2 2
4819 046/300 Read block 1702 len 12288
4820 047/300 Flush
4821 048/300 Flush
4822 049/300 Read block 3368 len 4096
4823 050/300 Write block 4980 len 32768 data: 3 3 3 3 3 2 2 3
4824 051/300 Write block 4829 len 16384 data: 3 3 3 3
4825 052/300 Read block 3572 len 36864
4826 053/300 Read block 94 len 28672
4827 054/300 Read block 287 len 4096
4828 055/300 Flush
4829 056/300 Read block 4390 len 12288
4830 057/300 Write block 1165 len 16384 data: 2 2 2 3
4831 058/300 Read block 890 len 16384
4832 059/300 Read block 546 len 16384
4833 060/300 Read block 4843 len 40960
4834 061/300 Read block 4125 len 24576
4835 062/300 Read block 560 len 28672
4836 063/300 Read block 1115 len 36864
4837 064/300 Write block 2984 len 12288 data: 2 3 3
4838 065/300 Read block 84 len 12288
4839 066/300 Read block 2287 len 40960
4840 067/300 Read block 2033 len 16384
4841 068/300 Read block 1651 len 8192
4842 069/300 Flush
4843 070/300 Write block 2632 len 8192 data: 3 3
4844 071/300 Read block 2328 len 40960
4845 072/300 Read block 3206 len 24576
4846 073/300 Write block 2303 len 8192 data: 4 2
4847 074/300 Read block 3649 len 40960
4848 075/300 Write block 3551 len 20480 data: 4 4 4 4 3
4849 076/300 Write block 1887 len 24576 data: 3 3 3 4 4 3
4850 077/300 Flush
4851 078/300 Write block 1092 len 8192 data: 3 3
4852 079/300 Read block 3916 len 8192
4853 080/300 Write block 1562 len 16384 data: 2 2 2 2
4854 081/300 Read block 4198 len 8192
4855 082/300 Read block 3518 len 20480
4856 083/300 Write block 3857 len 36864 data: 2 2 2 2 2 2 2 2 2
4857 084/300 Read block 3989 len 40960
4858 085/300 Read block 431 len 28672
4859 086/300 Read block 3466 len 32768
4860 087/300 Read block 564 len 28672
4861 088/300 Write block 612 len 36864 data: 2 2 2 2 2 2 2 2 2
4862 089/300 Read block 2757 len 20480
4863 090/300 Write block 3285 len 28672 data: 2 2 3 3 2 2 2
4864 091/300 Read block 310 len 8192
4865 092/300 Write block 4564 len 4096 data: 2
4866 093/300 Read block 1822 len 32768
4867 094/300 Flush
4868 095/300 Write block 3403 len 8192 data: 3 4
4869 096/300 Write block 1936 len 4096 data: 3
4870 097/300 Write block 3228 len 20480 data: 3 3 3 3 3
4871 098/300 Read block 1220 len 16384
4872 099/300 Read block 2642 len 12288
4873 100/300 Write block 3921 len 12288 data: 2 2 2
4874 101/300 Write block 3730 len 4096 data: 4
4875 102/300 Write block 4619 len 36864 data: 2 2 2 2 2 2 2 2 2
4876 103/300 Write block 3744 len 12288 data: 2 2 2
4877 104/300 Write block 1580 len 12288 data: 3 2 2
4878 105/300 Write block 3960 len 8192 data: 2 2
4879 106/300 Read block 1574 len 32768
4880 107/300 Write block 3399 len 32768 data: 4 3 4 4 4 5 4 4
4881 108/300 Read block 4903 len 32768
4882 109/300 Read block 515 len 40960
4883 110/300 Read block 3930 len 40960
4884 111/300 Read block 4184 len 20480
4885 112/300 Read block 3434 len 36864
4886 113/300 Read block 1990 len 40960
4887 114/300 Write block 1797 len 36864 data: 2 2 2 2 2 2 2 2 2
4888 115/300 Read block 3736 len 4096
4889 116/300 Write block 2546 len 28672 data: 3 2 2 2 2 2 2
4890 117/300 Read block 2561 len 12288
4891 118/300 Write block 3564 len 20480 data: 2 2 2 2 2
4892 119/300 Write block 1489 len 12288 data: 3 4 4
4893 120/300 Write block 1658 len 16384 data: 2 2 2 2
4894 121/300 Read block 577 len 8192
4895 122/300 Read block 4571 len 32768
4896 123/300 Flush
4897 124/300 Flush
4898 125/300 Write block 224 len 24576 data: 3 3 3 3 3 3
4899 126/300 Write block 4780 len 4096 data: 2
4900 127/300 Write block 2057 len 36864 data: 2 2 2 2 2 2 2 2 2
4901 128/300 Write block 2512 len 4096 data: 2
4902 129/300 Write block 874 len 12288 data: 4 4 3
4903 130/300 Read block 4284 len 24576
4904 131/300 Read block 344 len 16384
4905 132/300 Write block 586 len 24576 data: 2 2 2 2 2 2
4906 133/300 Write block 349 len 40960 data: 2 2 2 2 2 2 3 3 3 3
4907 134/300 Write block 2802 len 4096 data: 2
4908 135/300 Write block 925 len 24576 data: 2 3 2 2 3 3
4909 136/300 Read block 2275 len 8192
4910 137/300 Flush
4911 138/300 Read block 2890 len 12288
4912 139/300 Write block 459 len 12288 data: 2 2 3
4913 140/300 Read block 4748 len 36864
4914 141/300 Write block 246 len 8192 data: 3 3
4915 142/300 Read block 2183 len 40960
4916 143/300 Read block 2908 len 20480
4917 144/300 Write block 3603 len 24576 data: 2 2 2 2 2 2
4918 145/300 Write block 2859 len 8192 data: 2 2
4919 146/300 Read block 1301 len 40960
4920 147/300 Write block 2743 len 24576 data: 2 2 2 2 2 2
4921 148/300 Write block 2516 len 16384 data: 3 3 3 3
4922 149/300 Write block 1938 len 28672 data: 5 5 4 4 4 4 4
4923 150/300 Read block 2381 len 28672
4924 151/300 Read block 544 len 16384
4925 152/300 Flush
4926 153/300 Read block 2706 len 20480
4927 154/300 Read block 4537 len 40960
4928 155/300 Read block 2093 len 4096
4929 156/300 Read block 3106 len 28672
4930 157/300 Read block 3532 len 8192
4931 158/300 Read block 4861 len 12288
4932 159/300 Read block 4316 len 8192
4933 160/300 Read block 269 len 16384
4934 161/300 Read block 485 len 20480
4935 162/300 Write block 4159 len 40960 data: 2 2 2 2 2 2 2 2 2 2
4936 163/300 Flush
4937 164/300 Read block 4217 len 28672
4938 165/300 Write block 1939 len 40960 data: 6 5 5 5 5 5 4 3 3 3
4939 166/300 Write block 3155 len 28672 data: 2 2 2 2 2 3 3
4940 167/300 Write block 144 len 4096 data: 2
4941 168/300 Read block 1654 len 20480
4942 169/300 Write block 4097 len 8192 data: 2 2
4943 170/300 Read block 2763 len 16384
4944 171/300 Read block 2646 len 28672
4945 172/300 Write block 4799 len 16384 data: 2 2 3 4
4946 173/300 Write block 362 len 12288 data: 4 5 4
4947 174/300 Read block 1817 len 32768
4948 175/300 Write block 270 len 20480 data: 2 2 2 2 2
4949 176/300 Read block 907 len 4096
4950 177/300 Write block 1671 len 28672 data: 3 3 3 3 2 2 2
4951 178/300 Read block 4082 len 28672
4952 179/300 Read block 3417 len 36864
4953 180/300 Flush
4954 181/300 Write block 2828 len 20480 data: 2 2 2 2 2
4955 182/300 Write block 4893 len 16384 data: 2 2 2 2
4956 183/300 Read block 2906 len 20480
4957 184/300 Write block 3599 len 12288 data: 3 2 3
4958 185/300 Write block 2041 len 8192 data: 2 2
4959 186/300 Read block 8 len 8192
4960 187/300 Read block 2408 len 24576
4961 188/300 Flush
4962 189/300 Write block 359 len 28672 data: 3 3 4 5 6 5 4
4963 190/300 Write block 3941 len 4096 data: 2
4964 191/300 Read block 4197 len 40960
4965 192/300 Read block 313 len 16384
4966 193/300 Write block 224 len 24576 data: 4 4 4 4 4 4
4967 194/300 Read block 811 len 20480
4968 195/300 Read block 3567 len 8192
4969 196/300 Read block 4096 len 28672
4970 197/300 Write block 4961 len 20480 data: 3 3 3 3 4
4971 198/300 Write block 1446 len 12288 data: 2 2 2
4972 199/300 Write block 657 len 16384 data: 2 2 2 2
4973 200/300 Read block 273 len 24576
4974 201/300 Read block 531 len 8192
4975 202/300 Read block 865 len 12288
4976 203/300 Flush
4977 204/300 Read block 1648 len 16384
4978 205/300 Read block 3295 len 8192
4979 206/300 Write block 1354 len 24576 data: 3 3 3 2 2 2
4980 207/300 Read block 1603 len 16384
4981 208/300 Read block 67 len 24576
4982 209/300 Write block 1029 len 4096 data: 2
4983 210/300 Flush
4984 211/300 Read block 2250 len 40960
4985 212/300 Read block 3386 len 16384
4986 213/300 Write block 262 len 8192 data: 3 3
4987 214/300 Flush
4988 215/300 Flush
4989 216/300 Read block 2150 len 20480
4990 217/300 Read block 41 len 8192
4991 218/300 Write block 4575 len 24576 data: 4 3 2 2 2 2
4992 219/300 Read block 2681 len 32768
4993 220/300 Read block 3976 len 28672
4994 221/300 Read block 2621 len 32768
4995 222/300 Write block 365 len 20480 data: 5 4 4 3 3
4996 223/300 Read block 4341 len 8192
4997 224/300 Write block 96 len 36864 data: 2 2 2 2 2 3 3 3 3
4998 225/300 Read block 2460 len 12288
4999 226/300 Write block 3838 len 20480 data: 2 2 2 2 2
5000 227/300 Read block 3014 len 36864
5001 228/300 Write block 4003 len 36864 data: 3 3 3 3 3 3 3 3 3
5002 229/300 Flush
5003 230/300 Read block 4292 len 24576
5004 231/300 Read block 2917 len 36864
5005 232/300 Read block 3156 len 8192
5006 233/300 Flush
5007 234/300 Write block 1518 len 4096 data: 2
5008 235/300 Read block 4451 len 16384
5009 236/300 Write block 4809 len 36864 data: 5 4 3 3 3 3 3 3 3
5010 237/300 Read block 110 len 16384
5011 238/300 Write block 2600 len 36864 data: 2 2 2 2 2 2 2 3 3
5012 239/300 Read block 1828 len 24576
5013 240/300 Write block 4822 len 40960 data: 2 2 2 2 2 3 3 4 4 4
5014 241/300 Write block 3845 len 16384 data: 2 2 2 2
5015 242/300 Write block 3699 len 32768 data: 3 3 3 2 2 2 2 2
5016 243/300 Read block 844 len 16384
5017 244/300 Read block 2242 len 8192
5018 245/300 Read block 2636 len 8192
5019 246/300 Flush
5020 247/300 Read block 507 len 4096
5021 248/300 Write block 2566 len 8192 data: 2 3
5022 249/300 Write block 1004 len 40960 data: 2 2 2 2 2 2 2 2 2 2
5023 250/300 Read block 3980 len 24576
5024 251/300 Read block 4311 len 28672
5025 252/300 Read block 4496 len 16384
5026 253/300 Write block 4867 len 16384 data: 2 2 2 2
5027 254/300 Write block 96 len 28672 data: 3 3 3 3 3 4 4
5028 255/300 Read block 4284 len 8192
5029 256/300 Read block 1507 len 8192
5030 257/300 Read block 4543 len 16384
5031 258/300 Read block 4762 len 8192
5032 259/300 Read block 3131 len 20480
5033 260/300 Read block 797 len 8192
5034 261/300 Read block 2798 len 16384
5035 262/300 Read block 457 len 8192
5036 263/300 Write block 403 len 20480 data: 3 3 3 2 3
5037 264/300 Flush
5038 265/300 Write block 2058 len 36864 data: 3 3 3 3 3 3 3 3 2
5039 266/300 Flush
5040 267/300 Write block 2048 len 4096 data: 2
5041 268/300 Read block 570 len 28672
5042 269/300 Write block 4362 len 28672 data: 4 3 3 3 3 3 3
5043 270/300 Write block 2253 len 8192 data: 3 3
5044 271/300 Read block 1829 len 12288
5045 272/300 Read block 4921 len 40960
5046 273/300 Read block 2448 len 40960
5047 274/300 Read block 2382 len 12288
5048 275/300 Write block 2351 len 28672 data: 2 2 2 2 2 2 2
5049 276/300 Read block 3841 len 24576
5050 277/300 Flush
5051 278/300 Write block 1350 len 12288 data: 3 3 3
5052 279/300 Write block 2160 len 40960 data: 3 3 3 2 2 4 4 3 3 3
5053 280/300 Read block 3978 len 24576
5054 281/300 Write block 1364 len 20480 data: 2 2 2 2 2
5055 282/300 Write block 2969 len 4096 data: 2
5056 283/300 Flush
5057 284/300 Read block 2345 len 24576
5058 285/300 Write block 4349 len 20480 data: 2 2 2 2 2
5059 286/300 Write block 4677 len 32768 data: 2 2 2 2 2 2 2 2
5060 287/300 Write block 954 len 28672 data: 2 2 2 2 3 3 3
5061 288/300 Write block 2700 len 4096 data: 2
5062 289/300 Write block 1380 len 20480 data: 2 2 2 2 2
5063 290/300 Read block 4252 len 32768
5064 291/300 Read block 3847 len 16384
5065 292/300 Read block 2319 len 28672
5066 293/300 Write block 2422 len 24576 data: 3 3 3 2 2 2
5067 294/300 Read block 4130 len 12288
5068 295/300 Flush
5069 296/300 Write block 4763 len 32768 data: 2 2 2 2 2 2 2 2
5070 297/300 Write block 666 len 40960 data: 2 2 2 2 2 2 3 3 3 3
5071 298/300 Write block 2710 len 40960 data: 2 2 2 2 2 2 2 2 2 2
5072 299/300 Write block 1821 len 40960 data: 3 3 2 2 2 2 2 3 3 3
5073 300/300 Read block 1710 len 4096
5074 301/300 Read block 703 len 12288
5075 Main task has work to do, go find it
5076 got dsc Start(0)
5077 start 0
5078 Nov 26 22:05:37.617 INFO request completed, latency_us: 198, response_code: 204, uri: /start/cid/0, method: POST, req_id: ffbce624-2043-4ae9-810f-c2f910825204, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
5079 Replay: started 0, returned:Ok(())
5080 ----------------------------------------------------------------
5081 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
5082 States: Offline Active Active
5083 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
5084 2208 Acked Flush 0 Sent Done Done false
5085 2209 Acked Write 9 Sent Done Done false
5086 2210 Acked Read 10 Sent Done Done false
5087 2211 Acked Write 3 Sent Done Done false
5088 2212 Acked Read 2 Sent Done Done false
5089 2213 Acked Write 1 Sent Done Done false
5090 2214 Acked Write 7 Sent Done Done false
5091 2215 Acked Read 3 Sent Done Done false
5092 2216 Acked Read 4 Sent Done Done false
5093 2217 Acked Write 8 Sent Done Done false
5094 2218 Acked Read 1 Sent Done Done false
5095 2219 Acked Read 5 Sent Done Done false
5096 2220 Acked Read 8 Sent Done Done false
5097 2221 Acked Read 2 Sent Done Done false
5098 2222 Acked Flush 0 Sent Done Done false
5099 2223 Acked Read 9 Sent Done Done false
5100 2224 Acked Write 10 Sent Done Done false
5101 2225 Acked Read 7 Sent Done Done false
5102 2226 Acked Read 7 Sent Done Done false
5103 2227 Acked Flush 0 Sent Done Done false
5104 2228 Acked Write 9 Sent Done Done false
5105 2229 Acked Flush 0 Sent Done Done false
5106 2230 Acked Write 2 Sent Done Done false
5107 2231 Acked Write 5 Sent Done Done false
5108 2232 Acked Write 9 Sent Done Done false
5109 2233 Acked Read 7 Sent Done Done false
5110 2234 Acked Write 1 Sent Done Done false
5111 2235 Acked Read 10 Sent Done Done false
5112 2236 Acked Write 1 Sent Done Done false
5113 2237 Acked Write 9 Sent Done Done false
5114 2238 Acked Read 2 Sent Done Done false
5115 2239 Acked Write 8 Sent Done Done false
5116 2240 Acked Read 5 Sent Done Done false
5117 2241 Acked Read 2 Sent Done Done false
5118 2242 Acked Read 10 Sent Done Done false
5119 2243 Acked Write 8 Sent Done Done false
5120 2244 Acked Read 9 Sent Done Done false
5121 2245 Acked Write 7 Sent Done Done false
5122 2246 Acked Read 3 Sent Done Done false
5123 2247 Acked Write 2 Sent Done Done false
5124 2248 Acked Read 4 Sent Done Done false
5125 2249 Acked Flush 0 Sent Done Done false
5126 2250 Acked Write 5 Sent Done Done false
5127 2251 Acked Write 8 Sent Done Done false
5128 2252 Acked Write 9 Sent Done Done false
5129 2253 Acked Read 3 Sent Done Done false
5130 2254 Acked Flush 0 Sent Done Done false
5131 2255 Acked Flush 0 Sent Done Done false
5132 2256 Acked Read 1 Sent Done Done false
5133 2257 Acked Write 8 Sent Done Done false
5134 2258 Acked Write 4 Sent Done Done false
5135 2259 Acked Read 9 Sent Done Done false
5136 2260 Acked Read 7 Sent Done Done false
5137 2261 Acked Read 1 Sent Done Done false
5138 2262 Acked Flush 0 Sent Done Done false
5139 2263 Acked Read 3 Sent Done Done false
5140 2264 Acked Write 4 Sent Done Done false
5141 2265 Acked Read 4 Sent Done Done false
5142 2266 Acked Read 4 Sent Done Done false
5143 2267 Acked Read 10 Sent Done Done false
5144 2268 Acked Read 6 Sent Done Done false
5145 2269 Acked Read 7 Sent Done Done false
5146 2270 Acked Read 9 Sent Done Done false
5147 2271 Acked Write 3 Sent Done Done false
5148 2272 Acked Read 3 Sent Done Done false
5149 2273 Acked Read 10 Sent Done Done false
5150 2274 Acked Read 4 Sent Done Done false
5151 2275 Acked Read 2 Sent Done Done false
5152 2276 Acked Flush 0 Sent Done Done false
5153 2277 Acked Write 2 Sent Done Done false
5154 2278 Acked Read 10 Sent Done Done false
5155 2279 Acked Read 6 Sent Done Done false
5156 2280 Acked Write 2 Sent Done Done false
5157 2281 Acked Read 10 Sent Done Done false
5158 2282 Acked Write 5 Sent Done Done false
5159 2283 Acked Write 6 Sent Done Done false
5160 2284 Acked Flush 0 Sent Done Done false
5161 2285 Acked Write 2 Sent Done Done false
5162 2286 Acked Read 2 Sent Done Done false
5163 2287 Acked Write 4 Sent Done Done false
5164 2288 Acked Read 2 Sent Done Done false
5165 2289 Acked Read 5 Sent Done Done false
5166 2290 Acked Write 9 Sent Done Done false
5167 2291 Acked Read 10 Sent Done Done false
5168 2292 Acked Read 7 Sent Done Done false
5169 2293 Acked Read 8 Sent Done Done false
5170 2294 Acked Read 7 Sent Done Done false
5171 2295 Acked Write 9 Sent Done Done false
5172 2296 Acked Read 5 Sent Done Done false
5173 2297 Acked Write 7 Sent Done Done false
5174 2298 Acked Read 2 Sent Done Done false
5175 2299 Acked Write 1 Sent Done Done false
5176 2300 Acked Read 8 Sent Done Done false
5177 2301 Acked Flush 0 Sent Done Done false
5178 2302 Acked Write 2 Sent Done Done false
5179 2303 Acked Write 1 Sent Done Done false
5180 2304 Acked Write 5 Sent Done Done false
5181 2305 Acked Read 4 Sent Done Done false
5182 2306 Acked Read 3 Sent Done Done false
5183 2307 Acked Write 3 Sent Done Done false
5184 2308 Acked Write 1 Sent Done Done false
5185 2309 Acked Write 9 Sent Done Done false
5186 2310 Acked Write 3 Sent Done Done false
5187 2311 Acked Write 3 Sent Done Done false
5188 2312 Acked Write 2 Sent Done Done false
5189 2313 Acked Read 8 Sent Done Done false
5190 2314 Acked Write 8 Sent Done Done false
5191 2315 Acked Read 8 Sent Done Done false
5192 2316 Acked Read 10 Sent Done Done false
5193 2317 Acked Read 10 Sent Done Done false
5194 2318 Acked Read 5 Sent Done Done false
5195 2319 Acked Read 9 Sent Done Done false
5196 2320 Acked Read 10 Sent Done Done false
5197 2321 Acked Write 9 Sent Done Done false
5198 2322 Acked Read 1 Sent Done Done false
5199 2323 Acked Write 7 Sent Done Done false
5200 2324 Acked Read 3 Sent Done Done false
5201 2325 Acked Write 5 Sent Done Done false
5202 2326 Acked Write 3 Sent Done Done false
5203 2327 Acked Write 4 Sent Done Done false
5204 2328 Acked Read 2 Sent Done Done false
5205 2329 Acked Read 8 Sent Done Done false
5206 2330 Acked Flush 0 Sent Done Done false
5207 2331 Acked Flush 0 Sent Done Done false
5208 2332 Acked Write 6 Sent Done Done false
5209 2333 Acked Write 1 Sent Done Done false
5210 2334 Acked Write 9 Sent Done Done false
5211 2335 Acked Write 1 Sent Done Done false
5212 2336 Acked Write 3 Sent Done Done false
5213 2337 Acked Read 6 Sent Done Done false
5214 2338 Acked Read 4 Sent Done Done false
5215 2339 Acked Write 6 Sent Done Done false
5216 2340 Acked Write 10 Sent Done Done false
5217 2341 Acked Write 1 Sent Done Done false
5218 2342 Acked Write 6 Sent Done Done false
5219 2343 Acked Read 2 Sent Done Done false
5220 2344 Acked Flush 0 Sent Done Done false
5221 2345 Acked Read 3 Sent Done Done false
5222 2346 Acked Write 3 Sent Done Done false
5223 2347 Acked Read 9 Sent Done Done false
5224 2348 Acked Write 2 Sent Done Done false
5225 2349 Acked Read 10 Sent Done Done false
5226 2350 Acked Read 5 Sent Done Done false
5227 2351 Acked Write 6 Sent Done Done false
5228 2352 Acked Write 2 Sent Done Done false
5229 2353 Acked Read 10 Sent Done Done false
5230 2354 Acked Write 6 Sent Done Done false
5231 2355 Acked Write 4 Sent Done Done false
5232 2356 Acked Write 7 Sent Done Done false
5233 2357 Acked Read 7 Sent Done Done false
5234 2358 Acked Read 4 Sent Done Done false
5235 2359 Acked Flush 0 Sent Done Done false
5236 2360 Acked Read 5 Sent Done Done false
5237 2361 Acked Read 10 Sent Done Done false
5238 2362 Acked Read 1 Sent Done Done false
5239 2363 Acked Read 7 Sent Done Done false
5240 2364 Acked Read 2 Sent Done Done false
5241 2365 Acked Read 3 Sent Done Done false
5242 2366 Acked Read 2 Sent Done Done false
5243 2367 Acked Read 4 Sent Done Done false
5244 2368 Acked Read 5 Sent Done Done false
5245 2369 Acked Write 10 Sent Done Done false
5246 2370 Acked Flush 0 Sent Done Done false
5247 2371 Acked Read 7 Sent Done Done false
5248 2372 Acked Write 10 Sent Done Done false
5249 2373 Acked Write 7 Sent Done Done false
5250 2374 Acked Write 1 Sent Done Done false
5251 2375 Acked Read 5 Sent Done Done false
5252 2376 Acked Write 2 Sent Done Done false
5253 2377 Acked Read 4 Sent Done Done false
5254 2378 Acked Read 7 Sent Done Done false
5255 2379 Acked Write 4 Sent Done Done false
5256 2380 Acked Write 3 Sent Done Done false
5257 2381 Acked Read 8 Sent Done Done false
5258 2382 Acked Write 5 Sent Done Done false
5259 2383 Acked Read 1 Sent Done Done false
5260 2384 Acked Write 7 Sent Done Done false
5261 2385 Acked Read 7 Sent Done Done false
5262 2386 Acked Read 9 Sent Done Done false
5263 2387 Acked Flush 0 Sent Done Done false
5264 2388 Acked Write 5 Sent Done Done false
5265 2389 Acked Write 4 Sent Done Done false
5266 2390 Acked Read 5 Sent Done Done false
5267 2391 Acked Write 3 Sent Done Done false
5268 2392 Acked Write 2 Sent Done Done false
5269 2393 Acked Read 2 Sent Done Done false
5270 2394 Acked Read 6 Sent Done Done false
5271 2395 Acked Flush 0 Sent Done Done false
5272 2396 Acked Write 7 Sent Done Done false
5273 2397 Acked Write 1 Sent Done Done false
5274 2398 Acked Read 10 Sent Done Done false
5275 2399 Acked Read 4 Sent Done Done false
5276 2400 Acked Write 6 Sent Done Done false
5277 2401 Acked Read 5 Sent Done Done false
5278 2402 Acked Read 2 Sent Done Done false
5279 2403 Acked Read 7 Sent Done Done false
5280 2404 Acked Write 5 Sent Done Done false
5281 2405 Acked Write 3 Sent Done Done false
5282 2406 Acked Write 4 Sent Done Done false
5283 2407 Acked Read 6 Sent Done Done false
5284 2408 Acked Read 2 Sent Done Done false
5285 2409 Acked Read 3 Sent Done Done false
5286 2410 Acked Flush 0 Sent Done Done false
5287 2411 Acked Read 4 Sent Done Done false
5288 2412 Acked Read 2 Sent Done Done false
5289 2413 Acked Write 6 Sent Done Done false
5290 2414 Acked Read 4 Sent Done Done false
5291 2415 Acked Read 6 Sent Done Done false
5292 2416 Acked Write 1 Sent Done Done false
5293 2417 Acked Flush 0 Sent Done Done false
5294 2418 Acked Read 10 Sent Done Done false
5295 2419 Acked Read 4 Sent Done Done false
5296 2420 Acked Write 2 Sent Done Done false
5297 2421 Acked Flush 0 Sent Done Done false
5298 2422 Acked Flush 0 Sent Done Done false
5299 2423 Acked Read 5 Sent Done Done false
5300 2424 Acked Read 2 Sent Done Done false
5301 2425 Acked Write 6 Sent Done Done false
5302 2426 Acked Read 8 Sent Done Done false
5303 2427 Acked Read 7 Sent Done Done false
5304 2428 Acked Read 8 Sent Done Done false
5305 2429 Acked Write 5 Sent Done Done false
5306 2430 Acked Read 2 Sent Done Done false
5307 2431 Acked Write 9 Sent Done Done false
5308 2432 Acked Read 3 Sent Done Done false
5309 2433 Acked Write 5 Sent Done Done false
5310 2434 Acked Read 9 Sent Done Done false
5311 2435 Acked Write 9 Sent Done Done false
5312 2436 Acked Flush 0 Sent Done Done false
5313 2437 Acked Read 6 Sent Done Done false
5314 2438 Acked Read 9 Sent Done Done false
5315 2439 Acked Read 2 Sent Done Done false
5316 2440 Acked Flush 0 Sent Done Done false
5317 2441 Acked Write 1 Sent Done Done false
5318 2442 Acked Read 4 Sent Done Done false
5319 2443 Acked Write 9 Sent Done Done false
5320 2444 Acked Read 4 Sent Done Done false
5321 2445 Acked Write 9 Sent Done Done false
5322 2446 Acked Read 6 Sent Done Done false
5323 2447 Acked Write 10 Sent Done Done false
5324 2448 Acked Write 4 Sent Done Done false
5325 2449 Acked Write 8 Sent Done Done false
5326 2450 Acked Read 4 Sent Done Done false
5327 2451 Acked Read 2 Sent Done Done false
5328 2452 Acked Read 2 Sent Done Done false
5329 2453 Acked Flush 0 Sent Done Done false
5330 2454 Acked Read 1 Sent Done Done false
5331 2455 Acked Write 2 Sent Done Done false
5332 2456 Acked Write 10 Sent Done Done false
5333 2457 Acked Read 6 Sent Done Done false
5334 2458 Acked Read 7 Sent Done Done false
5335 2459 Acked Read 4 Sent Done Done false
5336 2460 Acked Write 4 Sent Done Done false
5337 2461 Acked Write 7 Sent Done Done false
5338 2462 Acked Read 2 Sent Done Done false
5339 2463 Acked Read 2 Sent Done Done false
5340 2464 Acked Read 4 Sent Done Done false
5341 2465 Acked Read 2 Sent Done Done false
5342 2466 Acked Read 5 Sent Done Done false
5343 2467 Acked Read 2 Sent Done Done false
5344 2468 Acked Read 4 Sent Done Done false
5345 2469 Acked Read 2 Sent Done Done false
5346 2470 Acked Write 5 Sent Done Done false
5347 2471 Acked Flush 0 Sent Done Done false
5348 2472 Acked Write 9 Sent Done Done false
5349 2473 Acked Flush 0 Sent Done Done false
5350 2474 Acked Write 1 Sent Done Done false
5351 2475 Acked Read 7 Sent Done Done false
5352 2476 Acked Write 7 Sent Done Done false
5353 2477 Acked Write 2 Sent Done Done false
5354 2478 Acked Read 3 Sent Done Done false
5355 2479 Acked Read 10 Sent Done Done false
5356 2480 Acked Read 10 Sent Done Done false
5357 2481 Acked Read 3 Sent Done Done false
5358 2482 Acked Write 7 Sent Done Done false
5359 2483 Acked Read 6 Sent Done Done false
5360 2484 Acked Flush 0 Sent Done Done false
5361 2485 Acked Write 3 Sent Done Done false
5362 2486 Acked Write 10 Sent Done Done false
5363 2487 Acked Read 6 Sent Done Done false
5364 2488 Acked Write 5 Sent Done Done false
5365 2489 Acked Write 1 Sent Done Done false
5366 2490 Acked Flush 0 Sent Done Done false
5367 2491 Acked Read 6 Sent Done Done false
5368 2492 Acked Write 5 Sent Done Done false
5369 2493 Acked Write 8 Sent Done Done false
5370 2494 Acked Write 7 Sent Done Done false
5371 2495 Acked Write 1 Sent Done Done false
5372 2496 Acked Write 5 Sent Done Done false
5373 2497 Acked Read 8 Sent Done Done false
5374 2498 Acked Read 4 Sent Done Done false
5375 2499 Acked Read 7 Sent Done Done false
5376 2500 Acked Write 6 Sent Done Done false
5377 2501 Acked Read 3 Sent Done Done false
5378 2502 Acked Flush 0 Sent Done Done false
5379 2503 Acked Write 8 Sent Done Done false
5380 2504 Acked Write 10 Sent Done Done false
5381 2505 Acked Write 10 Sent Done Done false
5382 2506 Acked Write 10 Sent Done Done false
5383 2507 Acked Read 1 Sent Done Done false
5384 2508 Acked Read 3 Sent Done Done false
5385 STATES DS:0 DS:1 DS:2 TOTAL
5386 Sent 301 0 0 301
5387 Done 0 301 301 602
5388 Skipped 0 0 0 0
5389 Error 0 0 0 0
5390 Last Flush: 2207 2502 2502
5391 Downstairs last five completed: 2207 2206 2205 2204 2203
5392 Upstairs last five completed: 2508 2507 2506 2505 2504
5393 CLIENT: Up:0 ds:301 act:2
5394 [8810] got start action
5395 [8810] I am going to restart
5396 Starting downstairs at port 8810
5397 Make output file at "/tmp/dsc/downstairs-8810.txt"
5398 [8810][0] reports Starting
5399 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1208)
5400 [8810][0] reports Running
5401 ----------------------------------------------------------------
5402 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
5403 States: Offline Active Active
5404 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
5405 2208 Acked Flush 0 Sent Done Done false
5406 2209 Acked Write 9 Sent Done Done false
5407 2210 Acked Read 10 Sent Done Done false
5408 2211 Acked Write 3 Sent Done Done false
5409 2212 Acked Read 2 Sent Done Done false
5410 2213 Acked Write 1 Sent Done Done false
5411 2214 Acked Write 7 Sent Done Done false
5412 2215 Acked Read 3 Sent Done Done false
5413 2216 Acked Read 4 Sent Done Done false
5414 2217 Acked Write 8 Sent Done Done false
5415 2218 Acked Read 1 Sent Done Done false
5416 2219 Acked Read 5 Sent Done Done false
5417 2220 Acked Read 8 Sent Done Done false
5418 2221 Acked Read 2 Sent Done Done false
5419 2222 Acked Flush 0 Sent Done Done false
5420 2223 Acked Read 9 Sent Done Done false
5421 2224 Acked Write 10 Sent Done Done false
5422 2225 Acked Read 7 Sent Done Done false
5423 2226 Acked Read 7 Sent Done Done false
5424 2227 Acked Flush 0 Sent Done Done false
5425 2228 Acked Write 9 Sent Done Done false
5426 2229 Acked Flush 0 Sent Done Done false
5427 2230 Acked Write 2 Sent Done Done false
5428 2231 Acked Write 5 Sent Done Done false
5429 2232 Acked Write 9 Sent Done Done false
5430 2233 Acked Read 7 Sent Done Done false
5431 2234 Acked Write 1 Sent Done Done false
5432 2235 Acked Read 10 Sent Done Done false
5433 2236 Acked Write 1 Sent Done Done false
5434 2237 Acked Write 9 Sent Done Done false
5435 2238 Acked Read 2 Sent Done Done false
5436 2239 Acked Write 8 Sent Done Done false
5437 2240 Acked Read 5 Sent Done Done false
5438 2241 Acked Read 2 Sent Done Done false
5439 2242 Acked Read 10 Sent Done Done false
5440 2243 Acked Write 8 Sent Done Done false
5441 2244 Acked Read 9 Sent Done Done false
5442 2245 Acked Write 7 Sent Done Done false
5443 2246 Acked Read 3 Sent Done Done false
5444 2247 Acked Write 2 Sent Done Done false
5445 2248 Acked Read 4 Sent Done Done false
5446 2249 Acked Flush 0 Sent Done Done false
5447 2250 Acked Write 5 Sent Done Done false
5448 2251 Acked Write 8 Sent Done Done false
5449 2252 Acked Write 9 Sent Done Done false
5450 2253 Acked Read 3 Sent Done Done false
5451 2254 Acked Flush 0 Sent Done Done false
5452 2255 Acked Flush 0 Sent Done Done false
5453 2256 Acked Read 1 Sent Done Done false
5454 2257 Acked Write 8 Sent Done Done false
5455 2258 Acked Write 4 Sent Done Done false
5456 2259 Acked Read 9 Sent Done Done false
5457 2260 Acked Read 7 Sent Done Done false
5458 2261 Acked Read 1 Sent Done Done false
5459 2262 Acked Flush 0 Sent Done Done false
5460 2263 Acked Read 3 Sent Done Done false
5461 2264 Acked Write 4 Sent Done Done false
5462 2265 Acked Read 4 Sent Done Done false
5463 2266 Acked Read 4 Sent Done Done false
5464 2267 Acked Read 10 Sent Done Done false
5465 2268 Acked Read 6 Sent Done Done false
5466 2269 Acked Read 7 Sent Done Done false
5467 2270 Acked Read 9 Sent Done Done false
5468 2271 Acked Write 3 Sent Done Done false
5469 2272 Acked Read 3 Sent Done Done false
5470 2273 Acked Read 10 Sent Done Done false
5471 2274 Acked Read 4 Sent Done Done false
5472 2275 Acked Read 2 Sent Done Done false
5473 2276 Acked Flush 0 Sent Done Done false
5474 2277 Acked Write 2 Sent Done Done false
5475 2278 Acked Read 10 Sent Done Done false
5476 2279 Acked Read 6 Sent Done Done false
5477 2280 Acked Write 2 Sent Done Done false
5478 2281 Acked Read 10 Sent Done Done false
5479 2282 Acked Write 5 Sent Done Done false
5480 2283 Acked Write 6 Sent Done Done false
5481 2284 Acked Flush 0 Sent Done Done false
5482 2285 Acked Write 2 Sent Done Done false
5483 2286 Acked Read 2 Sent Done Done false
5484 2287 Acked Write 4 Sent Done Done false
5485 2288 Acked Read 2 Sent Done Done false
5486 2289 Acked Read 5 Sent Done Done false
5487 2290 Acked Write 9 Sent Done Done false
5488 2291 Acked Read 10 Sent Done Done false
5489 2292 Acked Read 7 Sent Done Done false
5490 2293 Acked Read 8 Sent Done Done false
5491 2294 Acked Read 7 Sent Done Done false
5492 2295 Acked Write 9 Sent Done Done false
5493 2296 Acked Read 5 Sent Done Done false
5494 2297 Acked Write 7 Sent Done Done false
5495 2298 Acked Read 2 Sent Done Done false
5496 2299 Acked Write 1 Sent Done Done false
5497 2300 Acked Read 8 Sent Done Done false
5498 2301 Acked Flush 0 Sent Done Done false
5499 2302 Acked Write 2 Sent Done Done false
5500 2303 Acked Write 1 Sent Done Done false
5501 2304 Acked Write 5 Sent Done Done false
5502 2305 Acked Read 4 Sent Done Done false
5503 2306 Acked Read 3 Sent Done Done false
5504 2307 Acked Write 3 Sent Done Done false
5505 2308 Acked Write 1 Sent Done Done false
5506 2309 Acked Write 9 Sent Done Done false
5507 2310 Acked Write 3 Sent Done Done false
5508 2311 Acked Write 3 Sent Done Done false
5509 2312 Acked Write 2 Sent Done Done false
5510 2313 Acked Read 8 Sent Done Done false
5511 2314 Acked Write 8 Sent Done Done false
5512 2315 Acked Read 8 Sent Done Done false
5513 2316 Acked Read 10 Sent Done Done false
5514 2317 Acked Read 10 Sent Done Done false
5515 2318 Acked Read 5 Sent Done Done false
5516 2319 Acked Read 9 Sent Done Done false
5517 2320 Acked Read 10 Sent Done Done false
5518 2321 Acked Write 9 Sent Done Done false
5519 2322 Acked Read 1 Sent Done Done false
5520 2323 Acked Write 7 Sent Done Done false
5521 2324 Acked Read 3 Sent Done Done false
5522 2325 Acked Write 5 Sent Done Done false
5523 2326 Acked Write 3 Sent Done Done false
5524 2327 Acked Write 4 Sent Done Done false
5525 2328 Acked Read 2 Sent Done Done false
5526 2329 Acked Read 8 Sent Done Done false
5527 2330 Acked Flush 0 Sent Done Done false
5528 2331 Acked Flush 0 Sent Done Done false
5529 2332 Acked Write 6 Sent Done Done false
5530 2333 Acked Write 1 Sent Done Done false
5531 2334 Acked Write 9 Sent Done Done false
5532 2335 Acked Write 1 Sent Done Done false
5533 2336 Acked Write 3 Sent Done Done false
5534 2337 Acked Read 6 Sent Done Done false
5535 2338 Acked Read 4 Sent Done Done false
5536 2339 Acked Write 6 Sent Done Done false
5537 2340 Acked Write 10 Sent Done Done false
5538 2341 Acked Write 1 Sent Done Done false
5539 2342 Acked Write 6 Sent Done Done false
5540 2343 Acked Read 2 Sent Done Done false
5541 2344 Acked Flush 0 Sent Done Done false
5542 2345 Acked Read 3 Sent Done Done false
5543 2346 Acked Write 3 Sent Done Done false
5544 2347 Acked Read 9 Sent Done Done false
5545 2348 Acked Write 2 Sent Done Done false
5546 2349 Acked Read 10 Sent Done Done false
5547 2350 Acked Read 5 Sent Done Done false
5548 2351 Acked Write 6 Sent Done Done false
5549 2352 Acked Write 2 Sent Done Done false
5550 2353 Acked Read 10 Sent Done Done false
5551 2354 Acked Write 6 Sent Done Done false
5552 2355 Acked Write 4 Sent Done Done false
5553 2356 Acked Write 7 Sent Done Done false
5554 2357 Acked Read 7 Sent Done Done false
5555 2358 Acked Read 4 Sent Done Done false
5556 2359 Acked Flush 0 Sent Done Done false
5557 2360 Acked Read 5 Sent Done Done false
5558 2361 Acked Read 10 Sent Done Done false
5559 2362 Acked Read 1 Sent Done Done false
5560 2363 Acked Read 7 Sent Done Done false
5561 2364 Acked Read 2 Sent Done Done false
5562 2365 Acked Read 3 Sent Done Done false
5563 2366 Acked Read 2 Sent Done Done false
5564 2367 Acked Read 4 Sent Done Done false
5565 2368 Acked Read 5 Sent Done Done false
5566 2369 Acked Write 10 Sent Done Done false
5567 2370 Acked Flush 0 Sent Done Done false
5568 2371 Acked Read 7 Sent Done Done false
5569 2372 Acked Write 10 Sent Done Done false
5570 2373 Acked Write 7 Sent Done Done false
5571 2374 Acked Write 1 Sent Done Done false
5572 2375 Acked Read 5 Sent Done Done false
5573 2376 Acked Write 2 Sent Done Done false
5574 2377 Acked Read 4 Sent Done Done false
5575 2378 Acked Read 7 Sent Done Done false
5576 2379 Acked Write 4 Sent Done Done false
5577 2380 Acked Write 3 Sent Done Done false
5578 2381 Acked Read 8 Sent Done Done false
5579 2382 Acked Write 5 Sent Done Done false
5580 2383 Acked Read 1 Sent Done Done false
5581 2384 Acked Write 7 Sent Done Done false
5582 2385 Acked Read 7 Sent Done Done false
5583 2386 Acked Read 9 Sent Done Done false
5584 2387 Acked Flush 0 Sent Done Done false
5585 2388 Acked Write 5 Sent Done Done false
5586 2389 Acked Write 4 Sent Done Done false
5587 2390 Acked Read 5 Sent Done Done false
5588 2391 Acked Write 3 Sent Done Done false
5589 2392 Acked Write 2 Sent Done Done false
5590 2393 Acked Read 2 Sent Done Done false
5591 2394 Acked Read 6 Sent Done Done false
5592 2395 Acked Flush 0 Sent Done Done false
5593 2396 Acked Write 7 Sent Done Done false
5594 2397 Acked Write 1 Sent Done Done false
5595 2398 Acked Read 10 Sent Done Done false
5596 2399 Acked Read 4 Sent Done Done false
5597 2400 Acked Write 6 Sent Done Done false
5598 2401 Acked Read 5 Sent Done Done false
5599 2402 Acked Read 2 Sent Done Done false
5600 2403 Acked Read 7 Sent Done Done false
5601 2404 Acked Write 5 Sent Done Done false
5602 2405 Acked Write 3 Sent Done Done false
5603 2406 Acked Write 4 Sent Done Done false
5604 2407 Acked Read 6 Sent Done Done false
5605 2408 Acked Read 2 Sent Done Done false
5606 2409 Acked Read 3 Sent Done Done false
5607 2410 Acked Flush 0 Sent Done Done false
5608 2411 Acked Read 4 Sent Done Done false
5609 2412 Acked Read 2 Sent Done Done false
5610 2413 Acked Write 6 Sent Done Done false
5611 2414 Acked Read 4 Sent Done Done false
5612 2415 Acked Read 6 Sent Done Done false
5613 2416 Acked Write 1 Sent Done Done false
5614 2417 Acked Flush 0 Sent Done Done false
5615 2418 Acked Read 10 Sent Done Done false
5616 2419 Acked Read 4 Sent Done Done false
5617 2420 Acked Write 2 Sent Done Done false
5618 2421 Acked Flush 0 Sent Done Done false
5619 2422 Acked Flush 0 Sent Done Done false
5620 2423 Acked Read 5 Sent Done Done false
5621 2424 Acked Read 2 Sent Done Done false
5622 2425 Acked Write 6 Sent Done Done false
5623 2426 Acked Read 8 Sent Done Done false
5624 2427 Acked Read 7 Sent Done Done false
5625 2428 Acked Read 8 Sent Done Done false
5626 2429 Acked Write 5 Sent Done Done false
5627 2430 Acked Read 2 Sent Done Done false
5628 2431 Acked Write 9 Sent Done Done false
5629 2432 Acked Read 3 Sent Done Done false
5630 2433 Acked Write 5 Sent Done Done false
5631 2434 Acked Read 9 Sent Done Done false
5632 2435 Acked Write 9 Sent Done Done false
5633 2436 Acked Flush 0 Sent Done Done false
5634 2437 Acked Read 6 Sent Done Done false
5635 2438 Acked Read 9 Sent Done Done false
5636 2439 Acked Read 2 Sent Done Done false
5637 2440 Acked Flush 0 Sent Done Done false
5638 2441 Acked Write 1 Sent Done Done false
5639 2442 Acked Read 4 Sent Done Done false
5640 2443 Acked Write 9 Sent Done Done false
5641 2444 Acked Read 4 Sent Done Done false
5642 2445 Acked Write 9 Sent Done Done false
5643 2446 Acked Read 6 Sent Done Done false
5644 2447 Acked Write 10 Sent Done Done false
5645 2448 Acked Write 4 Sent Done Done false
5646 2449 Acked Write 8 Sent Done Done false
5647 2450 Acked Read 4 Sent Done Done false
5648 2451 Acked Read 2 Sent Done Done false
5649 2452 Acked Read 2 Sent Done Done false
5650 2453 Acked Flush 0 Sent Done Done false
5651 2454 Acked Read 1 Sent Done Done false
5652 2455 Acked Write 2 Sent Done Done false
5653 2456 Acked Write 10 Sent Done Done false
5654 2457 Acked Read 6 Sent Done Done false
5655 2458 Acked Read 7 Sent Done Done false
5656 2459 Acked Read 4 Sent Done Done false
5657 2460 Acked Write 4 Sent Done Done false
5658 2461 Acked Write 7 Sent Done Done false
5659 2462 Acked Read 2 Sent Done Done false
5660 2463 Acked Read 2 Sent Done Done false
5661 2464 Acked Read 4 Sent Done Done false
5662 2465 Acked Read 2 Sent Done Done false
5663 2466 Acked Read 5 Sent Done Done false
5664 2467 Acked Read 2 Sent Done Done false
5665 2468 Acked Read 4 Sent Done Done false
5666 2469 Acked Read 2 Sent Done Done false
5667 2470 Acked Write 5 Sent Done Done false
5668 2471 Acked Flush 0 Sent Done Done false
5669 2472 Acked Write 9 Sent Done Done false
5670 2473 Acked Flush 0 Sent Done Done false
5671 2474 Acked Write 1 Sent Done Done false
5672 2475 Acked Read 7 Sent Done Done false
5673 2476 Acked Write 7 Sent Done Done false
5674 2477 Acked Write 2 Sent Done Done false
5675 2478 Acked Read 3 Sent Done Done false
5676 2479 Acked Read 10 Sent Done Done false
5677 2480 Acked Read 10 Sent Done Done false
5678 2481 Acked Read 3 Sent Done Done false
5679 2482 Acked Write 7 Sent Done Done false
5680 2483 Acked Read 6 Sent Done Done false
5681 2484 Acked Flush 0 Sent Done Done false
5682 2485 Acked Write 3 Sent Done Done false
5683 2486 Acked Write 10 Sent Done Done false
5684 2487 Acked Read 6 Sent Done Done false
5685 2488 Acked Write 5 Sent Done Done false
5686 2489 Acked Write 1 Sent Done Done false
5687 2490 Acked Flush 0 Sent Done Done false
5688 2491 Acked Read 6 Sent Done Done false
5689 2492 Acked Write 5 Sent Done Done false
5690 2493 Acked Write 8 Sent Done Done false
5691 2494 Acked Write 7 Sent Done Done false
5692 2495 Acked Write 1 Sent Done Done false
5693 2496 Acked Write 5 Sent Done Done false
5694 2497 Acked Read 8 Sent Done Done false
5695 2498 Acked Read 4 Sent Done Done false
5696 2499 Acked Read 7 Sent Done Done false
5697 2500 Acked Write 6 Sent Done Done false
5698 2501 Acked Read 3 Sent Done Done false
5699 2502 Acked Flush 0 Sent Done Done false
5700 2503 Acked Write 8 Sent Done Done false
5701 2504 Acked Write 10 Sent Done Done false
5702 2505 Acked Write 10 Sent Done Done false
5703 2506 Acked Write 10 Sent Done Done false
5704 2507 Acked Read 1 Sent Done Done false
5705 2508 Acked Read 3 Sent Done Done false
5706 2509 Acked Flush 0 Sent Done Done false
5707 STATES DS:0 DS:1 DS:2 TOTAL
5708 Sent 302 0 0 302
5709 Done 0 302 302 604
5710 Skipped 0 0 0 0
5711 Error 0 0 0 0
5712 Last Flush: 2207 2509 2509
5713 Downstairs last five completed: 2207 2206 2205 2204 2203
5714 Upstairs last five completed: 2509 2508 2507 2506 2505
5715 CLIENT: Up:0 ds:302 act:2
57162024-11-26T22:05:43.388ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
57172024-11-26T22:05:43.388ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:54111) = io task client = 0 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
57182024-11-26T22:05:43.389ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
57192024-11-26T22:05:43.389ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
57202024-11-26T22:05:43.389ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
57212024-11-26T22:05:43.389ZINFOcrucible: send last flush ID to this DS: 2207 = downstairs client = 0 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
57222024-11-26T22:05:43.389ZINFOcrucible: Replied this last flush ID: 2207 = downstairs client = 0 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
57232024-11-26T22:05:43.390ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
57242024-11-26T22:05:43.390ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
57252024-11-26T22:05:43.390ZINFOcrucible: [0] client re-new 302 jobs since flush 2207 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
57262024-11-26T22:05:43.390ZINFOcrucible: [0] Replayed 302 jobs since flush: 2207 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
5727 ----------------------------------------------------------------
5728 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
5729 Downstairs last five completed: 2509 2508 2507 2506 2505
5730 Upstairs last five completed: 2509 2508 2507 2506 2505
5731 CLIENT: Up:0 ds:0 act:3
5732 Replay: All jobs finished
5733 Main task has work to do, go find it
5734 got dsc Stop(1)
5735 stop 1
5736 Nov 26 22:05:45.622 INFO request completed, latency_us: 313, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 77835dc2-1740-45f7-a43e-37331202ad1c, [8820] Got stop action so:false kr:false
5737 remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
57382024-11-26T22:05:45.622ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
57392024-11-26T22:05:45.622ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
57402024-11-26T22:05:45.622ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
57412024-11-26T22:05:45.622ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
57422024-11-26T22:05:45.623ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
57432024-11-26T22:05:45.623ZINFOcrucible: client task is exiting = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
5744 Nov 26 22:05:45.623 INFO request completed, latency_us: 118, response_code: 200, uri: /state/cid/1, method: GET, req_id: af7df293-32eb-4410-8e5c-cf796fd06901, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
5745 [8820] Exited with: ExitStatus(unix_wait_status(9))
5746 [8820][1] reports Exit
5747 Nov 26 22:05:49.624 INFO request completed, latency_us: 214, response_code: 200, uri: /state/cid/1, method: GET, req_id: bfb9af35-1436-44ab-ae69-5adfca6ec540, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
5748 001/300 Read block 560 len 36864
5749 002/300 Write block 3079 len 12288 data: 2 2 2
5750 003/300 Read block 2452 len 32768
5751 004/300 Write block 2663 len 36864 data: 3 3 3 2 3 3 2 2 2
5752 005/300 Write block 2861 len 36864 data: 2 2 2 2 2 2 2 2 2
5753 006/300 Write block 3653 len 24576 data: 5 4 2 2 2 2
5754 007/300 Write block 4741 len 4096 data: 2
5755 008/300 Write block 448 len 32768 data: 2 2 2 2 2 2 2 2
5756 009/300 Write block 289 len 40960 data: 4 4 5 5 5 5 5 4 3 2
5757 010/300 Write block 2421 len 24576 data: 2 4 4 4 3 3
5758 011/300 Write block 3914 len 32768 data: 2 2 2 2 2 2 2 3
5759 012/300 Read block 4198 len 40960
5760 013/300 Write block 3873 len 28672 data: 3 3 3 2 2 2 2
5761 014/300 Read block 657 len 24576
5762 015/300 Read block 3705 len 20480
5763 016/300 Write block 1504 len 20480 data: 3 2 2 2 2
5764 017/300 Read block 1466 len 12288
5765 018/300 Read block 4656 len 24576
5766 019/300 Write block 2136 len 4096 data: 3
5767 020/300 Read block 3246 len 28672
5768 021/300 Write block 4965 len 20480 data: 5 4 3 3 3
5769 022/300 Read block 3365 len 32768
5770 023/300 Read block 2103 len 28672
5771 024/300 Write block 2342 len 20480 data: 2 2 2 2 2
5772 025/300 Read block 3750 len 12288
5773 026/300 Write block 2829 len 28672 data: 3 3 3 3 2 2 2
5774 027/300 Write block 333 len 32768 data: 2 2 2 2 3 3 3 3
5775 028/300 Read block 1208 len 20480
5776 029/300 Write block 4618 len 32768 data: 2 3 3 3 3 3 3 3
5777 030/300 Flush
5778 031/300 Read block 4888 len 40960
5779 032/300 Write block 2623 len 8192 data: 2 2
5780 033/300 Write block 376 len 16384 data: 2 2 2 3
5781 034/300 Read block 3649 len 36864
5782 035/300 Write block 559 len 28672 data: 2 3 3 3 3 3 3
5783 036/300 Write block 4700 len 32768 data: 2 2 2 3 3 2 2 2
5784 037/300 Write block 944 len 16384 data: 3 3 3 3
5785 038/300 Read block 1956 len 32768
5786 039/300 Read block 3538 len 32768
5787 040/300 Read block 3910 len 28672
5788 041/300 Flush
5789 042/300 Read block 1613 len 24576
5790 043/300 Read block 3990 len 8192
5791 044/300 Read block 1051 len 16384
5792 045/300 Read block 1861 len 16384
5793 046/300 Read block 3429 len 28672
5794 047/300 Read block 1561 len 40960
5795 048/300 Read block 677 len 28672
5796 049/300 Read block 215 len 20480
5797 050/300 Flush
5798 051/300 Read block 4747 len 4096
5799 052/300 Read block 609 len 8192
5800 053/300 Write block 4785 len 8192 data: 2 2
5801 054/300 Write block 2952 len 20480 data: 2 2 2 2 2
5802 055/300 Read block 799 len 4096
5803 056/300 Read block 4334 len 32768
5804 057/300 Flush
5805 058/300 Write block 4492 len 16384 data: 4 4 3 3
5806 059/300 Write block 1927 len 32768 data: 3 3 3 3 2 3 3 3
5807 060/300 Read block 2219 len 4096
5808 061/300 Write block 3 len 28672 data: 3 3 3 3 3 3 2
5809 062/300 Write block 1653 len 36864 data: 2 2 2 2 2 3 3 3 3
5810 063/300 Write block 636 len 40960 data: 2 2 2 3 3 3 3 3 3 2
5811 064/300 Read block 491 len 4096
5812 065/300 Write block 4217 len 20480 data: 3 3 2 2 2
5813 066/300 Read block 3920 len 24576
5814 067/300 Write block 852 len 4096 data: 2
5815 068/300 Write block 3339 len 24576 data: 2 2 2 2 2 3
5816 069/300 Write block 1020 len 16384 data: 2 3 2 2
5817 070/300 Write block 4442 len 12288 data: 3 3 3
5818 071/300 Flush
5819 072/300 Read block 940 len 24576
5820 073/300 Write block 742 len 8192 data: 2 2
5821 074/300 Write block 4655 len 40960 data: 2 2 2 2 2 2 2 2 2 2
5822 075/300 Read block 763 len 32768
5823 076/300 Read block 4753 len 32768
5824 077/300 Write block 2062 len 40960 data: 4 4 4 4 3 2 3 3 3 3
5825 078/300 Write block 4088 len 16384 data: 2 2 2 2
5826 079/300 Write block 702 len 8192 data: 2 2
5827 080/300 Read block 2070 len 4096
5828 081/300 Write block 420 len 40960 data: 2 2 2 2 2 2 2 2 2 2
5829 082/300 Flush
5830 083/300 Read block 1054 len 24576
5831 084/300 Write block 1696 len 28672 data: 2 2 2 2 2 2 2
5832 085/300 Flush
5833 086/300 Read block 1682 len 24576
5834 087/300 Read block 2681 len 16384
5835 088/300 Write block 741 len 28672 data: 2 3 3 2 3 3 2
5836 089/300 Write block 323 len 8192 data: 2 2
5837 090/300 Read block 743 len 36864
5838 091/300 Read block 1926 len 20480
5839 092/300 Read block 4501 len 12288
5840 093/300 Write block 1673 len 16384 data: 4 4 3 3
5841 094/300 Read block 4120 len 24576
5842 095/300 Write block 2999 len 8192 data: 3 3
5843 096/300 Read block 575 len 32768
5844 097/300 Read block 4981 len 12288
5845 098/300 Write block 4863 len 12288 data: 5 4 2
5846 099/300 Write block 664 len 16384 data: 2 2 3 3
5847 100/300 Read block 552 len 36864
5848 101/300 Write block 1982 len 28672 data: 2 2 2 2 3 3 3
5849 102/300 Read block 267 len 4096
5850 103/300 Read block 1646 len 8192
5851 104/300 Read block 4788 len 8192
5852 105/300 Read block 2106 len 4096
5853 106/300 Write block 2280 len 12288 data: 2 2 3
5854 107/300 Read block 1778 len 36864
5855 108/300 Read block 2843 len 12288
5856 109/300 Read block 1410 len 28672
5857 110/300 Read block 1604 len 28672
5858 111/300 Read block 1411 len 4096
5859 112/300 Write block 3117 len 36864 data: 2 3 3 3 3 3 2 2 2
5860 113/300 Write block 2370 len 36864 data: 2 2 2 3 3 3 2 2 3
5861 114/300 Flush
5862 115/300 Read block 1763 len 8192
5863 116/300 Write block 3619 len 4096 data: 2
5864 117/300 Write block 3360 len 8192 data: 2 2
5865 118/300 Read block 21 len 16384
5866 119/300 Write block 1196 len 40960 data: 2 3 3 3 3 4 4 4 4 4
5867 120/300 Write block 4939 len 8192 data: 3 2
5868 121/300 Read block 4346 len 4096
5869 122/300 Flush
5870 123/300 Write block 2161 len 40960 data: 4 4 3 3 5 5 4 4 4 3
5871 124/300 Read block 3061 len 28672
5872 125/300 Read block 4468 len 16384
5873 126/300 Read block 4067 len 4096
5874 127/300 Flush
5875 128/300 Read block 3530 len 24576
5876 129/300 Write block 1504 len 12288 data: 4 3 3
5877 130/300 Read block 4565 len 28672
5878 131/300 Write block 2642 len 4096 data: 3
5879 132/300 Read block 1368 len 40960
5880 133/300 Read block 276 len 24576
5881 134/300 Write block 737 len 32768 data: 2 2 2 2 3 4 4 3
5882 135/300 Read block 1942 len 24576
5883 136/300 Flush
5884 137/300 Read block 1053 len 4096
5885 138/300 Flush
5886 139/300 Write block 3675 len 40960 data: 4 6 6 5 4 4 4 5 5 4
5887 140/300 Flush
5888 141/300 Write block 93 len 32768 data: 2 2 2 4 4 4 4 4
5889 142/300 Write block 1911 len 4096 data: 3
5890 143/300 Flush
5891 144/300 Write block 4315 len 24576 data: 4 4 3 3 3 3
5892 145/300 Read block 390 len 32768
5893 146/300 Write block 4547 len 32768 data: 2 2 2 2 2 2 2 2
5894 147/300 Write block 1870 len 28672 data: 3 2 2 2 2 3 2
5895 148/300 Read block 3329 len 24576
5896 149/300 Write block 2387 len 16384 data: 2 2 2 3
5897 150/300 Write block 929 len 20480 data: 4 4 2 2 2
5898 151/300 Read block 561 len 24576
5899 152/300 Read block 1675 len 16384
5900 153/300 Read block 634 len 4096
5901 154/300 Read block 1083 len 28672
5902 155/300 Write block 1119 len 36864 data: 3 3 3 3 3 2 2 2 2
5903 156/300 Read block 905 len 4096
5904 157/300 Write block 4672 len 40960 data: 2 2 2 2 2 3 3 3 3 3
5905 158/300 Read block 3671 len 24576
5906 159/300 Read block 4265 len 12288
5907 160/300 Write block 4473 len 32768 data: 2 2 3 3 3 2 2 2
5908 161/300 Flush
5909 162/300 Write block 2800 len 4096 data: 2
5910 163/300 Flush
5911 164/300 Read block 775 len 16384
5912 165/300 Read block 1390 len 32768
5913 166/300 Read block 2973 len 8192
5914 167/300 Read block 2453 len 40960
5915 168/300 Write block 2836 len 8192 data: 2 2
5916 169/300 Read block 4903 len 36864
5917 170/300 Read block 696 len 32768
5918 171/300 Write block 3056 len 20480 data: 2 2 2 3 3
5919 172/300 Flush
5920 173/300 Read block 2085 len 4096
5921 174/300 Write block 4590 len 8192 data: 3 3
5922 175/300 Write block 3641 len 8192 data: 2 2
5923 176/300 Read block 1020 len 32768
5924 177/300 Read block 1495 len 12288
5925 178/300 Flush
5926 179/300 Write block 723 len 24576 data: 3 3 3 2 3 3
5927 180/300 Write block 1189 len 36864 data: 2 2 2 2 2 2 2 3 4
5928 181/300 Read block 4377 len 28672
5929 182/300 Read block 4374 len 16384
5930 183/300 Write block 2238 len 28672 data: 2 2 2 2 2 2 2
5931 184/300 Flush
5932 185/300 Read block 492 len 20480
5933 186/300 Read block 2763 len 16384
5934 187/300 Write block 2145 len 8192 data: 2 2
5935 188/300 Read block 1518 len 24576
5936 189/300 Flush
5937 190/300 Write block 3445 len 36864 data: 2 2 2 2 2 2 2 2 2
5938 191/300 Read block 4123 len 12288
5939 192/300 Read block 1573 len 24576
5940 193/300 Read block 4977 len 8192
5941 194/300 Write block 1355 len 8192 data: 4 4
5942 195/300 Write block 4833 len 36864 data: 3 3 3 3 2 2 2 2 2
5943 196/300 Write block 3421 len 8192 data: 2 2
5944 197/300 Write block 3552 len 8192 data: 5 5
5945 198/300 Read block 1100 len 24576
5946 199/300 Write block 1976 len 8192 data: 2 2
5947 200/300 Write block 2892 len 12288 data: 2 2 3
5948 201/300 Read block 807 len 4096
5949 202/300 Write block 1791 len 4096 data: 3
5950 203/300 Write block 4072 len 4096 data: 3
5951 204/300 Write block 4290 len 16384 data: 3 3 3 3
5952 205/300 Read block 1196 len 24576
5953 206/300 Write block 3145 len 32768 data: 4 4 4 3 2 2 2 2
5954 207/300 Read block 1242 len 28672
5955 208/300 Flush
5956 209/300 Write block 3556 len 8192 data: 3 2
5957 210/300 Write block 4951 len 16384 data: 2 2 2 2
5958 211/300 Write block 3314 len 4096 data: 2
5959 212/300 Read block 3530 len 20480
5960 213/300 Read block 490 len 20480
5961 214/300 Read block 2078 len 8192
5962 215/300 Read block 4264 len 20480
5963 216/300 Read block 608 len 40960
5964 217/300 Read block 1386 len 8192
5965 218/300 Write block 370 len 28672 data: 2 2 2 2 2 2 3
5966 219/300 Read block 3125 len 12288
5967 220/300 Read block 860 len 28672
5968 221/300 Read block 742 len 16384
5969 222/300 Write block 2886 len 4096 data: 2
5970 223/300 Flush
5971 224/300 Read block 3428 len 16384
5972 225/300 Write block 4885 len 12288 data: 2 2 2
5973 226/300 Write block 1747 len 24576 data: 2 2 2 2 2 2
5974 227/300 Write block 3854 len 32768 data: 2 2 2 3 3 3 3 3
5975 228/300 Flush
5976 229/300 Read block 1419 len 20480
5977 230/300 Flush
5978 231/300 Write block 2727 len 20480 data: 2 2 2 2 2
5979 232/300 Read block 1710 len 8192
5980 233/300 Flush
5981 234/300 Flush
5982 235/300 Read block 3514 len 36864
5983 236/300 Read block 3030 len 28672
5984 237/300 Read block 49 len 12288
5985 238/300 Write block 2742 len 32768 data: 2 3 3 3 3 3 3 2
5986 239/300 Read block 3160 len 32768
5987 240/300 Read block 485 len 32768
5988 241/300 Read block 2793 len 8192
5989 242/300 Flush
5990 243/300 Write block 160 len 8192 data: 3 3
5991 244/300 Read block 788 len 16384
5992 245/300 Read block 1741 len 32768
5993 246/300 Read block 2335 len 8192
5994 247/300 Read block 17 len 4096
5995 248/300 Read block 4550 len 12288
5996 249/300 Write block 3641 len 32768 data: 3 3 2 2 2 2 3 3
5997 250/300 Write block 2918 len 32768 data: 3 2 2 3 3 3 2 2
5998 251/300 Read block 4519 len 36864
5999 252/300 Read block 2561 len 16384
6000 253/300 Read block 1420 len 8192
6001 254/300 Read block 3955 len 28672
6002 255/300 Flush
6003 256/300 Read block 871 len 4096
6004 257/300 Write block 462 len 32768 data: 4 3 3 3 4 3 3 4
6005 258/300 Read block 2285 len 16384
6006 259/300 Read block 4922 len 12288
6007 260/300 Write block 4191 len 32768 data: 2 3 3 3 3 3 3 3
6008 261/300 Read block 3432 len 24576
6009 262/300 Read block 816 len 24576
6010 263/300 Write block 3321 len 40960 data: 2 2 2 2 2 2 2 2 2 2
6011 264/300 Write block 1885 len 4096 data: 3
6012 265/300 Write block 3843 len 4096 data: 2
6013 266/300 Write block 4495 len 8192 data: 4 3
6014 267/300 Read block 1053 len 24576
6015 268/300 Read block 732 len 12288
6016 269/300 Read block 4587 len 36864
6017 270/300 Read block 4878 len 32768
6018 271/300 Read block 4132 len 40960
6019 272/300 Read block 3313 len 24576
6020 273/300 Read block 757 len 28672
6021 274/300 Read block 3147 len 12288
6022 275/300 Read block 2893 len 32768
6023 276/300 Write block 1083 len 36864 data: 3 3 2 2 2 2 2 2 2
6024 277/300 Read block 2156 len 12288
6025 278/300 Read block 3814 len 28672
6026 279/300 Write block 3415 len 20480 data: 2 2 2 2 2
6027 280/300 Read block 632 len 12288
6028 281/300 Flush
6029 282/300 Read block 1753 len 12288
6030 283/300 Read block 1752 len 8192
6031 284/300 Read block 4751 len 8192
6032 285/300 Write block 3429 len 12288 data: 2 2 2
6033 286/300 Read block 1166 len 32768
6034 287/300 Write block 2883 len 32768 data: 2 2 2 3 2 2 2 2
6035 288/300 Read block 2191 len 36864
6036 289/300 Read block 2080 len 8192
6037 290/300 Write block 1338 len 20480 data: 2 2 2 2 2
6038 291/300 Write block 2739 len 36864 data: 3 3 3 3 4 4 4 4 4
6039 292/300 Write block 4451 len 16384 data: 3 4 3 3
6040 293/300 Read block 3700 len 24576
6041 294/300 Read block 3151 len 40960
6042 295/300 Write block 4109 len 32768 data: 3 3 3 3 2 2 2 2
6043 296/300 Flush
6044 297/300 Read block 595 len 12288
6045 298/300 Write block 347 len 32768 data: 2 2 3 3 3 3 3 3
6046 299/300 Read block 1865 len 36864
6047 300/300 Write block 4717 len 32768 data: 3 3 3 3 3 3 2 2
6048 301/300 Read block 1665 len 16384
6049 Main task has work to do, go find it
6050 got dsc Start(1)
6051 start 1
6052 Nov 26 22:05:49.856 INFO request completed, latency_us: 205, response_code: 204, uri: /start/cid/1, method: POST, req_id: 3ab638c5-6b58-4ba5-b714-48a0d7c00c81, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
6053 Replay: started 1, returned:Ok(())
6054 ----------------------------------------------------------------
6055 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
6056 States: Active Offline Active
6057 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
6058 2510 Acked Read 9 Done Sent Done false
6059 2511 Acked Write 3 Done Sent Done false
6060 2512 Acked Read 8 Done Sent Done false
6061 2513 Acked Write 9 Done Sent Done false
6062 2514 Acked Write 9 Done Sent Done false
6063 2515 Acked Write 6 Done Sent Done false
6064 2516 Acked Write 1 Done Sent Done false
6065 2517 Acked Write 8 Done Sent Done false
6066 2518 Acked Write 10 Done Sent Done false
6067 2519 Acked Write 6 Done Sent Done false
6068 2520 Acked Write 8 Done Sent Done false
6069 2521 Acked Read 10 Done Sent Done false
6070 2522 Acked Write 7 Done Sent Done false
6071 2523 Acked Read 6 Done Sent Done false
6072 2524 Acked Read 5 Done Sent Done false
6073 2525 Acked Write 5 Done Sent Done false
6074 2526 Acked Read 3 Done Sent Done false
6075 2527 Acked Read 6 Done Sent Done false
6076 2528 Acked Write 1 Done Sent Done false
6077 2529 Acked Read 7 Done Sent Done false
6078 2530 Acked Write 5 Done Sent Done false
6079 2531 Acked Read 8 Done Sent Done false
6080 2532 Acked Read 7 Done Sent Done false
6081 2533 Acked Write 5 Done Sent Done false
6082 2534 Acked Read 3 Done Sent Done false
6083 2535 Acked Write 7 Done Sent Done false
6084 2536 Acked Write 8 Done Sent Done false
6085 2537 Acked Read 5 Done Sent Done false
6086 2538 Acked Write 8 Done Sent Done false
6087 2539 Acked Flush 0 Done Sent Done false
6088 2540 Acked Read 10 Done Sent Done false
6089 2541 Acked Write 2 Done Sent Done false
6090 2542 Acked Write 4 Done Sent Done false
6091 2543 Acked Read 9 Done Sent Done false
6092 2544 Acked Write 7 Done Sent Done false
6093 2545 Acked Write 8 Done Sent Done false
6094 2546 Acked Write 4 Done Sent Done false
6095 2547 Acked Read 8 Done Sent Done false
6096 2548 Acked Read 8 Done Sent Done false
6097 2549 Acked Read 7 Done Sent Done false
6098 2550 Acked Flush 0 Done Sent Done false
6099 2551 Acked Read 6 Done Sent Done false
6100 2552 Acked Read 2 Done Sent Done false
6101 2553 Acked Read 4 Done Sent Done false
6102 2554 Acked Read 4 Done Sent Done false
6103 2555 Acked Read 7 Done Sent Done false
6104 2556 Acked Read 10 Done Sent Done false
6105 2557 Acked Read 7 Done Sent Done false
6106 2558 Acked Read 5 Done Sent Done false
6107 2559 Acked Flush 0 Done Sent Done false
6108 2560 Acked Read 1 Done Sent Done false
6109 2561 Acked Read 2 Done Sent Done false
6110 2562 Acked Write 2 Done Sent Done false
6111 2563 Acked Write 5 Done Sent Done false
6112 2564 Acked Read 1 Done Sent Done false
6113 2565 Acked Read 8 Done Sent Done false
6114 2566 Acked Flush 0 Done Sent Done false
6115 2567 Acked Write 4 Done Sent Done false
6116 2568 Acked Write 8 Done Sent Done false
6117 2569 Acked Read 1 Done Sent Done false
6118 2570 Acked Write 7 Done Sent Done false
6119 2571 Acked Write 9 Done Sent Done false
6120 2572 Acked Write 10 Done Sent Done false
6121 2573 Acked Read 1 Done Sent Done false
6122 2574 Acked Write 5 Done Sent Done false
6123 2575 Acked Read 6 Done Sent Done false
6124 2576 Acked Write 1 Done Sent Done false
6125 2577 Acked Write 6 Done Sent Done false
6126 2578 Acked Write 4 Done Sent Done false
6127 2579 Acked Write 3 Done Sent Done false
6128 2580 Acked Flush 0 Done Sent Done false
6129 2581 Acked Read 6 Done Sent Done false
6130 2582 Acked Write 2 Done Sent Done false
6131 2583 Acked Write 10 Done Sent Done false
6132 2584 Acked Read 8 Done Sent Done false
6133 2585 Acked Read 8 Done Sent Done false
6134 2586 Acked Write 10 Done Sent Done false
6135 2587 Acked Write 4 Done Sent Done false
6136 2588 Acked Write 2 Done Sent Done false
6137 2589 Acked Read 1 Done Sent Done false
6138 2590 Acked Write 10 Done Sent Done false
6139 2591 Acked Flush 0 Done Sent Done false
6140 2592 Acked Read 6 Done Sent Done false
6141 2593 Acked Write 7 Done Sent Done false
6142 2594 Acked Flush 0 Done Sent Done false
6143 2595 Acked Read 6 Done Sent Done false
6144 2596 Acked Read 4 Done Sent Done false
6145 2597 Acked Write 7 Done Sent Done false
6146 2598 Acked Write 2 Done Sent Done false
6147 2599 Acked Read 9 Done Sent Done false
6148 2600 Acked Read 5 Done Sent Done false
6149 2601 Acked Read 3 Done Sent Done false
6150 2602 Acked Write 4 Done Sent Done false
6151 2603 Acked Read 6 Done Sent Done false
6152 2604 Acked Write 2 Done Sent Done false
6153 2605 Acked Read 8 Done Sent Done false
6154 2606 Acked Read 3 Done Sent Done false
6155 2607 Acked Write 3 Done Sent Done false
6156 2608 Acked Write 4 Done Sent Done false
6157 2609 Acked Read 9 Done Sent Done false
6158 2610 Acked Write 7 Done Sent Done false
6159 2611 Acked Read 1 Done Sent Done false
6160 2612 Acked Read 2 Done Sent Done false
6161 2613 Acked Read 2 Done Sent Done false
6162 2614 Acked Read 1 Done Sent Done false
6163 2615 Acked Write 3 Done Sent Done false
6164 2616 Acked Read 9 Done Sent Done false
6165 2617 Acked Read 3 Done Sent Done false
6166 2618 Acked Read 7 Done Sent Done false
6167 2619 Acked Read 7 Done Sent Done false
6168 2620 Acked Read 1 Done Sent Done false
6169 2621 Acked Write 9 Done Sent Done false
6170 2622 Acked Write 9 Done Sent Done false
6171 2623 Acked Flush 0 Done Sent Done false
6172 2624 Acked Read 2 Done Sent Done false
6173 2625 Acked Write 1 Done Sent Done false
6174 2626 Acked Write 2 Done Sent Done false
6175 2627 Acked Read 4 Done Sent Done false
6176 2628 Acked Write 10 Done Sent Done false
6177 2629 Acked Write 2 Done Sent Done false
6178 2630 Acked Read 1 Done Sent Done false
6179 2631 Acked Flush 0 Done Sent Done false
6180 2632 Acked Write 10 Done Sent Done false
6181 2633 Acked Read 7 Done Sent Done false
6182 2634 Acked Read 4 Done Sent Done false
6183 2635 Acked Read 1 Done Sent Done false
6184 2636 Acked Flush 0 Done Sent Done false
6185 2637 Acked Read 6 Done Sent Done false
6186 2638 Acked Write 3 Done Sent Done false
6187 2639 Acked Read 7 Done Sent Done false
6188 2640 Acked Write 1 Done Sent Done false
6189 2641 Acked Read 10 Done Sent Done false
6190 2642 Acked Read 6 Done Sent Done false
6191 2643 Acked Write 8 Done Sent Done false
6192 2644 Acked Read 6 Done Sent Done false
6193 2645 Acked Flush 0 Done Sent Done false
6194 2646 Acked Read 1 Done Sent Done false
6195 2647 Acked Flush 0 Done Sent Done false
6196 2648 Acked Write 10 Done Sent Done false
6197 2649 Acked Flush 0 Done Sent Done false
6198 2650 Acked Write 8 Done Sent Done false
6199 2651 Acked Write 1 Done Sent Done false
6200 2652 Acked Flush 0 Done Sent Done false
6201 2653 Acked Write 6 Done Sent Done false
6202 2654 Acked Read 8 Done Sent Done false
6203 2655 Acked Write 8 Done Sent Done false
6204 2656 Acked Write 7 Done Sent Done false
6205 2657 Acked Read 6 Done Sent Done false
6206 2658 Acked Write 4 Done Sent Done false
6207 2659 Acked Write 5 Done Sent Done false
6208 2660 Acked Read 6 Done Sent Done false
6209 2661 Acked Read 4 Done Sent Done false
6210 2662 Acked Read 1 Done Sent Done false
6211 2663 Acked Read 7 Done Sent Done false
6212 2664 Acked Write 9 Done Sent Done false
6213 2665 Acked Read 1 Done Sent Done false
6214 2666 Acked Write 10 Done Sent Done false
6215 2667 Acked Read 6 Done Sent Done false
6216 2668 Acked Read 3 Done Sent Done false
6217 2669 Acked Write 8 Done Sent Done false
6218 2670 Acked Flush 0 Done Sent Done false
6219 2671 Acked Write 1 Done Sent Done false
6220 2672 Acked Flush 0 Done Sent Done false
6221 2673 Acked Read 4 Done Sent Done false
6222 2674 Acked Read 8 Done Sent Done false
6223 2675 Acked Read 2 Done Sent Done false
6224 2676 Acked Read 10 Done Sent Done false
6225 2677 Acked Write 2 Done Sent Done false
6226 2678 Acked Read 9 Done Sent Done false
6227 2679 Acked Read 8 Done Sent Done false
6228 2680 Acked Write 5 Done Sent Done false
6229 2681 Acked Flush 0 Done Sent Done false
6230 2682 Acked Read 1 Done Sent Done false
6231 2683 Acked Write 2 Done Sent Done false
6232 2684 Acked Write 2 Done Sent Done false
6233 2685 Acked Read 8 Done Sent Done false
6234 2686 Acked Read 3 Done Sent Done false
6235 2687 Acked Flush 0 Done Sent Done false
6236 2688 Acked Write 6 Done Sent Done false
6237 2689 Acked Write 9 Done Sent Done false
6238 2690 Acked Read 7 Done Sent Done false
6239 2691 Acked Read 4 Done Sent Done false
6240 2692 Acked Write 7 Done Sent Done false
6241 2693 Acked Flush 0 Done Sent Done false
6242 2694 Acked Read 5 Done Sent Done false
6243 2695 Acked Read 4 Done Sent Done false
6244 2696 Acked Write 2 Done Sent Done false
6245 2697 Acked Read 6 Done Sent Done false
6246 2698 Acked Flush 0 Done Sent Done false
6247 2699 Acked Write 9 Done Sent Done false
6248 2700 Acked Read 3 Done Sent Done false
6249 2701 Acked Read 6 Done Sent Done false
6250 2702 Acked Read 2 Done Sent Done false
6251 2703 Acked Write 2 Done Sent Done false
6252 2704 Acked Write 9 Done Sent Done false
6253 2705 Acked Write 2 Done Sent Done false
6254 2706 Acked Write 2 Done Sent Done false
6255 2707 Acked Read 6 Done Sent Done false
6256 2708 Acked Write 2 Done Sent Done false
6257 2709 Acked Write 3 Done Sent Done false
6258 2710 Acked Read 1 Done Sent Done false
6259 2711 Acked Write 1 Done Sent Done false
6260 2712 Acked Write 1 Done Sent Done false
6261 2713 Acked Write 4 Done Sent Done false
6262 2714 Acked Read 6 Done Sent Done false
6263 2715 Acked Write 8 Done Sent Done false
6264 2716 Acked Read 7 Done Sent Done false
6265 2717 Acked Flush 0 Done Sent Done false
6266 2718 Acked Write 2 Done Sent Done false
6267 2719 Acked Write 4 Done Sent Done false
6268 2720 Acked Write 1 Done Sent Done false
6269 2721 Acked Read 5 Done Sent Done false
6270 2722 Acked Read 5 Done Sent Done false
6271 2723 Acked Read 2 Done Sent Done false
6272 2724 Acked Read 5 Done Sent Done false
6273 2725 Acked Read 10 Done Sent Done false
6274 2726 Acked Read 2 Done Sent Done false
6275 2727 Acked Write 7 Done Sent Done false
6276 2728 Acked Read 3 Done Sent Done false
6277 2729 Acked Read 7 Done Sent Done false
6278 2730 Acked Read 4 Done Sent Done false
6279 2731 Acked Write 1 Done Sent Done false
6280 2732 Acked Flush 0 Done Sent Done false
6281 2733 Acked Read 4 Done Sent Done false
6282 2734 Acked Write 3 Done Sent Done false
6283 2735 Acked Write 6 Done Sent Done false
6284 2736 Acked Write 8 Done Sent Done false
6285 2737 Acked Flush 0 Done Sent Done false
6286 2738 Acked Read 5 Done Sent Done false
6287 2739 Acked Flush 0 Done Sent Done false
6288 2740 Acked Write 5 Done Sent Done false
6289 2741 Acked Read 2 Done Sent Done false
6290 2742 Acked Flush 0 Done Sent Done false
6291 2743 Acked Flush 0 Done Sent Done false
6292 2744 Acked Read 9 Done Sent Done false
6293 2745 Acked Read 7 Done Sent Done false
6294 2746 Acked Read 3 Done Sent Done false
6295 2747 Acked Write 8 Done Sent Done false
6296 2748 Acked Read 8 Done Sent Done false
6297 2749 Acked Read 8 Done Sent Done false
6298 2750 Acked Read 2 Done Sent Done false
6299 2751 Acked Flush 0 Done Sent Done false
6300 2752 Acked Write 2 Done Sent Done false
6301 2753 Acked Read 4 Done Sent Done false
6302 2754 Acked Read 8 Done Sent Done false
6303 2755 Acked Read 2 Done Sent Done false
6304 2756 Acked Read 1 Done Sent Done false
6305 2757 Acked Read 3 Done Sent Done false
6306 2758 Acked Write 8 Done Sent Done false
6307 2759 Acked Write 8 Done Sent Done false
6308 2760 Acked Read 9 Done Sent Done false
6309 2761 Acked Read 4 Done Sent Done false
6310 2762 Acked Read 2 Done Sent Done false
6311 2763 Acked Read 7 Done Sent Done false
6312 2764 Acked Flush 0 Done Sent Done false
6313 2765 Acked Read 1 Done Sent Done false
6314 2766 Acked Write 8 Done Sent Done false
6315 2767 Acked Read 4 Done Sent Done false
6316 2768 Acked Read 3 Done Sent Done false
6317 2769 Acked Write 8 Done Sent Done false
6318 2770 Acked Read 6 Done Sent Done false
6319 2771 Acked Read 6 Done Sent Done false
6320 2772 Acked Write 10 Done Sent Done false
6321 2773 Acked Write 1 Done Sent Done false
6322 2774 Acked Write 1 Done Sent Done false
6323 2775 Acked Write 2 Done Sent Done false
6324 2776 Acked Read 6 Done Sent Done false
6325 2777 Acked Read 3 Done Sent Done false
6326 2778 Acked Read 9 Done Sent Done false
6327 2779 Acked Read 8 Done Sent Done false
6328 2780 Acked Read 10 Done Sent Done false
6329 2781 Acked Read 6 Done Sent Done false
6330 2782 Acked Read 7 Done Sent Done false
6331 2783 Acked Read 3 Done Sent Done false
6332 2784 Acked Read 8 Done Sent Done false
6333 2785 Acked Write 9 Done Sent Done false
6334 2786 Acked Read 3 Done Sent Done false
6335 2787 Acked Read 7 Done Sent Done false
6336 2788 Acked Write 5 Done Sent Done false
6337 2789 Acked Read 3 Done Sent Done false
6338 2790 Acked Flush 0 Done Sent Done false
6339 2791 Acked Read 3 Done Sent Done false
6340 2792 Acked Read 2 Done Sent Done false
6341 2793 Acked Read 2 Done Sent Done false
6342 2794 Acked Write 3 Done Sent Done false
6343 2795 Acked Read 8 Done Sent Done false
6344 2796 Acked Write 8 Done Sent Done false
6345 2797 Acked Read 9 Done Sent Done false
6346 2798 Acked Read 2 Done Sent Done false
6347 2799 Acked Write 5 Done Sent Done false
6348 2800 Acked Write 9 Done Sent Done false
6349 2801 Acked Write 4 Done Sent Done false
6350 2802 Acked Read 6 Done Sent Done false
6351 2803 Acked Read 10 Done Sent Done false
6352 2804 Acked Write 8 Done Sent Done false
6353 2805 Acked Flush 0 Done Sent Done false
6354 2806 Acked Read 3 Done Sent Done false
6355 2807 Acked Write 8 Done Sent Done false
6356 2808 Acked Read 9 Done Sent Done false
6357 2809 Acked Write 8 Done Sent Done false
6358 2810 Acked Read 4 Done Sent Done false
6359 STATES DS:0 DS:1 DS:2 TOTAL
6360 Sent 0 301 0 301
6361 Done 301 0 301 602
6362 Skipped 0 0 0 0
6363 Error 0 0 0 0
6364 Last Flush: 2805 2509 2805
6365 Downstairs last five completed: 2509 2508 2507 2506 2505
6366 Upstairs last five completed: 2810 2809 2808 2807 2806
6367 CLIENT: Up:0 ds:301 act:2
6368 ----------------------------------------------------------------
6369 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
6370 States: Active Offline Active
6371 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
6372 2510 Acked Read 9 Done Sent Done false
6373 2511 Acked Write 3 Done Sent Done false
6374 2512 Acked Read 8 Done Sent Done false
6375 2513 Acked Write 9 Done Sent Done false
6376 2514 Acked Write 9 Done Sent Done false
6377 2515 Acked Write 6 Done Sent Done false
6378 2516 Acked Write 1 Done Sent Done false
6379 2517 Acked Write 8 Done Sent Done false
6380 2518 Acked Write 10 Done Sent Done false
6381 2519 Acked Write 6 Done Sent Done false
6382 2520 Acked Write 8 Done Sent Done false
6383 2521 Acked Read 10 Done Sent Done false
6384 2522 Acked Write 7 Done Sent Done false
6385 2523 Acked Read 6 Done Sent Done false
6386 2524 Acked Read 5 Done Sent Done false
6387 2525 Acked Write 5 Done Sent Done false
6388 2526 Acked Read 3 Done Sent Done false
6389 2527 Acked Read 6 Done Sent Done false
6390 2528 Acked Write 1 Done Sent Done false
6391 2529 Acked Read 7 Done Sent Done false
6392 2530 Acked Write 5 Done Sent Done false
6393 2531 Acked Read 8 Done Sent Done false
6394 2532 Acked Read 7 Done Sent Done false
6395 2533 Acked Write 5 Done Sent Done false
6396 2534 Acked Read 3 Done Sent Done false
6397 2535 Acked Write 7 Done Sent Done false
6398 2536 Acked Write 8 Done Sent Done false
6399 2537 Acked Read 5 Done Sent Done false
6400 2538 Acked Write 8 Done Sent Done false
6401 2539 Acked Flush 0 Done Sent Done false
6402 2540 Acked Read 10 Done Sent Done false
6403 2541 Acked Write 2 Done Sent Done false
6404 2542 Acked Write 4 Done Sent Done false
6405 2543 Acked Read 9 Done Sent Done false
6406 2544 Acked Write 7 Done Sent Done false
6407 2545 Acked Write 8 Done Sent Done false
6408 2546 Acked Write 4 Done Sent Done false
6409 2547 Acked Read 8 Done Sent Done false
6410 2548 Acked Read 8 Done Sent Done false
6411 2549 Acked Read 7 Done Sent Done false
6412 2550 Acked Flush 0 Done Sent Done false
6413 2551 Acked Read 6 Done Sent Done false
6414 2552 Acked Read 2 Done Sent Done false
6415 2553 Acked Read 4 Done Sent Done false
6416 2554 Acked Read 4 Done Sent Done false
6417 2555 Acked Read 7 Done Sent Done false
6418 2556 Acked Read 10 Done Sent Done false
6419 2557 Acked Read 7 Done Sent Done false
6420 2558 Acked Read 5 Done Sent Done false
6421 2559 Acked Flush 0 Done Sent Done false
6422 2560 Acked Read 1 Done Sent Done false
6423 2561 Acked Read 2 Done Sent Done false
6424 2562 Acked Write 2 Done Sent Done false
6425 2563 Acked Write 5 Done Sent Done false
6426 2564 Acked Read 1 Done Sent Done false
6427 2565 Acked Read 8 Done Sent Done false
6428 2566 Acked Flush 0 Done Sent Done false
6429 2567 Acked Write 4 Done Sent Done false
6430 2568 Acked Write 8 Done Sent Done false
6431 2569 Acked Read 1 Done Sent Done false
6432 2570 Acked Write 7 Done Sent Done false
6433 2571 Acked Write 9 Done Sent Done false
6434 2572 Acked Write 10 Done Sent Done false
6435 2573 Acked Read 1 Done Sent Done false
6436 2574 Acked Write 5 Done Sent Done false
6437 2575 Acked Read 6 Done Sent Done false
6438 2576 Acked Write 1 Done Sent Done false
6439 2577 Acked Write 6 Done Sent Done false
6440 2578 Acked Write 4 Done Sent Done false
6441 2579 Acked Write 3 Done Sent Done false
6442 2580 Acked Flush 0 Done Sent Done false
6443 2581 Acked Read 6 Done Sent Done false
6444 2582 Acked Write 2 Done Sent Done false
6445 2583 Acked Write 10 Done Sent Done false
6446 2584 Acked Read 8 Done Sent Done false
6447 2585 Acked Read 8 Done Sent Done false
6448 2586 Acked Write 10 Done Sent Done false
6449 2587 Acked Write 4 Done Sent Done false
6450 2588 Acked Write 2 Done Sent Done false
6451 2589 Acked Read 1 Done Sent Done false
6452 2590 Acked Write 10 Done Sent Done false
6453 2591 Acked Flush 0 Done Sent Done false
6454 2592 Acked Read 6 Done Sent Done false
6455 2593 Acked Write 7 Done Sent Done false
6456 2594 Acked Flush 0 Done Sent Done false
6457 2595 Acked Read 6 Done Sent Done false
6458 2596 Acked Read 4 Done Sent Done false
6459 2597 Acked Write 7 Done Sent Done false
6460 2598 Acked Write 2 Done Sent Done false
6461 2599 Acked Read 9 Done Sent Done false
6462 2600 Acked Read 5 Done Sent Done false
6463 2601 Acked Read 3 Done Sent Done false
6464 2602 Acked Write 4 Done Sent Done false
6465 2603 Acked Read 6 Done Sent Done false
6466 2604 Acked Write 2 Done Sent Done false
6467 2605 Acked Read 8 Done Sent Done false
6468 2606 Acked Read 3 Done Sent Done false
6469 2607 Acked Write 3 Done Sent Done false
6470 2608 Acked Write 4 Done Sent Done false
6471 2609 Acked Read 9 Done Sent Done false
6472 2610 Acked Write 7 Done Sent Done false
6473 2611 Acked Read 1 Done Sent Done false
6474 2612 Acked Read 2 Done Sent Done false
6475 2613 Acked Read 2 Done Sent Done false
6476 2614 Acked Read 1 Done Sent Done false
6477 2615 Acked Write 3 Done Sent Done false
6478 2616 Acked Read 9 Done Sent Done false
6479 2617 Acked Read 3 Done Sent Done false
6480 2618 Acked Read 7 Done Sent Done false
6481 2619 Acked Read 7 Done Sent Done false
6482 2620 Acked Read 1 Done Sent Done false
6483 2621 Acked Write 9 Done Sent Done false
6484 2622 Acked Write 9 Done Sent Done false
6485 2623 Acked Flush 0 Done Sent Done false
6486 2624 Acked Read 2 Done Sent Done false
6487 2625 Acked Write 1 Done Sent Done false
6488 2626 Acked Write 2 Done Sent Done false
6489 2627 Acked Read 4 Done Sent Done false
6490 2628 Acked Write 10 Done Sent Done false
6491 2629 Acked Write 2 Done Sent Done false
6492 2630 Acked Read 1 Done Sent Done false
6493 2631 Acked Flush 0 Done Sent Done false
6494 2632 Acked Write 10 Done Sent Done false
6495 2633 Acked Read 7 Done Sent Done false
6496 2634 Acked Read 4 Done Sent Done false
6497 2635 Acked Read 1 Done Sent Done false
6498 2636 Acked Flush 0 Done Sent Done false
6499 2637 Acked Read 6 Done Sent Done false
6500 2638 Acked Write 3 Done Sent Done false
6501 2639 Acked Read 7 Done Sent Done false
6502 2640 Acked Write 1 Done Sent Done false
6503 2641 Acked Read 10 Done Sent Done false
6504 2642 Acked Read 6 Done Sent Done false
6505 2643 Acked Write 8 Done Sent Done false
6506 2644 Acked Read 6 Done Sent Done false
6507 2645 Acked Flush 0 Done Sent Done false
6508 2646 Acked Read 1 Done Sent Done false
6509 2647 Acked Flush 0 Done Sent Done false
6510 2648 Acked Write 10 Done Sent Done false
6511 2649 Acked Flush 0 Done Sent Done false
6512 2650 Acked Write 8 Done Sent Done false
6513 2651 Acked Write 1 Done Sent Done false
6514 2652 Acked Flush 0 Done Sent Done false
6515 2653 Acked Write 6 Done Sent Done false
6516 2654 Acked Read 8 Done Sent Done false
6517 2655 Acked Write 8 Done Sent Done false
6518 2656 Acked Write 7 Done Sent Done false
6519 2657 Acked Read 6 Done Sent Done false
6520 2658 Acked Write 4 Done Sent Done false
6521 2659 Acked Write 5 Done Sent Done false
6522 2660 Acked Read 6 Done Sent Done false
6523 2661 Acked Read 4 Done Sent Done false
6524 2662 Acked Read 1 Done Sent Done false
6525 2663 Acked Read 7 Done Sent Done false
6526 2664 Acked Write 9 Done Sent Done false
6527 2665 Acked Read 1 Done Sent Done false
6528 2666 Acked Write 10 Done Sent Done false
6529 2667 Acked Read 6 Done Sent Done false
6530 2668 Acked Read 3 Done Sent Done false
6531 2669 Acked Write 8 Done Sent Done false
6532 2670 Acked Flush 0 Done Sent Done false
6533 2671 Acked Write 1 Done Sent Done false
6534 2672 Acked Flush 0 Done Sent Done false
6535 2673 Acked Read 4 Done Sent Done false
6536 2674 Acked Read 8 Done Sent Done false
6537 2675 Acked Read 2 Done Sent Done false
6538 2676 Acked Read 10 Done Sent Done false
6539 2677 Acked Write 2 Done Sent Done false
6540 2678 Acked Read 9 Done Sent Done false
6541 2679 Acked Read 8 Done Sent Done false
6542 2680 Acked Write 5 Done Sent Done false
6543 2681 Acked Flush 0 Done Sent Done false
6544 2682 Acked Read 1 Done Sent Done false
6545 2683 Acked Write 2 Done Sent Done false
6546 2684 Acked Write 2 Done Sent Done false
6547 2685 Acked Read 8 Done Sent Done false
6548 2686 Acked Read 3 Done Sent Done false
6549 2687 Acked Flush 0 Done Sent Done false
6550 2688 Acked Write 6 Done Sent Done false
6551 2689 Acked Write 9 Done Sent Done false
6552 2690 Acked Read 7 Done Sent Done false
6553 2691 Acked Read 4 Done Sent Done false
6554 2692 Acked Write 7 Done Sent Done false
6555 2693 Acked Flush 0 Done Sent Done false
6556 2694 Acked Read 5 Done Sent Done false
6557 2695 Acked Read 4 Done Sent Done false
6558 2696 Acked Write 2 Done Sent Done false
6559 2697 Acked Read 6 Done Sent Done false
6560 2698 Acked Flush 0 Done Sent Done false
6561 2699 Acked Write 9 Done Sent Done false
6562 2700 Acked Read 3 Done Sent Done false
6563 2701 Acked Read 6 Done Sent Done false
6564 2702 Acked Read 2 Done Sent Done false
6565 2703 Acked Write 2 Done Sent Done false
6566 2704 Acked Write 9 Done Sent Done false
6567 2705 Acked Write 2 Done Sent Done false
6568 2706 Acked Write 2 Done Sent Done false
6569 2707 Acked Read 6 Done Sent Done false
6570 2708 Acked Write 2 Done Sent Done false
6571 2709 Acked Write 3 Done Sent Done false
6572 2710 Acked Read 1 Done Sent Done false
6573 2711 Acked Write 1 Done Sent Done false
6574 2712 Acked Write 1 Done Sent Done false
6575 2713 Acked Write 4 Done Sent Done false
6576 2714 Acked Read 6 Done Sent Done false
6577 2715 Acked Write 8 Done Sent Done false
6578 2716 Acked Read 7 Done Sent Done false
6579 2717 Acked Flush 0 Done Sent Done false
6580 2718 Acked Write 2 Done Sent Done false
6581 2719 Acked Write 4 Done Sent Done false
6582 2720 Acked Write 1 Done Sent Done false
6583 2721 Acked Read 5 Done Sent Done false
6584 2722 Acked Read 5 Done Sent Done false
6585 2723 Acked Read 2 Done Sent Done false
6586 2724 Acked Read 5 Done Sent Done false
6587 2725 Acked Read 10 Done Sent Done false
6588 2726 Acked Read 2 Done Sent Done false
6589 2727 Acked Write 7 Done Sent Done false
6590 2728 Acked Read 3 Done Sent Done false
6591 2729 Acked Read 7 Done Sent Done false
6592 2730 Acked Read 4 Done Sent Done false
6593 2731 Acked Write 1 Done Sent Done false
6594 2732 Acked Flush 0 Done Sent Done false
6595 2733 Acked Read 4 Done Sent Done false
6596 2734 Acked Write 3 Done Sent Done false
6597 2735 Acked Write 6 Done Sent Done false
6598 2736 Acked Write 8 Done Sent Done false
6599 2737 Acked Flush 0 Done Sent Done false
6600 2738 Acked Read 5 Done Sent Done false
6601 2739 Acked Flush 0 Done Sent Done false
6602 2740 Acked Write 5 Done Sent Done false
6603 2741 Acked Read 2 Done Sent Done false
6604 2742 Acked Flush 0 Done Sent Done false
6605 2743 Acked Flush 0 Done Sent Done false
6606 2744 Acked Read 9 Done Sent Done false
6607 2745 Acked Read 7 Done Sent Done false
6608 2746 Acked Read 3 Done Sent Done false
6609 2747 Acked Write 8 Done Sent Done false
6610 2748 Acked Read 8 Done Sent Done false
6611 2749 Acked Read 8 Done Sent Done false
6612 2750 Acked Read 2 Done Sent Done false
6613 2751 Acked Flush 0 Done Sent Done false
6614 2752 Acked Write 2 Done Sent Done false
6615 2753 Acked Read 4 Done Sent Done false
6616 2754 Acked Read 8 Done Sent Done false
6617 2755 Acked Read 2 Done Sent Done false
6618 2756 Acked Read 1 Done Sent Done false
6619 2757 Acked Read 3 Done Sent Done false
6620 2758 Acked Write 8 Done Sent Done false
6621 2759 Acked Write 8 Done Sent Done false
6622 2760 Acked Read 9 Done Sent Done false
6623 2761 Acked Read 4 Done Sent Done false
6624 2762 Acked Read 2 Done Sent Done false
6625 2763 Acked Read 7 Done Sent Done false
6626 2764 Acked Flush 0 Done Sent Done false
6627 2765 Acked Read 1 Done Sent Done false
6628 2766 Acked Write 8 Done Sent Done false
6629 2767 Acked Read 4 Done Sent Done false
6630 2768 Acked Read 3 Done Sent Done false
6631 2769 Acked Write 8 Done Sent Done false
6632 2770 Acked Read 6 Done Sent Done false
6633 2771 Acked Read 6 Done Sent Done false
6634 2772 Acked Write 10 Done Sent Done false
6635 2773 Acked Write 1 Done Sent Done false
6636 2774 Acked Write 1 Done Sent Done false
6637 2775 Acked Write 2 Done Sent Done false
6638 2776 Acked Read 6 Done Sent Done false
6639 2777 Acked Read 3 Done Sent Done false
6640 2778 Acked Read 9 Done Sent Done false
6641 2779 Acked Read 8 Done Sent Done false
6642 2780 Acked Read 10 Done Sent Done false
6643 2781 Acked Read 6 Done Sent Done false
6644 2782 Acked Read 7 Done Sent Done false
6645 2783 Acked Read 3 Done Sent Done false
6646 2784 Acked Read 8 Done Sent Done false
6647 2785 Acked Write 9 Done Sent Done false
6648 2786 Acked Read 3 Done Sent Done false
6649 2787 Acked Read 7 Done Sent Done false
6650 2788 Acked Write 5 Done Sent Done false
6651 2789 Acked Read 3 Done Sent Done false
6652 2790 Acked Flush 0 Done Sent Done false
6653 2791 Acked Read 3 Done Sent Done false
6654 2792 Acked Read 2 Done Sent Done false
6655 2793 Acked Read 2 Done Sent Done false
6656 2794 Acked Write 3 Done Sent Done false
6657 2795 Acked Read 8 Done Sent Done false
6658 2796 Acked Write 8 Done Sent Done false
6659 2797 Acked Read 9 Done Sent Done false
6660 2798 Acked Read 2 Done Sent Done false
6661 2799 Acked Write 5 Done Sent Done false
6662 2800 Acked Write 9 Done Sent Done false
6663 2801 Acked Write 4 Done Sent Done false
6664 2802 Acked Read 6 Done Sent Done false
6665 2803 Acked Read 10 Done Sent Done false
6666 2804 Acked Write 8 Done Sent Done false
6667 2805 Acked Flush 0 Done Sent Done false
6668 2806 Acked Read 3 Done Sent Done false
6669 2807 Acked Write 8 Done Sent Done false
6670 2808 Acked Read 9 Done Sent Done false
6671 2809 Acked Write 8 Done Sent Done false
6672 2810 Acked Read 4 Done Sent Done false
6673 2811 Acked Flush 0 Done Sent Done false
6674 STATES DS:0 DS:1 DS:2 TOTAL
6675 Sent 0 302 0 302
6676 Done 302 0 302 604
6677 Skipped 0 0 0 0
6678 Error 0 0 0 0
6679 Last Flush: 2811 2509 2811
6680 Downstairs last five completed: 2509 2508 2507 2506 2505
6681 Upstairs last five completed: 2811 2810 2809 2808 2807
6682 CLIENT: Up:0 ds:302 act:2
66832024-11-26T22:05:55.624ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
66842024-11-26T22:05:55.624ZWARNcrucible: ds_connection connect to 127.0.0.1:8820 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
66852024-11-26T22:05:55.624ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
66862024-11-26T22:05:55.624ZWARNcrucible: downstairs task for 1 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
66872024-11-26T22:05:55.624ZINFOcrucible: client task is exiting = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
6688 ----------------------------------------------------------------
6689 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
6690 States: Active Offline Active
6691 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
6692 2510 Acked Read 9 Done Sent Done false
6693 2511 Acked Write 3 Done Sent Done false
6694 2512 Acked Read 8 Done Sent Done false
6695 2513 Acked Write 9 Done Sent Done false
6696 2514 Acked Write 9 Done Sent Done false
6697 2515 Acked Write 6 Done Sent Done false
6698 2516 Acked Write 1 Done Sent Done false
6699 2517 Acked Write 8 Done Sent Done false
6700 2518 Acked Write 10 Done Sent Done false
6701 2519 Acked Write 6 Done Sent Done false
6702 2520 Acked Write 8 Done Sent Done false
6703 2521 Acked Read 10 Done Sent Done false
6704 2522 Acked Write 7 Done Sent Done false
6705 2523 Acked Read 6 Done Sent Done false
6706 2524 Acked Read 5 Done Sent Done false
6707 2525 Acked Write 5 Done Sent Done false
6708 2526 Acked Read 3 Done Sent Done false
6709 2527 Acked Read 6 Done Sent Done false
6710 2528 Acked Write 1 Done Sent Done false
6711 2529 Acked Read 7 Done Sent Done false
6712 2530 Acked Write 5 Done Sent Done false
6713 2531 Acked Read 8 Done Sent Done false
6714 2532 Acked Read 7 Done Sent Done false
6715 2533 Acked Write 5 Done Sent Done false
6716 2534 Acked Read 3 Done Sent Done false
6717 2535 Acked Write 7 Done Sent Done false
6718 2536 Acked Write 8 Done Sent Done false
6719 2537 Acked Read 5 Done Sent Done false
6720 2538 Acked Write 8 Done Sent Done false
6721 2539 Acked Flush 0 Done Sent Done false
6722 2540 Acked Read 10 Done Sent Done false
6723 2541 Acked Write 2 Done Sent Done false
6724 2542 Acked Write 4 Done Sent Done false
6725 2543 Acked Read 9 Done Sent Done false
6726 2544 Acked Write 7 Done Sent Done false
6727 2545 Acked Write 8 Done Sent Done false
6728 2546 Acked Write 4 Done Sent Done false
6729 2547 Acked Read 8 Done Sent Done false
6730 2548 Acked Read 8 Done Sent Done false
6731 2549 Acked Read 7 Done Sent Done false
6732 2550 Acked Flush 0 Done Sent Done false
6733 2551 Acked Read 6 Done Sent Done false
6734 2552 Acked Read 2 Done Sent Done false
6735 2553 Acked Read 4 Done Sent Done false
6736 2554 Acked Read 4 Done Sent Done false
6737 2555 Acked Read 7 Done Sent Done false
6738 2556 Acked Read 10 Done Sent Done false
6739 2557 Acked Read 7 Done Sent Done false
6740 2558 Acked Read 5 Done Sent Done false
6741 2559 Acked Flush 0 Done Sent Done false
6742 2560 Acked Read 1 Done Sent Done false
6743 2561 Acked Read 2 Done Sent Done false
6744 2562 Acked Write 2 Done Sent Done false
6745 2563 Acked Write 5 Done Sent Done false
6746 2564 Acked Read 1 Done Sent Done false
6747 2565 Acked Read 8 Done Sent Done false
6748 2566 Acked Flush 0 Done Sent Done false
6749 2567 Acked Write 4 Done Sent Done false
6750 2568 Acked Write 8 Done Sent Done false
6751 2569 Acked Read 1 Done Sent Done false
6752 2570 Acked Write 7 Done Sent Done false
6753 2571 Acked Write 9 Done Sent Done false
6754 2572 Acked Write 10 Done Sent Done false
6755 2573 Acked Read 1 Done Sent Done false
6756 2574 Acked Write 5 Done Sent Done false
6757 2575 Acked Read 6 Done Sent Done false
6758 2576 Acked Write 1 Done Sent Done false
6759 2577 Acked Write 6 Done Sent Done false
6760 2578 Acked Write 4 Done Sent Done false
6761 2579 Acked Write 3 Done Sent Done false
6762 2580 Acked Flush 0 Done Sent Done false
6763 2581 Acked Read 6 Done Sent Done false
6764 2582 Acked Write 2 Done Sent Done false
6765 2583 Acked Write 10 Done Sent Done false
6766 2584 Acked Read 8 Done Sent Done false
6767 2585 Acked Read 8 Done Sent Done false
6768 2586 Acked Write 10 Done Sent Done false
6769 2587 Acked Write 4 Done Sent Done false
6770 2588 Acked Write 2 Done Sent Done false
6771 2589 Acked Read 1 Done Sent Done false
6772 2590 Acked Write 10 Done Sent Done false
6773 2591 Acked Flush 0 Done Sent Done false
6774 2592 Acked Read 6 Done Sent Done false
6775 2593 Acked Write 7 Done Sent Done false
6776 2594 Acked Flush 0 Done Sent Done false
6777 2595 Acked Read 6 Done Sent Done false
6778 2596 Acked Read 4 Done Sent Done false
6779 2597 Acked Write 7 Done Sent Done false
6780 2598 Acked Write 2 Done Sent Done false
6781 2599 Acked Read 9 Done Sent Done false
6782 2600 Acked Read 5 Done Sent Done false
6783 2601 Acked Read 3 Done Sent Done false
6784 2602 Acked Write 4 Done Sent Done false
6785 2603 Acked Read 6 Done Sent Done false
6786 2604 Acked Write 2 Done Sent Done false
6787 2605 Acked Read 8 Done Sent Done false
6788 2606 Acked Read 3 Done Sent Done false
6789 2607 Acked Write 3 Done Sent Done false
6790 2608 Acked Write 4 Done Sent Done false
6791 2609 Acked Read 9 Done Sent Done false
6792 2610 Acked Write 7 Done Sent Done false
6793 2611 Acked Read 1 Done Sent Done false
6794 2612 Acked Read 2 Done Sent Done false
6795 2613 Acked Read 2 Done Sent Done false
6796 2614 Acked Read 1 Done Sent Done false
6797 2615 Acked Write 3 Done Sent Done false
6798 2616 Acked Read 9 Done Sent Done false
6799 2617 Acked Read 3 Done Sent Done false
6800 2618 Acked Read 7 Done Sent Done false
6801 2619 Acked Read 7 Done Sent Done false
6802 2620 Acked Read 1 Done Sent Done false
6803 2621 Acked Write 9 Done Sent Done false
6804 2622 Acked Write 9 Done Sent Done false
6805 2623 Acked Flush 0 Done Sent Done false
6806 2624 Acked Read 2 Done Sent Done false
6807 2625 Acked Write 1 Done Sent Done false
6808 2626 Acked Write 2 Done Sent Done false
6809 2627 Acked Read 4 Done Sent Done false
6810 2628 Acked Write 10 Done Sent Done false
6811 2629 Acked Write 2 Done Sent Done false
6812 2630 Acked Read 1 Done Sent Done false
6813 2631 Acked Flush 0 Done Sent Done false
6814 2632 Acked Write 10 Done Sent Done false
6815 2633 Acked Read 7 Done Sent Done false
6816 2634 Acked Read 4 Done Sent Done false
6817 2635 Acked Read 1 Done Sent Done false
6818 2636 Acked Flush 0 Done Sent Done false
6819 2637 Acked Read 6 Done Sent Done false
6820 2638 Acked Write 3 Done Sent Done false
6821 2639 Acked Read 7 Done Sent Done false
6822 2640 Acked Write 1 Done Sent Done false
6823 2641 Acked Read 10 Done Sent Done false
6824 2642 Acked Read 6 Done Sent Done false
6825 2643 Acked Write 8 Done Sent Done false
6826 2644 Acked Read 6 Done Sent Done false
6827 2645 Acked Flush 0 Done Sent Done false
6828 2646 Acked Read 1 Done Sent Done false
6829 2647 Acked Flush 0 Done Sent Done false
6830 2648 Acked Write 10 Done Sent Done false
6831 2649 Acked Flush 0 Done Sent Done false
6832 2650 Acked Write 8 Done Sent Done false
6833 2651 Acked Write 1 Done Sent Done false
6834 2652 Acked Flush 0 Done Sent Done false
6835 2653 Acked Write 6 Done Sent Done false
6836 2654 Acked Read 8 Done Sent Done false
6837 2655 Acked Write 8 Done Sent Done false
6838 2656 Acked Write 7 Done Sent Done false
6839 2657 Acked Read 6 Done Sent Done false
6840 2658 Acked Write 4 Done Sent Done false
6841 2659 Acked Write 5 Done Sent Done false
6842 2660 Acked Read 6 Done Sent Done false
6843 2661 Acked Read 4 Done Sent Done false
6844 2662 Acked Read 1 Done Sent Done false
6845 2663 Acked Read 7 Done Sent Done false
6846 2664 Acked Write 9 Done Sent Done false
6847 2665 Acked Read 1 Done Sent Done false
6848 2666 Acked Write 10 Done Sent Done false
6849 2667 Acked Read 6 Done Sent Done false
6850 2668 Acked Read 3 Done Sent Done false
6851 2669 Acked Write 8 Done Sent Done false
6852 2670 Acked Flush 0 Done Sent Done false
6853 2671 Acked Write 1 Done Sent Done false
6854 2672 Acked Flush 0 Done Sent Done false
6855 2673 Acked Read 4 Done Sent Done false
6856 2674 Acked Read 8 Done Sent Done false
6857 2675 Acked Read 2 Done Sent Done false
6858 2676 Acked Read 10 Done Sent Done false
6859 2677 Acked Write 2 Done Sent Done false
6860 2678 Acked Read 9 Done Sent Done false
6861 2679 Acked Read 8 Done Sent Done false
6862 2680 Acked Write 5 Done Sent Done false
6863 2681 Acked Flush 0 Done Sent Done false
6864 2682 Acked Read 1 Done Sent Done false
6865 2683 Acked Write 2 Done Sent Done false
6866 2684 Acked Write 2 Done Sent Done false
6867 2685 Acked Read 8 Done Sent Done false
6868 2686 Acked Read 3 Done Sent Done false
6869 2687 Acked Flush 0 Done Sent Done false
6870 2688 Acked Write 6 Done Sent Done false
6871 2689 Acked Write 9 Done Sent Done false
6872 2690 Acked Read 7 Done Sent Done false
6873 2691 Acked Read 4 Done Sent Done false
6874 2692 Acked Write 7 Done Sent Done false
6875 2693 Acked Flush 0 Done Sent Done false
6876 2694 Acked Read 5 Done Sent Done false
6877 2695 Acked Read 4 Done Sent Done false
6878 2696 Acked Write 2 Done Sent Done false
6879 2697 Acked Read 6 Done Sent Done false
6880 2698 Acked Flush 0 Done Sent Done false
6881 2699 Acked Write 9 Done Sent Done false
6882 2700 Acked Read 3 Done Sent Done false
6883 2701 Acked Read 6 Done Sent Done false
6884 2702 Acked Read 2 Done Sent Done false
6885 2703 Acked Write 2 Done Sent Done false
6886 2704 Acked Write 9 Done Sent Done false
6887 2705 Acked Write 2 Done Sent Done false
6888 2706 Acked Write 2 Done Sent Done false
6889 2707 Acked Read 6 Done Sent Done false
6890 2708 Acked Write 2 Done Sent Done false
6891 2709 Acked Write 3 Done Sent Done false
6892 2710 Acked Read 1 Done Sent Done false
6893 2711 Acked Write 1 Done Sent Done false
6894 2712 Acked Write 1 Done Sent Done false
6895 2713 Acked Write 4 Done Sent Done false
6896 2714 Acked Read 6 Done Sent Done false
6897 2715 Acked Write 8 Done Sent Done false
6898 2716 Acked Read 7 Done Sent Done false
6899 2717 Acked Flush 0 Done Sent Done false
6900 2718 Acked Write 2 Done Sent Done false
6901 2719 Acked Write 4 Done Sent Done false
6902 2720 Acked Write 1 Done Sent Done false
6903 2721 Acked Read 5 Done Sent Done false
6904 2722 Acked Read 5 Done Sent Done false
6905 2723 Acked Read 2 Done Sent Done false
6906 2724 Acked Read 5 Done Sent Done false
6907 2725 Acked Read 10 Done Sent Done false
6908 2726 Acked Read 2 Done Sent Done false
6909 2727 Acked Write 7 Done Sent Done false
6910 2728 Acked Read 3 Done Sent Done false
6911 2729 Acked Read 7 Done Sent Done false
6912 2730 Acked Read 4 Done Sent Done false
6913 2731 Acked Write 1 Done Sent Done false
6914 2732 Acked Flush 0 Done Sent Done false
6915 2733 Acked Read 4 Done Sent Done false
6916 2734 Acked Write 3 Done Sent Done false
6917 2735 Acked Write 6 Done Sent Done false
6918 2736 Acked Write 8 Done Sent Done false
6919 2737 Acked Flush 0 Done Sent Done false
6920 2738 Acked Read 5 Done Sent Done false
6921 2739 Acked Flush 0 Done Sent Done false
6922 2740 Acked Write 5 Done Sent Done false
6923 2741 Acked Read 2 Done Sent Done false
6924 2742 Acked Flush 0 Done Sent Done false
6925 2743 Acked Flush 0 Done Sent Done false
6926 2744 Acked Read 9 Done Sent Done false
6927 2745 Acked Read 7 Done Sent Done false
6928 2746 Acked Read 3 Done Sent Done false
6929 2747 Acked Write 8 Done Sent Done false
6930 2748 Acked Read 8 Done Sent Done false
6931 2749 Acked Read 8 Done Sent Done false
6932 2750 Acked Read 2 Done Sent Done false
6933 2751 Acked Flush 0 Done Sent Done false
6934 2752 Acked Write 2 Done Sent Done false
6935 2753 Acked Read 4 Done Sent Done false
6936 2754 Acked Read 8 Done Sent Done false
6937 2755 Acked Read 2 Done Sent Done false
6938 2756 Acked Read 1 Done Sent Done false
6939 2757 Acked Read 3 Done Sent Done false
6940 2758 Acked Write 8 Done Sent Done false
6941 2759 Acked Write 8 Done Sent Done false
6942 2760 Acked Read 9 Done Sent Done false
6943 2761 Acked Read 4 Done Sent Done false
6944 2762 Acked Read 2 Done Sent Done false
6945 2763 Acked Read 7 Done Sent Done false
6946 2764 Acked Flush 0 Done Sent Done false
6947 2765 Acked Read 1 Done Sent Done false
6948 2766 Acked Write 8 Done Sent Done false
6949 2767 Acked Read 4 Done Sent Done false
6950 2768 Acked Read 3 Done Sent Done false
6951 2769 Acked Write 8 Done Sent Done false
6952 2770 Acked Read 6 Done Sent Done false
6953 2771 Acked Read 6 Done Sent Done false
6954 2772 Acked Write 10 Done Sent Done false
6955 2773 Acked Write 1 Done Sent Done false
6956 2774 Acked Write 1 Done Sent Done false
6957 2775 Acked Write 2 Done Sent Done false
6958 2776 Acked Read 6 Done Sent Done false
6959 2777 Acked Read 3 Done Sent Done false
6960 2778 Acked Read 9 Done Sent Done false
6961 2779 Acked Read 8 Done Sent Done false
6962 2780 Acked Read 10 Done Sent Done false
6963 2781 Acked Read 6 Done Sent Done false
6964 2782 Acked Read 7 Done Sent Done false
6965 2783 Acked Read 3 Done Sent Done false
6966 2784 Acked Read 8 Done Sent Done false
6967 2785 Acked Write 9 Done Sent Done false
6968 2786 Acked Read 3 Done Sent Done false
6969 2787 Acked Read 7 Done Sent Done false
6970 2788 Acked Write 5 Done Sent Done false
6971 2789 Acked Read 3 Done Sent Done false
6972 2790 Acked Flush 0 Done Sent Done false
6973 2791 Acked Read 3 Done Sent Done false
6974 2792 Acked Read 2 Done Sent Done false
6975 2793 Acked Read 2 Done Sent Done false
6976 2794 Acked Write 3 Done Sent Done false
6977 2795 Acked Read 8 Done Sent Done false
6978 2796 Acked Write 8 Done Sent Done false
6979 2797 Acked Read 9 Done Sent Done false
6980 2798 Acked Read 2 Done Sent Done false
6981 2799 Acked Write 5 Done Sent Done false
6982 2800 Acked Write 9 Done Sent Done false
6983 2801 Acked Write 4 Done Sent Done false
6984 2802 Acked Read 6 Done Sent Done false
6985 2803 Acked Read 10 Done Sent Done false
6986 2804 Acked Write 8 Done Sent Done false
6987 2805 Acked Flush 0 Done Sent Done false
6988 2806 Acked Read 3 Done Sent Done false
6989 2807 Acked Write 8 Done Sent Done false
6990 2808 Acked Read 9 Done Sent Done false
6991 2809 Acked Write 8 Done Sent Done false
6992 2810 Acked Read 4 Done Sent Done false
6993 2811 Acked Flush 0 Done Sent Done false
6994 STATES DS:0 DS:1 DS:2 TOTAL
6995 Sent 0 302 0 302
6996 Done 302 0 302 604
6997 Skipped 0 0 0 0
6998 Error 0 0 0 0
6999 Last Flush: 2811 2509 2811
7000 Downstairs last five completed: 2509 2508 2507 2506 2505
7001 Upstairs last five completed: 2811 2810 2809 2808 2807
7002 CLIENT: Up:0 ds:302 act:2
7003 [8820] got start action
7004 [8820] I am going to restart
7005 Starting downstairs at port 8820
7006 Make output file at "/tmp/dsc/downstairs-8820.txt"
7007 [8820][1] reports Starting
7008 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1209)
7009 [8820][1] reports Running
7010 ----------------------------------------------------------------
7011 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
7012 States: Active Offline Active
7013 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
7014 2510 Acked Read 9 Done Sent Done false
7015 2511 Acked Write 3 Done Sent Done false
7016 2512 Acked Read 8 Done Sent Done false
7017 2513 Acked Write 9 Done Sent Done false
7018 2514 Acked Write 9 Done Sent Done false
7019 2515 Acked Write 6 Done Sent Done false
7020 2516 Acked Write 1 Done Sent Done false
7021 2517 Acked Write 8 Done Sent Done false
7022 2518 Acked Write 10 Done Sent Done false
7023 2519 Acked Write 6 Done Sent Done false
7024 2520 Acked Write 8 Done Sent Done false
7025 2521 Acked Read 10 Done Sent Done false
7026 2522 Acked Write 7 Done Sent Done false
7027 2523 Acked Read 6 Done Sent Done false
7028 2524 Acked Read 5 Done Sent Done false
7029 2525 Acked Write 5 Done Sent Done false
7030 2526 Acked Read 3 Done Sent Done false
7031 2527 Acked Read 6 Done Sent Done false
7032 2528 Acked Write 1 Done Sent Done false
7033 2529 Acked Read 7 Done Sent Done false
7034 2530 Acked Write 5 Done Sent Done false
7035 2531 Acked Read 8 Done Sent Done false
7036 2532 Acked Read 7 Done Sent Done false
7037 2533 Acked Write 5 Done Sent Done false
7038 2534 Acked Read 3 Done Sent Done false
7039 2535 Acked Write 7 Done Sent Done false
7040 2536 Acked Write 8 Done Sent Done false
7041 2537 Acked Read 5 Done Sent Done false
7042 2538 Acked Write 8 Done Sent Done false
7043 2539 Acked Flush 0 Done Sent Done false
7044 2540 Acked Read 10 Done Sent Done false
7045 2541 Acked Write 2 Done Sent Done false
7046 2542 Acked Write 4 Done Sent Done false
7047 2543 Acked Read 9 Done Sent Done false
7048 2544 Acked Write 7 Done Sent Done false
7049 2545 Acked Write 8 Done Sent Done false
7050 2546 Acked Write 4 Done Sent Done false
7051 2547 Acked Read 8 Done Sent Done false
7052 2548 Acked Read 8 Done Sent Done false
7053 2549 Acked Read 7 Done Sent Done false
7054 2550 Acked Flush 0 Done Sent Done false
7055 2551 Acked Read 6 Done Sent Done false
7056 2552 Acked Read 2 Done Sent Done false
7057 2553 Acked Read 4 Done Sent Done false
7058 2554 Acked Read 4 Done Sent Done false
7059 2555 Acked Read 7 Done Sent Done false
7060 2556 Acked Read 10 Done Sent Done false
7061 2557 Acked Read 7 Done Sent Done false
7062 2558 Acked Read 5 Done Sent Done false
7063 2559 Acked Flush 0 Done Sent Done false
7064 2560 Acked Read 1 Done Sent Done false
7065 2561 Acked Read 2 Done Sent Done false
7066 2562 Acked Write 2 Done Sent Done false
7067 2563 Acked Write 5 Done Sent Done false
7068 2564 Acked Read 1 Done Sent Done false
7069 2565 Acked Read 8 Done Sent Done false
7070 2566 Acked Flush 0 Done Sent Done false
7071 2567 Acked Write 4 Done Sent Done false
7072 2568 Acked Write 8 Done Sent Done false
7073 2569 Acked Read 1 Done Sent Done false
7074 2570 Acked Write 7 Done Sent Done false
7075 2571 Acked Write 9 Done Sent Done false
7076 2572 Acked Write 10 Done Sent Done false
7077 2573 Acked Read 1 Done Sent Done false
7078 2574 Acked Write 5 Done Sent Done false
7079 2575 Acked Read 6 Done Sent Done false
7080 2576 Acked Write 1 Done Sent Done false
7081 2577 Acked Write 6 Done Sent Done false
7082 2578 Acked Write 4 Done Sent Done false
7083 2579 Acked Write 3 Done Sent Done false
7084 2580 Acked Flush 0 Done Sent Done false
7085 2581 Acked Read 6 Done Sent Done false
7086 2582 Acked Write 2 Done Sent Done false
7087 2583 Acked Write 10 Done Sent Done false
7088 2584 Acked Read 8 Done Sent Done false
7089 2585 Acked Read 8 Done Sent Done false
7090 2586 Acked Write 10 Done Sent Done false
7091 2587 Acked Write 4 Done Sent Done false
7092 2588 Acked Write 2 Done Sent Done false
7093 2589 Acked Read 1 Done Sent Done false
7094 2590 Acked Write 10 Done Sent Done false
7095 2591 Acked Flush 0 Done Sent Done false
7096 2592 Acked Read 6 Done Sent Done false
7097 2593 Acked Write 7 Done Sent Done false
7098 2594 Acked Flush 0 Done Sent Done false
7099 2595 Acked Read 6 Done Sent Done false
7100 2596 Acked Read 4 Done Sent Done false
7101 2597 Acked Write 7 Done Sent Done false
7102 2598 Acked Write 2 Done Sent Done false
7103 2599 Acked Read 9 Done Sent Done false
7104 2600 Acked Read 5 Done Sent Done false
7105 2601 Acked Read 3 Done Sent Done false
7106 2602 Acked Write 4 Done Sent Done false
7107 2603 Acked Read 6 Done Sent Done false
7108 2604 Acked Write 2 Done Sent Done false
7109 2605 Acked Read 8 Done Sent Done false
7110 2606 Acked Read 3 Done Sent Done false
7111 2607 Acked Write 3 Done Sent Done false
7112 2608 Acked Write 4 Done Sent Done false
7113 2609 Acked Read 9 Done Sent Done false
7114 2610 Acked Write 7 Done Sent Done false
7115 2611 Acked Read 1 Done Sent Done false
7116 2612 Acked Read 2 Done Sent Done false
7117 2613 Acked Read 2 Done Sent Done false
7118 2614 Acked Read 1 Done Sent Done false
7119 2615 Acked Write 3 Done Sent Done false
7120 2616 Acked Read 9 Done Sent Done false
7121 2617 Acked Read 3 Done Sent Done false
7122 2618 Acked Read 7 Done Sent Done false
7123 2619 Acked Read 7 Done Sent Done false
7124 2620 Acked Read 1 Done Sent Done false
7125 2621 Acked Write 9 Done Sent Done false
7126 2622 Acked Write 9 Done Sent Done false
7127 2623 Acked Flush 0 Done Sent Done false
7128 2624 Acked Read 2 Done Sent Done false
7129 2625 Acked Write 1 Done Sent Done false
7130 2626 Acked Write 2 Done Sent Done false
7131 2627 Acked Read 4 Done Sent Done false
7132 2628 Acked Write 10 Done Sent Done false
7133 2629 Acked Write 2 Done Sent Done false
7134 2630 Acked Read 1 Done Sent Done false
7135 2631 Acked Flush 0 Done Sent Done false
7136 2632 Acked Write 10 Done Sent Done false
7137 2633 Acked Read 7 Done Sent Done false
7138 2634 Acked Read 4 Done Sent Done false
7139 2635 Acked Read 1 Done Sent Done false
7140 2636 Acked Flush 0 Done Sent Done false
7141 2637 Acked Read 6 Done Sent Done false
7142 2638 Acked Write 3 Done Sent Done false
7143 2639 Acked Read 7 Done Sent Done false
7144 2640 Acked Write 1 Done Sent Done false
7145 2641 Acked Read 10 Done Sent Done false
7146 2642 Acked Read 6 Done Sent Done false
7147 2643 Acked Write 8 Done Sent Done false
7148 2644 Acked Read 6 Done Sent Done false
7149 2645 Acked Flush 0 Done Sent Done false
7150 2646 Acked Read 1 Done Sent Done false
7151 2647 Acked Flush 0 Done Sent Done false
7152 2648 Acked Write 10 Done Sent Done false
7153 2649 Acked Flush 0 Done Sent Done false
7154 2650 Acked Write 8 Done Sent Done false
7155 2651 Acked Write 1 Done Sent Done false
7156 2652 Acked Flush 0 Done Sent Done false
7157 2653 Acked Write 6 Done Sent Done false
7158 2654 Acked Read 8 Done Sent Done false
7159 2655 Acked Write 8 Done Sent Done false
7160 2656 Acked Write 7 Done Sent Done false
7161 2657 Acked Read 6 Done Sent Done false
7162 2658 Acked Write 4 Done Sent Done false
7163 2659 Acked Write 5 Done Sent Done false
7164 2660 Acked Read 6 Done Sent Done false
7165 2661 Acked Read 4 Done Sent Done false
7166 2662 Acked Read 1 Done Sent Done false
7167 2663 Acked Read 7 Done Sent Done false
7168 2664 Acked Write 9 Done Sent Done false
7169 2665 Acked Read 1 Done Sent Done false
7170 2666 Acked Write 10 Done Sent Done false
7171 2667 Acked Read 6 Done Sent Done false
7172 2668 Acked Read 3 Done Sent Done false
7173 2669 Acked Write 8 Done Sent Done false
7174 2670 Acked Flush 0 Done Sent Done false
7175 2671 Acked Write 1 Done Sent Done false
7176 2672 Acked Flush 0 Done Sent Done false
7177 2673 Acked Read 4 Done Sent Done false
7178 2674 Acked Read 8 Done Sent Done false
7179 2675 Acked Read 2 Done Sent Done false
7180 2676 Acked Read 10 Done Sent Done false
7181 2677 Acked Write 2 Done Sent Done false
7182 2678 Acked Read 9 Done Sent Done false
7183 2679 Acked Read 8 Done Sent Done false
7184 2680 Acked Write 5 Done Sent Done false
7185 2681 Acked Flush 0 Done Sent Done false
7186 2682 Acked Read 1 Done Sent Done false
7187 2683 Acked Write 2 Done Sent Done false
7188 2684 Acked Write 2 Done Sent Done false
7189 2685 Acked Read 8 Done Sent Done false
7190 2686 Acked Read 3 Done Sent Done false
7191 2687 Acked Flush 0 Done Sent Done false
7192 2688 Acked Write 6 Done Sent Done false
7193 2689 Acked Write 9 Done Sent Done false
7194 2690 Acked Read 7 Done Sent Done false
7195 2691 Acked Read 4 Done Sent Done false
7196 2692 Acked Write 7 Done Sent Done false
7197 2693 Acked Flush 0 Done Sent Done false
7198 2694 Acked Read 5 Done Sent Done false
7199 2695 Acked Read 4 Done Sent Done false
7200 2696 Acked Write 2 Done Sent Done false
7201 2697 Acked Read 6 Done Sent Done false
7202 2698 Acked Flush 0 Done Sent Done false
7203 2699 Acked Write 9 Done Sent Done false
7204 2700 Acked Read 3 Done Sent Done false
7205 2701 Acked Read 6 Done Sent Done false
7206 2702 Acked Read 2 Done Sent Done false
7207 2703 Acked Write 2 Done Sent Done false
7208 2704 Acked Write 9 Done Sent Done false
7209 2705 Acked Write 2 Done Sent Done false
7210 2706 Acked Write 2 Done Sent Done false
7211 2707 Acked Read 6 Done Sent Done false
7212 2708 Acked Write 2 Done Sent Done false
7213 2709 Acked Write 3 Done Sent Done false
7214 2710 Acked Read 1 Done Sent Done false
7215 2711 Acked Write 1 Done Sent Done false
7216 2712 Acked Write 1 Done Sent Done false
7217 2713 Acked Write 4 Done Sent Done false
7218 2714 Acked Read 6 Done Sent Done false
7219 2715 Acked Write 8 Done Sent Done false
7220 2716 Acked Read 7 Done Sent Done false
7221 2717 Acked Flush 0 Done Sent Done false
7222 2718 Acked Write 2 Done Sent Done false
7223 2719 Acked Write 4 Done Sent Done false
7224 2720 Acked Write 1 Done Sent Done false
7225 2721 Acked Read 5 Done Sent Done false
7226 2722 Acked Read 5 Done Sent Done false
7227 2723 Acked Read 2 Done Sent Done false
7228 2724 Acked Read 5 Done Sent Done false
7229 2725 Acked Read 10 Done Sent Done false
7230 2726 Acked Read 2 Done Sent Done false
7231 2727 Acked Write 7 Done Sent Done false
7232 2728 Acked Read 3 Done Sent Done false
7233 2729 Acked Read 7 Done Sent Done false
7234 2730 Acked Read 4 Done Sent Done false
7235 2731 Acked Write 1 Done Sent Done false
7236 2732 Acked Flush 0 Done Sent Done false
7237 2733 Acked Read 4 Done Sent Done false
7238 2734 Acked Write 3 Done Sent Done false
7239 2735 Acked Write 6 Done Sent Done false
7240 2736 Acked Write 8 Done Sent Done false
7241 2737 Acked Flush 0 Done Sent Done false
7242 2738 Acked Read 5 Done Sent Done false
7243 2739 Acked Flush 0 Done Sent Done false
7244 2740 Acked Write 5 Done Sent Done false
7245 2741 Acked Read 2 Done Sent Done false
7246 2742 Acked Flush 0 Done Sent Done false
7247 2743 Acked Flush 0 Done Sent Done false
7248 2744 Acked Read 9 Done Sent Done false
7249 2745 Acked Read 7 Done Sent Done false
7250 2746 Acked Read 3 Done Sent Done false
7251 2747 Acked Write 8 Done Sent Done false
7252 2748 Acked Read 8 Done Sent Done false
7253 2749 Acked Read 8 Done Sent Done false
7254 2750 Acked Read 2 Done Sent Done false
7255 2751 Acked Flush 0 Done Sent Done false
7256 2752 Acked Write 2 Done Sent Done false
7257 2753 Acked Read 4 Done Sent Done false
7258 2754 Acked Read 8 Done Sent Done false
7259 2755 Acked Read 2 Done Sent Done false
7260 2756 Acked Read 1 Done Sent Done false
7261 2757 Acked Read 3 Done Sent Done false
7262 2758 Acked Write 8 Done Sent Done false
7263 2759 Acked Write 8 Done Sent Done false
7264 2760 Acked Read 9 Done Sent Done false
7265 2761 Acked Read 4 Done Sent Done false
7266 2762 Acked Read 2 Done Sent Done false
7267 2763 Acked Read 7 Done Sent Done false
7268 2764 Acked Flush 0 Done Sent Done false
7269 2765 Acked Read 1 Done Sent Done false
7270 2766 Acked Write 8 Done Sent Done false
7271 2767 Acked Read 4 Done Sent Done false
7272 2768 Acked Read 3 Done Sent Done false
7273 2769 Acked Write 8 Done Sent Done false
7274 2770 Acked Read 6 Done Sent Done false
7275 2771 Acked Read 6 Done Sent Done false
7276 2772 Acked Write 10 Done Sent Done false
7277 2773 Acked Write 1 Done Sent Done false
7278 2774 Acked Write 1 Done Sent Done false
7279 2775 Acked Write 2 Done Sent Done false
7280 2776 Acked Read 6 Done Sent Done false
7281 2777 Acked Read 3 Done Sent Done false
7282 2778 Acked Read 9 Done Sent Done false
7283 2779 Acked Read 8 Done Sent Done false
7284 2780 Acked Read 10 Done Sent Done false
7285 2781 Acked Read 6 Done Sent Done false
7286 2782 Acked Read 7 Done Sent Done false
7287 2783 Acked Read 3 Done Sent Done false
7288 2784 Acked Read 8 Done Sent Done false
7289 2785 Acked Write 9 Done Sent Done false
7290 2786 Acked Read 3 Done Sent Done false
7291 2787 Acked Read 7 Done Sent Done false
7292 2788 Acked Write 5 Done Sent Done false
7293 2789 Acked Read 3 Done Sent Done false
7294 2790 Acked Flush 0 Done Sent Done false
7295 2791 Acked Read 3 Done Sent Done false
7296 2792 Acked Read 2 Done Sent Done false
7297 2793 Acked Read 2 Done Sent Done false
7298 2794 Acked Write 3 Done Sent Done false
7299 2795 Acked Read 8 Done Sent Done false
7300 2796 Acked Write 8 Done Sent Done false
7301 2797 Acked Read 9 Done Sent Done false
7302 2798 Acked Read 2 Done Sent Done false
7303 2799 Acked Write 5 Done Sent Done false
7304 2800 Acked Write 9 Done Sent Done false
7305 2801 Acked Write 4 Done Sent Done false
7306 2802 Acked Read 6 Done Sent Done false
7307 2803 Acked Read 10 Done Sent Done false
7308 2804 Acked Write 8 Done Sent Done false
7309 2805 Acked Flush 0 Done Sent Done false
7310 2806 Acked Read 3 Done Sent Done false
7311 2807 Acked Write 8 Done Sent Done false
7312 2808 Acked Read 9 Done Sent Done false
7313 2809 Acked Write 8 Done Sent Done false
7314 2810 Acked Read 4 Done Sent Done false
7315 2811 Acked Flush 0 Done Sent Done false
7316 STATES DS:0 DS:1 DS:2 TOTAL
7317 Sent 0 302 0 302
7318 Done 302 0 302 604
7319 Skipped 0 0 0 0
7320 Error 0 0 0 0
7321 Last Flush: 2811 2509 2811
7322 Downstairs last five completed: 2509 2508 2507 2506 2505
7323 Upstairs last five completed: 2811 2810 2809 2808 2807
7324 CLIENT: Up:0 ds:302 act:2
73252024-11-26T22:06:05.625ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
73262024-11-26T22:06:05.625ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:62754) = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
73272024-11-26T22:06:05.626ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
73282024-11-26T22:06:05.627ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
73292024-11-26T22:06:05.627ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
73302024-11-26T22:06:05.627ZINFOcrucible: send last flush ID to this DS: 2509 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
73312024-11-26T22:06:05.627ZINFOcrucible: Replied this last flush ID: 2509 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
73322024-11-26T22:06:05.627ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
73332024-11-26T22:06:05.627ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
73342024-11-26T22:06:05.627ZINFOcrucible: [1] client re-new 302 jobs since flush 2509 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
73352024-11-26T22:06:05.627ZINFOcrucible: [1] Replayed 302 jobs since flush: 2509 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
7336 ----------------------------------------------------------------
7337 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
7338 Downstairs last five completed: 2811 2810 2809 2808 2807
7339 Upstairs last five completed: 2811 2810 2809 2808 2807
7340 CLIENT: Up:0 ds:0 act:3
7341 Replay: All jobs finished
7342 Main task has work to do, go find it
7343 got dsc Stop(0)
7344 stop 0
7345 Nov 26 22:06:05.865 INFO request completed, latency_us: 375, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 4ec98945-8f52-4c21-b4d0-a3922e90f434, remote_addr: [8810] Got stop action so:false kr:false
7346 127.0.0.1:36829, local_addr: 127.0.0.1:9998
73472024-11-26T22:06:05.866ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
73482024-11-26T22:06:05.866ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
73492024-11-26T22:06:05.866ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
73502024-11-26T22:06:05.866ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
73512024-11-26T22:06:05.866ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
7352 Nov 26 22:06:05.866 INFO request completed, latency_us: 142, response_code: 200, uri: /state/cid/0, method: GET, {"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2024-11-26T22:06:05.866803326Z","hostname":"w-01JDN7ZQ17ABQG15KHKN3F2D26","pid":1203,"":"io task","client":"0","":"downstairs","session_id":"86a36c17-3718-484c-87fd-2b8a1dc2ab18"}
7353 req_id: 44edf52f-9758-46bd-9b0f-9d9db163022d, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
7354 [8810] Exited with: ExitStatus(unix_wait_status(9))
7355 [8810][0] reports Exit
7356 Nov 26 22:06:09.869 INFO request completed, latency_us: 195, response_code: 200, uri: /state/cid/0, method: GET, req_id: 5e4298ef-4f56-4caf-80d7-37059e4a1b86, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
7357 001/300 Read block 1120 len 40960
7358 002/300 Read block 4519 len 28672
7359 003/300 Write block 2775 len 24576 data: 3 3 3 3 3 2
7360 004/300 Write block 3579 len 36864 data: 2 2 2 2 2 2 2 2 2
7361 005/300 Write block 4871 len 32768 data: 3 3 3 3 3 3 3 3
7362 006/300 Read block 2240 len 12288
7363 007/300 Flush
7364 008/300 Read block 4688 len 4096
7365 009/300 Read block 396 len 20480
7366 010/300 Read block 2433 len 4096
7367 011/300 Write block 4083 len 4096 data: 2
7368 012/300 Flush
7369 013/300 Flush
7370 014/300 Flush
7371 015/300 Read block 1483 len 28672
7372 016/300 Read block 2472 len 36864
7373 017/300 Read block 764 len 36864
7374 018/300 Read block 527 len 40960
7375 019/300 Write block 1147 len 32768 data: 3 3 3 3 3 2 2 2
7376 020/300 Read block 1051 len 28672
7377 021/300 Read block 3139 len 28672
7378 022/300 Write block 2025 len 8192 data: 3 3
7379 023/300 Write block 3872 len 12288 data: 3 4 4
7380 024/300 Write block 4324 len 36864 data: 3 3 3 4 4 3 3 3 3
7381 025/300 Read block 2427 len 36864
7382 026/300 Flush
7383 027/300 Read block 1645 len 28672
7384 028/300 Write block 1478 len 24576 data: 4 4 3 3 3 3
7385 029/300 Read block 3394 len 40960
7386 030/300 Flush
7387 031/300 Write block 1631 len 12288 data: 2 2 2
7388 032/300 Read block 4222 len 40960
7389 033/300 Flush
7390 034/300 Read block 2495 len 12288
7391 035/300 Write block 2067 len 8192 data: 3 4
7392 036/300 Write block 2863 len 36864 data: 3 3 3 3 3 3 3 2 2
7393 037/300 Flush
7394 038/300 Read block 802 len 8192
7395 039/300 Read block 3272 len 28672
7396 040/300 Read block 1574 len 36864
7397 041/300 Read block 233 len 12288
7398 042/300 Read block 2047 len 24576
7399 043/300 Read block 496 len 16384
7400 044/300 Read block 1644 len 28672
7401 045/300 Write block 1710 len 40960 data: 2 2 2 2 2 3 3 3 2 2
7402 046/300 Write block 163 len 12288 data: 2 2 2
7403 047/300 Write block 3941 len 32768 data: 3 2 2 4 3 3 4 4
7404 048/300 Read block 4754 len 16384
7405 049/300 Flush
7406 050/300 Write block 3624 len 8192 data: 3 3
7407 051/300 Write block 1036 len 20480 data: 2 2 2 2 2
7408 052/300 Write block 1745 len 28672 data: 2 2 3 3 3 3 3
7409 053/300 Write block 785 len 36864 data: 3 3 3 3 2 2 2 2 2
7410 054/300 Read block 2662 len 28672
7411 055/300 Flush
7412 056/300 Read block 840 len 4096
7413 057/300 Write block 1823 len 16384 data: 3 3 3 3
7414 058/300 Read block 3598 len 20480
7415 059/300 Write block 1569 len 8192 data: 3 2
7416 060/300 Write block 1956 len 12288 data: 3 2 2
7417 061/300 Flush
7418 062/300 Write block 4255 len 20480 data: 3 2 2 2 2
7419 063/300 Read block 1112 len 36864
7420 064/300 Write block 2386 len 40960 data: 2 3 3 3 4 3 2 2 2 2
7421 065/300 Read block 4686 len 32768
7422 066/300 Read block 933 len 4096
7423 067/300 Read block 2424 len 28672
7424 068/300 Read block 431 len 32768
7425 069/300 Read block 4026 len 16384
7426 070/300 Read block 996 len 32768
7427 071/300 Write block 1067 len 8192 data: 2 2
7428 072/300 Read block 2515 len 4096
7429 073/300 Write block 3847 len 4096 data: 3
7430 074/300 Write block 3909 len 16384 data: 2 2 2 2
7431 075/300 Read block 2269 len 16384
7432 076/300 Write block 3325 len 20480 data: 3 3 3 3 3
7433 077/300 Read block 4846 len 36864
7434 078/300 Write block 3087 len 20480 data: 3 3 3 3 2
7435 079/300 Read block 3299 len 32768
7436 080/300 Write block 2772 len 8192 data: 3 3
7437 081/300 Write block 238 len 28672 data: 3 3 3 3 3 3 2
7438 082/300 Read block 2550 len 40960
7439 083/300 Read block 2464 len 40960
7440 084/300 Read block 4816 len 28672
7441 085/300 Read block 1891 len 36864
7442 086/300 Flush
7443 087/300 Write block 3040 len 24576 data: 3 2 2 2 2 2
7444 088/300 Flush
7445 089/300 Read block 1800 len 40960
7446 090/300 Write block 4028 len 20480 data: 3 3 3 2 2
7447 091/300 Write block 227 len 8192 data: 5 5
7448 092/300 Read block 1737 len 12288
7449 093/300 Write block 2471 len 40960 data: 4 4 4 3 2 2 2 3 3 4
7450 094/300 Write block 1228 len 36864 data: 2 2 2 2 2 2 2 2 2
7451 095/300 Write block 2 len 4096 data: 3
7452 096/300 Write block 4197 len 12288 data: 4 4 3
7453 097/300 Read block 1269 len 28672
7454 098/300 Read block 4107 len 32768
7455 099/300 Read block 262 len 40960
7456 100/300 Read block 594 len 28672
7457 101/300 Write block 542 len 8192 data: 3 3
7458 102/300 Write block 4634 len 24576 data: 2 2 2 2 2 2
7459 103/300 Read block 1373 len 12288
7460 104/300 Flush
7461 105/300 Read block 526 len 20480
7462 106/300 Write block 533 len 28672 data: 2 2 2 2 2 2 2
7463 107/300 Flush
7464 108/300 Write block 4705 len 28672 data: 3 3 3 2 2 2 2
7465 109/300 Read block 4798 len 16384
7466 110/300 Read block 1342 len 16384
7467 111/300 Write block 4693 len 12288 data: 5 5 5
7468 112/300 Read block 2599 len 36864
7469 113/300 Read block 3904 len 28672
7470 114/300 Write block 1895 len 16384 data: 2 2 2 2
7471 115/300 Write block 118 len 20480 data: 3 2 3 4 5
7472 116/300 Write block 2105 len 28672 data: 3 2 2 2 2 2 2
7473 117/300 Read block 1213 len 20480
7474 118/300 Write block 3861 len 8192 data: 4 3
7475 119/300 Write block 2491 len 16384 data: 2 2 2 2
7476 120/300 Write block 3021 len 24576 data: 2 2 2 2 2 2
7477 121/300 Write block 1599 len 24576 data: 3 3 3 3 3 3
7478 122/300 Read block 3870 len 40960
7479 123/300 Read block 3300 len 12288
7480 124/300 Write block 1250 len 8192 data: 2 2
7481 125/300 Read block 1184 len 40960
7482 126/300 Write block 1522 len 32768 data: 2 3 3 4 4 4 5 5
7483 127/300 Read block 858 len 20480
7484 128/300 Write block 3658 len 12288 data: 3 2 2
7485 129/300 Write block 2480 len 28672 data: 5 4 4 4 4 4 4
7486 130/300 Write block 4603 len 32768 data: 3 3 3 2 2 3 3 3
7487 131/300 Write block 4055 len 20480 data: 3 3 3 3 4
7488 132/300 Write block 1632 len 20480 data: 3 3 2 2 2
7489 133/300 Write block 1609 len 20480 data: 2 2 2 2 2
7490 134/300 Read block 1476 len 4096
7491 135/300 Read block 1628 len 24576
7492 136/300 Write block 910 len 28672 data: 4 4 4 4 4 4 4
7493 137/300 Write block 3690 len 40960 data: 2 2 2 2 2 2 2 2 3 4
7494 138/300 Write block 3228 len 12288 data: 4 4 4
7495 139/300 Write block 1689 len 16384 data: 3 3 3 3
7496 140/300 Write block 4596 len 40960 data: 3 3 3 3 3 3 3 4 4 4
7497 141/300 Read block 410 len 40960
7498 142/300 Write block 316 len 32768 data: 2 2 2 2 2 2 2 3
7499 143/300 Read block 3891 len 24576
7500 144/300 Read block 4923 len 8192
7501 145/300 Read block 1112 len 36864
7502 146/300 Write block 537 len 12288 data: 3 3 3
7503 147/300 Write block 2389 len 40960 data: 4 5 4 3 3 3 3 3 3 3
7504 148/300 Read block 1484 len 4096
7505 149/300 Write block 4364 len 20480 data: 4 4 4 4 4
7506 150/300 Flush
7507 151/300 Write block 614 len 4096 data: 3
7508 152/300 Write block 2059 len 36864 data: 4 4 4 5 5 5 5 4 4
7509 153/300 Write block 3108 len 20480 data: 2 2 2 2 2
7510 154/300 Read block 1338 len 24576
7511 155/300 Read block 2760 len 40960
7512 156/300 Read block 1977 len 32768
7513 157/300 Read block 4668 len 28672
7514 158/300 Read block 1321 len 32768
7515 159/300 Write block 2911 len 20480 data: 3 3 3 3 4
7516 160/300 Flush
7517 161/300 Read block 671 len 8192
7518 162/300 Read block 3982 len 8192
7519 163/300 Read block 3328 len 24576
7520 164/300 Write block 2392 len 16384 data: 4 4 4 4
7521 165/300 Write block 434 len 40960 data: 3 3 3 2 2 2 2 2 2 2
7522 166/300 Read block 2392 len 36864
7523 167/300 Write block 3227 len 28672 data: 3 5 5 5 4 4 3
7524 168/300 Read block 1286 len 24576
7525 169/300 Write block 407 len 24576 data: 4 3 3 3 3 3
7526 170/300 Write block 428 len 28672 data: 3 3 2 2 3 3 4
7527 171/300 Read block 615 len 20480
7528 172/300 Read block 3366 len 32768
7529 173/300 Read block 4164 len 40960
7530 174/300 Write block 1953 len 40960 data: 2 2 3 4 3 3 2 2 2 2
7531 175/300 Write block 1281 len 40960 data: 3 3 3 3 3 2 2 2 2 2
7532 176/300 Write block 382 len 36864 data: 3 3 3 2 2 2 2 2 2
7533 177/300 Write block 586 len 32768 data: 3 3 3 3 3 3 2 2
7534 178/300 Write block 1079 len 36864 data: 4 4 3 3 4 4 3 3 3
7535 179/300 Flush
7536 180/300 Write block 2541 len 8192 data: 3 3
7537 181/300 Read block 2904 len 4096
7538 182/300 Write block 4348 len 16384 data: 2 3 3 3
7539 183/300 Read block 3212 len 20480
7540 184/300 Read block 3346 len 24576
7541 185/300 Read block 3813 len 12288
7542 186/300 Read block 1416 len 12288
7543 187/300 Write block 4842 len 28672 data: 2 2 2 2 2 2 2
7544 188/300 Write block 1679 len 16384 data: 2 2 2 2
7545 189/300 Read block 1334 len 28672
7546 190/300 Read block 4704 len 40960
7547 191/300 Read block 2254 len 40960
7548 192/300 Write block 4371 len 16384 data: 2 3 3 3
7549 193/300 Read block 1898 len 40960
7550 194/300 Write block 4583 len 12288 data: 2 2 2
7551 195/300 Read block 872 len 40960
7552 196/300 Write block 3965 len 8192 data: 2 3
7553 197/300 Write block 4870 len 12288 data: 3 4 4
7554 198/300 Read block 2069 len 16384
7555 199/300 Write block 3688 len 20480 data: 3 2 3 3 3
7556 200/300 Read block 827 len 32768
7557 201/300 Read block 314 len 36864
7558 202/300 Read block 4288 len 32768
7559 203/300 Write block 4968 len 28672 data: 4 4 3 2 2 2 2
7560 204/300 Flush
7561 205/300 Read block 1523 len 24576
7562 206/300 Write block 602 len 32768 data: 2 2 2 2 2 2 2 2
7563 207/300 Read block 1730 len 36864
7564 208/300 Write block 3217 len 4096 data: 3
7565 209/300 Read block 1046 len 36864
7566 210/300 Read block 4642 len 32768
7567 211/300 Write block 3502 len 32768 data: 2 2 2 2 2 2 2 2
7568 212/300 Read block 3464 len 16384
7569 213/300 Read block 23 len 8192
7570 214/300 Write block 345 len 20480 data: 3 2 3 3 4
7571 215/300 Write block 588 len 8192 data: 4 4
7572 216/300 Write block 1837 len 12288 data: 2 2 2
7573 217/300 Read block 405 len 8192
7574 218/300 Write block 3120 len 8192 data: 4 4
7575 219/300 Write block 3090 len 12288 data: 4 3 2
7576 220/300 Write block 615 len 12288 data: 3 3 3
7577 221/300 Read block 3186 len 8192
7578 222/300 Write block 4632 len 40960 data: 2 2 3 3 3 3 3 3 2 2
7579 223/300 Write block 3474 len 40960 data: 3 3 3 3 3 3 3 3 2 2
7580 224/300 Write block 4141 len 16384 data: 2 2 2 2
7581 225/300 Read block 4461 len 8192
7582 226/300 Write block 3803 len 40960 data: 3 2 2 2 2 2 2 3 3 3
7583 227/300 Read block 1763 len 16384
7584 228/300 Write block 3048 len 20480 data: 2 2 2 2 2
7585 229/300 Write block 4680 len 12288 data: 4 4 3
7586 230/300 Read block 1414 len 24576
7587 231/300 Write block 1378 len 20480 data: 2 2 3 3 3
7588 232/300 Write block 1981 len 36864 data: 2 3 3 3 3 4 4 4 3
7589 233/300 Read block 1062 len 8192
7590 234/300 Write block 2943 len 8192 data: 2 2
7591 235/300 Read block 3309 len 36864
7592 236/300 Write block 4725 len 12288 data: 2 2 2
7593 237/300 Write block 4650 len 24576 data: 3 3 3 3 2 3
7594 238/300 Write block 3003 len 8192 data: 2 2
7595 239/300 Read block 3501 len 28672
7596 240/300 Write block 2480 len 40960 data: 6 5 5 5 5 5 5 3 3 2
7597 241/300 Write block 801 len 4096 data: 3
7598 242/300 Write block 4686 len 36864 data: 2 2 3 4 4 5 5 6 6
7599 243/300 Read block 804 len 32768
7600 244/300 Read block 4890 len 32768
7601 245/300 Read block 3167 len 32768
7602 246/300 Flush
7603 247/300 Flush
7604 248/300 Write block 4806 len 20480 data: 4 4 5 6 5
7605 249/300 Read block 4343 len 36864
7606 250/300 Flush
7607 251/300 Read block 841 len 4096
7608 252/300 Read block 2244 len 28672
7609 253/300 Write block 3477 len 4096 data: 4
7610 254/300 Read block 2537 len 36864
7611 255/300 Read block 3431 len 8192
7612 256/300 Flush
7613 257/300 Read block 3381 len 40960
7614 258/300 Write block 2875 len 12288 data: 2 2 2
7615 259/300 Read block 3440 len 32768
7616 260/300 Write block 4810 len 40960 data: 6 4 4 4 4 4 4 4 3 3
7617 261/300 Read block 4777 len 20480
7618 262/300 Read block 3317 len 8192
7619 263/300 Write block 4263 len 24576 data: 2 2 2 2 3 2
7620 264/300 Read block 2004 len 36864
7621 265/300 Write block 1594 len 28672 data: 2 2 2 2 2 4 4
7622 266/300 Flush
7623 267/300 Read block 4455 len 4096
7624 268/300 Read block 849 len 32768
7625 269/300 Write block 637 len 40960 data: 3 3 4 4 4 4 4 4 3 2
7626 270/300 Write block 655 len 12288 data: 2 2 3
7627 271/300 Write block 4023 len 16384 data: 2 2 3 3
7628 272/300 Read block 573 len 36864
7629 273/300 Write block 3718 len 8192 data: 5 5
7630 274/300 Write block 621 len 36864 data: 2 3 4 4 3 3 3 3 3
7631 275/300 Read block 4346 len 36864
7632 276/300 Read block 2896 len 20480
7633 277/300 Write block 2840 len 16384 data: 4 4 4 4
7634 278/300 Write block 4990 len 24576 data: 4 4 4 4 4 4
7635 279/300 Flush
7636 280/300 Flush
7637 281/300 Read block 3981 len 20480
7638 282/300 Read block 1208 len 28672
7639 283/300 Read block 551 len 4096
7640 284/300 Write block 1641 len 36864 data: 2 2 2 3 3 3 3 3 4
7641 285/300 Read block 483 len 20480
7642 286/300 Write block 4818 len 32768 data: 4 4 2 2 3 3 3 3
7643 287/300 Read block 1639 len 24576
7644 288/300 Read block 1190 len 8192
7645 289/300 Read block 2336 len 40960
7646 290/300 Write block 1349 len 40960 data: 2 4 4 4 3 4 5 5 3 3
7647 291/300 Write block 1273 len 8192 data: 2 2
7648 292/300 Write block 1088 len 4096 data: 3
7649 293/300 Read block 3840 len 8192
7650 294/300 Write block 4861 len 4096 data: 4
7651 295/300 Write block 2899 len 4096 data: 3
7652 296/300 Read block 1250 len 32768
7653 297/300 Write block 812 len 8192 data: 2 2
7654 298/300 Read block 4884 len 36864
7655 299/300 Read block 4807 len 16384
7656 300/300 Write block 2992 len 32768 data: 2 2 2 2 2 2 3 4
7657 301/300 Read block 1457 len 20480
7658 Main task has work to do, go find it
7659 got dsc Start(0)
7660 start 0
7661 Nov 26 22:06:10.104 INFO request completed, latency_us: 216, response_code: 204, uri: /start/cid/0, method: POST, req_id: 8465f654-f8b7-4452-83cd-97bc932f416f, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
7662 Replay: started 0, returned:Ok(())
7663 ----------------------------------------------------------------
7664 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
7665 States: Offline Active Active
7666 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
7667 2812 Acked Read 10 Sent Done Done false
7668 2813 Acked Read 7 Sent Done Done false
7669 2814 Acked Write 6 Sent Done Done false
7670 2815 Acked Write 9 Sent Done Done false
7671 2816 Acked Write 8 Sent Done Done false
7672 2817 Acked Read 3 Sent Done Done false
7673 2818 Acked Flush 0 Sent Done Done false
7674 2819 Acked Read 1 Sent Done Done false
7675 2820 Acked Read 5 Sent Done Done false
7676 2821 Acked Read 1 Sent Done Done false
7677 2822 Acked Write 1 Sent Done Done false
7678 2823 Acked Flush 0 Sent Done Done false
7679 2824 Acked Flush 0 Sent Done Done false
7680 2825 Acked Flush 0 Sent Done Done false
7681 2826 Acked Read 7 Sent Done Done false
7682 2827 Acked Read 9 Sent Done Done false
7683 2828 Acked Read 9 Sent Done Done false
7684 2829 Acked Read 10 Sent Done Done false
7685 2830 Acked Write 8 Sent Done Done false
7686 2831 Acked Read 7 Sent Done Done false
7687 2832 Acked Read 7 Sent Done Done false
7688 2833 Acked Write 2 Sent Done Done false
7689 2834 Acked Write 3 Sent Done Done false
7690 2835 Acked Write 9 Sent Done Done false
7691 2836 Acked Read 9 Sent Done Done false
7692 2837 Acked Flush 0 Sent Done Done false
7693 2838 Acked Read 7 Sent Done Done false
7694 2839 Acked Write 6 Sent Done Done false
7695 2840 Acked Read 10 Sent Done Done false
7696 2841 Acked Flush 0 Sent Done Done false
7697 2842 Acked Write 3 Sent Done Done false
7698 2843 Acked Read 10 Sent Done Done false
7699 2844 Acked Flush 0 Sent Done Done false
7700 2845 Acked Read 3 Sent Done Done false
7701 2846 Acked Write 2 Sent Done Done false
7702 2847 Acked Write 9 Sent Done Done false
7703 2848 Acked Flush 0 Sent Done Done false
7704 2849 Acked Read 2 Sent Done Done false
7705 2850 Acked Read 7 Sent Done Done false
7706 2851 Acked Read 9 Sent Done Done false
7707 2852 Acked Read 3 Sent Done Done false
7708 2853 Acked Read 6 Sent Done Done false
7709 2854 Acked Read 4 Sent Done Done false
7710 2855 Acked Read 7 Sent Done Done false
7711 2856 Acked Write 10 Sent Done Done false
7712 2857 Acked Write 3 Sent Done Done false
7713 2858 Acked Write 8 Sent Done Done false
7714 2859 Acked Read 4 Sent Done Done false
7715 2860 Acked Flush 0 Sent Done Done false
7716 2861 Acked Write 2 Sent Done Done false
7717 2862 Acked Write 5 Sent Done Done false
7718 2863 Acked Write 7 Sent Done Done false
7719 2864 Acked Write 9 Sent Done Done false
7720 2865 Acked Read 7 Sent Done Done false
7721 2866 Acked Flush 0 Sent Done Done false
7722 2867 Acked Read 1 Sent Done Done false
7723 2868 Acked Write 4 Sent Done Done false
7724 2869 Acked Read 5 Sent Done Done false
7725 2870 Acked Write 2 Sent Done Done false
7726 2871 Acked Write 3 Sent Done Done false
7727 2872 Acked Flush 0 Sent Done Done false
7728 2873 Acked Write 5 Sent Done Done false
7729 2874 Acked Read 9 Sent Done Done false
7730 2875 Acked Write 10 Sent Done Done false
7731 2876 Acked Read 8 Sent Done Done false
7732 2877 Acked Read 1 Sent Done Done false
7733 2878 Acked Read 7 Sent Done Done false
7734 2879 Acked Read 8 Sent Done Done false
7735 2880 Acked Read 4 Sent Done Done false
7736 2881 Acked Read 8 Sent Done Done false
7737 2882 Acked Write 2 Sent Done Done false
7738 2883 Acked Read 1 Sent Done Done false
7739 2884 Acked Write 1 Sent Done Done false
7740 2885 Acked Write 4 Sent Done Done false
7741 2886 Acked Read 4 Sent Done Done false
7742 2887 Acked Write 5 Sent Done Done false
7743 2888 Acked Read 9 Sent Done Done false
7744 2889 Acked Write 5 Sent Done Done false
7745 2890 Acked Read 8 Sent Done Done false
7746 2891 Acked Write 2 Sent Done Done false
7747 2892 Acked Write 7 Sent Done Done false
7748 2893 Acked Read 10 Sent Done Done false
7749 2894 Acked Read 10 Sent Done Done false
7750 2895 Acked Read 7 Sent Done Done false
7751 2896 Acked Read 9 Sent Done Done false
7752 2897 Acked Flush 0 Sent Done Done false
7753 2898 Acked Write 6 Sent Done Done false
7754 2899 Acked Flush 0 Sent Done Done false
7755 2900 Acked Read 10 Sent Done Done false
7756 2901 Acked Write 5 Sent Done Done false
7757 2902 Acked Write 2 Sent Done Done false
7758 2903 Acked Read 3 Sent Done Done false
7759 2904 Acked Write 10 Sent Done Done false
7760 2905 Acked Write 9 Sent Done Done false
7761 2906 Acked Write 1 Sent Done Done false
7762 2907 Acked Write 3 Sent Done Done false
7763 2908 Acked Read 7 Sent Done Done false
7764 2909 Acked Read 8 Sent Done Done false
7765 2910 Acked Read 10 Sent Done Done false
7766 2911 Acked Read 7 Sent Done Done false
7767 2912 Acked Write 2 Sent Done Done false
7768 2913 Acked Write 6 Sent Done Done false
7769 2914 Acked Read 3 Sent Done Done false
7770 2915 Acked Flush 0 Sent Done Done false
7771 2916 Acked Read 5 Sent Done Done false
7772 2917 Acked Write 7 Sent Done Done false
7773 2918 Acked Flush 0 Sent Done Done false
7774 2919 Acked Write 7 Sent Done Done false
7775 2920 Acked Read 4 Sent Done Done false
7776 2921 Acked Read 4 Sent Done Done false
7777 2922 Acked Write 3 Sent Done Done false
7778 2923 Acked Read 9 Sent Done Done false
7779 2924 Acked Read 7 Sent Done Done false
7780 2925 Acked Write 4 Sent Done Done false
7781 2926 Acked Write 5 Sent Done Done false
7782 2927 Acked Write 7 Sent Done Done false
7783 2928 Acked Read 5 Sent Done Done false
7784 2929 Acked Write 2 Sent Done Done false
7785 2930 Acked Write 4 Sent Done Done false
7786 2931 Acked Write 6 Sent Done Done false
7787 2932 Acked Write 6 Sent Done Done false
7788 2933 Acked Read 10 Sent Done Done false
7789 2934 Acked Read 3 Sent Done Done false
7790 2935 Acked Write 2 Sent Done Done false
7791 2936 Acked Read 10 Sent Done Done false
7792 2937 Acked Write 8 Sent Done Done false
7793 2938 Acked Read 5 Sent Done Done false
7794 2939 Acked Write 3 Sent Done Done false
7795 2940 Acked Write 7 Sent Done Done false
7796 2941 Acked Write 8 Sent Done Done false
7797 2942 Acked Write 5 Sent Done Done false
7798 2943 Acked Write 5 Sent Done Done false
7799 2944 Acked Write 5 Sent Done Done false
7800 2945 Acked Read 1 Sent Done Done false
7801 2946 Acked Read 6 Sent Done Done false
7802 2947 Acked Write 7 Sent Done Done false
7803 2948 Acked Write 10 Sent Done Done false
7804 2949 Acked Write 3 Sent Done Done false
7805 2950 Acked Write 4 Sent Done Done false
7806 2951 Acked Write 10 Sent Done Done false
7807 2952 Acked Read 10 Sent Done Done false
7808 2953 Acked Write 8 Sent Done Done false
7809 2954 Acked Read 6 Sent Done Done false
7810 2955 Acked Read 2 Sent Done Done false
7811 2956 Acked Read 9 Sent Done Done false
7812 2957 Acked Write 3 Sent Done Done false
7813 2958 Acked Write 10 Sent Done Done false
7814 2959 Acked Read 1 Sent Done Done false
7815 2960 Acked Write 5 Sent Done Done false
7816 2961 Acked Flush 0 Sent Done Done false
7817 2962 Acked Write 1 Sent Done Done false
7818 2963 Acked Write 9 Sent Done Done false
7819 2964 Acked Write 5 Sent Done Done false
7820 2965 Acked Read 6 Sent Done Done false
7821 2966 Acked Read 10 Sent Done Done false
7822 2967 Acked Read 8 Sent Done Done false
7823 2968 Acked Read 7 Sent Done Done false
7824 2969 Acked Read 8 Sent Done Done false
7825 2970 Acked Write 5 Sent Done Done false
7826 2971 Acked Flush 0 Sent Done Done false
7827 2972 Acked Read 2 Sent Done Done false
7828 2973 Acked Read 2 Sent Done Done false
7829 2974 Acked Read 6 Sent Done Done false
7830 2975 Acked Write 4 Sent Done Done false
7831 2976 Acked Write 10 Sent Done Done false
7832 2977 Acked Read 9 Sent Done Done false
7833 2978 Acked Write 7 Sent Done Done false
7834 2979 Acked Read 6 Sent Done Done false
7835 2980 Acked Write 6 Sent Done Done false
7836 2981 Acked Write 7 Sent Done Done false
7837 2982 Acked Read 5 Sent Done Done false
7838 2983 Acked Read 8 Sent Done Done false
7839 2984 Acked Read 10 Sent Done Done false
7840 2985 Acked Write 10 Sent Done Done false
7841 2986 Acked Write 10 Sent Done Done false
7842 2987 Acked Write 9 Sent Done Done false
7843 2988 Acked Write 8 Sent Done Done false
7844 2989 Acked Write 9 Sent Done Done false
7845 2990 Acked Flush 0 Sent Done Done false
7846 2991 Acked Write 2 Sent Done Done false
7847 2992 Acked Read 1 Sent Done Done false
7848 2993 Acked Write 4 Sent Done Done false
7849 2994 Acked Read 5 Sent Done Done false
7850 2995 Acked Read 6 Sent Done Done false
7851 2996 Acked Read 3 Sent Done Done false
7852 2997 Acked Read 3 Sent Done Done false
7853 2998 Acked Write 7 Sent Done Done false
7854 2999 Acked Write 4 Sent Done Done false
7855 3000 Acked Read 7 Sent Done Done false
7856 3001 Acked Read 10 Sent Done Done false
7857 3002 Acked Read 10 Sent Done Done false
7858 3003 Acked Write 4 Sent Done Done false
7859 3004 Acked Read 10 Sent Done Done false
7860 3005 Acked Write 3 Sent Done Done false
7861 3006 Acked Read 10 Sent Done Done false
7862 3007 Acked Write 2 Sent Done Done false
7863 3008 Acked Write 3 Sent Done Done false
7864 3009 Acked Read 4 Sent Done Done false
7865 3010 Acked Write 5 Sent Done Done false
7866 3011 Acked Read 8 Sent Done Done false
7867 3012 Acked Read 9 Sent Done Done false
7868 3013 Acked Read 8 Sent Done Done false
7869 3014 Acked Write 7 Sent Done Done false
7870 3015 Acked Flush 0 Sent Done Done false
7871 3016 Acked Read 6 Sent Done Done false
7872 3017 Acked Write 8 Sent Done Done false
7873 3018 Acked Read 9 Sent Done Done false
7874 3019 Acked Write 1 Sent Done Done false
7875 3020 Acked Read 9 Sent Done Done false
7876 3021 Acked Read 8 Sent Done Done false
7877 3022 Acked Write 8 Sent Done Done false
7878 3023 Acked Read 4 Sent Done Done false
7879 3024 Acked Read 2 Sent Done Done false
7880 3025 Acked Write 5 Sent Done Done false
7881 3026 Acked Write 2 Sent Done Done false
7882 3027 Acked Write 3 Sent Done Done false
7883 3028 Acked Read 2 Sent Done Done false
7884 3029 Acked Write 2 Sent Done Done false
7885 3030 Acked Write 3 Sent Done Done false
7886 3031 Acked Write 3 Sent Done Done false
7887 3032 Acked Read 2 Sent Done Done false
7888 3033 Acked Write 10 Sent Done Done false
7889 3034 Acked Write 10 Sent Done Done false
7890 3035 Acked Write 4 Sent Done Done false
7891 3036 Acked Read 2 Sent Done Done false
7892 3037 Acked Write 10 Sent Done Done false
7893 3038 Acked Read 4 Sent Done Done false
7894 3039 Acked Write 5 Sent Done Done false
7895 3040 Acked Write 3 Sent Done Done false
7896 3041 Acked Read 6 Sent Done Done false
7897 3042 Acked Write 5 Sent Done Done false
7898 3043 Acked Write 9 Sent Done Done false
7899 3044 Acked Read 2 Sent Done Done false
7900 3045 Acked Write 2 Sent Done Done false
7901 3046 Acked Read 9 Sent Done Done false
7902 3047 Acked Write 3 Sent Done Done false
7903 3048 Acked Write 6 Sent Done Done false
7904 3049 Acked Write 2 Sent Done Done false
7905 3050 Acked Read 7 Sent Done Done false
7906 3051 Acked Write 10 Sent Done Done false
7907 3052 Acked Write 1 Sent Done Done false
7908 3053 Acked Write 9 Sent Done Done false
7909 3054 Acked Read 8 Sent Done Done false
7910 3055 Acked Read 8 Sent Done Done false
7911 3056 Acked Read 8 Sent Done Done false
7912 3057 Acked Flush 0 Sent Done Done false
7913 3058 Acked Flush 0 Sent Done Done false
7914 3059 Acked Write 5 Sent Done Done false
7915 3060 Acked Read 9 Sent Done Done false
7916 3061 Acked Flush 0 Sent Done Done false
7917 3062 Acked Read 1 Sent Done Done false
7918 3063 Acked Read 7 Sent Done Done false
7919 3064 Acked Write 1 Sent Done Done false
7920 3065 Acked Read 9 Sent Done Done false
7921 3066 Acked Read 2 Sent Done Done false
7922 3067 Acked Flush 0 Sent Done Done false
7923 3068 Acked Read 10 Sent Done Done false
7924 3069 Acked Write 3 Sent Done Done false
7925 3070 Acked Read 8 Sent Done Done false
7926 3071 Acked Write 10 Sent Done Done false
7927 3072 Acked Read 5 Sent Done Done false
7928 3073 Acked Read 2 Sent Done Done false
7929 3074 Acked Write 6 Sent Done Done false
7930 3075 Acked Read 9 Sent Done Done false
7931 3076 Acked Write 7 Sent Done Done false
7932 3077 Acked Flush 0 Sent Done Done false
7933 3078 Acked Read 1 Sent Done Done false
7934 3079 Acked Read 8 Sent Done Done false
7935 3080 Acked Write 10 Sent Done Done false
7936 3081 Acked Write 3 Sent Done Done false
7937 3082 Acked Write 4 Sent Done Done false
7938 3083 Acked Read 9 Sent Done Done false
7939 3084 Acked Write 2 Sent Done Done false
7940 3085 Acked Write 9 Sent Done Done false
7941 3086 Acked Read 9 Sent Done Done false
7942 3087 Acked Read 5 Sent Done Done false
7943 3088 Acked Write 4 Sent Done Done false
7944 3089 Acked Write 6 Sent Done Done false
7945 3090 Acked Flush 0 Sent Done Done false
7946 3091 Acked Flush 0 Sent Done Done false
7947 3092 Acked Read 5 Sent Done Done false
7948 3093 Acked Read 7 Sent Done Done false
7949 3094 Acked Read 1 Sent Done Done false
7950 3095 Acked Write 9 Sent Done Done false
7951 3096 Acked Read 5 Sent Done Done false
7952 3097 Acked Write 8 Sent Done Done false
7953 3098 Acked Read 6 Sent Done Done false
7954 3099 Acked Read 2 Sent Done Done false
7955 3100 Acked Read 10 Sent Done Done false
7956 3101 Acked Write 10 Sent Done Done false
7957 3102 Acked Write 2 Sent Done Done false
7958 3103 Acked Write 1 Sent Done Done false
7959 3104 Acked Read 2 Sent Done Done false
7960 3105 Acked Write 1 Sent Done Done false
7961 3106 Acked Write 1 Sent Done Done false
7962 3107 Acked Read 8 Sent Done Done false
7963 3108 Acked Write 2 Sent Done Done false
7964 3109 Acked Read 9 Sent Done Done false
7965 3110 Acked Read 4 Sent Done Done false
7966 3111 Acked Write 8 Sent Done Done false
7967 3112 Acked Read 5 Sent Done Done false
7968 STATES DS:0 DS:1 DS:2 TOTAL
7969 Sent 301 0 0 301
7970 Done 0 301 301 602
7971 Skipped 0 0 0 0
7972 Error 0 0 0 0
7973 Last Flush: 2811 3091 3091
7974 Downstairs last five completed: 2811 2810 2809 2808 2807
7975 Upstairs last five completed: 3112 3111 3110 3109 3108
7976 CLIENT: Up:0 ds:301 act:2
7977 [8810] got start action
7978 [8810] I am going to restart
7979 Starting downstairs at port 8810
7980 Make output file at "/tmp/dsc/downstairs-8810.txt"
7981 [8810][0] reports Starting
7982 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1210)
7983 [8810][0] reports Running
7984 ----------------------------------------------------------------
7985 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
7986 States: Offline Active Active
7987 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
7988 2812 Acked Read 10 Sent Done Done false
7989 2813 Acked Read 7 Sent Done Done false
7990 2814 Acked Write 6 Sent Done Done false
7991 2815 Acked Write 9 Sent Done Done false
7992 2816 Acked Write 8 Sent Done Done false
7993 2817 Acked Read 3 Sent Done Done false
7994 2818 Acked Flush 0 Sent Done Done false
7995 2819 Acked Read 1 Sent Done Done false
7996 2820 Acked Read 5 Sent Done Done false
7997 2821 Acked Read 1 Sent Done Done false
7998 2822 Acked Write 1 Sent Done Done false
7999 2823 Acked Flush 0 Sent Done Done false
8000 2824 Acked Flush 0 Sent Done Done false
8001 2825 Acked Flush 0 Sent Done Done false
8002 2826 Acked Read 7 Sent Done Done false
8003 2827 Acked Read 9 Sent Done Done false
8004 2828 Acked Read 9 Sent Done Done false
8005 2829 Acked Read 10 Sent Done Done false
8006 2830 Acked Write 8 Sent Done Done false
8007 2831 Acked Read 7 Sent Done Done false
8008 2832 Acked Read 7 Sent Done Done false
8009 2833 Acked Write 2 Sent Done Done false
8010 2834 Acked Write 3 Sent Done Done false
8011 2835 Acked Write 9 Sent Done Done false
8012 2836 Acked Read 9 Sent Done Done false
8013 2837 Acked Flush 0 Sent Done Done false
8014 2838 Acked Read 7 Sent Done Done false
8015 2839 Acked Write 6 Sent Done Done false
8016 2840 Acked Read 10 Sent Done Done false
8017 2841 Acked Flush 0 Sent Done Done false
8018 2842 Acked Write 3 Sent Done Done false
8019 2843 Acked Read 10 Sent Done Done false
8020 2844 Acked Flush 0 Sent Done Done false
8021 2845 Acked Read 3 Sent Done Done false
8022 2846 Acked Write 2 Sent Done Done false
8023 2847 Acked Write 9 Sent Done Done false
8024 2848 Acked Flush 0 Sent Done Done false
8025 2849 Acked Read 2 Sent Done Done false
8026 2850 Acked Read 7 Sent Done Done false
8027 2851 Acked Read 9 Sent Done Done false
8028 2852 Acked Read 3 Sent Done Done false
8029 2853 Acked Read 6 Sent Done Done false
8030 2854 Acked Read 4 Sent Done Done false
8031 2855 Acked Read 7 Sent Done Done false
8032 2856 Acked Write 10 Sent Done Done false
8033 2857 Acked Write 3 Sent Done Done false
8034 2858 Acked Write 8 Sent Done Done false
8035 2859 Acked Read 4 Sent Done Done false
8036 2860 Acked Flush 0 Sent Done Done false
8037 2861 Acked Write 2 Sent Done Done false
8038 2862 Acked Write 5 Sent Done Done false
8039 2863 Acked Write 7 Sent Done Done false
8040 2864 Acked Write 9 Sent Done Done false
8041 2865 Acked Read 7 Sent Done Done false
8042 2866 Acked Flush 0 Sent Done Done false
8043 2867 Acked Read 1 Sent Done Done false
8044 2868 Acked Write 4 Sent Done Done false
8045 2869 Acked Read 5 Sent Done Done false
8046 2870 Acked Write 2 Sent Done Done false
8047 2871 Acked Write 3 Sent Done Done false
8048 2872 Acked Flush 0 Sent Done Done false
8049 2873 Acked Write 5 Sent Done Done false
8050 2874 Acked Read 9 Sent Done Done false
8051 2875 Acked Write 10 Sent Done Done false
8052 2876 Acked Read 8 Sent Done Done false
8053 2877 Acked Read 1 Sent Done Done false
8054 2878 Acked Read 7 Sent Done Done false
8055 2879 Acked Read 8 Sent Done Done false
8056 2880 Acked Read 4 Sent Done Done false
8057 2881 Acked Read 8 Sent Done Done false
8058 2882 Acked Write 2 Sent Done Done false
8059 2883 Acked Read 1 Sent Done Done false
8060 2884 Acked Write 1 Sent Done Done false
8061 2885 Acked Write 4 Sent Done Done false
8062 2886 Acked Read 4 Sent Done Done false
8063 2887 Acked Write 5 Sent Done Done false
8064 2888 Acked Read 9 Sent Done Done false
8065 2889 Acked Write 5 Sent Done Done false
8066 2890 Acked Read 8 Sent Done Done false
8067 2891 Acked Write 2 Sent Done Done false
8068 2892 Acked Write 7 Sent Done Done false
8069 2893 Acked Read 10 Sent Done Done false
8070 2894 Acked Read 10 Sent Done Done false
8071 2895 Acked Read 7 Sent Done Done false
8072 2896 Acked Read 9 Sent Done Done false
8073 2897 Acked Flush 0 Sent Done Done false
8074 2898 Acked Write 6 Sent Done Done false
8075 2899 Acked Flush 0 Sent Done Done false
8076 2900 Acked Read 10 Sent Done Done false
8077 2901 Acked Write 5 Sent Done Done false
8078 2902 Acked Write 2 Sent Done Done false
8079 2903 Acked Read 3 Sent Done Done false
8080 2904 Acked Write 10 Sent Done Done false
8081 2905 Acked Write 9 Sent Done Done false
8082 2906 Acked Write 1 Sent Done Done false
8083 2907 Acked Write 3 Sent Done Done false
8084 2908 Acked Read 7 Sent Done Done false
8085 2909 Acked Read 8 Sent Done Done false
8086 2910 Acked Read 10 Sent Done Done false
8087 2911 Acked Read 7 Sent Done Done false
8088 2912 Acked Write 2 Sent Done Done false
8089 2913 Acked Write 6 Sent Done Done false
8090 2914 Acked Read 3 Sent Done Done false
8091 2915 Acked Flush 0 Sent Done Done false
8092 2916 Acked Read 5 Sent Done Done false
8093 2917 Acked Write 7 Sent Done Done false
8094 2918 Acked Flush 0 Sent Done Done false
8095 2919 Acked Write 7 Sent Done Done false
8096 2920 Acked Read 4 Sent Done Done false
8097 2921 Acked Read 4 Sent Done Done false
8098 2922 Acked Write 3 Sent Done Done false
8099 2923 Acked Read 9 Sent Done Done false
8100 2924 Acked Read 7 Sent Done Done false
8101 2925 Acked Write 4 Sent Done Done false
8102 2926 Acked Write 5 Sent Done Done false
8103 2927 Acked Write 7 Sent Done Done false
8104 2928 Acked Read 5 Sent Done Done false
8105 2929 Acked Write 2 Sent Done Done false
8106 2930 Acked Write 4 Sent Done Done false
8107 2931 Acked Write 6 Sent Done Done false
8108 2932 Acked Write 6 Sent Done Done false
8109 2933 Acked Read 10 Sent Done Done false
8110 2934 Acked Read 3 Sent Done Done false
8111 2935 Acked Write 2 Sent Done Done false
8112 2936 Acked Read 10 Sent Done Done false
8113 2937 Acked Write 8 Sent Done Done false
8114 2938 Acked Read 5 Sent Done Done false
8115 2939 Acked Write 3 Sent Done Done false
8116 2940 Acked Write 7 Sent Done Done false
8117 2941 Acked Write 8 Sent Done Done false
8118 2942 Acked Write 5 Sent Done Done false
8119 2943 Acked Write 5 Sent Done Done false
8120 2944 Acked Write 5 Sent Done Done false
8121 2945 Acked Read 1 Sent Done Done false
8122 2946 Acked Read 6 Sent Done Done false
8123 2947 Acked Write 7 Sent Done Done false
8124 2948 Acked Write 10 Sent Done Done false
8125 2949 Acked Write 3 Sent Done Done false
8126 2950 Acked Write 4 Sent Done Done false
8127 2951 Acked Write 10 Sent Done Done false
8128 2952 Acked Read 10 Sent Done Done false
8129 2953 Acked Write 8 Sent Done Done false
8130 2954 Acked Read 6 Sent Done Done false
8131 2955 Acked Read 2 Sent Done Done false
8132 2956 Acked Read 9 Sent Done Done false
8133 2957 Acked Write 3 Sent Done Done false
8134 2958 Acked Write 10 Sent Done Done false
8135 2959 Acked Read 1 Sent Done Done false
8136 2960 Acked Write 5 Sent Done Done false
8137 2961 Acked Flush 0 Sent Done Done false
8138 2962 Acked Write 1 Sent Done Done false
8139 2963 Acked Write 9 Sent Done Done false
8140 2964 Acked Write 5 Sent Done Done false
8141 2965 Acked Read 6 Sent Done Done false
8142 2966 Acked Read 10 Sent Done Done false
8143 2967 Acked Read 8 Sent Done Done false
8144 2968 Acked Read 7 Sent Done Done false
8145 2969 Acked Read 8 Sent Done Done false
8146 2970 Acked Write 5 Sent Done Done false
8147 2971 Acked Flush 0 Sent Done Done false
8148 2972 Acked Read 2 Sent Done Done false
8149 2973 Acked Read 2 Sent Done Done false
8150 2974 Acked Read 6 Sent Done Done false
8151 2975 Acked Write 4 Sent Done Done false
8152 2976 Acked Write 10 Sent Done Done false
8153 2977 Acked Read 9 Sent Done Done false
8154 2978 Acked Write 7 Sent Done Done false
8155 2979 Acked Read 6 Sent Done Done false
8156 2980 Acked Write 6 Sent Done Done false
8157 2981 Acked Write 7 Sent Done Done false
8158 2982 Acked Read 5 Sent Done Done false
8159 2983 Acked Read 8 Sent Done Done false
8160 2984 Acked Read 10 Sent Done Done false
8161 2985 Acked Write 10 Sent Done Done false
8162 2986 Acked Write 10 Sent Done Done false
8163 2987 Acked Write 9 Sent Done Done false
8164 2988 Acked Write 8 Sent Done Done false
8165 2989 Acked Write 9 Sent Done Done false
8166 2990 Acked Flush 0 Sent Done Done false
8167 2991 Acked Write 2 Sent Done Done false
8168 2992 Acked Read 1 Sent Done Done false
8169 2993 Acked Write 4 Sent Done Done false
8170 2994 Acked Read 5 Sent Done Done false
8171 2995 Acked Read 6 Sent Done Done false
8172 2996 Acked Read 3 Sent Done Done false
8173 2997 Acked Read 3 Sent Done Done false
8174 2998 Acked Write 7 Sent Done Done false
8175 2999 Acked Write 4 Sent Done Done false
8176 3000 Acked Read 7 Sent Done Done false
8177 3001 Acked Read 10 Sent Done Done false
8178 3002 Acked Read 10 Sent Done Done false
8179 3003 Acked Write 4 Sent Done Done false
8180 3004 Acked Read 10 Sent Done Done false
8181 3005 Acked Write 3 Sent Done Done false
8182 3006 Acked Read 10 Sent Done Done false
8183 3007 Acked Write 2 Sent Done Done false
8184 3008 Acked Write 3 Sent Done Done false
8185 3009 Acked Read 4 Sent Done Done false
8186 3010 Acked Write 5 Sent Done Done false
8187 3011 Acked Read 8 Sent Done Done false
8188 3012 Acked Read 9 Sent Done Done false
8189 3013 Acked Read 8 Sent Done Done false
8190 3014 Acked Write 7 Sent Done Done false
8191 3015 Acked Flush 0 Sent Done Done false
8192 3016 Acked Read 6 Sent Done Done false
8193 3017 Acked Write 8 Sent Done Done false
8194 3018 Acked Read 9 Sent Done Done false
8195 3019 Acked Write 1 Sent Done Done false
8196 3020 Acked Read 9 Sent Done Done false
8197 3021 Acked Read 8 Sent Done Done false
8198 3022 Acked Write 8 Sent Done Done false
8199 3023 Acked Read 4 Sent Done Done false
8200 3024 Acked Read 2 Sent Done Done false
8201 3025 Acked Write 5 Sent Done Done false
8202 3026 Acked Write 2 Sent Done Done false
8203 3027 Acked Write 3 Sent Done Done false
8204 3028 Acked Read 2 Sent Done Done false
8205 3029 Acked Write 2 Sent Done Done false
8206 3030 Acked Write 3 Sent Done Done false
8207 3031 Acked Write 3 Sent Done Done false
8208 3032 Acked Read 2 Sent Done Done false
8209 3033 Acked Write 10 Sent Done Done false
8210 3034 Acked Write 10 Sent Done Done false
8211 3035 Acked Write 4 Sent Done Done false
8212 3036 Acked Read 2 Sent Done Done false
8213 3037 Acked Write 10 Sent Done Done false
8214 3038 Acked Read 4 Sent Done Done false
8215 3039 Acked Write 5 Sent Done Done false
8216 3040 Acked Write 3 Sent Done Done false
8217 3041 Acked Read 6 Sent Done Done false
8218 3042 Acked Write 5 Sent Done Done false
8219 3043 Acked Write 9 Sent Done Done false
8220 3044 Acked Read 2 Sent Done Done false
8221 3045 Acked Write 2 Sent Done Done false
8222 3046 Acked Read 9 Sent Done Done false
8223 3047 Acked Write 3 Sent Done Done false
8224 3048 Acked Write 6 Sent Done Done false
8225 3049 Acked Write 2 Sent Done Done false
8226 3050 Acked Read 7 Sent Done Done false
8227 3051 Acked Write 10 Sent Done Done false
8228 3052 Acked Write 1 Sent Done Done false
8229 3053 Acked Write 9 Sent Done Done false
8230 3054 Acked Read 8 Sent Done Done false
8231 3055 Acked Read 8 Sent Done Done false
8232 3056 Acked Read 8 Sent Done Done false
8233 3057 Acked Flush 0 Sent Done Done false
8234 3058 Acked Flush 0 Sent Done Done false
8235 3059 Acked Write 5 Sent Done Done false
8236 3060 Acked Read 9 Sent Done Done false
8237 3061 Acked Flush 0 Sent Done Done false
8238 3062 Acked Read 1 Sent Done Done false
8239 3063 Acked Read 7 Sent Done Done false
8240 3064 Acked Write 1 Sent Done Done false
8241 3065 Acked Read 9 Sent Done Done false
8242 3066 Acked Read 2 Sent Done Done false
8243 3067 Acked Flush 0 Sent Done Done false
8244 3068 Acked Read 10 Sent Done Done false
8245 3069 Acked Write 3 Sent Done Done false
8246 3070 Acked Read 8 Sent Done Done false
8247 3071 Acked Write 10 Sent Done Done false
8248 3072 Acked Read 5 Sent Done Done false
8249 3073 Acked Read 2 Sent Done Done false
8250 3074 Acked Write 6 Sent Done Done false
8251 3075 Acked Read 9 Sent Done Done false
8252 3076 Acked Write 7 Sent Done Done false
8253 3077 Acked Flush 0 Sent Done Done false
8254 3078 Acked Read 1 Sent Done Done false
8255 3079 Acked Read 8 Sent Done Done false
8256 3080 Acked Write 10 Sent Done Done false
8257 3081 Acked Write 3 Sent Done Done false
8258 3082 Acked Write 4 Sent Done Done false
8259 3083 Acked Read 9 Sent Done Done false
8260 3084 Acked Write 2 Sent Done Done false
8261 3085 Acked Write 9 Sent Done Done false
8262 3086 Acked Read 9 Sent Done Done false
8263 3087 Acked Read 5 Sent Done Done false
8264 3088 Acked Write 4 Sent Done Done false
8265 3089 Acked Write 6 Sent Done Done false
8266 3090 Acked Flush 0 Sent Done Done false
8267 3091 Acked Flush 0 Sent Done Done false
8268 3092 Acked Read 5 Sent Done Done false
8269 3093 Acked Read 7 Sent Done Done false
8270 3094 Acked Read 1 Sent Done Done false
8271 3095 Acked Write 9 Sent Done Done false
8272 3096 Acked Read 5 Sent Done Done false
8273 3097 Acked Write 8 Sent Done Done false
8274 3098 Acked Read 6 Sent Done Done false
8275 3099 Acked Read 2 Sent Done Done false
8276 3100 Acked Read 10 Sent Done Done false
8277 3101 Acked Write 10 Sent Done Done false
8278 3102 Acked Write 2 Sent Done Done false
8279 3103 Acked Write 1 Sent Done Done false
8280 3104 Acked Read 2 Sent Done Done false
8281 3105 Acked Write 1 Sent Done Done false
8282 3106 Acked Write 1 Sent Done Done false
8283 3107 Acked Read 8 Sent Done Done false
8284 3108 Acked Write 2 Sent Done Done false
8285 3109 Acked Read 9 Sent Done Done false
8286 3110 Acked Read 4 Sent Done Done false
8287 3111 Acked Write 8 Sent Done Done false
8288 3112 Acked Read 5 Sent Done Done false
8289 3113 Acked Flush 0 Sent Done Done false
8290 STATES DS:0 DS:1 DS:2 TOTAL
8291 Sent 302 0 0 302
8292 Done 0 302 302 604
8293 Skipped 0 0 0 0
8294 Error 0 0 0 0
8295 Last Flush: 2811 3113 3113
8296 Downstairs last five completed: 2811 2810 2809 2808 2807
8297 Upstairs last five completed: 3113 3112 3111 3110 3109
8298 CLIENT: Up:0 ds:302 act:2
82992024-11-26T22:06:15.867ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
83002024-11-26T22:06:15.867ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:58113) = io task client = 0 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
83012024-11-26T22:06:15.868ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
83022024-11-26T22:06:15.868ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
83032024-11-26T22:06:15.868ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
83042024-11-26T22:06:15.868ZINFOcrucible: send last flush ID to this DS: 2811 = downstairs client = 0 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
83052024-11-26T22:06:15.868ZINFOcrucible: Replied this last flush ID: 2811 = downstairs client = 0 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
83062024-11-26T22:06:15.869ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
83072024-11-26T22:06:15.869ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
83082024-11-26T22:06:15.869ZINFOcrucible: [0] client re-new 302 jobs since flush 2811 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
83092024-11-26T22:06:15.869ZINFOcrucible: [0] Replayed 302 jobs since flush: 2811 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
8310 ----------------------------------------------------------------
8311 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
8312 Downstairs last five completed: 3113 3112 3111 3110 3109
8313 Upstairs last five completed: 3113 3112 3111 3110 3109
8314 CLIENT: Up:0 ds:0 act:3
8315 Replay: All jobs finished
8316 Main task has work to do, go find it
8317 got dsc Stop(0)
8318 stop 0
8319 Nov 26 22:06:18.109 INFO request completed, latency_us: 310, response_code: [8810] Got stop action so:false kr:false
8320 204, uri: /stop/cid/0, method: POST, req_id: b759721a-ed6e-4125-a258-b321d2791459, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
83212024-11-26T22:06:18.110ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
83222024-11-26T22:06:18.110ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
8323 Nov 26 22:06:18.110 INFO request completed, latency_us: 117, response_code: 200{"msg":"downstairs task for 0 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2024-11-26T22:06:18.110421789Z","hostname":"w-01JDN7ZQ17ABQG15KHKN3F2D26","pid":1203,"session_id":"86a36c17-3718-484c-87fd-2b8a1dc2ab18"}
8324 , uri: /state/cid/0, method: GET, req_id: 07df56bb-cbbf-4675-ad78-a5778e72bb1c, remote_addr: 127.0.0.{"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-26T22:06:18.110445726Z","hostname":"w-01JDN7ZQ17ABQG15KHKN3F2D26","pid":1203,"client":"0","":"downstairs","session_id":"86a36c17-3718-484c-87fd-2b8a1dc2ab18"}
8325 1:36829, local_addr: 127.0.0.1:9998{"msg":"[0] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-26T22:06:18.110481122Z","hostname":"w-01JDN7ZQ17ABQG15KHKN3F2D26","pid":1203,"client":"0","":"downstairs","session_id":"86a36c17-3718-484c-87fd-2b8a1dc2ab18"}
8326 
83272024-11-26T22:06:18.110ZINFOcrucible: client task is exiting = io task client = 0 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
8328 [8810] Exited with: ExitStatus(unix_wait_status(9))
8329 [8810][0] reports Exit
8330 Nov 26 22:06:22.112 INFO request completed, latency_us: 215, response_code: 200, uri: /state/cid/0, method: GET, req_id: 2bc84bc8-784b-44df-a846-48721c7737ea, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
8331 001/300 Write block 3500 len 28672 data: 2 2 3 3 3 3 3
8332 002/300 Write block 3341 len 12288 data: 3 3 3
8333 003/300 Write block 4060 len 24576 data: 4 3 4 4 4 3
8334 004/300 Read block 2450 len 36864
8335 005/300 Read block 1793 len 32768
8336 006/300 Read block 681 len 8192
8337 007/300 Flush
8338 008/300 Write block 740 len 12288 data: 3 4 5
8339 009/300 Write block 349 len 28672 data: 5 4 4 4 4 4 4
8340 010/300 Read block 211 len 24576
8341 011/300 Flush
8342 012/300 Read block 3155 len 8192
8343 013/300 Read block 433 len 24576
8344 014/300 Read block 2759 len 12288
8345 015/300 Write block 4195 len 40960 data: 4 4 5 5 4 3 2 2 2 2
8346 016/300 Write block 2022 len 20480 data: 2 2 3 4 4
8347 017/300 Read block 3707 len 32768
8348 018/300 Read block 2440 len 28672
8349 019/300 Write block 1100 len 16384 data: 2 2 2 2
8350 020/300 Read block 882 len 36864
8351 021/300 Write block 2009 len 8192 data: 2 2
8352 022/300 Write block 1870 len 32768 data: 4 3 3 3 3 4 3 2
8353 023/300 Write block 1357 len 12288 data: 4 4 3
8354 024/300 Write block 2162 len 36864 data: 5 4 4 6 6 5 5 5 4
8355 025/300 Read block 888 len 20480
8356 026/300 Write block 4582 len 8192 data: 2 3
8357 027/300 Write block 1168 len 8192 data: 4 3
8358 028/300 Write block 2813 len 8192 data: 3 2
8359 029/300 Write block 1358 len 8192 data: 5 4
8360 030/300 Read block 4606 len 40960
8361 031/300 Flush
8362 032/300 Read block 1992 len 28672
8363 033/300 Read block 374 len 36864
8364 034/300 Write block 2995 len 36864 data: 3 3 3 4 5 4 3 2 3
8365 035/300 Write block 448 len 12288 data: 3 3 3
8366 036/300 Write block 3293 len 8192 data: 2 2
8367 037/300 Read block 3205 len 20480
8368 038/300 Read block 2286 len 36864
8369 039/300 Read block 174 len 16384
8370 040/300 Read block 3479 len 20480
8371 041/300 Write block 4003 len 28672 data: 4 4 4 4 4 4 4
8372 042/300 Write block 4891 len 16384 data: 2 2 3 3
8373 043/300 Write block 3142 len 16384 data: 4 4 4 5
8374 044/300 Read block 4570 len 12288
8375 045/300 Read block 4397 len 32768
8376 046/300 Write block 328 len 20480 data: 2 2 2 2 2
8377 047/300 Flush
8378 048/300 Read block 2118 len 4096
8379 049/300 Read block 1377 len 32768
8380 050/300 Read block 2878 len 28672
8381 051/300 Write block 1492 len 16384 data: 3 3 3 2
8382 052/300 Read block 534 len 28672
8383 053/300 Write block 2026 len 36864 data: 5 3 3 2 2 2 3 3 4
8384 054/300 Write block 3258 len 20480 data: 3 3 3 3 3
8385 055/300 Read block 1656 len 40960
8386 056/300 Write block 2730 len 20480 data: 3 3 2 2 3
8387 057/300 Write block 3312 len 40960 data: 3 2 3 4 4 3 3 3 3 3
8388 058/300 Flush
8389 059/300 Write block 2143 len 12288 data: 2 2 3
8390 060/300 Write block 1255 len 36864 data: 2 2 2 2 2 2 2 2 2
8391 061/300 Write block 4443 len 24576 data: 4 4 3 3 3 4
8392 062/300 Read block 3699 len 16384
8393 063/300 Read block 2488 len 4096
8394 064/300 Read block 2564 len 36864
8395 065/300 Read block 3904 len 12288
8396 066/300 Write block 3317 len 24576 data: 4 4 4 4 4 3
8397 067/300 Write block 2604 len 16384 data: 3 3 3 4
8398 068/300 Read block 2893 len 20480
8399 069/300 Write block 3018 len 32768 data: 4 2 2 3 3 3 3 3
8400 070/300 Read block 2011 len 24576
8401 071/300 Read block 1045 len 40960
8402 072/300 Read block 4211 len 36864
8403 073/300 Write block 1958 len 4096 data: 4
8404 074/300 Read block 1575 len 40960
8405 075/300 Read block 3727 len 28672
8406 076/300 Read block 350 len 24576
8407 077/300 Read block 708 len 4096
8408 078/300 Write block 2963 len 24576 data: 2 2 2 2 2 2
8409 079/300 Read block 1976 len 28672
8410 080/300 Read block 2031 len 40960
8411 081/300 Write block 3947 len 16384 data: 5 5 4 4
8412 082/300 Read block 2762 len 24576
8413 083/300 Read block 4269 len 20480
8414 084/300 Read block 4543 len 24576
8415 085/300 Write block 2670 len 28672 data: 3 3 2 2 2 2 2
8416 086/300 Write block 3219 len 12288 data: 3 3 2
8417 087/300 Write block 2348 len 36864 data: 2 2 2 3 3 3 3 3 3
8418 088/300 Write block 4469 len 28672 data: 3 3 2 2 3 3 4
8419 089/300 Flush
8420 090/300 Read block 3335 len 12288
8421 091/300 Read block 1789 len 16384
8422 092/300 Write block 1325 len 12288 data: 2 2 2
8423 093/300 Write block 2692 len 16384 data: 2 2 2 2
8424 094/300 Flush
8425 095/300 Read block 2911 len 20480
8426 096/300 Write block 1087 len 4096 data: 4
8427 097/300 Write block 2401 len 20480 data: 3 3 3 3 3
8428 098/300 Read block 1507 len 24576
8429 099/300 Write block 3418 len 12288 data: 3 3 2
8430 100/300 Write block 2366 len 4096 data: 2
8431 101/300 Read block 2634 len 32768
8432 102/300 Write block 3093 len 12288 data: 2 2 2
8433 103/300 Flush
8434 104/300 Read block 4964 len 24576
8435 105/300 Write block 4507 len 28672 data: 2 2 2 2 2 2 3
8436 106/300 Write block 3346 len 32768 data: 3 3 3 3 3 3 2 2
8437 107/300 Write block 2285 len 12288 data: 3 3 3
8438 108/300 Read block 4286 len 12288
8439 109/300 Write block 1787 len 8192 data: 2 2
8440 110/300 Flush
8441 111/300 Read block 419 len 20480
8442 112/300 Read block 3891 len 20480
8443 113/300 Read block 853 len 16384
8444 114/300 Write block 3186 len 8192 data: 2 2
8445 115/300 Read block 4799 len 40960
8446 116/300 Read block 234 len 28672
8447 117/300 Write block 4768 len 40960 data: 3 3 3 2 2 2 2 2 2 2
8448 118/300 Read block 3697 len 16384
8449 119/300 Write block 2895 len 32768 data: 3 3 3 3 4 3 4 4
8450 120/300 Read block 2138 len 16384
8451 121/300 Write block 853 len 12288 data: 2 2 2
8452 122/300 Read block 159 len 12288
8453 123/300 Write block 4465 len 20480 data: 3 3 4 4 4
8454 124/300 Write block 377 len 12288 data: 3 3 4
8455 125/300 Read block 2311 len 20480
8456 126/300 Write block 1241 len 40960 data: 2 2 2 2 2 2 2 2 2 3
8457 127/300 Write block 1892 len 24576 data: 4 3 2 3 3 3
8458 128/300 Write block 572 len 16384 data: 4 3 3 4
8459 129/300 Write block 3361 len 32768 data: 3 2 2 2 2 2 2 2
8460 130/300 Write block 4370 len 32768 data: 3 3 4 4 4 2 2 2
8461 131/300 Flush
8462 132/300 Read block 302 len 20480
8463 133/300 Write block 1528 len 32768 data: 6 6 5 4 4 4 4 4
8464 134/300 Flush
8465 135/300 Read block 372 len 4096
8466 136/300 Read block 3211 len 8192
8467 137/300 Flush
8468 138/300 Flush
8469 139/300 Write block 4312 len 12288 data: 2 4 4
8470 140/300 Read block 4046 len 12288
8471 141/300 Write block 1719 len 36864 data: 3 2 3 3 3 3 3 3 3
8472 142/300 Read block 4953 len 20480
8473 143/300 Write block 4975 len 36864 data: 2 2 3 3 3 4 4 4 4
8474 144/300 Read block 2970 len 12288
8475 145/300 Read block 2338 len 40960
8476 146/300 Write block 97 len 4096 data: 5
8477 147/300 Read block 1138 len 12288
8478 148/300 Write block 3546 len 8192 data: 3 2
8479 149/300 Read block 435 len 4096
8480 150/300 Flush
8481 151/300 Read block 3416 len 40960
8482 152/300 Read block 4841 len 16384
8483 153/300 Write block 186 len 36864 data: 3 3 4 3 4 4 4 4 4
8484 154/300 Write block 4371 len 4096 data: 4
8485 155/300 Write block 2808 len 28672 data: 3 3 3 4 3 4 3
8486 156/300 Write block 3028 len 28672 data: 2 2 2 2 2 2 3
8487 157/300 Write block 794 len 4096 data: 2
8488 158/300 Read block 4245 len 8192
8489 159/300 Write block 3413 len 20480 data: 2 2 3 3 3
8490 160/300 Write block 2798 len 24576 data: 3 2 3 2 3 2
8491 161/300 Read block 1045 len 28672
8492 162/300 Flush
8493 163/300 Write block 924 len 24576 data: 2 3 4 3 3 5
8494 164/300 Flush
8495 165/300 Read block 126 len 20480
8496 166/300 Write block 3066 len 16384 data: 3 3 3 3
8497 167/300 Read block 35 len 24576
8498 168/300 Read block 815 len 4096
8499 169/300 Write block 2034 len 24576 data: 5 4 4 4 4 4
8500 170/300 Write block 1472 len 32768 data: 4 4 4 4 4 4 5 5
8501 171/300 Read block 1210 len 36864
8502 172/300 Read block 971 len 32768
8503 173/300 Write block 1652 len 36864 data: 2 3 3 3 3 3 4 4 4
8504 174/300 Write block 2487 len 24576 data: 4 4 3 2 3 3
8505 175/300 Read block 4480 len 28672
8506 176/300 Read block 2047 len 36864
8507 177/300 Read block 218 len 8192
8508 178/300 Read block 130 len 8192
8509 179/300 Write block 3564 len 20480 data: 3 3 3 3 3
8510 180/300 Write block 2054 len 16384 data: 3 2 2 3
8511 181/300 Write block 1284 len 36864 data: 4 4 3 3 3 3 3 2 2
8512 182/300 Read block 4323 len 40960
8513 183/300 Read block 3652 len 24576
8514 184/300 Write block 3904 len 16384 data: 2 2 2 3
8515 185/300 Read block 3523 len 36864
8516 186/300 Read block 2294 len 4096
8517 187/300 Write block 3261 len 32768 data: 4 4 3 2 2 2 2 2
8518 188/300 Read block 3559 len 40960
8519 189/300 Read block 3720 len 4096
8520 190/300 Read block 323 len 32768
8521 191/300 Read block 4196 len 20480
8522 192/300 Read block 4294 len 16384
8523 193/300 Write block 1037 len 8192 data: 3 3
8524 194/300 Write block 254 len 16384 data: 2 2 2 2
8525 195/300 Write block 3941 len 20480 data: 4 3 3 5 4
8526 196/300 Flush
8527 197/300 Read block 128 len 40960
8528 198/300 Flush
8529 199/300 Write block 4110 len 24576 data: 4 4 4 3 3 3
8530 200/300 Read block 3526 len 16384
8531 201/300 Write block 4033 len 8192 data: 2 2
8532 202/300 Write block 2239 len 8192 data: 3 3
8533 203/300 Flush
8534 204/300 Read block 2688 len 16384
8535 205/300 Write block 2603 len 36864 data: 3 4 4 4 5 4 3 3 4
8536 206/300 Flush
8537 207/300 Read block 2966 len 4096
8538 208/300 Write block 674 len 36864 data: 4 4 3 3 3 3 3 2 2
8539 209/300 Write block 2329 len 36864 data: 2 3 2 2 2 2 2 2 2
8540 210/300 Flush
8541 211/300 Read block 1642 len 16384
8542 212/300 Flush
8543 213/300 Write block 1061 len 16384 data: 2 2 2 2
8544 214/300 Read block 4823 len 32768
8545 215/300 Read block 2879 len 12288
8546 216/300 Read block 4401 len 12288
8547 217/300 Read block 4540 len 12288
8548 218/300 Flush
8549 219/300 Read block 1733 len 28672
8550 220/300 Write block 4065 len 36864 data: 4 4 4 5 4 4 3 4 3
8551 221/300 Read block 15 len 36864
8552 222/300 Read block 2862 len 16384
8553 223/300 Write block 1291 len 32768 data: 3 3 2 2 2 2 2 2
8554 224/300 Read block 1638 len 20480
8555 225/300 Read block 305 len 8192
8556 226/300 Read block 1176 len 16384
8557 227/300 Read block 1392 len 12288
8558 228/300 Read block 4350 len 24576
8559 229/300 Flush
8560 230/300 Read block 2984 len 32768
8561 231/300 Read block 3959 len 32768
8562 232/300 Write block 2459 len 4096 data: 3
8563 233/300 Read block 3454 len 12288
8564 234/300 Write block 4294 len 8192 data: 2 2
8565 235/300 Write block 4077 len 24576 data: 2 2 2 2 2 2
8566 236/300 Read block 599 len 36864
8567 237/300 Flush
8568 238/300 Read block 3608 len 36864
8569 239/300 Read block 2663 len 16384
8570 240/300 Read block 307 len 16384
8571 241/300 Read block 4121 len 36864
8572 242/300 Write block 4617 len 12288 data: 2 3 4
8573 243/300 Write block 82 len 32768 data: 2 2 2 2 2 2 2 2
8574 244/300 Write block 4988 len 20480 data: 3 4 5 5 5
8575 245/300 Write block 2338 len 32768 data: 2 2 3 3 3 3 3 3
8576 246/300 Write block 4750 len 32768 data: 3 2 2 2 2 2 2 2
8577 247/300 Write block 4359 len 8192 data: 3 4
8578 248/300 Read block 4407 len 4096
8579 249/300 Write block 2986 len 24576 data: 4 3 3 3 3 2
8580 250/300 Flush
8581 251/300 Write block 832 len 20480 data: 2 2 3 2 2
8582 252/300 Read block 3626 len 36864
8583 253/300 Read block 2458 len 28672
8584 254/300 Read block 3769 len 12288
8585 255/300 Read block 379 len 8192
8586 256/300 Read block 3214 len 40960
8587 257/300 Write block 3710 len 4096 data: 3
8588 258/300 Read block 1225 len 36864
8589 259/300 Write block 3184 len 8192 data: 3 2
8590 260/300 Write block 909 len 24576 data: 3 5 5 5 5 5
8591 261/300 Write block 3453 len 24576 data: 3 2 2 2 2 2
8592 262/300 Read block 1994 len 24576
8593 263/300 Write block 808 len 8192 data: 2 2
8594 264/300 Write block 3486 len 4096 data: 2
8595 265/300 Read block 3436 len 16384
8596 266/300 Read block 2368 len 4096
8597 267/300 Read block 2230 len 40960
8598 268/300 Read block 4692 len 24576
8599 269/300 Read block 1311 len 8192
8600 270/300 Read block 3054 len 12288
8601 271/300 Read block 153 len 24576
8602 272/300 Write block 4073 len 36864 data: 4 3 3 2 3 3 3 3 3
8603 273/300 Write block 1552 len 32768 data: 3 4 4 4 4 3 3 2
8604 274/300 Write block 4605 len 24576 data: 5 3 3 4 4 4
8605 275/300 Write block 4158 len 32768 data: 2 3 3 3 3 3 3 3
8606 276/300 Flush
8607 277/300 Write block 2925 len 24576 data: 3 2 2 2 2 2
8608 278/300 Write block 2958 len 8192 data: 2 2
8609 279/300 Read block 453 len 8192
8610 280/300 Write block 3229 len 4096 data: 6
8611 281/300 Write block 374 len 4096 data: 3
8612 282/300 Write block 1362 len 24576 data: 2 2 3 3 3 3
8613 283/300 Read block 707 len 20480
8614 284/300 Read block 977 len 28672
8615 285/300 Write block 3160 len 24576 data: 4 4 3 3 3 3
8616 286/300 Read block 2502 len 40960
8617 287/300 Write block 4143 len 16384 data: 3 3 2 2
8618 288/300 Read block 2507 len 4096
8619 289/300 Write block 2484 len 40960 data: 6 6 6 5 5 4 3 4 4 3
8620 290/300 Read block 52 len 32768
8621 291/300 Read block 3783 len 12288
8622 292/300 Read block 2798 len 8192
8623 293/300 Write block 441 len 20480 data: 3 3 3 2 2
8624 294/300 Write block 1282 len 20480 data: 4 4 5 5 4
8625 295/300 Read block 3548 len 4096
8626 296/300 Flush
8627 297/300 Write block 3950 len 12288 data: 5 4 3
8628 298/300 Write block 1405 len 12288 data: 2 2 2
8629 299/300 Flush
8630 300/300 Write block 1753 len 20480 data: 2 2 2 2 3
8631 301/300 Read block 3944 len 16384
8632 Main task has work to do, go find it
8633 got dsc Start(0)
8634 start 0
8635 Nov 26 22:06:22.338 INFO request completed, latency_us: 191, response_code: 204, uri: /start/cid/0, method: POST, req_id: 895622a2-e789-4094-8984-f97260ddfe3a, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
8636 Replay: started 0, returned:Ok(())
8637 ----------------------------------------------------------------
8638 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
8639 States: Offline Active Active
8640 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
8641 3114 Acked Write 7 Sent Done Done false
8642 3115 Acked Write 3 Sent Done Done false
8643 3116 Acked Write 6 Sent Done Done false
8644 3117 Acked Read 9 Sent Done Done false
8645 3118 Acked Read 8 Sent Done Done false
8646 3119 Acked Read 2 Sent Done Done false
8647 3120 Acked Flush 0 Sent Done Done false
8648 3121 Acked Write 3 Sent Done Done false
8649 3122 Acked Write 7 Sent Done Done false
8650 3123 Acked Read 6 Sent Done Done false
8651 3124 Acked Flush 0 Sent Done Done false
8652 3125 Acked Read 2 Sent Done Done false
8653 3126 Acked Read 6 Sent Done Done false
8654 3127 Acked Read 3 Sent Done Done false
8655 3128 Acked Write 10 Sent Done Done false
8656 3129 Acked Write 5 Sent Done Done false
8657 3130 Acked Read 8 Sent Done Done false
8658 3131 Acked Read 7 Sent Done Done false
8659 3132 Acked Write 4 Sent Done Done false
8660 3133 Acked Read 9 Sent Done Done false
8661 3134 Acked Write 2 Sent Done Done false
8662 3135 Acked Write 8 Sent Done Done false
8663 3136 Acked Write 3 Sent Done Done false
8664 3137 Acked Write 9 Sent Done Done false
8665 3138 Acked Read 5 Sent Done Done false
8666 3139 Acked Write 2 Sent Done Done false
8667 3140 Acked Write 2 Sent Done Done false
8668 3141 Acked Write 2 Sent Done Done false
8669 3142 Acked Write 2 Sent Done Done false
8670 3143 Acked Read 10 Sent Done Done false
8671 3144 Acked Flush 0 Sent Done Done false
8672 3145 Acked Read 7 Sent Done Done false
8673 3146 Acked Read 9 Sent Done Done false
8674 3147 Acked Write 9 Sent Done Done false
8675 3148 Acked Write 3 Sent Done Done false
8676 3149 Acked Write 2 Sent Done Done false
8677 3150 Acked Read 5 Sent Done Done false
8678 3151 Acked Read 9 Sent Done Done false
8679 3152 Acked Read 4 Sent Done Done false
8680 3153 Acked Read 5 Sent Done Done false
8681 3154 Acked Write 7 Sent Done Done false
8682 3155 Acked Write 4 Sent Done Done false
8683 3156 Acked Write 4 Sent Done Done false
8684 3157 Acked Read 3 Sent Done Done false
8685 3158 Acked Read 8 Sent Done Done false
8686 3159 Acked Write 5 Sent Done Done false
8687 3160 Acked Flush 0 Sent Done Done false
8688 3161 Acked Read 1 Sent Done Done false
8689 3162 Acked Read 8 Sent Done Done false
8690 3163 Acked Read 7 Sent Done Done false
8691 3164 Acked Write 4 Sent Done Done false
8692 3165 Acked Read 7 Sent Done Done false
8693 3166 Acked Write 9 Sent Done Done false
8694 3167 Acked Write 5 Sent Done Done false
8695 3168 Acked Read 10 Sent Done Done false
8696 3169 Acked Write 5 Sent Done Done false
8697 3170 Acked Write 10 Sent Done Done false
8698 3171 Acked Flush 0 Sent Done Done false
8699 3172 Acked Write 3 Sent Done Done false
8700 3173 Acked Write 9 Sent Done Done false
8701 3174 Acked Write 6 Sent Done Done false
8702 3175 Acked Read 4 Sent Done Done false
8703 3176 Acked Read 1 Sent Done Done false
8704 3177 Acked Read 9 Sent Done Done false
8705 3178 Acked Read 3 Sent Done Done false
8706 3179 Acked Write 6 Sent Done Done false
8707 3180 Acked Write 4 Sent Done Done false
8708 3181 Acked Read 5 Sent Done Done false
8709 3182 Acked Write 8 Sent Done Done false
8710 3183 Acked Read 6 Sent Done Done false
8711 3184 Acked Read 10 Sent Done Done false
8712 3185 Acked Read 9 Sent Done Done false
8713 3186 Acked Write 1 Sent Done Done false
8714 3187 Acked Read 10 Sent Done Done false
8715 3188 Acked Read 7 Sent Done Done false
8716 3189 Acked Read 6 Sent Done Done false
8717 3190 Acked Read 1 Sent Done Done false
8718 3191 Acked Write 6 Sent Done Done false
8719 3192 Acked Read 7 Sent Done Done false
8720 3193 Acked Read 10 Sent Done Done false
8721 3194 Acked Write 4 Sent Done Done false
8722 3195 Acked Read 6 Sent Done Done false
8723 3196 Acked Read 5 Sent Done Done false
8724 3197 Acked Read 6 Sent Done Done false
8725 3198 Acked Write 7 Sent Done Done false
8726 3199 Acked Write 3 Sent Done Done false
8727 3200 Acked Write 9 Sent Done Done false
8728 3201 Acked Write 7 Sent Done Done false
8729 3202 Acked Flush 0 Sent Done Done false
8730 3203 Acked Read 3 Sent Done Done false
8731 3204 Acked Read 4 Sent Done Done false
8732 3205 Acked Write 3 Sent Done Done false
8733 3206 Acked Write 4 Sent Done Done false
8734 3207 Acked Flush 0 Sent Done Done false
8735 3208 Acked Read 5 Sent Done Done false
8736 3209 Acked Write 1 Sent Done Done false
8737 3210 Acked Write 5 Sent Done Done false
8738 3211 Acked Read 6 Sent Done Done false
8739 3212 Acked Write 3 Sent Done Done false
8740 3213 Acked Write 1 Sent Done Done false
8741 3214 Acked Read 8 Sent Done Done false
8742 3215 Acked Write 3 Sent Done Done false
8743 3216 Acked Flush 0 Sent Done Done false
8744 3217 Acked Read 6 Sent Done Done false
8745 3218 Acked Write 7 Sent Done Done false
8746 3219 Acked Write 8 Sent Done Done false
8747 3220 Acked Write 3 Sent Done Done false
8748 3221 Acked Read 3 Sent Done Done false
8749 3222 Acked Write 2 Sent Done Done false
8750 3223 Acked Flush 0 Sent Done Done false
8751 3224 Acked Read 5 Sent Done Done false
8752 3225 Acked Read 5 Sent Done Done false
8753 3226 Acked Read 4 Sent Done Done false
8754 3227 Acked Write 2 Sent Done Done false
8755 3228 Acked Read 10 Sent Done Done false
8756 3229 Acked Read 7 Sent Done Done false
8757 3230 Acked Write 10 Sent Done Done false
8758 3231 Acked Read 4 Sent Done Done false
8759 3232 Acked Write 8 Sent Done Done false
8760 3233 Acked Read 4 Sent Done Done false
8761 3234 Acked Write 3 Sent Done Done false
8762 3235 Acked Read 3 Sent Done Done false
8763 3236 Acked Write 5 Sent Done Done false
8764 3237 Acked Write 3 Sent Done Done false
8765 3238 Acked Read 5 Sent Done Done false
8766 3239 Acked Write 10 Sent Done Done false
8767 3240 Acked Write 6 Sent Done Done false
8768 3241 Acked Write 4 Sent Done Done false
8769 3242 Acked Write 8 Sent Done Done false
8770 3243 Acked Write 8 Sent Done Done false
8771 3244 Acked Flush 0 Sent Done Done false
8772 3245 Acked Read 5 Sent Done Done false
8773 3246 Acked Write 8 Sent Done Done false
8774 3247 Acked Flush 0 Sent Done Done false
8775 3248 Acked Read 1 Sent Done Done false
8776 3249 Acked Read 2 Sent Done Done false
8777 3250 Acked Flush 0 Sent Done Done false
8778 3251 Acked Flush 0 Sent Done Done false
8779 3252 Acked Write 3 Sent Done Done false
8780 3253 Acked Read 3 Sent Done Done false
8781 3254 Acked Write 9 Sent Done Done false
8782 3255 Acked Read 5 Sent Done Done false
8783 3256 Acked Write 9 Sent Done Done false
8784 3257 Acked Read 3 Sent Done Done false
8785 3258 Acked Read 10 Sent Done Done false
8786 3259 Acked Write 1 Sent Done Done false
8787 3260 Acked Read 3 Sent Done Done false
8788 3261 Acked Write 2 Sent Done Done false
8789 3262 Acked Read 1 Sent Done Done false
8790 3263 Acked Flush 0 Sent Done Done false
8791 3264 Acked Read 10 Sent Done Done false
8792 3265 Acked Read 4 Sent Done Done false
8793 3266 Acked Write 9 Sent Done Done false
8794 3267 Acked Write 1 Sent Done Done false
8795 3268 Acked Write 7 Sent Done Done false
8796 3269 Acked Write 7 Sent Done Done false
8797 3270 Acked Write 1 Sent Done Done false
8798 3271 Acked Read 2 Sent Done Done false
8799 3272 Acked Write 5 Sent Done Done false
8800 3273 Acked Write 6 Sent Done Done false
8801 3274 Acked Read 7 Sent Done Done false
8802 3275 Acked Flush 0 Sent Done Done false
8803 3276 Acked Write 6 Sent Done Done false
8804 3277 Acked Flush 0 Sent Done Done false
8805 3278 Acked Read 5 Sent Done Done false
8806 3279 Acked Write 4 Sent Done Done false
8807 3280 Acked Read 6 Sent Done Done false
8808 3281 Acked Read 1 Sent Done Done false
8809 3282 Acked Write 6 Sent Done Done false
8810 3283 Acked Write 8 Sent Done Done false
8811 3284 Acked Read 9 Sent Done Done false
8812 3285 Acked Read 8 Sent Done Done false
8813 3286 Acked Write 9 Sent Done Done false
8814 3287 Acked Write 6 Sent Done Done false
8815 3288 Acked Read 7 Sent Done Done false
8816 3289 Acked Read 9 Sent Done Done false
8817 3290 Acked Read 2 Sent Done Done false
8818 3291 Acked Read 2 Sent Done Done false
8819 3292 Acked Write 5 Sent Done Done false
8820 3293 Acked Write 4 Sent Done Done false
8821 3294 Acked Write 9 Sent Done Done false
8822 3295 Acked Read 10 Sent Done Done false
8823 3296 Acked Read 6 Sent Done Done false
8824 3297 Acked Write 4 Sent Done Done false
8825 3298 Acked Read 9 Sent Done Done false
8826 3299 Acked Read 1 Sent Done Done false
8827 3300 Acked Write 8 Sent Done Done false
8828 3301 Acked Read 10 Sent Done Done false
8829 3302 Acked Read 1 Sent Done Done false
8830 3303 Acked Read 8 Sent Done Done false
8831 3304 Acked Read 5 Sent Done Done false
8832 3305 Acked Read 4 Sent Done Done false
8833 3306 Acked Write 2 Sent Done Done false
8834 3307 Acked Write 4 Sent Done Done false
8835 3308 Acked Write 5 Sent Done Done false
8836 3309 Acked Flush 0 Sent Done Done false
8837 3310 Acked Read 10 Sent Done Done false
8838 3311 Acked Flush 0 Sent Done Done false
8839 3312 Acked Write 6 Sent Done Done false
8840 3313 Acked Read 4 Sent Done Done false
8841 3314 Acked Write 2 Sent Done Done false
8842 3315 Acked Write 2 Sent Done Done false
8843 3316 Acked Flush 0 Sent Done Done false
8844 3317 Acked Read 4 Sent Done Done false
8845 3318 Acked Write 9 Sent Done Done false
8846 3319 Acked Flush 0 Sent Done Done false
8847 3320 Acked Read 1 Sent Done Done false
8848 3321 Acked Write 9 Sent Done Done false
8849 3322 Acked Write 9 Sent Done Done false
8850 3323 Acked Flush 0 Sent Done Done false
8851 3324 Acked Read 4 Sent Done Done false
8852 3325 Acked Flush 0 Sent Done Done false
8853 3326 Acked Write 4 Sent Done Done false
8854 3327 Acked Read 8 Sent Done Done false
8855 3328 Acked Read 3 Sent Done Done false
8856 3329 Acked Read 3 Sent Done Done false
8857 3330 Acked Read 3 Sent Done Done false
8858 3331 Acked Flush 0 Sent Done Done false
8859 3332 Acked Read 7 Sent Done Done false
8860 3333 Acked Write 9 Sent Done Done false
8861 3334 Acked Read 9 Sent Done Done false
8862 3335 Acked Read 4 Sent Done Done false
8863 3336 Acked Write 8 Sent Done Done false
8864 3337 Acked Read 5 Sent Done Done false
8865 3338 Acked Read 2 Sent Done Done false
8866 3339 Acked Read 4 Sent Done Done false
8867 3340 Acked Read 3 Sent Done Done false
8868 3341 Acked Read 6 Sent Done Done false
8869 3342 Acked Flush 0 Sent Done Done false
8870 3343 Acked Read 8 Sent Done Done false
8871 3344 Acked Read 8 Sent Done Done false
8872 3345 Acked Write 1 Sent Done Done false
8873 3346 Acked Read 3 Sent Done Done false
8874 3347 Acked Write 2 Sent Done Done false
8875 3348 Acked Write 6 Sent Done Done false
8876 3349 Acked Read 9 Sent Done Done false
8877 3350 Acked Flush 0 Sent Done Done false
8878 3351 Acked Read 9 Sent Done Done false
8879 3352 Acked Read 4 Sent Done Done false
8880 3353 Acked Read 4 Sent Done Done false
8881 3354 Acked Read 9 Sent Done Done false
8882 3355 Acked Write 3 Sent Done Done false
8883 3356 Acked Write 8 Sent Done Done false
8884 3357 Acked Write 5 Sent Done Done false
8885 3358 Acked Write 8 Sent Done Done false
8886 3359 Acked Write 8 Sent Done Done false
8887 3360 Acked Write 2 Sent Done Done false
8888 3361 Acked Read 1 Sent Done Done false
8889 3362 Acked Write 6 Sent Done Done false
8890 3363 Acked Flush 0 Sent Done Done false
8891 3364 Acked Write 5 Sent Done Done false
8892 3365 Acked Read 9 Sent Done Done false
8893 3366 Acked Read 7 Sent Done Done false
8894 3367 Acked Read 3 Sent Done Done false
8895 3368 Acked Read 2 Sent Done Done false
8896 3369 Acked Read 10 Sent Done Done false
8897 3370 Acked Write 1 Sent Done Done false
8898 3371 Acked Read 9 Sent Done Done false
8899 3372 Acked Write 2 Sent Done Done false
8900 3373 Acked Write 6 Sent Done Done false
8901 3374 Acked Write 6 Sent Done Done false
8902 3375 Acked Read 6 Sent Done Done false
8903 3376 Acked Write 2 Sent Done Done false
8904 3377 Acked Write 1 Sent Done Done false
8905 3378 Acked Read 4 Sent Done Done false
8906 3379 Acked Read 1 Sent Done Done false
8907 3380 Acked Read 10 Sent Done Done false
8908 3381 Acked Read 6 Sent Done Done false
8909 3382 Acked Read 2 Sent Done Done false
8910 3383 Acked Read 3 Sent Done Done false
8911 3384 Acked Read 6 Sent Done Done false
8912 3385 Acked Write 9 Sent Done Done false
8913 3386 Acked Write 8 Sent Done Done false
8914 3387 Acked Write 6 Sent Done Done false
8915 3388 Acked Write 8 Sent Done Done false
8916 3389 Acked Flush 0 Sent Done Done false
8917 3390 Acked Write 6 Sent Done Done false
8918 3391 Acked Write 2 Sent Done Done false
8919 3392 Acked Read 2 Sent Done Done false
8920 3393 Acked Write 1 Sent Done Done false
8921 3394 Acked Write 1 Sent Done Done false
8922 3395 Acked Write 6 Sent Done Done false
8923 3396 Acked Read 5 Sent Done Done false
8924 3397 Acked Read 7 Sent Done Done false
8925 3398 Acked Write 6 Sent Done Done false
8926 3399 Acked Read 10 Sent Done Done false
8927 3400 Acked Write 4 Sent Done Done false
8928 3401 Acked Read 1 Sent Done Done false
8929 3402 Acked Write 10 Sent Done Done false
8930 3403 Acked Read 8 Sent Done Done false
8931 3404 Acked Read 3 Sent Done Done false
8932 3405 Acked Read 2 Sent Done Done false
8933 3406 Acked Write 5 Sent Done Done false
8934 3407 Acked Write 5 Sent Done Done false
8935 3408 Acked Read 1 Sent Done Done false
8936 3409 Acked Flush 0 Sent Done Done false
8937 3410 Acked Write 3 Sent Done Done false
8938 3411 Acked Write 3 Sent Done Done false
8939 3412 Acked Flush 0 Sent Done Done false
8940 3413 Acked Write 5 Sent Done Done false
8941 3414 Acked Read 4 Sent Done Done false
8942 STATES DS:0 DS:1 DS:2 TOTAL
8943 Sent 301 0 0 301
8944 Done 0 301 301 602
8945 Skipped 0 0 0 0
8946 Error 0 0 0 0
8947 Last Flush: 3113 3412 3412
8948 Downstairs last five completed: 3113 3112 3111 3110 3109
8949 Upstairs last five completed: 3414 3413 3412 3411 3410
8950 CLIENT: Up:0 ds:301 act:2
8951 [8810] got start action
8952 [8810] I am going to restart
8953 Starting downstairs at port 8810
8954 Make output file at "/tmp/dsc/downstairs-8810.txt"
8955 [8810][0] reports Starting
8956 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1211)
8957 [8810][0] reports Running
8958 ----------------------------------------------------------------
8959 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
8960 States: Offline Active Active
8961 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
8962 3114 Acked Write 7 Sent Done Done false
8963 3115 Acked Write 3 Sent Done Done false
8964 3116 Acked Write 6 Sent Done Done false
8965 3117 Acked Read 9 Sent Done Done false
8966 3118 Acked Read 8 Sent Done Done false
8967 3119 Acked Read 2 Sent Done Done false
8968 3120 Acked Flush 0 Sent Done Done false
8969 3121 Acked Write 3 Sent Done Done false
8970 3122 Acked Write 7 Sent Done Done false
8971 3123 Acked Read 6 Sent Done Done false
8972 3124 Acked Flush 0 Sent Done Done false
8973 3125 Acked Read 2 Sent Done Done false
8974 3126 Acked Read 6 Sent Done Done false
8975 3127 Acked Read 3 Sent Done Done false
8976 3128 Acked Write 10 Sent Done Done false
8977 3129 Acked Write 5 Sent Done Done false
8978 3130 Acked Read 8 Sent Done Done false
8979 3131 Acked Read 7 Sent Done Done false
8980 3132 Acked Write 4 Sent Done Done false
8981 3133 Acked Read 9 Sent Done Done false
8982 3134 Acked Write 2 Sent Done Done false
8983 3135 Acked Write 8 Sent Done Done false
8984 3136 Acked Write 3 Sent Done Done false
8985 3137 Acked Write 9 Sent Done Done false
8986 3138 Acked Read 5 Sent Done Done false
8987 3139 Acked Write 2 Sent Done Done false
8988 3140 Acked Write 2 Sent Done Done false
8989 3141 Acked Write 2 Sent Done Done false
8990 3142 Acked Write 2 Sent Done Done false
8991 3143 Acked Read 10 Sent Done Done false
8992 3144 Acked Flush 0 Sent Done Done false
8993 3145 Acked Read 7 Sent Done Done false
8994 3146 Acked Read 9 Sent Done Done false
8995 3147 Acked Write 9 Sent Done Done false
8996 3148 Acked Write 3 Sent Done Done false
8997 3149 Acked Write 2 Sent Done Done false
8998 3150 Acked Read 5 Sent Done Done false
8999 3151 Acked Read 9 Sent Done Done false
9000 3152 Acked Read 4 Sent Done Done false
9001 3153 Acked Read 5 Sent Done Done false
9002 3154 Acked Write 7 Sent Done Done false
9003 3155 Acked Write 4 Sent Done Done false
9004 3156 Acked Write 4 Sent Done Done false
9005 3157 Acked Read 3 Sent Done Done false
9006 3158 Acked Read 8 Sent Done Done false
9007 3159 Acked Write 5 Sent Done Done false
9008 3160 Acked Flush 0 Sent Done Done false
9009 3161 Acked Read 1 Sent Done Done false
9010 3162 Acked Read 8 Sent Done Done false
9011 3163 Acked Read 7 Sent Done Done false
9012 3164 Acked Write 4 Sent Done Done false
9013 3165 Acked Read 7 Sent Done Done false
9014 3166 Acked Write 9 Sent Done Done false
9015 3167 Acked Write 5 Sent Done Done false
9016 3168 Acked Read 10 Sent Done Done false
9017 3169 Acked Write 5 Sent Done Done false
9018 3170 Acked Write 10 Sent Done Done false
9019 3171 Acked Flush 0 Sent Done Done false
9020 3172 Acked Write 3 Sent Done Done false
9021 3173 Acked Write 9 Sent Done Done false
9022 3174 Acked Write 6 Sent Done Done false
9023 3175 Acked Read 4 Sent Done Done false
9024 3176 Acked Read 1 Sent Done Done false
9025 3177 Acked Read 9 Sent Done Done false
9026 3178 Acked Read 3 Sent Done Done false
9027 3179 Acked Write 6 Sent Done Done false
9028 3180 Acked Write 4 Sent Done Done false
9029 3181 Acked Read 5 Sent Done Done false
9030 3182 Acked Write 8 Sent Done Done false
9031 3183 Acked Read 6 Sent Done Done false
9032 3184 Acked Read 10 Sent Done Done false
9033 3185 Acked Read 9 Sent Done Done false
9034 3186 Acked Write 1 Sent Done Done false
9035 3187 Acked Read 10 Sent Done Done false
9036 3188 Acked Read 7 Sent Done Done false
9037 3189 Acked Read 6 Sent Done Done false
9038 3190 Acked Read 1 Sent Done Done false
9039 3191 Acked Write 6 Sent Done Done false
9040 3192 Acked Read 7 Sent Done Done false
9041 3193 Acked Read 10 Sent Done Done false
9042 3194 Acked Write 4 Sent Done Done false
9043 3195 Acked Read 6 Sent Done Done false
9044 3196 Acked Read 5 Sent Done Done false
9045 3197 Acked Read 6 Sent Done Done false
9046 3198 Acked Write 7 Sent Done Done false
9047 3199 Acked Write 3 Sent Done Done false
9048 3200 Acked Write 9 Sent Done Done false
9049 3201 Acked Write 7 Sent Done Done false
9050 3202 Acked Flush 0 Sent Done Done false
9051 3203 Acked Read 3 Sent Done Done false
9052 3204 Acked Read 4 Sent Done Done false
9053 3205 Acked Write 3 Sent Done Done false
9054 3206 Acked Write 4 Sent Done Done false
9055 3207 Acked Flush 0 Sent Done Done false
9056 3208 Acked Read 5 Sent Done Done false
9057 3209 Acked Write 1 Sent Done Done false
9058 3210 Acked Write 5 Sent Done Done false
9059 3211 Acked Read 6 Sent Done Done false
9060 3212 Acked Write 3 Sent Done Done false
9061 3213 Acked Write 1 Sent Done Done false
9062 3214 Acked Read 8 Sent Done Done false
9063 3215 Acked Write 3 Sent Done Done false
9064 3216 Acked Flush 0 Sent Done Done false
9065 3217 Acked Read 6 Sent Done Done false
9066 3218 Acked Write 7 Sent Done Done false
9067 3219 Acked Write 8 Sent Done Done false
9068 3220 Acked Write 3 Sent Done Done false
9069 3221 Acked Read 3 Sent Done Done false
9070 3222 Acked Write 2 Sent Done Done false
9071 3223 Acked Flush 0 Sent Done Done false
9072 3224 Acked Read 5 Sent Done Done false
9073 3225 Acked Read 5 Sent Done Done false
9074 3226 Acked Read 4 Sent Done Done false
9075 3227 Acked Write 2 Sent Done Done false
9076 3228 Acked Read 10 Sent Done Done false
9077 3229 Acked Read 7 Sent Done Done false
9078 3230 Acked Write 10 Sent Done Done false
9079 3231 Acked Read 4 Sent Done Done false
9080 3232 Acked Write 8 Sent Done Done false
9081 3233 Acked Read 4 Sent Done Done false
9082 3234 Acked Write 3 Sent Done Done false
9083 3235 Acked Read 3 Sent Done Done false
9084 3236 Acked Write 5 Sent Done Done false
9085 3237 Acked Write 3 Sent Done Done false
9086 3238 Acked Read 5 Sent Done Done false
9087 3239 Acked Write 10 Sent Done Done false
9088 3240 Acked Write 6 Sent Done Done false
9089 3241 Acked Write 4 Sent Done Done false
9090 3242 Acked Write 8 Sent Done Done false
9091 3243 Acked Write 8 Sent Done Done false
9092 3244 Acked Flush 0 Sent Done Done false
9093 3245 Acked Read 5 Sent Done Done false
9094 3246 Acked Write 8 Sent Done Done false
9095 3247 Acked Flush 0 Sent Done Done false
9096 3248 Acked Read 1 Sent Done Done false
9097 3249 Acked Read 2 Sent Done Done false
9098 3250 Acked Flush 0 Sent Done Done false
9099 3251 Acked Flush 0 Sent Done Done false
9100 3252 Acked Write 3 Sent Done Done false
9101 3253 Acked Read 3 Sent Done Done false
9102 3254 Acked Write 9 Sent Done Done false
9103 3255 Acked Read 5 Sent Done Done false
9104 3256 Acked Write 9 Sent Done Done false
9105 3257 Acked Read 3 Sent Done Done false
9106 3258 Acked Read 10 Sent Done Done false
9107 3259 Acked Write 1 Sent Done Done false
9108 3260 Acked Read 3 Sent Done Done false
9109 3261 Acked Write 2 Sent Done Done false
9110 3262 Acked Read 1 Sent Done Done false
9111 3263 Acked Flush 0 Sent Done Done false
9112 3264 Acked Read 10 Sent Done Done false
9113 3265 Acked Read 4 Sent Done Done false
9114 3266 Acked Write 9 Sent Done Done false
9115 3267 Acked Write 1 Sent Done Done false
9116 3268 Acked Write 7 Sent Done Done false
9117 3269 Acked Write 7 Sent Done Done false
9118 3270 Acked Write 1 Sent Done Done false
9119 3271 Acked Read 2 Sent Done Done false
9120 3272 Acked Write 5 Sent Done Done false
9121 3273 Acked Write 6 Sent Done Done false
9122 3274 Acked Read 7 Sent Done Done false
9123 3275 Acked Flush 0 Sent Done Done false
9124 3276 Acked Write 6 Sent Done Done false
9125 3277 Acked Flush 0 Sent Done Done false
9126 3278 Acked Read 5 Sent Done Done false
9127 3279 Acked Write 4 Sent Done Done false
9128 3280 Acked Read 6 Sent Done Done false
9129 3281 Acked Read 1 Sent Done Done false
9130 3282 Acked Write 6 Sent Done Done false
9131 3283 Acked Write 8 Sent Done Done false
9132 3284 Acked Read 9 Sent Done Done false
9133 3285 Acked Read 8 Sent Done Done false
9134 3286 Acked Write 9 Sent Done Done false
9135 3287 Acked Write 6 Sent Done Done false
9136 3288 Acked Read 7 Sent Done Done false
9137 3289 Acked Read 9 Sent Done Done false
9138 3290 Acked Read 2 Sent Done Done false
9139 3291 Acked Read 2 Sent Done Done false
9140 3292 Acked Write 5 Sent Done Done false
9141 3293 Acked Write 4 Sent Done Done false
9142 3294 Acked Write 9 Sent Done Done false
9143 3295 Acked Read 10 Sent Done Done false
9144 3296 Acked Read 6 Sent Done Done false
9145 3297 Acked Write 4 Sent Done Done false
9146 3298 Acked Read 9 Sent Done Done false
9147 3299 Acked Read 1 Sent Done Done false
9148 3300 Acked Write 8 Sent Done Done false
9149 3301 Acked Read 10 Sent Done Done false
9150 3302 Acked Read 1 Sent Done Done false
9151 3303 Acked Read 8 Sent Done Done false
9152 3304 Acked Read 5 Sent Done Done false
9153 3305 Acked Read 4 Sent Done Done false
9154 3306 Acked Write 2 Sent Done Done false
9155 3307 Acked Write 4 Sent Done Done false
9156 3308 Acked Write 5 Sent Done Done false
9157 3309 Acked Flush 0 Sent Done Done false
9158 3310 Acked Read 10 Sent Done Done false
9159 3311 Acked Flush 0 Sent Done Done false
9160 3312 Acked Write 6 Sent Done Done false
9161 3313 Acked Read 4 Sent Done Done false
9162 3314 Acked Write 2 Sent Done Done false
9163 3315 Acked Write 2 Sent Done Done false
9164 3316 Acked Flush 0 Sent Done Done false
9165 3317 Acked Read 4 Sent Done Done false
9166 3318 Acked Write 9 Sent Done Done false
9167 3319 Acked Flush 0 Sent Done Done false
9168 3320 Acked Read 1 Sent Done Done false
9169 3321 Acked Write 9 Sent Done Done false
9170 3322 Acked Write 9 Sent Done Done false
9171 3323 Acked Flush 0 Sent Done Done false
9172 3324 Acked Read 4 Sent Done Done false
9173 3325 Acked Flush 0 Sent Done Done false
9174 3326 Acked Write 4 Sent Done Done false
9175 3327 Acked Read 8 Sent Done Done false
9176 3328 Acked Read 3 Sent Done Done false
9177 3329 Acked Read 3 Sent Done Done false
9178 3330 Acked Read 3 Sent Done Done false
9179 3331 Acked Flush 0 Sent Done Done false
9180 3332 Acked Read 7 Sent Done Done false
9181 3333 Acked Write 9 Sent Done Done false
9182 3334 Acked Read 9 Sent Done Done false
9183 3335 Acked Read 4 Sent Done Done false
9184 3336 Acked Write 8 Sent Done Done false
9185 3337 Acked Read 5 Sent Done Done false
9186 3338 Acked Read 2 Sent Done Done false
9187 3339 Acked Read 4 Sent Done Done false
9188 3340 Acked Read 3 Sent Done Done false
9189 3341 Acked Read 6 Sent Done Done false
9190 3342 Acked Flush 0 Sent Done Done false
9191 3343 Acked Read 8 Sent Done Done false
9192 3344 Acked Read 8 Sent Done Done false
9193 3345 Acked Write 1 Sent Done Done false
9194 3346 Acked Read 3 Sent Done Done false
9195 3347 Acked Write 2 Sent Done Done false
9196 3348 Acked Write 6 Sent Done Done false
9197 3349 Acked Read 9 Sent Done Done false
9198 3350 Acked Flush 0 Sent Done Done false
9199 3351 Acked Read 9 Sent Done Done false
9200 3352 Acked Read 4 Sent Done Done false
9201 3353 Acked Read 4 Sent Done Done false
9202 3354 Acked Read 9 Sent Done Done false
9203 3355 Acked Write 3 Sent Done Done false
9204 3356 Acked Write 8 Sent Done Done false
9205 3357 Acked Write 5 Sent Done Done false
9206 3358 Acked Write 8 Sent Done Done false
9207 3359 Acked Write 8 Sent Done Done false
9208 3360 Acked Write 2 Sent Done Done false
9209 3361 Acked Read 1 Sent Done Done false
9210 3362 Acked Write 6 Sent Done Done false
9211 3363 Acked Flush 0 Sent Done Done false
9212 3364 Acked Write 5 Sent Done Done false
9213 3365 Acked Read 9 Sent Done Done false
9214 3366 Acked Read 7 Sent Done Done false
9215 3367 Acked Read 3 Sent Done Done false
9216 3368 Acked Read 2 Sent Done Done false
9217 3369 Acked Read 10 Sent Done Done false
9218 3370 Acked Write 1 Sent Done Done false
9219 3371 Acked Read 9 Sent Done Done false
9220 3372 Acked Write 2 Sent Done Done false
9221 3373 Acked Write 6 Sent Done Done false
9222 3374 Acked Write 6 Sent Done Done false
9223 3375 Acked Read 6 Sent Done Done false
9224 3376 Acked Write 2 Sent Done Done false
9225 3377 Acked Write 1 Sent Done Done false
9226 3378 Acked Read 4 Sent Done Done false
9227 3379 Acked Read 1 Sent Done Done false
9228 3380 Acked Read 10 Sent Done Done false
9229 3381 Acked Read 6 Sent Done Done false
9230 3382 Acked Read 2 Sent Done Done false
9231 3383 Acked Read 3 Sent Done Done false
9232 3384 Acked Read 6 Sent Done Done false
9233 3385 Acked Write 9 Sent Done Done false
9234 3386 Acked Write 8 Sent Done Done false
9235 3387 Acked Write 6 Sent Done Done false
9236 3388 Acked Write 8 Sent Done Done false
9237 3389 Acked Flush 0 Sent Done Done false
9238 3390 Acked Write 6 Sent Done Done false
9239 3391 Acked Write 2 Sent Done Done false
9240 3392 Acked Read 2 Sent Done Done false
9241 3393 Acked Write 1 Sent Done Done false
9242 3394 Acked Write 1 Sent Done Done false
9243 3395 Acked Write 6 Sent Done Done false
9244 3396 Acked Read 5 Sent Done Done false
9245 3397 Acked Read 7 Sent Done Done false
9246 3398 Acked Write 6 Sent Done Done false
9247 3399 Acked Read 10 Sent Done Done false
9248 3400 Acked Write 4 Sent Done Done false
9249 3401 Acked Read 1 Sent Done Done false
9250 3402 Acked Write 10 Sent Done Done false
9251 3403 Acked Read 8 Sent Done Done false
9252 3404 Acked Read 3 Sent Done Done false
9253 3405 Acked Read 2 Sent Done Done false
9254 3406 Acked Write 5 Sent Done Done false
9255 3407 Acked Write 5 Sent Done Done false
9256 3408 Acked Read 1 Sent Done Done false
9257 3409 Acked Flush 0 Sent Done Done false
9258 3410 Acked Write 3 Sent Done Done false
9259 3411 Acked Write 3 Sent Done Done false
9260 3412 Acked Flush 0 Sent Done Done false
9261 3413 Acked Write 5 Sent Done Done false
9262 3414 Acked Read 4 Sent Done Done false
9263 3415 Acked Flush 0 Sent Done Done false
9264 STATES DS:0 DS:1 DS:2 TOTAL
9265 Sent 302 0 0 302
9266 Done 0 302 302 604
9267 Skipped 0 0 0 0
9268 Error 0 0 0 0
9269 Last Flush: 3113 3415 3415
9270 Downstairs last five completed: 3113 3112 3111 3110 3109
9271 Upstairs last five completed: 3415 3414 3413 3412 3411
9272 CLIENT: Up:0 ds:302 act:2
92732024-11-26T22:06:28.111ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
92742024-11-26T22:06:28.111ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:46261) = io task client = 0 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
92752024-11-26T22:06:28.112ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
92762024-11-26T22:06:28.113ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
92772024-11-26T22:06:28.113ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
92782024-11-26T22:06:28.113ZINFOcrucible: send last flush ID to this DS: 3113 = downstairs client = 0 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
92792024-11-26T22:06:28.113ZINFOcrucible: Replied this last flush ID: 3113 = downstairs client = 0 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
92802024-11-26T22:06:28.113ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
92812024-11-26T22:06:28.113ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
92822024-11-26T22:06:28.113ZINFOcrucible: [0] client re-new 302 jobs since flush 3113 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
92832024-11-26T22:06:28.113ZINFOcrucible: [0] Replayed 302 jobs since flush: 3113 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
9284 ----------------------------------------------------------------
9285 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
9286 Downstairs last five completed: 3415 3414 3413 3412 3411
9287 Upstairs last five completed: 3415 3414 3413 3412 3411
9288 CLIENT: Up:0 ds:0 act:3
9289 Replay: All jobs finished
9290 Main task has work to do, go find it
9291 got dsc Stop(1)
9292 stop 1
9293 Nov 26 22:06:30.343 INFO request completed, latency_us: [8820] Got stop action so:false kr:false
9294 348, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 0ef2430c-0efd-4156-acd7-ad3e15595e24, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
9295 Nov 26 22:06:30.344 INFO request completed, latency_us: 137, response_code: 200, uri: {"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2024-11-26T22:06:30.344532702Z","hostname":"w-01JDN7ZQ17ABQG15KHKN3F2D26","pid":1203,"":"io task","client":"1","":"downstairs","session_id":"86a36c17-3718-484c-87fd-2b8a1dc2ab18"}
9296 /state/cid/1, method: GET, req_id: d59791b0-ddb6-4044-8735-429530ecb275, remote_addr: 127{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2024-11-26T22:06:30.344617732Z","hostname":"w-01JDN7ZQ17ABQG15KHKN3F2D26","pid":1203,"":"io task","client":"1","":"downstairs","session_id":"86a36c17-3718-484c-87fd-2b8a1dc2ab18"}
9297 .{"msg":"downstairs task for 1 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2024-11-26T22:06:30.344639809Z","hostname":"w-01JDN7ZQ17ABQG15KHKN3F2D26","pid":1203,"session_id":"86a36c17-3718-484c-87fd-2b8a1dc2ab18"}
9298 0.0.1:36829{"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-26T22:06:30.344691113Z","hostname":"w-01JDN7ZQ17ABQG15KHKN3F2D26","pid":1203,"client":"1","":"downstairs","session_id":"86a36c17-3718-484c-87fd-2b8a1dc2ab18"}
9299 , local_addr: 127.0.0.1:9998
93002024-11-26T22:06:30.344ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
93012024-11-26T22:06:30.344ZINFOcrucible: client task is exiting = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
9302 [8820] Exited with: ExitStatus(unix_wait_status(9))
9303 [8820][1] reports Exit
9304 Nov 26 22:06:34.347 INFO request completed, latency_us: 196, response_code: 200, uri: /state/cid/1, method: GET, req_id: f17becc3-7fb5-41d4-9418-cf6700cce598, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
9305 001/300 Read block 4569 len 40960
9306 002/300 Write block 1785 len 16384 data: 2 2 3 3
9307 003/300 Read block 4915 len 36864
9308 004/300 Read block 549 len 28672
9309 005/300 Write block 3610 len 36864 data: 3 3 3 3 3 3 2 2 2
9310 006/300 Read block 4990 len 32768
9311 007/300 Read block 4868 len 8192
9312 008/300 Write block 3952 len 36864 data: 4 3 3 3 3 2 2 2 3
9313 009/300 Read block 4301 len 40960
9314 010/300 Write block 1935 len 16384 data: 3 4 5 6
9315 011/300 Write block 3087 len 8192 data: 4 4
9316 012/300 Write block 2143 len 12288 data: 3 3 4
9317 013/300 Read block 334 len 32768
9318 014/300 Write block 4034 len 40960 data: 3 2 2 2 2 2 2 3 3 4
9319 015/300 Read block 2716 len 32768
9320 016/300 Write block 2883 len 40960 data: 3 3 3 4 3 3 3 3 2 3
9321 017/300 Read block 682 len 8192
9322 018/300 Read block 4615 len 4096
9323 019/300 Read block 3458 len 4096
9324 020/300 Flush
9325 021/300 Read block 2668 len 12288
9326 022/300 Read block 1770 len 40960
9327 023/300 Read block 2899 len 8192
9328 024/300 Write block 2709 len 36864 data: 4 3 3 3 3 3 3 3 3
9329 025/300 Flush
9330 026/300 Read block 1158 len 20480
9331 027/300 Write block 2675 len 16384 data: 3 3 2 2
9332 028/300 Read block 3895 len 32768
9333 029/300 Write block 4702 len 36864 data: 3 4 4 4 4 4 3 3 3
9334 030/300 Read block 2917 len 20480
9335 031/300 Write block 1480 len 12288 data: 4 4 4
9336 032/300 Flush
9337 033/300 Write block 1552 len 8192 data: 4 5
9338 034/300 Write block 2335 len 32768 data: 3 3 3 3 3 4 4 4
9339 035/300 Write block 3082 len 40960 data: 2 2 2 2 2 5 5 4 5 4
9340 036/300 Write block 267 len 24576 data: 3 3 3 3 3 3
9341 037/300 Write block 1716 len 12288 data: 4 4 3
9342 038/300 Flush
9343 039/300 Flush
9344 040/300 Read block 4135 len 4096
9345 041/300 Write block 2877 len 12288 data: 3 2 2
9346 042/300 Flush
9347 043/300 Read block 2210 len 40960
9348 044/300 Read block 969 len 32768
9349 045/300 Flush
9350 046/300 Read block 3713 len 16384
9351 047/300 Write block 3825 len 36864 data: 2 2 4 4 4 4 4 3 3
9352 048/300 Write block 1423 len 20480 data: 2 2 2 2 2
9353 049/300 Read block 2083 len 24576
9354 050/300 Read block 1361 len 20480
9355 051/300 Flush
9356 052/300 Read block 2454 len 12288
9357 053/300 Read block 652 len 4096
9358 054/300 Read block 1063 len 28672
9359 055/300 Read block 1798 len 16384
9360 056/300 Flush
9361 057/300 Flush
9362 058/300 Write block 4037 len 4096 data: 3
9363 059/300 Read block 3018 len 24576
9364 060/300 Write block 1832 len 20480 data: 3 3 2 2 2
9365 061/300 Write block 4875 len 28672 data: 4 4 4 4 3 3 2
9366 062/300 Flush
9367 063/300 Write block 1102 len 8192 data: 3 3
9368 064/300 Read block 4021 len 40960
9369 065/300 Read block 1537 len 20480
9370 066/300 Write block 2265 len 40960 data: 2 3 3 3 3 3 3 3 2 2
9371 067/300 Read block 3521 len 20480
9372 068/300 Write block 1769 len 40960 data: 2 2 2 2 2 3 3 3 4 4
9373 069/300 Write block 1827 len 24576 data: 3 4 4 4 3 4
9374 070/300 Write block 2626 len 16384 data: 2 2 2 2
9375 071/300 Flush
9376 072/300 Read block 396 len 12288
9377 073/300 Write block 4746 len 12288 data: 2 2 2
9378 074/300 Read block 877 len 24576
9379 075/300 Write block 2054 len 24576 data: 4 3 3 4 4 5
9380 076/300 Write block 1193 len 28672 data: 3 3 3 4 5 4 4
9381 077/300 Write block 3987 len 36864 data: 2 2 3 4 5 5 5 4 4
9382 078/300 Read block 4076 len 12288
9383 079/300 Read block 319 len 32768
9384 080/300 Read block 1459 len 40960
9385 081/300 Write block 40 len 28672 data: 3 3 3 2 2 3 4
9386 082/300 Read block 1581 len 20480
9387 083/300 Flush
9388 084/300 Flush
9389 085/300 Read block 1672 len 40960
9390 086/300 Read block 3002 len 36864
9391 087/300 Flush
9392 088/300 Write block 1692 len 32768 data: 4 2 2 2 3 3 3 3
9393 089/300 Read block 1246 len 12288
9394 090/300 Write block 1531 len 4096 data: 5
9395 091/300 Write block 1022 len 36864 data: 3 3 2 2 2 2 2 3 2
9396 092/300 Flush
9397 093/300 Flush
9398 094/300 Flush
9399 095/300 Read block 4510 len 8192
9400 096/300 Read block 4978 len 28672
9401 097/300 Read block 4434 len 20480
9402 098/300 Write block 1833 len 12288 data: 4 3 3
9403 099/300 Read block 2628 len 16384
9404 100/300 Read block 4655 len 12288
9405 101/300 Read block 8 len 16384
9406 102/300 Read block 2871 len 32768
9407 103/300 Read block 610 len 20480
9408 104/300 Write block 3280 len 28672 data: 2 2 2 2 2 3 3
9409 105/300 Read block 3333 len 28672
9410 106/300 Read block 2063 len 8192
9411 107/300 Write block 963 len 32768 data: 2 2 3 3 3 3 3 2
9412 108/300 Read block 4931 len 16384
9413 109/300 Read block 154 len 16384
9414 110/300 Write block 1159 len 12288 data: 2 2 2
9415 111/300 Write block 2463 len 4096 data: 2
9416 112/300 Write block 704 len 4096 data: 2
9417 113/300 Read block 1736 len 8192
9418 114/300 Write block 1133 len 12288 data: 3 2 2
9419 115/300 Write block 4828 len 28672 data: 4 5 5 5 4 4 4
9420 116/300 Write block 789 len 28672 data: 3 3 3 3 3 3 2
9421 117/300 Read block 171 len 28672
9422 118/300 Read block 564 len 36864
9423 119/300 Read block 3110 len 24576
9424 120/300 Write block 1642 len 16384 data: 3 3 4 4
9425 121/300 Read block 4341 len 24576
9426 122/300 Write block 3593 len 40960 data: 5 5 4 4 4 4 4 3 4 3
9427 123/300 Flush
9428 124/300 Read block 1460 len 4096
9429 125/300 Read block 4282 len 12288
9430 126/300 Read block 3817 len 4096
9431 127/300 Read block 2043 len 40960
9432 128/300 Write block 2048 len 20480 data: 3 2 2 2 2
9433 129/300 Read block 4926 len 24576
9434 130/300 Read block 294 len 12288
9435 131/300 Read block 1808 len 12288
9436 132/300 Read block 4220 len 28672
9437 133/300 Read block 4161 len 12288
9438 134/300 Read block 4563 len 32768
9439 135/300 Write block 2598 len 8192 data: 2 2
9440 136/300 Read block 1247 len 4096
9441 137/300 Write block 4478 len 24576 data: 3 3 3 2 2 2
9442 138/300 Write block 3954 len 4096 data: 4
9443 139/300 Write block 3241 len 16384 data: 2 3 3 4
9444 140/300 Read block 3915 len 32768
9445 141/300 Write block 3859 len 16384 data: 4 4 5 4
9446 142/300 Write block 2391 len 8192 data: 5 5
9447 143/300 Read block 1645 len 20480
9448 144/300 Flush
9449 145/300 Write block 4342 len 32768 data: 3 3 3 2 2 2 3 4
9450 146/300 Write block 4436 len 36864 data: 5 5 5 5 3 3 4 5 5
9451 147/300 Read block 4426 len 28672
9452 148/300 Write block 2875 len 12288 data: 3 3 4
9453 149/300 Read block 2759 len 4096
9454 150/300 Write block 761 len 32768 data: 3 3 3 3 3 3 3 3
9455 151/300 Read block 4104 len 8192
9456 152/300 Read block 3633 len 16384
9457 153/300 Write block 711 len 32768 data: 2 2 2 2 2 2 2 2
9458 154/300 Write block 4042 len 24576 data: 4 5 4 4 4 4
9459 155/300 Read block 160 len 8192
9460 156/300 Read block 4746 len 16384
9461 157/300 Write block 190 len 4096 data: 5
9462 158/300 Flush
9463 159/300 Read block 3762 len 24576
9464 160/300 Read block 1536 len 24576
9465 161/300 Read block 182 len 28672
9466 162/300 Read block 2476 len 28672
9467 163/300 Read block 175 len 8192
9468 164/300 Write block 926 len 8192 data: 5 4
9469 165/300 Write block 4763 len 20480 data: 3 3 3 3 3
9470 166/300 Read block 105 len 24576
9471 167/300 Write block 698 len 12288 data: 2 2 2
9472 168/300 Write block 3490 len 40960 data: 2 3 3 3 3 3 3 2 2 2
9473 169/300 Read block 468 len 8192
9474 170/300 Read block 4903 len 20480
9475 171/300 Read block 1539 len 24576
9476 172/300 Flush
9477 173/300 Write block 3470 len 32768 data: 2 2 2 3 4 4 4 5
9478 174/300 Read block 4026 len 28672
9479 175/300 Read block 2241 len 36864
9480 176/300 Read block 4165 len 20480
9481 177/300 Write block 2883 len 12288 data: 4 4 4
9482 178/300 Write block 4071 len 16384 data: 4 5 5 4
9483 179/300 Flush
9484 180/300 Read block 235 len 24576
9485 181/300 Read block 3200 len 12288
9486 182/300 Read block 4623 len 16384
9487 183/300 Write block 2932 len 32768 data: 2 2 2 2 2 2 2 2
9488 184/300 Read block 1662 len 36864
9489 185/300 Write block 140 len 8192 data: 2 2
9490 186/300 Write block 4060 len 20480 data: 5 4 5 5 5
9491 187/300 Write block 4014 len 36864 data: 2 2 2 2 2 2 2 2 2
9492 188/300 Read block 1771 len 16384
9493 189/300 Write block 1580 len 12288 data: 4 3 3
9494 190/300 Write block 521 len 16384 data: 3 3 3 3
9495 191/300 Read block 3442 len 16384
9496 192/300 Read block 2964 len 8192
9497 193/300 Write block 1715 len 16384 data: 4 5 5 4
9498 194/300 Read block 3076 len 8192
9499 195/300 Write block 1522 len 24576 data: 3 4 4 5 5 5
9500 196/300 Write block 4206 len 24576 data: 2 2 2 2 2 2
9501 197/300 Read block 3828 len 36864
9502 198/300 Read block 775 len 24576
9503 199/300 Read block 4474 len 20480
9504 200/300 Write block 2705 len 4096 data: 3
9505 201/300 Read block 4917 len 4096
9506 202/300 Write block 1231 len 24576 data: 3 3 3 3 3 3
9507 203/300 Write block 4395 len 8192 data: 2 2
9508 204/300 Read block 1665 len 20480
9509 205/300 Write block 3870 len 36864 data: 2 2 4 5 5 4 3 3 3
9510 206/300 Flush
9511 207/300 Write block 3857 len 20480 data: 4 4 5 5 6
9512 208/300 Write block 1273 len 28672 data: 3 3 2 2 3 3 3
9513 209/300 Flush
9514 210/300 Read block 825 len 8192
9515 211/300 Write block 1150 len 12288 data: 4 4 3
9516 212/300 Write block 1693 len 8192 data: 3 3
9517 213/300 Write block 3166 len 36864 data: 3 3 3 2 2 2 2 2 2
9518 214/300 Write block 747 len 24576 data: 3 2 2 2 2 2
9519 215/300 Flush
9520 216/300 Read block 2708 len 40960
9521 217/300 Write block 667 len 20480 data: 4 3 3 3 3
9522 218/300 Write block 4156 len 16384 data: 2 2 3 4
9523 219/300 Read block 290 len 28672
9524 220/300 Write block 3081 len 12288 data: 3 3 3
9525 221/300 Flush
9526 222/300 Read block 2922 len 20480
9527 223/300 Read block 1402 len 32768
9528 224/300 Write block 1671 len 40960 data: 4 4 5 5 4 4 3 2 3 3
9529 225/300 Read block 1025 len 16384
9530 226/300 Read block 4088 len 28672
9531 227/300 Read block 15 len 20480
9532 228/300 Read block 2436 len 12288
9533 229/300 Write block 2321 len 12288 data: 3 3 3
9534 230/300 Read block 531 len 20480
9535 231/300 Read block 4776 len 4096
9536 232/300 Read block 2847 len 36864
9537 233/300 Read block 4254 len 28672
9538 234/300 Write block 4989 len 40960 data: 5 6 6 6 5 5 5 4 3 2
9539 235/300 Write block 1193 len 8192 data: 4 4
9540 236/300 Write block 4144 len 24576 data: 4 3 3 2 2 2
9541 237/300 Read block 3524 len 32768
9542 238/300 Write block 4747 len 36864 data: 3 3 2 4 3 3 3 3 3
9543 239/300 Read block 166 len 4096
9544 240/300 Write block 394 len 36864 data: 2 2 2 2 2 2 2 2 2
9545 241/300 Read block 3117 len 24576
9546 242/300 Write block 4248 len 16384 data: 3 3 3 3
9547 243/300 Read block 1919 len 8192
9548 244/300 Write block 781 len 8192 data: 3 3
9549 245/300 Read block 2359 len 36864
9550 246/300 Read block 2743 len 40960
9551 247/300 Read block 1183 len 24576
9552 248/300 Read block 49 len 24576
9553 249/300 Write block 802 len 36864 data: 3 3 3 2 2 2 3 3 2
9554 250/300 Write block 3135 len 32768 data: 2 2 2 2 2 4 4 5
9555 251/300 Write block 1499 len 4096 data: 4
9556 252/300 Read block 3290 len 36864
9557 253/300 Read block 3794 len 4096
9558 254/300 Read block 754 len 32768
9559 255/300 Write block 1844 len 36864 data: 3 3 4 5 5 5 5 5 5
9560 256/300 Write block 94 len 32768 data: 3 3 5 6 5 5 5 5
9561 257/300 Write block 3300 len 40960 data: 2 2 2 2 2 2 2 2 2 2
9562 258/300 Read block 4007 len 12288
9563 259/300 Read block 3322 len 24576
9564 260/300 Write block 3505 len 24576 data: 4 4 3 3 3 2
9565 261/300 Read block 329 len 12288
9566 262/300 Read block 4312 len 28672
9567 263/300 Write block 3049 len 28672 data: 3 3 3 3 2 3 2
9568 264/300 Write block 498 len 40960 data: 2 2 2 2 2 2 2 2 2 2
9569 265/300 Read block 4805 len 28672
9570 266/300 Flush
9571 267/300 Read block 3828 len 32768
9572 268/300 Read block 2276 len 20480
9573 269/300 Read block 4975 len 24576
9574 270/300 Write block 4528 len 40960 data: 3 3 3 3 3 3 3 3 4 5
9575 271/300 Read block 4956 len 12288
9576 272/300 Flush
9577 273/300 Write block 3153 len 16384 data: 2 2 3 3
9578 274/300 Write block 183 len 12288 data: 4 4 4
9579 275/300 Write block 2612 len 4096 data: 3
9580 276/300 Read block 995 len 36864
9581 277/300 Write block 4296 len 24576 data: 2 2 2 2 2 2
9582 278/300 Read block 1768 len 40960
9583 279/300 Read block 3516 len 28672
9584 280/300 Write block 2516 len 24576 data: 4 4 4 4 3 3
9585 281/300 Write block 3412 len 4096 data: 2
9586 282/300 Write block 734 len 36864 data: 3 3 2 3 3 3 4 5 6
9587 283/300 Flush
9588 284/300 Flush
9589 285/300 Write block 789 len 12288 data: 4 4 4
9590 286/300 Read block 1500 len 36864
9591 287/300 Read block 2809 len 20480
9592 288/300 Read block 4187 len 16384
9593 289/300 Write block 4661 len 16384 data: 3 3 3 3
9594 290/300 Write block 2148 len 8192 data: 2 2
9595 291/300 Read block 54 len 40960
9596 292/300 Flush
9597 293/300 Read block 1980 len 28672
9598 294/300 Write block 402 len 8192 data: 3 4
9599 295/300 Write block 2286 len 12288 data: 4 4 3
9600 296/300 Write block 932 len 8192 data: 3 3
9601 297/300 Read block 4049 len 40960
9602 298/300 Read block 1361 len 16384
9603 299/300 Write block 2956 len 12288 data: 3 2 3
9604 300/300 Write block 1239 len 40960 data: 2 2 3 3 3 3 3 3 3 3
9605 301/300 Write block 47 len 40960 data: 4 4 4 3 3 3 3 3 2 3
9606 Main task has work to do, go find it
9607 got dsc Start(1)
9608 start 1
9609 Nov 26 22:06:34.579 INFO request completed, latency_us: 204, response_code: 204, uri: /start/cid/1, method: POST, req_id: aa7fc004-01b8-47fc-9783-a1c9323ed9e4, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
9610 Replay: started 1, returned:Ok(())
9611 ----------------------------------------------------------------
9612 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
9613 States: Active Offline Active
9614 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
9615 3416 Acked Read 10 Done Sent Done false
9616 3417 Acked Write 4 Done Sent Done false
9617 3418 Acked Read 9 Done Sent Done false
9618 3419 Acked Read 7 Done Sent Done false
9619 3420 Acked Write 9 Done Sent Done false
9620 3421 Acked Read 8 Done Sent Done false
9621 3422 Acked Read 2 Done Sent Done false
9622 3423 Acked Write 9 Done Sent Done false
9623 3424 Acked Read 10 Done Sent Done false
9624 3425 Acked Write 4 Done Sent Done false
9625 3426 Acked Write 2 Done Sent Done false
9626 3427 Acked Write 3 Done Sent Done false
9627 3428 Acked Read 8 Done Sent Done false
9628 3429 Acked Write 10 Done Sent Done false
9629 3430 Acked Read 8 Done Sent Done false
9630 3431 Acked Write 10 Done Sent Done false
9631 3432 Acked Read 2 Done Sent Done false
9632 3433 Acked Read 1 Done Sent Done false
9633 3434 Acked Read 1 Done Sent Done false
9634 3435 Acked Flush 0 Done Sent Done false
9635 3436 Acked Read 3 Done Sent Done false
9636 3437 Acked Read 10 Done Sent Done false
9637 3438 Acked Read 2 Done Sent Done false
9638 3439 Acked Write 9 Done Sent Done false
9639 3440 Acked Flush 0 Done Sent Done false
9640 3441 Acked Read 5 Done Sent Done false
9641 3442 Acked Write 4 Done Sent Done false
9642 3443 Acked Read 8 Done Sent Done false
9643 3444 Acked Write 9 Done Sent Done false
9644 3445 Acked Read 5 Done Sent Done false
9645 3446 Acked Write 3 Done Sent Done false
9646 3447 Acked Flush 0 Done Sent Done false
9647 3448 Acked Write 2 Done Sent Done false
9648 3449 Acked Write 8 Done Sent Done false
9649 3450 Acked Write 10 Done Sent Done false
9650 3451 Acked Write 6 Done Sent Done false
9651 3452 Acked Write 3 Done Sent Done false
9652 3453 Acked Flush 0 Done Sent Done false
9653 3454 Acked Flush 0 Done Sent Done false
9654 3455 Acked Read 1 Done Sent Done false
9655 3456 Acked Write 3 Done Sent Done false
9656 3457 Acked Flush 0 Done Sent Done false
9657 3458 Acked Read 10 Done Sent Done false
9658 3459 Acked Read 8 Done Sent Done false
9659 3460 Acked Flush 0 Done Sent Done false
9660 3461 Acked Read 4 Done Sent Done false
9661 3462 Acked Write 9 Done Sent Done false
9662 3463 Acked Write 5 Done Sent Done false
9663 3464 Acked Read 6 Done Sent Done false
9664 3465 Acked Read 5 Done Sent Done false
9665 3466 Acked Flush 0 Done Sent Done false
9666 3467 Acked Read 3 Done Sent Done false
9667 3468 Acked Read 1 Done Sent Done false
9668 3469 Acked Read 7 Done Sent Done false
9669 3470 Acked Read 4 Done Sent Done false
9670 3471 Acked Flush 0 Done Sent Done false
9671 3472 Acked Flush 0 Done Sent Done false
9672 3473 Acked Write 1 Done Sent Done false
9673 3474 Acked Read 6 Done Sent Done false
9674 3475 Acked Write 5 Done Sent Done false
9675 3476 Acked Write 7 Done Sent Done false
9676 3477 Acked Flush 0 Done Sent Done false
9677 3478 Acked Write 2 Done Sent Done false
9678 3479 Acked Read 10 Done Sent Done false
9679 3480 Acked Read 5 Done Sent Done false
9680 3481 Acked Write 10 Done Sent Done false
9681 3482 Acked Read 5 Done Sent Done false
9682 3483 Acked Write 10 Done Sent Done false
9683 3484 Acked Write 6 Done Sent Done false
9684 3485 Acked Write 4 Done Sent Done false
9685 3486 Acked Flush 0 Done Sent Done false
9686 3487 Acked Read 3 Done Sent Done false
9687 3488 Acked Write 3 Done Sent Done false
9688 3489 Acked Read 6 Done Sent Done false
9689 3490 Acked Write 6 Done Sent Done false
9690 3491 Acked Write 7 Done Sent Done false
9691 3492 Acked Write 9 Done Sent Done false
9692 3493 Acked Read 3 Done Sent Done false
9693 3494 Acked Read 8 Done Sent Done false
9694 3495 Acked Read 10 Done Sent Done false
9695 3496 Acked Write 7 Done Sent Done false
9696 3497 Acked Read 5 Done Sent Done false
9697 3498 Acked Flush 0 Done Sent Done false
9698 3499 Acked Flush 0 Done Sent Done false
9699 3500 Acked Read 10 Done Sent Done false
9700 3501 Acked Read 9 Done Sent Done false
9701 3502 Acked Flush 0 Done Sent Done false
9702 3503 Acked Write 8 Done Sent Done false
9703 3504 Acked Read 3 Done Sent Done false
9704 3505 Acked Write 1 Done Sent Done false
9705 3506 Acked Write 9 Done Sent Done false
9706 3507 Acked Flush 0 Done Sent Done false
9707 3508 Acked Flush 0 Done Sent Done false
9708 3509 Acked Flush 0 Done Sent Done false
9709 3510 Acked Read 2 Done Sent Done false
9710 3511 Acked Read 7 Done Sent Done false
9711 3512 Acked Read 5 Done Sent Done false
9712 3513 Acked Write 3 Done Sent Done false
9713 3514 Acked Read 4 Done Sent Done false
9714 3515 Acked Read 3 Done Sent Done false
9715 3516 Acked Read 4 Done Sent Done false
9716 3517 Acked Read 8 Done Sent Done false
9717 3518 Acked Read 5 Done Sent Done false
9718 3519 Acked Write 7 Done Sent Done false
9719 3520 Acked Read 7 Done Sent Done false
9720 3521 Acked Read 2 Done Sent Done false
9721 3522 Acked Write 8 Done Sent Done false
9722 3523 Acked Read 4 Done Sent Done false
9723 3524 Acked Read 4 Done Sent Done false
9724 3525 Acked Write 3 Done Sent Done false
9725 3526 Acked Write 1 Done Sent Done false
9726 3527 Acked Write 1 Done Sent Done false
9727 3528 Acked Read 2 Done Sent Done false
9728 3529 Acked Write 3 Done Sent Done false
9729 3530 Acked Write 7 Done Sent Done false
9730 3531 Acked Write 7 Done Sent Done false
9731 3532 Acked Read 7 Done Sent Done false
9732 3533 Acked Read 9 Done Sent Done false
9733 3534 Acked Read 6 Done Sent Done false
9734 3535 Acked Write 4 Done Sent Done false
9735 3536 Acked Read 6 Done Sent Done false
9736 3537 Acked Write 10 Done Sent Done false
9737 3538 Acked Flush 0 Done Sent Done false
9738 3539 Acked Read 1 Done Sent Done false
9739 3540 Acked Read 3 Done Sent Done false
9740 3541 Acked Read 1 Done Sent Done false
9741 3542 Acked Read 10 Done Sent Done false
9742 3543 Acked Write 5 Done Sent Done false
9743 3544 Acked Read 6 Done Sent Done false
9744 3545 Acked Read 3 Done Sent Done false
9745 3546 Acked Read 3 Done Sent Done false
9746 3547 Acked Read 7 Done Sent Done false
9747 3548 Acked Read 3 Done Sent Done false
9748 3549 Acked Read 8 Done Sent Done false
9749 3550 Acked Write 2 Done Sent Done false
9750 3551 Acked Read 1 Done Sent Done false
9751 3552 Acked Write 6 Done Sent Done false
9752 3553 Acked Write 1 Done Sent Done false
9753 3554 Acked Write 4 Done Sent Done false
9754 3555 Acked Read 8 Done Sent Done false
9755 3556 Acked Write 4 Done Sent Done false
9756 3557 Acked Write 2 Done Sent Done false
9757 3558 Acked Read 5 Done Sent Done false
9758 3559 Acked Flush 0 Done Sent Done false
9759 3560 Acked Write 8 Done Sent Done false
9760 3561 Acked Write 9 Done Sent Done false
9761 3562 Acked Read 7 Done Sent Done false
9762 3563 Acked Write 3 Done Sent Done false
9763 3564 Acked Read 1 Done Sent Done false
9764 3565 Acked Write 8 Done Sent Done false
9765 3566 Acked Read 2 Done Sent Done false
9766 3567 Acked Read 4 Done Sent Done false
9767 3568 Acked Write 8 Done Sent Done false
9768 3569 Acked Write 6 Done Sent Done false
9769 3570 Acked Read 2 Done Sent Done false
9770 3571 Acked Read 4 Done Sent Done false
9771 3572 Acked Write 1 Done Sent Done false
9772 3573 Acked Flush 0 Done Sent Done false
9773 3574 Acked Read 6 Done Sent Done false
9774 3575 Acked Read 6 Done Sent Done false
9775 3576 Acked Read 7 Done Sent Done false
9776 3577 Acked Read 7 Done Sent Done false
9777 3578 Acked Read 2 Done Sent Done false
9778 3579 Acked Write 2 Done Sent Done false
9779 3580 Acked Write 5 Done Sent Done false
9780 3581 Acked Read 6 Done Sent Done false
9781 3582 Acked Write 3 Done Sent Done false
9782 3583 Acked Write 10 Done Sent Done false
9783 3584 Acked Read 2 Done Sent Done false
9784 3585 Acked Read 5 Done Sent Done false
9785 3586 Acked Read 6 Done Sent Done false
9786 3587 Acked Flush 0 Done Sent Done false
9787 3588 Acked Write 8 Done Sent Done false
9788 3589 Acked Read 7 Done Sent Done false
9789 3590 Acked Read 9 Done Sent Done false
9790 3591 Acked Read 5 Done Sent Done false
9791 3592 Acked Write 3 Done Sent Done false
9792 3593 Acked Write 4 Done Sent Done false
9793 3594 Acked Flush 0 Done Sent Done false
9794 3595 Acked Read 6 Done Sent Done false
9795 3596 Acked Read 3 Done Sent Done false
9796 3597 Acked Read 4 Done Sent Done false
9797 3598 Acked Write 8 Done Sent Done false
9798 3599 Acked Read 9 Done Sent Done false
9799 3600 Acked Write 2 Done Sent Done false
9800 3601 Acked Write 5 Done Sent Done false
9801 3602 Acked Write 9 Done Sent Done false
9802 3603 Acked Read 4 Done Sent Done false
9803 3604 Acked Write 3 Done Sent Done false
9804 3605 Acked Write 4 Done Sent Done false
9805 3606 Acked Read 4 Done Sent Done false
9806 3607 Acked Read 2 Done Sent Done false
9807 3608 Acked Write 4 Done Sent Done false
9808 3609 Acked Read 2 Done Sent Done false
9809 3610 Acked Write 6 Done Sent Done false
9810 3611 Acked Write 6 Done Sent Done false
9811 3612 Acked Read 9 Done Sent Done false
9812 3613 Acked Read 6 Done Sent Done false
9813 3614 Acked Read 5 Done Sent Done false
9814 3615 Acked Write 1 Done Sent Done false
9815 3616 Acked Read 1 Done Sent Done false
9816 3617 Acked Write 6 Done Sent Done false
9817 3618 Acked Write 2 Done Sent Done false
9818 3619 Acked Read 5 Done Sent Done false
9819 3620 Acked Write 9 Done Sent Done false
9820 3621 Acked Flush 0 Done Sent Done false
9821 3622 Acked Write 5 Done Sent Done false
9822 3623 Acked Write 7 Done Sent Done false
9823 3624 Acked Flush 0 Done Sent Done false
9824 3625 Acked Read 2 Done Sent Done false
9825 3626 Acked Write 3 Done Sent Done false
9826 3627 Acked Write 2 Done Sent Done false
9827 3628 Acked Write 9 Done Sent Done false
9828 3629 Acked Write 6 Done Sent Done false
9829 3630 Acked Flush 0 Done Sent Done false
9830 3631 Acked Read 10 Done Sent Done false
9831 3632 Acked Write 5 Done Sent Done false
9832 3633 Acked Write 4 Done Sent Done false
9833 3634 Acked Read 7 Done Sent Done false
9834 3635 Acked Write 3 Done Sent Done false
9835 3636 Acked Flush 0 Done Sent Done false
9836 3637 Acked Read 5 Done Sent Done false
9837 3638 Acked Read 8 Done Sent Done false
9838 3639 Acked Write 10 Done Sent Done false
9839 3640 Acked Read 4 Done Sent Done false
9840 3641 Acked Read 7 Done Sent Done false
9841 3642 Acked Read 5 Done Sent Done false
9842 3643 Acked Read 3 Done Sent Done false
9843 3644 Acked Write 3 Done Sent Done false
9844 3645 Acked Read 5 Done Sent Done false
9845 3646 Acked Read 1 Done Sent Done false
9846 3647 Acked Read 9 Done Sent Done false
9847 3648 Acked Read 7 Done Sent Done false
9848 3649 Acked Write 10 Done Sent Done false
9849 3650 Acked Write 2 Done Sent Done false
9850 3651 Acked Write 6 Done Sent Done false
9851 3652 Acked Read 8 Done Sent Done false
9852 3653 Acked Write 9 Done Sent Done false
9853 3654 Acked Read 1 Done Sent Done false
9854 3655 Acked Write 9 Done Sent Done false
9855 3656 Acked Read 6 Done Sent Done false
9856 3657 Acked Write 4 Done Sent Done false
9857 3658 Acked Read 2 Done Sent Done false
9858 3659 Acked Write 2 Done Sent Done false
9859 3660 Acked Read 9 Done Sent Done false
9860 3661 Acked Read 10 Done Sent Done false
9861 3662 Acked Read 6 Done Sent Done false
9862 3663 Acked Read 6 Done Sent Done false
9863 3664 Acked Write 9 Done Sent Done false
9864 3665 Acked Write 8 Done Sent Done false
9865 3666 Acked Write 1 Done Sent Done false
9866 3667 Acked Read 9 Done Sent Done false
9867 3668 Acked Read 1 Done Sent Done false
9868 3669 Acked Read 8 Done Sent Done false
9869 3670 Acked Write 9 Done Sent Done false
9870 3671 Acked Write 8 Done Sent Done false
9871 3672 Acked Write 10 Done Sent Done false
9872 3673 Acked Read 3 Done Sent Done false
9873 3674 Acked Read 6 Done Sent Done false
9874 3675 Acked Write 6 Done Sent Done false
9875 3676 Acked Read 3 Done Sent Done false
9876 3677 Acked Read 7 Done Sent Done false
9877 3678 Acked Write 7 Done Sent Done false
9878 3679 Acked Write 10 Done Sent Done false
9879 3680 Acked Read 7 Done Sent Done false
9880 3681 Acked Flush 0 Done Sent Done false
9881 3682 Acked Read 8 Done Sent Done false
9882 3683 Acked Read 5 Done Sent Done false
9883 3684 Acked Read 6 Done Sent Done false
9884 3685 Acked Write 10 Done Sent Done false
9885 3686 Acked Read 3 Done Sent Done false
9886 3687 Acked Flush 0 Done Sent Done false
9887 3688 Acked Write 4 Done Sent Done false
9888 3689 Acked Write 3 Done Sent Done false
9889 3690 Acked Write 1 Done Sent Done false
9890 3691 Acked Read 9 Done Sent Done false
9891 3692 Acked Write 6 Done Sent Done false
9892 3693 Acked Read 10 Done Sent Done false
9893 3694 Acked Read 7 Done Sent Done false
9894 3695 Acked Write 6 Done Sent Done false
9895 3696 Acked Write 1 Done Sent Done false
9896 3697 Acked Write 9 Done Sent Done false
9897 3698 Acked Flush 0 Done Sent Done false
9898 3699 Acked Flush 0 Done Sent Done false
9899 3700 Acked Write 3 Done Sent Done false
9900 3701 Acked Read 9 Done Sent Done false
9901 3702 Acked Read 5 Done Sent Done false
9902 3703 Acked Read 4 Done Sent Done false
9903 3704 Acked Write 4 Done Sent Done false
9904 3705 Acked Write 2 Done Sent Done false
9905 3706 Acked Read 10 Done Sent Done false
9906 3707 Acked Flush 0 Done Sent Done false
9907 3708 Acked Read 7 Done Sent Done false
9908 3709 Acked Write 2 Done Sent Done false
9909 3710 Acked Write 3 Done Sent Done false
9910 3711 Acked Write 2 Done Sent Done false
9911 3712 Acked Read 10 Done Sent Done false
9912 3713 Acked Read 4 Done Sent Done false
9913 3714 Acked Write 3 Done Sent Done false
9914 3715 Acked Write 10 Done Sent Done false
9915 3716 Acked Write 10 Sent Sent Done false
9916 STATES DS:0 DS:1 DS:2 TOTAL
9917 Sent 1 301 0 302
9918 Done 300 0 301 601
9919 Skipped 0 0 0 0
9920 Error 0 0 0 0
9921 Last Flush: 3707 3415 3707
9922 Downstairs last five completed: 3415 3414 3413 3412 3411
9923 Upstairs last five completed: 3716 3715 3714 3713 3712
9924 CLIENT: Up:0 ds:301 act:2
9925 [8820] got start action
9926 [8820] I am going to restart
9927 Starting downstairs at port 8820
9928 Make output file at "/tmp/dsc/downstairs-8820.txt"
9929 [8820][1] reports Starting
9930 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1212)
9931 [8820][1] reports Running
9932 ----------------------------------------------------------------
9933 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
9934 States: Active Offline Active
9935 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
9936 3416 Acked Read 10 Done Sent Done false
9937 3417 Acked Write 4 Done Sent Done false
9938 3418 Acked Read 9 Done Sent Done false
9939 3419 Acked Read 7 Done Sent Done false
9940 3420 Acked Write 9 Done Sent Done false
9941 3421 Acked Read 8 Done Sent Done false
9942 3422 Acked Read 2 Done Sent Done false
9943 3423 Acked Write 9 Done Sent Done false
9944 3424 Acked Read 10 Done Sent Done false
9945 3425 Acked Write 4 Done Sent Done false
9946 3426 Acked Write 2 Done Sent Done false
9947 3427 Acked Write 3 Done Sent Done false
9948 3428 Acked Read 8 Done Sent Done false
9949 3429 Acked Write 10 Done Sent Done false
9950 3430 Acked Read 8 Done Sent Done false
9951 3431 Acked Write 10 Done Sent Done false
9952 3432 Acked Read 2 Done Sent Done false
9953 3433 Acked Read 1 Done Sent Done false
9954 3434 Acked Read 1 Done Sent Done false
9955 3435 Acked Flush 0 Done Sent Done false
9956 3436 Acked Read 3 Done Sent Done false
9957 3437 Acked Read 10 Done Sent Done false
9958 3438 Acked Read 2 Done Sent Done false
9959 3439 Acked Write 9 Done Sent Done false
9960 3440 Acked Flush 0 Done Sent Done false
9961 3441 Acked Read 5 Done Sent Done false
9962 3442 Acked Write 4 Done Sent Done false
9963 3443 Acked Read 8 Done Sent Done false
9964 3444 Acked Write 9 Done Sent Done false
9965 3445 Acked Read 5 Done Sent Done false
9966 3446 Acked Write 3 Done Sent Done false
9967 3447 Acked Flush 0 Done Sent Done false
9968 3448 Acked Write 2 Done Sent Done false
9969 3449 Acked Write 8 Done Sent Done false
9970 3450 Acked Write 10 Done Sent Done false
9971 3451 Acked Write 6 Done Sent Done false
9972 3452 Acked Write 3 Done Sent Done false
9973 3453 Acked Flush 0 Done Sent Done false
9974 3454 Acked Flush 0 Done Sent Done false
9975 3455 Acked Read 1 Done Sent Done false
9976 3456 Acked Write 3 Done Sent Done false
9977 3457 Acked Flush 0 Done Sent Done false
9978 3458 Acked Read 10 Done Sent Done false
9979 3459 Acked Read 8 Done Sent Done false
9980 3460 Acked Flush 0 Done Sent Done false
9981 3461 Acked Read 4 Done Sent Done false
9982 3462 Acked Write 9 Done Sent Done false
9983 3463 Acked Write 5 Done Sent Done false
9984 3464 Acked Read 6 Done Sent Done false
9985 3465 Acked Read 5 Done Sent Done false
9986 3466 Acked Flush 0 Done Sent Done false
9987 3467 Acked Read 3 Done Sent Done false
9988 3468 Acked Read 1 Done Sent Done false
9989 3469 Acked Read 7 Done Sent Done false
9990 3470 Acked Read 4 Done Sent Done false
9991 3471 Acked Flush 0 Done Sent Done false
9992 3472 Acked Flush 0 Done Sent Done false
9993 3473 Acked Write 1 Done Sent Done false
9994 3474 Acked Read 6 Done Sent Done false
9995 3475 Acked Write 5 Done Sent Done false
9996 3476 Acked Write 7 Done Sent Done false
9997 3477 Acked Flush 0 Done Sent Done false
9998 3478 Acked Write 2 Done Sent Done false
9999 3479 Acked Read 10 Done Sent Done false
10000 3480 Acked Read 5 Done Sent Done false
10001 3481 Acked Write 10 Done Sent Done false
10002 3482 Acked Read 5 Done Sent Done false
10003 3483 Acked Write 10 Done Sent Done false
10004 3484 Acked Write 6 Done Sent Done false
10005 3485 Acked Write 4 Done Sent Done false
10006 3486 Acked Flush 0 Done Sent Done false
10007 3487 Acked Read 3 Done Sent Done false
10008 3488 Acked Write 3 Done Sent Done false
10009 3489 Acked Read 6 Done Sent Done false
10010 3490 Acked Write 6 Done Sent Done false
10011 3491 Acked Write 7 Done Sent Done false
10012 3492 Acked Write 9 Done Sent Done false
10013 3493 Acked Read 3 Done Sent Done false
10014 3494 Acked Read 8 Done Sent Done false
10015 3495 Acked Read 10 Done Sent Done false
10016 3496 Acked Write 7 Done Sent Done false
10017 3497 Acked Read 5 Done Sent Done false
10018 3498 Acked Flush 0 Done Sent Done false
10019 3499 Acked Flush 0 Done Sent Done false
10020 3500 Acked Read 10 Done Sent Done false
10021 3501 Acked Read 9 Done Sent Done false
10022 3502 Acked Flush 0 Done Sent Done false
10023 3503 Acked Write 8 Done Sent Done false
10024 3504 Acked Read 3 Done Sent Done false
10025 3505 Acked Write 1 Done Sent Done false
10026 3506 Acked Write 9 Done Sent Done false
10027 3507 Acked Flush 0 Done Sent Done false
10028 3508 Acked Flush 0 Done Sent Done false
10029 3509 Acked Flush 0 Done Sent Done false
10030 3510 Acked Read 2 Done Sent Done false
10031 3511 Acked Read 7 Done Sent Done false
10032 3512 Acked Read 5 Done Sent Done false
10033 3513 Acked Write 3 Done Sent Done false
10034 3514 Acked Read 4 Done Sent Done false
10035 3515 Acked Read 3 Done Sent Done false
10036 3516 Acked Read 4 Done Sent Done false
10037 3517 Acked Read 8 Done Sent Done false
10038 3518 Acked Read 5 Done Sent Done false
10039 3519 Acked Write 7 Done Sent Done false
10040 3520 Acked Read 7 Done Sent Done false
10041 3521 Acked Read 2 Done Sent Done false
10042 3522 Acked Write 8 Done Sent Done false
10043 3523 Acked Read 4 Done Sent Done false
10044 3524 Acked Read 4 Done Sent Done false
10045 3525 Acked Write 3 Done Sent Done false
10046 3526 Acked Write 1 Done Sent Done false
10047 3527 Acked Write 1 Done Sent Done false
10048 3528 Acked Read 2 Done Sent Done false
10049 3529 Acked Write 3 Done Sent Done false
10050 3530 Acked Write 7 Done Sent Done false
10051 3531 Acked Write 7 Done Sent Done false
10052 3532 Acked Read 7 Done Sent Done false
10053 3533 Acked Read 9 Done Sent Done false
10054 3534 Acked Read 6 Done Sent Done false
10055 3535 Acked Write 4 Done Sent Done false
10056 3536 Acked Read 6 Done Sent Done false
10057 3537 Acked Write 10 Done Sent Done false
10058 3538 Acked Flush 0 Done Sent Done false
10059 3539 Acked Read 1 Done Sent Done false
10060 3540 Acked Read 3 Done Sent Done false
10061 3541 Acked Read 1 Done Sent Done false
10062 3542 Acked Read 10 Done Sent Done false
10063 3543 Acked Write 5 Done Sent Done false
10064 3544 Acked Read 6 Done Sent Done false
10065 3545 Acked Read 3 Done Sent Done false
10066 3546 Acked Read 3 Done Sent Done false
10067 3547 Acked Read 7 Done Sent Done false
10068 3548 Acked Read 3 Done Sent Done false
10069 3549 Acked Read 8 Done Sent Done false
10070 3550 Acked Write 2 Done Sent Done false
10071 3551 Acked Read 1 Done Sent Done false
10072 3552 Acked Write 6 Done Sent Done false
10073 3553 Acked Write 1 Done Sent Done false
10074 3554 Acked Write 4 Done Sent Done false
10075 3555 Acked Read 8 Done Sent Done false
10076 3556 Acked Write 4 Done Sent Done false
10077 3557 Acked Write 2 Done Sent Done false
10078 3558 Acked Read 5 Done Sent Done false
10079 3559 Acked Flush 0 Done Sent Done false
10080 3560 Acked Write 8 Done Sent Done false
10081 3561 Acked Write 9 Done Sent Done false
10082 3562 Acked Read 7 Done Sent Done false
10083 3563 Acked Write 3 Done Sent Done false
10084 3564 Acked Read 1 Done Sent Done false
10085 3565 Acked Write 8 Done Sent Done false
10086 3566 Acked Read 2 Done Sent Done false
10087 3567 Acked Read 4 Done Sent Done false
10088 3568 Acked Write 8 Done Sent Done false
10089 3569 Acked Write 6 Done Sent Done false
10090 3570 Acked Read 2 Done Sent Done false
10091 3571 Acked Read 4 Done Sent Done false
10092 3572 Acked Write 1 Done Sent Done false
10093 3573 Acked Flush 0 Done Sent Done false
10094 3574 Acked Read 6 Done Sent Done false
10095 3575 Acked Read 6 Done Sent Done false
10096 3576 Acked Read 7 Done Sent Done false
10097 3577 Acked Read 7 Done Sent Done false
10098 3578 Acked Read 2 Done Sent Done false
10099 3579 Acked Write 2 Done Sent Done false
10100 3580 Acked Write 5 Done Sent Done false
10101 3581 Acked Read 6 Done Sent Done false
10102 3582 Acked Write 3 Done Sent Done false
10103 3583 Acked Write 10 Done Sent Done false
10104 3584 Acked Read 2 Done Sent Done false
10105 3585 Acked Read 5 Done Sent Done false
10106 3586 Acked Read 6 Done Sent Done false
10107 3587 Acked Flush 0 Done Sent Done false
10108 3588 Acked Write 8 Done Sent Done false
10109 3589 Acked Read 7 Done Sent Done false
10110 3590 Acked Read 9 Done Sent Done false
10111 3591 Acked Read 5 Done Sent Done false
10112 3592 Acked Write 3 Done Sent Done false
10113 3593 Acked Write 4 Done Sent Done false
10114 3594 Acked Flush 0 Done Sent Done false
10115 3595 Acked Read 6 Done Sent Done false
10116 3596 Acked Read 3 Done Sent Done false
10117 3597 Acked Read 4 Done Sent Done false
10118 3598 Acked Write 8 Done Sent Done false
10119 3599 Acked Read 9 Done Sent Done false
10120 3600 Acked Write 2 Done Sent Done false
10121 3601 Acked Write 5 Done Sent Done false
10122 3602 Acked Write 9 Done Sent Done false
10123 3603 Acked Read 4 Done Sent Done false
10124 3604 Acked Write 3 Done Sent Done false
10125 3605 Acked Write 4 Done Sent Done false
10126 3606 Acked Read 4 Done Sent Done false
10127 3607 Acked Read 2 Done Sent Done false
10128 3608 Acked Write 4 Done Sent Done false
10129 3609 Acked Read 2 Done Sent Done false
10130 3610 Acked Write 6 Done Sent Done false
10131 3611 Acked Write 6 Done Sent Done false
10132 3612 Acked Read 9 Done Sent Done false
10133 3613 Acked Read 6 Done Sent Done false
10134 3614 Acked Read 5 Done Sent Done false
10135 3615 Acked Write 1 Done Sent Done false
10136 3616 Acked Read 1 Done Sent Done false
10137 3617 Acked Write 6 Done Sent Done false
10138 3618 Acked Write 2 Done Sent Done false
10139 3619 Acked Read 5 Done Sent Done false
10140 3620 Acked Write 9 Done Sent Done false
10141 3621 Acked Flush 0 Done Sent Done false
10142 3622 Acked Write 5 Done Sent Done false
10143 3623 Acked Write 7 Done Sent Done false
10144 3624 Acked Flush 0 Done Sent Done false
10145 3625 Acked Read 2 Done Sent Done false
10146 3626 Acked Write 3 Done Sent Done false
10147 3627 Acked Write 2 Done Sent Done false
10148 3628 Acked Write 9 Done Sent Done false
10149 3629 Acked Write 6 Done Sent Done false
10150 3630 Acked Flush 0 Done Sent Done false
10151 3631 Acked Read 10 Done Sent Done false
10152 3632 Acked Write 5 Done Sent Done false
10153 3633 Acked Write 4 Done Sent Done false
10154 3634 Acked Read 7 Done Sent Done false
10155 3635 Acked Write 3 Done Sent Done false
10156 3636 Acked Flush 0 Done Sent Done false
10157 3637 Acked Read 5 Done Sent Done false
10158 3638 Acked Read 8 Done Sent Done false
10159 3639 Acked Write 10 Done Sent Done false
10160 3640 Acked Read 4 Done Sent Done false
10161 3641 Acked Read 7 Done Sent Done false
10162 3642 Acked Read 5 Done Sent Done false
10163 3643 Acked Read 3 Done Sent Done false
10164 3644 Acked Write 3 Done Sent Done false
10165 3645 Acked Read 5 Done Sent Done false
10166 3646 Acked Read 1 Done Sent Done false
10167 3647 Acked Read 9 Done Sent Done false
10168 3648 Acked Read 7 Done Sent Done false
10169 3649 Acked Write 10 Done Sent Done false
10170 3650 Acked Write 2 Done Sent Done false
10171 3651 Acked Write 6 Done Sent Done false
10172 3652 Acked Read 8 Done Sent Done false
10173 3653 Acked Write 9 Done Sent Done false
10174 3654 Acked Read 1 Done Sent Done false
10175 3655 Acked Write 9 Done Sent Done false
10176 3656 Acked Read 6 Done Sent Done false
10177 3657 Acked Write 4 Done Sent Done false
10178 3658 Acked Read 2 Done Sent Done false
10179 3659 Acked Write 2 Done Sent Done false
10180 3660 Acked Read 9 Done Sent Done false
10181 3661 Acked Read 10 Done Sent Done false
10182 3662 Acked Read 6 Done Sent Done false
10183 3663 Acked Read 6 Done Sent Done false
10184 3664 Acked Write 9 Done Sent Done false
10185 3665 Acked Write 8 Done Sent Done false
10186 3666 Acked Write 1 Done Sent Done false
10187 3667 Acked Read 9 Done Sent Done false
10188 3668 Acked Read 1 Done Sent Done false
10189 3669 Acked Read 8 Done Sent Done false
10190 3670 Acked Write 9 Done Sent Done false
10191 3671 Acked Write 8 Done Sent Done false
10192 3672 Acked Write 10 Done Sent Done false
10193 3673 Acked Read 3 Done Sent Done false
10194 3674 Acked Read 6 Done Sent Done false
10195 3675 Acked Write 6 Done Sent Done false
10196 3676 Acked Read 3 Done Sent Done false
10197 3677 Acked Read 7 Done Sent Done false
10198 3678 Acked Write 7 Done Sent Done false
10199 3679 Acked Write 10 Done Sent Done false
10200 3680 Acked Read 7 Done Sent Done false
10201 3681 Acked Flush 0 Done Sent Done false
10202 3682 Acked Read 8 Done Sent Done false
10203 3683 Acked Read 5 Done Sent Done false
10204 3684 Acked Read 6 Done Sent Done false
10205 3685 Acked Write 10 Done Sent Done false
10206 3686 Acked Read 3 Done Sent Done false
10207 3687 Acked Flush 0 Done Sent Done false
10208 3688 Acked Write 4 Done Sent Done false
10209 3689 Acked Write 3 Done Sent Done false
10210 3690 Acked Write 1 Done Sent Done false
10211 3691 Acked Read 9 Done Sent Done false
10212 3692 Acked Write 6 Done Sent Done false
10213 3693 Acked Read 10 Done Sent Done false
10214 3694 Acked Read 7 Done Sent Done false
10215 3695 Acked Write 6 Done Sent Done false
10216 3696 Acked Write 1 Done Sent Done false
10217 3697 Acked Write 9 Done Sent Done false
10218 3698 Acked Flush 0 Done Sent Done false
10219 3699 Acked Flush 0 Done Sent Done false
10220 3700 Acked Write 3 Done Sent Done false
10221 3701 Acked Read 9 Done Sent Done false
10222 3702 Acked Read 5 Done Sent Done false
10223 3703 Acked Read 4 Done Sent Done false
10224 3704 Acked Write 4 Done Sent Done false
10225 3705 Acked Write 2 Done Sent Done false
10226 3706 Acked Read 10 Done Sent Done false
10227 3707 Acked Flush 0 Done Sent Done false
10228 3708 Acked Read 7 Done Sent Done false
10229 3709 Acked Write 2 Done Sent Done false
10230 3710 Acked Write 3 Done Sent Done false
10231 3711 Acked Write 2 Done Sent Done false
10232 3712 Acked Read 10 Done Sent Done false
10233 3713 Acked Read 4 Done Sent Done false
10234 3714 Acked Write 3 Done Sent Done false
10235 3715 Acked Write 10 Done Sent Done false
10236 3716 Acked Write 10 Done Sent Done false
10237 3717 Acked Flush 0 Done Sent Done false
10238 STATES DS:0 DS:1 DS:2 TOTAL
10239 Sent 0 302 0 302
10240 Done 302 0 302 604
10241 Skipped 0 0 0 0
10242 Error 0 0 0 0
10243 Last Flush: 3717 3415 3717
10244 Downstairs last five completed: 3415 3414 3413 3412 3411
10245 Upstairs last five completed: 3717 3716 3715 3714 3713
10246 CLIENT: Up:0 ds:302 act:2
102472024-11-26T22:06:40.345ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
102482024-11-26T22:06:40.345ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:64502) = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
102492024-11-26T22:06:40.346ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
102502024-11-26T22:06:40.347ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
102512024-11-26T22:06:40.347ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
102522024-11-26T22:06:40.347ZINFOcrucible: send last flush ID to this DS: 3415 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
102532024-11-26T22:06:40.347ZINFOcrucible: Replied this last flush ID: 3415 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
102542024-11-26T22:06:40.347ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
102552024-11-26T22:06:40.347ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
102562024-11-26T22:06:40.347ZINFOcrucible: [1] client re-new 302 jobs since flush 3415 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
102572024-11-26T22:06:40.347ZINFOcrucible: [1] Replayed 302 jobs since flush: 3415 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
10258 ----------------------------------------------------------------
10259 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
10260 Downstairs last five completed: 3717 3716 3715 3714 3713
10261 Upstairs last five completed: 3717 3716 3715 3714 3713
10262 CLIENT: Up:0 ds:0 act:3
10263 Replay: All jobs finished
10264 Main task has work to do, go find it
10265 got dsc Stop(2)
10266 stop 2
10267 Nov 26 22:06:42.584 INFO request completed, latency_us: 374[8830] Got stop action so:false kr:false
10268 , response_code: 204, uri: /stop/cid/2, method: POST, req_id: 11ac3c94-e838-4741-b6fc-cb91730a51b4, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
102692024-11-26T22:06:42.585ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
10270 Nov 26 22:06:42.585 INFO request completed, {"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2024-11-26T22:06:42.585667466Z","hostname":"w-01JDN7ZQ17ABQG15KHKN3F2D26","pid":1203,"":"io task","client":"2","":"downstairs","session_id":"86a36c17-3718-484c-87fd-2b8a1dc2ab18"}
10271 latency_us: 111, response_code: 200, uri: /state/cid/2, method:{"msg":"downstairs task for 2 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2024-11-26T22:06:42.585736127Z","hostname":"w-01JDN7ZQ17ABQG15KHKN3F2D26","pid":1203,"session_id":"86a36c17-3718-484c-87fd-2b8a1dc2ab18"}
102722024-11-26T22:06:42.585ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
10273 GET, req_id: befa8c8e-542b-4e4e-8252-392000eca8ab, remote_addr: 127.0.0.1:36829{"msg":"[2] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-26T22:06:42.585805549Z","hostname":"w-01JDN7ZQ17ABQG15KHKN3F2D26","pid":1203,"client":"2","":"downstairs","session_id":"86a36c17-3718-484c-87fd-2b8a1dc2ab18"}
10274 , local_addr: 127.0.0.1:9998
102752024-11-26T22:06:42.585ZINFOcrucible: client task is exiting = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
10276 [8830] Exited with: ExitStatus(unix_wait_status(9))
10277 [8830][2] reports Exit
10278 Nov 26 22:06:46.587 INFO request completed, latency_us: 208, response_code: 200, uri: /state/cid/2, method: GET, req_id: a6893d05-4069-4779-9fe7-1f6c17c727c7, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
10279 001/300 Read block 4238 len 20480
10280 002/300 Read block 3518 len 36864
10281 003/300 Write block 2853 len 36864 data: 2 2 2 2 2 2 3 3 3
10282 004/300 Write block 1571 len 28672 data: 2 3 3 3 3 3 3
10283 005/300 Write block 3015 len 4096 data: 4
10284 006/300 Read block 2743 len 12288
10285 007/300 Read block 2764 len 36864
10286 008/300 Read block 2958 len 12288
10287 009/300 Read block 3572 len 32768
10288 010/300 Write block 2887 len 16384 data: 4 4 4 4
10289 011/300 Write block 2707 len 12288 data: 3 3 5
10290 012/300 Read block 816 len 32768
10291 013/300 Read block 1412 len 20480
10292 014/300 Write block 4697 len 28672 data: 3 3 2 3 3 4 5
10293 015/300 Write block 2428 len 20480 data: 2 2 2 2 2
10294 016/300 Flush
10295 017/300 Read block 3465 len 20480
10296 018/300 Write block 2728 len 36864 data: 3 3 4 4 3 3 4 4 3
10297 019/300 Read block 2820 len 32768
10298 020/300 Write block 571 len 32768 data: 4 5 4 4 5 3 2 2
10299 021/300 Write block 924 len 4096 data: 3
10300 022/300 Read block 10 len 24576
10301 023/300 Read block 2060 len 24576
10302 024/300 Read block 1729 len 8192
10303 025/300 Write block 751 len 32768 data: 3 3 2 2 3 2 2 3
10304 026/300 Read block 876 len 20480
10305 027/300 Write block 3975 len 32768 data: 2 2 2 2 2 2 2 2
10306 028/300 Read block 2591 len 24576
10307 029/300 Read block 4539 len 16384
10308 030/300 Read block 375 len 8192
10309 031/300 Read block 4936 len 32768
10310 032/300 Write block 4854 len 12288 data: 2 2 3
10311 033/300 Write block 1638 len 4096 data: 2
10312 034/300 Write block 3661 len 40960 data: 3 3 3 4 4 4 4 3 3 3
10313 035/300 Write block 4988 len 32768 data: 4 6 7 7 7 6 6 6
10314 036/300 Read block 1963 len 16384
10315 037/300 Write block 810 len 24576 data: 3 2 3 3 2 2
10316 038/300 Write block 1244 len 20480 data: 4 4 4 4 4
10317 039/300 Read block 4974 len 8192
10318 040/300 Read block 11 len 12288
10319 041/300 Read block 165 len 16384
10320 042/300 Read block 927 len 28672
10321 043/300 Read block 3122 len 12288
10322 044/300 Write block 1611 len 20480 data: 3 3 3 2 2
10323 045/300 Read block 4329 len 12288
10324 046/300 Read block 4675 len 16384
10325 047/300 Flush
10326 048/300 Read block 3995 len 40960
10327 049/300 Read block 2511 len 20480
10328 050/300 Read block 3426 len 8192
10329 051/300 Write block 3745 len 28672 data: 3 3 2 2 3 3 3
10330 052/300 Flush
10331 053/300 Write block 3438 len 28672 data: 2 2 2 2 2 2 2
10332 054/300 Read block 4800 len 40960
10333 055/300 Flush
10334 056/300 Write block 535 len 32768 data: 3 3 4 4 4 3 3 4
10335 057/300 Write block 351 len 24576 data: 5 5 5 5 5 4
10336 058/300 Read block 4119 len 8192
10337 059/300 Read block 4941 len 28672
10338 060/300 Write block 4347 len 8192 data: 3 4
10339 061/300 Write block 2049 len 16384 data: 3 3 3 3
10340 062/300 Read block 1117 len 12288
10341 063/300 Write block 3927 len 36864 data: 3 2 2 2 2 2 2 3 2
10342 064/300 Read block 2223 len 8192
10343 065/300 Write block 2607 len 40960 data: 6 5 4 4 5 4 2 2 2 2
10344 066/300 Read block 1428 len 20480
10345 067/300 Read block 1249 len 36864
10346 068/300 Read block 585 len 28672
10347 069/300 Write block 3374 len 8192 data: 3 3
10348 070/300 Write block 2771 len 16384 data: 3 4 4 3
10349 071/300 Read block 4786 len 40960
10350 072/300 Read block 4816 len 12288
10351 073/300 Write block 2816 len 4096 data: 2
10352 074/300 Read block 1402 len 28672
10353 075/300 Flush
10354 076/300 Read block 1588 len 24576
10355 077/300 Write block 2489 len 20480 data: 5 4 5 5 4
10356 078/300 Read block 4443 len 36864
10357 079/300 Read block 2131 len 32768
10358 080/300 Read block 2062 len 8192
10359 081/300 Read block 522 len 12288
10360 082/300 Flush
10361 083/300 Read block 3794 len 28672
10362 084/300 Write block 2486 len 28672 data: 7 6 6 6 5 6 6
10363 085/300 Read block 4191 len 24576
10364 086/300 Write block 2946 len 8192 data: 2 2
10365 087/300 Write block 3449 len 36864 data: 3 3 3 3 4 3 3 3 3
10366 088/300 Write block 2731 len 8192 data: 5 4
10367 089/300 Read block 3813 len 24576
10368 090/300 Write block 3154 len 28672 data: 3 4 4 3 3 3 5
10369 091/300 Write block 2050 len 36864 data: 4 4 4 2 5 4 4 5 5
10370 092/300 Write block 3670 len 40960 data: 4 4 5 5 5 5 7 7 6 5
10371 093/300 Read block 3407 len 16384
10372 094/300 Write block 4474 len 16384 data: 4 5 4 4
10373 095/300 Read block 1416 len 40960
10374 096/300 Read block 3540 len 40960
10375 097/300 Write block 2625 len 24576 data: 2 3 3 3 3 3
10376 098/300 Read block 15 len 20480
10377 099/300 Read block 2978 len 28672
10378 100/300 Write block 1258 len 8192 data: 3 3
10379 101/300 Read block 2597 len 4096
10380 102/300 Read block 424 len 8192
10381 103/300 Write block 3976 len 32768 data: 3 3 3 3 3 3 3 2
10382 104/300 Write block 3990 len 40960 data: 5 6 6 6 5 5 4 4 3 2
10383 105/300 Read block 2416 len 16384
10384 106/300 Flush
10385 107/300 Read block 3893 len 32768
10386 108/300 Read block 3735 len 24576
10387 109/300 Write block 1098 len 4096 data: 3
10388 110/300 Write block 2043 len 16384 data: 2 2 2 2
10389 111/300 Write block 4147 len 40960 data: 3 3 3 2 2 2 2 2 2 3
10390 112/300 Write block 218 len 24576 data: 2 2 2 2 3 3
10391 113/300 Write block 3681 len 32768 data: 5 6 6 5 4 3 3 4
10392 114/300 Read block 3100 len 8192
10393 115/300 Write block 3124 len 12288 data: 3 3 2
10394 116/300 Read block 1937 len 28672
10395 117/300 Read block 705 len 20480
10396 118/300 Read block 4478 len 4096
10397 119/300 Write block 158 len 24576 data: 3 3 4 4 2 3
10398 120/300 Read block 1139 len 28672
10399 121/300 Read block 687 len 8192
10400 122/300 Flush
10401 123/300 Read block 2389 len 40960
10402 124/300 Write block 369 len 28672 data: 4 3 3 3 3 4 3
10403 125/300 Write block 3351 len 20480 data: 4 3 3 2 2
10404 126/300 Read block 376 len 12288
10405 127/300 Read block 2447 len 40960
10406 128/300 Flush
10407 129/300 Write block 3371 len 36864 data: 2 2 3 4 4 3 2 3 3
10408 130/300 Write block 1760 len 12288 data: 3 4 4
10409 131/300 Read block 2323 len 36864
10410 132/300 Flush
10411 133/300 Read block 4148 len 40960
10412 134/300 Read block 4429 len 16384
10413 135/300 Flush
10414 136/300 Read block 1860 len 20480
10415 137/300 Read block 4374 len 28672
10416 138/300 Flush
10417 139/300 Read block 2037 len 36864
10418 140/300 Read block 1384 len 36864
10419 141/300 Read block 2926 len 20480
10420 142/300 Flush
10421 143/300 Read block 2709 len 36864
10422 144/300 Read block 4442 len 4096
10423 145/300 Write block 4544 len 28672 data: 2 2 2 3 3 3 3
10424 146/300 Write block 3221 len 16384 data: 3 2 2 2
10425 147/300 Read block 625 len 4096
10426 148/300 Write block 2405 len 20480 data: 4 2 2 2 2
10427 149/300 Write block 932 len 12288 data: 4 4 2
10428 150/300 Write block 4255 len 16384 data: 4 3 3 3
10429 151/300 Write block 4416 len 8192 data: 3 3
10430 152/300 Write block 1002 len 28672 data: 2 2 3 3 3 3 3
10431 153/300 Read block 3679 len 40960
10432 154/300 Read block 2473 len 28672
10433 155/300 Write block 1376 len 16384 data: 2 2 3 3
10434 156/300 Read block 4344 len 32768
10435 157/300 Read block 562 len 28672
10436 158/300 Write block 1244 len 16384 data: 5 5 5 5
10437 159/300 Write block 1170 len 28672 data: 3 3 2 2 2 2 2
10438 160/300 Read block 3492 len 40960
10439 161/300 Read block 4039 len 40960
10440 162/300 Write block 4184 len 32768 data: 2 3 3 2 2 2 2 3
10441 163/300 Read block 2431 len 28672
10442 164/300 Read block 4220 len 20480
10443 165/300 Flush
10444 166/300 Read block 677 len 16384
10445 167/300 Write block 2349 len 24576 data: 3 3 4 4 4 4
10446 168/300 Read block 2331 len 40960
10447 169/300 Write block 2162 len 8192 data: 6 5
10448 170/300 Read block 4431 len 40960
10449 171/300 Write block 3475 len 16384 data: 5 5 6 4
10450 172/300 Read block 103 len 20480
10451 173/300 Read block 2240 len 8192
10452 174/300 Read block 1885 len 36864
10453 175/300 Flush
10454 176/300 Read block 3838 len 40960
10455 177/300 Flush
10456 178/300 Read block 1267 len 20480
10457 179/300 Read block 1768 len 16384
10458 180/300 Write block 3224 len 40960 data: 3 2 3 4 6 7 6 5 5 4
10459 181/300 Write block 78 len 16384 data: 3 3 2 2
10460 182/300 Write block 3673 len 40960 data: 6 6 6 8 8 7 6 5 6 7
10461 183/300 Read block 3962 len 36864
10462 184/300 Read block 3861 len 36864
10463 185/300 Flush
10464 186/300 Read block 3824 len 40960
10465 187/300 Flush
10466 188/300 Write block 3421 len 36864 data: 3 3 2 2 2 2 2 2 3
10467 189/300 Read block 3420 len 12288
10468 190/300 Write block 903 len 36864 data: 2 2 2 2 2 3 4 6 6
10469 191/300 Read block 1228 len 12288
10470 192/300 Write block 1618 len 40960 data: 3 3 3 2 2 2 3 4 3 3
10471 193/300 Write block 807 len 40960 data: 3 4 4 4 3 4 4 3 3 2
10472 194/300 Read block 4637 len 16384
10473 195/300 Read block 3202 len 12288
10474 196/300 Read block 2095 len 20480
10475 197/300 Read block 4635 len 4096
10476 198/300 Read block 3599 len 20480
10477 199/300 Flush
10478 200/300 Write block 1570 len 28672 data: 3 3 4 4 4 4 4
10479 201/300 Read block 3916 len 40960
10480 202/300 Flush
10481 203/300 Read block 1702 len 36864
10482 204/300 Write block 497 len 4096 data: 3
10483 205/300 Read block 586 len 4096
10484 206/300 Write block 4464 len 12288 data: 2 4 4
10485 207/300 Write block 4729 len 20480 data: 2 2 3 3 3
10486 208/300 Write block 4671 len 40960 data: 2 3 3 3 3 3 4 4 4 5
10487 209/300 Read block 682 len 16384
10488 210/300 Flush
10489 211/300 Read block 2580 len 16384
10490 212/300 Read block 1245 len 28672
10491 213/300 Read block 2861 len 4096
10492 214/300 Read block 4314 len 12288
10493 215/300 Write block 3021 len 16384 data: 4 4 4 4
10494 216/300 Write block 1990 len 8192 data: 3 3
10495 217/300 Flush
10496 218/300 Read block 291 len 28672
10497 219/300 Read block 2792 len 20480
10498 220/300 Flush
10499 221/300 Read block 2253 len 4096
10500 222/300 Read block 3863 len 24576
10501 223/300 Read block 4468 len 20480
10502 224/300 Read block 1095 len 12288
10503 225/300 Read block 4026 len 24576
10504 226/300 Read block 3163 len 28672
10505 227/300 Write block 4874 len 32768 data: 4 5 5 5 5 4 4 3
10506 228/300 Write block 2065 len 24576 data: 6 5 5 5 4 4
10507 229/300 Read block 3767 len 12288
10508 230/300 Flush
10509 231/300 Read block 2820 len 40960
10510 232/300 Read block 3844 len 28672
10511 233/300 Read block 1232 len 4096
10512 234/300 Read block 1972 len 8192
10513 235/300 Flush
10514 236/300 Read block 3802 len 24576
10515 237/300 Write block 716 len 36864 data: 3 3 3 2 2 2 3 4 4
10516 238/300 Read block 1936 len 20480
10517 239/300 Read block 3584 len 16384
10518 240/300 Read block 3350 len 24576
10519 241/300 Flush
10520 242/300 Read block 3218 len 28672
10521 243/300 Read block 699 len 36864
10522 244/300 Write block 1402 len 4096 data: 2
10523 245/300 Write block 2558 len 40960 data: 4 4 4 4 4 4 4 4 3 4
10524 246/300 Read block 4808 len 32768
10525 247/300 Read block 4788 len 40960
10526 248/300 Read block 1234 len 20480
10527 249/300 Write block 138 len 28672 data: 2 2 3 3 2 2 3
10528 250/300 Write block 4255 len 20480 data: 5 4 4 4 3
10529 251/300 Read block 4226 len 8192
10530 252/300 Read block 1159 len 28672
10531 253/300 Read block 626 len 24576
10532 254/300 Flush
10533 255/300 Write block 4896 len 24576 data: 3 2 2 2 2 2
10534 256/300 Read block 3016 len 12288
10535 257/300 Read block 2692 len 40960
10536 258/300 Read block 3231 len 36864
10537 259/300 Read block 4363 len 24576
10538 260/300 Flush
10539 261/300 Read block 1497 len 36864
10540 262/300 Read block 3541 len 28672
10541 263/300 Write block 2016 len 16384 data: 2 2 2 2
10542 264/300 Write block 1235 len 4096 data: 4
10543 265/300 Read block 4229 len 4096
10544 266/300 Read block 3597 len 32768
10545 267/300 Read block 991 len 12288
10546 268/300 Write block 4509 len 4096 data: 3
10547 269/300 Write block 1233 len 8192 data: 4 4
10548 270/300 Write block 2705 len 8192 data: 4 3
10549 271/300 Read block 2489 len 8192
10550 272/300 Read block 3593 len 8192
10551 273/300 Read block 3197 len 36864
10552 274/300 Write block 4207 len 20480 data: 3 3 3 3 3
10553 275/300 Flush
10554 276/300 Write block 748 len 4096 data: 3
10555 277/300 Read block 2525 len 16384
10556 278/300 Write block 1107 len 36864 data: 2 2 2 2 2 2 2 2 2
10557 279/300 Read block 2932 len 40960
10558 280/300 Read block 4328 len 12288
10559 281/300 Write block 2395 len 20480 data: 5 4 4 4 3
10560 282/300 Read block 4263 len 28672
10561 283/300 Read block 839 len 24576
10562 284/300 Write block 536 len 40960 data: 4 5 5 5 4 4 5 4 3 3
10563 285/300 Flush
10564 286/300 Write block 215 len 40960 data: 2 2 2 3 3 3 3 4 4 5
10565 287/300 Read block 4972 len 4096
10566 288/300 Read block 1778 len 16384
10567 289/300 Read block 1641 len 16384
10568 290/300 Read block 4555 len 4096
10569 291/300 Read block 2400 len 28672
10570 292/300 Read block 899 len 36864
10571 293/300 Read block 1674 len 4096
10572 294/300 Read block 2308 len 40960
10573 295/300 Read block 1731 len 8192
10574 296/300 Read block 4194 len 24576
10575 297/300 Read block 4699 len 36864
10576 298/300 Read block 92 len 36864
10577 299/300 Flush
10578 300/300 Write block 112 len 16384 data: 2 2 3 3
10579 301/300 Read block 893 len 28672
10580 Main task has work to do, go find it
10581 got dsc Start(2)
10582 start 2
10583 Nov 26 22:06:46.838 INFO request completed, latency_us: 204, response_code: 204, uri: /start/cid/2, method: POST, req_id: 0332a9d7-879c-4619-a66c-ca8561b4ab7a, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
10584 Replay: started 2, returned:Ok(())
10585 ----------------------------------------------------------------
10586 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
10587 States: Active Active Offline
10588 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
10589 3718 Acked Read 5 Done Done Sent false
10590 3719 Acked Read 9 Done Done Sent false
10591 3720 Acked Write 9 Done Done Sent false
10592 3721 Acked Write 7 Done Done Sent false
10593 3722 Acked Write 1 Done Done Sent false
10594 3723 Acked Read 3 Done Done Sent false
10595 3724 Acked Read 9 Done Done Sent false
10596 3725 Acked Read 3 Done Done Sent false
10597 3726 Acked Read 8 Done Done Sent false
10598 3727 Acked Write 4 Done Done Sent false
10599 3728 Acked Write 3 Done Done Sent false
10600 3729 Acked Read 8 Done Done Sent false
10601 3730 Acked Read 5 Done Done Sent false
10602 3731 Acked Write 7 Done Done Sent false
10603 3732 Acked Write 5 Done Done Sent false
10604 3733 Acked Flush 0 Done Done Sent false
10605 3734 Acked Read 5 Done Done Sent false
10606 3735 Acked Write 9 Done Done Sent false
10607 3736 Acked Read 8 Done Done Sent false
10608 3737 Acked Write 8 Done Done Sent false
10609 3738 Acked Write 1 Done Done Sent false
10610 3739 Acked Read 6 Done Done Sent false
10611 3740 Acked Read 6 Done Done Sent false
10612 3741 Acked Read 2 Done Done Sent false
10613 3742 Acked Write 8 Done Done Sent false
10614 3743 Acked Read 5 Done Done Sent false
10615 3744 Acked Write 8 Done Done Sent false
10616 3745 Acked Read 6 Done Done Sent false
10617 3746 Acked Read 4 Done Done Sent false
10618 3747 Acked Read 2 Done Done Sent false
10619 3748 Acked Read 8 Done Done Sent false
10620 3749 Acked Write 3 Done Done Sent false
10621 3750 Acked Write 1 Done Done Sent false
10622 3751 Acked Write 10 Done Done Sent false
10623 3752 Acked Write 8 Done Done Sent false
10624 3753 Acked Read 4 Done Done Sent false
10625 3754 Acked Write 6 Done Done Sent false
10626 3755 Acked Write 5 Done Done Sent false
10627 3756 Acked Read 2 Done Done Sent false
10628 3757 Acked Read 3 Done Done Sent false
10629 3758 Acked Read 4 Done Done Sent false
10630 3759 Acked Read 7 Done Done Sent false
10631 3760 Acked Read 3 Done Done Sent false
10632 3761 Acked Write 5 Done Done Sent false
10633 3762 Acked Read 3 Done Done Sent false
10634 3763 Acked Read 4 Done Done Sent false
10635 3764 Acked Flush 0 Done Done Sent false
10636 3765 Acked Read 10 Done Done Sent false
10637 3766 Acked Read 5 Done Done Sent false
10638 3767 Acked Read 2 Done Done Sent false
10639 3768 Acked Write 7 Done Done Sent false
10640 3769 Acked Flush 0 Done Done Sent false
10641 3770 Acked Write 7 Done Done Sent false
10642 3771 Acked Read 10 Done Done Sent false
10643 3772 Acked Flush 0 Done Done Sent false
10644 3773 Acked Write 8 Done Done Sent false
10645 3774 Acked Write 6 Done Done Sent false
10646 3775 Acked Read 2 Done Done Sent false
10647 3776 Acked Read 7 Done Done Sent false
10648 3777 Acked Write 2 Done Done Sent false
10649 3778 Acked Write 4 Done Done Sent false
10650 3779 Acked Read 3 Done Done Sent false
10651 3780 Acked Write 9 Done Done Sent false
10652 3781 Acked Read 2 Done Done Sent false
10653 3782 Acked Write 10 Done Done Sent false
10654 3783 Acked Read 5 Done Done Sent false
10655 3784 Acked Read 9 Done Done Sent false
10656 3785 Acked Read 7 Done Done Sent false
10657 3786 Acked Write 2 Done Done Sent false
10658 3787 Acked Write 4 Done Done Sent false
10659 3788 Acked Read 10 Done Done Sent false
10660 3789 Acked Read 3 Done Done Sent false
10661 3790 Acked Write 1 Done Done Sent false
10662 3791 Acked Read 7 Done Done Sent false
10663 3792 Acked Flush 0 Done Done Sent false
10664 3793 Acked Read 6 Done Done Sent false
10665 3794 Acked Write 5 Done Done Sent false
10666 3795 Acked Read 9 Done Done Sent false
10667 3796 Acked Read 8 Done Done Sent false
10668 3797 Acked Read 2 Done Done Sent false
10669 3798 Acked Read 3 Done Done Sent false
10670 3799 Acked Flush 0 Done Done Sent false
10671 3800 Acked Read 7 Done Done Sent false
10672 3801 Acked Write 7 Done Done Sent false
10673 3802 Acked Read 6 Done Done Sent false
10674 3803 Acked Write 2 Done Done Sent false
10675 3804 Acked Write 9 Done Done Sent false
10676 3805 Acked Write 2 Done Done Sent false
10677 3806 Acked Read 6 Done Done Sent false
10678 3807 Acked Write 7 Done Done Sent false
10679 3808 Acked Write 9 Done Done Sent false
10680 3809 Acked Write 10 Done Done Sent false
10681 3810 Acked Read 4 Done Done Sent false
10682 3811 Acked Write 4 Done Done Sent false
10683 3812 Acked Read 10 Done Done Sent false
10684 3813 Acked Read 10 Done Done Sent false
10685 3814 Acked Write 6 Done Done Sent false
10686 3815 Acked Read 5 Done Done Sent false
10687 3816 Acked Read 7 Done Done Sent false
10688 3817 Acked Write 2 Done Done Sent false
10689 3818 Acked Read 1 Done Done Sent false
10690 3819 Acked Read 2 Done Done Sent false
10691 3820 Acked Write 8 Done Done Sent false
10692 3821 Acked Write 10 Done Done Sent false
10693 3822 Acked Read 4 Done Done Sent false
10694 3823 Acked Flush 0 Done Done Sent false
10695 3824 Acked Read 8 Done Done Sent false
10696 3825 Acked Read 6 Done Done Sent false
10697 3826 Acked Write 1 Done Done Sent false
10698 3827 Acked Write 4 Done Done Sent false
10699 3828 Acked Write 10 Done Done Sent false
10700 3829 Acked Write 6 Done Done Sent false
10701 3830 Acked Write 8 Done Done Sent false
10702 3831 Acked Read 2 Done Done Sent false
10703 3832 Acked Write 3 Done Done Sent false
10704 3833 Acked Read 7 Done Done Sent false
10705 3834 Acked Read 5 Done Done Sent false
10706 3835 Acked Read 1 Done Done Sent false
10707 3836 Acked Write 6 Done Done Sent false
10708 3837 Acked Read 7 Done Done Sent false
10709 3838 Acked Read 2 Done Done Sent false
10710 3839 Acked Flush 0 Done Done Sent false
10711 3840 Acked Read 10 Done Done Sent false
10712 3841 Acked Write 7 Done Done Sent false
10713 3842 Acked Write 5 Done Done Sent false
10714 3843 Acked Read 3 Done Done Sent false
10715 3844 Acked Read 10 Done Done Sent false
10716 3845 Acked Flush 0 Done Done Sent false
10717 3846 Acked Write 9 Done Done Sent false
10718 3847 Acked Write 3 Done Done Sent false
10719 3848 Acked Read 9 Done Done Sent false
10720 3849 Acked Flush 0 Done Done Sent false
10721 3850 Acked Read 10 Done Done Sent false
10722 3851 Acked Read 4 Done Done Sent false
10723 3852 Acked Flush 0 Done Done Sent false
10724 3853 Acked Read 5 Done Done Sent false
10725 3854 Acked Read 7 Done Done Sent false
10726 3855 Acked Flush 0 Done Done Sent false
10727 3856 Acked Read 9 Done Done Sent false
10728 3857 Acked Read 9 Done Done Sent false
10729 3858 Acked Read 5 Done Done Sent false
10730 3859 Acked Flush 0 Done Done Sent false
10731 3860 Acked Read 9 Done Done Sent false
10732 3861 Acked Read 1 Done Done Sent false
10733 3862 Acked Write 7 Done Done Sent false
10734 3863 Acked Write 4 Done Done Sent false
10735 3864 Acked Read 1 Done Done Sent false
10736 3865 Acked Write 5 Done Done Sent false
10737 3866 Acked Write 3 Done Done Sent false
10738 3867 Acked Write 4 Done Done Sent false
10739 3868 Acked Write 2 Done Done Sent false
10740 3869 Acked Write 7 Done Done Sent false
10741 3870 Acked Read 10 Done Done Sent false
10742 3871 Acked Read 7 Done Done Sent false
10743 3872 Acked Write 4 Done Done Sent false
10744 3873 Acked Read 8 Done Done Sent false
10745 3874 Acked Read 7 Done Done Sent false
10746 3875 Acked Write 4 Done Done Sent false
10747 3876 Acked Write 7 Done Done Sent false
10748 3877 Acked Read 10 Done Done Sent false
10749 3878 Acked Read 10 Done Done Sent false
10750 3879 Acked Write 8 Done Done Sent false
10751 3880 Acked Read 7 Done Done Sent false
10752 3881 Acked Read 5 Done Done Sent false
10753 3882 Acked Flush 0 Done Done Sent false
10754 3883 Acked Read 4 Done Done Sent false
10755 3884 Acked Write 6 Done Done Sent false
10756 3885 Acked Read 10 Done Done Sent false
10757 3886 Acked Write 2 Done Done Sent false
10758 3887 Acked Read 10 Done Done Sent false
10759 3888 Acked Write 4 Done Done Sent false
10760 3889 Acked Read 5 Done Done Sent false
10761 3890 Acked Read 2 Done Done Sent false
10762 3891 Acked Read 9 Done Done Sent false
10763 3892 Acked Flush 0 Done Done Sent false
10764 3893 Acked Read 10 Done Done Sent false
10765 3894 Acked Flush 0 Done Done Sent false
10766 3895 Acked Read 5 Done Done Sent false
10767 3896 Acked Read 4 Done Done Sent false
10768 3897 Acked Write 10 Done Done Sent false
10769 3898 Acked Write 4 Done Done Sent false
10770 3899 Acked Write 10 Done Done Sent false
10771 3900 Acked Read 9 Done Done Sent false
10772 3901 Acked Read 9 Done Done Sent false
10773 3902 Acked Flush 0 Done Done Sent false
10774 3903 Acked Read 10 Done Done Sent false
10775 3904 Acked Flush 0 Done Done Sent false
10776 3905 Acked Write 9 Done Done Sent false
10777 3906 Acked Read 3 Done Done Sent false
10778 3907 Acked Write 9 Done Done Sent false
10779 3908 Acked Read 3 Done Done Sent false
10780 3909 Acked Write 10 Done Done Sent false
10781 3910 Acked Write 10 Done Done Sent false
10782 3911 Acked Read 4 Done Done Sent false
10783 3912 Acked Read 3 Done Done Sent false
10784 3913 Acked Read 5 Done Done Sent false
10785 3914 Acked Read 1 Done Done Sent false
10786 3915 Acked Read 5 Done Done Sent false
10787 3916 Acked Flush 0 Done Done Sent false
10788 3917 Acked Write 7 Done Done Sent false
10789 3918 Acked Read 10 Done Done Sent false
10790 3919 Acked Flush 0 Done Done Sent false
10791 3920 Acked Read 9 Done Done Sent false
10792 3921 Acked Write 1 Done Done Sent false
10793 3922 Acked Read 1 Done Done Sent false
10794 3923 Acked Write 3 Done Done Sent false
10795 3924 Acked Write 5 Done Done Sent false
10796 3925 Acked Write 10 Done Done Sent false
10797 3926 Acked Read 4 Done Done Sent false
10798 3927 Acked Flush 0 Done Done Sent false
10799 3928 Acked Read 4 Done Done Sent false
10800 3929 Acked Read 7 Done Done Sent false
10801 3930 Acked Read 1 Done Done Sent false
10802 3931 Acked Read 3 Done Done Sent false
10803 3932 Acked Write 4 Done Done Sent false
10804 3933 Acked Write 2 Done Done Sent false
10805 3934 Acked Flush 0 Done Done Sent false
10806 3935 Acked Read 7 Done Done Sent false
10807 3936 Acked Read 5 Done Done Sent false
10808 3937 Acked Flush 0 Done Done Sent false
10809 3938 Acked Read 1 Done Done Sent false
10810 3939 Acked Read 6 Done Done Sent false
10811 3940 Acked Read 5 Done Done Sent false
10812 3941 Acked Read 3 Done Done Sent false
10813 3942 Acked Read 6 Done Done Sent false
10814 3943 Acked Read 7 Done Done Sent false
10815 3944 Acked Write 8 Done Done Sent false
10816 3945 Acked Write 6 Done Done Sent false
10817 3946 Acked Read 3 Done Done Sent false
10818 3947 Acked Flush 0 Done Done Sent false
10819 3948 Acked Read 10 Done Done Sent false
10820 3949 Acked Read 7 Done Done Sent false
10821 3950 Acked Read 1 Done Done Sent false
10822 3951 Acked Read 2 Done Done Sent false
10823 3952 Acked Flush 0 Done Done Sent false
10824 3953 Acked Read 6 Done Done Sent false
10825 3954 Acked Write 9 Done Done Sent false
10826 3955 Acked Read 5 Done Done Sent false
10827 3956 Acked Read 4 Done Done Sent false
10828 3957 Acked Read 6 Done Done Sent false
10829 3958 Acked Flush 0 Done Done Sent false
10830 3959 Acked Read 7 Done Done Sent false
10831 3960 Acked Read 9 Done Done Sent false
10832 3961 Acked Write 1 Done Done Sent false
10833 3962 Acked Write 10 Done Done Sent false
10834 3963 Acked Read 8 Done Done Sent false
10835 3964 Acked Read 10 Done Done Sent false
10836 3965 Acked Read 5 Done Done Sent false
10837 3966 Acked Write 7 Done Done Sent false
10838 3967 Acked Write 5 Done Done Sent false
10839 3968 Acked Read 2 Done Done Sent false
10840 3969 Acked Read 7 Done Done Sent false
10841 3970 Acked Read 6 Done Done Sent false
10842 3971 Acked Flush 0 Done Done Sent false
10843 3972 Acked Write 6 Done Done Sent false
10844 3973 Acked Read 3 Done Done Sent false
10845 3974 Acked Read 10 Done Done Sent false
10846 3975 Acked Read 9 Done Done Sent false
10847 3976 Acked Read 6 Done Done Sent false
10848 3977 Acked Flush 0 Done Done Sent false
10849 3978 Acked Read 9 Done Done Sent false
10850 3979 Acked Read 7 Done Done Sent false
10851 3980 Acked Write 4 Done Done Sent false
10852 3981 Acked Write 1 Done Done Sent false
10853 3982 Acked Read 1 Done Done Sent false
10854 3983 Acked Read 8 Done Done Sent false
10855 3984 Acked Read 3 Done Done Sent false
10856 3985 Acked Write 1 Done Done Sent false
10857 3986 Acked Write 2 Done Done Sent false
10858 3987 Acked Write 2 Done Done Sent false
10859 3988 Acked Read 2 Done Done Sent false
10860 3989 Acked Read 2 Done Done Sent false
10861 3990 Acked Read 9 Done Done Sent false
10862 3991 Acked Write 5 Done Done Sent false
10863 3992 Acked Flush 0 Done Done Sent false
10864 3993 Acked Write 1 Done Done Sent false
10865 3994 Acked Read 4 Done Done Sent false
10866 3995 Acked Write 9 Done Done Sent false
10867 3996 Acked Read 10 Done Done Sent false
10868 3997 Acked Read 3 Done Done Sent false
10869 3998 Acked Write 5 Done Done Sent false
10870 3999 Acked Read 7 Done Done Sent false
10871 4000 Acked Read 6 Done Done Sent false
10872 4001 Acked Write 10 Done Done Sent false
10873 4002 Acked Flush 0 Done Done Sent false
10874 4003 Acked Write 10 Done Done Sent false
10875 4004 Acked Read 1 Done Done Sent false
10876 4005 Acked Read 4 Done Done Sent false
10877 4006 Acked Read 4 Done Done Sent false
10878 4007 Acked Read 1 Done Done Sent false
10879 4008 Acked Read 7 Done Done Sent false
10880 4009 Acked Read 9 Done Done Sent false
10881 4010 Acked Read 1 Done Done Sent false
10882 4011 Acked Read 10 Done Done Sent false
10883 4012 Acked Read 2 Done Done Sent false
10884 4013 Acked Read 6 Done Done Sent false
10885 4014 Acked Read 9 Done Done Sent false
10886 4015 Acked Read 9 Done Done Sent false
10887 4016 Acked Flush 0 Done Done Sent false
10888 4017 Acked Write 4 Done Done Sent false
10889 4018 Acked Read 7 Done Done Sent false
10890 STATES DS:0 DS:1 DS:2 TOTAL
10891 Sent 0 0 301 301
10892 Done 301 301 0 602
10893 Skipped 0 0 0 0
10894 Error 0 0 0 0
10895 Last Flush: 4016 4016 3717
10896 Downstairs last five completed: 3717 3716 3715 3714 3713
10897 Upstairs last five completed: 4018 4017 4016 4015 4014
10898 CLIENT: Up:0 ds:301 act:2
10899 [8830] got start action
10900 [8830] I am going to restart
10901 Starting downstairs at port 8830
10902 Make output file at "/tmp/dsc/downstairs-8830.txt"
10903 [8830][2] reports Starting
10904 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1213)
10905 [8830][2] reports Running
10906 ----------------------------------------------------------------
10907 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
10908 States: Active Active Offline
10909 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
10910 3718 Acked Read 5 Done Done Sent false
10911 3719 Acked Read 9 Done Done Sent false
10912 3720 Acked Write 9 Done Done Sent false
10913 3721 Acked Write 7 Done Done Sent false
10914 3722 Acked Write 1 Done Done Sent false
10915 3723 Acked Read 3 Done Done Sent false
10916 3724 Acked Read 9 Done Done Sent false
10917 3725 Acked Read 3 Done Done Sent false
10918 3726 Acked Read 8 Done Done Sent false
10919 3727 Acked Write 4 Done Done Sent false
10920 3728 Acked Write 3 Done Done Sent false
10921 3729 Acked Read 8 Done Done Sent false
10922 3730 Acked Read 5 Done Done Sent false
10923 3731 Acked Write 7 Done Done Sent false
10924 3732 Acked Write 5 Done Done Sent false
10925 3733 Acked Flush 0 Done Done Sent false
10926 3734 Acked Read 5 Done Done Sent false
10927 3735 Acked Write 9 Done Done Sent false
10928 3736 Acked Read 8 Done Done Sent false
10929 3737 Acked Write 8 Done Done Sent false
10930 3738 Acked Write 1 Done Done Sent false
10931 3739 Acked Read 6 Done Done Sent false
10932 3740 Acked Read 6 Done Done Sent false
10933 3741 Acked Read 2 Done Done Sent false
10934 3742 Acked Write 8 Done Done Sent false
10935 3743 Acked Read 5 Done Done Sent false
10936 3744 Acked Write 8 Done Done Sent false
10937 3745 Acked Read 6 Done Done Sent false
10938 3746 Acked Read 4 Done Done Sent false
10939 3747 Acked Read 2 Done Done Sent false
10940 3748 Acked Read 8 Done Done Sent false
10941 3749 Acked Write 3 Done Done Sent false
10942 3750 Acked Write 1 Done Done Sent false
10943 3751 Acked Write 10 Done Done Sent false
10944 3752 Acked Write 8 Done Done Sent false
10945 3753 Acked Read 4 Done Done Sent false
10946 3754 Acked Write 6 Done Done Sent false
10947 3755 Acked Write 5 Done Done Sent false
10948 3756 Acked Read 2 Done Done Sent false
10949 3757 Acked Read 3 Done Done Sent false
10950 3758 Acked Read 4 Done Done Sent false
10951 3759 Acked Read 7 Done Done Sent false
10952 3760 Acked Read 3 Done Done Sent false
10953 3761 Acked Write 5 Done Done Sent false
10954 3762 Acked Read 3 Done Done Sent false
10955 3763 Acked Read 4 Done Done Sent false
10956 3764 Acked Flush 0 Done Done Sent false
10957 3765 Acked Read 10 Done Done Sent false
10958 3766 Acked Read 5 Done Done Sent false
10959 3767 Acked Read 2 Done Done Sent false
10960 3768 Acked Write 7 Done Done Sent false
10961 3769 Acked Flush 0 Done Done Sent false
10962 3770 Acked Write 7 Done Done Sent false
10963 3771 Acked Read 10 Done Done Sent false
10964 3772 Acked Flush 0 Done Done Sent false
10965 3773 Acked Write 8 Done Done Sent false
10966 3774 Acked Write 6 Done Done Sent false
10967 3775 Acked Read 2 Done Done Sent false
10968 3776 Acked Read 7 Done Done Sent false
10969 3777 Acked Write 2 Done Done Sent false
10970 3778 Acked Write 4 Done Done Sent false
10971 3779 Acked Read 3 Done Done Sent false
10972 3780 Acked Write 9 Done Done Sent false
10973 3781 Acked Read 2 Done Done Sent false
10974 3782 Acked Write 10 Done Done Sent false
10975 3783 Acked Read 5 Done Done Sent false
10976 3784 Acked Read 9 Done Done Sent false
10977 3785 Acked Read 7 Done Done Sent false
10978 3786 Acked Write 2 Done Done Sent false
10979 3787 Acked Write 4 Done Done Sent false
10980 3788 Acked Read 10 Done Done Sent false
10981 3789 Acked Read 3 Done Done Sent false
10982 3790 Acked Write 1 Done Done Sent false
10983 3791 Acked Read 7 Done Done Sent false
10984 3792 Acked Flush 0 Done Done Sent false
10985 3793 Acked Read 6 Done Done Sent false
10986 3794 Acked Write 5 Done Done Sent false
10987 3795 Acked Read 9 Done Done Sent false
10988 3796 Acked Read 8 Done Done Sent false
10989 3797 Acked Read 2 Done Done Sent false
10990 3798 Acked Read 3 Done Done Sent false
10991 3799 Acked Flush 0 Done Done Sent false
10992 3800 Acked Read 7 Done Done Sent false
10993 3801 Acked Write 7 Done Done Sent false
10994 3802 Acked Read 6 Done Done Sent false
10995 3803 Acked Write 2 Done Done Sent false
10996 3804 Acked Write 9 Done Done Sent false
10997 3805 Acked Write 2 Done Done Sent false
10998 3806 Acked Read 6 Done Done Sent false
10999 3807 Acked Write 7 Done Done Sent false
11000 3808 Acked Write 9 Done Done Sent false
11001 3809 Acked Write 10 Done Done Sent false
11002 3810 Acked Read 4 Done Done Sent false
11003 3811 Acked Write 4 Done Done Sent false
11004 3812 Acked Read 10 Done Done Sent false
11005 3813 Acked Read 10 Done Done Sent false
11006 3814 Acked Write 6 Done Done Sent false
11007 3815 Acked Read 5 Done Done Sent false
11008 3816 Acked Read 7 Done Done Sent false
11009 3817 Acked Write 2 Done Done Sent false
11010 3818 Acked Read 1 Done Done Sent false
11011 3819 Acked Read 2 Done Done Sent false
11012 3820 Acked Write 8 Done Done Sent false
11013 3821 Acked Write 10 Done Done Sent false
11014 3822 Acked Read 4 Done Done Sent false
11015 3823 Acked Flush 0 Done Done Sent false
11016 3824 Acked Read 8 Done Done Sent false
11017 3825 Acked Read 6 Done Done Sent false
11018 3826 Acked Write 1 Done Done Sent false
11019 3827 Acked Write 4 Done Done Sent false
11020 3828 Acked Write 10 Done Done Sent false
11021 3829 Acked Write 6 Done Done Sent false
11022 3830 Acked Write 8 Done Done Sent false
11023 3831 Acked Read 2 Done Done Sent false
11024 3832 Acked Write 3 Done Done Sent false
11025 3833 Acked Read 7 Done Done Sent false
11026 3834 Acked Read 5 Done Done Sent false
11027 3835 Acked Read 1 Done Done Sent false
11028 3836 Acked Write 6 Done Done Sent false
11029 3837 Acked Read 7 Done Done Sent false
11030 3838 Acked Read 2 Done Done Sent false
11031 3839 Acked Flush 0 Done Done Sent false
11032 3840 Acked Read 10 Done Done Sent false
11033 3841 Acked Write 7 Done Done Sent false
11034 3842 Acked Write 5 Done Done Sent false
11035 3843 Acked Read 3 Done Done Sent false
11036 3844 Acked Read 10 Done Done Sent false
11037 3845 Acked Flush 0 Done Done Sent false
11038 3846 Acked Write 9 Done Done Sent false
11039 3847 Acked Write 3 Done Done Sent false
11040 3848 Acked Read 9 Done Done Sent false
11041 3849 Acked Flush 0 Done Done Sent false
11042 3850 Acked Read 10 Done Done Sent false
11043 3851 Acked Read 4 Done Done Sent false
11044 3852 Acked Flush 0 Done Done Sent false
11045 3853 Acked Read 5 Done Done Sent false
11046 3854 Acked Read 7 Done Done Sent false
11047 3855 Acked Flush 0 Done Done Sent false
11048 3856 Acked Read 9 Done Done Sent false
11049 3857 Acked Read 9 Done Done Sent false
11050 3858 Acked Read 5 Done Done Sent false
11051 3859 Acked Flush 0 Done Done Sent false
11052 3860 Acked Read 9 Done Done Sent false
11053 3861 Acked Read 1 Done Done Sent false
11054 3862 Acked Write 7 Done Done Sent false
11055 3863 Acked Write 4 Done Done Sent false
11056 3864 Acked Read 1 Done Done Sent false
11057 3865 Acked Write 5 Done Done Sent false
11058 3866 Acked Write 3 Done Done Sent false
11059 3867 Acked Write 4 Done Done Sent false
11060 3868 Acked Write 2 Done Done Sent false
11061 3869 Acked Write 7 Done Done Sent false
11062 3870 Acked Read 10 Done Done Sent false
11063 3871 Acked Read 7 Done Done Sent false
11064 3872 Acked Write 4 Done Done Sent false
11065 3873 Acked Read 8 Done Done Sent false
11066 3874 Acked Read 7 Done Done Sent false
11067 3875 Acked Write 4 Done Done Sent false
11068 3876 Acked Write 7 Done Done Sent false
11069 3877 Acked Read 10 Done Done Sent false
11070 3878 Acked Read 10 Done Done Sent false
11071 3879 Acked Write 8 Done Done Sent false
11072 3880 Acked Read 7 Done Done Sent false
11073 3881 Acked Read 5 Done Done Sent false
11074 3882 Acked Flush 0 Done Done Sent false
11075 3883 Acked Read 4 Done Done Sent false
11076 3884 Acked Write 6 Done Done Sent false
11077 3885 Acked Read 10 Done Done Sent false
11078 3886 Acked Write 2 Done Done Sent false
11079 3887 Acked Read 10 Done Done Sent false
11080 3888 Acked Write 4 Done Done Sent false
11081 3889 Acked Read 5 Done Done Sent false
11082 3890 Acked Read 2 Done Done Sent false
11083 3891 Acked Read 9 Done Done Sent false
11084 3892 Acked Flush 0 Done Done Sent false
11085 3893 Acked Read 10 Done Done Sent false
11086 3894 Acked Flush 0 Done Done Sent false
11087 3895 Acked Read 5 Done Done Sent false
11088 3896 Acked Read 4 Done Done Sent false
11089 3897 Acked Write 10 Done Done Sent false
11090 3898 Acked Write 4 Done Done Sent false
11091 3899 Acked Write 10 Done Done Sent false
11092 3900 Acked Read 9 Done Done Sent false
11093 3901 Acked Read 9 Done Done Sent false
11094 3902 Acked Flush 0 Done Done Sent false
11095 3903 Acked Read 10 Done Done Sent false
11096 3904 Acked Flush 0 Done Done Sent false
11097 3905 Acked Write 9 Done Done Sent false
11098 3906 Acked Read 3 Done Done Sent false
11099 3907 Acked Write 9 Done Done Sent false
11100 3908 Acked Read 3 Done Done Sent false
11101 3909 Acked Write 10 Done Done Sent false
11102 3910 Acked Write 10 Done Done Sent false
11103 3911 Acked Read 4 Done Done Sent false
11104 3912 Acked Read 3 Done Done Sent false
11105 3913 Acked Read 5 Done Done Sent false
11106 3914 Acked Read 1 Done Done Sent false
11107 3915 Acked Read 5 Done Done Sent false
11108 3916 Acked Flush 0 Done Done Sent false
11109 3917 Acked Write 7 Done Done Sent false
11110 3918 Acked Read 10 Done Done Sent false
11111 3919 Acked Flush 0 Done Done Sent false
11112 3920 Acked Read 9 Done Done Sent false
11113 3921 Acked Write 1 Done Done Sent false
11114 3922 Acked Read 1 Done Done Sent false
11115 3923 Acked Write 3 Done Done Sent false
11116 3924 Acked Write 5 Done Done Sent false
11117 3925 Acked Write 10 Done Done Sent false
11118 3926 Acked Read 4 Done Done Sent false
11119 3927 Acked Flush 0 Done Done Sent false
11120 3928 Acked Read 4 Done Done Sent false
11121 3929 Acked Read 7 Done Done Sent false
11122 3930 Acked Read 1 Done Done Sent false
11123 3931 Acked Read 3 Done Done Sent false
11124 3932 Acked Write 4 Done Done Sent false
11125 3933 Acked Write 2 Done Done Sent false
11126 3934 Acked Flush 0 Done Done Sent false
11127 3935 Acked Read 7 Done Done Sent false
11128 3936 Acked Read 5 Done Done Sent false
11129 3937 Acked Flush 0 Done Done Sent false
11130 3938 Acked Read 1 Done Done Sent false
11131 3939 Acked Read 6 Done Done Sent false
11132 3940 Acked Read 5 Done Done Sent false
11133 3941 Acked Read 3 Done Done Sent false
11134 3942 Acked Read 6 Done Done Sent false
11135 3943 Acked Read 7 Done Done Sent false
11136 3944 Acked Write 8 Done Done Sent false
11137 3945 Acked Write 6 Done Done Sent false
11138 3946 Acked Read 3 Done Done Sent false
11139 3947 Acked Flush 0 Done Done Sent false
11140 3948 Acked Read 10 Done Done Sent false
11141 3949 Acked Read 7 Done Done Sent false
11142 3950 Acked Read 1 Done Done Sent false
11143 3951 Acked Read 2 Done Done Sent false
11144 3952 Acked Flush 0 Done Done Sent false
11145 3953 Acked Read 6 Done Done Sent false
11146 3954 Acked Write 9 Done Done Sent false
11147 3955 Acked Read 5 Done Done Sent false
11148 3956 Acked Read 4 Done Done Sent false
11149 3957 Acked Read 6 Done Done Sent false
11150 3958 Acked Flush 0 Done Done Sent false
11151 3959 Acked Read 7 Done Done Sent false
11152 3960 Acked Read 9 Done Done Sent false
11153 3961 Acked Write 1 Done Done Sent false
11154 3962 Acked Write 10 Done Done Sent false
11155 3963 Acked Read 8 Done Done Sent false
11156 3964 Acked Read 10 Done Done Sent false
11157 3965 Acked Read 5 Done Done Sent false
11158 3966 Acked Write 7 Done Done Sent false
11159 3967 Acked Write 5 Done Done Sent false
11160 3968 Acked Read 2 Done Done Sent false
11161 3969 Acked Read 7 Done Done Sent false
11162 3970 Acked Read 6 Done Done Sent false
11163 3971 Acked Flush 0 Done Done Sent false
11164 3972 Acked Write 6 Done Done Sent false
11165 3973 Acked Read 3 Done Done Sent false
11166 3974 Acked Read 10 Done Done Sent false
11167 3975 Acked Read 9 Done Done Sent false
11168 3976 Acked Read 6 Done Done Sent false
11169 3977 Acked Flush 0 Done Done Sent false
11170 3978 Acked Read 9 Done Done Sent false
11171 3979 Acked Read 7 Done Done Sent false
11172 3980 Acked Write 4 Done Done Sent false
11173 3981 Acked Write 1 Done Done Sent false
11174 3982 Acked Read 1 Done Done Sent false
11175 3983 Acked Read 8 Done Done Sent false
11176 3984 Acked Read 3 Done Done Sent false
11177 3985 Acked Write 1 Done Done Sent false
11178 3986 Acked Write 2 Done Done Sent false
11179 3987 Acked Write 2 Done Done Sent false
11180 3988 Acked Read 2 Done Done Sent false
11181 3989 Acked Read 2 Done Done Sent false
11182 3990 Acked Read 9 Done Done Sent false
11183 3991 Acked Write 5 Done Done Sent false
11184 3992 Acked Flush 0 Done Done Sent false
11185 3993 Acked Write 1 Done Done Sent false
11186 3994 Acked Read 4 Done Done Sent false
11187 3995 Acked Write 9 Done Done Sent false
11188 3996 Acked Read 10 Done Done Sent false
11189 3997 Acked Read 3 Done Done Sent false
11190 3998 Acked Write 5 Done Done Sent false
11191 3999 Acked Read 7 Done Done Sent false
11192 4000 Acked Read 6 Done Done Sent false
11193 4001 Acked Write 10 Done Done Sent false
11194 4002 Acked Flush 0 Done Done Sent false
11195 4003 Acked Write 10 Done Done Sent false
11196 4004 Acked Read 1 Done Done Sent false
11197 4005 Acked Read 4 Done Done Sent false
11198 4006 Acked Read 4 Done Done Sent false
11199 4007 Acked Read 1 Done Done Sent false
11200 4008 Acked Read 7 Done Done Sent false
11201 4009 Acked Read 9 Done Done Sent false
11202 4010 Acked Read 1 Done Done Sent false
11203 4011 Acked Read 10 Done Done Sent false
11204 4012 Acked Read 2 Done Done Sent false
11205 4013 Acked Read 6 Done Done Sent false
11206 4014 Acked Read 9 Done Done Sent false
11207 4015 Acked Read 9 Done Done Sent false
11208 4016 Acked Flush 0 Done Done Sent false
11209 4017 Acked Write 4 Done Done Sent false
11210 4018 Acked Read 7 Done Done Sent false
11211 4019 Acked Flush 0 Done Done Sent false
11212 STATES DS:0 DS:1 DS:2 TOTAL
11213 Sent 0 0 302 302
11214 Done 302 302 0 604
11215 Skipped 0 0 0 0
11216 Error 0 0 0 0
11217 Last Flush: 4019 4019 3717
11218 Downstairs last five completed: 3717 3716 3715 3714 3713
11219 Upstairs last five completed: 4019 4018 4017 4016 4015
11220 CLIENT: Up:0 ds:302 act:2
112212024-11-26T22:06:52.586ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
112222024-11-26T22:06:52.586ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:39645) = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
112232024-11-26T22:06:52.587ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
112242024-11-26T22:06:52.588ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
112252024-11-26T22:06:52.588ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
112262024-11-26T22:06:52.588ZINFOcrucible: send last flush ID to this DS: 3717 = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
112272024-11-26T22:06:52.588ZINFOcrucible: Replied this last flush ID: 3717 = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
112282024-11-26T22:06:52.588ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
112292024-11-26T22:06:52.588ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
112302024-11-26T22:06:52.588ZINFOcrucible: [2] client re-new 302 jobs since flush 3717 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
112312024-11-26T22:06:52.588ZINFOcrucible: [2] Replayed 302 jobs since flush: 3717 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
11232 ----------------------------------------------------------------
11233 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
11234 Downstairs last five completed: 4019 4018 4017 4016 4015
11235 Upstairs last five completed: 4019 4018 4017 4016 4015
11236 CLIENT: Up:0 ds:0 act:3
11237 Replay: All jobs finished
11238 Main task has work to do, go find it
11239 got dsc Stop(1)
11240 stop 1
11241 Nov 26 22:06:54.843[8820] Got stop action so:false kr:false
11242 INFO request completed, latency_us: 352, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 3fd2327f-209c-4533-8c3a-44030c4bde12, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
11243 Nov 26 22:06:54.844 INFO request completed, latency_us: 130, response_code: 200, uri: /state/cid/1, method: GET, req_id: aa8b463b-bac9-4060-b303-16385dff8bc4, remote_addr: {"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2024-11-26T22:06:54.844488037Z","hostname":"w-01JDN7ZQ17ABQG15KHKN3F2D26","pid":1203,"":"io task","client":"1","":"downstairs","session_id":"86a36c17-3718-484c-87fd-2b8a1dc2ab18"}
11244 127.0.0.1:36829, local_addr: 127.0.0.1:9998
112452024-11-26T22:06:54.844ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
112462024-11-26T22:06:54.844ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
112472024-11-26T22:06:54.844ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
112482024-11-26T22:06:54.844ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
112492024-11-26T22:06:54.844ZINFOcrucible: client task is exiting = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
11250 [8820] Exited with: ExitStatus(unix_wait_status(9))
11251 [8820][1] reports Exit
11252 Nov 26 22:06:58.847 INFO request completed, latency_us: 200, response_code: 200, uri: /state/cid/1, method: GET, req_id: 5d305269-c59d-4422-987c-3e6fb5543b20, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
11253 001/300 Flush
11254 002/300 Read block 185 len 4096
11255 003/300 Write block 2495 len 40960 data: 2 2 2 2 2 4 3 3 3 3
11256 004/300 Read block 3717 len 36864
11257 005/300 Write block 3358 len 24576 data: 2 2 3 4 3 3
11258 006/300 Read block 4730 len 28672
11259 007/300 Read block 3983 len 12288
11260 008/300 Read block 3615 len 4096
11261 009/300 Write block 1931 len 40960 data: 3 4 4 4 4 5 6 7 7 6
11262 010/300 Write block 4659 len 32768 data: 3 3 4 4 4 4 2 2
11263 011/300 Write block 2372 len 8192 data: 3 4
11264 012/300 Read block 4929 len 12288
11265 013/300 Read block 1213 len 36864
11266 014/300 Read block 865 len 4096
11267 015/300 Flush
11268 016/300 Write block 2336 len 16384 data: 4 4 4 4
11269 017/300 Flush
11270 018/300 Read block 750 len 36864
11271 019/300 Flush
11272 020/300 Write block 1882 len 24576 data: 3 3 3 4 3 4
11273 021/300 Read block 3294 len 12288
11274 022/300 Read block 3986 len 24576
11275 023/300 Read block 1196 len 4096
11276 024/300 Read block 1405 len 32768
11277 025/300 Read block 397 len 36864
11278 026/300 Read block 3717 len 12288
11279 027/300 Flush
11280 028/300 Write block 4472 len 28672 data: 3 4 5 6 5 5 4
11281 029/300 Read block 4641 len 16384
11282 030/300 Read block 550 len 4096
11283 031/300 Write block 3160 len 12288 data: 6 5 4
11284 032/300 Write block 719 len 32768 data: 3 3 3 4 5 5 4 3
11285 033/300 Read block 3740 len 4096
11286 034/300 Read block 3656 len 40960
11287 035/300 Read block 3669 len 8192
11288 036/300 Read block 2770 len 40960
11289 037/300 Write block 4946 len 12288 data: 2 2 2
11290 038/300 Read block 1798 len 40960
11291 039/300 Read block 2331 len 20480
11292 040/300 Write block 4074 len 16384 data: 5 4 3 4
11293 041/300 Write block 1368 len 28672 data: 3 2 2 2 2 2 2
11294 042/300 Read block 4455 len 20480
11295 043/300 Flush
11296 044/300 Read block 3375 len 32768
11297 045/300 Read block 852 len 4096
11298 046/300 Write block 2234 len 4096 data: 2
11299 047/300 Read block 3881 len 20480
11300 048/300 Write block 889 len 24576 data: 4 4 4 4 3 2
11301 049/300 Write block 4421 len 28672 data: 2 2 2 2 2 2 2
11302 050/300 Read block 368 len 20480
11303 051/300 Write block 195 len 32768 data: 3 2 2 2 2 2 2 2
11304 052/300 Read block 3673 len 36864
11305 053/300 Read block 2189 len 32768
11306 054/300 Write block 2102 len 12288 data: 3 3 3
11307 055/300 Write block 4126 len 12288 data: 2 2 2
11308 056/300 Read block 214 len 8192
11309 057/300 Read block 1266 len 40960
11310 058/300 Read block 1566 len 40960
11311 059/300 Write block 2809 len 4096 data: 4
11312 060/300 Write block 216 len 24576 data: 3 3 4 4 4 4
11313 061/300 Read block 1891 len 28672
11314 062/300 Read block 3694 len 36864
11315 063/300 Flush
11316 064/300 Read block 1316 len 28672
11317 065/300 Read block 2744 len 20480
11318 066/300 Read block 507 len 12288
11319 067/300 Read block 4278 len 32768
11320 068/300 Read block 813 len 40960
11321 069/300 Write block 287 len 16384 data: 2 3 5 5
11322 070/300 Write block 1793 len 16384 data: 3 2 2 2
11323 071/300 Read block 3043 len 12288
11324 072/300 Read block 1787 len 40960
11325 073/300 Write block 3643 len 28672 data: 3 3 3 3 4 4 3
11326 074/300 Flush
11327 075/300 Flush
11328 076/300 Read block 753 len 16384
11329 077/300 Read block 3843 len 16384
11330 078/300 Read block 2509 len 4096
11331 079/300 Read block 4644 len 16384
11332 080/300 Read block 1982 len 36864
11333 081/300 Flush
11334 082/300 Write block 231 len 8192 data: 2 2
11335 083/300 Write block 3082 len 40960 data: 4 4 3 3 3 6 6 5 6 5
11336 084/300 Read block 1806 len 12288
11337 085/300 Write block 2084 len 40960 data: 3 3 3 4 3 3 3 2 2 2
11338 086/300 Read block 1736 len 20480
11339 087/300 Flush
11340 088/300 Read block 4481 len 36864
11341 089/300 Read block 4983 len 32768
11342 090/300 Read block 4478 len 12288
11343 091/300 Flush
11344 092/300 Write block 356 len 40960 data: 5 4 4 4 4 5 6 7 6 6
11345 093/300 Read block 3373 len 36864
11346 094/300 Read block 2447 len 36864
11347 095/300 Read block 3879 len 20480
11348 096/300 Read block 3167 len 28672
11349 097/300 Write block 2956 len 32768 data: 4 3 4 3 3 3 3 3
11350 098/300 Write block 4306 len 16384 data: 2 2 2 2
11351 099/300 Read block 3259 len 8192
11352 100/300 Write block 2812 len 28672 data: 4 5 4 2 3 2 2
11353 101/300 Read block 3503 len 20480
11354 102/300 Write block 1779 len 40960 data: 5 5 3 3 2 2 3 3 4 4
11355 103/300 Read block 4572 len 32768
11356 104/300 Write block 2356 len 20480 data: 4 3 2 2 2
11357 105/300 Write block 524 len 24576 data: 4 3 2 2 2 2
11358 106/300 Read block 1955 len 36864
11359 107/300 Read block 3637 len 24576
11360 108/300 Write block 3792 len 28672 data: 2 3 3 3 4 3 3
11361 109/300 Flush
11362 110/300 Read block 1781 len 32768
11363 111/300 Flush
11364 112/300 Write block 1122 len 24576 data: 4 4 3 3 3 3
11365 113/300 Read block 682 len 16384
11366 114/300 Read block 2059 len 4096
11367 115/300 Read block 2930 len 8192
11368 116/300 Flush
11369 117/300 Write block 4356 len 36864 data: 3 3 3 4 5 4 5 4 5
11370 118/300 Read block 2054 len 12288
11371 119/300 Read block 1454 len 24576
11372 120/300 Read block 4607 len 28672
11373 121/300 Read block 73 len 12288
11374 122/300 Write block 4984 len 4096 data: 4
11375 123/300 Write block 3889 len 36864 data: 2 2 2 2 2 2 2 2 2
11376 124/300 Write block 1790 len 4096 data: 3
11377 125/300 Read block 1705 len 36864
11378 126/300 Flush
11379 127/300 Write block 2619 len 28672 data: 3 3 2 2 3 3 3
11380 128/300 Write block 3930 len 8192 data: 3 3
11381 129/300 Write block 592 len 40960 data: 3 3 2 2 2 2 2 2 2 2
11382 130/300 Read block 1433 len 24576
11383 131/300 Read block 1865 len 20480
11384 132/300 Write block 1589 len 8192 data: 2 2
11385 133/300 Write block 3915 len 16384 data: 3 3 3 3
11386 134/300 Write block 2432 len 12288 data: 3 3 3
11387 135/300 Write block 1382 len 28672 data: 4 3 3 2 2 2 2
11388 136/300 Write block 3866 len 12288 data: 2 2 2
11389 137/300 Write block 747 len 12288 data: 4 4 3
11390 138/300 Read block 147 len 4096
11391 139/300 Write block 3241 len 40960 data: 3 4 4 5 4 4 5 3 3 2
11392 140/300 Read block 78 len 8192
11393 141/300 Read block 2960 len 4096
11394 142/300 Read block 1772 len 4096
11395 143/300 Read block 3132 len 32768
11396 144/300 Read block 2602 len 36864
11397 145/300 Read block 4262 len 8192
11398 146/300 Write block 3528 len 28672 data: 2 2 2 3 3 3 3
11399 147/300 Write block 3139 len 4096 data: 3
11400 148/300 Write block 4446 len 24576 data: 4 4 5 3 3 4
11401 149/300 Write block 4677 len 20480 data: 5 5 5 6 5
11402 150/300 Write block 3302 len 12288 data: 3 3 3
11403 151/300 Read block 1683 len 4096
11404 152/300 Read block 3482 len 20480
11405 153/300 Read block 2139 len 12288
11406 154/300 Read block 1505 len 24576
11407 155/300 Read block 4383 len 40960
11408 156/300 Write block 2400 len 4096 data: 3
11409 157/300 Read block 3972 len 12288
11410 158/300 Write block 2267 len 12288 data: 4 4 4
11411 159/300 Read block 1283 len 24576
11412 160/300 Write block 779 len 4096 data: 2
11413 161/300 Read block 972 len 40960
11414 162/300 Write block 3893 len 24576 data: 3 3 3 3 3 2
11415 163/300 Read block 2050 len 24576
11416 164/300 Write block 1096 len 16384 data: 3 3 4 2
11417 165/300 Read block 4135 len 40960
11418 166/300 Read block 11 len 16384
11419 167/300 Read block 1183 len 12288
11420 168/300 Write block 1836 len 28672 data: 3 3 3 3 2 2 3
11421 169/300 Flush
11422 170/300 Read block 1454 len 32768
11423 171/300 Read block 4410 len 16384
11424 172/300 Read block 4920 len 20480
11425 173/300 Write block 2596 len 36864 data: 3 2 3 3 3 3 3 4 5
11426 174/300 Write block 2198 len 32768 data: 3 3 3 2 2 2 2 2
11427 175/300 Write block 2587 len 8192 data: 2 2
11428 176/300 Write block 1807 len 20480 data: 2 2 2 2 2
11429 177/300 Read block 703 len 28672
11430 178/300 Write block 564 len 32768 data: 4 4 3 3 3 3 3 5
11431 179/300 Read block 1420 len 20480
11432 180/300 Read block 3664 len 32768
11433 181/300 Read block 867 len 28672
11434 182/300 Read block 3996 len 36864
11435 183/300 Write block 3469 len 32768 data: 2 3 3 3 4 5 6 6
11436 184/300 Write block 4758 len 32768 data: 2 2 2 2 2 4 4 4
11437 185/300 Write block 4025 len 16384 data: 4 4 3 4
11438 186/300 Flush
11439 187/300 Write block 3850 len 36864 data: 2 2 2 2 3 3 3 5 5
11440 188/300 Read block 3652 len 12288
11441 189/300 Write block 2296 len 24576 data: 3 3 3 4 4 4
11442 190/300 Write block 3163 len 16384 data: 4 4 4 4
11443 191/300 Flush
11444 192/300 Read block 2729 len 32768
11445 193/300 Write block 2958 len 8192 data: 5 4
11446 194/300 Flush
11447 195/300 Write block 2210 len 32768 data: 2 2 2 2 2 2 2 2
11448 196/300 Read block 2700 len 16384
11449 197/300 Read block 3332 len 4096
11450 198/300 Read block 863 len 12288
11451 199/300 Write block 1594 len 24576 data: 3 3 3 3 3 5
11452 200/300 Read block 1070 len 36864
11453 201/300 Read block 2438 len 36864
11454 202/300 Write block 4687 len 4096 data: 3
11455 203/300 Read block 1758 len 20480
11456 204/300 Read block 3997 len 12288
11457 205/300 Write block 4964 len 32768 data: 4 6 5 4 5 5 4 3
11458 206/300 Write block 2247 len 32768 data: 4 4 3 3 3 3 4 4
11459 207/300 Read block 4742 len 4096
11460 208/300 Read block 3093 len 4096
11461 209/300 Write block 2473 len 16384 data: 5 4 3 3
11462 210/300 Read block 4956 len 40960
11463 211/300 Read block 2829 len 24576
11464 212/300 Write block 3696 len 28672 data: 3 3 4 5 4 4 3
11465 213/300 Read block 4298 len 36864
11466 214/300 Read block 169 len 8192
11467 215/300 Write block 565 len 4096 data: 5
11468 216/300 Read block 1105 len 36864
11469 217/300 Write block 1720 len 20480 data: 3 4 4 4 4
11470 218/300 Write block 3221 len 40960 data: 4 3 3 4 3 4 5 7 8 7
11471 219/300 Read block 628 len 12288
11472 220/300 Write block 4398 len 8192 data: 2 2
11473 221/300 Read block 4995 len 16384
11474 222/300 Read block 506 len 4096
11475 223/300 Write block 1498 len 24576 data: 2 5 4 3 3 3
11476 224/300 Read block 319 len 36864
11477 225/300 Read block 1585 len 28672
11478 226/300 Read block 3960 len 40960
11479 227/300 Write block 2465 len 12288 data: 3 3 4
11480 228/300 Read block 4967 len 20480
11481 229/300 Write block 1144 len 16384 data: 3 3 3 4
11482 230/300 Read block 2710 len 20480
11483 231/300 Read block 1003 len 4096
11484 232/300 Write block 2888 len 4096 data: 5
11485 233/300 Read block 1087 len 28672
11486 234/300 Write block 44 len 4096 data: 3
11487 235/300 Flush
11488 236/300 Read block 3390 len 40960
11489 237/300 Read block 2110 len 24576
11490 238/300 Flush
11491 239/300 Read block 4645 len 32768
11492 240/300 Write block 2880 len 4096 data: 2
11493 241/300 Write block 1907 len 20480 data: 2 2 2 2 4
11494 242/300 Write block 216 len 20480 data: 4 4 5 5 5
11495 243/300 Write block 4487 len 20480 data: 3 3 4 4 4
11496 244/300 Read block 1874 len 12288
11497 245/300 Write block 1790 len 16384 data: 4 4 3 4
11498 246/300 Write block 3363 len 24576 data: 4 3 3 3 3 3
11499 247/300 Write block 4011 len 32768 data: 4 2 2 3 3 3 3 3
11500 248/300 Write block 2064 len 8192 data: 6 7
11501 249/300 Write block 1919 len 40960 data: 2 2 2 2 2 3 3 3 4 4
11502 250/300 Write block 2574 len 32768 data: 2 2 2 2 2 2 2 2
11503 251/300 Read block 1583 len 36864
11504 252/300 Read block 1148 len 24576
11505 253/300 Flush
11506 254/300 Write block 4252 len 28672 data: 3 3 3 6 5 5 5
11507 255/300 Read block 1426 len 16384
11508 256/300 Read block 4979 len 4096
11509 257/300 Read block 1308 len 8192
11510 258/300 Write block 3663 len 32768 data: 4 5 5 5 5 4 4 5
11511 259/300 Write block 3640 len 36864 data: 2 4 4 4 4 4 4 5 5
11512 260/300 Write block 3873 len 12288 data: 6 6 5
11513 261/300 Read block 4643 len 16384
11514 262/300 Read block 1509 len 40960
11515 263/300 Write block 1852 len 24576 data: 6 4 3 4 4 4
11516 264/300 Read block 4313 len 32768
11517 265/300 Write block 3950 len 28672 data: 6 5 5 4 5 4 4
11518 266/300 Read block 3036 len 20480
11519 267/300 Write block 1921 len 28672 data: 3 3 3 4 4 4 5
11520 268/300 Write block 576 len 32768 data: 4 3 3 2 2 2 2 2
11521 269/300 Read block 3663 len 16384
11522 270/300 Flush
11523 271/300 Read block 3808 len 24576
11524 272/300 Read block 1900 len 24576
11525 273/300 Write block 1340 len 28672 data: 3 3 3 4 3 3 3
11526 274/300 Write block 1155 len 36864 data: 2 2 2 2 3 3 3 2 3
11527 275/300 Read block 236 len 4096
11528 276/300 Read block 442 len 36864
11529 277/300 Read block 1063 len 36864
11530 278/300 Read block 2750 len 8192
11531 279/300 Write block 2356 len 12288 data: 5 4 3
11532 280/300 Read block 1997 len 28672
11533 281/300 Read block 3813 len 20480
11534 282/300 Read block 2563 len 12288
11535 283/300 Flush
11536 284/300 Read block 902 len 12288
11537 285/300 Write block 1116 len 12288 data: 2 3 3
11538 286/300 Read block 1209 len 32768
11539 287/300 Read block 3948 len 28672
11540 288/300 Write block 3758 len 36864 data: 2 2 2 2 2 3 3 2 2
11541 289/300 Read block 4129 len 36864
11542 290/300 Read block 1343 len 12288
11543 291/300 Read block 3034 len 20480
11544 292/300 Write block 1370 len 20480 data: 3 3 3 3 3
11545 293/300 Read block 4681 len 32768
11546 294/300 Flush
11547 295/300 Read block 205 len 16384
11548 296/300 Read block 4492 len 16384
11549 297/300 Read block 1143 len 12288
11550 298/300 Read block 3595 len 32768
11551 299/300 Read block 2418 len 12288
11552 300/300 Write block 1185 len 32768 data: 3 3 3 3 3 3 3 3
11553 301/300 Flush
11554 Main task has work to do, go find it
11555 got dsc Start(1)
11556 start 1
11557 Nov 26 22:06:59.089 INFO request completed, latency_us: 203, response_code: 204, uri: /start/cid/1, method: POST, req_id: 918261d8-e0ec-4a6a-bb55-13dbb1032b93, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
11558 Replay: started 1, returned:Ok(())
11559 ----------------------------------------------------------------
11560 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
11561 States: Active Offline Active
11562 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
11563 4020 Acked Flush 0 Done Sent Done false
11564 4021 Acked Read 1 Done Sent Done false
11565 4022 Acked Write 10 Done Sent Done false
11566 4023 Acked Read 9 Done Sent Done false
11567 4024 Acked Write 6 Done Sent Done false
11568 4025 Acked Read 7 Done Sent Done false
11569 4026 Acked Read 3 Done Sent Done false
11570 4027 Acked Read 1 Done Sent Done false
11571 4028 Acked Write 10 Done Sent Done false
11572 4029 Acked Write 8 Done Sent Done false
11573 4030 Acked Write 2 Done Sent Done false
11574 4031 Acked Read 3 Done Sent Done false
11575 4032 Acked Read 9 Done Sent Done false
11576 4033 Acked Read 1 Done Sent Done false
11577 4034 Acked Flush 0 Done Sent Done false
11578 4035 Acked Write 4 Done Sent Done false
11579 4036 Acked Flush 0 Done Sent Done false
11580 4037 Acked Read 9 Done Sent Done false
11581 4038 Acked Flush 0 Done Sent Done false
11582 4039 Acked Write 6 Done Sent Done false
11583 4040 Acked Read 3 Done Sent Done false
11584 4041 Acked Read 6 Done Sent Done false
11585 4042 Acked Read 1 Done Sent Done false
11586 4043 Acked Read 8 Done Sent Done false
11587 4044 Acked Read 9 Done Sent Done false
11588 4045 Acked Read 3 Done Sent Done false
11589 4046 Acked Flush 0 Done Sent Done false
11590 4047 Acked Write 7 Done Sent Done false
11591 4048 Acked Read 4 Done Sent Done false
11592 4049 Acked Read 1 Done Sent Done false
11593 4050 Acked Write 3 Done Sent Done false
11594 4051 Acked Write 8 Done Sent Done false
11595 4052 Acked Read 1 Done Sent Done false
11596 4053 Acked Read 10 Done Sent Done false
11597 4054 Acked Read 2 Done Sent Done false
11598 4055 Acked Read 10 Done Sent Done false
11599 4056 Acked Write 3 Done Sent Done false
11600 4057 Acked Read 10 Done Sent Done false
11601 4058 Acked Read 5 Done Sent Done false
11602 4059 Acked Write 4 Done Sent Done false
11603 4060 Acked Write 7 Done Sent Done false
11604 4061 Acked Read 5 Done Sent Done false
11605 4062 Acked Flush 0 Done Sent Done false
11606 4063 Acked Read 8 Done Sent Done false
11607 4064 Acked Read 1 Done Sent Done false
11608 4065 Acked Write 1 Done Sent Done false
11609 4066 Acked Read 5 Done Sent Done false
11610 4067 Acked Write 6 Done Sent Done false
11611 4068 Acked Write 7 Done Sent Done false
11612 4069 Acked Read 5 Done Sent Done false
11613 4070 Acked Write 8 Done Sent Done false
11614 4071 Acked Read 9 Done Sent Done false
11615 4072 Acked Read 8 Done Sent Done false
11616 4073 Acked Write 3 Done Sent Done false
11617 4074 Acked Write 3 Done Sent Done false
11618 4075 Acked Read 2 Done Sent Done false
11619 4076 Acked Read 10 Done Sent Done false
11620 4077 Acked Read 10 Done Sent Done false
11621 4078 Acked Write 1 Done Sent Done false
11622 4079 Acked Write 6 Done Sent Done false
11623 4080 Acked Read 7 Done Sent Done false
11624 4081 Acked Read 9 Done Sent Done false
11625 4082 Acked Flush 0 Done Sent Done false
11626 4083 Acked Read 7 Done Sent Done false
11627 4084 Acked Read 5 Done Sent Done false
11628 4085 Acked Read 3 Done Sent Done false
11629 4086 Acked Read 8 Done Sent Done false
11630 4087 Acked Read 10 Done Sent Done false
11631 4088 Acked Write 4 Done Sent Done false
11632 4089 Acked Write 4 Done Sent Done false
11633 4090 Acked Read 3 Done Sent Done false
11634 4091 Acked Read 10 Done Sent Done false
11635 4092 Acked Write 7 Done Sent Done false
11636 4093 Acked Flush 0 Done Sent Done false
11637 4094 Acked Flush 0 Done Sent Done false
11638 4095 Acked Read 4 Done Sent Done false
11639 4096 Acked Read 4 Done Sent Done false
11640 4097 Acked Read 1 Done Sent Done false
11641 4098 Acked Read 4 Done Sent Done false
11642 4099 Acked Read 9 Done Sent Done false
11643 4100 Acked Flush 0 Done Sent Done false
11644 4101 Acked Write 2 Done Sent Done false
11645 4102 Acked Write 10 Done Sent Done false
11646 4103 Acked Read 3 Done Sent Done false
11647 4104 Acked Write 10 Done Sent Done false
11648 4105 Acked Read 5 Done Sent Done false
11649 4106 Acked Flush 0 Done Sent Done false
11650 4107 Acked Read 9 Done Sent Done false
11651 4108 Acked Read 8 Done Sent Done false
11652 4109 Acked Read 3 Done Sent Done false
11653 4110 Acked Flush 0 Done Sent Done false
11654 4111 Acked Write 10 Done Sent Done false
11655 4112 Acked Read 9 Done Sent Done false
11656 4113 Acked Read 9 Done Sent Done false
11657 4114 Acked Read 5 Done Sent Done false
11658 4115 Acked Read 7 Done Sent Done false
11659 4116 Acked Write 8 Done Sent Done false
11660 4117 Acked Write 4 Done Sent Done false
11661 4118 Acked Read 2 Done Sent Done false
11662 4119 Acked Write 7 Done Sent Done false
11663 4120 Acked Read 5 Done Sent Done false
11664 4121 Acked Write 10 Done Sent Done false
11665 4122 Acked Read 8 Done Sent Done false
11666 4123 Acked Write 5 Done Sent Done false
11667 4124 Acked Write 6 Done Sent Done false
11668 4125 Acked Read 9 Done Sent Done false
11669 4126 Acked Read 6 Done Sent Done false
11670 4127 Acked Write 7 Done Sent Done false
11671 4128 Acked Flush 0 Done Sent Done false
11672 4129 Acked Read 8 Done Sent Done false
11673 4130 Acked Flush 0 Done Sent Done false
11674 4131 Acked Write 6 Done Sent Done false
11675 4132 Acked Read 4 Done Sent Done false
11676 4133 Acked Read 1 Done Sent Done false
11677 4134 Acked Read 2 Done Sent Done false
11678 4135 Acked Flush 0 Done Sent Done false
11679 4136 Acked Write 9 Done Sent Done false
11680 4137 Acked Read 3 Done Sent Done false
11681 4138 Acked Read 6 Done Sent Done false
11682 4139 Acked Read 7 Done Sent Done false
11683 4140 Acked Read 3 Done Sent Done false
11684 4141 Acked Write 1 Done Sent Done false
11685 4142 Acked Write 9 Done Sent Done false
11686 4143 Acked Write 1 Done Sent Done false
11687 4144 Acked Read 9 Done Sent Done false
11688 4145 Acked Flush 0 Done Sent Done false
11689 4146 Acked Write 7 Done Sent Done false
11690 4147 Acked Write 2 Done Sent Done false
11691 4148 Acked Write 10 Done Sent Done false
11692 4149 Acked Read 6 Done Sent Done false
11693 4150 Acked Read 5 Done Sent Done false
11694 4151 Acked Write 2 Done Sent Done false
11695 4152 Acked Write 4 Done Sent Done false
11696 4153 Acked Write 3 Done Sent Done false
11697 4154 Acked Write 7 Done Sent Done false
11698 4155 Acked Write 3 Done Sent Done false
11699 4156 Acked Write 3 Done Sent Done false
11700 4157 Acked Read 1 Done Sent Done false
11701 4158 Acked Write 10 Done Sent Done false
11702 4159 Acked Read 2 Done Sent Done false
11703 4160 Acked Read 1 Done Sent Done false
11704 4161 Acked Read 1 Done Sent Done false
11705 4162 Acked Read 8 Done Sent Done false
11706 4163 Acked Read 9 Done Sent Done false
11707 4164 Acked Read 2 Done Sent Done false
11708 4165 Acked Write 7 Done Sent Done false
11709 4166 Acked Write 1 Done Sent Done false
11710 4167 Acked Write 6 Done Sent Done false
11711 4168 Acked Write 5 Done Sent Done false
11712 4169 Acked Write 3 Done Sent Done false
11713 4170 Acked Read 1 Done Sent Done false
11714 4171 Acked Read 5 Done Sent Done false
11715 4172 Acked Read 3 Done Sent Done false
11716 4173 Acked Read 6 Done Sent Done false
11717 4174 Acked Read 10 Done Sent Done false
11718 4175 Acked Write 1 Done Sent Done false
11719 4176 Acked Read 3 Done Sent Done false
11720 4177 Acked Write 3 Done Sent Done false
11721 4178 Acked Read 6 Done Sent Done false
11722 4179 Acked Write 1 Done Sent Done false
11723 4180 Acked Read 10 Done Sent Done false
11724 4181 Acked Write 6 Done Sent Done false
11725 4182 Acked Read 6 Done Sent Done false
11726 4183 Acked Write 4 Done Sent Done false
11727 4184 Acked Read 10 Done Sent Done false
11728 4185 Acked Read 4 Done Sent Done false
11729 4186 Acked Read 3 Done Sent Done false
11730 4187 Acked Write 7 Done Sent Done false
11731 4188 Acked Flush 0 Done Sent Done false
11732 4189 Acked Read 8 Done Sent Done false
11733 4190 Acked Read 4 Done Sent Done false
11734 4191 Acked Read 5 Done Sent Done false
11735 4192 Acked Write 9 Done Sent Done false
11736 4193 Acked Write 8 Done Sent Done false
11737 4194 Acked Write 2 Done Sent Done false
11738 4195 Acked Write 5 Done Sent Done false
11739 4196 Acked Read 7 Done Sent Done false
11740 4197 Acked Write 8 Done Sent Done false
11741 4198 Acked Read 5 Done Sent Done false
11742 4199 Acked Read 8 Done Sent Done false
11743 4200 Acked Read 7 Done Sent Done false
11744 4201 Acked Read 9 Done Sent Done false
11745 4202 Acked Write 8 Done Sent Done false
11746 4203 Acked Write 8 Done Sent Done false
11747 4204 Acked Write 4 Done Sent Done false
11748 4205 Acked Flush 0 Done Sent Done false
11749 4206 Acked Write 9 Done Sent Done false
11750 4207 Acked Read 3 Done Sent Done false
11751 4208 Acked Write 6 Done Sent Done false
11752 4209 Acked Write 4 Done Sent Done false
11753 4210 Acked Flush 0 Done Sent Done false
11754 4211 Acked Read 8 Done Sent Done false
11755 4212 Acked Write 2 Done Sent Done false
11756 4213 Acked Flush 0 Done Sent Done false
11757 4214 Acked Write 8 Done Sent Done false
11758 4215 Acked Read 4 Done Sent Done false
11759 4216 Acked Read 1 Done Sent Done false
11760 4217 Acked Read 3 Done Sent Done false
11761 4218 Acked Write 6 Done Sent Done false
11762 4219 Acked Read 9 Done Sent Done false
11763 4220 Acked Read 9 Done Sent Done false
11764 4221 Acked Write 1 Done Sent Done false
11765 4222 Acked Read 5 Done Sent Done false
11766 4223 Acked Read 3 Done Sent Done false
11767 4224 Acked Write 8 Done Sent Done false
11768 4225 Acked Write 8 Done Sent Done false
11769 4226 Acked Read 1 Done Sent Done false
11770 4227 Acked Read 1 Done Sent Done false
11771 4228 Acked Write 4 Done Sent Done false
11772 4229 Acked Read 10 Done Sent Done false
11773 4230 Acked Read 6 Done Sent Done false
11774 4231 Acked Write 7 Done Sent Done false
11775 4232 Acked Read 9 Done Sent Done false
11776 4233 Acked Read 2 Done Sent Done false
11777 4234 Acked Write 1 Done Sent Done false
11778 4235 Acked Read 9 Done Sent Done false
11779 4236 Acked Write 5 Done Sent Done false
11780 4237 Acked Write 10 Done Sent Done false
11781 4238 Acked Read 3 Done Sent Done false
11782 4239 Acked Write 2 Done Sent Done false
11783 4240 Acked Read 4 Done Sent Done false
11784 4241 Acked Read 1 Done Sent Done false
11785 4242 Acked Write 6 Done Sent Done false
11786 4243 Acked Read 9 Done Sent Done false
11787 4244 Acked Read 7 Done Sent Done false
11788 4245 Acked Read 10 Done Sent Done false
11789 4246 Acked Write 3 Done Sent Done false
11790 4247 Acked Read 5 Done Sent Done false
11791 4248 Acked Write 4 Done Sent Done false
11792 4249 Acked Read 5 Done Sent Done false
11793 4250 Acked Read 1 Done Sent Done false
11794 4251 Acked Write 1 Done Sent Done false
11795 4252 Acked Read 7 Done Sent Done false
11796 4253 Acked Write 1 Done Sent Done false
11797 4254 Acked Flush 0 Done Sent Done false
11798 4255 Acked Read 10 Done Sent Done false
11799 4256 Acked Read 6 Done Sent Done false
11800 4257 Acked Flush 0 Done Sent Done false
11801 4258 Acked Read 8 Done Sent Done false
11802 4259 Acked Write 1 Done Sent Done false
11803 4260 Acked Write 5 Done Sent Done false
11804 4261 Acked Write 5 Done Sent Done false
11805 4262 Acked Write 5 Done Sent Done false
11806 4263 Acked Read 3 Done Sent Done false
11807 4264 Acked Write 4 Done Sent Done false
11808 4265 Acked Write 6 Done Sent Done false
11809 4266 Acked Write 8 Done Sent Done false
11810 4267 Acked Write 2 Done Sent Done false
11811 4268 Acked Write 10 Done Sent Done false
11812 4269 Acked Write 8 Done Sent Done false
11813 4270 Acked Read 9 Done Sent Done false
11814 4271 Acked Read 6 Done Sent Done false
11815 4272 Acked Flush 0 Done Sent Done false
11816 4273 Acked Write 7 Done Sent Done false
11817 4274 Acked Read 4 Done Sent Done false
11818 4275 Acked Read 1 Done Sent Done false
11819 4276 Acked Read 2 Done Sent Done false
11820 4277 Acked Write 8 Done Sent Done false
11821 4278 Acked Write 9 Done Sent Done false
11822 4279 Acked Write 3 Done Sent Done false
11823 4280 Acked Read 4 Done Sent Done false
11824 4281 Acked Read 10 Done Sent Done false
11825 4282 Acked Write 6 Done Sent Done false
11826 4283 Acked Read 8 Done Sent Done false
11827 4284 Acked Write 7 Done Sent Done false
11828 4285 Acked Read 5 Done Sent Done false
11829 4286 Acked Write 7 Done Sent Done false
11830 4287 Acked Write 8 Done Sent Done false
11831 4288 Acked Read 4 Done Sent Done false
11832 4289 Acked Flush 0 Done Sent Done false
11833 4290 Acked Read 6 Done Sent Done false
11834 4291 Acked Read 6 Done Sent Done false
11835 4292 Acked Write 7 Done Sent Done false
11836 4293 Acked Write 9 Done Sent Done false
11837 4294 Acked Read 1 Done Sent Done false
11838 4295 Acked Read 9 Done Sent Done false
11839 4296 Acked Read 9 Done Sent Done false
11840 4297 Acked Read 2 Done Sent Done false
11841 4298 Acked Write 3 Done Sent Done false
11842 4299 Acked Read 7 Done Sent Done false
11843 4300 Acked Read 5 Done Sent Done false
11844 4301 Acked Read 3 Done Sent Done false
11845 4302 Acked Flush 0 Done Sent Done false
11846 4303 Acked Read 3 Done Sent Done false
11847 4304 Acked Write 3 Done Sent Done false
11848 4305 Acked Read 8 Done Sent Done false
11849 4306 Acked Read 7 Done Sent Done false
11850 4307 Acked Write 9 Done Sent Done false
11851 4308 Acked Read 9 Done Sent Done false
11852 4309 Acked Read 3 Done Sent Done false
11853 4310 Acked Read 5 Done Sent Done false
11854 4311 Acked Write 5 Done Sent Done false
11855 4312 Acked Read 8 Done Sent Done false
11856 4313 Acked Flush 0 Done Sent Done false
11857 4314 Acked Read 4 Done Sent Done false
11858 4315 Acked Read 4 Done Sent Done false
11859 4316 Acked Read 3 Done Sent Done false
11860 4317 Acked Read 8 Done Sent Done false
11861 4318 Acked Read 3 Done Sent Done false
11862 4319 Acked Write 8 Done Sent Done false
11863 4320 Acked Flush 0 Done Sent Done false
11864 STATES DS:0 DS:1 DS:2 TOTAL
11865 Sent 0 301 0 301
11866 Done 301 0 301 602
11867 Skipped 0 0 0 0
11868 Error 0 0 0 0
11869 Last Flush: 4320 4019 4320
11870 Downstairs last five completed: 4019 4018 4017 4016 4015
11871 Upstairs last five completed: 4320 4319 4318 4317 4316
11872 CLIENT: Up:0 ds:301 act:2
11873 [8820] got start action
11874 [8820] I am going to restart
11875 Starting downstairs at port 8820
11876 Make output file at "/tmp/dsc/downstairs-8820.txt"
11877 [8820][1] reports Starting
11878 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1214)
11879 [8820][1] reports Running
11880 ----------------------------------------------------------------
11881 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
11882 States: Active Offline Active
11883 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
11884 4020 Acked Flush 0 Done Sent Done false
11885 4021 Acked Read 1 Done Sent Done false
11886 4022 Acked Write 10 Done Sent Done false
11887 4023 Acked Read 9 Done Sent Done false
11888 4024 Acked Write 6 Done Sent Done false
11889 4025 Acked Read 7 Done Sent Done false
11890 4026 Acked Read 3 Done Sent Done false
11891 4027 Acked Read 1 Done Sent Done false
11892 4028 Acked Write 10 Done Sent Done false
11893 4029 Acked Write 8 Done Sent Done false
11894 4030 Acked Write 2 Done Sent Done false
11895 4031 Acked Read 3 Done Sent Done false
11896 4032 Acked Read 9 Done Sent Done false
11897 4033 Acked Read 1 Done Sent Done false
11898 4034 Acked Flush 0 Done Sent Done false
11899 4035 Acked Write 4 Done Sent Done false
11900 4036 Acked Flush 0 Done Sent Done false
11901 4037 Acked Read 9 Done Sent Done false
11902 4038 Acked Flush 0 Done Sent Done false
11903 4039 Acked Write 6 Done Sent Done false
11904 4040 Acked Read 3 Done Sent Done false
11905 4041 Acked Read 6 Done Sent Done false
11906 4042 Acked Read 1 Done Sent Done false
11907 4043 Acked Read 8 Done Sent Done false
11908 4044 Acked Read 9 Done Sent Done false
11909 4045 Acked Read 3 Done Sent Done false
11910 4046 Acked Flush 0 Done Sent Done false
11911 4047 Acked Write 7 Done Sent Done false
11912 4048 Acked Read 4 Done Sent Done false
11913 4049 Acked Read 1 Done Sent Done false
11914 4050 Acked Write 3 Done Sent Done false
11915 4051 Acked Write 8 Done Sent Done false
11916 4052 Acked Read 1 Done Sent Done false
11917 4053 Acked Read 10 Done Sent Done false
11918 4054 Acked Read 2 Done Sent Done false
11919 4055 Acked Read 10 Done Sent Done false
11920 4056 Acked Write 3 Done Sent Done false
11921 4057 Acked Read 10 Done Sent Done false
11922 4058 Acked Read 5 Done Sent Done false
11923 4059 Acked Write 4 Done Sent Done false
11924 4060 Acked Write 7 Done Sent Done false
11925 4061 Acked Read 5 Done Sent Done false
11926 4062 Acked Flush 0 Done Sent Done false
11927 4063 Acked Read 8 Done Sent Done false
11928 4064 Acked Read 1 Done Sent Done false
11929 4065 Acked Write 1 Done Sent Done false
11930 4066 Acked Read 5 Done Sent Done false
11931 4067 Acked Write 6 Done Sent Done false
11932 4068 Acked Write 7 Done Sent Done false
11933 4069 Acked Read 5 Done Sent Done false
11934 4070 Acked Write 8 Done Sent Done false
11935 4071 Acked Read 9 Done Sent Done false
11936 4072 Acked Read 8 Done Sent Done false
11937 4073 Acked Write 3 Done Sent Done false
11938 4074 Acked Write 3 Done Sent Done false
11939 4075 Acked Read 2 Done Sent Done false
11940 4076 Acked Read 10 Done Sent Done false
11941 4077 Acked Read 10 Done Sent Done false
11942 4078 Acked Write 1 Done Sent Done false
11943 4079 Acked Write 6 Done Sent Done false
11944 4080 Acked Read 7 Done Sent Done false
11945 4081 Acked Read 9 Done Sent Done false
11946 4082 Acked Flush 0 Done Sent Done false
11947 4083 Acked Read 7 Done Sent Done false
11948 4084 Acked Read 5 Done Sent Done false
11949 4085 Acked Read 3 Done Sent Done false
11950 4086 Acked Read 8 Done Sent Done false
11951 4087 Acked Read 10 Done Sent Done false
11952 4088 Acked Write 4 Done Sent Done false
11953 4089 Acked Write 4 Done Sent Done false
11954 4090 Acked Read 3 Done Sent Done false
11955 4091 Acked Read 10 Done Sent Done false
11956 4092 Acked Write 7 Done Sent Done false
11957 4093 Acked Flush 0 Done Sent Done false
11958 4094 Acked Flush 0 Done Sent Done false
11959 4095 Acked Read 4 Done Sent Done false
11960 4096 Acked Read 4 Done Sent Done false
11961 4097 Acked Read 1 Done Sent Done false
11962 4098 Acked Read 4 Done Sent Done false
11963 4099 Acked Read 9 Done Sent Done false
11964 4100 Acked Flush 0 Done Sent Done false
11965 4101 Acked Write 2 Done Sent Done false
11966 4102 Acked Write 10 Done Sent Done false
11967 4103 Acked Read 3 Done Sent Done false
11968 4104 Acked Write 10 Done Sent Done false
11969 4105 Acked Read 5 Done Sent Done false
11970 4106 Acked Flush 0 Done Sent Done false
11971 4107 Acked Read 9 Done Sent Done false
11972 4108 Acked Read 8 Done Sent Done false
11973 4109 Acked Read 3 Done Sent Done false
11974 4110 Acked Flush 0 Done Sent Done false
11975 4111 Acked Write 10 Done Sent Done false
11976 4112 Acked Read 9 Done Sent Done false
11977 4113 Acked Read 9 Done Sent Done false
11978 4114 Acked Read 5 Done Sent Done false
11979 4115 Acked Read 7 Done Sent Done false
11980 4116 Acked Write 8 Done Sent Done false
11981 4117 Acked Write 4 Done Sent Done false
11982 4118 Acked Read 2 Done Sent Done false
11983 4119 Acked Write 7 Done Sent Done false
11984 4120 Acked Read 5 Done Sent Done false
11985 4121 Acked Write 10 Done Sent Done false
11986 4122 Acked Read 8 Done Sent Done false
11987 4123 Acked Write 5 Done Sent Done false
11988 4124 Acked Write 6 Done Sent Done false
11989 4125 Acked Read 9 Done Sent Done false
11990 4126 Acked Read 6 Done Sent Done false
11991 4127 Acked Write 7 Done Sent Done false
11992 4128 Acked Flush 0 Done Sent Done false
11993 4129 Acked Read 8 Done Sent Done false
11994 4130 Acked Flush 0 Done Sent Done false
11995 4131 Acked Write 6 Done Sent Done false
11996 4132 Acked Read 4 Done Sent Done false
11997 4133 Acked Read 1 Done Sent Done false
11998 4134 Acked Read 2 Done Sent Done false
11999 4135 Acked Flush 0 Done Sent Done false
12000 4136 Acked Write 9 Done Sent Done false
12001 4137 Acked Read 3 Done Sent Done false
12002 4138 Acked Read 6 Done Sent Done false
12003 4139 Acked Read 7 Done Sent Done false
12004 4140 Acked Read 3 Done Sent Done false
12005 4141 Acked Write 1 Done Sent Done false
12006 4142 Acked Write 9 Done Sent Done false
12007 4143 Acked Write 1 Done Sent Done false
12008 4144 Acked Read 9 Done Sent Done false
12009 4145 Acked Flush 0 Done Sent Done false
12010 4146 Acked Write 7 Done Sent Done false
12011 4147 Acked Write 2 Done Sent Done false
12012 4148 Acked Write 10 Done Sent Done false
12013 4149 Acked Read 6 Done Sent Done false
12014 4150 Acked Read 5 Done Sent Done false
12015 4151 Acked Write 2 Done Sent Done false
12016 4152 Acked Write 4 Done Sent Done false
12017 4153 Acked Write 3 Done Sent Done false
12018 4154 Acked Write 7 Done Sent Done false
12019 4155 Acked Write 3 Done Sent Done false
12020 4156 Acked Write 3 Done Sent Done false
12021 4157 Acked Read 1 Done Sent Done false
12022 4158 Acked Write 10 Done Sent Done false
12023 4159 Acked Read 2 Done Sent Done false
12024 4160 Acked Read 1 Done Sent Done false
12025 4161 Acked Read 1 Done Sent Done false
12026 4162 Acked Read 8 Done Sent Done false
12027 4163 Acked Read 9 Done Sent Done false
12028 4164 Acked Read 2 Done Sent Done false
12029 4165 Acked Write 7 Done Sent Done false
12030 4166 Acked Write 1 Done Sent Done false
12031 4167 Acked Write 6 Done Sent Done false
12032 4168 Acked Write 5 Done Sent Done false
12033 4169 Acked Write 3 Done Sent Done false
12034 4170 Acked Read 1 Done Sent Done false
12035 4171 Acked Read 5 Done Sent Done false
12036 4172 Acked Read 3 Done Sent Done false
12037 4173 Acked Read 6 Done Sent Done false
12038 4174 Acked Read 10 Done Sent Done false
12039 4175 Acked Write 1 Done Sent Done false
12040 4176 Acked Read 3 Done Sent Done false
12041 4177 Acked Write 3 Done Sent Done false
12042 4178 Acked Read 6 Done Sent Done false
12043 4179 Acked Write 1 Done Sent Done false
12044 4180 Acked Read 10 Done Sent Done false
12045 4181 Acked Write 6 Done Sent Done false
12046 4182 Acked Read 6 Done Sent Done false
12047 4183 Acked Write 4 Done Sent Done false
12048 4184 Acked Read 10 Done Sent Done false
12049 4185 Acked Read 4 Done Sent Done false
12050 4186 Acked Read 3 Done Sent Done false
12051 4187 Acked Write 7 Done Sent Done false
12052 4188 Acked Flush 0 Done Sent Done false
12053 4189 Acked Read 8 Done Sent Done false
12054 4190 Acked Read 4 Done Sent Done false
12055 4191 Acked Read 5 Done Sent Done false
12056 4192 Acked Write 9 Done Sent Done false
12057 4193 Acked Write 8 Done Sent Done false
12058 4194 Acked Write 2 Done Sent Done false
12059 4195 Acked Write 5 Done Sent Done false
12060 4196 Acked Read 7 Done Sent Done false
12061 4197 Acked Write 8 Done Sent Done false
12062 4198 Acked Read 5 Done Sent Done false
12063 4199 Acked Read 8 Done Sent Done false
12064 4200 Acked Read 7 Done Sent Done false
12065 4201 Acked Read 9 Done Sent Done false
12066 4202 Acked Write 8 Done Sent Done false
12067 4203 Acked Write 8 Done Sent Done false
12068 4204 Acked Write 4 Done Sent Done false
12069 4205 Acked Flush 0 Done Sent Done false
12070 4206 Acked Write 9 Done Sent Done false
12071 4207 Acked Read 3 Done Sent Done false
12072 4208 Acked Write 6 Done Sent Done false
12073 4209 Acked Write 4 Done Sent Done false
12074 4210 Acked Flush 0 Done Sent Done false
12075 4211 Acked Read 8 Done Sent Done false
12076 4212 Acked Write 2 Done Sent Done false
12077 4213 Acked Flush 0 Done Sent Done false
12078 4214 Acked Write 8 Done Sent Done false
12079 4215 Acked Read 4 Done Sent Done false
12080 4216 Acked Read 1 Done Sent Done false
12081 4217 Acked Read 3 Done Sent Done false
12082 4218 Acked Write 6 Done Sent Done false
12083 4219 Acked Read 9 Done Sent Done false
12084 4220 Acked Read 9 Done Sent Done false
12085 4221 Acked Write 1 Done Sent Done false
12086 4222 Acked Read 5 Done Sent Done false
12087 4223 Acked Read 3 Done Sent Done false
12088 4224 Acked Write 8 Done Sent Done false
12089 4225 Acked Write 8 Done Sent Done false
12090 4226 Acked Read 1 Done Sent Done false
12091 4227 Acked Read 1 Done Sent Done false
12092 4228 Acked Write 4 Done Sent Done false
12093 4229 Acked Read 10 Done Sent Done false
12094 4230 Acked Read 6 Done Sent Done false
12095 4231 Acked Write 7 Done Sent Done false
12096 4232 Acked Read 9 Done Sent Done false
12097 4233 Acked Read 2 Done Sent Done false
12098 4234 Acked Write 1 Done Sent Done false
12099 4235 Acked Read 9 Done Sent Done false
12100 4236 Acked Write 5 Done Sent Done false
12101 4237 Acked Write 10 Done Sent Done false
12102 4238 Acked Read 3 Done Sent Done false
12103 4239 Acked Write 2 Done Sent Done false
12104 4240 Acked Read 4 Done Sent Done false
12105 4241 Acked Read 1 Done Sent Done false
12106 4242 Acked Write 6 Done Sent Done false
12107 4243 Acked Read 9 Done Sent Done false
12108 4244 Acked Read 7 Done Sent Done false
12109 4245 Acked Read 10 Done Sent Done false
12110 4246 Acked Write 3 Done Sent Done false
12111 4247 Acked Read 5 Done Sent Done false
12112 4248 Acked Write 4 Done Sent Done false
12113 4249 Acked Read 5 Done Sent Done false
12114 4250 Acked Read 1 Done Sent Done false
12115 4251 Acked Write 1 Done Sent Done false
12116 4252 Acked Read 7 Done Sent Done false
12117 4253 Acked Write 1 Done Sent Done false
12118 4254 Acked Flush 0 Done Sent Done false
12119 4255 Acked Read 10 Done Sent Done false
12120 4256 Acked Read 6 Done Sent Done false
12121 4257 Acked Flush 0 Done Sent Done false
12122 4258 Acked Read 8 Done Sent Done false
12123 4259 Acked Write 1 Done Sent Done false
12124 4260 Acked Write 5 Done Sent Done false
12125 4261 Acked Write 5 Done Sent Done false
12126 4262 Acked Write 5 Done Sent Done false
12127 4263 Acked Read 3 Done Sent Done false
12128 4264 Acked Write 4 Done Sent Done false
12129 4265 Acked Write 6 Done Sent Done false
12130 4266 Acked Write 8 Done Sent Done false
12131 4267 Acked Write 2 Done Sent Done false
12132 4268 Acked Write 10 Done Sent Done false
12133 4269 Acked Write 8 Done Sent Done false
12134 4270 Acked Read 9 Done Sent Done false
12135 4271 Acked Read 6 Done Sent Done false
12136 4272 Acked Flush 0 Done Sent Done false
12137 4273 Acked Write 7 Done Sent Done false
12138 4274 Acked Read 4 Done Sent Done false
12139 4275 Acked Read 1 Done Sent Done false
12140 4276 Acked Read 2 Done Sent Done false
12141 4277 Acked Write 8 Done Sent Done false
12142 4278 Acked Write 9 Done Sent Done false
12143 4279 Acked Write 3 Done Sent Done false
12144 4280 Acked Read 4 Done Sent Done false
12145 4281 Acked Read 10 Done Sent Done false
12146 4282 Acked Write 6 Done Sent Done false
12147 4283 Acked Read 8 Done Sent Done false
12148 4284 Acked Write 7 Done Sent Done false
12149 4285 Acked Read 5 Done Sent Done false
12150 4286 Acked Write 7 Done Sent Done false
12151 4287 Acked Write 8 Done Sent Done false
12152 4288 Acked Read 4 Done Sent Done false
12153 4289 Acked Flush 0 Done Sent Done false
12154 4290 Acked Read 6 Done Sent Done false
12155 4291 Acked Read 6 Done Sent Done false
12156 4292 Acked Write 7 Done Sent Done false
12157 4293 Acked Write 9 Done Sent Done false
12158 4294 Acked Read 1 Done Sent Done false
12159 4295 Acked Read 9 Done Sent Done false
12160 4296 Acked Read 9 Done Sent Done false
12161 4297 Acked Read 2 Done Sent Done false
12162 4298 Acked Write 3 Done Sent Done false
12163 4299 Acked Read 7 Done Sent Done false
12164 4300 Acked Read 5 Done Sent Done false
12165 4301 Acked Read 3 Done Sent Done false
12166 4302 Acked Flush 0 Done Sent Done false
12167 4303 Acked Read 3 Done Sent Done false
12168 4304 Acked Write 3 Done Sent Done false
12169 4305 Acked Read 8 Done Sent Done false
12170 4306 Acked Read 7 Done Sent Done false
12171 4307 Acked Write 9 Done Sent Done false
12172 4308 Acked Read 9 Done Sent Done false
12173 4309 Acked Read 3 Done Sent Done false
12174 4310 Acked Read 5 Done Sent Done false
12175 4311 Acked Write 5 Done Sent Done false
12176 4312 Acked Read 8 Done Sent Done false
12177 4313 Acked Flush 0 Done Sent Done false
12178 4314 Acked Read 4 Done Sent Done false
12179 4315 Acked Read 4 Done Sent Done false
12180 4316 Acked Read 3 Done Sent Done false
12181 4317 Acked Read 8 Done Sent Done false
12182 4318 Acked Read 3 Done Sent Done false
12183 4319 Acked Write 8 Done Sent Done false
12184 4320 Acked Flush 0 Done Sent Done false
12185 STATES DS:0 DS:1 DS:2 TOTAL
12186 Sent 0 301 0 301
12187 Done 301 0 301 602
12188 Skipped 0 0 0 0
12189 Error 0 0 0 0
12190 Last Flush: 4320 4019 4320
12191 Downstairs last five completed: 4019 4018 4017 4016 4015
12192 Upstairs last five completed: 4320 4319 4318 4317 4316
12193 CLIENT: Up:0 ds:301 act:2
121942024-11-26T22:07:04.845ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
121952024-11-26T22:07:04.845ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:61138) = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
121962024-11-26T22:07:04.846ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
121972024-11-26T22:07:04.847ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
121982024-11-26T22:07:04.847ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
121992024-11-26T22:07:04.847ZINFOcrucible: send last flush ID to this DS: 4019 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
122002024-11-26T22:07:04.847ZINFOcrucible: Replied this last flush ID: 4019 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
122012024-11-26T22:07:04.847ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
122022024-11-26T22:07:04.847ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
122032024-11-26T22:07:04.847ZINFOcrucible: [1] client re-new 301 jobs since flush 4019 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
122042024-11-26T22:07:04.847ZINFOcrucible: [1] Replayed 301 jobs since flush: 4019 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
12205 ----------------------------------------------------------------
12206 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
12207 Downstairs last five completed: 4320 4319 4318 4317 4316
12208 Upstairs last five completed: 4320 4319 4318 4317 4316
12209 CLIENT: Up:0 ds:0 act:3
12210 Replay: All jobs finished
12211 Main task has work to do, go find it
12212 got dsc Stop(2)
12213 stop 2
12214 Nov 26 22:07:07.094 INFO request completed, latency_us: 355, response_code: 204, uri: /stop/cid/2[8830] Got stop action so:false kr:false
12215 , method: POST, req_id: 178676d6-0e2d-4512-a1f0-0e2f312bdcb4, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
12216 Nov 26 22:07:07.094 INFO request completed, latency_us: 115, response_code: 200, uri: /state/cid/2, method: GET, req_id: dad994f9-02ab-49c1-9963-506da80349e0, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
122172024-11-26T22:07:07.094ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
122182024-11-26T22:07:07.094ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
122192024-11-26T22:07:07.094ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
122202024-11-26T22:07:07.094ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
122212024-11-26T22:07:07.094ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
122222024-11-26T22:07:07.094ZINFOcrucible: client task is exiting = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
12223 [8830] Exited with: ExitStatus(unix_wait_status(9))
12224 [8830][2] reports Exit
12225 Nov 26 22:07:11.097 INFO request completed, latency_us: 214, response_code: 200, uri: /state/cid/2, method: GET, req_id: b23b8e57-77bf-4fd4-a6ed-d75defb44193, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
12226 001/300 Write block 2828 len 24576 data: 3 4 4 4 4 3
12227 002/300 Read block 2610 len 36864
12228 003/300 Write block 2597 len 4096 data: 3
12229 004/300 Read block 3973 len 16384
12230 005/300 Write block 1893 len 20480 data: 4 3 4 4 4
12231 006/300 Write block 652 len 40960 data: 2 2 2 3 3 4 3 3 3 2
12232 007/300 Write block 2272 len 32768 data: 4 3 3 2 2 2 2 2
12233 008/300 Read block 909 len 4096
12234 009/300 Read block 4628 len 28672
12235 010/300 Flush
12236 011/300 Write block 1535 len 36864 data: 5 4 4 4 4 5 5 4 3
12237 012/300 Write block 2029 len 40960 data: 3 3 3 4 4 6 5 5 5 5
12238 013/300 Read block 649 len 4096
12239 014/300 Read block 4455 len 32768
12240 015/300 Write block 1080 len 4096 data: 5
12241 016/300 Read block 3475 len 16384
12242 017/300 Read block 4989 len 28672
12243 018/300 Read block 3372 len 28672
12244 019/300 Write block 4556 len 40960 data: 2 2 2 2 2 2 2 2 3 2
12245 020/300 Write block 1099 len 8192 data: 3 3
12246 021/300 Read block 4320 len 4096
12247 022/300 Write block 4357 len 40960 data: 4 4 5 6 5 6 5 6 5 5
12248 023/300 Flush
12249 024/300 Write block 4581 len 8192 data: 2 3
12250 025/300 Write block 3768 len 32768 data: 3 3 2 2 2 2 2 2
12251 026/300 Write block 2980 len 24576 data: 2 2 2 2 3 4
12252 027/300 Write block 3775 len 32768 data: 3 2 3 3 3 3 3 4
12253 028/300 Read block 4331 len 12288
12254 029/300 Write block 2766 len 8192 data: 2 2
12255 030/300 Read block 2619 len 20480
12256 031/300 Read block 4881 len 16384
12257 032/300 Read block 4669 len 24576
12258 033/300 Flush
12259 034/300 Write block 481 len 8192 data: 4 5
12260 035/300 Read block 3225 len 24576
12261 036/300 Read block 3206 len 8192
12262 037/300 Write block 4392 len 12288 data: 2 3 3
12263 038/300 Write block 508 len 24576 data: 2 4 4 4 4 4
12264 039/300 Read block 3685 len 32768
12265 040/300 Read block 4840 len 4096
12266 041/300 Flush
12267 042/300 Read block 166 len 28672
12268 043/300 Flush
12269 044/300 Read block 3387 len 20480
12270 045/300 Flush
12271 046/300 Flush
12272 047/300 Read block 2828 len 4096
12273 048/300 Read block 4113 len 36864
12274 049/300 Flush
12275 050/300 Read block 4808 len 28672
12276 051/300 Write block 1811 len 40960 data: 3 2 2 2 3 3 2 2 2 2
12277 052/300 Read block 2900 len 12288
12278 053/300 Read block 1592 len 40960
12279 054/300 Write block 1771 len 24576 data: 3 3 3 4 4 4
12280 055/300 Write block 2689 len 20480 data: 3 2 2 3 3
12281 056/300 Write block 2585 len 24576 data: 2 2 3 3 3 3
12282 057/300 Read block 1041 len 20480
12283 058/300 Read block 4883 len 40960
12284 059/300 Read block 2158 len 32768
12285 060/300 Write block 1202 len 40960 data: 5 5 5 5 3 2 2 4 4 4
12286 061/300 Read block 4869 len 28672
12287 062/300 Read block 1296 len 28672
12288 063/300 Read block 3953 len 24576
12289 064/300 Flush
12290 065/300 Write block 139 len 12288 data: 3 4 4
12291 066/300 Read block 1776 len 12288
12292 067/300 Flush
12293 068/300 Write block 6 len 24576 data: 4 4 4 3 3 2
12294 069/300 Write block 3879 len 40960 data: 3 2 2 2 2 2 2 2 2 2
12295 070/300 Read block 1285 len 4096
12296 071/300 Flush
12297 072/300 Read block 830 len 24576
12298 073/300 Read block 1371 len 32768
12299 074/300 Read block 4854 len 16384
12300 075/300 Read block 1597 len 8192
12301 076/300 Write block 4853 len 40960 data: 2 3 3 4 3 3 3 4 5 5
12302 077/300 Read block 4933 len 28672
12303 078/300 Write block 410 len 20480 data: 4 4 4 3 3
12304 079/300 Write block 1048 len 4096 data: 2
12305 080/300 Read block 2384 len 4096
12306 081/300 Read block 2072 len 12288
12307 082/300 Write block 3582 len 28672 data: 3 3 3 3 3 3 2
12308 083/300 Write block 2508 len 16384 data: 2 2 2 2
12309 084/300 Write block 1919 len 20480 data: 3 3 4 4 4
12310 085/300 Read block 146 len 24576
12311 086/300 Read block 689 len 24576
12312 087/300 Write block 2101 len 16384 data: 3 4 4 4
12313 088/300 Write block 2844 len 20480 data: 3 3 4 3 3
12314 089/300 Write block 1318 len 36864 data: 3 3 3 2 2 2 2 3 3
12315 090/300 Read block 3103 len 8192
12316 091/300 Write block 1147 len 20480 data: 5 4 4 5 5
12317 092/300 Write block 2874 len 32768 data: 2 4 4 5 3 3 3 2
12318 093/300 Write block 3557 len 36864 data: 3 2 2 3 3 2 2 4 4
12319 094/300 Flush
12320 095/300 Read block 3683 len 20480
12321 096/300 Write block 4872 len 4096 data: 5
12322 097/300 Read block 4750 len 36864
12323 098/300 Flush
12324 099/300 Read block 914 len 12288
12325 100/300 Write block 1586 len 36864 data: 2 2 2 3 3 2 2 2 4
12326 101/300 Read block 3531 len 32768
12327 102/300 Read block 4177 len 36864
12328 103/300 Read block 2676 len 24576
12329 104/300 Read block 613 len 24576
12330 105/300 Write block 1998 len 8192 data: 2 3
12331 106/300 Flush
12332 107/300 Read block 779 len 24576
12333 108/300 Write block 3921 len 8192 data: 4 3
12334 109/300 Read block 1573 len 20480
12335 110/300 Write block 3091 len 12288 data: 6 3 3
12336 111/300 Write block 2913 len 36864 data: 4 4 5 4 4 4 3 3 4
12337 112/300 Flush
12338 113/300 Write block 2401 len 28672 data: 4 4 4 4 5 3 3
12339 114/300 Read block 1155 len 4096
12340 115/300 Read block 3052 len 24576
12341 116/300 Write block 2853 len 8192 data: 3 3
12342 117/300 Write block 368 len 12288 data: 4 5 4
12343 118/300 Write block 164 len 4096 data: 3
12344 119/300 Read block 4416 len 4096
12345 120/300 Flush
12346 121/300 Write block 660 len 20480 data: 4 3 2 2 3
12347 122/300 Write block 2136 len 28672 data: 4 3 3 3 3 3 3
12348 123/300 Read block 3838 len 12288
12349 124/300 Read block 4829 len 16384
12350 125/300 Write block 3460 len 20480 data: 2 3 3 3 2
12351 126/300 Write block 3195 len 20480 data: 2 2 2 2 2
12352 127/300 Read block 3845 len 8192
12353 128/300 Write block 286 len 16384 data: 3 3 4 6
12354 129/300 Read block 1938 len 20480
12355 130/300 Read block 832 len 28672
12356 131/300 Read block 3517 len 8192
12357 132/300 Write block 2768 len 36864 data: 2 2 2 4 5 5 4 4 4
12358 133/300 Write block 3742 len 16384 data: 2 2 3 4
12359 134/300 Write block 1799 len 36864 data: 3 3 3 3 3 3 3 2 3
12360 135/300 Read block 2449 len 4096
12361 136/300 Read block 741 len 32768
12362 137/300 Read block 4954 len 20480
12363 138/300 Read block 526 len 40960
12364 139/300 Write block 3801 len 28672 data: 3 3 4 3 3 3 3
12365 140/300 Write block 3669 len 28672 data: 5 6 5 6 7 7 7
12366 141/300 Read block 3914 len 24576
12367 142/300 Flush
12368 143/300 Read block 3070 len 32768
12369 144/300 Read block 3797 len 28672
12370 145/300 Read block 2725 len 32768
12371 146/300 Read block 3882 len 20480
12372 147/300 Flush
12373 148/300 Read block 2826 len 36864
12374 149/300 Flush
12375 150/300 Write block 1965 len 32768 data: 2 2 2 2 2 2 2 2
12376 151/300 Read block 384 len 36864
12377 152/300 Read block 2359 len 32768
12378 153/300 Read block 3390 len 20480
12379 154/300 Write block 4396 len 40960 data: 3 2 3 3 2 2 2 2 2 3
12380 155/300 Read block 3643 len 20480
12381 156/300 Write block 1258 len 12288 data: 4 4 3
12382 157/300 Write block 2967 len 36864 data: 3 3 3 2 2 2 2 2 2
12383 158/300 Read block 1814 len 20480
12384 159/300 Read block 4866 len 28672
12385 160/300 Read block 803 len 12288
12386 161/300 Write block 1729 len 28672 data: 2 2 2 2 2 2 2
12387 162/300 Read block 4774 len 8192
12388 163/300 Write block 310 len 24576 data: 2 2 2 2 2 2
12389 164/300 Write block 1868 len 40960 data: 3 3 5 4 4 4 4 5 4 3
12390 165/300 Read block 3022 len 12288
12391 166/300 Read block 3923 len 12288
12392 167/300 Read block 4429 len 36864
12393 168/300 Write block 1280 len 12288 data: 3 4 5
12394 169/300 Write block 1256 len 8192 data: 3 3
12395 170/300 Write block 4171 len 28672 data: 2 2 2 2 2 2 2
12396 171/300 Read block 4635 len 36864
12397 172/300 Read block 2041 len 36864
12398 173/300 Write block 2595 len 20480 data: 3 4 4 4 4
12399 174/300 Read block 2842 len 36864
12400 175/300 Write block 4768 len 24576 data: 4 4 4 3 3 3
12401 176/300 Flush
12402 177/300 Read block 1039 len 36864
12403 178/300 Read block 3022 len 40960
12404 179/300 Read block 3603 len 12288
12405 180/300 Read block 3655 len 36864
12406 181/300 Write block 3929 len 28672 data: 3 4 4 3 3 4 3
12407 182/300 Read block 1438 len 20480
12408 183/300 Write block 3081 len 24576 data: 4 5 5 4 4 4
12409 184/300 Flush
12410 185/300 Write block 4084 len 36864 data: 2 2 2 2 3 3 3 3 2
12411 186/300 Write block 1303 len 28672 data: 2 2 2 2 2 2 2
12412 187/300 Read block 3402 len 24576
12413 188/300 Write block 2219 len 12288 data: 2 2 2
12414 189/300 Read block 2878 len 28672
12415 190/300 Read block 2641 len 16384
12416 191/300 Flush
12417 192/300 Read block 1209 len 20480
12418 193/300 Flush
12419 194/300 Read block 3991 len 32768
12420 195/300 Write block 3308 len 20480 data: 3 3 2 2 4
12421 196/300 Write block 4101 len 12288 data: 2 3 3
12422 197/300 Read block 3738 len 20480
12423 198/300 Read block 3574 len 8192
12424 199/300 Read block 3337 len 20480
12425 200/300 Write block 4570 len 16384 data: 3 3 3 3
12426 201/300 Write block 2853 len 16384 data: 4 4 3 3
12427 202/300 Read block 3626 len 4096
12428 203/300 Read block 1735 len 20480
12429 204/300 Write block 517 len 36864 data: 4 3 3 3 4 4 4 5 4
12430 205/300 Read block 2454 len 20480
12431 206/300 Read block 2811 len 36864
12432 207/300 Write block 3107 len 20480 data: 2 3 3 3 3
12433 208/300 Write block 2043 len 24576 data: 3 3 3 3 2 4
12434 209/300 Read block 2612 len 24576
12435 210/300 Read block 3567 len 24576
12436 211/300 Write block 2647 len 4096 data: 2
12437 212/300 Write block 165 len 36864 data: 3 2 2 2 2 2 2 2 2
12438 213/300 Read block 3560 len 36864
12439 214/300 Read block 428 len 40960
12440 215/300 Read block 1790 len 8192
12441 216/300 Write block 3385 len 16384 data: 3 2 2 2
12442 217/300 Read block 386 len 36864
12443 218/300 Write block 3226 len 40960 data: 5 6 8 9 8 6 6 5 3 2
12444 219/300 Read block 40 len 20480
12445 220/300 Read block 3454 len 32768
12446 221/300 Write block 2077 len 16384 data: 2 2 2 3
12447 222/300 Write block 1652 len 28672 data: 3 4 4 4 4 4 5
12448 223/300 Write block 609 len 4096 data: 3
12449 224/300 Write block 3493 len 40960 data: 4 4 4 4 3 3 3 3 3 4
12450 225/300 Read block 3425 len 36864
12451 226/300 Read block 2054 len 32768
12452 227/300 Flush
12453 228/300 Read block 4944 len 16384
12454 229/300 Flush
12455 230/300 Read block 4876 len 40960
12456 231/300 Flush
12457 232/300 Write block 1057 len 12288 data: 3 3 3
12458 233/300 Read block 2369 len 8192
12459 234/300 Read block 4900 len 24576
12460 235/300 Write block 4765 len 12288 data: 5 4 4
12461 236/300 Write block 2353 len 4096 data: 5
12462 237/300 Read block 218 len 40960
12463 238/300 Read block 379 len 20480
12464 239/300 Write block 2887 len 8192 data: 5 6
12465 240/300 Write block 4099 len 40960 data: 2 2 3 4 4 3 4 3 3 3
12466 241/300 Read block 3701 len 40960
12467 242/300 Read block 462 len 28672
12468 243/300 Write block 4770 len 36864 data: 5 4 4 4 3 3 3 3 2
12469 244/300 Write block 1748 len 40960 data: 4 4 4 4 3 3 3 3 3 4
12470 245/300 Read block 2052 len 28672
12471 246/300 Write block 366 len 8192 data: 5 5
12472 247/300 Write block 3273 len 12288 data: 3 3 3
12473 248/300 Read block 691 len 28672
12474 249/300 Read block 1579 len 12288
12475 250/300 Write block 392 len 8192 data: 2 2
12476 251/300 Read block 4315 len 28672
12477 252/300 Read block 961 len 24576
12478 253/300 Read block 3520 len 16384
12479 254/300 Write block 3404 len 28672 data: 6 5 5 4 4 4 2
12480 255/300 Read block 3481 len 8192
12481 256/300 Read block 1298 len 36864
12482 257/300 Write block 1462 len 28672 data: 2 2 2 2 2 2 2
12483 258/300 Flush
12484 259/300 Read block 1134 len 4096
12485 260/300 Write block 2700 len 8192 data: 3 2
12486 261/300 Write block 36 len 20480 data: 3 3 3 3 4
12487 262/300 Read block 4661 len 4096
12488 263/300 Write block 2152 len 4096 data: 2
12489 264/300 Flush
12490 265/300 Read block 740 len 24576
12491 266/300 Write block 659 len 20480 data: 4 5 4 3 3
12492 267/300 Write block 88 len 24576 data: 3 3 2 2 2 3
12493 268/300 Read block 1339 len 40960
12494 269/300 Read block 1433 len 32768
12495 270/300 Read block 1982 len 4096
12496 271/300 Write block 2312 len 16384 data: 2 2 3 3
12497 272/300 Write block 446 len 12288 data: 2 2 4
12498 273/300 Write block 4162 len 4096 data: 4
12499 274/300 Write block 1621 len 24576 data: 3 3 3 4 5 4
12500 275/300 Write block 4924 len 20480 data: 3 4 4 4 4
12501 276/300 Read block 4671 len 20480
12502 277/300 Read block 2437 len 12288
12503 278/300 Write block 4110 len 28672 data: 5 5 5 4 4 4 3
12504 279/300 Write block 844 len 40960 data: 2 2 2 2 2 2 2 2 3 3
12505 280/300 Read block 2241 len 32768
12506 281/300 Write block 4182 len 4096 data: 2
12507 282/300 Write block 4627 len 12288 data: 3 2 2
12508 283/300 Write block 726 len 36864 data: 4 4 4 3 3 3 3 3 4
12509 284/300 Write block 4074 len 4096 data: 6
12510 285/300 Read block 432 len 4096
12511 286/300 Read block 2377 len 20480
12512 287/300 Read block 4727 len 28672
12513 288/300 Write block 126 len 20480 data: 5 5 4 4 4
12514 289/300 Read block 2094 len 12288
12515 290/300 Flush
12516 291/300 Write block 57 len 36864 data: 3 3 3 3 4 3 3 3 3
12517 292/300 Read block 2723 len 12288
12518 293/300 Read block 3347 len 28672
12519 294/300 Read block 813 len 32768
12520 295/300 Write block 1069 len 12288 data: 2 2 2
12521 296/300 Write block 444 len 12288 data: 3 3 3
12522 297/300 Read block 1236 len 40960
12523 298/300 Write block 4537 len 20480 data: 6 4 4 3 2
12524 299/300 Read block 4366 len 4096
12525 300/300 Read block 3999 len 4096
12526 301/300 Read block 2461 len 40960
12527 Main task has work to do, go find it
12528 got dsc Start(2)
12529 start 2
12530 Nov 26 22:07:11.332 INFO request completed, latency_us: 200, response_code: 204, uri: /start/cid/2, method: POST, req_id: 2280bd38-ac64-4bc4-9e22-763ae4d990aa, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
12531 Replay: started 2, returned:Ok(())
12532 ----------------------------------------------------------------
12533 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
12534 States: Active Active Offline
12535 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
12536 4321 Acked Write 6 Done Done Sent false
12537 4322 Acked Read 9 Done Done Sent false
12538 4323 Acked Write 1 Done Done Sent false
12539 4324 Acked Read 4 Done Done Sent false
12540 4325 Acked Write 5 Done Done Sent false
12541 4326 Acked Write 10 Done Done Sent false
12542 4327 Acked Write 8 Done Done Sent false
12543 4328 Acked Read 1 Done Done Sent false
12544 4329 Acked Read 7 Done Done Sent false
12545 4330 Acked Flush 0 Done Done Sent false
12546 4331 Acked Write 9 Done Done Sent false
12547 4332 Acked Write 10 Done Done Sent false
12548 4333 Acked Read 1 Done Done Sent false
12549 4334 Acked Read 8 Done Done Sent false
12550 4335 Acked Write 1 Done Done Sent false
12551 4336 Acked Read 4 Done Done Sent false
12552 4337 Acked Read 7 Done Done Sent false
12553 4338 Acked Read 7 Done Done Sent false
12554 4339 Acked Write 10 Done Done Sent false
12555 4340 Acked Write 2 Done Done Sent false
12556 4341 Acked Read 1 Done Done Sent false
12557 4342 Acked Write 10 Done Done Sent false
12558 4343 Acked Flush 0 Done Done Sent false
12559 4344 Acked Write 2 Done Done Sent false
12560 4345 Acked Write 8 Done Done Sent false
12561 4346 Acked Write 6 Done Done Sent false
12562 4347 Acked Write 8 Done Done Sent false
12563 4348 Acked Read 3 Done Done Sent false
12564 4349 Acked Write 2 Done Done Sent false
12565 4350 Acked Read 5 Done Done Sent false
12566 4351 Acked Read 4 Done Done Sent false
12567 4352 Acked Read 6 Done Done Sent false
12568 4353 Acked Flush 0 Done Done Sent false
12569 4354 Acked Write 2 Done Done Sent false
12570 4355 Acked Read 6 Done Done Sent false
12571 4356 Acked Read 2 Done Done Sent false
12572 4357 Acked Write 3 Done Done Sent false
12573 4358 Acked Write 6 Done Done Sent false
12574 4359 Acked Read 8 Done Done Sent false
12575 4360 Acked Read 1 Done Done Sent false
12576 4361 Acked Flush 0 Done Done Sent false
12577 4362 Acked Read 7 Done Done Sent false
12578 4363 Acked Flush 0 Done Done Sent false
12579 4364 Acked Read 5 Done Done Sent false
12580 4365 Acked Flush 0 Done Done Sent false
12581 4366 Acked Flush 0 Done Done Sent false
12582 4367 Acked Read 1 Done Done Sent false
12583 4368 Acked Read 9 Done Done Sent false
12584 4369 Acked Flush 0 Done Done Sent false
12585 4370 Acked Read 7 Done Done Sent false
12586 4371 Acked Write 10 Done Done Sent false
12587 4372 Acked Read 3 Done Done Sent false
12588 4373 Acked Read 10 Done Done Sent false
12589 4374 Acked Write 6 Done Done Sent false
12590 4375 Acked Write 5 Done Done Sent false
12591 4376 Acked Write 6 Done Done Sent false
12592 4377 Acked Read 5 Done Done Sent false
12593 4378 Acked Read 10 Done Done Sent false
12594 4379 Acked Read 8 Done Done Sent false
12595 4380 Acked Write 10 Done Done Sent false
12596 4381 Acked Read 7 Done Done Sent false
12597 4382 Acked Read 7 Done Done Sent false
12598 4383 Acked Read 6 Done Done Sent false
12599 4384 Acked Flush 0 Done Done Sent false
12600 4385 Acked Write 3 Done Done Sent false
12601 4386 Acked Read 3 Done Done Sent false
12602 4387 Acked Flush 0 Done Done Sent false
12603 4388 Acked Write 6 Done Done Sent false
12604 4389 Acked Write 10 Done Done Sent false
12605 4390 Acked Read 1 Done Done Sent false
12606 4391 Acked Flush 0 Done Done Sent false
12607 4392 Acked Read 6 Done Done Sent false
12608 4393 Acked Read 8 Done Done Sent false
12609 4394 Acked Read 4 Done Done Sent false
12610 4395 Acked Read 2 Done Done Sent false
12611 4396 Acked Write 10 Done Done Sent false
12612 4397 Acked Read 7 Done Done Sent false
12613 4398 Acked Write 5 Done Done Sent false
12614 4399 Acked Write 1 Done Done Sent false
12615 4400 Acked Read 1 Done Done Sent false
12616 4401 Acked Read 3 Done Done Sent false
12617 4402 Acked Write 7 Done Done Sent false
12618 4403 Acked Write 4 Done Done Sent false
12619 4404 Acked Write 5 Done Done Sent false
12620 4405 Acked Read 6 Done Done Sent false
12621 4406 Acked Read 6 Done Done Sent false
12622 4407 Acked Write 4 Done Done Sent false
12623 4408 Acked Write 5 Done Done Sent false
12624 4409 Acked Write 9 Done Done Sent false
12625 4410 Acked Read 2 Done Done Sent false
12626 4411 Acked Write 5 Done Done Sent false
12627 4412 Acked Write 8 Done Done Sent false
12628 4413 Acked Write 9 Done Done Sent false
12629 4414 Acked Flush 0 Done Done Sent false
12630 4415 Acked Read 5 Done Done Sent false
12631 4416 Acked Write 1 Done Done Sent false
12632 4417 Acked Read 9 Done Done Sent false
12633 4418 Acked Flush 0 Done Done Sent false
12634 4419 Acked Read 3 Done Done Sent false
12635 4420 Acked Write 9 Done Done Sent false
12636 4421 Acked Read 8 Done Done Sent false
12637 4422 Acked Read 9 Done Done Sent false
12638 4423 Acked Read 6 Done Done Sent false
12639 4424 Acked Read 6 Done Done Sent false
12640 4425 Acked Write 2 Done Done Sent false
12641 4426 Acked Flush 0 Done Done Sent false
12642 4427 Acked Read 6 Done Done Sent false
12643 4428 Acked Write 2 Done Done Sent false
12644 4429 Acked Read 5 Done Done Sent false
12645 4430 Acked Write 3 Done Done Sent false
12646 4431 Acked Write 9 Done Done Sent false
12647 4432 Acked Flush 0 Done Done Sent false
12648 4433 Acked Write 7 Done Done Sent false
12649 4434 Acked Read 1 Done Done Sent false
12650 4435 Acked Read 6 Done Done Sent false
12651 4436 Acked Write 2 Done Done Sent false
12652 4437 Acked Write 3 Done Done Sent false
12653 4438 Acked Write 1 Done Done Sent false
12654 4439 Acked Read 1 Done Done Sent false
12655 4440 Acked Flush 0 Done Done Sent false
12656 4441 Acked Write 5 Done Done Sent false
12657 4442 Acked Write 7 Done Done Sent false
12658 4443 Acked Read 3 Done Done Sent false
12659 4444 Acked Read 4 Done Done Sent false
12660 4445 Acked Write 5 Done Done Sent false
12661 4446 Acked Write 5 Done Done Sent false
12662 4447 Acked Read 2 Done Done Sent false
12663 4448 Acked Write 4 Done Done Sent false
12664 4449 Acked Read 5 Done Done Sent false
12665 4450 Acked Read 7 Done Done Sent false
12666 4451 Acked Read 2 Done Done Sent false
12667 4452 Acked Write 9 Done Done Sent false
12668 4453 Acked Write 4 Done Done Sent false
12669 4454 Acked Write 9 Done Done Sent false
12670 4455 Acked Read 1 Done Done Sent false
12671 4456 Acked Read 8 Done Done Sent false
12672 4457 Acked Read 5 Done Done Sent false
12673 4458 Acked Read 10 Done Done Sent false
12674 4459 Acked Write 7 Done Done Sent false
12675 4460 Acked Write 7 Done Done Sent false
12676 4461 Acked Read 6 Done Done Sent false
12677 4462 Acked Flush 0 Done Done Sent false
12678 4463 Acked Read 8 Done Done Sent false
12679 4464 Acked Read 7 Done Done Sent false
12680 4465 Acked Read 8 Done Done Sent false
12681 4466 Acked Read 5 Done Done Sent false
12682 4467 Acked Flush 0 Done Done Sent false
12683 4468 Acked Read 9 Done Done Sent false
12684 4469 Acked Flush 0 Done Done Sent false
12685 4470 Acked Write 8 Done Done Sent false
12686 4471 Acked Read 9 Done Done Sent false
12687 4472 Acked Read 8 Done Done Sent false
12688 4473 Acked Read 5 Done Done Sent false
12689 4474 Acked Write 10 Done Done Sent false
12690 4475 Acked Read 5 Done Done Sent false
12691 4476 Acked Write 3 Done Done Sent false
12692 4477 Acked Write 9 Done Done Sent false
12693 4478 Acked Read 5 Done Done Sent false
12694 4479 Acked Read 7 Done Done Sent false
12695 4480 Acked Read 3 Done Done Sent false
12696 4481 Acked Write 7 Done Done Sent false
12697 4482 Acked Read 2 Done Done Sent false
12698 4483 Acked Write 6 Done Done Sent false
12699 4484 Acked Write 10 Done Done Sent false
12700 4485 Acked Read 3 Done Done Sent false
12701 4486 Acked Read 3 Done Done Sent false
12702 4487 Acked Read 9 Done Done Sent false
12703 4488 Acked Write 3 Done Done Sent false
12704 4489 Acked Write 2 Done Done Sent false
12705 4490 Acked Write 7 Done Done Sent false
12706 4491 Acked Read 9 Done Done Sent false
12707 4492 Acked Read 9 Done Done Sent false
12708 4493 Acked Write 5 Done Done Sent false
12709 4494 Acked Read 9 Done Done Sent false
12710 4495 Acked Write 6 Done Done Sent false
12711 4496 Acked Flush 0 Done Done Sent false
12712 4497 Acked Read 9 Done Done Sent false
12713 4498 Acked Read 10 Done Done Sent false
12714 4499 Acked Read 3 Done Done Sent false
12715 4500 Acked Read 9 Done Done Sent false
12716 4501 Acked Write 7 Done Done Sent false
12717 4502 Acked Read 5 Done Done Sent false
12718 4503 Acked Write 6 Done Done Sent false
12719 4504 Acked Flush 0 Done Done Sent false
12720 4505 Acked Write 9 Done Done Sent false
12721 4506 Acked Write 7 Done Done Sent false
12722 4507 Acked Read 6 Done Done Sent false
12723 4508 Acked Write 3 Done Done Sent false
12724 4509 Acked Read 7 Done Done Sent false
12725 4510 Acked Read 4 Done Done Sent false
12726 4511 Acked Flush 0 Done Done Sent false
12727 4512 Acked Read 5 Done Done Sent false
12728 4513 Acked Flush 0 Done Done Sent false
12729 4514 Acked Read 8 Done Done Sent false
12730 4515 Acked Write 5 Done Done Sent false
12731 4516 Acked Write 3 Done Done Sent false
12732 4517 Acked Read 5 Done Done Sent false
12733 4518 Acked Read 2 Done Done Sent false
12734 4519 Acked Read 5 Done Done Sent false
12735 4520 Acked Write 4 Done Done Sent false
12736 4521 Acked Write 4 Done Done Sent false
12737 4522 Acked Read 1 Done Done Sent false
12738 4523 Acked Read 5 Done Done Sent false
12739 4524 Acked Write 9 Done Done Sent false
12740 4525 Acked Read 5 Done Done Sent false
12741 4526 Acked Read 9 Done Done Sent false
12742 4527 Acked Write 5 Done Done Sent false
12743 4528 Acked Write 6 Done Done Sent false
12744 4529 Acked Read 6 Done Done Sent false
12745 4530 Acked Read 6 Done Done Sent false
12746 4531 Acked Write 1 Done Done Sent false
12747 4532 Acked Write 9 Done Done Sent false
12748 4533 Acked Read 9 Done Done Sent false
12749 4534 Acked Read 10 Done Done Sent false
12750 4535 Acked Read 2 Done Done Sent false
12751 4536 Acked Write 4 Done Done Sent false
12752 4537 Acked Read 9 Done Done Sent false
12753 4538 Acked Write 10 Done Done Sent false
12754 4539 Acked Read 5 Done Done Sent false
12755 4540 Acked Read 8 Done Done Sent false
12756 4541 Acked Write 4 Done Done Sent false
12757 4542 Acked Write 7 Done Done Sent false
12758 4543 Acked Write 1 Done Done Sent false
12759 4544 Acked Write 10 Done Done Sent false
12760 4545 Acked Read 9 Done Done Sent false
12761 4546 Acked Read 8 Done Done Sent false
12762 4547 Acked Flush 0 Done Done Sent false
12763 4548 Acked Read 4 Done Done Sent false
12764 4549 Acked Flush 0 Done Done Sent false
12765 4550 Acked Read 10 Done Done Sent false
12766 4551 Acked Flush 0 Done Done Sent false
12767 4552 Acked Write 3 Done Done Sent false
12768 4553 Acked Read 2 Done Done Sent false
12769 4554 Acked Read 6 Done Done Sent false
12770 4555 Acked Write 3 Done Done Sent false
12771 4556 Acked Write 1 Done Done Sent false
12772 4557 Acked Read 10 Done Done Sent false
12773 4558 Acked Read 5 Done Done Sent false
12774 4559 Acked Write 2 Done Done Sent false
12775 4560 Acked Write 10 Done Done Sent false
12776 4561 Acked Read 10 Done Done Sent false
12777 4562 Acked Read 7 Done Done Sent false
12778 4563 Acked Write 9 Done Done Sent false
12779 4564 Acked Write 10 Done Done Sent false
12780 4565 Acked Read 7 Done Done Sent false
12781 4566 Acked Write 2 Done Done Sent false
12782 4567 Acked Write 3 Done Done Sent false
12783 4568 Acked Read 7 Done Done Sent false
12784 4569 Acked Read 3 Done Done Sent false
12785 4570 Acked Write 2 Done Done Sent false
12786 4571 Acked Read 7 Done Done Sent false
12787 4572 Acked Read 6 Done Done Sent false
12788 4573 Acked Read 4 Done Done Sent false
12789 4574 Acked Write 7 Done Done Sent false
12790 4575 Acked Read 2 Done Done Sent false
12791 4576 Acked Read 9 Done Done Sent false
12792 4577 Acked Write 7 Done Done Sent false
12793 4578 Acked Flush 0 Done Done Sent false
12794 4579 Acked Read 1 Done Done Sent false
12795 4580 Acked Write 2 Done Done Sent false
12796 4581 Acked Write 5 Done Done Sent false
12797 4582 Acked Read 1 Done Done Sent false
12798 4583 Acked Write 1 Done Done Sent false
12799 4584 Acked Flush 0 Done Done Sent false
12800 4585 Acked Read 6 Done Done Sent false
12801 4586 Acked Write 5 Done Done Sent false
12802 4587 Acked Write 6 Done Done Sent false
12803 4588 Acked Read 10 Done Done Sent false
12804 4589 Acked Read 8 Done Done Sent false
12805 4590 Acked Read 1 Done Done Sent false
12806 4591 Acked Write 4 Done Done Sent false
12807 4592 Acked Write 3 Done Done Sent false
12808 4593 Acked Write 1 Done Done Sent false
12809 4594 Acked Write 6 Done Done Sent false
12810 4595 Acked Write 5 Done Done Sent false
12811 4596 Acked Read 5 Done Done Sent false
12812 4597 Acked Read 3 Done Done Sent false
12813 4598 Acked Write 7 Done Done Sent false
12814 4599 Acked Write 10 Done Done Sent false
12815 4600 Acked Read 8 Done Done Sent false
12816 4601 Acked Write 1 Done Done Sent false
12817 4602 Acked Write 3 Done Done Sent false
12818 4603 Acked Write 9 Done Done Sent false
12819 4604 Acked Write 1 Done Done Sent false
12820 4605 Acked Read 1 Done Done Sent false
12821 4606 Acked Read 5 Done Done Sent false
12822 4607 Acked Read 7 Done Done Sent false
12823 4608 Acked Write 5 Done Done Sent false
12824 4609 Acked Read 3 Done Done Sent false
12825 4610 Acked Flush 0 Done Done Sent false
12826 4611 Acked Write 9 Done Done Sent false
12827 4612 Acked Read 3 Done Done Sent false
12828 4613 Acked Read 7 Done Done Sent false
12829 4614 Acked Read 8 Done Done Sent false
12830 4615 Acked Write 3 Done Done Sent false
12831 4616 Acked Write 3 Done Done Sent false
12832 4617 Acked Read 10 Done Done Sent false
12833 4618 Acked Write 5 Done Done Sent false
12834 4619 Acked Read 1 Done Done Sent false
12835 4620 Acked Read 1 Done Done Sent false
12836 4621 Acked Read 10 Done Done Sent false
12837 STATES DS:0 DS:1 DS:2 TOTAL
12838 Sent 0 0 301 301
12839 Done 301 301 0 602
12840 Skipped 0 0 0 0
12841 Error 0 0 0 0
12842 Last Flush: 4610 4610 4320
12843 Downstairs last five completed: 4320 4319 4318 4317 4316
12844 Upstairs last five completed: 4621 4620 4619 4618 4617
12845 CLIENT: Up:0 ds:301 act:2
12846 [8830] got start action
12847 [8830] I am going to restart
12848 Starting downstairs at port 8830
12849 Make output file at "/tmp/dsc/downstairs-8830.txt"
12850 [8830][2] reports Starting
12851 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1215)
12852 [8830][2] reports Running
12853 ----------------------------------------------------------------
12854 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
12855 States: Active Active Offline
12856 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
12857 4321 Acked Write 6 Done Done Sent false
12858 4322 Acked Read 9 Done Done Sent false
12859 4323 Acked Write 1 Done Done Sent false
12860 4324 Acked Read 4 Done Done Sent false
12861 4325 Acked Write 5 Done Done Sent false
12862 4326 Acked Write 10 Done Done Sent false
12863 4327 Acked Write 8 Done Done Sent false
12864 4328 Acked Read 1 Done Done Sent false
12865 4329 Acked Read 7 Done Done Sent false
12866 4330 Acked Flush 0 Done Done Sent false
12867 4331 Acked Write 9 Done Done Sent false
12868 4332 Acked Write 10 Done Done Sent false
12869 4333 Acked Read 1 Done Done Sent false
12870 4334 Acked Read 8 Done Done Sent false
12871 4335 Acked Write 1 Done Done Sent false
12872 4336 Acked Read 4 Done Done Sent false
12873 4337 Acked Read 7 Done Done Sent false
12874 4338 Acked Read 7 Done Done Sent false
12875 4339 Acked Write 10 Done Done Sent false
12876 4340 Acked Write 2 Done Done Sent false
12877 4341 Acked Read 1 Done Done Sent false
12878 4342 Acked Write 10 Done Done Sent false
12879 4343 Acked Flush 0 Done Done Sent false
12880 4344 Acked Write 2 Done Done Sent false
12881 4345 Acked Write 8 Done Done Sent false
12882 4346 Acked Write 6 Done Done Sent false
12883 4347 Acked Write 8 Done Done Sent false
12884 4348 Acked Read 3 Done Done Sent false
12885 4349 Acked Write 2 Done Done Sent false
12886 4350 Acked Read 5 Done Done Sent false
12887 4351 Acked Read 4 Done Done Sent false
12888 4352 Acked Read 6 Done Done Sent false
12889 4353 Acked Flush 0 Done Done Sent false
12890 4354 Acked Write 2 Done Done Sent false
12891 4355 Acked Read 6 Done Done Sent false
12892 4356 Acked Read 2 Done Done Sent false
12893 4357 Acked Write 3 Done Done Sent false
12894 4358 Acked Write 6 Done Done Sent false
12895 4359 Acked Read 8 Done Done Sent false
12896 4360 Acked Read 1 Done Done Sent false
12897 4361 Acked Flush 0 Done Done Sent false
12898 4362 Acked Read 7 Done Done Sent false
12899 4363 Acked Flush 0 Done Done Sent false
12900 4364 Acked Read 5 Done Done Sent false
12901 4365 Acked Flush 0 Done Done Sent false
12902 4366 Acked Flush 0 Done Done Sent false
12903 4367 Acked Read 1 Done Done Sent false
12904 4368 Acked Read 9 Done Done Sent false
12905 4369 Acked Flush 0 Done Done Sent false
12906 4370 Acked Read 7 Done Done Sent false
12907 4371 Acked Write 10 Done Done Sent false
12908 4372 Acked Read 3 Done Done Sent false
12909 4373 Acked Read 10 Done Done Sent false
12910 4374 Acked Write 6 Done Done Sent false
12911 4375 Acked Write 5 Done Done Sent false
12912 4376 Acked Write 6 Done Done Sent false
12913 4377 Acked Read 5 Done Done Sent false
12914 4378 Acked Read 10 Done Done Sent false
12915 4379 Acked Read 8 Done Done Sent false
12916 4380 Acked Write 10 Done Done Sent false
12917 4381 Acked Read 7 Done Done Sent false
12918 4382 Acked Read 7 Done Done Sent false
12919 4383 Acked Read 6 Done Done Sent false
12920 4384 Acked Flush 0 Done Done Sent false
12921 4385 Acked Write 3 Done Done Sent false
12922 4386 Acked Read 3 Done Done Sent false
12923 4387 Acked Flush 0 Done Done Sent false
12924 4388 Acked Write 6 Done Done Sent false
12925 4389 Acked Write 10 Done Done Sent false
12926 4390 Acked Read 1 Done Done Sent false
12927 4391 Acked Flush 0 Done Done Sent false
12928 4392 Acked Read 6 Done Done Sent false
12929 4393 Acked Read 8 Done Done Sent false
12930 4394 Acked Read 4 Done Done Sent false
12931 4395 Acked Read 2 Done Done Sent false
12932 4396 Acked Write 10 Done Done Sent false
12933 4397 Acked Read 7 Done Done Sent false
12934 4398 Acked Write 5 Done Done Sent false
12935 4399 Acked Write 1 Done Done Sent false
12936 4400 Acked Read 1 Done Done Sent false
12937 4401 Acked Read 3 Done Done Sent false
12938 4402 Acked Write 7 Done Done Sent false
12939 4403 Acked Write 4 Done Done Sent false
12940 4404 Acked Write 5 Done Done Sent false
12941 4405 Acked Read 6 Done Done Sent false
12942 4406 Acked Read 6 Done Done Sent false
12943 4407 Acked Write 4 Done Done Sent false
12944 4408 Acked Write 5 Done Done Sent false
12945 4409 Acked Write 9 Done Done Sent false
12946 4410 Acked Read 2 Done Done Sent false
12947 4411 Acked Write 5 Done Done Sent false
12948 4412 Acked Write 8 Done Done Sent false
12949 4413 Acked Write 9 Done Done Sent false
12950 4414 Acked Flush 0 Done Done Sent false
12951 4415 Acked Read 5 Done Done Sent false
12952 4416 Acked Write 1 Done Done Sent false
12953 4417 Acked Read 9 Done Done Sent false
12954 4418 Acked Flush 0 Done Done Sent false
12955 4419 Acked Read 3 Done Done Sent false
12956 4420 Acked Write 9 Done Done Sent false
12957 4421 Acked Read 8 Done Done Sent false
12958 4422 Acked Read 9 Done Done Sent false
12959 4423 Acked Read 6 Done Done Sent false
12960 4424 Acked Read 6 Done Done Sent false
12961 4425 Acked Write 2 Done Done Sent false
12962 4426 Acked Flush 0 Done Done Sent false
12963 4427 Acked Read 6 Done Done Sent false
12964 4428 Acked Write 2 Done Done Sent false
12965 4429 Acked Read 5 Done Done Sent false
12966 4430 Acked Write 3 Done Done Sent false
12967 4431 Acked Write 9 Done Done Sent false
12968 4432 Acked Flush 0 Done Done Sent false
12969 4433 Acked Write 7 Done Done Sent false
12970 4434 Acked Read 1 Done Done Sent false
12971 4435 Acked Read 6 Done Done Sent false
12972 4436 Acked Write 2 Done Done Sent false
12973 4437 Acked Write 3 Done Done Sent false
12974 4438 Acked Write 1 Done Done Sent false
12975 4439 Acked Read 1 Done Done Sent false
12976 4440 Acked Flush 0 Done Done Sent false
12977 4441 Acked Write 5 Done Done Sent false
12978 4442 Acked Write 7 Done Done Sent false
12979 4443 Acked Read 3 Done Done Sent false
12980 4444 Acked Read 4 Done Done Sent false
12981 4445 Acked Write 5 Done Done Sent false
12982 4446 Acked Write 5 Done Done Sent false
12983 4447 Acked Read 2 Done Done Sent false
12984 4448 Acked Write 4 Done Done Sent false
12985 4449 Acked Read 5 Done Done Sent false
12986 4450 Acked Read 7 Done Done Sent false
12987 4451 Acked Read 2 Done Done Sent false
12988 4452 Acked Write 9 Done Done Sent false
12989 4453 Acked Write 4 Done Done Sent false
12990 4454 Acked Write 9 Done Done Sent false
12991 4455 Acked Read 1 Done Done Sent false
12992 4456 Acked Read 8 Done Done Sent false
12993 4457 Acked Read 5 Done Done Sent false
12994 4458 Acked Read 10 Done Done Sent false
12995 4459 Acked Write 7 Done Done Sent false
12996 4460 Acked Write 7 Done Done Sent false
12997 4461 Acked Read 6 Done Done Sent false
12998 4462 Acked Flush 0 Done Done Sent false
12999 4463 Acked Read 8 Done Done Sent false
13000 4464 Acked Read 7 Done Done Sent false
13001 4465 Acked Read 8 Done Done Sent false
13002 4466 Acked Read 5 Done Done Sent false
13003 4467 Acked Flush 0 Done Done Sent false
13004 4468 Acked Read 9 Done Done Sent false
13005 4469 Acked Flush 0 Done Done Sent false
13006 4470 Acked Write 8 Done Done Sent false
13007 4471 Acked Read 9 Done Done Sent false
13008 4472 Acked Read 8 Done Done Sent false
13009 4473 Acked Read 5 Done Done Sent false
13010 4474 Acked Write 10 Done Done Sent false
13011 4475 Acked Read 5 Done Done Sent false
13012 4476 Acked Write 3 Done Done Sent false
13013 4477 Acked Write 9 Done Done Sent false
13014 4478 Acked Read 5 Done Done Sent false
13015 4479 Acked Read 7 Done Done Sent false
13016 4480 Acked Read 3 Done Done Sent false
13017 4481 Acked Write 7 Done Done Sent false
13018 4482 Acked Read 2 Done Done Sent false
13019 4483 Acked Write 6 Done Done Sent false
13020 4484 Acked Write 10 Done Done Sent false
13021 4485 Acked Read 3 Done Done Sent false
13022 4486 Acked Read 3 Done Done Sent false
13023 4487 Acked Read 9 Done Done Sent false
13024 4488 Acked Write 3 Done Done Sent false
13025 4489 Acked Write 2 Done Done Sent false
13026 4490 Acked Write 7 Done Done Sent false
13027 4491 Acked Read 9 Done Done Sent false
13028 4492 Acked Read 9 Done Done Sent false
13029 4493 Acked Write 5 Done Done Sent false
13030 4494 Acked Read 9 Done Done Sent false
13031 4495 Acked Write 6 Done Done Sent false
13032 4496 Acked Flush 0 Done Done Sent false
13033 4497 Acked Read 9 Done Done Sent false
13034 4498 Acked Read 10 Done Done Sent false
13035 4499 Acked Read 3 Done Done Sent false
13036 4500 Acked Read 9 Done Done Sent false
13037 4501 Acked Write 7 Done Done Sent false
13038 4502 Acked Read 5 Done Done Sent false
13039 4503 Acked Write 6 Done Done Sent false
13040 4504 Acked Flush 0 Done Done Sent false
13041 4505 Acked Write 9 Done Done Sent false
13042 4506 Acked Write 7 Done Done Sent false
13043 4507 Acked Read 6 Done Done Sent false
13044 4508 Acked Write 3 Done Done Sent false
13045 4509 Acked Read 7 Done Done Sent false
13046 4510 Acked Read 4 Done Done Sent false
13047 4511 Acked Flush 0 Done Done Sent false
13048 4512 Acked Read 5 Done Done Sent false
13049 4513 Acked Flush 0 Done Done Sent false
13050 4514 Acked Read 8 Done Done Sent false
13051 4515 Acked Write 5 Done Done Sent false
13052 4516 Acked Write 3 Done Done Sent false
13053 4517 Acked Read 5 Done Done Sent false
13054 4518 Acked Read 2 Done Done Sent false
13055 4519 Acked Read 5 Done Done Sent false
13056 4520 Acked Write 4 Done Done Sent false
13057 4521 Acked Write 4 Done Done Sent false
13058 4522 Acked Read 1 Done Done Sent false
13059 4523 Acked Read 5 Done Done Sent false
13060 4524 Acked Write 9 Done Done Sent false
13061 4525 Acked Read 5 Done Done Sent false
13062 4526 Acked Read 9 Done Done Sent false
13063 4527 Acked Write 5 Done Done Sent false
13064 4528 Acked Write 6 Done Done Sent false
13065 4529 Acked Read 6 Done Done Sent false
13066 4530 Acked Read 6 Done Done Sent false
13067 4531 Acked Write 1 Done Done Sent false
13068 4532 Acked Write 9 Done Done Sent false
13069 4533 Acked Read 9 Done Done Sent false
13070 4534 Acked Read 10 Done Done Sent false
13071 4535 Acked Read 2 Done Done Sent false
13072 4536 Acked Write 4 Done Done Sent false
13073 4537 Acked Read 9 Done Done Sent false
13074 4538 Acked Write 10 Done Done Sent false
13075 4539 Acked Read 5 Done Done Sent false
13076 4540 Acked Read 8 Done Done Sent false
13077 4541 Acked Write 4 Done Done Sent false
13078 4542 Acked Write 7 Done Done Sent false
13079 4543 Acked Write 1 Done Done Sent false
13080 4544 Acked Write 10 Done Done Sent false
13081 4545 Acked Read 9 Done Done Sent false
13082 4546 Acked Read 8 Done Done Sent false
13083 4547 Acked Flush 0 Done Done Sent false
13084 4548 Acked Read 4 Done Done Sent false
13085 4549 Acked Flush 0 Done Done Sent false
13086 4550 Acked Read 10 Done Done Sent false
13087 4551 Acked Flush 0 Done Done Sent false
13088 4552 Acked Write 3 Done Done Sent false
13089 4553 Acked Read 2 Done Done Sent false
13090 4554 Acked Read 6 Done Done Sent false
13091 4555 Acked Write 3 Done Done Sent false
13092 4556 Acked Write 1 Done Done Sent false
13093 4557 Acked Read 10 Done Done Sent false
13094 4558 Acked Read 5 Done Done Sent false
13095 4559 Acked Write 2 Done Done Sent false
13096 4560 Acked Write 10 Done Done Sent false
13097 4561 Acked Read 10 Done Done Sent false
13098 4562 Acked Read 7 Done Done Sent false
13099 4563 Acked Write 9 Done Done Sent false
13100 4564 Acked Write 10 Done Done Sent false
13101 4565 Acked Read 7 Done Done Sent false
13102 4566 Acked Write 2 Done Done Sent false
13103 4567 Acked Write 3 Done Done Sent false
13104 4568 Acked Read 7 Done Done Sent false
13105 4569 Acked Read 3 Done Done Sent false
13106 4570 Acked Write 2 Done Done Sent false
13107 4571 Acked Read 7 Done Done Sent false
13108 4572 Acked Read 6 Done Done Sent false
13109 4573 Acked Read 4 Done Done Sent false
13110 4574 Acked Write 7 Done Done Sent false
13111 4575 Acked Read 2 Done Done Sent false
13112 4576 Acked Read 9 Done Done Sent false
13113 4577 Acked Write 7 Done Done Sent false
13114 4578 Acked Flush 0 Done Done Sent false
13115 4579 Acked Read 1 Done Done Sent false
13116 4580 Acked Write 2 Done Done Sent false
13117 4581 Acked Write 5 Done Done Sent false
13118 4582 Acked Read 1 Done Done Sent false
13119 4583 Acked Write 1 Done Done Sent false
13120 4584 Acked Flush 0 Done Done Sent false
13121 4585 Acked Read 6 Done Done Sent false
13122 4586 Acked Write 5 Done Done Sent false
13123 4587 Acked Write 6 Done Done Sent false
13124 4588 Acked Read 10 Done Done Sent false
13125 4589 Acked Read 8 Done Done Sent false
13126 4590 Acked Read 1 Done Done Sent false
13127 4591 Acked Write 4 Done Done Sent false
13128 4592 Acked Write 3 Done Done Sent false
13129 4593 Acked Write 1 Done Done Sent false
13130 4594 Acked Write 6 Done Done Sent false
13131 4595 Acked Write 5 Done Done Sent false
13132 4596 Acked Read 5 Done Done Sent false
13133 4597 Acked Read 3 Done Done Sent false
13134 4598 Acked Write 7 Done Done Sent false
13135 4599 Acked Write 10 Done Done Sent false
13136 4600 Acked Read 8 Done Done Sent false
13137 4601 Acked Write 1 Done Done Sent false
13138 4602 Acked Write 3 Done Done Sent false
13139 4603 Acked Write 9 Done Done Sent false
13140 4604 Acked Write 1 Done Done Sent false
13141 4605 Acked Read 1 Done Done Sent false
13142 4606 Acked Read 5 Done Done Sent false
13143 4607 Acked Read 7 Done Done Sent false
13144 4608 Acked Write 5 Done Done Sent false
13145 4609 Acked Read 3 Done Done Sent false
13146 4610 Acked Flush 0 Done Done Sent false
13147 4611 Acked Write 9 Done Done Sent false
13148 4612 Acked Read 3 Done Done Sent false
13149 4613 Acked Read 7 Done Done Sent false
13150 4614 Acked Read 8 Done Done Sent false
13151 4615 Acked Write 3 Done Done Sent false
13152 4616 Acked Write 3 Done Done Sent false
13153 4617 Acked Read 10 Done Done Sent false
13154 4618 Acked Write 5 Done Done Sent false
13155 4619 Acked Read 1 Done Done Sent false
13156 4620 Acked Read 1 Done Done Sent false
13157 4621 Acked Read 10 Done Done Sent false
13158 4622 Acked Flush 0 Done Done Sent false
13159 STATES DS:0 DS:1 DS:2 TOTAL
13160 Sent 0 0 302 302
13161 Done 302 302 0 604
13162 Skipped 0 0 0 0
13163 Error 0 0 0 0
13164 Last Flush: 4622 4622 4320
13165 Downstairs last five completed: 4320 4319 4318 4317 4316
13166 Upstairs last five completed: 4622 4621 4620 4619 4618
13167 CLIENT: Up:0 ds:302 act:2
131682024-11-26T22:07:17.096ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
131692024-11-26T22:07:17.096ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:57760) = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
131702024-11-26T22:07:17.097ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
131712024-11-26T22:07:17.097ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
131722024-11-26T22:07:17.097ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
131732024-11-26T22:07:17.097ZINFOcrucible: send last flush ID to this DS: 4320 = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
131742024-11-26T22:07:17.097ZINFOcrucible: Replied this last flush ID: 4320 = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
131752024-11-26T22:07:17.097ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
131762024-11-26T22:07:17.098ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
131772024-11-26T22:07:17.098ZINFOcrucible: [2] client re-new 302 jobs since flush 4320 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
131782024-11-26T22:07:17.098ZINFOcrucible: [2] Replayed 302 jobs since flush: 4320 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
13179 ----------------------------------------------------------------
13180 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
13181 Downstairs last five completed: 4622 4621 4620 4619 4618
13182 Upstairs last five completed: 4622 4621 4620 4619 4618
13183 CLIENT: Up:0 ds:0 act:3
13184 Replay: All jobs finished
13185 Main task has work to do, go find it
13186 got dsc Stop(1)
13187 stop 1
13188 Nov 26 22:07:19.336[8820] Got stop action so:false kr:false
13189 INFO request completed, latency_us: 340, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 793e67f4-8f9c-4afa-9934-05b2061d2363, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
131902024-11-26T22:07:19.337ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
131912024-11-26T22:07:19.337ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
131922024-11-26T22:07:19.337ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
131932024-11-26T22:07:19.337ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
13194 Nov 26 22:07:19.337 INFO request completed, latency_us:{"msg":"[1] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-26T22:07:19.337773879Z","hostname":"w-01JDN7ZQ17ABQG15KHKN3F2D26","pid":1203,"client":"1","":"downstairs","session_id":"86a36c17-3718-484c-87fd-2b8a1dc2ab18"}
13195 151, response_code: 200, uri: /state/cid/1, method: GET, req_id: fc51f35a-5bc6-4956-9525-19be5fa3dad6, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
131962024-11-26T22:07:19.337ZINFOcrucible: client task is exiting = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
13197 [8820] Exited with: ExitStatus(unix_wait_status(9))
13198 [8820][1] reports Exit
13199 Nov 26 22:07:23.340 INFO request completed, latency_us: 205, response_code: 200, uri: /state/cid/1, method: GET, req_id: 3f3198ab-cf57-4a82-b64e-13b1e62c089a, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
13200 001/300 Read block 978 len 12288
13201 002/300 Write block 3568 len 32768 data: 4 3 4 4 4 4 4 3
13202 003/300 Read block 2792 len 24576
13203 004/300 Read block 3459 len 8192
13204 005/300 Write block 1036 len 4096 data: 3
13205 006/300 Read block 526 len 20480
13206 007/300 Read block 1992 len 24576
13207 008/300 Read block 4656 len 40960
13208 009/300 Read block 1776 len 36864
13209 010/300 Read block 4623 len 32768
13210 011/300 Read block 2022 len 32768
13211 012/300 Write block 3390 len 36864 data: 2 3 3 3 4 5 5 5 5
13212 013/300 Write block 1213 len 16384 data: 4 4 4 3
13213 014/300 Write block 3435 len 16384 data: 2 2 2 3
13214 015/300 Read block 2343 len 36864
13215 016/300 Write block 3110 len 20480 data: 4 4 3 2 3
13216 017/300 Write block 4981 len 40960 data: 5 5 5 5 3 3 4 5 7 8
13217 018/300 Write block 341 len 24576 data: 3 3 3 3 4 3
13218 019/300 Read block 4686 len 8192
13219 020/300 Write block 1393 len 20480 data: 2 2 2 3 3
13220 021/300 Write block 1607 len 16384 data: 2 2 3 3
13221 022/300 Flush
13222 023/300 Flush
13223 024/300 Write block 930 len 36864 data: 5 3 5 5 3 2 2 2 2
13224 025/300 Flush
13225 026/300 Read block 4025 len 12288
13226 027/300 Write block 4357 len 4096 data: 5
13227 028/300 Read block 3676 len 40960
13228 029/300 Read block 1245 len 20480
13229 030/300 Write block 3454 len 8192 data: 4 4
13230 031/300 Write block 4404 len 36864 data: 3 4 3 3 3 3 3 3 3
13231 032/300 Write block 3569 len 4096 data: 4
13232 033/300 Read block 551 len 4096
13233 034/300 Write block 2367 len 28672 data: 2 2 2 3 3 4 5
13234 035/300 Read block 170 len 40960
13235 036/300 Read block 4706 len 28672
13236 037/300 Flush
13237 038/300 Read block 1479 len 4096
13238 039/300 Read block 2341 len 28672
13239 040/300 Flush
13240 041/300 Read block 2650 len 36864
13241 042/300 Write block 178 len 4096 data: 2
13242 043/300 Write block 451 len 20480 data: 3 3 3 3 3
13243 044/300 Read block 4496 len 20480
13244 045/300 Read block 3197 len 40960
13245 046/300 Write block 1989 len 8192 data: 4 4
13246 047/300 Read block 899 len 36864
13247 048/300 Read block 506 len 40960
13248 049/300 Write block 4894 len 24576 data: 4 3 4 3 3 3
13249 050/300 Read block 2616 len 36864
13250 051/300 Read block 2871 len 12288
13251 052/300 Write block 1410 len 20480 data: 2 2 2 2 3
13252 053/300 Write block 3596 len 28672 data: 5 5 5 5 4 5 4
13253 054/300 Read block 1238 len 4096
13254 055/300 Read block 4330 len 24576
13255 056/300 Read block 37 len 24576
13256 057/300 Write block 930 len 20480 data: 6 4 6 6 4
13257 058/300 Write block 1552 len 16384 data: 5 6 5 5
13258 059/300 Read block 1806 len 8192
13259 060/300 Read block 2933 len 4096
13260 061/300 Write block 2930 len 32768 data: 3 2 3 3 3 3 3 3
13261 062/300 Read block 4096 len 4096
13262 063/300 Write block 2351 len 16384 data: 5 5 6 5
13263 064/300 Flush
13264 065/300 Read block 2136 len 4096
13265 066/300 Read block 2525 len 24576
13266 067/300 Write block 557 len 8192 data: 2 2
13267 068/300 Write block 4239 len 20480 data: 3 3 3 3 3
13268 069/300 Read block 457 len 36864
13269 070/300 Read block 2905 len 24576
13270 071/300 Read block 1717 len 20480
13271 072/300 Read block 3954 len 40960
13272 073/300 Write block 1106 len 12288 data: 2 3 3
13273 074/300 Read block 3414 len 28672
13274 075/300 Write block 234 len 36864 data: 3 3 3 3 4 4 4 4 4
13275 076/300 Write block 512 len 24576 data: 5 5 4 4 4 5
13276 077/300 Read block 2892 len 12288
13277 078/300 Read block 31 len 8192
13278 079/300 Read block 3434 len 16384
13279 080/300 Write block 1477 len 32768 data: 5 6 6 5 5 5 4 3
13280 081/300 Flush
13281 082/300 Read block 1169 len 28672
13282 083/300 Write block 2838 len 36864 data: 4 4 5 5 5 5 4 4 5
13283 084/300 Write block 2514 len 28672 data: 2 2 5 5 5 5 4
13284 085/300 Read block 1150 len 24576
13285 086/300 Write block 3133 len 40960 data: 2 2 3 3 3 3 4 5 5 6
13286 087/300 Read block 53 len 20480
13287 088/300 Write block 4562 len 24576 data: 3 3 4 3 2 2
13288 089/300 Write block 909 len 20480 data: 5 7 7 6 6
13289 090/300 Read block 4857 len 32768
13290 091/300 Read block 77 len 12288
13291 092/300 Read block 92 len 4096
13292 093/300 Read block 2790 len 32768
13293 094/300 Read block 3882 len 12288
13294 095/300 Read block 1719 len 4096
13295 096/300 Flush
13296 097/300 Read block 2718 len 16384
13297 098/300 Write block 1759 len 8192 data: 3 4
13298 099/300 Write block 1723 len 32768 data: 5 5 4 4 4 2 3 3
13299 100/300 Read block 3230 len 8192
13300 101/300 Read block 4847 len 40960
13301 102/300 Write block 1058 len 12288 data: 4 4 3
13302 103/300 Read block 4949 len 20480
13303 104/300 Read block 3819 len 12288
13304 105/300 Write block 349 len 32768 data: 6 5 6 6 6 6 6 6
13305 106/300 Write block 4932 len 28672 data: 3 3 3 2 2 3 3
13306 107/300 Write block 2246 len 32768 data: 4 5 5 4 4 4 4 5
13307 108/300 Write block 3917 len 24576 data: 4 4 3 3 5 4
13308 109/300 Write block 4246 len 16384 data: 3 3 4 4
13309 110/300 Read block 206 len 12288
13310 111/300 Read block 2792 len 4096
13311 112/300 Write block 4342 len 16384 data: 4 4 4 3
13312 113/300 Read block 761 len 8192
13313 114/300 Read block 624 len 32768
13314 115/300 Read block 1418 len 16384
13315 116/300 Write block 1091 len 36864 data: 3 4 4 3 4 4 4 5 4
13316 117/300 Write block 2386 len 8192 data: 3 4
13317 118/300 Read block 1776 len 28672
13318 119/300 Read block 3363 len 16384
13319 120/300 Read block 3700 len 4096
13320 121/300 Write block 631 len 40960 data: 3 3 2 2 2 3 4 4 5 5
13321 122/300 Write block 3847 len 8192 data: 4 3
13322 123/300 Read block 2781 len 4096
13323 124/300 Read block 510 len 20480
13324 125/300 Write block 4969 len 4096 data: 6
13325 126/300 Flush
13326 127/300 Write block 3279 len 32768 data: 2 3 3 3 3 3 4 4
13327 128/300 Read block 1760 len 12288
13328 129/300 Read block 1768 len 4096
13329 130/300 Read block 1696 len 36864
13330 131/300 Read block 2199 len 36864
13331 132/300 Write block 2075 len 20480 data: 2 2 3 3 3
13332 133/300 Write block 2747 len 16384 data: 5 4 3 2
13333 134/300 Write block 3081 len 12288 data: 5 6 6
13334 135/300 Read block 607 len 20480
13335 136/300 Read block 687 len 36864
13336 137/300 Flush
13337 138/300 Read block 2511 len 28672
13338 139/300 Read block 1682 len 4096
13339 140/300 Flush
13340 141/300 Read block 4382 len 32768
13341 142/300 Read block 4130 len 32768
13342 143/300 Read block 3200 len 32768
13343 144/300 Read block 3287 len 40960
13344 145/300 Read block 3275 len 16384
13345 146/300 Write block 4616 len 20480 data: 4 3 4 5 4
13346 147/300 Write block 2087 len 20480 data: 5 4 4 4 3
13347 148/300 Read block 2620 len 4096
13348 149/300 Read block 3956 len 24576
13349 150/300 Read block 734 len 40960
13350 151/300 Write block 1158 len 4096 data: 3
13351 152/300 Read block 1980 len 16384
13352 153/300 Read block 3737 len 40960
13353 154/300 Write block 4194 len 40960 data: 4 5 5 6 6 5 4 3 3 3
13354 155/300 Write block 1993 len 36864 data: 3 3 2 2 2 3 4 3 3
13355 156/300 Write block 90 len 32768 data: 3 3 3 4 4 4 6 7
13356 157/300 Read block 289 len 16384
13357 158/300 Read block 4834 len 8192
13358 159/300 Read block 4597 len 40960
13359 160/300 Read block 974 len 12288
13360 161/300 Read block 4108 len 40960
13361 162/300 Read block 756 len 8192
13362 163/300 Flush
13363 164/300 Read block 3779 len 8192
13364 165/300 Read block 369 len 40960
13365 166/300 Read block 4148 len 12288
13366 167/300 Write block 3905 len 16384 data: 3 3 4 3
13367 168/300 Write block 3590 len 40960 data: 3 3 4 6 6 5 6 6 6 6
13368 169/300 Read block 4821 len 8192
13369 170/300 Read block 2238 len 16384
13370 171/300 Read block 3380 len 40960
13371 172/300 Read block 3011 len 8192
13372 173/300 Read block 3611 len 24576
13373 174/300 Read block 3611 len 40960
13374 175/300 Read block 1541 len 24576
13375 176/300 Write block 2696 len 8192 data: 2 2
13376 177/300 Write block 3134 len 20480 data: 3 4 4 4 4
13377 178/300 Read block 2255 len 16384
13378 179/300 Write block 4341 len 32768 data: 4 5 5 5 4 3 4 5
13379 180/300 Read block 559 len 16384
13380 181/300 Write block 2201 len 8192 data: 3 3
13381 182/300 Read block 4145 len 28672
13382 183/300 Write block 2485 len 16384 data: 7 8 7 7
13383 184/300 Read block 4013 len 16384
13384 185/300 Flush
13385 186/300 Read block 3822 len 16384
13386 187/300 Read block 4255 len 32768
13387 188/300 Read block 1635 len 40960
13388 189/300 Write block 4843 len 36864 data: 3 3 3 3 3 3 2 2 2
13389 190/300 Read block 454 len 32768
13390 191/300 Read block 4332 len 4096
13391 192/300 Read block 3033 len 40960
13392 193/300 Write block 4354 len 16384 data: 2 2 4 6
13393 194/300 Write block 684 len 12288 data: 2 2 2
13394 195/300 Read block 2646 len 32768
13395 196/300 Flush
13396 197/300 Read block 2395 len 36864
13397 198/300 Write block 3636 len 8192 data: 2 2
13398 199/300 Read block 2721 len 40960
13399 200/300 Write block 444 len 36864 data: 4 4 4 3 5 4 4 4 4
13400 201/300 Read block 2671 len 16384
13401 202/300 Read block 753 len 40960
13402 203/300 Read block 4979 len 24576
13403 204/300 Read block 704 len 40960
13404 205/300 Write block 4941 len 40960 data: 2 2 2 2 2 3 3 3 2 2
13405 206/300 Read block 1773 len 4096
13406 207/300 Read block 1515 len 4096
13407 208/300 Read block 2456 len 28672
13408 209/300 Read block 3677 len 4096
13409 210/300 Read block 2023 len 12288
13410 211/300 Write block 3666 len 32768 data: 6 6 5 6 7 6 7 8
13411 212/300 Write block 2264 len 8192 data: 2 3
13412 213/300 Read block 3637 len 8192
13413 214/300 Write block 136 len 40960 data: 2 2 3 4 5 5 3 3 4 2
13414 215/300 Read block 1 len 40960
13415 216/300 Flush
13416 217/300 Write block 3668 len 20480 data: 6 7 8 7 8
13417 218/300 Read block 587 len 24576
13418 219/300 Read block 1546 len 20480
13419 220/300 Write block 1524 len 12288 data: 5 6 6
13420 221/300 Write block 4311 len 8192 data: 2 3
13421 222/300 Write block 402 len 4096 data: 4
13422 223/300 Read block 3290 len 4096
13423 224/300 Read block 2637 len 8192
13424 225/300 Flush
13425 226/300 Write block 86 len 12288 data: 3 3 4
13426 227/300 Read block 4550 len 24576
13427 228/300 Read block 762 len 24576
13428 229/300 Flush
13429 230/300 Read block 1645 len 16384
13430 231/300 Read block 995 len 16384
13431 232/300 Write block 4018 len 4096 data: 4
13432 233/300 Write block 2106 len 36864 data: 3 3 3 3 3 3 2 2 2
13433 234/300 Read block 3559 len 8192
13434 235/300 Read block 4979 len 28672
13435 236/300 Read block 371 len 28672
13436 237/300 Write block 3740 len 36864 data: 2 2 3 3 4 5 4 3 3
13437 238/300 Flush
13438 239/300 Read block 2113 len 28672
13439 240/300 Write block 476 len 16384 data: 3 2 2 3
13440 241/300 Write block 812 len 4096 data: 5
13441 242/300 Write block 4680 len 28672 data: 7 6 4 3 3 2 3
13442 243/300 Read block 1992 len 4096
13443 244/300 Read block 1087 len 40960
13444 245/300 Write block 504 len 8192 data: 3 3
13445 246/300 Read block 330 len 36864
13446 247/300 Write block 2102 len 24576 data: 5 5 5 4 4 4
13447 248/300 Read block 3547 len 24576
13448 249/300 Read block 3294 len 8192
13449 250/300 Flush
13450 251/300 Write block 400 len 20480 data: 3 3 5 5 4
13451 252/300 Read block 2843 len 28672
13452 253/300 Read block 4264 len 32768
13453 254/300 Read block 1910 len 12288
13454 255/300 Read block 347 len 40960
13455 256/300 Read block 2842 len 28672
13456 257/300 Read block 69 len 40960
13457 258/300 Flush
13458 259/300 Read block 565 len 36864
13459 260/300 Write block 4289 len 4096 data: 2
13460 261/300 Read block 1693 len 4096
13461 262/300 Read block 1438 len 36864
13462 263/300 Write block 1335 len 4096 data: 2
13463 264/300 Read block 2327 len 12288
13464 265/300 Write block 1855 len 16384 data: 5 5 5 5
13465 266/300 Read block 1684 len 16384
13466 267/300 Read block 2113 len 12288
13467 268/300 Read block 4089 len 4096
13468 269/300 Read block 1210 len 28672
13469 270/300 Read block 1945 len 12288
13470 271/300 Read block 1209 len 24576
13471 272/300 Write block 287 len 24576 data: 4 5 7 6 6 6
13472 273/300 Read block 779 len 8192
13473 274/300 Write block 1500 len 16384 data: 5 4 4 4
13474 275/300 Read block 1425 len 8192
13475 276/300 Read block 4204 len 16384
13476 277/300 Write block 169 len 40960 data: 3 3 3 3 3 2 2 2 2 3
13477 278/300 Write block 452 len 8192 data: 5 4
13478 279/300 Write block 4726 len 24576 data: 3 3 2 3 3 4
13479 280/300 Write block 1986 len 4096 data: 5
13480 281/300 Write block 4011 len 36864 data: 5 3 3 4 4 4 4 5 3
13481 282/300 Read block 2727 len 24576
13482 283/300 Write block 2381 len 16384 data: 2 2 2 2
13483 284/300 Read block 4566 len 12288
13484 285/300 Write block 397 len 12288 data: 3 3 3
13485 286/300 Flush
13486 287/300 Read block 2176 len 20480
13487 288/300 Flush
13488 289/300 Read block 2710 len 12288
13489 290/300 Write block 1547 len 4096 data: 2
13490 291/300 Read block 3420 len 12288
13491 292/300 Write block 39 len 40960 data: 4 5 4 4 3 4 4 5 5 5
13492 293/300 Read block 3303 len 36864
13493 294/300 Read block 1934 len 4096
13494 295/300 Flush
13495 296/300 Flush
13496 297/300 Write block 4635 len 40960 data: 4 4 4 4 4 3 3 2 2 3
13497 298/300 Write block 4200 len 32768 data: 5 4 4 4 3 2 3 4
13498 299/300 Write block 4431 len 8192 data: 2 2
13499 300/300 Write block 3530 len 32768 data: 3 4 4 4 4 3 3 2
13500 301/300 Write block 2386 len 40960 data: 4 5 4 5 6 6 6 5 5 6
13501 Main task has work to do, go find it
13502 got dsc Start(1)
13503 start 1
13504 Nov 26 22:07:23.580 INFO request completed, latency_us: 185, response_code: 204, uri: /start/cid/1, method: POST, req_id: 54f2a280-e193-445b-bcfc-0af7b95efbc4, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
13505 Replay: started 1, returned:Ok(())
13506 ----------------------------------------------------------------
13507 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
13508 States: Active Offline Active
13509 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
13510 4623 Acked Read 3 Done Sent Done false
13511 4624 Acked Write 8 Done Sent Done false
13512 4625 Acked Read 6 Done Sent Done false
13513 4626 Acked Read 2 Done Sent Done false
13514 4627 Acked Write 1 Done Sent Done false
13515 4628 Acked Read 5 Done Sent Done false
13516 4629 Acked Read 6 Done Sent Done false
13517 4630 Acked Read 10 Done Sent Done false
13518 4631 Acked Read 9 Done Sent Done false
13519 4632 Acked Read 8 Done Sent Done false
13520 4633 Acked Read 8 Done Sent Done false
13521 4634 Acked Write 9 Done Sent Done false
13522 4635 Acked Write 4 Done Sent Done false
13523 4636 Acked Write 4 Done Sent Done false
13524 4637 Acked Read 9 Done Sent Done false
13525 4638 Acked Write 5 Done Sent Done false
13526 4639 Acked Write 10 Done Sent Done false
13527 4640 Acked Write 6 Done Sent Done false
13528 4641 Acked Read 2 Done Sent Done false
13529 4642 Acked Write 5 Done Sent Done false
13530 4643 Acked Write 4 Done Sent Done false
13531 4644 Acked Flush 0 Done Sent Done false
13532 4645 Acked Flush 0 Done Sent Done false
13533 4646 Acked Write 9 Done Sent Done false
13534 4647 Acked Flush 0 Done Sent Done false
13535 4648 Acked Read 3 Done Sent Done false
13536 4649 Acked Write 1 Done Sent Done false
13537 4650 Acked Read 10 Done Sent Done false
13538 4651 Acked Read 5 Done Sent Done false
13539 4652 Acked Write 2 Done Sent Done false
13540 4653 Acked Write 9 Done Sent Done false
13541 4654 Acked Write 1 Done Sent Done false
13542 4655 Acked Read 1 Done Sent Done false
13543 4656 Acked Write 7 Done Sent Done false
13544 4657 Acked Read 10 Done Sent Done false
13545 4658 Acked Read 7 Done Sent Done false
13546 4659 Acked Flush 0 Done Sent Done false
13547 4660 Acked Read 1 Done Sent Done false
13548 4661 Acked Read 7 Done Sent Done false
13549 4662 Acked Flush 0 Done Sent Done false
13550 4663 Acked Read 9 Done Sent Done false
13551 4664 Acked Write 1 Done Sent Done false
13552 4665 Acked Write 5 Done Sent Done false
13553 4666 Acked Read 5 Done Sent Done false
13554 4667 Acked Read 10 Done Sent Done false
13555 4668 Acked Write 2 Done Sent Done false
13556 4669 Acked Read 9 Done Sent Done false
13557 4670 Acked Read 10 Done Sent Done false
13558 4671 Acked Write 6 Done Sent Done false
13559 4672 Acked Read 9 Done Sent Done false
13560 4673 Acked Read 3 Done Sent Done false
13561 4674 Acked Write 5 Done Sent Done false
13562 4675 Acked Write 7 Done Sent Done false
13563 4676 Acked Read 1 Done Sent Done false
13564 4677 Acked Read 6 Done Sent Done false
13565 4678 Acked Read 6 Done Sent Done false
13566 4679 Acked Write 5 Done Sent Done false
13567 4680 Acked Write 4 Done Sent Done false
13568 4681 Acked Read 2 Done Sent Done false
13569 4682 Acked Read 1 Done Sent Done false
13570 4683 Acked Write 8 Done Sent Done false
13571 4684 Acked Read 1 Done Sent Done false
13572 4685 Acked Write 4 Done Sent Done false
13573 4686 Acked Flush 0 Done Sent Done false
13574 4687 Acked Read 1 Done Sent Done false
13575 4688 Acked Read 6 Done Sent Done false
13576 4689 Acked Write 2 Done Sent Done false
13577 4690 Acked Write 5 Done Sent Done false
13578 4691 Acked Read 9 Done Sent Done false
13579 4692 Acked Read 6 Done Sent Done false
13580 4693 Acked Read 5 Done Sent Done false
13581 4694 Acked Read 10 Done Sent Done false
13582 4695 Acked Write 3 Done Sent Done false
13583 4696 Acked Read 7 Done Sent Done false
13584 4697 Acked Write 9 Done Sent Done false
13585 4698 Acked Write 6 Done Sent Done false
13586 4699 Acked Read 3 Done Sent Done false
13587 4700 Acked Read 2 Done Sent Done false
13588 4701 Acked Read 4 Done Sent Done false
13589 4702 Acked Write 8 Done Sent Done false
13590 4703 Acked Flush 0 Done Sent Done false
13591 4704 Acked Read 7 Done Sent Done false
13592 4705 Acked Write 9 Done Sent Done false
13593 4706 Acked Write 7 Done Sent Done false
13594 4707 Acked Read 6 Done Sent Done false
13595 4708 Acked Write 10 Done Sent Done false
13596 4709 Acked Read 5 Done Sent Done false
13597 4710 Acked Write 6 Done Sent Done false
13598 4711 Acked Write 5 Done Sent Done false
13599 4712 Acked Read 8 Done Sent Done false
13600 4713 Acked Read 3 Done Sent Done false
13601 4714 Acked Read 1 Done Sent Done false
13602 4715 Acked Read 8 Done Sent Done false
13603 4716 Acked Read 3 Done Sent Done false
13604 4717 Acked Read 1 Done Sent Done false
13605 4718 Acked Flush 0 Done Sent Done false
13606 4719 Acked Read 4 Done Sent Done false
13607 4720 Acked Write 2 Done Sent Done false
13608 4721 Acked Write 8 Done Sent Done false
13609 4722 Acked Read 2 Done Sent Done false
13610 4723 Acked Read 10 Done Sent Done false
13611 4724 Acked Write 3 Done Sent Done false
13612 4725 Acked Read 5 Done Sent Done false
13613 4726 Acked Read 3 Done Sent Done false
13614 4727 Acked Write 8 Done Sent Done false
13615 4728 Acked Write 7 Done Sent Done false
13616 4729 Acked Write 8 Done Sent Done false
13617 4730 Acked Write 6 Done Sent Done false
13618 4731 Acked Write 4 Done Sent Done false
13619 4732 Acked Read 3 Done Sent Done false
13620 4733 Acked Read 1 Done Sent Done false
13621 4734 Acked Write 4 Done Sent Done false
13622 4735 Acked Read 2 Done Sent Done false
13623 4736 Acked Read 8 Done Sent Done false
13624 4737 Acked Read 4 Done Sent Done false
13625 4738 Acked Write 9 Done Sent Done false
13626 4739 Acked Write 2 Done Sent Done false
13627 4740 Acked Read 7 Done Sent Done false
13628 4741 Acked Read 4 Done Sent Done false
13629 4742 Acked Read 1 Done Sent Done false
13630 4743 Acked Write 10 Done Sent Done false
13631 4744 Acked Write 2 Done Sent Done false
13632 4745 Acked Read 1 Done Sent Done false
13633 4746 Acked Read 5 Done Sent Done false
13634 4747 Acked Write 1 Done Sent Done false
13635 4748 Acked Flush 0 Done Sent Done false
13636 4749 Acked Write 8 Done Sent Done false
13637 4750 Acked Read 3 Done Sent Done false
13638 4751 Acked Read 1 Done Sent Done false
13639 4752 Acked Read 9 Done Sent Done false
13640 4753 Acked Read 9 Done Sent Done false
13641 4754 Acked Write 5 Done Sent Done false
13642 4755 Acked Write 4 Done Sent Done false
13643 4756 Acked Write 3 Done Sent Done false
13644 4757 Acked Read 5 Done Sent Done false
13645 4758 Acked Read 9 Done Sent Done false
13646 4759 Acked Flush 0 Done Sent Done false
13647 4760 Acked Read 7 Done Sent Done false
13648 4761 Acked Read 1 Done Sent Done false
13649 4762 Acked Flush 0 Done Sent Done false
13650 4763 Acked Read 8 Done Sent Done false
13651 4764 Acked Read 8 Done Sent Done false
13652 4765 Acked Read 8 Done Sent Done false
13653 4766 Acked Read 10 Done Sent Done false
13654 4767 Acked Read 4 Done Sent Done false
13655 4768 Acked Write 5 Done Sent Done false
13656 4769 Acked Write 5 Done Sent Done false
13657 4770 Acked Read 1 Done Sent Done false
13658 4771 Acked Read 6 Done Sent Done false
13659 4772 Acked Read 10 Done Sent Done false
13660 4773 Acked Write 1 Done Sent Done false
13661 4774 Acked Read 4 Done Sent Done false
13662 4775 Acked Read 10 Done Sent Done false
13663 4776 Acked Write 10 Done Sent Done false
13664 4777 Acked Write 9 Done Sent Done false
13665 4778 Acked Write 8 Done Sent Done false
13666 4779 Acked Read 4 Done Sent Done false
13667 4780 Acked Read 2 Done Sent Done false
13668 4781 Acked Read 10 Done Sent Done false
13669 4782 Acked Read 3 Done Sent Done false
13670 4783 Acked Read 10 Done Sent Done false
13671 4784 Acked Read 2 Done Sent Done false
13672 4785 Acked Flush 0 Done Sent Done false
13673 4786 Acked Read 2 Done Sent Done false
13674 4787 Acked Read 10 Done Sent Done false
13675 4788 Acked Read 3 Done Sent Done false
13676 4789 Acked Write 4 Done Sent Done false
13677 4790 Acked Write 10 Done Sent Done false
13678 4791 Acked Read 2 Done Sent Done false
13679 4792 Acked Read 4 Done Sent Done false
13680 4793 Acked Read 10 Done Sent Done false
13681 4794 Acked Read 2 Done Sent Done false
13682 4795 Acked Read 6 Done Sent Done false
13683 4796 Acked Read 10 Done Sent Done false
13684 4797 Acked Read 6 Done Sent Done false
13685 4798 Acked Write 2 Done Sent Done false
13686 4799 Acked Write 5 Done Sent Done false
13687 4800 Acked Read 4 Done Sent Done false
13688 4801 Acked Write 8 Done Sent Done false
13689 4802 Acked Read 4 Done Sent Done false
13690 4803 Acked Write 2 Done Sent Done false
13691 4804 Acked Read 7 Done Sent Done false
13692 4805 Acked Write 4 Done Sent Done false
13693 4806 Acked Read 4 Done Sent Done false
13694 4807 Acked Flush 0 Done Sent Done false
13695 4808 Acked Read 4 Done Sent Done false
13696 4809 Acked Read 8 Done Sent Done false
13697 4810 Acked Read 10 Done Sent Done false
13698 4811 Acked Write 9 Done Sent Done false
13699 4812 Acked Read 8 Done Sent Done false
13700 4813 Acked Read 1 Done Sent Done false
13701 4814 Acked Read 10 Done Sent Done false
13702 4815 Acked Write 4 Done Sent Done false
13703 4816 Acked Write 3 Done Sent Done false
13704 4817 Acked Read 8 Done Sent Done false
13705 4818 Acked Flush 0 Done Sent Done false
13706 4819 Acked Read 9 Done Sent Done false
13707 4820 Acked Write 2 Done Sent Done false
13708 4821 Acked Read 10 Done Sent Done false
13709 4822 Acked Write 9 Done Sent Done false
13710 4823 Acked Read 4 Done Sent Done false
13711 4824 Acked Read 10 Done Sent Done false
13712 4825 Acked Read 6 Done Sent Done false
13713 4826 Acked Read 10 Done Sent Done false
13714 4827 Acked Write 10 Done Sent Done false
13715 4828 Acked Read 1 Done Sent Done false
13716 4829 Acked Read 1 Done Sent Done false
13717 4830 Acked Read 7 Done Sent Done false
13718 4831 Acked Read 1 Done Sent Done false
13719 4832 Acked Read 3 Done Sent Done false
13720 4833 Acked Write 8 Done Sent Done false
13721 4834 Acked Write 2 Done Sent Done false
13722 4835 Acked Read 2 Done Sent Done false
13723 4836 Acked Write 10 Done Sent Done false
13724 4837 Acked Read 10 Done Sent Done false
13725 4838 Acked Flush 0 Done Sent Done false
13726 4839 Acked Write 5 Done Sent Done false
13727 4840 Acked Read 6 Done Sent Done false
13728 4841 Acked Read 5 Done Sent Done false
13729 4842 Acked Write 3 Done Sent Done false
13730 4843 Acked Write 2 Done Sent Done false
13731 4844 Acked Write 1 Done Sent Done false
13732 4845 Acked Read 1 Done Sent Done false
13733 4846 Acked Read 2 Done Sent Done false
13734 4847 Acked Flush 0 Done Sent Done false
13735 4848 Acked Write 3 Done Sent Done false
13736 4849 Acked Read 6 Done Sent Done false
13737 4850 Acked Read 6 Done Sent Done false
13738 4851 Acked Flush 0 Done Sent Done false
13739 4852 Acked Read 4 Done Sent Done false
13740 4853 Acked Read 4 Done Sent Done false
13741 4854 Acked Write 1 Done Sent Done false
13742 4855 Acked Write 9 Done Sent Done false
13743 4856 Acked Read 2 Done Sent Done false
13744 4857 Acked Read 7 Done Sent Done false
13745 4858 Acked Read 7 Done Sent Done false
13746 4859 Acked Write 9 Done Sent Done false
13747 4860 Acked Flush 0 Done Sent Done false
13748 4861 Acked Read 7 Done Sent Done false
13749 4862 Acked Write 4 Done Sent Done false
13750 4863 Acked Write 1 Done Sent Done false
13751 4864 Acked Write 7 Done Sent Done false
13752 4865 Acked Read 1 Done Sent Done false
13753 4866 Acked Read 10 Done Sent Done false
13754 4867 Acked Write 2 Done Sent Done false
13755 4868 Acked Read 9 Done Sent Done false
13756 4869 Acked Write 6 Done Sent Done false
13757 4870 Acked Read 6 Done Sent Done false
13758 4871 Acked Read 2 Done Sent Done false
13759 4872 Acked Flush 0 Done Sent Done false
13760 4873 Acked Write 5 Done Sent Done false
13761 4874 Acked Read 7 Done Sent Done false
13762 4875 Acked Read 8 Done Sent Done false
13763 4876 Acked Read 3 Done Sent Done false
13764 4877 Acked Read 10 Done Sent Done false
13765 4878 Acked Read 7 Done Sent Done false
13766 4879 Acked Read 10 Done Sent Done false
13767 4880 Acked Flush 0 Done Sent Done false
13768 4881 Acked Read 9 Done Sent Done false
13769 4882 Acked Write 1 Done Sent Done false
13770 4883 Acked Read 1 Done Sent Done false
13771 4884 Acked Read 9 Done Sent Done false
13772 4885 Acked Write 1 Done Sent Done false
13773 4886 Acked Read 3 Done Sent Done false
13774 4887 Acked Write 4 Done Sent Done false
13775 4888 Acked Read 4 Done Sent Done false
13776 4889 Acked Read 3 Done Sent Done false
13777 4890 Acked Read 1 Done Sent Done false
13778 4891 Acked Read 7 Done Sent Done false
13779 4892 Acked Read 3 Done Sent Done false
13780 4893 Acked Read 6 Done Sent Done false
13781 4894 Acked Write 6 Done Sent Done false
13782 4895 Acked Read 2 Done Sent Done false
13783 4896 Acked Write 4 Done Sent Done false
13784 4897 Acked Read 2 Done Sent Done false
13785 4898 Acked Read 4 Done Sent Done false
13786 4899 Acked Write 10 Done Sent Done false
13787 4900 Acked Write 2 Done Sent Done false
13788 4901 Acked Write 6 Done Sent Done false
13789 4902 Acked Write 1 Done Sent Done false
13790 4903 Acked Write 9 Done Sent Done false
13791 4904 Acked Read 6 Done Sent Done false
13792 4905 Acked Write 4 Done Sent Done false
13793 4906 Acked Read 3 Done Sent Done false
13794 4907 Acked Write 3 Done Sent Done false
13795 4908 Acked Flush 0 Done Sent Done false
13796 4909 Acked Read 5 Done Sent Done false
13797 4910 Acked Flush 0 Done Sent Done false
13798 4911 Acked Read 3 Done Sent Done false
13799 4912 Acked Write 1 Done Sent Done false
13800 4913 Acked Read 3 Done Sent Done false
13801 4914 Acked Write 10 Done Sent Done false
13802 4915 Acked Read 9 Done Sent Done false
13803 4916 Acked Read 1 Done Sent Done false
13804 4917 Acked Flush 0 Done Sent Done false
13805 4918 Acked Flush 0 Done Sent Done false
13806 4919 Acked Write 10 Done Sent Done false
13807 4920 Acked Write 8 Done Sent Done false
13808 4921 Acked Write 2 Done Sent Done false
13809 4922 Acked Write 8 Done Sent Done false
13810 4923 Acked Write 10 Sent Sent Sent false
13811 STATES DS:0 DS:1 DS:2 TOTAL
13812 Sent 1 301 1 303
13813 Done 300 0 300 600
13814 Skipped 0 0 0 0
13815 Error 0 0 0 0
13816 Last Flush: 4918 4622 4918
13817 Downstairs last five completed: 4622 4621 4620 4619 4618
13818 Upstairs last five completed: 4923 4922 4921 4920 4919
13819 CLIENT: Up:0 ds:301 act:2
13820 [8820] got start action
13821 [8820] I am going to restart
13822 Starting downstairs at port 8820
13823 Make output file at "/tmp/dsc/downstairs-8820.txt"
13824 [8820][1] reports Starting
13825 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1216)
13826 [8820][1] reports Running
13827 ----------------------------------------------------------------
13828 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
13829 States: Active Offline Active
13830 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
13831 4623 Acked Read 3 Done Sent Done false
13832 4624 Acked Write 8 Done Sent Done false
13833 4625 Acked Read 6 Done Sent Done false
13834 4626 Acked Read 2 Done Sent Done false
13835 4627 Acked Write 1 Done Sent Done false
13836 4628 Acked Read 5 Done Sent Done false
13837 4629 Acked Read 6 Done Sent Done false
13838 4630 Acked Read 10 Done Sent Done false
13839 4631 Acked Read 9 Done Sent Done false
13840 4632 Acked Read 8 Done Sent Done false
13841 4633 Acked Read 8 Done Sent Done false
13842 4634 Acked Write 9 Done Sent Done false
13843 4635 Acked Write 4 Done Sent Done false
13844 4636 Acked Write 4 Done Sent Done false
13845 4637 Acked Read 9 Done Sent Done false
13846 4638 Acked Write 5 Done Sent Done false
13847 4639 Acked Write 10 Done Sent Done false
13848 4640 Acked Write 6 Done Sent Done false
13849 4641 Acked Read 2 Done Sent Done false
13850 4642 Acked Write 5 Done Sent Done false
13851 4643 Acked Write 4 Done Sent Done false
13852 4644 Acked Flush 0 Done Sent Done false
13853 4645 Acked Flush 0 Done Sent Done false
13854 4646 Acked Write 9 Done Sent Done false
13855 4647 Acked Flush 0 Done Sent Done false
13856 4648 Acked Read 3 Done Sent Done false
13857 4649 Acked Write 1 Done Sent Done false
13858 4650 Acked Read 10 Done Sent Done false
13859 4651 Acked Read 5 Done Sent Done false
13860 4652 Acked Write 2 Done Sent Done false
13861 4653 Acked Write 9 Done Sent Done false
13862 4654 Acked Write 1 Done Sent Done false
13863 4655 Acked Read 1 Done Sent Done false
13864 4656 Acked Write 7 Done Sent Done false
13865 4657 Acked Read 10 Done Sent Done false
13866 4658 Acked Read 7 Done Sent Done false
13867 4659 Acked Flush 0 Done Sent Done false
13868 4660 Acked Read 1 Done Sent Done false
13869 4661 Acked Read 7 Done Sent Done false
13870 4662 Acked Flush 0 Done Sent Done false
13871 4663 Acked Read 9 Done Sent Done false
13872 4664 Acked Write 1 Done Sent Done false
13873 4665 Acked Write 5 Done Sent Done false
13874 4666 Acked Read 5 Done Sent Done false
13875 4667 Acked Read 10 Done Sent Done false
13876 4668 Acked Write 2 Done Sent Done false
13877 4669 Acked Read 9 Done Sent Done false
13878 4670 Acked Read 10 Done Sent Done false
13879 4671 Acked Write 6 Done Sent Done false
13880 4672 Acked Read 9 Done Sent Done false
13881 4673 Acked Read 3 Done Sent Done false
13882 4674 Acked Write 5 Done Sent Done false
13883 4675 Acked Write 7 Done Sent Done false
13884 4676 Acked Read 1 Done Sent Done false
13885 4677 Acked Read 6 Done Sent Done false
13886 4678 Acked Read 6 Done Sent Done false
13887 4679 Acked Write 5 Done Sent Done false
13888 4680 Acked Write 4 Done Sent Done false
13889 4681 Acked Read 2 Done Sent Done false
13890 4682 Acked Read 1 Done Sent Done false
13891 4683 Acked Write 8 Done Sent Done false
13892 4684 Acked Read 1 Done Sent Done false
13893 4685 Acked Write 4 Done Sent Done false
13894 4686 Acked Flush 0 Done Sent Done false
13895 4687 Acked Read 1 Done Sent Done false
13896 4688 Acked Read 6 Done Sent Done false
13897 4689 Acked Write 2 Done Sent Done false
13898 4690 Acked Write 5 Done Sent Done false
13899 4691 Acked Read 9 Done Sent Done false
13900 4692 Acked Read 6 Done Sent Done false
13901 4693 Acked Read 5 Done Sent Done false
13902 4694 Acked Read 10 Done Sent Done false
13903 4695 Acked Write 3 Done Sent Done false
13904 4696 Acked Read 7 Done Sent Done false
13905 4697 Acked Write 9 Done Sent Done false
13906 4698 Acked Write 6 Done Sent Done false
13907 4699 Acked Read 3 Done Sent Done false
13908 4700 Acked Read 2 Done Sent Done false
13909 4701 Acked Read 4 Done Sent Done false
13910 4702 Acked Write 8 Done Sent Done false
13911 4703 Acked Flush 0 Done Sent Done false
13912 4704 Acked Read 7 Done Sent Done false
13913 4705 Acked Write 9 Done Sent Done false
13914 4706 Acked Write 7 Done Sent Done false
13915 4707 Acked Read 6 Done Sent Done false
13916 4708 Acked Write 10 Done Sent Done false
13917 4709 Acked Read 5 Done Sent Done false
13918 4710 Acked Write 6 Done Sent Done false
13919 4711 Acked Write 5 Done Sent Done false
13920 4712 Acked Read 8 Done Sent Done false
13921 4713 Acked Read 3 Done Sent Done false
13922 4714 Acked Read 1 Done Sent Done false
13923 4715 Acked Read 8 Done Sent Done false
13924 4716 Acked Read 3 Done Sent Done false
13925 4717 Acked Read 1 Done Sent Done false
13926 4718 Acked Flush 0 Done Sent Done false
13927 4719 Acked Read 4 Done Sent Done false
13928 4720 Acked Write 2 Done Sent Done false
13929 4721 Acked Write 8 Done Sent Done false
13930 4722 Acked Read 2 Done Sent Done false
13931 4723 Acked Read 10 Done Sent Done false
13932 4724 Acked Write 3 Done Sent Done false
13933 4725 Acked Read 5 Done Sent Done false
13934 4726 Acked Read 3 Done Sent Done false
13935 4727 Acked Write 8 Done Sent Done false
13936 4728 Acked Write 7 Done Sent Done false
13937 4729 Acked Write 8 Done Sent Done false
13938 4730 Acked Write 6 Done Sent Done false
13939 4731 Acked Write 4 Done Sent Done false
13940 4732 Acked Read 3 Done Sent Done false
13941 4733 Acked Read 1 Done Sent Done false
13942 4734 Acked Write 4 Done Sent Done false
13943 4735 Acked Read 2 Done Sent Done false
13944 4736 Acked Read 8 Done Sent Done false
13945 4737 Acked Read 4 Done Sent Done false
13946 4738 Acked Write 9 Done Sent Done false
13947 4739 Acked Write 2 Done Sent Done false
13948 4740 Acked Read 7 Done Sent Done false
13949 4741 Acked Read 4 Done Sent Done false
13950 4742 Acked Read 1 Done Sent Done false
13951 4743 Acked Write 10 Done Sent Done false
13952 4744 Acked Write 2 Done Sent Done false
13953 4745 Acked Read 1 Done Sent Done false
13954 4746 Acked Read 5 Done Sent Done false
13955 4747 Acked Write 1 Done Sent Done false
13956 4748 Acked Flush 0 Done Sent Done false
13957 4749 Acked Write 8 Done Sent Done false
13958 4750 Acked Read 3 Done Sent Done false
13959 4751 Acked Read 1 Done Sent Done false
13960 4752 Acked Read 9 Done Sent Done false
13961 4753 Acked Read 9 Done Sent Done false
13962 4754 Acked Write 5 Done Sent Done false
13963 4755 Acked Write 4 Done Sent Done false
13964 4756 Acked Write 3 Done Sent Done false
13965 4757 Acked Read 5 Done Sent Done false
13966 4758 Acked Read 9 Done Sent Done false
13967 4759 Acked Flush 0 Done Sent Done false
13968 4760 Acked Read 7 Done Sent Done false
13969 4761 Acked Read 1 Done Sent Done false
13970 4762 Acked Flush 0 Done Sent Done false
13971 4763 Acked Read 8 Done Sent Done false
13972 4764 Acked Read 8 Done Sent Done false
13973 4765 Acked Read 8 Done Sent Done false
13974 4766 Acked Read 10 Done Sent Done false
13975 4767 Acked Read 4 Done Sent Done false
13976 4768 Acked Write 5 Done Sent Done false
13977 4769 Acked Write 5 Done Sent Done false
13978 4770 Acked Read 1 Done Sent Done false
13979 4771 Acked Read 6 Done Sent Done false
13980 4772 Acked Read 10 Done Sent Done false
13981 4773 Acked Write 1 Done Sent Done false
13982 4774 Acked Read 4 Done Sent Done false
13983 4775 Acked Read 10 Done Sent Done false
13984 4776 Acked Write 10 Done Sent Done false
13985 4777 Acked Write 9 Done Sent Done false
13986 4778 Acked Write 8 Done Sent Done false
13987 4779 Acked Read 4 Done Sent Done false
13988 4780 Acked Read 2 Done Sent Done false
13989 4781 Acked Read 10 Done Sent Done false
13990 4782 Acked Read 3 Done Sent Done false
13991 4783 Acked Read 10 Done Sent Done false
13992 4784 Acked Read 2 Done Sent Done false
13993 4785 Acked Flush 0 Done Sent Done false
13994 4786 Acked Read 2 Done Sent Done false
13995 4787 Acked Read 10 Done Sent Done false
13996 4788 Acked Read 3 Done Sent Done false
13997 4789 Acked Write 4 Done Sent Done false
13998 4790 Acked Write 10 Done Sent Done false
13999 4791 Acked Read 2 Done Sent Done false
14000 4792 Acked Read 4 Done Sent Done false
14001 4793 Acked Read 10 Done Sent Done false
14002 4794 Acked Read 2 Done Sent Done false
14003 4795 Acked Read 6 Done Sent Done false
14004 4796 Acked Read 10 Done Sent Done false
14005 4797 Acked Read 6 Done Sent Done false
14006 4798 Acked Write 2 Done Sent Done false
14007 4799 Acked Write 5 Done Sent Done false
14008 4800 Acked Read 4 Done Sent Done false
14009 4801 Acked Write 8 Done Sent Done false
14010 4802 Acked Read 4 Done Sent Done false
14011 4803 Acked Write 2 Done Sent Done false
14012 4804 Acked Read 7 Done Sent Done false
14013 4805 Acked Write 4 Done Sent Done false
14014 4806 Acked Read 4 Done Sent Done false
14015 4807 Acked Flush 0 Done Sent Done false
14016 4808 Acked Read 4 Done Sent Done false
14017 4809 Acked Read 8 Done Sent Done false
14018 4810 Acked Read 10 Done Sent Done false
14019 4811 Acked Write 9 Done Sent Done false
14020 4812 Acked Read 8 Done Sent Done false
14021 4813 Acked Read 1 Done Sent Done false
14022 4814 Acked Read 10 Done Sent Done false
14023 4815 Acked Write 4 Done Sent Done false
14024 4816 Acked Write 3 Done Sent Done false
14025 4817 Acked Read 8 Done Sent Done false
14026 4818 Acked Flush 0 Done Sent Done false
14027 4819 Acked Read 9 Done Sent Done false
14028 4820 Acked Write 2 Done Sent Done false
14029 4821 Acked Read 10 Done Sent Done false
14030 4822 Acked Write 9 Done Sent Done false
14031 4823 Acked Read 4 Done Sent Done false
14032 4824 Acked Read 10 Done Sent Done false
14033 4825 Acked Read 6 Done Sent Done false
14034 4826 Acked Read 10 Done Sent Done false
14035 4827 Acked Write 10 Done Sent Done false
14036 4828 Acked Read 1 Done Sent Done false
14037 4829 Acked Read 1 Done Sent Done false
14038 4830 Acked Read 7 Done Sent Done false
14039 4831 Acked Read 1 Done Sent Done false
14040 4832 Acked Read 3 Done Sent Done false
14041 4833 Acked Write 8 Done Sent Done false
14042 4834 Acked Write 2 Done Sent Done false
14043 4835 Acked Read 2 Done Sent Done false
14044 4836 Acked Write 10 Done Sent Done false
14045 4837 Acked Read 10 Done Sent Done false
14046 4838 Acked Flush 0 Done Sent Done false
14047 4839 Acked Write 5 Done Sent Done false
14048 4840 Acked Read 6 Done Sent Done false
14049 4841 Acked Read 5 Done Sent Done false
14050 4842 Acked Write 3 Done Sent Done false
14051 4843 Acked Write 2 Done Sent Done false
14052 4844 Acked Write 1 Done Sent Done false
14053 4845 Acked Read 1 Done Sent Done false
14054 4846 Acked Read 2 Done Sent Done false
14055 4847 Acked Flush 0 Done Sent Done false
14056 4848 Acked Write 3 Done Sent Done false
14057 4849 Acked Read 6 Done Sent Done false
14058 4850 Acked Read 6 Done Sent Done false
14059 4851 Acked Flush 0 Done Sent Done false
14060 4852 Acked Read 4 Done Sent Done false
14061 4853 Acked Read 4 Done Sent Done false
14062 4854 Acked Write 1 Done Sent Done false
14063 4855 Acked Write 9 Done Sent Done false
14064 4856 Acked Read 2 Done Sent Done false
14065 4857 Acked Read 7 Done Sent Done false
14066 4858 Acked Read 7 Done Sent Done false
14067 4859 Acked Write 9 Done Sent Done false
14068 4860 Acked Flush 0 Done Sent Done false
14069 4861 Acked Read 7 Done Sent Done false
14070 4862 Acked Write 4 Done Sent Done false
14071 4863 Acked Write 1 Done Sent Done false
14072 4864 Acked Write 7 Done Sent Done false
14073 4865 Acked Read 1 Done Sent Done false
14074 4866 Acked Read 10 Done Sent Done false
14075 4867 Acked Write 2 Done Sent Done false
14076 4868 Acked Read 9 Done Sent Done false
14077 4869 Acked Write 6 Done Sent Done false
14078 4870 Acked Read 6 Done Sent Done false
14079 4871 Acked Read 2 Done Sent Done false
14080 4872 Acked Flush 0 Done Sent Done false
14081 4873 Acked Write 5 Done Sent Done false
14082 4874 Acked Read 7 Done Sent Done false
14083 4875 Acked Read 8 Done Sent Done false
14084 4876 Acked Read 3 Done Sent Done false
14085 4877 Acked Read 10 Done Sent Done false
14086 4878 Acked Read 7 Done Sent Done false
14087 4879 Acked Read 10 Done Sent Done false
14088 4880 Acked Flush 0 Done Sent Done false
14089 4881 Acked Read 9 Done Sent Done false
14090 4882 Acked Write 1 Done Sent Done false
14091 4883 Acked Read 1 Done Sent Done false
14092 4884 Acked Read 9 Done Sent Done false
14093 4885 Acked Write 1 Done Sent Done false
14094 4886 Acked Read 3 Done Sent Done false
14095 4887 Acked Write 4 Done Sent Done false
14096 4888 Acked Read 4 Done Sent Done false
14097 4889 Acked Read 3 Done Sent Done false
14098 4890 Acked Read 1 Done Sent Done false
14099 4891 Acked Read 7 Done Sent Done false
14100 4892 Acked Read 3 Done Sent Done false
14101 4893 Acked Read 6 Done Sent Done false
14102 4894 Acked Write 6 Done Sent Done false
14103 4895 Acked Read 2 Done Sent Done false
14104 4896 Acked Write 4 Done Sent Done false
14105 4897 Acked Read 2 Done Sent Done false
14106 4898 Acked Read 4 Done Sent Done false
14107 4899 Acked Write 10 Done Sent Done false
14108 4900 Acked Write 2 Done Sent Done false
14109 4901 Acked Write 6 Done Sent Done false
14110 4902 Acked Write 1 Done Sent Done false
14111 4903 Acked Write 9 Done Sent Done false
14112 4904 Acked Read 6 Done Sent Done false
14113 4905 Acked Write 4 Done Sent Done false
14114 4906 Acked Read 3 Done Sent Done false
14115 4907 Acked Write 3 Done Sent Done false
14116 4908 Acked Flush 0 Done Sent Done false
14117 4909 Acked Read 5 Done Sent Done false
14118 4910 Acked Flush 0 Done Sent Done false
14119 4911 Acked Read 3 Done Sent Done false
14120 4912 Acked Write 1 Done Sent Done false
14121 4913 Acked Read 3 Done Sent Done false
14122 4914 Acked Write 10 Done Sent Done false
14123 4915 Acked Read 9 Done Sent Done false
14124 4916 Acked Read 1 Done Sent Done false
14125 4917 Acked Flush 0 Done Sent Done false
14126 4918 Acked Flush 0 Done Sent Done false
14127 4919 Acked Write 10 Done Sent Done false
14128 4920 Acked Write 8 Done Sent Done false
14129 4921 Acked Write 2 Done Sent Done false
14130 4922 Acked Write 8 Done Sent Done false
14131 4923 Acked Write 10 Done Sent Done false
14132 4924 Acked Flush 0 Done Sent Done false
14133 STATES DS:0 DS:1 DS:2 TOTAL
14134 Sent 0 302 0 302
14135 Done 302 0 302 604
14136 Skipped 0 0 0 0
14137 Error 0 0 0 0
14138 Last Flush: 4924 4622 4924
14139 Downstairs last five completed: 4622 4621 4620 4619 4618
14140 Upstairs last five completed: 4924 4923 4922 4921 4920
14141 CLIENT: Up:0 ds:302 act:2
141422024-11-26T22:07:29.339ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
141432024-11-26T22:07:29.339ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:48628) = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
141442024-11-26T22:07:29.340ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
141452024-11-26T22:07:29.341ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
141462024-11-26T22:07:29.341ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
141472024-11-26T22:07:29.341ZINFOcrucible: send last flush ID to this DS: 4622 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
141482024-11-26T22:07:29.341ZINFOcrucible: Replied this last flush ID: 4622 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
141492024-11-26T22:07:29.341ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
141502024-11-26T22:07:29.341ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
141512024-11-26T22:07:29.341ZINFOcrucible: [1] client re-new 302 jobs since flush 4622 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
141522024-11-26T22:07:29.341ZINFOcrucible: [1] Replayed 302 jobs since flush: 4622 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
14153 ----------------------------------------------------------------
14154 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
14155 Downstairs last five completed: 4924 4923 4922 4921 4920
14156 Upstairs last five completed: 4924 4923 4922 4921 4920
14157 CLIENT: Up:0 ds:0 act:3
14158 Replay: All jobs finished
14159 Main task has work to do, go find it
14160 got dsc Stop(1)
14161 stop 1
14162 [8820] Got stop action so:false kr:false
14163 Nov 26 22:07:31.585 INFO request completed, latency_us: 341, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 5a0edcb3-3873-4630-85a0-ccf1709ac9f2, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
14164 Nov 26 22:07:31.586 INFO request completed, latency_us: 159, response_code: 200, uri: /state/cid/1, method: GET, req_id: d7360ab9-ada7-4a70-8a65-be86011385da, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
141652024-11-26T22:07:31.586ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
141662024-11-26T22:07:31.586ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
141672024-11-26T22:07:31.586ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
141682024-11-26T22:07:31.586ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
141692024-11-26T22:07:31.586ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
141702024-11-26T22:07:31.586ZINFOcrucible: client task is exiting = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
14171 [8820] Exited with: ExitStatus(unix_wait_status(9))
14172 [8820][1] reports Exit
14173 Nov 26 22:07:35.589 INFO request completed, latency_us: 225, response_code: 200, uri: /state/cid/1, method: GET, req_id: ea605089-7f53-4fd8-a003-fc672280a1be, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
14174 001/300 Write block 90 len 20480 data: 4 4 4 5 5
14175 002/300 Write block 2539 len 4096 data: 2
14176 003/300 Read block 2752 len 32768
14177 004/300 Read block 4459 len 4096
14178 005/300 Read block 930 len 24576
14179 006/300 Flush
14180 007/300 Read block 1032 len 40960
14181 008/300 Write block 113 len 36864 data: 3 4 4 3 3 4 3 4 5
14182 009/300 Flush
14183 010/300 Flush
14184 011/300 Read block 437 len 12288
14185 012/300 Read block 4405 len 12288
14186 013/300 Write block 148 len 16384 data: 2 2 2 2
14187 014/300 Write block 4343 len 12288 data: 6 6 5
14188 015/300 Read block 1641 len 28672
14189 016/300 Read block 4076 len 20480
14190 017/300 Read block 3844 len 8192
14191 018/300 Read block 56 len 4096
14192 019/300 Read block 4943 len 40960
14193 020/300 Read block 1592 len 8192
14194 021/300 Write block 3947 len 12288 data: 6 6 5
14195 022/300 Read block 1763 len 28672
14196 023/300 Write block 345 len 40960 data: 5 4 4 4 7 6 7 7 7 7
14197 024/300 Read block 745 len 40960
14198 025/300 Write block 117 len 12288 data: 4 5 4
14199 026/300 Write block 2501 len 20480 data: 4 4 4 4 3
14200 027/300 Write block 1237 len 8192 data: 2 2
14201 028/300 Write block 2494 len 20480 data: 3 3 3 3 3
14202 029/300 Write block 1714 len 20480 data: 3 5 6 6 5
14203 030/300 Write block 3080 len 4096 data: 3
14204 031/300 Read block 1456 len 16384
14205 032/300 Write block 3313 len 36864 data: 3 4 5 5 5 5 5 5 5
14206 033/300 Write block 4990 len 24576 data: 9 8 8 7 7 7
14207 034/300 Write block 1621 len 32768 data: 4 4 4 5 6 5 4 3
14208 035/300 Write block 4981 len 12288 data: 6 6 6
14209 036/300 Write block 1590 len 28672 data: 4 3 3 3 5 4 4
14210 037/300 Read block 135 len 20480
14211 038/300 Read block 4943 len 24576
14212 039/300 Read block 1366 len 8192
14213 040/300 Read block 2252 len 20480
14214 041/300 Write block 3018 len 8192 data: 5 3
14215 042/300 Read block 4754 len 12288
14216 043/300 Read block 4132 len 32768
14217 044/300 Write block 3740 len 12288 data: 3 3 4
14218 045/300 Write block 2702 len 32768 data: 3 3 3 5 4 4 4 6
14219 046/300 Read block 1172 len 12288
14220 047/300 Write block 1263 len 8192 data: 3 3
14221 048/300 Write block 3940 len 8192 data: 2 5
14222 049/300 Flush
14223 050/300 Flush
14224 051/300 Write block 3923 len 36864 data: 3 3 3 4 4 3 4 5 5
14225 052/300 Read block 1988 len 20480
14226 053/300 Write block 4693 len 16384 data: 7 7 6 3
14227 054/300 Read block 3515 len 28672
14228 055/300 Write block 4664 len 12288 data: 5 3 3
14229 056/300 Read block 1889 len 36864
14230 057/300 Flush
14231 058/300 Read block 467 len 20480
14232 059/300 Write block 3054 len 16384 data: 4 3 3 3
14233 060/300 Flush
14234 061/300 Write block 1045 len 20480 data: 2 2 2 3 2
14235 062/300 Read block 3877 len 24576
14236 063/300 Write block 4002 len 4096 data: 3
14237 064/300 Write block 2686 len 24576 data: 3 3 3 4 3 3
14238 065/300 Write block 3456 len 8192 data: 4 4
14239 066/300 Read block 4651 len 28672
14240 067/300 Write block 3984 len 24576 data: 2 2 2 3 3 4
14241 068/300 Write block 4084 len 4096 data: 3
14242 069/300 Write block 1552 len 8192 data: 6 7
14243 070/300 Write block 4037 len 36864 data: 4 3 3 3 4 5 6 5 5
14244 071/300 Read block 1594 len 8192
14245 072/300 Write block 3007 len 16384 data: 3 3 3 5
14246 073/300 Write block 4482 len 36864 data: 3 3 2 3 3 4 4 5 5
14247 074/300 Write block 3137 len 4096 data: 5
14248 075/300 Write block 3789 len 28672 data: 2 2 2 3 4 4 4
14249 076/300 Read block 4359 len 4096
14250 077/300 Flush
14251 078/300 Read block 2766 len 36864
14252 079/300 Write block 2998 len 20480 data: 5 6 5 4 3
14253 080/300 Read block 3667 len 36864
14254 081/300 Flush
14255 082/300 Flush
14256 083/300 Write block 2512 len 28672 data: 3 2 3 3 6 6 6
14257 084/300 Read block 2615 len 20480
14258 085/300 Write block 4711 len 28672 data: 3 2 2 2 3 3 4
14259 086/300 Read block 1135 len 36864
14260 087/300 Write block 4313 len 8192 data: 5 5
14261 088/300 Write block 347 len 4096 data: 5
14262 089/300 Read block 836 len 32768
14263 090/300 Write block 1654 len 4096 data: 5
14264 091/300 Write block 764 len 16384 data: 4 4 4 4
14265 092/300 Write block 3494 len 40960 data: 5 5 5 4 4 4 4 4 5 4
14266 093/300 Read block 3793 len 12288
14267 094/300 Read block 179 len 16384
14268 095/300 Read block 2465 len 36864
14269 096/300 Write block 1024 len 16384 data: 3 3 3 3
14270 097/300 Flush
14271 098/300 Read block 4639 len 24576
14272 099/300 Read block 307 len 32768
14273 100/300 Flush
14274 101/300 Write block 3670 len 24576 data: 9 8 9 9 8 8
14275 102/300 Write block 3757 len 8192 data: 2 3
14276 103/300 Write block 3166 len 24576 data: 5 4 4 3 3 3
14277 104/300 Write block 1360 len 28672 data: 2 2 3 3 4 4 4
14278 105/300 Read block 1388 len 16384
14279 106/300 Read block 3350 len 28672
14280 107/300 Read block 236 len 28672
14281 108/300 Read block 2538 len 40960
14282 109/300 Write block 3303 len 20480 data: 4 4 3 3 3
14283 110/300 Flush
14284 111/300 Write block 3004 len 40960 data: 3 3 3 4 4 4 6 5 4 4
14285 112/300 Write block 1185 len 4096 data: 4
14286 113/300 Flush
14287 114/300 Write block 1190 len 32768 data: 4 4 4 5 5 4 5 6
14288 115/300 Write block 1031 len 20480 data: 3 3 3 3 3
14289 116/300 Read block 1815 len 36864
14290 117/300 Write block 946 len 36864 data: 4 4 3 3 3 2 2 2 3
14291 118/300 Write block 1978 len 24576 data: 2 2 2 3 4 4
14292 119/300 Flush
14293 120/300 Read block 3184 len 32768
14294 121/300 Flush
14295 122/300 Flush
14296 123/300 Flush
14297 124/300 Read block 1139 len 32768
14298 125/300 Write block 2716 len 16384 data: 4 4 3 3
14299 126/300 Write block 2797 len 8192 data: 3 4
14300 127/300 Read block 2812 len 32768
14301 128/300 Read block 3162 len 12288
14302 129/300 Read block 3595 len 20480
14303 130/300 Read block 357 len 8192
14304 131/300 Read block 2951 len 24576
14305 132/300 Write block 3713 len 8192 data: 3 3
14306 133/300 Read block 2564 len 8192
14307 134/300 Write block 1150 len 32768 data: 6 6 4 3 3 3 3 3
14308 135/300 Write block 3507 len 24576 data: 4 4 4 3 2 2
14309 136/300 Write block 3512 len 12288 data: 3 2 2
14310 137/300 Read block 3388 len 32768
14311 138/300 Write block 726 len 4096 data: 5
14312 139/300 Write block 2885 len 36864 data: 5 5 6 7 5 5 3 4 3
14313 140/300 Read block 1627 len 36864
14314 141/300 Read block 1615 len 16384
14315 142/300 Read block 4328 len 20480
14316 143/300 Read block 1074 len 16384
14317 144/300 Write block 33 len 24576 data: 2 2 3 4 4 4
14318 145/300 Read block 4951 len 4096
14319 146/300 Read block 686 len 24576
14320 147/300 Write block 601 len 12288 data: 3 3 3
14321 148/300 Write block 3290 len 32768 data: 3 3 2 3 3 2 2 2
14322 149/300 Write block 2589 len 20480 data: 4 4 3 3 3
14323 150/300 Write block 2876 len 28672 data: 5 6 4 4 4 3 2
14324 151/300 Read block 3611 len 12288
14325 152/300 Write block 539 len 28672 data: 6 5 5 6 5 4 4
14326 153/300 Read block 1487 len 20480
14327 154/300 Read block 1315 len 4096
14328 155/300 Write block 234 len 32768 data: 4 4 4 4 5 5 5 5
14329 156/300 Flush
14330 157/300 Read block 181 len 20480
14331 158/300 Write block 2846 len 20480 data: 6 4 4 2 2
14332 159/300 Read block 4016 len 24576
14333 160/300 Write block 4640 len 20480 data: 4 4 3 3 4
14334 161/300 Flush
14335 162/300 Read block 709 len 4096
14336 163/300 Read block 3361 len 20480
14337 164/300 Read block 3638 len 24576
14338 165/300 Write block 2266 len 24576 data: 4 5 5 5 4 4
14339 166/300 Read block 3281 len 20480
14340 167/300 Read block 3532 len 24576
14341 168/300 Write block 1726 len 40960 data: 5 5 3 4 4 3 3 3 3 3
14342 169/300 Read block 415 len 28672
14343 170/300 Read block 2932 len 20480
14344 171/300 Read block 1587 len 36864
14345 172/300 Read block 4951 len 32768
14346 173/300 Read block 46 len 36864
14347 174/300 Read block 706 len 40960
14348 175/300 Write block 1560 len 32768 data: 2 2 3 3 3 3 3 3
14349 176/300 Write block 4421 len 20480 data: 3 3 3 3 3
14350 177/300 Read block 4170 len 28672
14351 178/300 Flush
14352 179/300 Read block 1889 len 16384
14353 180/300 Write block 4359 len 4096 data: 6
14354 181/300 Write block 4709 len 24576 data: 4 4 4 3 3 3
14355 182/300 Read block 3804 len 24576
14356 183/300 Read block 4122 len 12288
14357 184/300 Read block 660 len 12288
14358 185/300 Read block 2797 len 24576
14359 186/300 Flush
14360 187/300 Write block 1421 len 40960 data: 2 2 3 3 3 3 3 2 2 2
14361 188/300 Write block 1682 len 12288 data: 3 3 3
14362 189/300 Write block 3006 len 4096 data: 4
14363 190/300 Read block 3466 len 8192
14364 191/300 Read block 3002 len 16384
14365 192/300 Write block 641 len 36864 data: 5 5 5 5 4 3 2 2 2
14366 193/300 Write block 2710 len 24576 data: 4 4 4 4 4 4
14367 194/300 Read block 4231 len 28672
14368 195/300 Read block 4420 len 32768
14369 196/300 Write block 1936 len 28672 data: 6 7 8 8 7 6 6
14370 197/300 Write block 2076 len 28672 data: 3 4 4 4 4 4 4
14371 198/300 Read block 3802 len 36864
14372 199/300 Read block 30 len 20480
14373 200/300 Write block 1934 len 36864 data: 5 5 7 8 9 9 8 7 7
14374 201/300 Read block 1967 len 4096
14375 202/300 Write block 4280 len 32768 data: 3 2 2 2 2 2 2 2
14376 203/300 Write block 4385 len 24576 data: 4 4 4 4 3 2
14377 204/300 Write block 1351 len 16384 data: 5 5 4 5
14378 205/300 Write block 6 len 40960 data: 5 5 5 4 4 3 3 3 3 2
14379 206/300 Write block 4750 len 24576 data: 5 4 4 4 4 4
14380 207/300 Write block 4833 len 28672 data: 5 5 4 4 3 3 3
14381 208/300 Read block 1123 len 32768
14382 209/300 Write block 2905 len 16384 data: 4 4 4 4
14383 210/300 Write block 3751 len 28672 data: 4 3 2 2 3 2 3
14384 211/300 Read block 3322 len 12288
14385 212/300 Flush
14386 213/300 Read block 2280 len 4096
14387 214/300 Write block 369 len 28672 data: 6 5 4 4 4 5 4
14388 215/300 Read block 3726 len 4096
14389 216/300 Read block 2909 len 28672
14390 217/300 Read block 4321 len 40960
14391 218/300 Read block 2830 len 4096
14392 219/300 Read block 3236 len 4096
14393 220/300 Read block 4648 len 12288
14394 221/300 Flush
14395 222/300 Read block 4313 len 20480
14396 223/300 Write block 4431 len 4096 data: 3
14397 224/300 Write block 3113 len 12288 data: 3 4 2
14398 225/300 Write block 4441 len 20480 data: 4 5 6 6 4
14399 226/300 Read block 2106 len 40960
14400 227/300 Read block 1191 len 28672
14401 228/300 Read block 3959 len 12288
14402 229/300 Read block 1910 len 40960
14403 230/300 Write block 1207 len 40960 data: 3 3 5 5 5 4 5 5 5 4
14404 231/300 Write block 317 len 40960 data: 3 3 3 3 3 3 4 3 2 2
14405 232/300 Write block 1854 len 20480 data: 4 6 6 6 6
14406 233/300 Read block 3051 len 16384
14407 234/300 Write block 1127 len 36864 data: 4 2 2 2 2 2 4 3 3
14408 235/300 Write block 3766 len 32768 data: 3 2 4 4 3 3 3 3
14409 236/300 Flush
14410 237/300 Read block 3071 len 8192
14411 238/300 Flush
14412 239/300 Write block 2084 len 8192 data: 4 4
14413 240/300 Write block 1105 len 32768 data: 2 3 4 4 3 3 3 3
14414 241/300 Read block 478 len 8192
14415 242/300 Flush
14416 243/300 Read block 3673 len 32768
14417 244/300 Read block 772 len 32768
14418 245/300 Write block 4015 len 8192 data: 5 5
14419 246/300 Write block 4023 len 32768 data: 3 3 5 5 4 5 4 4
14420 247/300 Write block 4396 len 20480 data: 4 3 4 4 3
14421 248/300 Write block 3790 len 12288 data: 3 3 4
14422 249/300 Read block 831 len 12288
14423 250/300 Write block 1824 len 32768 data: 4 4 4 4 5 5 5 4
14424 251/300 Write block 669 len 20480 data: 4 4 4 4 4
14425 252/300 Read block 3248 len 32768
14426 253/300 Write block 4469 len 12288 data: 5 4 3
14427 254/300 Read block 3467 len 4096
14428 255/300 Flush
14429 256/300 Write block 4988 len 24576 data: 6 8 10 9 9 8
14430 257/300 Read block 4531 len 36864
14431 258/300 Write block 3259 len 4096 data: 4
14432 259/300 Read block 4744 len 32768
14433 260/300 Read block 4694 len 24576
14434 261/300 Write block 448 len 20480 data: 6 5 5 5 6
14435 262/300 Read block 1734 len 4096
14436 263/300 Write block 2413 len 32768 data: 3 3 2 2 2 2 2 2
14437 264/300 Read block 3814 len 40960
14438 265/300 Read block 210 len 20480
14439 266/300 Write block 3381 len 16384 data: 3 3 3 3
14440 267/300 Write block 2968 len 36864 data: 4 4 3 3 3 3 3 3 2
14441 268/300 Read block 2876 len 4096
14442 269/300 Write block 2910 len 8192 data: 2 4
14443 270/300 Read block 2263 len 32768
14444 271/300 Read block 2784 len 28672
14445 272/300 Flush
14446 273/300 Write block 287 len 28672 data: 5 6 8 7 7 7 6
14447 274/300 Read block 3820 len 4096
14448 275/300 Write block 709 len 20480 data: 2 2 3 3 3
14449 276/300 Read block 4550 len 16384
14450 277/300 Read block 3244 len 28672
14451 278/300 Write block 4449 len 36864 data: 4 4 5 5 4 4 3 3 3
14452 279/300 Read block 1205 len 24576
14453 280/300 Read block 2112 len 12288
14454 281/300 Flush
14455 282/300 Write block 3728 len 24576 data: 3 3 5 4 3 3
14456 283/300 Read block 191 len 24576
14457 284/300 Write block 4144 len 16384 data: 5 4 4 4
14458 285/300 Write block 228 len 28672 data: 6 5 3 3 3 3 5
14459 286/300 Read block 4972 len 16384
14460 287/300 Write block 416 len 24576 data: 2 2 2 2 3 3
14461 288/300 Read block 3472 len 28672
14462 289/300 Read block 1279 len 28672
14463 290/300 Read block 199 len 16384
14464 291/300 Read block 4579 len 28672
14465 292/300 Flush
14466 293/300 Write block 2320 len 40960 data: 3 4 4 4 3 2 2 2 2 3
14467 294/300 Write block 3030 len 12288 data: 3 3 3
14468 295/300 Write block 3992 len 4096 data: 7
14469 296/300 Read block 596 len 16384
14470 297/300 Read block 2087 len 4096
14471 298/300 Read block 1658 len 16384
14472 299/300 Read block 4598 len 36864
14473 300/300 Write block 2363 len 28672 data: 2 2 2 3 3 3 3
14474 301/300 Write block 3105 len 40960 data: 2 2 3 4 4 5 5 4 4 5
14475 Main task has work to do, go find it
14476 got dsc Start(1)
14477 start 1
14478 Nov 26 22:07:35.819 INFO request completed, latency_us: 250, response_code: 204, uri: /start/cid/1, method: POST, req_id: 2d2940aa-d7ac-4252-bbfd-a82bf4f01db3, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
14479 Replay: started 1, returned:Ok(())
14480 ----------------------------------------------------------------
14481 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
14482 States: Active Offline Active
14483 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
14484 4925 Acked Write 5 Done Sent Done false
14485 4926 Acked Write 1 Done Sent Done false
14486 4927 Acked Read 8 Done Sent Done false
14487 4928 Acked Read 1 Done Sent Done false
14488 4929 Acked Read 6 Done Sent Done false
14489 4930 Acked Flush 0 Done Sent Done false
14490 4931 Acked Read 10 Done Sent Done false
14491 4932 Acked Write 9 Done Sent Done false
14492 4933 Acked Flush 0 Done Sent Done false
14493 4934 Acked Flush 0 Done Sent Done false
14494 4935 Acked Read 3 Done Sent Done false
14495 4936 Acked Read 3 Done Sent Done false
14496 4937 Acked Write 4 Done Sent Done false
14497 4938 Acked Write 3 Done Sent Done false
14498 4939 Acked Read 7 Done Sent Done false
14499 4940 Acked Read 5 Done Sent Done false
14500 4941 Acked Read 2 Done Sent Done false
14501 4942 Acked Read 1 Done Sent Done false
14502 4943 Acked Read 10 Done Sent Done false
14503 4944 Acked Read 2 Done Sent Done false
14504 4945 Acked Write 3 Done Sent Done false
14505 4946 Acked Read 7 Done Sent Done false
14506 4947 Acked Write 10 Done Sent Done false
14507 4948 Acked Read 10 Done Sent Done false
14508 4949 Acked Write 3 Done Sent Done false
14509 4950 Acked Write 5 Done Sent Done false
14510 4951 Acked Write 2 Done Sent Done false
14511 4952 Acked Write 5 Done Sent Done false
14512 4953 Acked Write 5 Done Sent Done false
14513 4954 Acked Write 1 Done Sent Done false
14514 4955 Acked Read 4 Done Sent Done false
14515 4956 Acked Write 9 Done Sent Done false
14516 4957 Acked Write 6 Done Sent Done false
14517 4958 Acked Write 8 Done Sent Done false
14518 4959 Acked Write 3 Done Sent Done false
14519 4960 Acked Write 7 Done Sent Done false
14520 4961 Acked Read 5 Done Sent Done false
14521 4962 Acked Read 6 Done Sent Done false
14522 4963 Acked Read 2 Done Sent Done false
14523 4964 Acked Read 5 Done Sent Done false
14524 4965 Acked Write 2 Done Sent Done false
14525 4966 Acked Read 3 Done Sent Done false
14526 4967 Acked Read 8 Done Sent Done false
14527 4968 Acked Write 3 Done Sent Done false
14528 4969 Acked Write 8 Done Sent Done false
14529 4970 Acked Read 3 Done Sent Done false
14530 4971 Acked Write 2 Done Sent Done false
14531 4972 Acked Write 2 Done Sent Done false
14532 4973 Acked Flush 0 Done Sent Done false
14533 4974 Acked Flush 0 Done Sent Done false
14534 4975 Acked Write 9 Done Sent Done false
14535 4976 Acked Read 5 Done Sent Done false
14536 4977 Acked Write 4 Done Sent Done false
14537 4978 Acked Read 7 Done Sent Done false
14538 4979 Acked Write 3 Done Sent Done false
14539 4980 Acked Read 9 Done Sent Done false
14540 4981 Acked Flush 0 Done Sent Done false
14541 4982 Acked Read 5 Done Sent Done false
14542 4983 Acked Write 4 Done Sent Done false
14543 4984 Acked Flush 0 Done Sent Done false
14544 4985 Acked Write 5 Done Sent Done false
14545 4986 Acked Read 6 Done Sent Done false
14546 4987 Acked Write 1 Done Sent Done false
14547 4988 Acked Write 6 Done Sent Done false
14548 4989 Acked Write 2 Done Sent Done false
14549 4990 Acked Read 7 Done Sent Done false
14550 4991 Acked Write 6 Done Sent Done false
14551 4992 Acked Write 1 Done Sent Done false
14552 4993 Acked Write 2 Done Sent Done false
14553 4994 Acked Write 9 Done Sent Done false
14554 4995 Acked Read 2 Done Sent Done false
14555 4996 Acked Write 4 Done Sent Done false
14556 4997 Acked Write 9 Done Sent Done false
14557 4998 Acked Write 1 Done Sent Done false
14558 4999 Acked Write 7 Done Sent Done false
14559 5000 Acked Read 1 Done Sent Done false
14560 5001 Acked Flush 0 Done Sent Done false
14561 5002 Acked Read 9 Done Sent Done false
14562 5003 Acked Write 5 Done Sent Done false
14563 5004 Acked Read 9 Done Sent Done false
14564 5005 Acked Flush 0 Done Sent Done false
14565 5006 Acked Flush 0 Done Sent Done false
14566 5007 Acked Write 7 Done Sent Done false
14567 5008 Acked Read 5 Done Sent Done false
14568 5009 Acked Write 7 Done Sent Done false
14569 5010 Acked Read 9 Done Sent Done false
14570 5011 Acked Write 2 Done Sent Done false
14571 5012 Acked Write 1 Done Sent Done false
14572 5013 Acked Read 8 Done Sent Done false
14573 5014 Acked Write 1 Done Sent Done false
14574 5015 Acked Write 4 Done Sent Done false
14575 5016 Acked Write 10 Done Sent Done false
14576 5017 Acked Read 3 Done Sent Done false
14577 5018 Acked Read 4 Done Sent Done false
14578 5019 Acked Read 9 Done Sent Done false
14579 5020 Acked Write 4 Done Sent Done false
14580 5021 Acked Flush 0 Done Sent Done false
14581 5022 Acked Read 6 Done Sent Done false
14582 5023 Acked Read 8 Done Sent Done false
14583 5024 Acked Flush 0 Done Sent Done false
14584 5025 Acked Write 6 Done Sent Done false
14585 5026 Acked Write 2 Done Sent Done false
14586 5027 Acked Write 6 Done Sent Done false
14587 5028 Acked Write 7 Done Sent Done false
14588 5029 Acked Read 4 Done Sent Done false
14589 5030 Acked Read 7 Done Sent Done false
14590 5031 Acked Read 7 Done Sent Done false
14591 5032 Acked Read 10 Done Sent Done false
14592 5033 Acked Write 5 Done Sent Done false
14593 5034 Acked Flush 0 Done Sent Done false
14594 5035 Acked Write 10 Done Sent Done false
14595 5036 Acked Write 1 Done Sent Done false
14596 5037 Acked Flush 0 Done Sent Done false
14597 5038 Acked Write 8 Done Sent Done false
14598 5039 Acked Write 5 Done Sent Done false
14599 5040 Acked Read 9 Done Sent Done false
14600 5041 Acked Write 9 Done Sent Done false
14601 5042 Acked Write 6 Done Sent Done false
14602 5043 Acked Flush 0 Done Sent Done false
14603 5044 Acked Read 8 Done Sent Done false
14604 5045 Acked Flush 0 Done Sent Done false
14605 5046 Acked Flush 0 Done Sent Done false
14606 5047 Acked Flush 0 Done Sent Done false
14607 5048 Acked Read 8 Done Sent Done false
14608 5049 Acked Write 4 Done Sent Done false
14609 5050 Acked Write 2 Done Sent Done false
14610 5051 Acked Read 8 Done Sent Done false
14611 5052 Acked Read 3 Done Sent Done false
14612 5053 Acked Read 5 Done Sent Done false
14613 5054 Acked Read 2 Done Sent Done false
14614 5055 Acked Read 6 Done Sent Done false
14615 5056 Acked Write 2 Done Sent Done false
14616 5057 Acked Read 2 Done Sent Done false
14617 5058 Acked Write 8 Done Sent Done false
14618 5059 Acked Write 6 Done Sent Done false
14619 5060 Acked Write 3 Done Sent Done false
14620 5061 Acked Read 8 Done Sent Done false
14621 5062 Acked Write 1 Done Sent Done false
14622 5063 Acked Write 9 Done Sent Done false
14623 5064 Acked Read 9 Done Sent Done false
14624 5065 Acked Read 4 Done Sent Done false
14625 5066 Acked Read 5 Done Sent Done false
14626 5067 Acked Read 4 Done Sent Done false
14627 5068 Acked Write 6 Done Sent Done false
14628 5069 Acked Read 1 Done Sent Done false
14629 5070 Acked Read 6 Done Sent Done false
14630 5071 Acked Write 3 Done Sent Done false
14631 5072 Acked Write 8 Done Sent Done false
14632 5073 Acked Write 5 Done Sent Done false
14633 5074 Acked Write 7 Done Sent Done false
14634 5075 Acked Read 3 Done Sent Done false
14635 5076 Acked Write 7 Done Sent Done false
14636 5077 Acked Read 5 Done Sent Done false
14637 5078 Acked Read 1 Done Sent Done false
14638 5079 Acked Write 8 Done Sent Done false
14639 5080 Acked Flush 0 Done Sent Done false
14640 5081 Acked Read 5 Done Sent Done false
14641 5082 Acked Write 5 Done Sent Done false
14642 5083 Acked Read 6 Done Sent Done false
14643 5084 Acked Write 5 Done Sent Done false
14644 5085 Acked Flush 0 Done Sent Done false
14645 5086 Acked Read 1 Done Sent Done false
14646 5087 Acked Read 5 Done Sent Done false
14647 5088 Acked Read 6 Done Sent Done false
14648 5089 Acked Write 6 Done Sent Done false
14649 5090 Acked Read 5 Done Sent Done false
14650 5091 Acked Read 6 Done Sent Done false
14651 5092 Acked Write 10 Done Sent Done false
14652 5093 Acked Read 7 Done Sent Done false
14653 5094 Acked Read 5 Done Sent Done false
14654 5095 Acked Read 9 Done Sent Done false
14655 5096 Acked Read 8 Done Sent Done false
14656 5097 Acked Read 9 Done Sent Done false
14657 5098 Acked Read 10 Done Sent Done false
14658 5099 Acked Write 8 Done Sent Done false
14659 5100 Acked Write 5 Done Sent Done false
14660 5101 Acked Read 7 Done Sent Done false
14661 5102 Acked Flush 0 Done Sent Done false
14662 5103 Acked Read 4 Done Sent Done false
14663 5104 Acked Write 1 Done Sent Done false
14664 5105 Acked Write 6 Done Sent Done false
14665 5106 Acked Read 6 Done Sent Done false
14666 5107 Acked Read 3 Done Sent Done false
14667 5108 Acked Read 3 Done Sent Done false
14668 5109 Acked Read 6 Done Sent Done false
14669 5110 Acked Flush 0 Done Sent Done false
14670 5111 Acked Write 10 Done Sent Done false
14671 5112 Acked Write 3 Done Sent Done false
14672 5113 Acked Write 1 Done Sent Done false
14673 5114 Acked Read 2 Done Sent Done false
14674 5115 Acked Read 4 Done Sent Done false
14675 5116 Acked Write 9 Done Sent Done false
14676 5117 Acked Write 6 Done Sent Done false
14677 5118 Acked Read 7 Done Sent Done false
14678 5119 Acked Read 8 Done Sent Done false
14679 5120 Acked Write 7 Done Sent Done false
14680 5121 Acked Write 7 Done Sent Done false
14681 5122 Acked Read 9 Done Sent Done false
14682 5123 Acked Read 5 Done Sent Done false
14683 5124 Acked Write 9 Done Sent Done false
14684 5125 Acked Read 1 Done Sent Done false
14685 5126 Acked Write 8 Done Sent Done false
14686 5127 Acked Write 6 Done Sent Done false
14687 5128 Acked Write 4 Done Sent Done false
14688 5129 Acked Write 10 Done Sent Done false
14689 5130 Acked Write 6 Done Sent Done false
14690 5131 Acked Write 7 Done Sent Done false
14691 5132 Acked Read 8 Done Sent Done false
14692 5133 Acked Write 4 Done Sent Done false
14693 5134 Acked Write 7 Done Sent Done false
14694 5135 Acked Read 3 Done Sent Done false
14695 5136 Acked Flush 0 Done Sent Done false
14696 5137 Acked Read 1 Done Sent Done false
14697 5138 Acked Write 7 Done Sent Done false
14698 5139 Acked Read 1 Done Sent Done false
14699 5140 Acked Read 7 Done Sent Done false
14700 5141 Acked Read 10 Done Sent Done false
14701 5142 Acked Read 1 Done Sent Done false
14702 5143 Acked Read 1 Done Sent Done false
14703 5144 Acked Read 3 Done Sent Done false
14704 5145 Acked Flush 0 Done Sent Done false
14705 5146 Acked Read 5 Done Sent Done false
14706 5147 Acked Write 1 Done Sent Done false
14707 5148 Acked Write 3 Done Sent Done false
14708 5149 Acked Write 5 Done Sent Done false
14709 5150 Acked Read 10 Done Sent Done false
14710 5151 Acked Read 7 Done Sent Done false
14711 5152 Acked Read 3 Done Sent Done false
14712 5153 Acked Read 10 Done Sent Done false
14713 5154 Acked Write 10 Done Sent Done false
14714 5155 Acked Write 10 Done Sent Done false
14715 5156 Acked Write 5 Done Sent Done false
14716 5157 Acked Read 4 Done Sent Done false
14717 5158 Acked Write 9 Done Sent Done false
14718 5159 Acked Write 8 Done Sent Done false
14719 5160 Acked Flush 0 Done Sent Done false
14720 5161 Acked Read 2 Done Sent Done false
14721 5162 Acked Flush 0 Done Sent Done false
14722 5163 Acked Write 2 Done Sent Done false
14723 5164 Acked Write 8 Done Sent Done false
14724 5165 Acked Read 2 Done Sent Done false
14725 5166 Acked Flush 0 Done Sent Done false
14726 5167 Acked Read 8 Done Sent Done false
14727 5168 Acked Read 8 Done Sent Done false
14728 5169 Acked Write 2 Done Sent Done false
14729 5170 Acked Write 8 Done Sent Done false
14730 5171 Acked Write 5 Done Sent Done false
14731 5172 Acked Write 3 Done Sent Done false
14732 5173 Acked Read 3 Done Sent Done false
14733 5174 Acked Write 8 Done Sent Done false
14734 5175 Acked Write 5 Done Sent Done false
14735 5176 Acked Read 8 Done Sent Done false
14736 5177 Acked Write 3 Done Sent Done false
14737 5178 Acked Read 1 Done Sent Done false
14738 5179 Acked Flush 0 Done Sent Done false
14739 5180 Acked Write 6 Done Sent Done false
14740 5181 Acked Read 9 Done Sent Done false
14741 5182 Acked Write 1 Done Sent Done false
14742 5183 Acked Read 8 Done Sent Done false
14743 5184 Acked Read 6 Done Sent Done false
14744 5185 Acked Write 5 Done Sent Done false
14745 5186 Acked Read 1 Done Sent Done false
14746 5187 Acked Write 8 Done Sent Done false
14747 5188 Acked Read 10 Done Sent Done false
14748 5189 Acked Read 5 Done Sent Done false
14749 5190 Acked Write 4 Done Sent Done false
14750 5191 Acked Write 9 Done Sent Done false
14751 5192 Acked Read 1 Done Sent Done false
14752 5193 Acked Write 2 Done Sent Done false
14753 5194 Acked Read 8 Done Sent Done false
14754 5195 Acked Read 7 Done Sent Done false
14755 5196 Acked Flush 0 Done Sent Done false
14756 5197 Acked Write 7 Done Sent Done false
14757 5198 Acked Read 1 Done Sent Done false
14758 5199 Acked Write 5 Done Sent Done false
14759 5200 Acked Read 4 Done Sent Done false
14760 5201 Acked Read 7 Done Sent Done false
14761 5202 Acked Write 9 Done Sent Done false
14762 5203 Acked Read 6 Done Sent Done false
14763 5204 Acked Read 3 Done Sent Done false
14764 5205 Acked Flush 0 Done Sent Done false
14765 5206 Acked Write 6 Done Sent Done false
14766 5207 Acked Read 6 Done Sent Done false
14767 5208 Acked Write 4 Done Sent Done false
14768 5209 Acked Write 7 Done Sent Done false
14769 5210 Acked Read 4 Done Sent Done false
14770 5211 Acked Write 6 Done Sent Done false
14771 5212 Acked Read 7 Done Sent Done false
14772 5213 Acked Read 7 Done Sent Done false
14773 5214 Acked Read 4 Done Sent Done false
14774 5215 Acked Read 7 Done Sent Done false
14775 5216 Acked Flush 0 Done Sent Done false
14776 5217 Acked Write 10 Done Sent Done false
14777 5218 Acked Write 3 Done Sent Done false
14778 5219 Acked Write 1 Done Sent Done false
14779 5220 Acked Read 4 Done Sent Done false
14780 5221 Acked Read 1 Done Sent Done false
14781 5222 Acked Read 4 Done Sent Done false
14782 5223 Acked Read 9 Done Sent Done false
14783 5224 Acked Write 7 Done Sent Done false
14784 5225 Acked Write 10 Done Sent Done false
14785 STATES DS:0 DS:1 DS:2 TOTAL
14786 Sent 0 301 0 301
14787 Done 301 0 301 602
14788 Skipped 0 0 0 0
14789 Error 0 0 0 0
14790 Last Flush: 5216 4924 5216
14791 Downstairs last five completed: 4924 4923 4922 4921 4920
14792 Upstairs last five completed: 5225 5224 5223 5222 5221
14793 CLIENT: Up:0 ds:301 act:2
14794 [8820] got start action
14795 [8820] I am going to restart
14796 Starting downstairs at port 8820
14797 Make output file at "/tmp/dsc/downstairs-8820.txt"
14798 [8820][1] reports Starting
14799 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1217)
14800 [8820][1] reports Running
14801 ----------------------------------------------------------------
14802 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
14803 States: Active Offline Active
14804 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
14805 4925 Acked Write 5 Done Sent Done false
14806 4926 Acked Write 1 Done Sent Done false
14807 4927 Acked Read 8 Done Sent Done false
14808 4928 Acked Read 1 Done Sent Done false
14809 4929 Acked Read 6 Done Sent Done false
14810 4930 Acked Flush 0 Done Sent Done false
14811 4931 Acked Read 10 Done Sent Done false
14812 4932 Acked Write 9 Done Sent Done false
14813 4933 Acked Flush 0 Done Sent Done false
14814 4934 Acked Flush 0 Done Sent Done false
14815 4935 Acked Read 3 Done Sent Done false
14816 4936 Acked Read 3 Done Sent Done false
14817 4937 Acked Write 4 Done Sent Done false
14818 4938 Acked Write 3 Done Sent Done false
14819 4939 Acked Read 7 Done Sent Done false
14820 4940 Acked Read 5 Done Sent Done false
14821 4941 Acked Read 2 Done Sent Done false
14822 4942 Acked Read 1 Done Sent Done false
14823 4943 Acked Read 10 Done Sent Done false
14824 4944 Acked Read 2 Done Sent Done false
14825 4945 Acked Write 3 Done Sent Done false
14826 4946 Acked Read 7 Done Sent Done false
14827 4947 Acked Write 10 Done Sent Done false
14828 4948 Acked Read 10 Done Sent Done false
14829 4949 Acked Write 3 Done Sent Done false
14830 4950 Acked Write 5 Done Sent Done false
14831 4951 Acked Write 2 Done Sent Done false
14832 4952 Acked Write 5 Done Sent Done false
14833 4953 Acked Write 5 Done Sent Done false
14834 4954 Acked Write 1 Done Sent Done false
14835 4955 Acked Read 4 Done Sent Done false
14836 4956 Acked Write 9 Done Sent Done false
14837 4957 Acked Write 6 Done Sent Done false
14838 4958 Acked Write 8 Done Sent Done false
14839 4959 Acked Write 3 Done Sent Done false
14840 4960 Acked Write 7 Done Sent Done false
14841 4961 Acked Read 5 Done Sent Done false
14842 4962 Acked Read 6 Done Sent Done false
14843 4963 Acked Read 2 Done Sent Done false
14844 4964 Acked Read 5 Done Sent Done false
14845 4965 Acked Write 2 Done Sent Done false
14846 4966 Acked Read 3 Done Sent Done false
14847 4967 Acked Read 8 Done Sent Done false
14848 4968 Acked Write 3 Done Sent Done false
14849 4969 Acked Write 8 Done Sent Done false
14850 4970 Acked Read 3 Done Sent Done false
14851 4971 Acked Write 2 Done Sent Done false
14852 4972 Acked Write 2 Done Sent Done false
14853 4973 Acked Flush 0 Done Sent Done false
14854 4974 Acked Flush 0 Done Sent Done false
14855 4975 Acked Write 9 Done Sent Done false
14856 4976 Acked Read 5 Done Sent Done false
14857 4977 Acked Write 4 Done Sent Done false
14858 4978 Acked Read 7 Done Sent Done false
14859 4979 Acked Write 3 Done Sent Done false
14860 4980 Acked Read 9 Done Sent Done false
14861 4981 Acked Flush 0 Done Sent Done false
14862 4982 Acked Read 5 Done Sent Done false
14863 4983 Acked Write 4 Done Sent Done false
14864 4984 Acked Flush 0 Done Sent Done false
14865 4985 Acked Write 5 Done Sent Done false
14866 4986 Acked Read 6 Done Sent Done false
14867 4987 Acked Write 1 Done Sent Done false
14868 4988 Acked Write 6 Done Sent Done false
14869 4989 Acked Write 2 Done Sent Done false
14870 4990 Acked Read 7 Done Sent Done false
14871 4991 Acked Write 6 Done Sent Done false
14872 4992 Acked Write 1 Done Sent Done false
14873 4993 Acked Write 2 Done Sent Done false
14874 4994 Acked Write 9 Done Sent Done false
14875 4995 Acked Read 2 Done Sent Done false
14876 4996 Acked Write 4 Done Sent Done false
14877 4997 Acked Write 9 Done Sent Done false
14878 4998 Acked Write 1 Done Sent Done false
14879 4999 Acked Write 7 Done Sent Done false
14880 5000 Acked Read 1 Done Sent Done false
14881 5001 Acked Flush 0 Done Sent Done false
14882 5002 Acked Read 9 Done Sent Done false
14883 5003 Acked Write 5 Done Sent Done false
14884 5004 Acked Read 9 Done Sent Done false
14885 5005 Acked Flush 0 Done Sent Done false
14886 5006 Acked Flush 0 Done Sent Done false
14887 5007 Acked Write 7 Done Sent Done false
14888 5008 Acked Read 5 Done Sent Done false
14889 5009 Acked Write 7 Done Sent Done false
14890 5010 Acked Read 9 Done Sent Done false
14891 5011 Acked Write 2 Done Sent Done false
14892 5012 Acked Write 1 Done Sent Done false
14893 5013 Acked Read 8 Done Sent Done false
14894 5014 Acked Write 1 Done Sent Done false
14895 5015 Acked Write 4 Done Sent Done false
14896 5016 Acked Write 10 Done Sent Done false
14897 5017 Acked Read 3 Done Sent Done false
14898 5018 Acked Read 4 Done Sent Done false
14899 5019 Acked Read 9 Done Sent Done false
14900 5020 Acked Write 4 Done Sent Done false
14901 5021 Acked Flush 0 Done Sent Done false
14902 5022 Acked Read 6 Done Sent Done false
14903 5023 Acked Read 8 Done Sent Done false
14904 5024 Acked Flush 0 Done Sent Done false
14905 5025 Acked Write 6 Done Sent Done false
14906 5026 Acked Write 2 Done Sent Done false
14907 5027 Acked Write 6 Done Sent Done false
14908 5028 Acked Write 7 Done Sent Done false
14909 5029 Acked Read 4 Done Sent Done false
14910 5030 Acked Read 7 Done Sent Done false
14911 5031 Acked Read 7 Done Sent Done false
14912 5032 Acked Read 10 Done Sent Done false
14913 5033 Acked Write 5 Done Sent Done false
14914 5034 Acked Flush 0 Done Sent Done false
14915 5035 Acked Write 10 Done Sent Done false
14916 5036 Acked Write 1 Done Sent Done false
14917 5037 Acked Flush 0 Done Sent Done false
14918 5038 Acked Write 8 Done Sent Done false
14919 5039 Acked Write 5 Done Sent Done false
14920 5040 Acked Read 9 Done Sent Done false
14921 5041 Acked Write 9 Done Sent Done false
14922 5042 Acked Write 6 Done Sent Done false
14923 5043 Acked Flush 0 Done Sent Done false
14924 5044 Acked Read 8 Done Sent Done false
14925 5045 Acked Flush 0 Done Sent Done false
14926 5046 Acked Flush 0 Done Sent Done false
14927 5047 Acked Flush 0 Done Sent Done false
14928 5048 Acked Read 8 Done Sent Done false
14929 5049 Acked Write 4 Done Sent Done false
14930 5050 Acked Write 2 Done Sent Done false
14931 5051 Acked Read 8 Done Sent Done false
14932 5052 Acked Read 3 Done Sent Done false
14933 5053 Acked Read 5 Done Sent Done false
14934 5054 Acked Read 2 Done Sent Done false
14935 5055 Acked Read 6 Done Sent Done false
14936 5056 Acked Write 2 Done Sent Done false
14937 5057 Acked Read 2 Done Sent Done false
14938 5058 Acked Write 8 Done Sent Done false
14939 5059 Acked Write 6 Done Sent Done false
14940 5060 Acked Write 3 Done Sent Done false
14941 5061 Acked Read 8 Done Sent Done false
14942 5062 Acked Write 1 Done Sent Done false
14943 5063 Acked Write 9 Done Sent Done false
14944 5064 Acked Read 9 Done Sent Done false
14945 5065 Acked Read 4 Done Sent Done false
14946 5066 Acked Read 5 Done Sent Done false
14947 5067 Acked Read 4 Done Sent Done false
14948 5068 Acked Write 6 Done Sent Done false
14949 5069 Acked Read 1 Done Sent Done false
14950 5070 Acked Read 6 Done Sent Done false
14951 5071 Acked Write 3 Done Sent Done false
14952 5072 Acked Write 8 Done Sent Done false
14953 5073 Acked Write 5 Done Sent Done false
14954 5074 Acked Write 7 Done Sent Done false
14955 5075 Acked Read 3 Done Sent Done false
14956 5076 Acked Write 7 Done Sent Done false
14957 5077 Acked Read 5 Done Sent Done false
14958 5078 Acked Read 1 Done Sent Done false
14959 5079 Acked Write 8 Done Sent Done false
14960 5080 Acked Flush 0 Done Sent Done false
14961 5081 Acked Read 5 Done Sent Done false
14962 5082 Acked Write 5 Done Sent Done false
14963 5083 Acked Read 6 Done Sent Done false
14964 5084 Acked Write 5 Done Sent Done false
14965 5085 Acked Flush 0 Done Sent Done false
14966 5086 Acked Read 1 Done Sent Done false
14967 5087 Acked Read 5 Done Sent Done false
14968 5088 Acked Read 6 Done Sent Done false
14969 5089 Acked Write 6 Done Sent Done false
14970 5090 Acked Read 5 Done Sent Done false
14971 5091 Acked Read 6 Done Sent Done false
14972 5092 Acked Write 10 Done Sent Done false
14973 5093 Acked Read 7 Done Sent Done false
14974 5094 Acked Read 5 Done Sent Done false
14975 5095 Acked Read 9 Done Sent Done false
14976 5096 Acked Read 8 Done Sent Done false
14977 5097 Acked Read 9 Done Sent Done false
14978 5098 Acked Read 10 Done Sent Done false
14979 5099 Acked Write 8 Done Sent Done false
14980 5100 Acked Write 5 Done Sent Done false
14981 5101 Acked Read 7 Done Sent Done false
14982 5102 Acked Flush 0 Done Sent Done false
14983 5103 Acked Read 4 Done Sent Done false
14984 5104 Acked Write 1 Done Sent Done false
14985 5105 Acked Write 6 Done Sent Done false
14986 5106 Acked Read 6 Done Sent Done false
14987 5107 Acked Read 3 Done Sent Done false
14988 5108 Acked Read 3 Done Sent Done false
14989 5109 Acked Read 6 Done Sent Done false
14990 5110 Acked Flush 0 Done Sent Done false
14991 5111 Acked Write 10 Done Sent Done false
14992 5112 Acked Write 3 Done Sent Done false
14993 5113 Acked Write 1 Done Sent Done false
14994 5114 Acked Read 2 Done Sent Done false
14995 5115 Acked Read 4 Done Sent Done false
14996 5116 Acked Write 9 Done Sent Done false
14997 5117 Acked Write 6 Done Sent Done false
14998 5118 Acked Read 7 Done Sent Done false
14999 5119 Acked Read 8 Done Sent Done false
15000 5120 Acked Write 7 Done Sent Done false
15001 5121 Acked Write 7 Done Sent Done false
15002 5122 Acked Read 9 Done Sent Done false
15003 5123 Acked Read 5 Done Sent Done false
15004 5124 Acked Write 9 Done Sent Done false
15005 5125 Acked Read 1 Done Sent Done false
15006 5126 Acked Write 8 Done Sent Done false
15007 5127 Acked Write 6 Done Sent Done false
15008 5128 Acked Write 4 Done Sent Done false
15009 5129 Acked Write 10 Done Sent Done false
15010 5130 Acked Write 6 Done Sent Done false
15011 5131 Acked Write 7 Done Sent Done false
15012 5132 Acked Read 8 Done Sent Done false
15013 5133 Acked Write 4 Done Sent Done false
15014 5134 Acked Write 7 Done Sent Done false
15015 5135 Acked Read 3 Done Sent Done false
15016 5136 Acked Flush 0 Done Sent Done false
15017 5137 Acked Read 1 Done Sent Done false
15018 5138 Acked Write 7 Done Sent Done false
15019 5139 Acked Read 1 Done Sent Done false
15020 5140 Acked Read 7 Done Sent Done false
15021 5141 Acked Read 10 Done Sent Done false
15022 5142 Acked Read 1 Done Sent Done false
15023 5143 Acked Read 1 Done Sent Done false
15024 5144 Acked Read 3 Done Sent Done false
15025 5145 Acked Flush 0 Done Sent Done false
15026 5146 Acked Read 5 Done Sent Done false
15027 5147 Acked Write 1 Done Sent Done false
15028 5148 Acked Write 3 Done Sent Done false
15029 5149 Acked Write 5 Done Sent Done false
15030 5150 Acked Read 10 Done Sent Done false
15031 5151 Acked Read 7 Done Sent Done false
15032 5152 Acked Read 3 Done Sent Done false
15033 5153 Acked Read 10 Done Sent Done false
15034 5154 Acked Write 10 Done Sent Done false
15035 5155 Acked Write 10 Done Sent Done false
15036 5156 Acked Write 5 Done Sent Done false
15037 5157 Acked Read 4 Done Sent Done false
15038 5158 Acked Write 9 Done Sent Done false
15039 5159 Acked Write 8 Done Sent Done false
15040 5160 Acked Flush 0 Done Sent Done false
15041 5161 Acked Read 2 Done Sent Done false
15042 5162 Acked Flush 0 Done Sent Done false
15043 5163 Acked Write 2 Done Sent Done false
15044 5164 Acked Write 8 Done Sent Done false
15045 5165 Acked Read 2 Done Sent Done false
15046 5166 Acked Flush 0 Done Sent Done false
15047 5167 Acked Read 8 Done Sent Done false
15048 5168 Acked Read 8 Done Sent Done false
15049 5169 Acked Write 2 Done Sent Done false
15050 5170 Acked Write 8 Done Sent Done false
15051 5171 Acked Write 5 Done Sent Done false
15052 5172 Acked Write 3 Done Sent Done false
15053 5173 Acked Read 3 Done Sent Done false
15054 5174 Acked Write 8 Done Sent Done false
15055 5175 Acked Write 5 Done Sent Done false
15056 5176 Acked Read 8 Done Sent Done false
15057 5177 Acked Write 3 Done Sent Done false
15058 5178 Acked Read 1 Done Sent Done false
15059 5179 Acked Flush 0 Done Sent Done false
15060 5180 Acked Write 6 Done Sent Done false
15061 5181 Acked Read 9 Done Sent Done false
15062 5182 Acked Write 1 Done Sent Done false
15063 5183 Acked Read 8 Done Sent Done false
15064 5184 Acked Read 6 Done Sent Done false
15065 5185 Acked Write 5 Done Sent Done false
15066 5186 Acked Read 1 Done Sent Done false
15067 5187 Acked Write 8 Done Sent Done false
15068 5188 Acked Read 10 Done Sent Done false
15069 5189 Acked Read 5 Done Sent Done false
15070 5190 Acked Write 4 Done Sent Done false
15071 5191 Acked Write 9 Done Sent Done false
15072 5192 Acked Read 1 Done Sent Done false
15073 5193 Acked Write 2 Done Sent Done false
15074 5194 Acked Read 8 Done Sent Done false
15075 5195 Acked Read 7 Done Sent Done false
15076 5196 Acked Flush 0 Done Sent Done false
15077 5197 Acked Write 7 Done Sent Done false
15078 5198 Acked Read 1 Done Sent Done false
15079 5199 Acked Write 5 Done Sent Done false
15080 5200 Acked Read 4 Done Sent Done false
15081 5201 Acked Read 7 Done Sent Done false
15082 5202 Acked Write 9 Done Sent Done false
15083 5203 Acked Read 6 Done Sent Done false
15084 5204 Acked Read 3 Done Sent Done false
15085 5205 Acked Flush 0 Done Sent Done false
15086 5206 Acked Write 6 Done Sent Done false
15087 5207 Acked Read 6 Done Sent Done false
15088 5208 Acked Write 4 Done Sent Done false
15089 5209 Acked Write 7 Done Sent Done false
15090 5210 Acked Read 4 Done Sent Done false
15091 5211 Acked Write 6 Done Sent Done false
15092 5212 Acked Read 7 Done Sent Done false
15093 5213 Acked Read 7 Done Sent Done false
15094 5214 Acked Read 4 Done Sent Done false
15095 5215 Acked Read 7 Done Sent Done false
15096 5216 Acked Flush 0 Done Sent Done false
15097 5217 Acked Write 10 Done Sent Done false
15098 5218 Acked Write 3 Done Sent Done false
15099 5219 Acked Write 1 Done Sent Done false
15100 5220 Acked Read 4 Done Sent Done false
15101 5221 Acked Read 1 Done Sent Done false
15102 5222 Acked Read 4 Done Sent Done false
15103 5223 Acked Read 9 Done Sent Done false
15104 5224 Acked Write 7 Done Sent Done false
15105 5225 Acked Write 10 Done Sent Done false
15106 5226 Acked Flush 0 Done Sent Done false
15107 STATES DS:0 DS:1 DS:2 TOTAL
15108 Sent 0 302 0 302
15109 Done 302 0 302 604
15110 Skipped 0 0 0 0
15111 Error 0 0 0 0
15112 Last Flush: 5226 4924 5226
15113 Downstairs last five completed: 4924 4923 4922 4921 4920
15114 Upstairs last five completed: 5226 5225 5224 5223 5222
15115 CLIENT: Up:0 ds:302 act:2
151162024-11-26T22:07:41.588ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
151172024-11-26T22:07:41.588ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:40764) = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
151182024-11-26T22:07:41.589ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
151192024-11-26T22:07:41.590ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
151202024-11-26T22:07:41.590ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
151212024-11-26T22:07:41.590ZINFOcrucible: send last flush ID to this DS: 4924 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
151222024-11-26T22:07:41.590ZINFOcrucible: Replied this last flush ID: 4924 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
151232024-11-26T22:07:41.590ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
151242024-11-26T22:07:41.590ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
151252024-11-26T22:07:41.590ZINFOcrucible: [1] client re-new 302 jobs since flush 4924 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
151262024-11-26T22:07:41.590ZINFOcrucible: [1] Replayed 302 jobs since flush: 4924 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
15127 ----------------------------------------------------------------
15128 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
15129 Downstairs last five completed: 5226 5225 5224 5223 5222
15130 Upstairs last five completed: 5226 5225 5224 5223 5222
15131 CLIENT: Up:0 ds:0 act:3
15132 Replay: All jobs finished
15133 Main task has work to do, go find it
15134 got dsc Stop(1)
15135 stop 1
15136 Nov 26 22:07:43.824 INFO request completed, latency_us[8820] Got stop action so:false kr:false
15137 : 342, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 90d0defb-77cc-4339-a1d5-75167ad28c37, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
15138 Nov 26 22:07:43.824 INFO request completed, latency_us: 112, response_code: 200, uri: {"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2024-11-26T22:07:43.824757456Z","hostname":"w-01JDN7ZQ17ABQG15KHKN3F2D26","pid":1203,"":"io task","client":"1","":"downstairs","session_id":"86a36c17-3718-484c-87fd-2b8a1dc2ab18"}
15139 /state/cid/1, method: GET, req_id: c53f9b98-e54e-4772-87f3-330ee810130e, remote_addr: 127.{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2024-11-26T22:07:43.824845045Z","hostname":"w-01JDN7ZQ17ABQG15KHKN3F2D26","pid":1203,"":"io task","client":"1","":"downstairs","session_id":"86a36c17-3718-484c-87fd-2b8a1dc2ab18"}
15140 0{"msg":"downstairs task for 1 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2024-11-26T22:07:43.824911007Z","hostname":"w-01JDN7ZQ17ABQG15KHKN3F2D26","pid":1203,"session_id":"86a36c17-3718-484c-87fd-2b8a1dc2ab18"}
15141 .0.1:36829, local_addr: 127.0.0.1:9998
151422024-11-26T22:07:43.824ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
151432024-11-26T22:07:43.825ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
151442024-11-26T22:07:43.825ZINFOcrucible: client task is exiting = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
15145 [8820] Exited with: ExitStatus(unix_wait_status(9))
15146 [8820][1] reports Exit
15147 Nov 26 22:07:47.826 INFO request completed, latency_us: 210, response_code: 200, uri: /state/cid/1, method: GET, req_id: b65c523c-bb3d-463a-a33c-7293e6816f5d, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
15148 001/300 Write block 1143 len 16384 data: 2 4 4 4
15149 002/300 Read block 4104 len 28672
15150 003/300 Write block 549 len 40960 data: 2 2 2 2 2 2 2 2 3 3
15151 004/300 Read block 4516 len 32768
15152 005/300 Write block 1598 len 8192 data: 4 6
15153 006/300 Write block 4856 len 8192 data: 5 4
15154 007/300 Write block 922 len 40960 data: 2 2 4 4 6 5 4 6 7 5
15155 008/300 Read block 554 len 4096
15156 009/300 Write block 641 len 28672 data: 6 6 6 6 5 4 3
15157 010/300 Read block 2658 len 32768
15158 011/300 Write block 2817 len 20480 data: 3 3 2 2 2
15159 012/300 Flush
15160 013/300 Write block 1452 len 24576 data: 2 2 2 2 2 2
15161 014/300 Read block 2681 len 24576
15162 015/300 Read block 1583 len 20480
15163 016/300 Write block 4025 len 20480 data: 6 6 5 6 5
15164 017/300 Write block 2819 len 32768 data: 3 3 3 3 3 2 2 2
15165 018/300 Read block 755 len 24576
15166 019/300 Write block 2748 len 8192 data: 5 4
15167 020/300 Read block 4656 len 32768
15168 021/300 Write block 2513 len 40960 data: 3 4 4 7 7 7 6 5 4 3
15169 022/300 Read block 1166 len 40960
15170 023/300 Write block 2470 len 32768 data: 4 5 5 6 5 4 4 3
15171 024/300 Write block 1606 len 40960 data: 2 3 3 4 4 4 4 4 3 3
15172 025/300 Write block 4515 len 36864 data: 4 3 2 2 2 2 2 2 2
15173 026/300 Write block 4266 len 20480 data: 3 4 3 2 2
15174 027/300 Read block 4799 len 12288
15175 028/300 Flush
15176 029/300 Read block 376 len 24576
15177 030/300 Read block 4994 len 4096
15178 031/300 Read block 4553 len 24576
15179 032/300 Write block 4504 len 32768 data: 2 2 2 3 3 4 3 3
15180 033/300 Write block 4278 len 16384 data: 3 3 4 3
15181 034/300 Read block 2182 len 40960
15182 035/300 Write block 2860 len 20480 data: 4 4 3 4 4
15183 036/300 Read block 3600 len 16384
15184 037/300 Read block 2313 len 40960
15185 038/300 Write block 2713 len 20480 data: 5 5 5 5 5
15186 039/300 Write block 2013 len 32768 data: 2 2 2 3 3 3 3 2
15187 040/300 Read block 3479 len 36864
15188 041/300 Read block 3866 len 24576
15189 042/300 Read block 1778 len 12288
15190 043/300 Write block 2621 len 24576 data: 3 3 4 4 4 4
15191 044/300 Write block 3768 len 4096 data: 5
15192 045/300 Read block 3693 len 40960
15193 046/300 Read block 3040 len 8192
15194 047/300 Write block 3539 len 20480 data: 2 4 4 4 3
15195 048/300 Read block 1008 len 8192
15196 049/300 Write block 2445 len 32768 data: 4 3 2 2 2 2 2 2
15197 050/300 Read block 3714 len 20480
15198 051/300 Read block 1078 len 24576
15199 052/300 Read block 4302 len 8192
15200 053/300 Read block 4347 len 8192
15201 054/300 Write block 3616 len 24576 data: 3 3 3 3 2 2
15202 055/300 Flush
15203 056/300 Read block 2949 len 24576
15204 057/300 Write block 539 len 28672 data: 7 6 6 7 6 5 5
15205 058/300 Write block 2665 len 28672 data: 4 3 4 4 3 4 4
15206 059/300 Read block 3006 len 36864
15207 060/300 Read block 3040 len 32768
15208 061/300 Write block 3762 len 24576 data: 3 4 4 3 4 3
15209 062/300 Flush
15210 063/300 Read block 616 len 36864
15211 064/300 Read block 2510 len 20480
15212 065/300 Read block 1357 len 12288
15213 066/300 Flush
15214 067/300 Read block 3904 len 12288
15215 068/300 Write block 3845 len 8192 data: 3 3
15216 069/300 Read block 572 len 40960
15217 070/300 Write block 2252 len 20480 data: 5 6 5 2 2
15218 071/300 Read block 4008 len 20480
15219 072/300 Read block 4627 len 4096
15220 073/300 Write block 1298 len 28672 data: 3 2 2 2 2 3 3
15221 074/300 Read block 69 len 40960
15222 075/300 Write block 1847 len 40960 data: 6 6 6 6 6 7 5 5 7 7
15223 076/300 Read block 3589 len 36864
15224 077/300 Write block 2945 len 36864 data: 2 3 3 2 2 2 2 3 3
15225 078/300 Write block 1660 len 36864 data: 5 4 3 3 3 3 2 3 3
15226 079/300 Write block 3676 len 12288 data: 9 9 8
15227 080/300 Read block 2811 len 36864
15228 081/300 Read block 1203 len 8192
15229 082/300 Read block 302 len 28672
15230 083/300 Write block 3597 len 20480 data: 7 7 7 5 6
15231 084/300 Write block 631 len 24576 data: 4 4 3 3 3 4
15232 085/300 Read block 2386 len 40960
15233 086/300 Flush
15234 087/300 Read block 4461 len 40960
15235 088/300 Read block 195 len 4096
15236 089/300 Read block 1230 len 12288
15237 090/300 Flush
15238 091/300 Read block 1170 len 8192
15239 092/300 Read block 2887 len 28672
15240 093/300 Write block 2603 len 4096 data: 5
15241 094/300 Read block 2284 len 16384
15242 095/300 Read block 4703 len 4096
15243 096/300 Read block 4536 len 24576
15244 097/300 Read block 4053 len 20480
15245 098/300 Read block 4659 len 12288
15246 099/300 Write block 2436 len 36864 data: 3 2 2 2 2 3 3 4 4
15247 100/300 Read block 3927 len 16384
15248 101/300 Write block 1540 len 32768 data: 6 6 5 4 3 2 2 3
15249 102/300 Write block 2829 len 4096 data: 5
15250 103/300 Write block 1159 len 24576 data: 4 4 4 3 4 3
15251 104/300 Write block 391 len 8192 data: 2 3
15252 105/300 Write block 599 len 4096 data: 3
15253 106/300 Write block 543 len 8192 data: 7 6
15254 107/300 Read block 4797 len 12288
15255 108/300 Write block 1132 len 40960 data: 3 5 4 4 2 2 2 2 3 3
15256 109/300 Read block 1051 len 4096
15257 110/300 Write block 1607 len 36864 data: 4 4 5 5 5 5 5 4 4
15258 111/300 Write block 1192 len 40960 data: 5 6 6 5 6 7 5 5 4 5
15259 112/300 Read block 2944 len 4096
15260 113/300 Read block 4993 len 20480
15261 114/300 Write block 3230 len 20480 data: 9 7 7 6 4
15262 115/300 Read block 2258 len 36864
15263 116/300 Flush
15264 117/300 Read block 2376 len 12288
15265 118/300 Read block 4318 len 20480
15266 119/300 Read block 1092 len 8192
15267 120/300 Read block 3601 len 40960
15268 121/300 Read block 352 len 4096
15269 122/300 Read block 3998 len 4096
15270 123/300 Write block 317 len 36864 data: 4 4 4 4 4 4 5 4 3
15271 124/300 Read block 542 len 16384
15272 125/300 Write block 3942 len 24576 data: 4 4 6 5 4 7
15273 126/300 Write block 399 len 4096 data: 4
15274 127/300 Write block 1282 len 40960 data: 6 5 6 6 5 4 4 4 4 4
15275 128/300 Write block 1231 len 40960 data: 4 4 5 5 5 4 3 3 3 3
15276 129/300 Read block 483 len 40960
15277 130/300 Read block 4348 len 4096
15278 131/300 Read block 2171 len 40960
15279 132/300 Read block 2250 len 16384
15280 133/300 Read block 811 len 16384
15281 134/300 Flush
15282 135/300 Write block 1916 len 32768 data: 2 2 2 4 4 5 5 5
15283 136/300 Read block 2745 len 24576
15284 137/300 Read block 625 len 28672
15285 138/300 Write block 1806 len 40960 data: 3 4 3 3 3 4 3 3 3 4
15286 139/300 Read block 4304 len 24576
15287 140/300 Read block 156 len 24576
15288 141/300 Read block 2105 len 36864
15289 142/300 Write block 906 len 28672 data: 3 3 4 6 8 8 7
15290 143/300 Write block 1305 len 12288 data: 3 3 3
15291 144/300 Read block 1147 len 28672
15292 145/300 Read block 2038 len 28672
15293 146/300 Read block 2937 len 16384
15294 147/300 Write block 865 len 24576 data: 4 4 4 4 3 5
15295 148/300 Read block 107 len 16384
15296 149/300 Read block 1595 len 28672
15297 150/300 Read block 389 len 24576
15298 151/300 Read block 4581 len 36864
15299 152/300 Flush
15300 153/300 Flush
15301 154/300 Write block 214 len 24576 data: 2 3 5 5 6 6
15302 155/300 Read block 2864 len 20480
15303 156/300 Write block 1825 len 4096 data: 5
15304 157/300 Read block 3517 len 4096
15305 158/300 Write block 1100 len 20480 data: 4 3 4 4 2
15306 159/300 Read block 2735 len 8192
15307 160/300 Read block 1572 len 28672
15308 161/300 Write block 3510 len 20480 data: 4 3 4 3 3
15309 162/300 Read block 2271 len 20480
15310 163/300 Write block 1578 len 32768 data: 3 3 5 4 4 3 2 2
15311 164/300 Read block 650 len 40960
15312 165/300 Flush
15313 166/300 Write block 89 len 32768 data: 4 5 5 5 6 6 5 7
15314 167/300 Flush
15315 168/300 Write block 3750 len 16384 data: 4 5 4 3
15316 169/300 Read block 129 len 24576
15317 170/300 Flush
15318 171/300 Flush
15319 172/300 Write block 4772 len 12288 data: 5 5 4
15320 173/300 Write block 4592 len 32768 data: 3 4 4 3 4 4 4 4
15321 174/300 Read block 2602 len 24576
15322 175/300 Read block 1598 len 40960
15323 176/300 Flush
15324 177/300 Read block 753 len 12288
15325 178/300 Read block 1241 len 20480
15326 179/300 Read block 3592 len 28672
15327 180/300 Read block 2612 len 28672
15328 181/300 Read block 347 len 16384
15329 182/300 Read block 2949 len 16384
15330 183/300 Read block 2193 len 40960
15331 184/300 Write block 3520 len 24576 data: 2 2 2 2 2 2
15332 185/300 Read block 3793 len 4096
15333 186/300 Flush
15334 187/300 Write block 3108 len 32768 data: 5 5 6 6 5 5 6 3
15335 188/300 Write block 2395 len 28672 data: 7 5 5 5 4 4 5
15336 189/300 Write block 4771 len 24576 data: 5 6 6 5 4 4
15337 190/300 Read block 4966 len 12288
15338 191/300 Read block 1667 len 36864
15339 192/300 Write block 1960 len 36864 data: 3 3 3 2 2 3 3 3 3
15340 193/300 Read block 1905 len 40960
15341 194/300 Write block 3394 len 28672 data: 5 6 6 6 6 5 4
15342 195/300 Write block 4449 len 24576 data: 5 5 6 6 5 5
15343 196/300 Read block 2228 len 32768
15344 197/300 Read block 3135 len 16384
15345 198/300 Flush
15346 199/300 Read block 2134 len 12288
15347 200/300 Read block 1592 len 20480
15348 201/300 Read block 1015 len 20480
15349 202/300 Flush
15350 203/300 Write block 1274 len 24576 data: 4 3 3 4 4 4
15351 204/300 Write block 2887 len 4096 data: 7
15352 205/300 Flush
15353 206/300 Write block 690 len 8192 data: 2 2
15354 207/300 Flush
15355 208/300 Write block 4017 len 36864 data: 5 6 4 3 3 3 4 4 7
15356 209/300 Read block 2075 len 32768
15357 210/300 Flush
15358 211/300 Read block 3504 len 28672
15359 212/300 Write block 3851 len 32768 data: 3 3 3 4 4 4 6 6
15360 213/300 Read block 2687 len 4096
15361 214/300 Read block 3446 len 28672
15362 215/300 Read block 2748 len 32768
15363 216/300 Read block 3447 len 12288
15364 217/300 Write block 4915 len 16384 data: 2 2 2 2
15365 218/300 Read block 2331 len 40960
15366 219/300 Flush
15367 220/300 Write block 4904 len 24576 data: 3 3 3 3 2 2
15368 221/300 Write block 4279 len 32768 data: 4 5 4 3 3 3 3 3
15369 222/300 Read block 2009 len 24576
15370 223/300 Write block 3631 len 12288 data: 4 4 4
15371 224/300 Read block 3246 len 40960
15372 225/300 Write block 3155 len 4096 data: 5
15373 226/300 Write block 2327 len 20480 data: 3 3 4 4 3
15374 227/300 Flush
15375 228/300 Read block 4694 len 32768
15376 229/300 Read block 2368 len 32768
15377 230/300 Read block 74 len 16384
15378 231/300 Write block 335 len 4096 data: 3
15379 232/300 Write block 1131 len 24576 data: 3 4 6 5 5 3
15380 233/300 Write block 1963 len 28672 data: 3 3 4 4 4 4 3
15381 234/300 Read block 4755 len 36864
15382 235/300 Read block 1449 len 28672
15383 236/300 Read block 2917 len 36864
15384 237/300 Write block 4545 len 20480 data: 3 3 4 4 4
15385 238/300 Flush
15386 239/300 Flush
15387 240/300 Read block 663 len 32768
15388 241/300 Read block 3193 len 36864
15389 242/300 Read block 4408 len 36864
15390 243/300 Read block 131 len 8192
15391 244/300 Read block 138 len 24576
15392 245/300 Read block 3834 len 24576
15393 246/300 Read block 1614 len 36864
15394 247/300 Read block 1246 len 36864
15395 248/300 Read block 1971 len 24576
15396 249/300 Read block 1010 len 4096
15397 250/300 Read block 1883 len 8192
15398 251/300 Write block 4586 len 12288 data: 3 4 4
15399 252/300 Read block 2428 len 24576
15400 253/300 Read block 4656 len 4096
15401 254/300 Read block 4000 len 24576
15402 255/300 Read block 1308 len 4096
15403 256/300 Write block 796 len 24576 data: 2 2 2 3 3 4
15404 257/300 Read block 3508 len 20480
15405 258/300 Write block 1258 len 24576 data: 5 5 4 3 3 4
15406 259/300 Read block 2471 len 4096
15407 260/300 Read block 1896 len 32768
15408 261/300 Write block 4788 len 12288 data: 3 4 3
15409 262/300 Write block 4819 len 28672 data: 5 3 3 4 4 4 4
15410 263/300 Read block 3640 len 16384
15411 264/300 Read block 2028 len 40960
15412 265/300 Read block 3351 len 24576
15413 266/300 Write block 4396 len 16384 data: 5 4 5 5
15414 267/300 Read block 4603 len 8192
15415 268/300 Write block 3894 len 32768 data: 4 4 4 4 3 2 2 2
15416 269/300 Write block 338 len 32768 data: 4 4 4 4 4 4 4 6
15417 270/300 Flush
15418 271/300 Read block 4503 len 16384
15419 272/300 Write block 2551 len 36864 data: 3 3 2 2 2 4 4 5 5
15420 273/300 Read block 4685 len 28672
15421 274/300 Write block 966 len 12288 data: 4 4 4
15422 275/300 Read block 2185 len 20480
15423 276/300 Read block 2783 len 20480
15424 277/300 Write block 2108 len 4096 data: 4
15425 278/300 Read block 1108 len 12288
15426 279/300 Read block 2043 len 32768
15427 280/300 Read block 3321 len 24576
15428 281/300 Write block 1555 len 16384 data: 6 5 4 4
15429 282/300 Read block 3553 len 40960
15430 283/300 Read block 3266 len 20480
15431 284/300 Read block 1996 len 36864
15432 285/300 Write block 4600 len 28672 data: 4 4 4 5 5 6 4
15433 286/300 Write block 4125 len 32768 data: 2 3 3 3 2 2 2 2
15434 287/300 Flush
15435 288/300 Write block 2951 len 20480 data: 3 4 4 3 3
15436 289/300 Write block 3972 len 28672 data: 3 2 2 3 4 4 4
15437 290/300 Write block 4 len 8192 data: 4 4
15438 291/300 Read block 4694 len 40960
15439 292/300 Flush
15440 293/300 Write block 1275 len 20480 data: 4 4 5 5 5
15441 294/300 Read block 78 len 16384
15442 295/300 Write block 4948 len 28672 data: 4 3 3 3 3 3 3
15443 296/300 Read block 304 len 12288
15444 297/300 Read block 2596 len 24576
15445 298/300 Write block 2610 len 28672 data: 5 6 5 3 3 3 3
15446 299/300 Write block 172 len 40960 data: 4 4 3 3 3 3 4 2 3 3
15447 300/300 Write block 1176 len 32768 data: 3 2 2 3 3 4 3 3
15448 301/300 Write block 4169 len 4096 data: 2
15449 Main task has work to do, go find it
15450 got dsc Start(1)
15451 start 1
15452 Nov 26 22:07:48.072 INFO request completed, latency_us: 210, response_code: 204, uri: /start/cid/1, method: POST, req_id: 9aa3d6c2-4a72-4266-bf35-3f2d39f6b008, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
15453 Replay: started 1, returned:Ok(())
15454 ----------------------------------------------------------------
15455 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
15456 States: Active Offline Active
15457 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
15458 5227 Acked Write 4 Done Sent Done false
15459 5228 Acked Read 7 Done Sent Done false
15460 5229 Acked Write 10 Done Sent Done false
15461 5230 Acked Read 8 Done Sent Done false
15462 5231 Acked Write 2 Done Sent Done false
15463 5232 Acked Write 2 Done Sent Done false
15464 5233 Acked Write 10 Done Sent Done false
15465 5234 Acked Read 1 Done Sent Done false
15466 5235 Acked Write 7 Done Sent Done false
15467 5236 Acked Read 8 Done Sent Done false
15468 5237 Acked Write 5 Done Sent Done false
15469 5238 Acked Flush 0 Done Sent Done false
15470 5239 Acked Write 6 Done Sent Done false
15471 5240 Acked Read 6 Done Sent Done false
15472 5241 Acked Read 5 Done Sent Done false
15473 5242 Acked Write 5 Done Sent Done false
15474 5243 Acked Write 8 Done Sent Done false
15475 5244 Acked Read 6 Done Sent Done false
15476 5245 Acked Write 2 Done Sent Done false
15477 5246 Acked Read 8 Done Sent Done false
15478 5247 Acked Write 10 Done Sent Done false
15479 5248 Acked Read 10 Done Sent Done false
15480 5249 Acked Write 8 Done Sent Done false
15481 5250 Acked Write 10 Done Sent Done false
15482 5251 Acked Write 9 Done Sent Done false
15483 5252 Acked Write 5 Done Sent Done false
15484 5253 Acked Read 3 Done Sent Done false
15485 5254 Acked Flush 0 Done Sent Done false
15486 5255 Acked Read 6 Done Sent Done false
15487 5256 Acked Read 1 Done Sent Done false
15488 5257 Acked Read 6 Done Sent Done false
15489 5258 Acked Write 8 Done Sent Done false
15490 5259 Acked Write 4 Done Sent Done false
15491 5260 Acked Read 10 Done Sent Done false
15492 5261 Acked Write 5 Done Sent Done false
15493 5262 Acked Read 4 Done Sent Done false
15494 5263 Acked Read 10 Done Sent Done false
15495 5264 Acked Write 5 Done Sent Done false
15496 5265 Acked Write 8 Done Sent Done false
15497 5266 Acked Read 9 Done Sent Done false
15498 5267 Acked Read 6 Done Sent Done false
15499 5268 Acked Read 3 Done Sent Done false
15500 5269 Acked Write 6 Done Sent Done false
15501 5270 Acked Write 1 Done Sent Done false
15502 5271 Acked Read 10 Done Sent Done false
15503 5272 Acked Read 2 Done Sent Done false
15504 5273 Acked Write 5 Done Sent Done false
15505 5274 Acked Read 2 Done Sent Done false
15506 5275 Acked Write 8 Done Sent Done false
15507 5276 Acked Read 5 Done Sent Done false
15508 5277 Acked Read 6 Done Sent Done false
15509 5278 Acked Read 2 Done Sent Done false
15510 5279 Acked Read 2 Done Sent Done false
15511 5280 Acked Write 6 Done Sent Done false
15512 5281 Acked Flush 0 Done Sent Done false
15513 5282 Acked Read 6 Done Sent Done false
15514 5283 Acked Write 7 Done Sent Done false
15515 5284 Acked Write 7 Done Sent Done false
15516 5285 Acked Read 9 Done Sent Done false
15517 5286 Acked Read 8 Done Sent Done false
15518 5287 Acked Write 6 Done Sent Done false
15519 5288 Acked Flush 0 Done Sent Done false
15520 5289 Acked Read 9 Done Sent Done false
15521 5290 Acked Read 5 Done Sent Done false
15522 5291 Acked Read 3 Done Sent Done false
15523 5292 Acked Flush 0 Done Sent Done false
15524 5293 Acked Read 3 Done Sent Done false
15525 5294 Acked Write 2 Done Sent Done false
15526 5295 Acked Read 10 Done Sent Done false
15527 5296 Acked Write 5 Done Sent Done false
15528 5297 Acked Read 5 Done Sent Done false
15529 5298 Acked Read 1 Done Sent Done false
15530 5299 Acked Write 7 Done Sent Done false
15531 5300 Acked Read 10 Done Sent Done false
15532 5301 Acked Write 10 Done Sent Done false
15533 5302 Acked Read 9 Done Sent Done false
15534 5303 Acked Write 9 Done Sent Done false
15535 5304 Acked Write 9 Done Sent Done false
15536 5305 Acked Write 3 Done Sent Done false
15537 5306 Acked Read 9 Done Sent Done false
15538 5307 Acked Read 2 Done Sent Done false
15539 5308 Acked Read 7 Done Sent Done false
15540 5309 Acked Write 5 Done Sent Done false
15541 5310 Acked Write 6 Done Sent Done false
15542 5311 Acked Read 10 Done Sent Done false
15543 5312 Acked Flush 0 Done Sent Done false
15544 5313 Acked Read 10 Done Sent Done false
15545 5314 Acked Read 1 Done Sent Done false
15546 5315 Acked Read 3 Done Sent Done false
15547 5316 Acked Flush 0 Done Sent Done false
15548 5317 Acked Read 2 Done Sent Done false
15549 5318 Acked Read 7 Done Sent Done false
15550 5319 Acked Write 1 Done Sent Done false
15551 5320 Acked Read 4 Done Sent Done false
15552 5321 Acked Read 1 Done Sent Done false
15553 5322 Acked Read 6 Done Sent Done false
15554 5323 Acked Read 5 Done Sent Done false
15555 5324 Acked Read 3 Done Sent Done false
15556 5325 Acked Write 9 Done Sent Done false
15557 5326 Acked Read 4 Done Sent Done false
15558 5327 Acked Write 8 Done Sent Done false
15559 5328 Acked Write 1 Done Sent Done false
15560 5329 Acked Write 6 Done Sent Done false
15561 5330 Acked Write 2 Done Sent Done false
15562 5331 Acked Write 1 Done Sent Done false
15563 5332 Acked Write 2 Done Sent Done false
15564 5333 Acked Read 3 Done Sent Done false
15565 5334 Acked Write 10 Done Sent Done false
15566 5335 Acked Read 1 Done Sent Done false
15567 5336 Acked Write 9 Done Sent Done false
15568 5337 Acked Write 10 Done Sent Done false
15569 5338 Acked Read 1 Done Sent Done false
15570 5339 Acked Read 5 Done Sent Done false
15571 5340 Acked Write 5 Done Sent Done false
15572 5341 Acked Read 9 Done Sent Done false
15573 5342 Acked Flush 0 Done Sent Done false
15574 5343 Acked Read 3 Done Sent Done false
15575 5344 Acked Read 5 Done Sent Done false
15576 5345 Acked Read 2 Done Sent Done false
15577 5346 Acked Read 10 Done Sent Done false
15578 5347 Acked Read 1 Done Sent Done false
15579 5348 Acked Read 1 Done Sent Done false
15580 5349 Acked Write 9 Done Sent Done false
15581 5350 Acked Read 4 Done Sent Done false
15582 5351 Acked Write 6 Done Sent Done false
15583 5352 Acked Write 1 Done Sent Done false
15584 5353 Acked Write 10 Done Sent Done false
15585 5354 Acked Write 10 Done Sent Done false
15586 5355 Acked Read 10 Done Sent Done false
15587 5356 Acked Read 1 Done Sent Done false
15588 5357 Acked Read 10 Done Sent Done false
15589 5358 Acked Read 4 Done Sent Done false
15590 5359 Acked Read 4 Done Sent Done false
15591 5360 Acked Flush 0 Done Sent Done false
15592 5361 Acked Write 8 Done Sent Done false
15593 5362 Acked Read 6 Done Sent Done false
15594 5363 Acked Read 7 Done Sent Done false
15595 5364 Acked Write 10 Done Sent Done false
15596 5365 Acked Read 6 Done Sent Done false
15597 5366 Acked Read 6 Done Sent Done false
15598 5367 Acked Read 9 Done Sent Done false
15599 5368 Acked Write 7 Done Sent Done false
15600 5369 Acked Write 3 Done Sent Done false
15601 5370 Acked Read 7 Done Sent Done false
15602 5371 Acked Read 7 Done Sent Done false
15603 5372 Acked Read 4 Done Sent Done false
15604 5373 Acked Write 6 Done Sent Done false
15605 5374 Acked Read 4 Done Sent Done false
15606 5375 Acked Read 7 Done Sent Done false
15607 5376 Acked Read 6 Done Sent Done false
15608 5377 Acked Read 9 Done Sent Done false
15609 5378 Acked Flush 0 Done Sent Done false
15610 5379 Acked Flush 0 Done Sent Done false
15611 5380 Acked Write 6 Done Sent Done false
15612 5381 Acked Read 5 Done Sent Done false
15613 5382 Acked Write 1 Done Sent Done false
15614 5383 Acked Read 1 Done Sent Done false
15615 5384 Acked Write 5 Done Sent Done false
15616 5385 Acked Read 2 Done Sent Done false
15617 5386 Acked Read 7 Done Sent Done false
15618 5387 Acked Write 5 Done Sent Done false
15619 5388 Acked Read 5 Done Sent Done false
15620 5389 Acked Write 8 Done Sent Done false
15621 5390 Acked Read 10 Done Sent Done false
15622 5391 Acked Flush 0 Done Sent Done false
15623 5392 Acked Write 8 Done Sent Done false
15624 5393 Acked Flush 0 Done Sent Done false
15625 5394 Acked Write 4 Done Sent Done false
15626 5395 Acked Read 6 Done Sent Done false
15627 5396 Acked Flush 0 Done Sent Done false
15628 5397 Acked Flush 0 Done Sent Done false
15629 5398 Acked Write 3 Done Sent Done false
15630 5399 Acked Write 8 Done Sent Done false
15631 5400 Acked Read 6 Done Sent Done false
15632 5401 Acked Read 10 Done Sent Done false
15633 5402 Acked Flush 0 Done Sent Done false
15634 5403 Acked Read 3 Done Sent Done false
15635 5404 Acked Read 5 Done Sent Done false
15636 5405 Acked Read 7 Done Sent Done false
15637 5406 Acked Read 7 Done Sent Done false
15638 5407 Acked Read 4 Done Sent Done false
15639 5408 Acked Read 4 Done Sent Done false
15640 5409 Acked Read 10 Done Sent Done false
15641 5410 Acked Write 6 Done Sent Done false
15642 5411 Acked Read 1 Done Sent Done false
15643 5412 Acked Flush 0 Done Sent Done false
15644 5413 Acked Write 8 Done Sent Done false
15645 5414 Acked Write 7 Done Sent Done false
15646 5415 Acked Write 6 Done Sent Done false
15647 5416 Acked Read 3 Done Sent Done false
15648 5417 Acked Read 9 Done Sent Done false
15649 5418 Acked Write 9 Done Sent Done false
15650 5419 Acked Read 10 Done Sent Done false
15651 5420 Acked Write 7 Done Sent Done false
15652 5421 Acked Write 6 Done Sent Done false
15653 5422 Acked Read 8 Done Sent Done false
15654 5423 Acked Read 4 Done Sent Done false
15655 5424 Acked Flush 0 Done Sent Done false
15656 5425 Acked Read 3 Done Sent Done false
15657 5426 Acked Read 5 Done Sent Done false
15658 5427 Acked Read 5 Done Sent Done false
15659 5428 Acked Flush 0 Done Sent Done false
15660 5429 Acked Write 6 Done Sent Done false
15661 5430 Acked Write 1 Done Sent Done false
15662 5431 Acked Flush 0 Done Sent Done false
15663 5432 Acked Write 2 Done Sent Done false
15664 5433 Acked Flush 0 Done Sent Done false
15665 5434 Acked Write 9 Done Sent Done false
15666 5435 Acked Read 8 Done Sent Done false
15667 5436 Acked Flush 0 Done Sent Done false
15668 5437 Acked Read 7 Done Sent Done false
15669 5438 Acked Write 8 Done Sent Done false
15670 5439 Acked Read 1 Done Sent Done false
15671 5440 Acked Read 7 Done Sent Done false
15672 5441 Acked Read 8 Done Sent Done false
15673 5442 Acked Read 3 Done Sent Done false
15674 5443 Acked Write 4 Done Sent Done false
15675 5444 Acked Read 10 Done Sent Done false
15676 5445 Acked Flush 0 Done Sent Done false
15677 5446 Acked Write 6 Done Sent Done false
15678 5447 Acked Write 8 Done Sent Done false
15679 5448 Acked Read 6 Done Sent Done false
15680 5449 Acked Write 3 Done Sent Done false
15681 5450 Acked Read 10 Done Sent Done false
15682 5451 Acked Write 1 Done Sent Done false
15683 5452 Acked Write 5 Done Sent Done false
15684 5453 Acked Flush 0 Done Sent Done false
15685 5454 Acked Read 8 Done Sent Done false
15686 5455 Acked Read 8 Done Sent Done false
15687 5456 Acked Read 4 Done Sent Done false
15688 5457 Acked Write 1 Done Sent Done false
15689 5458 Acked Write 6 Done Sent Done false
15690 5459 Acked Write 7 Done Sent Done false
15691 5460 Acked Read 9 Done Sent Done false
15692 5461 Acked Read 7 Done Sent Done false
15693 5462 Acked Read 9 Done Sent Done false
15694 5463 Acked Write 5 Done Sent Done false
15695 5464 Acked Flush 0 Done Sent Done false
15696 5465 Acked Flush 0 Done Sent Done false
15697 5466 Acked Read 8 Done Sent Done false
15698 5467 Acked Read 9 Done Sent Done false
15699 5468 Acked Read 9 Done Sent Done false
15700 5469 Acked Read 2 Done Sent Done false
15701 5470 Acked Read 6 Done Sent Done false
15702 5471 Acked Read 6 Done Sent Done false
15703 5472 Acked Read 9 Done Sent Done false
15704 5473 Acked Read 9 Done Sent Done false
15705 5474 Acked Read 6 Done Sent Done false
15706 5475 Acked Read 1 Done Sent Done false
15707 5476 Acked Read 2 Done Sent Done false
15708 5477 Acked Write 3 Done Sent Done false
15709 5478 Acked Read 6 Done Sent Done false
15710 5479 Acked Read 1 Done Sent Done false
15711 5480 Acked Read 6 Done Sent Done false
15712 5481 Acked Read 1 Done Sent Done false
15713 5482 Acked Write 6 Done Sent Done false
15714 5483 Acked Read 5 Done Sent Done false
15715 5484 Acked Write 6 Done Sent Done false
15716 5485 Acked Read 1 Done Sent Done false
15717 5486 Acked Read 8 Done Sent Done false
15718 5487 Acked Write 3 Done Sent Done false
15719 5488 Acked Write 7 Done Sent Done false
15720 5489 Acked Read 4 Done Sent Done false
15721 5490 Acked Read 10 Done Sent Done false
15722 5491 Acked Read 6 Done Sent Done false
15723 5492 Acked Write 4 Done Sent Done false
15724 5493 Acked Read 2 Done Sent Done false
15725 5494 Acked Write 8 Done Sent Done false
15726 5495 Acked Write 8 Done Sent Done false
15727 5496 Acked Flush 0 Done Sent Done false
15728 5497 Acked Read 4 Done Sent Done false
15729 5498 Acked Write 9 Done Sent Done false
15730 5499 Acked Read 7 Done Sent Done false
15731 5500 Acked Write 3 Done Sent Done false
15732 5501 Acked Read 5 Done Sent Done false
15733 5502 Acked Read 5 Done Sent Done false
15734 5503 Acked Write 1 Done Sent Done false
15735 5504 Acked Read 3 Done Sent Done false
15736 5505 Acked Read 8 Done Sent Done false
15737 5506 Acked Read 6 Done Sent Done false
15738 5507 Acked Write 4 Done Sent Done false
15739 5508 Acked Read 10 Done Sent Done false
15740 5509 Acked Read 5 Done Sent Done false
15741 5510 Acked Read 9 Done Sent Done false
15742 5511 Acked Write 7 Done Sent Done false
15743 5512 Acked Write 8 Done Sent Done false
15744 5513 Acked Flush 0 Done Sent Done false
15745 5514 Acked Write 5 Done Sent Done false
15746 5515 Acked Write 7 Done Sent Done false
15747 5516 Acked Write 2 Done Sent Done false
15748 5517 Acked Read 10 Done Sent Done false
15749 5518 Acked Flush 0 Done Sent Done false
15750 5519 Acked Write 5 Done Sent Done false
15751 5520 Acked Read 4 Done Sent Done false
15752 5521 Acked Write 7 Done Sent Done false
15753 5522 Acked Read 3 Done Sent Done false
15754 5523 Acked Read 6 Done Sent Done false
15755 5524 Acked Write 7 Done Sent Done false
15756 5525 Acked Write 10 Done Sent Done false
15757 5526 Acked Write 8 Done Sent Sent false
15758 5527 Acked Write 1 Sent Sent Sent false
15759 STATES DS:0 DS:1 DS:2 TOTAL
15760 Sent 1 301 2 304
15761 Done 300 0 299 599
15762 Skipped 0 0 0 0
15763 Error 0 0 0 0
15764 Last Flush: 5518 5226 5518
15765 Downstairs last five completed: 5226 5225 5224 5223 5222
15766 Upstairs last five completed: 5527 5526 5525 5524 5523
15767 CLIENT: Up:0 ds:301 act:2
15768 [8820] got start action
15769 [8820] I am going to restart
15770 Starting downstairs at port 8820
15771 Make output file at "/tmp/dsc/downstairs-8820.txt"
15772 [8820][1] reports Starting
15773 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1218)
15774 [8820][1] reports Running
15775 ----------------------------------------------------------------
15776 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
15777 States: Active Offline Active
15778 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
15779 5227 Acked Write 4 Done Sent Done false
15780 5228 Acked Read 7 Done Sent Done false
15781 5229 Acked Write 10 Done Sent Done false
15782 5230 Acked Read 8 Done Sent Done false
15783 5231 Acked Write 2 Done Sent Done false
15784 5232 Acked Write 2 Done Sent Done false
15785 5233 Acked Write 10 Done Sent Done false
15786 5234 Acked Read 1 Done Sent Done false
15787 5235 Acked Write 7 Done Sent Done false
15788 5236 Acked Read 8 Done Sent Done false
15789 5237 Acked Write 5 Done Sent Done false
15790 5238 Acked Flush 0 Done Sent Done false
15791 5239 Acked Write 6 Done Sent Done false
15792 5240 Acked Read 6 Done Sent Done false
15793 5241 Acked Read 5 Done Sent Done false
15794 5242 Acked Write 5 Done Sent Done false
15795 5243 Acked Write 8 Done Sent Done false
15796 5244 Acked Read 6 Done Sent Done false
15797 5245 Acked Write 2 Done Sent Done false
15798 5246 Acked Read 8 Done Sent Done false
15799 5247 Acked Write 10 Done Sent Done false
15800 5248 Acked Read 10 Done Sent Done false
15801 5249 Acked Write 8 Done Sent Done false
15802 5250 Acked Write 10 Done Sent Done false
15803 5251 Acked Write 9 Done Sent Done false
15804 5252 Acked Write 5 Done Sent Done false
15805 5253 Acked Read 3 Done Sent Done false
15806 5254 Acked Flush 0 Done Sent Done false
15807 5255 Acked Read 6 Done Sent Done false
15808 5256 Acked Read 1 Done Sent Done false
15809 5257 Acked Read 6 Done Sent Done false
15810 5258 Acked Write 8 Done Sent Done false
15811 5259 Acked Write 4 Done Sent Done false
15812 5260 Acked Read 10 Done Sent Done false
15813 5261 Acked Write 5 Done Sent Done false
15814 5262 Acked Read 4 Done Sent Done false
15815 5263 Acked Read 10 Done Sent Done false
15816 5264 Acked Write 5 Done Sent Done false
15817 5265 Acked Write 8 Done Sent Done false
15818 5266 Acked Read 9 Done Sent Done false
15819 5267 Acked Read 6 Done Sent Done false
15820 5268 Acked Read 3 Done Sent Done false
15821 5269 Acked Write 6 Done Sent Done false
15822 5270 Acked Write 1 Done Sent Done false
15823 5271 Acked Read 10 Done Sent Done false
15824 5272 Acked Read 2 Done Sent Done false
15825 5273 Acked Write 5 Done Sent Done false
15826 5274 Acked Read 2 Done Sent Done false
15827 5275 Acked Write 8 Done Sent Done false
15828 5276 Acked Read 5 Done Sent Done false
15829 5277 Acked Read 6 Done Sent Done false
15830 5278 Acked Read 2 Done Sent Done false
15831 5279 Acked Read 2 Done Sent Done false
15832 5280 Acked Write 6 Done Sent Done false
15833 5281 Acked Flush 0 Done Sent Done false
15834 5282 Acked Read 6 Done Sent Done false
15835 5283 Acked Write 7 Done Sent Done false
15836 5284 Acked Write 7 Done Sent Done false
15837 5285 Acked Read 9 Done Sent Done false
15838 5286 Acked Read 8 Done Sent Done false
15839 5287 Acked Write 6 Done Sent Done false
15840 5288 Acked Flush 0 Done Sent Done false
15841 5289 Acked Read 9 Done Sent Done false
15842 5290 Acked Read 5 Done Sent Done false
15843 5291 Acked Read 3 Done Sent Done false
15844 5292 Acked Flush 0 Done Sent Done false
15845 5293 Acked Read 3 Done Sent Done false
15846 5294 Acked Write 2 Done Sent Done false
15847 5295 Acked Read 10 Done Sent Done false
15848 5296 Acked Write 5 Done Sent Done false
15849 5297 Acked Read 5 Done Sent Done false
15850 5298 Acked Read 1 Done Sent Done false
15851 5299 Acked Write 7 Done Sent Done false
15852 5300 Acked Read 10 Done Sent Done false
15853 5301 Acked Write 10 Done Sent Done false
15854 5302 Acked Read 9 Done Sent Done false
15855 5303 Acked Write 9 Done Sent Done false
15856 5304 Acked Write 9 Done Sent Done false
15857 5305 Acked Write 3 Done Sent Done false
15858 5306 Acked Read 9 Done Sent Done false
15859 5307 Acked Read 2 Done Sent Done false
15860 5308 Acked Read 7 Done Sent Done false
15861 5309 Acked Write 5 Done Sent Done false
15862 5310 Acked Write 6 Done Sent Done false
15863 5311 Acked Read 10 Done Sent Done false
15864 5312 Acked Flush 0 Done Sent Done false
15865 5313 Acked Read 10 Done Sent Done false
15866 5314 Acked Read 1 Done Sent Done false
15867 5315 Acked Read 3 Done Sent Done false
15868 5316 Acked Flush 0 Done Sent Done false
15869 5317 Acked Read 2 Done Sent Done false
15870 5318 Acked Read 7 Done Sent Done false
15871 5319 Acked Write 1 Done Sent Done false
15872 5320 Acked Read 4 Done Sent Done false
15873 5321 Acked Read 1 Done Sent Done false
15874 5322 Acked Read 6 Done Sent Done false
15875 5323 Acked Read 5 Done Sent Done false
15876 5324 Acked Read 3 Done Sent Done false
15877 5325 Acked Write 9 Done Sent Done false
15878 5326 Acked Read 4 Done Sent Done false
15879 5327 Acked Write 8 Done Sent Done false
15880 5328 Acked Write 1 Done Sent Done false
15881 5329 Acked Write 6 Done Sent Done false
15882 5330 Acked Write 2 Done Sent Done false
15883 5331 Acked Write 1 Done Sent Done false
15884 5332 Acked Write 2 Done Sent Done false
15885 5333 Acked Read 3 Done Sent Done false
15886 5334 Acked Write 10 Done Sent Done false
15887 5335 Acked Read 1 Done Sent Done false
15888 5336 Acked Write 9 Done Sent Done false
15889 5337 Acked Write 10 Done Sent Done false
15890 5338 Acked Read 1 Done Sent Done false
15891 5339 Acked Read 5 Done Sent Done false
15892 5340 Acked Write 5 Done Sent Done false
15893 5341 Acked Read 9 Done Sent Done false
15894 5342 Acked Flush 0 Done Sent Done false
15895 5343 Acked Read 3 Done Sent Done false
15896 5344 Acked Read 5 Done Sent Done false
15897 5345 Acked Read 2 Done Sent Done false
15898 5346 Acked Read 10 Done Sent Done false
15899 5347 Acked Read 1 Done Sent Done false
15900 5348 Acked Read 1 Done Sent Done false
15901 5349 Acked Write 9 Done Sent Done false
15902 5350 Acked Read 4 Done Sent Done false
15903 5351 Acked Write 6 Done Sent Done false
15904 5352 Acked Write 1 Done Sent Done false
15905 5353 Acked Write 10 Done Sent Done false
15906 5354 Acked Write 10 Done Sent Done false
15907 5355 Acked Read 10 Done Sent Done false
15908 5356 Acked Read 1 Done Sent Done false
15909 5357 Acked Read 10 Done Sent Done false
15910 5358 Acked Read 4 Done Sent Done false
15911 5359 Acked Read 4 Done Sent Done false
15912 5360 Acked Flush 0 Done Sent Done false
15913 5361 Acked Write 8 Done Sent Done false
15914 5362 Acked Read 6 Done Sent Done false
15915 5363 Acked Read 7 Done Sent Done false
15916 5364 Acked Write 10 Done Sent Done false
15917 5365 Acked Read 6 Done Sent Done false
15918 5366 Acked Read 6 Done Sent Done false
15919 5367 Acked Read 9 Done Sent Done false
15920 5368 Acked Write 7 Done Sent Done false
15921 5369 Acked Write 3 Done Sent Done false
15922 5370 Acked Read 7 Done Sent Done false
15923 5371 Acked Read 7 Done Sent Done false
15924 5372 Acked Read 4 Done Sent Done false
15925 5373 Acked Write 6 Done Sent Done false
15926 5374 Acked Read 4 Done Sent Done false
15927 5375 Acked Read 7 Done Sent Done false
15928 5376 Acked Read 6 Done Sent Done false
15929 5377 Acked Read 9 Done Sent Done false
15930 5378 Acked Flush 0 Done Sent Done false
15931 5379 Acked Flush 0 Done Sent Done false
15932 5380 Acked Write 6 Done Sent Done false
15933 5381 Acked Read 5 Done Sent Done false
15934 5382 Acked Write 1 Done Sent Done false
15935 5383 Acked Read 1 Done Sent Done false
15936 5384 Acked Write 5 Done Sent Done false
15937 5385 Acked Read 2 Done Sent Done false
15938 5386 Acked Read 7 Done Sent Done false
15939 5387 Acked Write 5 Done Sent Done false
15940 5388 Acked Read 5 Done Sent Done false
15941 5389 Acked Write 8 Done Sent Done false
15942 5390 Acked Read 10 Done Sent Done false
15943 5391 Acked Flush 0 Done Sent Done false
15944 5392 Acked Write 8 Done Sent Done false
15945 5393 Acked Flush 0 Done Sent Done false
15946 5394 Acked Write 4 Done Sent Done false
15947 5395 Acked Read 6 Done Sent Done false
15948 5396 Acked Flush 0 Done Sent Done false
15949 5397 Acked Flush 0 Done Sent Done false
15950 5398 Acked Write 3 Done Sent Done false
15951 5399 Acked Write 8 Done Sent Done false
15952 5400 Acked Read 6 Done Sent Done false
15953 5401 Acked Read 10 Done Sent Done false
15954 5402 Acked Flush 0 Done Sent Done false
15955 5403 Acked Read 3 Done Sent Done false
15956 5404 Acked Read 5 Done Sent Done false
15957 5405 Acked Read 7 Done Sent Done false
15958 5406 Acked Read 7 Done Sent Done false
15959 5407 Acked Read 4 Done Sent Done false
15960 5408 Acked Read 4 Done Sent Done false
15961 5409 Acked Read 10 Done Sent Done false
15962 5410 Acked Write 6 Done Sent Done false
15963 5411 Acked Read 1 Done Sent Done false
15964 5412 Acked Flush 0 Done Sent Done false
15965 5413 Acked Write 8 Done Sent Done false
15966 5414 Acked Write 7 Done Sent Done false
15967 5415 Acked Write 6 Done Sent Done false
15968 5416 Acked Read 3 Done Sent Done false
15969 5417 Acked Read 9 Done Sent Done false
15970 5418 Acked Write 9 Done Sent Done false
15971 5419 Acked Read 10 Done Sent Done false
15972 5420 Acked Write 7 Done Sent Done false
15973 5421 Acked Write 6 Done Sent Done false
15974 5422 Acked Read 8 Done Sent Done false
15975 5423 Acked Read 4 Done Sent Done false
15976 5424 Acked Flush 0 Done Sent Done false
15977 5425 Acked Read 3 Done Sent Done false
15978 5426 Acked Read 5 Done Sent Done false
15979 5427 Acked Read 5 Done Sent Done false
15980 5428 Acked Flush 0 Done Sent Done false
15981 5429 Acked Write 6 Done Sent Done false
15982 5430 Acked Write 1 Done Sent Done false
15983 5431 Acked Flush 0 Done Sent Done false
15984 5432 Acked Write 2 Done Sent Done false
15985 5433 Acked Flush 0 Done Sent Done false
15986 5434 Acked Write 9 Done Sent Done false
15987 5435 Acked Read 8 Done Sent Done false
15988 5436 Acked Flush 0 Done Sent Done false
15989 5437 Acked Read 7 Done Sent Done false
15990 5438 Acked Write 8 Done Sent Done false
15991 5439 Acked Read 1 Done Sent Done false
15992 5440 Acked Read 7 Done Sent Done false
15993 5441 Acked Read 8 Done Sent Done false
15994 5442 Acked Read 3 Done Sent Done false
15995 5443 Acked Write 4 Done Sent Done false
15996 5444 Acked Read 10 Done Sent Done false
15997 5445 Acked Flush 0 Done Sent Done false
15998 5446 Acked Write 6 Done Sent Done false
15999 5447 Acked Write 8 Done Sent Done false
16000 5448 Acked Read 6 Done Sent Done false
16001 5449 Acked Write 3 Done Sent Done false
16002 5450 Acked Read 10 Done Sent Done false
16003 5451 Acked Write 1 Done Sent Done false
16004 5452 Acked Write 5 Done Sent Done false
16005 5453 Acked Flush 0 Done Sent Done false
16006 5454 Acked Read 8 Done Sent Done false
16007 5455 Acked Read 8 Done Sent Done false
16008 5456 Acked Read 4 Done Sent Done false
16009 5457 Acked Write 1 Done Sent Done false
16010 5458 Acked Write 6 Done Sent Done false
16011 5459 Acked Write 7 Done Sent Done false
16012 5460 Acked Read 9 Done Sent Done false
16013 5461 Acked Read 7 Done Sent Done false
16014 5462 Acked Read 9 Done Sent Done false
16015 5463 Acked Write 5 Done Sent Done false
16016 5464 Acked Flush 0 Done Sent Done false
16017 5465 Acked Flush 0 Done Sent Done false
16018 5466 Acked Read 8 Done Sent Done false
16019 5467 Acked Read 9 Done Sent Done false
16020 5468 Acked Read 9 Done Sent Done false
16021 5469 Acked Read 2 Done Sent Done false
16022 5470 Acked Read 6 Done Sent Done false
16023 5471 Acked Read 6 Done Sent Done false
16024 5472 Acked Read 9 Done Sent Done false
16025 5473 Acked Read 9 Done Sent Done false
16026 5474 Acked Read 6 Done Sent Done false
16027 5475 Acked Read 1 Done Sent Done false
16028 5476 Acked Read 2 Done Sent Done false
16029 5477 Acked Write 3 Done Sent Done false
16030 5478 Acked Read 6 Done Sent Done false
16031 5479 Acked Read 1 Done Sent Done false
16032 5480 Acked Read 6 Done Sent Done false
16033 5481 Acked Read 1 Done Sent Done false
16034 5482 Acked Write 6 Done Sent Done false
16035 5483 Acked Read 5 Done Sent Done false
16036 5484 Acked Write 6 Done Sent Done false
16037 5485 Acked Read 1 Done Sent Done false
16038 5486 Acked Read 8 Done Sent Done false
16039 5487 Acked Write 3 Done Sent Done false
16040 5488 Acked Write 7 Done Sent Done false
16041 5489 Acked Read 4 Done Sent Done false
16042 5490 Acked Read 10 Done Sent Done false
16043 5491 Acked Read 6 Done Sent Done false
16044 5492 Acked Write 4 Done Sent Done false
16045 5493 Acked Read 2 Done Sent Done false
16046 5494 Acked Write 8 Done Sent Done false
16047 5495 Acked Write 8 Done Sent Done false
16048 5496 Acked Flush 0 Done Sent Done false
16049 5497 Acked Read 4 Done Sent Done false
16050 5498 Acked Write 9 Done Sent Done false
16051 5499 Acked Read 7 Done Sent Done false
16052 5500 Acked Write 3 Done Sent Done false
16053 5501 Acked Read 5 Done Sent Done false
16054 5502 Acked Read 5 Done Sent Done false
16055 5503 Acked Write 1 Done Sent Done false
16056 5504 Acked Read 3 Done Sent Done false
16057 5505 Acked Read 8 Done Sent Done false
16058 5506 Acked Read 6 Done Sent Done false
16059 5507 Acked Write 4 Done Sent Done false
16060 5508 Acked Read 10 Done Sent Done false
16061 5509 Acked Read 5 Done Sent Done false
16062 5510 Acked Read 9 Done Sent Done false
16063 5511 Acked Write 7 Done Sent Done false
16064 5512 Acked Write 8 Done Sent Done false
16065 5513 Acked Flush 0 Done Sent Done false
16066 5514 Acked Write 5 Done Sent Done false
16067 5515 Acked Write 7 Done Sent Done false
16068 5516 Acked Write 2 Done Sent Done false
16069 5517 Acked Read 10 Done Sent Done false
16070 5518 Acked Flush 0 Done Sent Done false
16071 5519 Acked Write 5 Done Sent Done false
16072 5520 Acked Read 4 Done Sent Done false
16073 5521 Acked Write 7 Done Sent Done false
16074 5522 Acked Read 3 Done Sent Done false
16075 5523 Acked Read 6 Done Sent Done false
16076 5524 Acked Write 7 Done Sent Done false
16077 5525 Acked Write 10 Done Sent Done false
16078 5526 Acked Write 8 Done Sent Done false
16079 5527 Acked Write 1 Done Sent Done false
16080 5528 Acked Flush 0 Done Sent Done false
16081 STATES DS:0 DS:1 DS:2 TOTAL
16082 Sent 0 302 0 302
16083 Done 302 0 302 604
16084 Skipped 0 0 0 0
16085 Error 0 0 0 0
16086 Last Flush: 5528 5226 5528
16087 Downstairs last five completed: 5226 5225 5224 5223 5222
16088 Upstairs last five completed: 5528 5527 5526 5525 5524
16089 CLIENT: Up:0 ds:302 act:2
160902024-11-26T22:07:53.826ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
160912024-11-26T22:07:53.826ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:38900) = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
160922024-11-26T22:07:53.827ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
160932024-11-26T22:07:53.827ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
160942024-11-26T22:07:53.827ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
160952024-11-26T22:07:53.827ZINFOcrucible: send last flush ID to this DS: 5226 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
160962024-11-26T22:07:53.828ZINFOcrucible: Replied this last flush ID: 5226 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
160972024-11-26T22:07:53.828ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
160982024-11-26T22:07:53.828ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
160992024-11-26T22:07:53.828ZINFOcrucible: [1] client re-new 302 jobs since flush 5226 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
161002024-11-26T22:07:53.828ZINFOcrucible: [1] Replayed 302 jobs since flush: 5226 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
16101 ----------------------------------------------------------------
16102 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
16103 Downstairs last five completed: 5528 5527 5526 5525 5524
16104 Upstairs last five completed: 5528 5527 5526 5525 5524
16105 CLIENT: Up:0 ds:0 act:3
16106 Replay: All jobs finished
16107 Main task has work to do, go find it
16108 got dsc Stop(1)
16109 stop 1
16110 Nov 26 22:07:56.077 INFO request completed, latency_us: 371, [8820] Got stop action so:false kr:false
16111 response_code: 204, uri: /stop/cid/1, method: POST, req_id: b782b1bc-7c79-45ab-8a0b-6c51f86df35a, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
161122024-11-26T22:07:56.078ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
161132024-11-26T22:07:56.078ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
161142024-11-26T22:07:56.078ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
161152024-11-26T22:07:56.078ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
161162024-11-26T22:07:56.078ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
16117 Nov 26 22:07:56.078 INFO request completed, latency_us: 142, response_code: 200, uri{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2024-11-26T22:07:56.078501471Z","hostname":"w-01JDN7ZQ17ABQG15KHKN3F2D26","pid":1203,"":"io task","client":"1","":"downstairs","session_id":"86a36c17-3718-484c-87fd-2b8a1dc2ab18"}
16118 : /state/cid/1, method: GET, req_id: 80203d54-0c09-4ecc-8267-0cb5ad3e9112, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
16119 [8820] Exited with: ExitStatus(unix_wait_status(9))
16120 [8820][1] reports Exit
16121 Nov 26 22:08:00.080 INFO request completed, latency_us: 199, response_code: 200, uri: /state/cid/1, method: GET, req_id: d1a7b60d-5340-4cf1-963d-02df586cf032, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
16122 001/300 Read block 2762 len 36864
16123 002/300 Write block 3351 len 36864 data: 5 4 4 3 3 2 2 3 3
16124 003/300 Read block 3249 len 24576
16125 004/300 Read block 1710 len 12288
16126 005/300 Read block 4357 len 4096
16127 006/300 Read block 4839 len 24576
16128 007/300 Write block 1211 len 8192 data: 6 5
16129 008/300 Flush
16130 009/300 Read block 4385 len 4096
16131 010/300 Write block 4343 len 20480 data: 7 7 6 4 5
16132 011/300 Flush
16133 012/300 Write block 159 len 20480 data: 4 5 5 3 4
16134 013/300 Read block 1639 len 16384
16135 014/300 Flush
16136 015/300 Read block 4412 len 12288
16137 016/300 Write block 2690 len 40960 data: 4 4 4 4 3 3 3 3 2 2
16138 017/300 Write block 3274 len 28672 data: 4 4 3 2 2 3 4
16139 018/300 Read block 135 len 28672
16140 019/300 Write block 1252 len 24576 data: 2 2 2 3 4 4
16141 020/300 Write block 4268 len 16384 data: 4 3 3 3
16142 021/300 Write block 4306 len 28672 data: 3 3 3 3 2 3 4
16143 022/300 Write block 2011 len 8192 data: 2 2
16144 023/300 Read block 2705 len 24576
16145 024/300 Write block 988 len 36864 data: 3 3 3 2 2 2 2 2 2
16146 025/300 Read block 2431 len 28672
16147 026/300 Write block 493 len 32768 data: 3 3 3 4 4 3 3 3
16148 027/300 Write block 4343 len 24576 data: 8 8 7 5 6 6
16149 028/300 Read block 4402 len 36864
16150 029/300 Write block 243 len 28672 data: 4 3 2 4 4 3 3
16151 030/300 Write block 3537 len 32768 data: 3 2 3 5 5 5 4 3
16152 031/300 Read block 3094 len 8192
16153 032/300 Write block 1335 len 12288 data: 3 2 2
16154 033/300 Write block 2741 len 36864 data: 4 4 5 5 5 5 6 6 5
16155 034/300 Read block 766 len 4096
16156 035/300 Flush
16157 036/300 Read block 1023 len 4096
16158 037/300 Flush
16159 038/300 Read block 4580 len 32768
16160 039/300 Read block 2510 len 20480
16161 040/300 Read block 3891 len 8192
16162 041/300 Write block 1455 len 40960 data: 3 3 3 2 2 2 2 3 3 3
16163 042/300 Read block 3203 len 32768
16164 043/300 Read block 4583 len 24576
16165 044/300 Write block 525 len 28672 data: 5 3 3 3 3 2 2
16166 045/300 Read block 1530 len 16384
16167 046/300 Read block 2307 len 20480
16168 047/300 Read block 598 len 8192
16169 048/300 Write block 4677 len 36864 data: 6 6 6 8 7 5 4 4 3
16170 049/300 Write block 717 len 24576 data: 4 4 4 4 4 5
16171 050/300 Read block 388 len 8192
16172 051/300 Write block 1542 len 12288 data: 6 5 4
16173 052/300 Write block 2212 len 32768 data: 3 3 3 3 3 3 2 3
16174 053/300 Flush
16175 054/300 Flush
16176 055/300 Write block 4343 len 4096 data: 9
16177 056/300 Write block 2934 len 8192 data: 4 4
16178 057/300 Write block 3673 len 24576 data: 10 9 9 10 10 9
16179 058/300 Read block 2755 len 12288
16180 059/300 Write block 3782 len 24576 data: 5 4 5 4 3 3
16181 060/300 Flush
16182 061/300 Read block 11 len 24576
16183 062/300 Write block 4322 len 4096 data: 3
16184 063/300 Write block 2065 len 32768 data: 8 6 6 6 5 5 4 3
16185 064/300 Write block 2401 len 32768 data: 6 5 5 5 6 4 4 3
16186 065/300 Flush
16187 066/300 Write block 4617 len 12288 data: 4 5 6
16188 067/300 Read block 786 len 32768
16189 068/300 Write block 1800 len 4096 data: 4
16190 069/300 Write block 1483 len 12288 data: 5 4 2
16191 070/300 Read block 3697 len 36864
16192 071/300 Read block 2530 len 4096
16193 072/300 Write block 923 len 4096 data: 3
16194 073/300 Read block 4592 len 32768
16195 074/300 Flush
16196 075/300 Read block 1698 len 20480
16197 076/300 Read block 3483 len 28672
16198 077/300 Read block 2135 len 36864
16199 078/300 Read block 1154 len 4096
16200 079/300 Read block 1928 len 20480
16201 080/300 Read block 4768 len 20480
16202 081/300 Read block 847 len 16384
16203 082/300 Read block 3317 len 12288
16204 083/300 Write block 3490 len 32768 data: 3 4 4 5 6 6 6 5
16205 084/300 Write block 1103 len 16384 data: 5 3 3 4
16206 085/300 Read block 3112 len 28672
16207 086/300 Read block 2097 len 8192
16208 087/300 Write block 4666 len 24576 data: 4 2 2 2 2 3
16209 088/300 Read block 1632 len 40960
16210 089/300 Flush
16211 090/300 Read block 1168 len 36864
16212 091/300 Write block 4648 len 16384 data: 3 3 4 4
16213 092/300 Write block 1481 len 36864 data: 6 6 6 5 3 2 2 3 4
16214 093/300 Read block 3480 len 8192
16215 094/300 Write block 3684 len 12288 data: 6 5 4
16216 095/300 Flush
16217 096/300 Write block 2272 len 12288 data: 5 4 4
16218 097/300 Write block 4026 len 32768 data: 7 6 7 6 5 3 3 3
16219 098/300 Read block 1805 len 20480
16220 099/300 Write block 2754 len 40960 data: 2 2 2 2 4 4 4 4 2 3
16221 100/300 Write block 4602 len 12288 data: 5 6 6
16222 101/300 Read block 4761 len 28672
16223 102/300 Write block 3082 len 24576 data: 7 7 5 5 5 7
16224 103/300 Flush
16225 104/300 Write block 3700 len 20480 data: 5 5 4 3 3
16226 105/300 Read block 586 len 36864
16227 106/300 Read block 1247 len 16384
16228 107/300 Write block 4076 len 32768 data: 4 5 4 4 4 4 3 3
16229 108/300 Write block 344 len 20480 data: 5 7 5 6 5
16230 109/300 Read block 1956 len 40960
16231 110/300 Read block 286 len 32768
16232 111/300 Read block 3567 len 4096
16233 112/300 Read block 1293 len 8192
16234 113/300 Write block 3562 len 16384 data: 3 3 5 5
16235 114/300 Read block 2965 len 24576
16236 115/300 Read block 1600 len 4096
16237 116/300 Write block 2715 len 36864 data: 6 6 6 4 4 2 2 2 2
16238 117/300 Write block 2001 len 4096 data: 4
16239 118/300 Write block 3331 len 4096 data: 2
16240 119/300 Flush
16241 120/300 Flush
16242 121/300 Read block 4919 len 36864
16243 122/300 Read block 3075 len 36864
16244 123/300 Write block 3331 len 4096 data: 3
16245 124/300 Read block 2395 len 36864
16246 125/300 Read block 9 len 28672
16247 126/300 Read block 1099 len 36864
16248 127/300 Read block 1877 len 20480
16249 128/300 Read block 2140 len 32768
16250 129/300 Read block 696 len 8192
16251 130/300 Read block 4334 len 4096
16252 131/300 Write block 856 len 40960 data: 2 2 2 2 3 3 3 3 4 5
16253 132/300 Read block 3725 len 12288
16254 133/300 Read block 95 len 28672
16255 134/300 Write block 3830 len 8192 data: 5 5
16256 135/300 Flush
16257 136/300 Read block 114 len 12288
16258 137/300 Read block 1177 len 32768
16259 138/300 Read block 4348 len 24576
16260 139/300 Read block 1183 len 28672
16261 140/300 Read block 4737 len 40960
16262 141/300 Write block 4088 len 40960 data: 4 4 4 4 3 2 2 2 2 3
16263 142/300 Write block 3456 len 36864 data: 5 5 3 2 3 4 4 4 3
16264 143/300 Read block 2026 len 20480
16265 144/300 Flush
16266 145/300 Read block 3381 len 8192
16267 146/300 Read block 218 len 24576
16268 147/300 Read block 3293 len 32768
16269 148/300 Read block 1390 len 36864
16270 149/300 Flush
16271 150/300 Read block 477 len 20480
16272 151/300 Write block 1851 len 32768 data: 7 8 6 6 8 8 7 7
16273 152/300 Flush
16274 153/300 Read block 1267 len 36864
16275 154/300 Read block 2555 len 4096
16276 155/300 Write block 1405 len 16384 data: 3 3 3 2
16277 156/300 Read block 2022 len 36864
16278 157/300 Write block 2744 len 28672 data: 6 6 6 7 7 6 3
16279 158/300 Read block 1725 len 20480
16280 159/300 Read block 1655 len 36864
16281 160/300 Flush
16282 161/300 Flush
16283 162/300 Read block 1098 len 24576
16284 163/300 Read block 304 len 24576
16285 164/300 Read block 4840 len 28672
16286 165/300 Write block 2830 len 32768 data: 5 5 5 4 3 3 3 3
16287 166/300 Write block 4550 len 8192 data: 4 3
16288 167/300 Read block 267 len 8192
16289 168/300 Flush
16290 169/300 Read block 4614 len 12288
16291 170/300 Write block 1853 len 16384 data: 7 7 9 9
16292 171/300 Write block 1562 len 4096 data: 4
16293 172/300 Read block 1034 len 36864
16294 173/300 Write block 3126 len 8192 data: 3 2
16295 174/300 Write block 1304 len 24576 data: 4 4 4 4 3 3
16296 175/300 Flush
16297 176/300 Read block 39 len 32768
16298 177/300 Write block 1736 len 12288 data: 2 2 2
16299 178/300 Read block 3009 len 16384
16300 179/300 Read block 3829 len 20480
16301 180/300 Write block 1841 len 32768 data: 3 4 3 4 4 5 7 7
16302 181/300 Write block 1608 len 28672 data: 5 6 6 6 6 6 5
16303 182/300 Read block 1469 len 24576
16304 183/300 Write block 3777 len 20480 data: 4 4 4 4 4
16305 184/300 Read block 2500 len 20480
16306 185/300 Write block 2964 len 12288 data: 3 3 3
16307 186/300 Read block 3724 len 4096
16308 187/300 Read block 2431 len 36864
16309 188/300 Read block 752 len 36864
16310 189/300 Read block 4514 len 12288
16311 190/300 Write block 4331 len 36864 data: 4 4 4 4 4 4 3 4 4
16312 191/300 Write block 2758 len 32768 data: 5 5 5 5 3 4 3 2
16313 192/300 Read block 2791 len 8192
16314 193/300 Write block 714 len 40960 data: 3 3 4 5 5 5 5 5 6 6
16315 194/300 Write block 4045 len 24576 data: 6 5 5 3 3 3
16316 195/300 Flush
16317 196/300 Write block 4270 len 16384 data: 4 4 3 3
16318 197/300 Read block 3576 len 36864
16319 198/300 Read block 1512 len 28672
16320 199/300 Flush
16321 200/300 Read block 600 len 4096
16322 201/300 Write block 1806 len 32768 data: 4 5 4 4 4 5 4 4
16323 202/300 Read block 4827 len 28672
16324 203/300 Write block 1302 len 8192 data: 3 4
16325 204/300 Write block 4449 len 40960 data: 6 6 7 7 6 6 4 4 4 3
16326 205/300 Write block 1011 len 4096 data: 3
16327 206/300 Read block 4310 len 16384
16328 207/300 Flush
16329 208/300 Write block 3230 len 12288 data: 10 8 8
16330 209/300 Read block 3569 len 32768
16331 210/300 Flush
16332 211/300 Read block 316 len 16384
16333 212/300 Write block 1831 len 24576 data: 5 5 5 4 4 4
16334 213/300 Read block 3534 len 16384
16335 214/300 Write block 3739 len 28672 data: 2 4 4 5 4 5 6
16336 215/300 Write block 1043 len 36864 data: 2 2 3 3 3 4 3 2 2
16337 216/300 Read block 3516 len 32768
16338 217/300 Read block 1000 len 12288
16339 218/300 Write block 1771 len 36864 data: 4 4 4 5 5 5 5 5 6
16340 219/300 Read block 1472 len 36864
16341 220/300 Write block 1060 len 16384 data: 4 3 3 3
16342 221/300 Write block 170 len 32768 data: 4 4 5 5 4 4 4 4
16343 222/300 Read block 4622 len 20480
16344 223/300 Write block 231 len 36864 data: 4 4 4 6 5 5 5 6 6
16345 224/300 Write block 700 len 20480 data: 3 2 3 3 3
16346 225/300 Write block 3988 len 28672 data: 4 5 6 7 8 7 6
16347 226/300 Write block 3725 len 20480 data: 2 3 3 4 4
16348 227/300 Read block 145 len 8192
16349 228/300 Write block 610 len 36864 data: 2 2 3 3 4 4 4 4 3
16350 229/300 Write block 3167 len 28672 data: 5 5 4 4 4 3 3
16351 230/300 Read block 1524 len 32768
16352 231/300 Flush
16353 232/300 Write block 2634 len 32768 data: 3 3 3 3 4 4 3 3
16354 233/300 Flush
16355 234/300 Read block 3285 len 8192
16356 235/300 Read block 4956 len 8192
16357 236/300 Flush
16358 237/300 Read block 4792 len 40960
16359 238/300 Read block 3206 len 28672
16360 239/300 Read block 149 len 32768
16361 240/300 Write block 1326 len 12288 data: 4 3 2
16362 241/300 Write block 4397 len 12288 data: 5 6 6
16363 242/300 Write block 2643 len 16384 data: 3 3 2 2
16364 243/300 Write block 1943 len 8192 data: 6 6
16365 244/300 Flush
16366 245/300 Read block 3602 len 32768
16367 246/300 Write block 1682 len 24576 data: 4 4 4 3 3 3
16368 247/300 Read block 4344 len 32768
16369 248/300 Read block 555 len 12288
16370 249/300 Write block 3591 len 40960 data: 4 5 7 7 6 7 8 8 8 6
16371 250/300 Write block 4726 len 36864 data: 4 4 3 4 4 5 4 4 2
16372 251/300 Write block 1940 len 28672 data: 9 8 8 7 7 5 4
16373 252/300 Read block 4250 len 16384
16374 253/300 Read block 2600 len 32768
16375 254/300 Write block 3647 len 36864 data: 6 6 4 3 3 3 6 5 3
16376 255/300 Read block 4864 len 40960
16377 256/300 Flush
16378 257/300 Read block 2037 len 28672
16379 258/300 Write block 4308 len 28672 data: 4 4 3 4 5 6 6
16380 259/300 Read block 1484 len 12288
16381 260/300 Read block 3317 len 24576
16382 261/300 Write block 2957 len 32768 data: 4 6 5 4 4 4 4 4
16383 262/300 Flush
16384 263/300 Read block 4887 len 16384
16385 264/300 Flush
16386 265/300 Write block 1028 len 20480 data: 3 4 3 4 4
16387 266/300 Read block 3741 len 32768
16388 267/300 Write block 2187 len 4096 data: 3
16389 268/300 Write block 4402 len 12288 data: 3 3 4
16390 269/300 Write block 2939 len 36864 data: 3 2 2 2 3 3 3 4 4
16391 270/300 Read block 29 len 40960
16392 271/300 Read block 577 len 36864
16393 272/300 Read block 566 len 32768
16394 273/300 Read block 1928 len 16384
16395 274/300 Read block 859 len 24576
16396 275/300 Write block 1337 len 8192 data: 3 3
16397 276/300 Read block 870 len 40960
16398 277/300 Read block 1924 len 12288
16399 278/300 Read block 831 len 24576
16400 279/300 Read block 4831 len 20480
16401 280/300 Write block 2207 len 8192 data: 2 2
16402 281/300 Read block 113 len 8192
16403 282/300 Flush
16404 283/300 Read block 838 len 4096
16405 284/300 Read block 2506 len 12288
16406 285/300 Write block 880 len 40960 data: 3 2 2 2 2 4 4 4 4 5
16407 286/300 Write block 1026 len 4096 data: 4
16408 287/300 Write block 4674 len 8192 data: 4 4
16409 288/300 Write block 770 len 36864 data: 3 3 2 2 2 2 2 2 2
16410 289/300 Write block 510 len 20480 data: 5 5 6 6 5
16411 290/300 Write block 3487 len 20480 data: 2 2 2 4 5
16412 291/300 Read block 3645 len 8192
16413 292/300 Read block 1277 len 4096
16414 293/300 Read block 1822 len 8192
16415 294/300 Read block 3907 len 36864
16416 295/300 Write block 1559 len 28672 data: 3 3 3 5 4 4 4
16417 296/300 Flush
16418 297/300 Write block 2675 len 8192 data: 4 4
16419 298/300 Write block 1096 len 12288 data: 5 5 6
16420 299/300 Write block 540 len 20480 data: 7 7 8 8 7
16421 300/300 Write block 3160 len 12288 data: 7 6 5
16422 301/300 Write block 1460 len 4096 data: 3
16423 Main task has work to do, go find it
16424 got dsc Start(1)
16425 start 1
16426 Nov 26 22:08:00.310 INFO request completed, latency_us: 219, response_code: 204, uri: /start/cid/1, method: POST, req_id: 4402d2e2-ef7d-47f1-91a8-a4cc03a23487, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
16427 Replay: started 1, returned:Ok(())
16428 ----------------------------------------------------------------
16429 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
16430 States: Active Offline Active
16431 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
16432 5529 Acked Read 9 Done Sent Done false
16433 5530 Acked Write 9 Done Sent Done false
16434 5531 Acked Read 6 Done Sent Done false
16435 5532 Acked Read 3 Done Sent Done false
16436 5533 Acked Read 1 Done Sent Done false
16437 5534 Acked Read 6 Done Sent Done false
16438 5535 Acked Write 2 Done Sent Done false
16439 5536 Acked Flush 0 Done Sent Done false
16440 5537 Acked Read 1 Done Sent Done false
16441 5538 Acked Write 5 Done Sent Done false
16442 5539 Acked Flush 0 Done Sent Done false
16443 5540 Acked Write 5 Done Sent Done false
16444 5541 Acked Read 4 Done Sent Done false
16445 5542 Acked Flush 0 Done Sent Done false
16446 5543 Acked Read 3 Done Sent Done false
16447 5544 Acked Write 10 Done Sent Done false
16448 5545 Acked Write 7 Done Sent Done false
16449 5546 Acked Read 7 Done Sent Done false
16450 5547 Acked Write 6 Done Sent Done false
16451 5548 Acked Write 4 Done Sent Done false
16452 5549 Acked Write 7 Done Sent Done false
16453 5550 Acked Write 2 Done Sent Done false
16454 5551 Acked Read 6 Done Sent Done false
16455 5552 Acked Write 9 Done Sent Done false
16456 5553 Acked Read 7 Done Sent Done false
16457 5554 Acked Write 8 Done Sent Done false
16458 5555 Acked Write 6 Done Sent Done false
16459 5556 Acked Read 9 Done Sent Done false
16460 5557 Acked Write 7 Done Sent Done false
16461 5558 Acked Write 8 Done Sent Done false
16462 5559 Acked Read 2 Done Sent Done false
16463 5560 Acked Write 3 Done Sent Done false
16464 5561 Acked Write 9 Done Sent Done false
16465 5562 Acked Read 1 Done Sent Done false
16466 5563 Acked Flush 0 Done Sent Done false
16467 5564 Acked Read 1 Done Sent Done false
16468 5565 Acked Flush 0 Done Sent Done false
16469 5566 Acked Read 8 Done Sent Done false
16470 5567 Acked Read 5 Done Sent Done false
16471 5568 Acked Read 2 Done Sent Done false
16472 5569 Acked Write 10 Done Sent Done false
16473 5570 Acked Read 8 Done Sent Done false
16474 5571 Acked Read 6 Done Sent Done false
16475 5572 Acked Write 7 Done Sent Done false
16476 5573 Acked Read 4 Done Sent Done false
16477 5574 Acked Read 5 Done Sent Done false
16478 5575 Acked Read 2 Done Sent Done false
16479 5576 Acked Write 9 Done Sent Done false
16480 5577 Acked Write 6 Done Sent Done false
16481 5578 Acked Read 2 Done Sent Done false
16482 5579 Acked Write 3 Done Sent Done false
16483 5580 Acked Write 8 Done Sent Done false
16484 5581 Acked Flush 0 Done Sent Done false
16485 5582 Acked Flush 0 Done Sent Done false
16486 5583 Acked Write 1 Done Sent Done false
16487 5584 Acked Write 2 Done Sent Done false
16488 5585 Acked Write 6 Done Sent Done false
16489 5586 Acked Read 3 Done Sent Done false
16490 5587 Acked Write 6 Done Sent Done false
16491 5588 Acked Flush 0 Done Sent Done false
16492 5589 Acked Read 6 Done Sent Done false
16493 5590 Acked Write 1 Done Sent Done false
16494 5591 Acked Write 8 Done Sent Done false
16495 5592 Acked Write 8 Done Sent Done false
16496 5593 Acked Flush 0 Done Sent Done false
16497 5594 Acked Write 3 Done Sent Done false
16498 5595 Acked Read 8 Done Sent Done false
16499 5596 Acked Write 1 Done Sent Done false
16500 5597 Acked Write 3 Done Sent Done false
16501 5598 Acked Read 9 Done Sent Done false
16502 5599 Acked Read 1 Done Sent Done false
16503 5600 Acked Write 1 Done Sent Done false
16504 5601 Acked Read 8 Done Sent Done false
16505 5602 Acked Flush 0 Done Sent Done false
16506 5603 Acked Read 5 Done Sent Done false
16507 5604 Acked Read 7 Done Sent Done false
16508 5605 Acked Read 9 Done Sent Done false
16509 5606 Acked Read 1 Done Sent Done false
16510 5607 Acked Read 5 Done Sent Done false
16511 5608 Acked Read 5 Done Sent Done false
16512 5609 Acked Read 4 Done Sent Done false
16513 5610 Acked Read 3 Done Sent Done false
16514 5611 Acked Write 8 Done Sent Done false
16515 5612 Acked Write 4 Done Sent Done false
16516 5613 Acked Read 7 Done Sent Done false
16517 5614 Acked Read 2 Done Sent Done false
16518 5615 Acked Write 6 Done Sent Done false
16519 5616 Acked Read 10 Done Sent Done false
16520 5617 Acked Flush 0 Done Sent Done false
16521 5618 Acked Read 9 Done Sent Done false
16522 5619 Acked Write 4 Done Sent Done false
16523 5620 Acked Write 9 Done Sent Done false
16524 5621 Acked Read 2 Done Sent Done false
16525 5622 Acked Write 3 Done Sent Done false
16526 5623 Acked Flush 0 Done Sent Done false
16527 5624 Acked Write 3 Done Sent Done false
16528 5625 Acked Write 8 Done Sent Done false
16529 5626 Acked Read 5 Done Sent Done false
16530 5627 Acked Write 10 Done Sent Done false
16531 5628 Acked Write 3 Done Sent Done false
16532 5629 Acked Read 7 Done Sent Done false
16533 5630 Acked Write 6 Done Sent Done false
16534 5631 Acked Flush 0 Done Sent Done false
16535 5632 Acked Write 5 Done Sent Done false
16536 5633 Acked Read 9 Done Sent Done false
16537 5634 Acked Read 4 Done Sent Done false
16538 5635 Acked Write 8 Done Sent Done false
16539 5636 Acked Write 5 Done Sent Done false
16540 5637 Acked Read 10 Done Sent Done false
16541 5638 Acked Read 8 Done Sent Done false
16542 5639 Acked Read 1 Done Sent Done false
16543 5640 Acked Read 2 Done Sent Done false
16544 5641 Acked Write 4 Done Sent Done false
16545 5642 Acked Read 6 Done Sent Done false
16546 5643 Acked Read 1 Done Sent Done false
16547 5644 Acked Write 9 Done Sent Done false
16548 5645 Acked Write 1 Done Sent Done false
16549 5646 Acked Write 1 Done Sent Done false
16550 5647 Acked Flush 0 Done Sent Done false
16551 5648 Acked Flush 0 Done Sent Done false
16552 5649 Acked Read 9 Done Sent Done false
16553 5650 Acked Read 9 Done Sent Done false
16554 5651 Acked Write 1 Done Sent Done false
16555 5652 Acked Read 9 Done Sent Done false
16556 5653 Acked Read 7 Done Sent Done false
16557 5654 Acked Read 9 Done Sent Done false
16558 5655 Acked Read 5 Done Sent Done false
16559 5656 Acked Read 8 Done Sent Done false
16560 5657 Acked Read 2 Done Sent Done false
16561 5658 Acked Read 1 Done Sent Done false
16562 5659 Acked Write 10 Done Sent Done false
16563 5660 Acked Read 3 Done Sent Done false
16564 5661 Acked Read 7 Done Sent Done false
16565 5662 Acked Write 2 Done Sent Done false
16566 5663 Acked Flush 0 Done Sent Done false
16567 5664 Acked Read 3 Done Sent Done false
16568 5665 Acked Read 8 Done Sent Done false
16569 5666 Acked Read 6 Done Sent Done false
16570 5667 Acked Read 7 Done Sent Done false
16571 5668 Acked Read 10 Done Sent Done false
16572 5669 Acked Write 10 Done Sent Done false
16573 5670 Acked Write 9 Done Sent Done false
16574 5671 Acked Read 5 Done Sent Done false
16575 5672 Acked Flush 0 Done Sent Done false
16576 5673 Acked Read 2 Done Sent Done false
16577 5674 Acked Read 6 Done Sent Done false
16578 5675 Acked Read 8 Done Sent Done false
16579 5676 Acked Read 9 Done Sent Done false
16580 5677 Acked Flush 0 Done Sent Done false
16581 5678 Acked Read 5 Done Sent Done false
16582 5679 Acked Write 8 Done Sent Done false
16583 5680 Acked Flush 0 Done Sent Done false
16584 5681 Acked Read 9 Done Sent Done false
16585 5682 Acked Read 1 Done Sent Done false
16586 5683 Acked Write 4 Done Sent Done false
16587 5684 Acked Read 9 Done Sent Done false
16588 5685 Acked Write 7 Done Sent Done false
16589 5686 Acked Read 5 Done Sent Done false
16590 5687 Acked Read 9 Done Sent Done false
16591 5688 Acked Flush 0 Done Sent Done false
16592 5689 Acked Flush 0 Done Sent Done false
16593 5690 Acked Read 6 Done Sent Done false
16594 5691 Acked Read 6 Done Sent Done false
16595 5692 Acked Read 7 Done Sent Done false
16596 5693 Acked Write 8 Done Sent Done false
16597 5694 Acked Write 2 Done Sent Done false
16598 5695 Acked Read 2 Done Sent Done false
16599 5696 Acked Flush 0 Done Sent Done false
16600 5697 Acked Read 3 Done Sent Done false
16601 5698 Acked Write 4 Done Sent Done false
16602 5699 Acked Write 1 Done Sent Done false
16603 5700 Acked Read 9 Done Sent Done false
16604 5701 Acked Write 2 Done Sent Done false
16605 5702 Acked Write 6 Done Sent Done false
16606 5703 Acked Flush 0 Done Sent Done false
16607 5704 Acked Read 8 Done Sent Done false
16608 5705 Acked Write 3 Done Sent Done false
16609 5706 Acked Read 4 Done Sent Done false
16610 5707 Acked Read 5 Done Sent Done false
16611 5708 Acked Write 8 Done Sent Done false
16612 5709 Acked Write 7 Done Sent Done false
16613 5710 Acked Read 6 Done Sent Done false
16614 5711 Acked Write 5 Done Sent Done false
16615 5712 Acked Read 5 Done Sent Done false
16616 5713 Acked Write 3 Done Sent Done false
16617 5714 Acked Read 1 Done Sent Done false
16618 5715 Acked Read 9 Done Sent Done false
16619 5716 Acked Read 9 Done Sent Done false
16620 5717 Acked Read 3 Done Sent Done false
16621 5718 Acked Write 9 Done Sent Done false
16622 5719 Acked Write 8 Done Sent Done false
16623 5720 Acked Read 2 Done Sent Done false
16624 5721 Acked Write 10 Done Sent Done false
16625 5722 Acked Write 6 Done Sent Done false
16626 5723 Acked Flush 0 Done Sent Done false
16627 5724 Acked Write 4 Done Sent Done false
16628 5725 Acked Read 9 Done Sent Done false
16629 5726 Acked Read 7 Done Sent Done false
16630 5727 Acked Flush 0 Done Sent Done false
16631 5728 Acked Read 1 Done Sent Done false
16632 5729 Acked Write 8 Done Sent Done false
16633 5730 Acked Read 7 Done Sent Done false
16634 5731 Acked Write 2 Done Sent Done false
16635 5732 Acked Write 10 Done Sent Done false
16636 5733 Acked Write 1 Done Sent Done false
16637 5734 Acked Read 4 Done Sent Done false
16638 5735 Acked Flush 0 Done Sent Done false
16639 5736 Acked Write 3 Done Sent Done false
16640 5737 Acked Read 8 Done Sent Done false
16641 5738 Acked Flush 0 Done Sent Done false
16642 5739 Acked Read 4 Done Sent Done false
16643 5740 Acked Write 6 Done Sent Done false
16644 5741 Acked Read 4 Done Sent Done false
16645 5742 Acked Write 7 Done Sent Done false
16646 5743 Acked Write 9 Done Sent Done false
16647 5744 Acked Read 8 Done Sent Done false
16648 5745 Acked Read 3 Done Sent Done false
16649 5746 Acked Write 9 Done Sent Done false
16650 5747 Acked Read 9 Done Sent Done false
16651 5748 Acked Write 4 Done Sent Done false
16652 5749 Acked Write 8 Done Sent Done false
16653 5750 Acked Read 5 Done Sent Done false
16654 5751 Acked Write 9 Done Sent Done false
16655 5752 Acked Write 5 Done Sent Done false
16656 5753 Acked Write 7 Done Sent Done false
16657 5754 Acked Write 5 Done Sent Done false
16658 5755 Acked Read 2 Done Sent Done false
16659 5756 Acked Write 9 Done Sent Done false
16660 5757 Acked Write 7 Done Sent Done false
16661 5758 Acked Read 8 Done Sent Done false
16662 5759 Acked Flush 0 Done Sent Done false
16663 5760 Acked Write 8 Done Sent Done false
16664 5761 Acked Flush 0 Done Sent Done false
16665 5762 Acked Read 2 Done Sent Done false
16666 5763 Acked Read 2 Done Sent Done false
16667 5764 Acked Flush 0 Done Sent Done false
16668 5765 Acked Read 10 Done Sent Done false
16669 5766 Acked Read 7 Done Sent Done false
16670 5767 Acked Read 8 Done Sent Done false
16671 5768 Acked Write 3 Done Sent Done false
16672 5769 Acked Write 3 Done Sent Done false
16673 5770 Acked Write 4 Done Sent Done false
16674 5771 Acked Write 2 Done Sent Done false
16675 5772 Acked Flush 0 Done Sent Done false
16676 5773 Acked Read 8 Done Sent Done false
16677 5774 Acked Write 6 Done Sent Done false
16678 5775 Acked Read 8 Done Sent Done false
16679 5776 Acked Read 3 Done Sent Done false
16680 5777 Acked Write 10 Done Sent Done false
16681 5778 Acked Write 9 Done Sent Done false
16682 5779 Acked Write 7 Done Sent Done false
16683 5780 Acked Read 4 Done Sent Done false
16684 5781 Acked Read 8 Done Sent Done false
16685 5782 Acked Write 9 Done Sent Done false
16686 5783 Acked Read 10 Done Sent Done false
16687 5784 Acked Flush 0 Done Sent Done false
16688 5785 Acked Read 7 Done Sent Done false
16689 5786 Acked Write 7 Done Sent Done false
16690 5787 Acked Read 3 Done Sent Done false
16691 5788 Acked Read 6 Done Sent Done false
16692 5789 Acked Write 8 Done Sent Done false
16693 5790 Acked Flush 0 Done Sent Done false
16694 5791 Acked Read 4 Done Sent Done false
16695 5792 Acked Flush 0 Done Sent Done false
16696 5793 Acked Write 5 Done Sent Done false
16697 5794 Acked Read 8 Done Sent Done false
16698 5795 Acked Write 1 Done Sent Done false
16699 5796 Acked Write 3 Done Sent Done false
16700 5797 Acked Write 9 Done Sent Done false
16701 5798 Acked Read 10 Done Sent Done false
16702 5799 Acked Read 9 Done Sent Done false
16703 5800 Acked Read 8 Done Sent Done false
16704 5801 Acked Read 4 Done Sent Done false
16705 5802 Acked Read 6 Done Sent Done false
16706 5803 Acked Write 2 Done Sent Done false
16707 5804 Acked Read 10 Done Sent Done false
16708 5805 Acked Read 3 Done Sent Done false
16709 5806 Acked Read 6 Done Sent Done false
16710 5807 Acked Read 5 Done Sent Done false
16711 5808 Acked Write 2 Done Sent Done false
16712 5809 Acked Read 2 Done Sent Done false
16713 5810 Acked Flush 0 Done Sent Done false
16714 5811 Acked Read 1 Done Sent Done false
16715 5812 Acked Read 3 Done Sent Done false
16716 5813 Acked Write 10 Done Sent Done false
16717 5814 Acked Write 1 Done Sent Done false
16718 5815 Acked Write 2 Done Sent Done false
16719 5816 Acked Write 9 Done Sent Done false
16720 5817 Acked Write 5 Done Sent Done false
16721 5818 Acked Write 5 Done Sent Done false
16722 5819 Acked Read 2 Done Sent Done false
16723 5820 Acked Read 1 Done Sent Done false
16724 5821 Acked Read 2 Done Sent Done false
16725 5822 Acked Read 9 Done Sent Done false
16726 5823 Acked Write 7 Done Sent Done false
16727 5824 Acked Flush 0 Done Sent Done false
16728 5825 Acked Write 2 Done Sent Done false
16729 5826 Acked Write 3 Done Sent Done false
16730 5827 Acked Write 5 Done Sent Done false
16731 5828 Acked Write 3 Done Sent Done false
16732 5829 Acked Write 1 Done Sent Done false
16733 STATES DS:0 DS:1 DS:2 TOTAL
16734 Sent 0 301 0 301
16735 Done 301 0 301 602
16736 Skipped 0 0 0 0
16737 Error 0 0 0 0
16738 Last Flush: 5824 5528 5824
16739 Downstairs last five completed: 5528 5527 5526 5525 5524
16740 Upstairs last five completed: 5829 5828 5827 5826 5825
16741 CLIENT: Up:0 ds:301 act:2
16742 [8820] got start action
16743 [8820] I am going to restart
16744 Starting downstairs at port 8820
16745 Make output file at "/tmp/dsc/downstairs-8820.txt"
16746 [8820][1] reports Starting
16747 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1219)
16748 [8820][1] reports Running
16749 ----------------------------------------------------------------
16750 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
16751 States: Active Offline Active
16752 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
16753 5529 Acked Read 9 Done Sent Done false
16754 5530 Acked Write 9 Done Sent Done false
16755 5531 Acked Read 6 Done Sent Done false
16756 5532 Acked Read 3 Done Sent Done false
16757 5533 Acked Read 1 Done Sent Done false
16758 5534 Acked Read 6 Done Sent Done false
16759 5535 Acked Write 2 Done Sent Done false
16760 5536 Acked Flush 0 Done Sent Done false
16761 5537 Acked Read 1 Done Sent Done false
16762 5538 Acked Write 5 Done Sent Done false
16763 5539 Acked Flush 0 Done Sent Done false
16764 5540 Acked Write 5 Done Sent Done false
16765 5541 Acked Read 4 Done Sent Done false
16766 5542 Acked Flush 0 Done Sent Done false
16767 5543 Acked Read 3 Done Sent Done false
16768 5544 Acked Write 10 Done Sent Done false
16769 5545 Acked Write 7 Done Sent Done false
16770 5546 Acked Read 7 Done Sent Done false
16771 5547 Acked Write 6 Done Sent Done false
16772 5548 Acked Write 4 Done Sent Done false
16773 5549 Acked Write 7 Done Sent Done false
16774 5550 Acked Write 2 Done Sent Done false
16775 5551 Acked Read 6 Done Sent Done false
16776 5552 Acked Write 9 Done Sent Done false
16777 5553 Acked Read 7 Done Sent Done false
16778 5554 Acked Write 8 Done Sent Done false
16779 5555 Acked Write 6 Done Sent Done false
16780 5556 Acked Read 9 Done Sent Done false
16781 5557 Acked Write 7 Done Sent Done false
16782 5558 Acked Write 8 Done Sent Done false
16783 5559 Acked Read 2 Done Sent Done false
16784 5560 Acked Write 3 Done Sent Done false
16785 5561 Acked Write 9 Done Sent Done false
16786 5562 Acked Read 1 Done Sent Done false
16787 5563 Acked Flush 0 Done Sent Done false
16788 5564 Acked Read 1 Done Sent Done false
16789 5565 Acked Flush 0 Done Sent Done false
16790 5566 Acked Read 8 Done Sent Done false
16791 5567 Acked Read 5 Done Sent Done false
16792 5568 Acked Read 2 Done Sent Done false
16793 5569 Acked Write 10 Done Sent Done false
16794 5570 Acked Read 8 Done Sent Done false
16795 5571 Acked Read 6 Done Sent Done false
16796 5572 Acked Write 7 Done Sent Done false
16797 5573 Acked Read 4 Done Sent Done false
16798 5574 Acked Read 5 Done Sent Done false
16799 5575 Acked Read 2 Done Sent Done false
16800 5576 Acked Write 9 Done Sent Done false
16801 5577 Acked Write 6 Done Sent Done false
16802 5578 Acked Read 2 Done Sent Done false
16803 5579 Acked Write 3 Done Sent Done false
16804 5580 Acked Write 8 Done Sent Done false
16805 5581 Acked Flush 0 Done Sent Done false
16806 5582 Acked Flush 0 Done Sent Done false
16807 5583 Acked Write 1 Done Sent Done false
16808 5584 Acked Write 2 Done Sent Done false
16809 5585 Acked Write 6 Done Sent Done false
16810 5586 Acked Read 3 Done Sent Done false
16811 5587 Acked Write 6 Done Sent Done false
16812 5588 Acked Flush 0 Done Sent Done false
16813 5589 Acked Read 6 Done Sent Done false
16814 5590 Acked Write 1 Done Sent Done false
16815 5591 Acked Write 8 Done Sent Done false
16816 5592 Acked Write 8 Done Sent Done false
16817 5593 Acked Flush 0 Done Sent Done false
16818 5594 Acked Write 3 Done Sent Done false
16819 5595 Acked Read 8 Done Sent Done false
16820 5596 Acked Write 1 Done Sent Done false
16821 5597 Acked Write 3 Done Sent Done false
16822 5598 Acked Read 9 Done Sent Done false
16823 5599 Acked Read 1 Done Sent Done false
16824 5600 Acked Write 1 Done Sent Done false
16825 5601 Acked Read 8 Done Sent Done false
16826 5602 Acked Flush 0 Done Sent Done false
16827 5603 Acked Read 5 Done Sent Done false
16828 5604 Acked Read 7 Done Sent Done false
16829 5605 Acked Read 9 Done Sent Done false
16830 5606 Acked Read 1 Done Sent Done false
16831 5607 Acked Read 5 Done Sent Done false
16832 5608 Acked Read 5 Done Sent Done false
16833 5609 Acked Read 4 Done Sent Done false
16834 5610 Acked Read 3 Done Sent Done false
16835 5611 Acked Write 8 Done Sent Done false
16836 5612 Acked Write 4 Done Sent Done false
16837 5613 Acked Read 7 Done Sent Done false
16838 5614 Acked Read 2 Done Sent Done false
16839 5615 Acked Write 6 Done Sent Done false
16840 5616 Acked Read 10 Done Sent Done false
16841 5617 Acked Flush 0 Done Sent Done false
16842 5618 Acked Read 9 Done Sent Done false
16843 5619 Acked Write 4 Done Sent Done false
16844 5620 Acked Write 9 Done Sent Done false
16845 5621 Acked Read 2 Done Sent Done false
16846 5622 Acked Write 3 Done Sent Done false
16847 5623 Acked Flush 0 Done Sent Done false
16848 5624 Acked Write 3 Done Sent Done false
16849 5625 Acked Write 8 Done Sent Done false
16850 5626 Acked Read 5 Done Sent Done false
16851 5627 Acked Write 10 Done Sent Done false
16852 5628 Acked Write 3 Done Sent Done false
16853 5629 Acked Read 7 Done Sent Done false
16854 5630 Acked Write 6 Done Sent Done false
16855 5631 Acked Flush 0 Done Sent Done false
16856 5632 Acked Write 5 Done Sent Done false
16857 5633 Acked Read 9 Done Sent Done false
16858 5634 Acked Read 4 Done Sent Done false
16859 5635 Acked Write 8 Done Sent Done false
16860 5636 Acked Write 5 Done Sent Done false
16861 5637 Acked Read 10 Done Sent Done false
16862 5638 Acked Read 8 Done Sent Done false
16863 5639 Acked Read 1 Done Sent Done false
16864 5640 Acked Read 2 Done Sent Done false
16865 5641 Acked Write 4 Done Sent Done false
16866 5642 Acked Read 6 Done Sent Done false
16867 5643 Acked Read 1 Done Sent Done false
16868 5644 Acked Write 9 Done Sent Done false
16869 5645 Acked Write 1 Done Sent Done false
16870 5646 Acked Write 1 Done Sent Done false
16871 5647 Acked Flush 0 Done Sent Done false
16872 5648 Acked Flush 0 Done Sent Done false
16873 5649 Acked Read 9 Done Sent Done false
16874 5650 Acked Read 9 Done Sent Done false
16875 5651 Acked Write 1 Done Sent Done false
16876 5652 Acked Read 9 Done Sent Done false
16877 5653 Acked Read 7 Done Sent Done false
16878 5654 Acked Read 9 Done Sent Done false
16879 5655 Acked Read 5 Done Sent Done false
16880 5656 Acked Read 8 Done Sent Done false
16881 5657 Acked Read 2 Done Sent Done false
16882 5658 Acked Read 1 Done Sent Done false
16883 5659 Acked Write 10 Done Sent Done false
16884 5660 Acked Read 3 Done Sent Done false
16885 5661 Acked Read 7 Done Sent Done false
16886 5662 Acked Write 2 Done Sent Done false
16887 5663 Acked Flush 0 Done Sent Done false
16888 5664 Acked Read 3 Done Sent Done false
16889 5665 Acked Read 8 Done Sent Done false
16890 5666 Acked Read 6 Done Sent Done false
16891 5667 Acked Read 7 Done Sent Done false
16892 5668 Acked Read 10 Done Sent Done false
16893 5669 Acked Write 10 Done Sent Done false
16894 5670 Acked Write 9 Done Sent Done false
16895 5671 Acked Read 5 Done Sent Done false
16896 5672 Acked Flush 0 Done Sent Done false
16897 5673 Acked Read 2 Done Sent Done false
16898 5674 Acked Read 6 Done Sent Done false
16899 5675 Acked Read 8 Done Sent Done false
16900 5676 Acked Read 9 Done Sent Done false
16901 5677 Acked Flush 0 Done Sent Done false
16902 5678 Acked Read 5 Done Sent Done false
16903 5679 Acked Write 8 Done Sent Done false
16904 5680 Acked Flush 0 Done Sent Done false
16905 5681 Acked Read 9 Done Sent Done false
16906 5682 Acked Read 1 Done Sent Done false
16907 5683 Acked Write 4 Done Sent Done false
16908 5684 Acked Read 9 Done Sent Done false
16909 5685 Acked Write 7 Done Sent Done false
16910 5686 Acked Read 5 Done Sent Done false
16911 5687 Acked Read 9 Done Sent Done false
16912 5688 Acked Flush 0 Done Sent Done false
16913 5689 Acked Flush 0 Done Sent Done false
16914 5690 Acked Read 6 Done Sent Done false
16915 5691 Acked Read 6 Done Sent Done false
16916 5692 Acked Read 7 Done Sent Done false
16917 5693 Acked Write 8 Done Sent Done false
16918 5694 Acked Write 2 Done Sent Done false
16919 5695 Acked Read 2 Done Sent Done false
16920 5696 Acked Flush 0 Done Sent Done false
16921 5697 Acked Read 3 Done Sent Done false
16922 5698 Acked Write 4 Done Sent Done false
16923 5699 Acked Write 1 Done Sent Done false
16924 5700 Acked Read 9 Done Sent Done false
16925 5701 Acked Write 2 Done Sent Done false
16926 5702 Acked Write 6 Done Sent Done false
16927 5703 Acked Flush 0 Done Sent Done false
16928 5704 Acked Read 8 Done Sent Done false
16929 5705 Acked Write 3 Done Sent Done false
16930 5706 Acked Read 4 Done Sent Done false
16931 5707 Acked Read 5 Done Sent Done false
16932 5708 Acked Write 8 Done Sent Done false
16933 5709 Acked Write 7 Done Sent Done false
16934 5710 Acked Read 6 Done Sent Done false
16935 5711 Acked Write 5 Done Sent Done false
16936 5712 Acked Read 5 Done Sent Done false
16937 5713 Acked Write 3 Done Sent Done false
16938 5714 Acked Read 1 Done Sent Done false
16939 5715 Acked Read 9 Done Sent Done false
16940 5716 Acked Read 9 Done Sent Done false
16941 5717 Acked Read 3 Done Sent Done false
16942 5718 Acked Write 9 Done Sent Done false
16943 5719 Acked Write 8 Done Sent Done false
16944 5720 Acked Read 2 Done Sent Done false
16945 5721 Acked Write 10 Done Sent Done false
16946 5722 Acked Write 6 Done Sent Done false
16947 5723 Acked Flush 0 Done Sent Done false
16948 5724 Acked Write 4 Done Sent Done false
16949 5725 Acked Read 9 Done Sent Done false
16950 5726 Acked Read 7 Done Sent Done false
16951 5727 Acked Flush 0 Done Sent Done false
16952 5728 Acked Read 1 Done Sent Done false
16953 5729 Acked Write 8 Done Sent Done false
16954 5730 Acked Read 7 Done Sent Done false
16955 5731 Acked Write 2 Done Sent Done false
16956 5732 Acked Write 10 Done Sent Done false
16957 5733 Acked Write 1 Done Sent Done false
16958 5734 Acked Read 4 Done Sent Done false
16959 5735 Acked Flush 0 Done Sent Done false
16960 5736 Acked Write 3 Done Sent Done false
16961 5737 Acked Read 8 Done Sent Done false
16962 5738 Acked Flush 0 Done Sent Done false
16963 5739 Acked Read 4 Done Sent Done false
16964 5740 Acked Write 6 Done Sent Done false
16965 5741 Acked Read 4 Done Sent Done false
16966 5742 Acked Write 7 Done Sent Done false
16967 5743 Acked Write 9 Done Sent Done false
16968 5744 Acked Read 8 Done Sent Done false
16969 5745 Acked Read 3 Done Sent Done false
16970 5746 Acked Write 9 Done Sent Done false
16971 5747 Acked Read 9 Done Sent Done false
16972 5748 Acked Write 4 Done Sent Done false
16973 5749 Acked Write 8 Done Sent Done false
16974 5750 Acked Read 5 Done Sent Done false
16975 5751 Acked Write 9 Done Sent Done false
16976 5752 Acked Write 5 Done Sent Done false
16977 5753 Acked Write 7 Done Sent Done false
16978 5754 Acked Write 5 Done Sent Done false
16979 5755 Acked Read 2 Done Sent Done false
16980 5756 Acked Write 9 Done Sent Done false
16981 5757 Acked Write 7 Done Sent Done false
16982 5758 Acked Read 8 Done Sent Done false
16983 5759 Acked Flush 0 Done Sent Done false
16984 5760 Acked Write 8 Done Sent Done false
16985 5761 Acked Flush 0 Done Sent Done false
16986 5762 Acked Read 2 Done Sent Done false
16987 5763 Acked Read 2 Done Sent Done false
16988 5764 Acked Flush 0 Done Sent Done false
16989 5765 Acked Read 10 Done Sent Done false
16990 5766 Acked Read 7 Done Sent Done false
16991 5767 Acked Read 8 Done Sent Done false
16992 5768 Acked Write 3 Done Sent Done false
16993 5769 Acked Write 3 Done Sent Done false
16994 5770 Acked Write 4 Done Sent Done false
16995 5771 Acked Write 2 Done Sent Done false
16996 5772 Acked Flush 0 Done Sent Done false
16997 5773 Acked Read 8 Done Sent Done false
16998 5774 Acked Write 6 Done Sent Done false
16999 5775 Acked Read 8 Done Sent Done false
17000 5776 Acked Read 3 Done Sent Done false
17001 5777 Acked Write 10 Done Sent Done false
17002 5778 Acked Write 9 Done Sent Done false
17003 5779 Acked Write 7 Done Sent Done false
17004 5780 Acked Read 4 Done Sent Done false
17005 5781 Acked Read 8 Done Sent Done false
17006 5782 Acked Write 9 Done Sent Done false
17007 5783 Acked Read 10 Done Sent Done false
17008 5784 Acked Flush 0 Done Sent Done false
17009 5785 Acked Read 7 Done Sent Done false
17010 5786 Acked Write 7 Done Sent Done false
17011 5787 Acked Read 3 Done Sent Done false
17012 5788 Acked Read 6 Done Sent Done false
17013 5789 Acked Write 8 Done Sent Done false
17014 5790 Acked Flush 0 Done Sent Done false
17015 5791 Acked Read 4 Done Sent Done false
17016 5792 Acked Flush 0 Done Sent Done false
17017 5793 Acked Write 5 Done Sent Done false
17018 5794 Acked Read 8 Done Sent Done false
17019 5795 Acked Write 1 Done Sent Done false
17020 5796 Acked Write 3 Done Sent Done false
17021 5797 Acked Write 9 Done Sent Done false
17022 5798 Acked Read 10 Done Sent Done false
17023 5799 Acked Read 9 Done Sent Done false
17024 5800 Acked Read 8 Done Sent Done false
17025 5801 Acked Read 4 Done Sent Done false
17026 5802 Acked Read 6 Done Sent Done false
17027 5803 Acked Write 2 Done Sent Done false
17028 5804 Acked Read 10 Done Sent Done false
17029 5805 Acked Read 3 Done Sent Done false
17030 5806 Acked Read 6 Done Sent Done false
17031 5807 Acked Read 5 Done Sent Done false
17032 5808 Acked Write 2 Done Sent Done false
17033 5809 Acked Read 2 Done Sent Done false
17034 5810 Acked Flush 0 Done Sent Done false
17035 5811 Acked Read 1 Done Sent Done false
17036 5812 Acked Read 3 Done Sent Done false
17037 5813 Acked Write 10 Done Sent Done false
17038 5814 Acked Write 1 Done Sent Done false
17039 5815 Acked Write 2 Done Sent Done false
17040 5816 Acked Write 9 Done Sent Done false
17041 5817 Acked Write 5 Done Sent Done false
17042 5818 Acked Write 5 Done Sent Done false
17043 5819 Acked Read 2 Done Sent Done false
17044 5820 Acked Read 1 Done Sent Done false
17045 5821 Acked Read 2 Done Sent Done false
17046 5822 Acked Read 9 Done Sent Done false
17047 5823 Acked Write 7 Done Sent Done false
17048 5824 Acked Flush 0 Done Sent Done false
17049 5825 Acked Write 2 Done Sent Done false
17050 5826 Acked Write 3 Done Sent Done false
17051 5827 Acked Write 5 Done Sent Done false
17052 5828 Acked Write 3 Done Sent Done false
17053 5829 Acked Write 1 Done Sent Done false
17054 5830 Acked Flush 0 Done Sent Done false
17055 STATES DS:0 DS:1 DS:2 TOTAL
17056 Sent 0 302 0 302
17057 Done 302 0 302 604
17058 Skipped 0 0 0 0
17059 Error 0 0 0 0
17060 Last Flush: 5830 5528 5830
17061 Downstairs last five completed: 5528 5527 5526 5525 5524
17062 Upstairs last five completed: 5830 5829 5828 5827 5826
17063 CLIENT: Up:0 ds:302 act:2
170642024-11-26T22:08:06.079ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
170652024-11-26T22:08:06.079ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:49024) = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
170662024-11-26T22:08:06.080ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
170672024-11-26T22:08:06.081ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
170682024-11-26T22:08:06.081ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
170692024-11-26T22:08:06.081ZINFOcrucible: send last flush ID to this DS: 5528 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
170702024-11-26T22:08:06.081ZINFOcrucible: Replied this last flush ID: 5528 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
170712024-11-26T22:08:06.081ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
170722024-11-26T22:08:06.081ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
170732024-11-26T22:08:06.081ZINFOcrucible: [1] client re-new 302 jobs since flush 5528 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
170742024-11-26T22:08:06.081ZINFOcrucible: [1] Replayed 302 jobs since flush: 5528 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
17075 ----------------------------------------------------------------
17076 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
17077 Downstairs last five completed: 5830 5829 5828 5827 5826
17078 Upstairs last five completed: 5830 5829 5828 5827 5826
17079 CLIENT: Up:0 ds:0 act:3
17080 Replay: All jobs finished
17081 Main task has work to do, go find it
17082 got dsc Stop(2)
17083 stop 2
17084 Nov 26 22:08:08.315 INFO request completed, latency_us: 360, response_code: 204, [8830] Got stop action so:false kr:false
17085 uri: /stop/cid/2, method: POST, req_id: fa686449-45d8-4b14-bd10-a14558b8fe0a, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
17086 Nov 26 22:08:08.316 {"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2024-11-26T22:08:08.31598569Z","hostname":"w-01JDN7ZQ17ABQG15KHKN3F2D26","pid":1203,"":"io task","client":"2","":"downstairs","session_id":"86a36c17-3718-484c-87fd-2b8a1dc2ab18"}
17087 INFO request completed, latency_us: 137, response_code: 200, uri: /state/cid/2, method: GET, {"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2024-11-26T22:08:08.316061461Z","hostname":"w-01JDN7ZQ17ABQG15KHKN3F2D26","pid":1203,"":"io task","client":"2","":"downstairs","session_id":"86a36c17-3718-484c-87fd-2b8a1dc2ab18"}
17088 req_id: e9bd88e1-a0f4-4f9e-a6e7-e2736fc06226, remote_addr: 127.0.0.{"msg":"downstairs task for 2 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2024-11-26T22:08:08.316111725Z","hostname":"w-01JDN7ZQ17ABQG15KHKN3F2D26","pid":1203,"session_id":"86a36c17-3718-484c-87fd-2b8a1dc2ab18"}
17089 1:36829, local_addr: 127.0.0.1:9998
170902024-11-26T22:08:08.316ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
170912024-11-26T22:08:08.316ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
170922024-11-26T22:08:08.316ZINFOcrucible: client task is exiting = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
17093 [8830] Exited with: ExitStatus(unix_wait_status(9))
17094 [8830][2] reports Exit
17095 Nov 26 22:08:12.318 INFO request completed, latency_us: 216, response_code: 200, uri: /state/cid/2, method: GET, req_id: 10ecf219-965f-4642-bed6-d36665a86c6a, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
17096 001/300 Write block 237 len 12288 data: 6 7 7
17097 002/300 Write block 2740 len 28672 data: 4 5 5 6 7 7 7
17098 003/300 Read block 2006 len 40960
17099 004/300 Read block 4689 len 20480
17100 005/300 Write block 3459 len 24576 data: 3 4 5 5 5 4
17101 006/300 Write block 4154 len 28672 data: 3 3 4 3 4 5 4
17102 007/300 Read block 2647 len 4096
17103 008/300 Write block 4691 len 40960 data: 6 6 8 8 7 4 4 4 3 4
17104 009/300 Read block 4493 len 4096
17105 010/300 Read block 3438 len 12288
17106 011/300 Read block 2770 len 36864
17107 012/300 Read block 2155 len 12288
17108 013/300 Read block 4029 len 4096
17109 014/300 Write block 4558 len 32768 data: 3 3 3 3 4 4 5 4
17110 015/300 Read block 3483 len 40960
17111 016/300 Write block 3568 len 40960 data: 5 5 5 5 5 5 5 4 2 2
17112 017/300 Read block 4679 len 12288
17113 018/300 Read block 942 len 28672
17114 019/300 Read block 2832 len 40960
17115 020/300 Read block 4629 len 24576
17116 021/300 Write block 2078 len 4096 data: 5
17117 022/300 Write block 1568 len 40960 data: 3 4 4 4 5 5 5 5 5 4
17118 023/300 Read block 870 len 20480
17119 024/300 Read block 1791 len 16384
17120 025/300 Write block 1812 len 12288 data: 5 5 4
17121 026/300 Read block 3417 len 12288
17122 027/300 Read block 1158 len 20480
17123 028/300 Write block 2858 len 16384 data: 3 4 5 5
17124 029/300 Read block 4068 len 24576
17125 030/300 Read block 2178 len 20480
17126 031/300 Read block 3456 len 40960
17127 032/300 Flush
17128 033/300 Write block 2125 len 4096 data: 2
17129 034/300 Read block 1085 len 32768
17130 035/300 Write block 26 len 16384 data: 3 3 4 3
17131 036/300 Read block 3790 len 24576
17132 037/300 Write block 637 len 32768 data: 5 5 6 6 7 7 7 7
17133 038/300 Read block 1463 len 36864
17134 039/300 Read block 944 len 32768
17135 040/300 Flush
17136 041/300 Read block 1629 len 4096
17137 042/300 Write block 1984 len 24576 data: 4 4 6 5 5 5
17138 043/300 Read block 3822 len 4096
17139 044/300 Write block 1234 len 28672 data: 6 6 5 4 4 4 4
17140 045/300 Read block 3813 len 12288
17141 046/300 Write block 1751 len 40960 data: 5 4 4 4 4 4 5 3 4 5
17142 047/300 Read block 48 len 12288
17143 048/300 Read block 1620 len 8192
17144 049/300 Write block 3882 len 40960 data: 3 3 3 3 3 3 3 3 3 3
17145 050/300 Write block 210 len 28672 data: 2 2 2 2 3 4 6
17146 051/300 Read block 921 len 32768
17147 052/300 Write block 3718 len 36864 data: 6 6 4 2 2 2 2 3 4
17148 053/300 Read block 2419 len 20480
17149 054/300 Read block 840 len 4096
17150 055/300 Write block 3046 len 4096 data: 2
17151 056/300 Read block 2158 len 20480
17152 057/300 Read block 1529 len 32768
17153 058/300 Read block 2170 len 40960
17154 059/300 Write block 1289 len 24576 data: 5 5 5 4 3 3
17155 060/300 Flush
17156 061/300 Read block 3910 len 20480
17157 062/300 Read block 1736 len 12288
17158 063/300 Read block 2969 len 40960
17159 064/300 Write block 4959 len 28672 data: 2 3 4 4 4 5 7
17160 065/300 Write block 953 len 32768 data: 3 4 3 3 3 4 4 4
17161 066/300 Write block 3823 len 40960 data: 2 2 3 3 5 5 5 6 6 4
17162 067/300 Write block 3895 len 8192 data: 5 5
17163 068/300 Write block 1587 len 8192 data: 3 3
17164 069/300 Write block 2558 len 20480 data: 6 6 5 5 5
17165 070/300 Write block 4037 len 24576 data: 5 4 4 4 5 6
17166 071/300 Read block 2656 len 16384
17167 072/300 Read block 1203 len 24576
17168 073/300 Write block 4744 len 12288 data: 2 2 3
17169 074/300 Write block 1047 len 36864 data: 4 5 4 3 3 3 3 4 4
17170 075/300 Read block 1161 len 36864
17171 076/300 Read block 1114 len 28672
17172 077/300 Read block 633 len 16384
17173 078/300 Read block 897 len 40960
17174 079/300 Read block 576 len 12288
17175 080/300 Read block 4873 len 16384
17176 081/300 Write block 261 len 4096 data: 3
17177 082/300 Write block 125 len 40960 data: 5 6 6 5 5 5 2 2 2 2
17178 083/300 Read block 3894 len 8192
17179 084/300 Read block 2580 len 32768
17180 085/300 Read block 1865 len 40960
17181 086/300 Flush
17182 087/300 Read block 3466 len 36864
17183 088/300 Read block 3637 len 16384
17184 089/300 Read block 1085 len 8192
17185 090/300 Write block 3253 len 32768 data: 2 2 2 3 3 4 5 4
17186 091/300 Read block 989 len 12288
17187 092/300 Read block 1371 len 12288
17188 093/300 Read block 3604 len 20480
17189 094/300 Write block 3053 len 20480 data: 3 5 4 4 4
17190 095/300 Flush
17191 096/300 Read block 2667 len 16384
17192 097/300 Write block 2918 len 28672 data: 5 4 4 5 4 4 3
17193 098/300 Flush
17194 099/300 Read block 2850 len 32768
17195 100/300 Write block 4258 len 20480 data: 6 4 2 2 2
17196 101/300 Read block 41 len 20480
17197 102/300 Flush
17198 103/300 Read block 2714 len 20480
17199 104/300 Flush
17200 105/300 Read block 1843 len 20480
17201 106/300 Read block 2945 len 8192
17202 107/300 Read block 4480 len 20480
17203 108/300 Read block 222 len 28672
17204 109/300 Flush
17205 110/300 Read block 4947 len 12288
17206 111/300 Read block 565 len 40960
17207 112/300 Write block 3778 len 36864 data: 5 5 5 5 6 5 6 5 4
17208 113/300 Read block 3275 len 20480
17209 114/300 Write block 2116 len 32768 data: 2 2 2 2 2 2 2 2
17210 115/300 Flush
17211 116/300 Write block 2663 len 20480 data: 4 4 5 4 5
17212 117/300 Read block 2266 len 20480
17213 118/300 Read block 4745 len 20480
17214 119/300 Read block 1694 len 4096
17215 120/300 Read block 1590 len 8192
17216 121/300 Write block 1383 len 20480 data: 4 4 3 3 3
17217 122/300 Read block 4131 len 24576
17218 123/300 Read block 3719 len 40960
17219 124/300 Write block 3272 len 4096 data: 3
17220 125/300 Read block 1439 len 12288
17221 126/300 Flush
17222 127/300 Flush
17223 128/300 Read block 4537 len 36864
17224 129/300 Write block 4142 len 12288 data: 3 4 6
17225 130/300 Read block 2334 len 24576
17226 131/300 Read block 3122 len 28672
17227 132/300 Read block 1411 len 24576
17228 133/300 Write block 1670 len 24576 data: 3 5 5 6 6 5
17229 134/300 Read block 117 len 24576
17230 135/300 Read block 92 len 4096
17231 136/300 Read block 961 len 8192
17232 137/300 Write block 1185 len 8192 data: 5 4
17233 138/300 Read block 3758 len 32768
17234 139/300 Write block 1559 len 32768 data: 4 4 4 6 5 5 5 4
17235 140/300 Read block 40 len 4096
17236 141/300 Read block 2793 len 4096
17237 142/300 Write block 102 len 40960 data: 5 4 4 2 2 2 2 2 2 2
17238 143/300 Read block 3396 len 28672
17239 144/300 Write block 4540 len 4096 data: 4
17240 145/300 Write block 2398 len 28672 data: 6 5 5 7 6 6 6
17241 146/300 Flush
17242 147/300 Read block 3479 len 28672
17243 148/300 Flush
17244 149/300 Read block 4553 len 12288
17245 150/300 Read block 4979 len 8192
17246 151/300 Write block 4861 len 20480 data: 6 6 6 5 3
17247 152/300 Read block 2283 len 12288
17248 153/300 Read block 1386 len 4096
17249 154/300 Read block 3929 len 20480
17250 155/300 Write block 4747 len 12288 data: 4 4 3
17251 156/300 Read block 4734 len 28672
17252 157/300 Write block 2842 len 36864 data: 6 6 5 5 7 5 5 3 3
17253 158/300 Read block 2797 len 24576
17254 159/300 Write block 4549 len 4096 data: 5
17255 160/300 Flush
17256 161/300 Write block 2530 len 8192 data: 3 3
17257 162/300 Read block 2813 len 40960
17258 163/300 Read block 2497 len 16384
17259 164/300 Write block 4015 len 4096 data: 6
17260 165/300 Write block 2252 len 32768 data: 6 7 6 3 3 2 2 3
17261 166/300 Read block 2315 len 40960
17262 167/300 Write block 3629 len 20480 data: 3 3 5 5 5
17263 168/300 Write block 707 len 24576 data: 2 2 3 3 4 4
17264 169/300 Write block 2812 len 8192 data: 5 6
17265 170/300 Write block 1364 len 24576 data: 5 5 5 4 4 3
17266 171/300 Read block 2475 len 32768
17267 172/300 Read block 3410 len 28672
17268 173/300 Read block 4075 len 24576
17269 174/300 Read block 1321 len 12288
17270 175/300 Read block 2434 len 20480
17271 176/300 Read block 1623 len 12288
17272 177/300 Write block 4331 len 28672 data: 5 5 5 5 5 5 4
17273 178/300 Read block 4450 len 28672
17274 179/300 Write block 1174 len 12288 data: 3 3 4
17275 180/300 Write block 4890 len 32768 data: 2 3 3 4 5 4 5 4
17276 181/300 Read block 4154 len 20480
17277 182/300 Write block 2951 len 16384 data: 4 5 5 4
17278 183/300 Read block 216 len 16384
17279 184/300 Flush
17280 185/300 Read block 1800 len 28672
17281 186/300 Write block 4946 len 12288 data: 4 4 5
17282 187/300 Read block 1823 len 40960
17283 188/300 Read block 2941 len 32768
17284 189/300 Read block 1577 len 28672
17285 190/300 Read block 2684 len 4096
17286 191/300 Read block 115 len 28672
17287 192/300 Read block 4542 len 4096
17288 193/300 Write block 340 len 20480 data: 5 5 5 5 6
17289 194/300 Read block 3775 len 24576
17290 195/300 Write block 3037 len 4096 data: 3
17291 196/300 Write block 4920 len 32768 data: 3 3 3 3 4 5 5 5
17292 197/300 Flush
17293 198/300 Read block 3396 len 36864
17294 199/300 Read block 4745 len 12288
17295 200/300 Read block 2390 len 28672
17296 201/300 Read block 3234 len 16384
17297 202/300 Write block 1981 len 24576 data: 4 5 5 5 5 7
17298 203/300 Read block 4926 len 24576
17299 204/300 Write block 1619 len 28672 data: 4 4 5 5 5 6 7
17300 205/300 Read block 902 len 24576
17301 206/300 Read block 1085 len 28672
17302 207/300 Read block 4691 len 36864
17303 208/300 Read block 744 len 4096
17304 209/300 Read block 506 len 40960
17305 210/300 Read block 2455 len 32768
17306 211/300 Write block 2987 len 12288 data: 4 4 4
17307 212/300 Read block 2826 len 24576
17308 213/300 Write block 3523 len 36864 data: 3 3 3 2 2 3 3 4 5
17309 214/300 Read block 1024 len 12288
17310 215/300 Write block 2967 len 16384 data: 4 5 5 4
17311 216/300 Write block 2710 len 36864 data: 5 5 5 6 6 7 7 7 5
17312 217/300 Read block 3378 len 8192
17313 218/300 Read block 3473 len 36864
17314 219/300 Read block 1620 len 24576
17315 220/300 Read block 2444 len 32768
17316 221/300 Write block 2467 len 12288 data: 5 4 4
17317 222/300 Write block 1304 len 16384 data: 5 5 5 5
17318 223/300 Read block 3672 len 8192
17319 224/300 Write block 646 len 24576 data: 5 4 3 3 2 2
17320 225/300 Read block 3334 len 40960
17321 226/300 Write block 1444 len 32768 data: 2 2 3 3 3 2 2 2
17322 227/300 Read block 3350 len 8192
17323 228/300 Write block 1591 len 16384 data: 4 4 4 6
17324 229/300 Write block 4070 len 4096 data: 5
17325 230/300 Read block 1821 len 40960
17326 231/300 Read block 1371 len 36864
17327 232/300 Write block 784 len 36864 data: 3 4 4 4 4 5 5 5 4
17328 233/300 Write block 4147 len 24576 data: 5 4 4 3 3 3
17329 234/300 Write block 3605 len 40960 data: 3 3 3 3 3 4 4 4 4 4
17330 235/300 Read block 1947 len 20480
17331 236/300 Write block 771 len 12288 data: 4 3 3
17332 237/300 Write block 2115 len 40960 data: 2 3 3 3 3 3 3 3 3 2
17333 238/300 Read block 4690 len 24576
17334 239/300 Read block 4706 len 32768
17335 240/300 Read block 4550 len 8192
17336 241/300 Read block 4966 len 8192
17337 242/300 Read block 984 len 28672
17338 243/300 Read block 1883 len 32768
17339 244/300 Write block 3481 len 8192 data: 4 3
17340 245/300 Write block 1059 len 36864 data: 5 5 4 4 4 3 2 2 3
17341 246/300 Read block 3273 len 12288
17342 247/300 Write block 855 len 40960 data: 3 3 3 3 3 4 4 4 4 5
17343 248/300 Write block 2254 len 32768 data: 7 4 4 3 3 4 3 3
17344 249/300 Read block 319 len 24576
17345 250/300 Write block 3895 len 12288 data: 6 6 5
17346 251/300 Read block 3144 len 28672
17347 252/300 Write block 1249 len 16384 data: 3 4 3 3
17348 253/300 Read block 3212 len 28672
17349 254/300 Write block 3600 len 4096 data: 7
17350 255/300 Read block 447 len 12288
17351 256/300 Write block 253 len 16384 data: 3 3 3 3
17352 257/300 Write block 4322 len 20480 data: 4 3 4 4 4
17353 258/300 Read block 3473 len 28672
17354 259/300 Write block 736 len 36864 data: 3 4 4 4 5 6 7 5 4
17355 260/300 Read block 2018 len 36864
17356 261/300 Read block 4364 len 12288
17357 262/300 Read block 2142 len 40960
17358 263/300 Read block 3349 len 40960
17359 264/300 Write block 4950 len 32768 data: 4 4 4 4 4 2 2 2
17360 265/300 Read block 4849 len 4096
17361 266/300 Read block 3897 len 24576
17362 267/300 Write block 292 len 24576 data: 8 7 6 6 5 4
17363 268/300 Read block 2670 len 12288
17364 269/300 Flush
17365 270/300 Write block 1249 len 8192 data: 4 5
17366 271/300 Read block 1366 len 32768
17367 272/300 Write block 1261 len 32768 data: 4 4 5 4 3 3 3 3
17368 273/300 Read block 2667 len 12288
17369 274/300 Write block 1176 len 36864 data: 5 3 3 4 4 5 4 4 3
17370 275/300 Read block 691 len 40960
17371 276/300 Read block 132 len 24576
17372 277/300 Read block 3602 len 8192
17373 278/300 Write block 600 len 28672 data: 3 4 4 4 3 3 3
17374 279/300 Read block 200 len 24576
17375 280/300 Write block 402 len 12288 data: 6 6 5
17376 281/300 Flush
17377 282/300 Read block 3863 len 32768
17378 283/300 Read block 1395 len 20480
17379 284/300 Write block 395 len 28672 data: 3 3 4 4 5 4 4
17380 285/300 Read block 1529 len 28672
17381 286/300 Read block 4046 len 28672
17382 287/300 Read block 3191 len 28672
17383 288/300 Write block 2884 len 16384 data: 5 6 6 8
17384 289/300 Read block 4680 len 12288
17385 290/300 Read block 1927 len 28672
17386 291/300 Read block 2127 len 12288
17387 292/300 Write block 4968 len 40960 data: 6 7 5 4 3 3 3 3 3 4
17388 293/300 Write block 708 len 40960 data: 3 4 4 5 5 4 4 4 5 6
17389 294/300 Write block 1017 len 28672 data: 2 2 2 3 4 4 4
17390 295/300 Write block 1066 len 28672 data: 3 4 3 3 3 3 2
17391 296/300 Write block 3920 len 8192 data: 4 6
17392 297/300 Read block 4322 len 12288
17393 298/300 Read block 4524 len 12288
17394 299/300 Write block 2058 len 16384 data: 6 6 5 5
17395 300/300 Write block 4957 len 16384 data: 3 2 3 4
17396 301/300 Read block 728 len 36864
17397 Main task has work to do, go find it
17398 got dsc Start(2)
17399 start 2
17400 Nov 26 22:08:12.568 INFO request completed, latency_us: 185, response_code: 204, uri: /start/cid/2, method: POST, req_id: 9405af2b-c5a8-4db4-9fae-52cf11fe9faf, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
17401 Replay: started 2, returned:Ok(())
17402 ----------------------------------------------------------------
17403 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
17404 States: Active Active Offline
17405 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
17406 5831 Acked Write 3 Done Done Sent false
17407 5832 Acked Write 7 Done Done Sent false
17408 5833 Acked Read 10 Done Done Sent false
17409 5834 Acked Read 5 Done Done Sent false
17410 5835 Acked Write 6 Done Done Sent false
17411 5836 Acked Write 7 Done Done Sent false
17412 5837 Acked Read 1 Done Done Sent false
17413 5838 Acked Write 10 Done Done Sent false
17414 5839 Acked Read 1 Done Done Sent false
17415 5840 Acked Read 3 Done Done Sent false
17416 5841 Acked Read 9 Done Done Sent false
17417 5842 Acked Read 3 Done Done Sent false
17418 5843 Acked Read 1 Done Done Sent false
17419 5844 Acked Write 8 Done Done Sent false
17420 5845 Acked Read 10 Done Done Sent false
17421 5846 Acked Write 10 Done Done Sent false
17422 5847 Acked Read 3 Done Done Sent false
17423 5848 Acked Read 7 Done Done Sent false
17424 5849 Acked Read 10 Done Done Sent false
17425 5850 Acked Read 6 Done Done Sent false
17426 5851 Acked Write 1 Done Done Sent false
17427 5852 Acked Write 10 Done Done Sent false
17428 5853 Acked Read 5 Done Done Sent false
17429 5854 Acked Read 4 Done Done Sent false
17430 5855 Acked Write 3 Done Done Sent false
17431 5856 Acked Read 3 Done Done Sent false
17432 5857 Acked Read 5 Done Done Sent false
17433 5858 Acked Write 4 Done Done Sent false
17434 5859 Acked Read 6 Done Done Sent false
17435 5860 Acked Read 5 Done Done Sent false
17436 5861 Acked Read 10 Done Done Sent false
17437 5862 Acked Flush 0 Done Done Sent false
17438 5863 Acked Write 1 Done Done Sent false
17439 5864 Acked Read 8 Done Done Sent false
17440 5865 Acked Write 4 Done Done Sent false
17441 5866 Acked Read 6 Done Done Sent false
17442 5867 Acked Write 8 Done Done Sent false
17443 5868 Acked Read 9 Done Done Sent false
17444 5869 Acked Read 8 Done Done Sent false
17445 5870 Acked Flush 0 Done Done Sent false
17446 5871 Acked Read 1 Done Done Sent false
17447 5872 Acked Write 6 Done Done Sent false
17448 5873 Acked Read 1 Done Done Sent false
17449 5874 Acked Write 7 Done Done Sent false
17450 5875 Acked Read 3 Done Done Sent false
17451 5876 Acked Write 10 Done Done Sent false
17452 5877 Acked Read 3 Done Done Sent false
17453 5878 Acked Read 2 Done Done Sent false
17454 5879 Acked Write 10 Done Done Sent false
17455 5880 Acked Write 7 Done Done Sent false
17456 5881 Acked Read 8 Done Done Sent false
17457 5882 Acked Write 9 Done Done Sent false
17458 5883 Acked Read 5 Done Done Sent false
17459 5884 Acked Read 1 Done Done Sent false
17460 5885 Acked Write 1 Done Done Sent false
17461 5886 Acked Read 5 Done Done Sent false
17462 5887 Acked Read 8 Done Done Sent false
17463 5888 Acked Read 10 Done Done Sent false
17464 5889 Acked Write 6 Done Done Sent false
17465 5890 Acked Flush 0 Done Done Sent false
17466 5891 Acked Read 5 Done Done Sent false
17467 5892 Acked Read 3 Done Done Sent false
17468 5893 Acked Read 10 Done Done Sent false
17469 5894 Acked Write 7 Done Done Sent false
17470 5895 Acked Write 8 Done Done Sent false
17471 5896 Acked Write 10 Done Done Sent false
17472 5897 Acked Write 2 Done Done Sent false
17473 5898 Acked Write 2 Done Done Sent false
17474 5899 Acked Write 5 Done Done Sent false
17475 5900 Acked Write 6 Done Done Sent false
17476 5901 Acked Read 4 Done Done Sent false
17477 5902 Acked Read 6 Done Done Sent false
17478 5903 Acked Write 3 Done Done Sent false
17479 5904 Acked Write 9 Done Done Sent false
17480 5905 Acked Read 9 Done Done Sent false
17481 5906 Acked Read 7 Done Done Sent false
17482 5907 Acked Read 4 Done Done Sent false
17483 5908 Acked Read 10 Done Done Sent false
17484 5909 Acked Read 3 Done Done Sent false
17485 5910 Acked Read 4 Done Done Sent false
17486 5911 Acked Write 1 Done Done Sent false
17487 5912 Acked Write 10 Done Done Sent false
17488 5913 Acked Read 2 Done Done Sent false
17489 5914 Acked Read 8 Done Done Sent false
17490 5915 Acked Read 10 Done Done Sent false
17491 5916 Acked Flush 0 Done Done Sent false
17492 5917 Acked Read 9 Done Done Sent false
17493 5918 Acked Read 4 Done Done Sent false
17494 5919 Acked Read 2 Done Done Sent false
17495 5920 Acked Write 8 Done Done Sent false
17496 5921 Acked Read 3 Done Done Sent false
17497 5922 Acked Read 3 Done Done Sent false
17498 5923 Acked Read 5 Done Done Sent false
17499 5924 Acked Write 5 Done Done Sent false
17500 5925 Acked Flush 0 Done Done Sent false
17501 5926 Acked Read 4 Done Done Sent false
17502 5927 Acked Write 7 Done Done Sent false
17503 5928 Acked Flush 0 Done Done Sent false
17504 5929 Acked Read 8 Done Done Sent false
17505 5930 Acked Write 5 Done Done Sent false
17506 5931 Acked Read 5 Done Done Sent false
17507 5932 Acked Flush 0 Done Done Sent false
17508 5933 Acked Read 5 Done Done Sent false
17509 5934 Acked Flush 0 Done Done Sent false
17510 5935 Acked Read 5 Done Done Sent false
17511 5936 Acked Read 2 Done Done Sent false
17512 5937 Acked Read 5 Done Done Sent false
17513 5938 Acked Read 7 Done Done Sent false
17514 5939 Acked Flush 0 Done Done Sent false
17515 5940 Acked Read 3 Done Done Sent false
17516 5941 Acked Read 10 Done Done Sent false
17517 5942 Acked Write 9 Done Done Sent false
17518 5943 Acked Read 5 Done Done Sent false
17519 5944 Acked Write 8 Done Done Sent false
17520 5945 Acked Flush 0 Done Done Sent false
17521 5946 Acked Write 5 Done Done Sent false
17522 5947 Acked Read 5 Done Done Sent false
17523 5948 Acked Read 5 Done Done Sent false
17524 5949 Acked Read 1 Done Done Sent false
17525 5950 Acked Read 2 Done Done Sent false
17526 5951 Acked Write 5 Done Done Sent false
17527 5952 Acked Read 6 Done Done Sent false
17528 5953 Acked Read 10 Done Done Sent false
17529 5954 Acked Write 1 Done Done Sent false
17530 5955 Acked Read 3 Done Done Sent false
17531 5956 Acked Flush 0 Done Done Sent false
17532 5957 Acked Flush 0 Done Done Sent false
17533 5958 Acked Read 9 Done Done Sent false
17534 5959 Acked Write 3 Done Done Sent false
17535 5960 Acked Read 6 Done Done Sent false
17536 5961 Acked Read 7 Done Done Sent false
17537 5962 Acked Read 6 Done Done Sent false
17538 5963 Acked Write 6 Done Done Sent false
17539 5964 Acked Read 6 Done Done Sent false
17540 5965 Acked Read 1 Done Done Sent false
17541 5966 Acked Read 2 Done Done Sent false
17542 5967 Acked Write 2 Done Done Sent false
17543 5968 Acked Read 8 Done Done Sent false
17544 5969 Acked Write 8 Done Done Sent false
17545 5970 Acked Read 1 Done Done Sent false
17546 5971 Acked Read 1 Done Done Sent false
17547 5972 Acked Write 10 Done Done Sent false
17548 5973 Acked Read 7 Done Done Sent false
17549 5974 Acked Write 1 Done Done Sent false
17550 5975 Acked Write 7 Done Done Sent false
17551 5976 Acked Flush 0 Done Done Sent false
17552 5977 Acked Read 7 Done Done Sent false
17553 5978 Acked Flush 0 Done Done Sent false
17554 5979 Acked Read 3 Done Done Sent false
17555 5980 Acked Read 2 Done Done Sent false
17556 5981 Acked Write 5 Done Done Sent false
17557 5982 Acked Read 3 Done Done Sent false
17558 5983 Acked Read 1 Done Done Sent false
17559 5984 Acked Read 5 Done Done Sent false
17560 5985 Acked Write 3 Done Done Sent false
17561 5986 Acked Read 7 Done Done Sent false
17562 5987 Acked Write 9 Done Done Sent false
17563 5988 Acked Read 6 Done Done Sent false
17564 5989 Acked Write 1 Done Done Sent false
17565 5990 Acked Flush 0 Done Done Sent false
17566 5991 Acked Write 2 Done Done Sent false
17567 5992 Acked Read 10 Done Done Sent false
17568 5993 Acked Read 4 Done Done Sent false
17569 5994 Acked Write 1 Done Done Sent false
17570 5995 Acked Write 8 Done Done Sent false
17571 5996 Acked Read 10 Done Done Sent false
17572 5997 Acked Write 5 Done Done Sent false
17573 5998 Acked Write 6 Done Done Sent false
17574 5999 Acked Write 2 Done Done Sent false
17575 6000 Acked Write 6 Done Done Sent false
17576 6001 Acked Read 8 Done Done Sent false
17577 6002 Acked Read 7 Done Done Sent false
17578 6003 Acked Read 6 Done Done Sent false
17579 6004 Acked Read 3 Done Done Sent false
17580 6005 Acked Read 5 Done Done Sent false
17581 6006 Acked Read 3 Done Done Sent false
17582 6007 Acked Write 7 Done Done Sent false
17583 6008 Acked Read 7 Done Done Sent false
17584 6009 Acked Write 3 Done Done Sent false
17585 6010 Acked Write 8 Done Done Sent false
17586 6011 Acked Read 5 Done Done Sent false
17587 6012 Acked Write 4 Done Done Sent false
17588 6013 Acked Read 4 Done Done Sent false
17589 6014 Acked Flush 0 Done Done Sent false
17590 6015 Acked Read 7 Done Done Sent false
17591 6016 Acked Write 3 Done Done Sent false
17592 6017 Acked Read 10 Done Done Sent false
17593 6018 Acked Read 8 Done Done Sent false
17594 6019 Acked Read 7 Done Done Sent false
17595 6020 Acked Read 1 Done Done Sent false
17596 6021 Acked Read 7 Done Done Sent false
17597 6022 Acked Read 1 Done Done Sent false
17598 6023 Acked Write 5 Done Done Sent false
17599 6024 Acked Read 6 Done Done Sent false
17600 6025 Acked Write 1 Done Done Sent false
17601 6026 Acked Write 8 Done Done Sent false
17602 6027 Acked Flush 0 Done Done Sent false
17603 6028 Acked Read 9 Done Done Sent false
17604 6029 Acked Read 3 Done Done Sent false
17605 6030 Acked Read 7 Done Done Sent false
17606 6031 Acked Read 4 Done Done Sent false
17607 6032 Acked Write 6 Done Done Sent false
17608 6033 Acked Read 6 Done Done Sent false
17609 6034 Acked Write 7 Done Done Sent false
17610 6035 Acked Read 6 Done Done Sent false
17611 6036 Acked Read 7 Done Done Sent false
17612 6037 Acked Read 9 Done Done Sent false
17613 6038 Acked Read 1 Done Done Sent false
17614 6039 Acked Read 10 Done Done Sent false
17615 6040 Acked Read 8 Done Done Sent false
17616 6041 Acked Write 3 Done Done Sent false
17617 6042 Acked Read 6 Done Done Sent false
17618 6043 Acked Write 9 Done Done Sent false
17619 6044 Acked Read 3 Done Done Sent false
17620 6045 Acked Write 4 Done Done Sent false
17621 6046 Acked Write 9 Done Done Sent false
17622 6047 Acked Read 2 Done Done Sent false
17623 6048 Acked Read 9 Done Done Sent false
17624 6049 Acked Read 6 Done Done Sent false
17625 6050 Acked Read 8 Done Done Sent false
17626 6051 Acked Write 3 Done Done Sent false
17627 6052 Acked Write 4 Done Done Sent false
17628 6053 Acked Read 2 Done Done Sent false
17629 6054 Acked Write 6 Done Done Sent false
17630 6055 Acked Read 10 Done Done Sent false
17631 6056 Acked Write 8 Done Done Sent false
17632 6057 Acked Read 2 Done Done Sent false
17633 6058 Acked Write 4 Done Done Sent false
17634 6059 Acked Write 1 Done Done Sent false
17635 6060 Acked Read 10 Done Done Sent false
17636 6061 Acked Read 9 Done Done Sent false
17637 6062 Acked Write 9 Done Done Sent false
17638 6063 Acked Write 6 Done Done Sent false
17639 6064 Acked Write 10 Done Done Sent false
17640 6065 Acked Read 5 Done Done Sent false
17641 6066 Acked Write 3 Done Done Sent false
17642 6067 Acked Write 10 Done Done Sent false
17643 6068 Acked Read 6 Done Done Sent false
17644 6069 Acked Read 8 Done Done Sent false
17645 6070 Acked Read 2 Done Done Sent false
17646 6071 Acked Read 2 Done Done Sent false
17647 6072 Acked Read 7 Done Done Sent false
17648 6073 Acked Read 8 Done Done Sent false
17649 6074 Acked Write 2 Done Done Sent false
17650 6075 Acked Write 9 Done Done Sent false
17651 6076 Acked Read 3 Done Done Sent false
17652 6077 Acked Write 10 Done Done Sent false
17653 6078 Acked Write 8 Done Done Sent false
17654 6079 Acked Read 6 Done Done Sent false
17655 6080 Acked Write 3 Done Done Sent false
17656 6081 Acked Read 7 Done Done Sent false
17657 6082 Acked Write 4 Done Done Sent false
17658 6083 Acked Read 7 Done Done Sent false
17659 6084 Acked Write 1 Done Done Sent false
17660 6085 Acked Read 3 Done Done Sent false
17661 6086 Acked Write 4 Done Done Sent false
17662 6087 Acked Write 5 Done Done Sent false
17663 6088 Acked Read 7 Done Done Sent false
17664 6089 Acked Write 9 Done Done Sent false
17665 6090 Acked Read 9 Done Done Sent false
17666 6091 Acked Read 3 Done Done Sent false
17667 6092 Acked Read 10 Done Done Sent false
17668 6093 Acked Read 10 Done Done Sent false
17669 6094 Acked Write 8 Done Done Sent false
17670 6095 Acked Read 1 Done Done Sent false
17671 6096 Acked Read 6 Done Done Sent false
17672 6097 Acked Write 6 Done Done Sent false
17673 6098 Acked Read 3 Done Done Sent false
17674 6099 Acked Flush 0 Done Done Sent false
17675 6100 Acked Write 2 Done Done Sent false
17676 6101 Acked Read 8 Done Done Sent false
17677 6102 Acked Write 8 Done Done Sent false
17678 6103 Acked Read 3 Done Done Sent false
17679 6104 Acked Write 9 Done Done Sent false
17680 6105 Acked Read 10 Done Done Sent false
17681 6106 Acked Read 6 Done Done Sent false
17682 6107 Acked Read 2 Done Done Sent false
17683 6108 Acked Write 7 Done Done Sent false
17684 6109 Acked Read 6 Done Done Sent false
17685 6110 Acked Write 3 Done Done Sent false
17686 6111 Acked Flush 0 Done Done Sent false
17687 6112 Acked Read 8 Done Done Sent false
17688 6113 Acked Read 5 Done Done Sent false
17689 6114 Acked Write 7 Done Done Sent false
17690 6115 Acked Read 7 Done Done Sent false
17691 6116 Acked Read 7 Done Done Sent false
17692 6117 Acked Read 7 Done Done Sent false
17693 6118 Acked Write 4 Done Done Sent false
17694 6119 Acked Read 3 Done Done Sent false
17695 6120 Acked Read 7 Done Done Sent false
17696 6121 Acked Read 3 Done Done Sent false
17697 6122 Acked Write 10 Done Done Sent false
17698 6123 Acked Write 10 Done Done Sent false
17699 6124 Acked Write 7 Done Done Sent false
17700 6125 Acked Write 7 Done Done Sent false
17701 6126 Acked Write 2 Done Done Sent false
17702 6127 Acked Read 3 Done Done Sent false
17703 6128 Acked Read 3 Done Done Sent false
17704 6129 Acked Write 4 Done Done Sent false
17705 6130 Acked Write 4 Done Done Sent false
17706 6131 Acked Read 9 Done Done Sent false
17707 STATES DS:0 DS:1 DS:2 TOTAL
17708 Sent 0 0 301 301
17709 Done 301 301 0 602
17710 Skipped 0 0 0 0
17711 Error 0 0 0 0
17712 Last Flush: 6111 6111 5830
17713 Downstairs last five completed: 5830 5829 5828 5827 5826
17714 Upstairs last five completed: 6131 6130 6129 6128 6127
17715 CLIENT: Up:0 ds:301 act:2
17716 ----------------------------------------------------------------
17717 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
17718 States: Active Active Offline
17719 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
17720 5831 Acked Write 3 Done Done Sent false
17721 5832 Acked Write 7 Done Done Sent false
17722 5833 Acked Read 10 Done Done Sent false
17723 5834 Acked Read 5 Done Done Sent false
17724 5835 Acked Write 6 Done Done Sent false
17725 5836 Acked Write 7 Done Done Sent false
17726 5837 Acked Read 1 Done Done Sent false
17727 5838 Acked Write 10 Done Done Sent false
17728 5839 Acked Read 1 Done Done Sent false
17729 5840 Acked Read 3 Done Done Sent false
17730 5841 Acked Read 9 Done Done Sent false
17731 5842 Acked Read 3 Done Done Sent false
17732 5843 Acked Read 1 Done Done Sent false
17733 5844 Acked Write 8 Done Done Sent false
17734 5845 Acked Read 10 Done Done Sent false
17735 5846 Acked Write 10 Done Done Sent false
17736 5847 Acked Read 3 Done Done Sent false
17737 5848 Acked Read 7 Done Done Sent false
17738 5849 Acked Read 10 Done Done Sent false
17739 5850 Acked Read 6 Done Done Sent false
17740 5851 Acked Write 1 Done Done Sent false
17741 5852 Acked Write 10 Done Done Sent false
17742 5853 Acked Read 5 Done Done Sent false
17743 5854 Acked Read 4 Done Done Sent false
17744 5855 Acked Write 3 Done Done Sent false
17745 5856 Acked Read 3 Done Done Sent false
17746 5857 Acked Read 5 Done Done Sent false
17747 5858 Acked Write 4 Done Done Sent false
17748 5859 Acked Read 6 Done Done Sent false
17749 5860 Acked Read 5 Done Done Sent false
17750 5861 Acked Read 10 Done Done Sent false
17751 5862 Acked Flush 0 Done Done Sent false
17752 5863 Acked Write 1 Done Done Sent false
17753 5864 Acked Read 8 Done Done Sent false
17754 5865 Acked Write 4 Done Done Sent false
17755 5866 Acked Read 6 Done Done Sent false
17756 5867 Acked Write 8 Done Done Sent false
17757 5868 Acked Read 9 Done Done Sent false
17758 5869 Acked Read 8 Done Done Sent false
17759 5870 Acked Flush 0 Done Done Sent false
17760 5871 Acked Read 1 Done Done Sent false
17761 5872 Acked Write 6 Done Done Sent false
17762 5873 Acked Read 1 Done Done Sent false
17763 5874 Acked Write 7 Done Done Sent false
17764 5875 Acked Read 3 Done Done Sent false
17765 5876 Acked Write 10 Done Done Sent false
17766 5877 Acked Read 3 Done Done Sent false
17767 5878 Acked Read 2 Done Done Sent false
17768 5879 Acked Write 10 Done Done Sent false
17769 5880 Acked Write 7 Done Done Sent false
17770 5881 Acked Read 8 Done Done Sent false
17771 5882 Acked Write 9 Done Done Sent false
17772 5883 Acked Read 5 Done Done Sent false
17773 5884 Acked Read 1 Done Done Sent false
17774 5885 Acked Write 1 Done Done Sent false
17775 5886 Acked Read 5 Done Done Sent false
17776 5887 Acked Read 8 Done Done Sent false
17777 5888 Acked Read 10 Done Done Sent false
17778 5889 Acked Write 6 Done Done Sent false
17779 5890 Acked Flush 0 Done Done Sent false
17780 5891 Acked Read 5 Done Done Sent false
17781 5892 Acked Read 3 Done Done Sent false
17782 5893 Acked Read 10 Done Done Sent false
17783 5894 Acked Write 7 Done Done Sent false
17784 5895 Acked Write 8 Done Done Sent false
17785 5896 Acked Write 10 Done Done Sent false
17786 5897 Acked Write 2 Done Done Sent false
17787 5898 Acked Write 2 Done Done Sent false
17788 5899 Acked Write 5 Done Done Sent false
17789 5900 Acked Write 6 Done Done Sent false
17790 5901 Acked Read 4 Done Done Sent false
17791 5902 Acked Read 6 Done Done Sent false
17792 5903 Acked Write 3 Done Done Sent false
17793 5904 Acked Write 9 Done Done Sent false
17794 5905 Acked Read 9 Done Done Sent false
17795 5906 Acked Read 7 Done Done Sent false
17796 5907 Acked Read 4 Done Done Sent false
17797 5908 Acked Read 10 Done Done Sent false
17798 5909 Acked Read 3 Done Done Sent false
17799 5910 Acked Read 4 Done Done Sent false
17800 5911 Acked Write 1 Done Done Sent false
17801 5912 Acked Write 10 Done Done Sent false
17802 5913 Acked Read 2 Done Done Sent false
17803 5914 Acked Read 8 Done Done Sent false
17804 5915 Acked Read 10 Done Done Sent false
17805 5916 Acked Flush 0 Done Done Sent false
17806 5917 Acked Read 9 Done Done Sent false
17807 5918 Acked Read 4 Done Done Sent false
17808 5919 Acked Read 2 Done Done Sent false
17809 5920 Acked Write 8 Done Done Sent false
17810 5921 Acked Read 3 Done Done Sent false
17811 5922 Acked Read 3 Done Done Sent false
17812 5923 Acked Read 5 Done Done Sent false
17813 5924 Acked Write 5 Done Done Sent false
17814 5925 Acked Flush 0 Done Done Sent false
17815 5926 Acked Read 4 Done Done Sent false
17816 5927 Acked Write 7 Done Done Sent false
17817 5928 Acked Flush 0 Done Done Sent false
17818 5929 Acked Read 8 Done Done Sent false
17819 5930 Acked Write 5 Done Done Sent false
17820 5931 Acked Read 5 Done Done Sent false
17821 5932 Acked Flush 0 Done Done Sent false
17822 5933 Acked Read 5 Done Done Sent false
17823 5934 Acked Flush 0 Done Done Sent false
17824 5935 Acked Read 5 Done Done Sent false
17825 5936 Acked Read 2 Done Done Sent false
17826 5937 Acked Read 5 Done Done Sent false
17827 5938 Acked Read 7 Done Done Sent false
17828 5939 Acked Flush 0 Done Done Sent false
17829 5940 Acked Read 3 Done Done Sent false
17830 5941 Acked Read 10 Done Done Sent false
17831 5942 Acked Write 9 Done Done Sent false
17832 5943 Acked Read 5 Done Done Sent false
17833 5944 Acked Write 8 Done Done Sent false
17834 5945 Acked Flush 0 Done Done Sent false
17835 5946 Acked Write 5 Done Done Sent false
17836 5947 Acked Read 5 Done Done Sent false
17837 5948 Acked Read 5 Done Done Sent false
17838 5949 Acked Read 1 Done Done Sent false
17839 5950 Acked Read 2 Done Done Sent false
17840 5951 Acked Write 5 Done Done Sent false
17841 5952 Acked Read 6 Done Done Sent false
17842 5953 Acked Read 10 Done Done Sent false
17843 5954 Acked Write 1 Done Done Sent false
17844 5955 Acked Read 3 Done Done Sent false
17845 5956 Acked Flush 0 Done Done Sent false
17846 5957 Acked Flush 0 Done Done Sent false
17847 5958 Acked Read 9 Done Done Sent false
17848 5959 Acked Write 3 Done Done Sent false
17849 5960 Acked Read 6 Done Done Sent false
17850 5961 Acked Read 7 Done Done Sent false
17851 5962 Acked Read 6 Done Done Sent false
17852 5963 Acked Write 6 Done Done Sent false
17853 5964 Acked Read 6 Done Done Sent false
17854 5965 Acked Read 1 Done Done Sent false
17855 5966 Acked Read 2 Done Done Sent false
17856 5967 Acked Write 2 Done Done Sent false
17857 5968 Acked Read 8 Done Done Sent false
17858 5969 Acked Write 8 Done Done Sent false
17859 5970 Acked Read 1 Done Done Sent false
17860 5971 Acked Read 1 Done Done Sent false
17861 5972 Acked Write 10 Done Done Sent false
17862 5973 Acked Read 7 Done Done Sent false
17863 5974 Acked Write 1 Done Done Sent false
17864 5975 Acked Write 7 Done Done Sent false
17865 5976 Acked Flush 0 Done Done Sent false
17866 5977 Acked Read 7 Done Done Sent false
17867 5978 Acked Flush 0 Done Done Sent false
17868 5979 Acked Read 3 Done Done Sent false
17869 5980 Acked Read 2 Done Done Sent false
17870 5981 Acked Write 5 Done Done Sent false
17871 5982 Acked Read 3 Done Done Sent false
17872 5983 Acked Read 1 Done Done Sent false
17873 5984 Acked Read 5 Done Done Sent false
17874 5985 Acked Write 3 Done Done Sent false
17875 5986 Acked Read 7 Done Done Sent false
17876 5987 Acked Write 9 Done Done Sent false
17877 5988 Acked Read 6 Done Done Sent false
17878 5989 Acked Write 1 Done Done Sent false
17879 5990 Acked Flush 0 Done Done Sent false
17880 5991 Acked Write 2 Done Done Sent false
17881 5992 Acked Read 10 Done Done Sent false
17882 5993 Acked Read 4 Done Done Sent false
17883 5994 Acked Write 1 Done Done Sent false
17884 5995 Acked Write 8 Done Done Sent false
17885 5996 Acked Read 10 Done Done Sent false
17886 5997 Acked Write 5 Done Done Sent false
17887 5998 Acked Write 6 Done Done Sent false
17888 5999 Acked Write 2 Done Done Sent false
17889 6000 Acked Write 6 Done Done Sent false
17890 6001 Acked Read 8 Done Done Sent false
17891 6002 Acked Read 7 Done Done Sent false
17892 6003 Acked Read 6 Done Done Sent false
17893 6004 Acked Read 3 Done Done Sent false
17894 6005 Acked Read 5 Done Done Sent false
17895 6006 Acked Read 3 Done Done Sent false
17896 6007 Acked Write 7 Done Done Sent false
17897 6008 Acked Read 7 Done Done Sent false
17898 6009 Acked Write 3 Done Done Sent false
17899 6010 Acked Write 8 Done Done Sent false
17900 6011 Acked Read 5 Done Done Sent false
17901 6012 Acked Write 4 Done Done Sent false
17902 6013 Acked Read 4 Done Done Sent false
17903 6014 Acked Flush 0 Done Done Sent false
17904 6015 Acked Read 7 Done Done Sent false
17905 6016 Acked Write 3 Done Done Sent false
17906 6017 Acked Read 10 Done Done Sent false
17907 6018 Acked Read 8 Done Done Sent false
17908 6019 Acked Read 7 Done Done Sent false
17909 6020 Acked Read 1 Done Done Sent false
17910 6021 Acked Read 7 Done Done Sent false
17911 6022 Acked Read 1 Done Done Sent false
17912 6023 Acked Write 5 Done Done Sent false
17913 6024 Acked Read 6 Done Done Sent false
17914 6025 Acked Write 1 Done Done Sent false
17915 6026 Acked Write 8 Done Done Sent false
17916 6027 Acked Flush 0 Done Done Sent false
17917 6028 Acked Read 9 Done Done Sent false
17918 6029 Acked Read 3 Done Done Sent false
17919 6030 Acked Read 7 Done Done Sent false
17920 6031 Acked Read 4 Done Done Sent false
17921 6032 Acked Write 6 Done Done Sent false
17922 6033 Acked Read 6 Done Done Sent false
17923 6034 Acked Write 7 Done Done Sent false
17924 6035 Acked Read 6 Done Done Sent false
17925 6036 Acked Read 7 Done Done Sent false
17926 6037 Acked Read 9 Done Done Sent false
17927 6038 Acked Read 1 Done Done Sent false
17928 6039 Acked Read 10 Done Done Sent false
17929 6040 Acked Read 8 Done Done Sent false
17930 6041 Acked Write 3 Done Done Sent false
17931 6042 Acked Read 6 Done Done Sent false
17932 6043 Acked Write 9 Done Done Sent false
17933 6044 Acked Read 3 Done Done Sent false
17934 6045 Acked Write 4 Done Done Sent false
17935 6046 Acked Write 9 Done Done Sent false
17936 6047 Acked Read 2 Done Done Sent false
17937 6048 Acked Read 9 Done Done Sent false
17938 6049 Acked Read 6 Done Done Sent false
17939 6050 Acked Read 8 Done Done Sent false
17940 6051 Acked Write 3 Done Done Sent false
17941 6052 Acked Write 4 Done Done Sent false
17942 6053 Acked Read 2 Done Done Sent false
17943 6054 Acked Write 6 Done Done Sent false
17944 6055 Acked Read 10 Done Done Sent false
17945 6056 Acked Write 8 Done Done Sent false
17946 6057 Acked Read 2 Done Done Sent false
17947 6058 Acked Write 4 Done Done Sent false
17948 6059 Acked Write 1 Done Done Sent false
17949 6060 Acked Read 10 Done Done Sent false
17950 6061 Acked Read 9 Done Done Sent false
17951 6062 Acked Write 9 Done Done Sent false
17952 6063 Acked Write 6 Done Done Sent false
17953 6064 Acked Write 10 Done Done Sent false
17954 6065 Acked Read 5 Done Done Sent false
17955 6066 Acked Write 3 Done Done Sent false
17956 6067 Acked Write 10 Done Done Sent false
17957 6068 Acked Read 6 Done Done Sent false
17958 6069 Acked Read 8 Done Done Sent false
17959 6070 Acked Read 2 Done Done Sent false
17960 6071 Acked Read 2 Done Done Sent false
17961 6072 Acked Read 7 Done Done Sent false
17962 6073 Acked Read 8 Done Done Sent false
17963 6074 Acked Write 2 Done Done Sent false
17964 6075 Acked Write 9 Done Done Sent false
17965 6076 Acked Read 3 Done Done Sent false
17966 6077 Acked Write 10 Done Done Sent false
17967 6078 Acked Write 8 Done Done Sent false
17968 6079 Acked Read 6 Done Done Sent false
17969 6080 Acked Write 3 Done Done Sent false
17970 6081 Acked Read 7 Done Done Sent false
17971 6082 Acked Write 4 Done Done Sent false
17972 6083 Acked Read 7 Done Done Sent false
17973 6084 Acked Write 1 Done Done Sent false
17974 6085 Acked Read 3 Done Done Sent false
17975 6086 Acked Write 4 Done Done Sent false
17976 6087 Acked Write 5 Done Done Sent false
17977 6088 Acked Read 7 Done Done Sent false
17978 6089 Acked Write 9 Done Done Sent false
17979 6090 Acked Read 9 Done Done Sent false
17980 6091 Acked Read 3 Done Done Sent false
17981 6092 Acked Read 10 Done Done Sent false
17982 6093 Acked Read 10 Done Done Sent false
17983 6094 Acked Write 8 Done Done Sent false
17984 6095 Acked Read 1 Done Done Sent false
17985 6096 Acked Read 6 Done Done Sent false
17986 6097 Acked Write 6 Done Done Sent false
17987 6098 Acked Read 3 Done Done Sent false
17988 6099 Acked Flush 0 Done Done Sent false
17989 6100 Acked Write 2 Done Done Sent false
17990 6101 Acked Read 8 Done Done Sent false
17991 6102 Acked Write 8 Done Done Sent false
17992 6103 Acked Read 3 Done Done Sent false
17993 6104 Acked Write 9 Done Done Sent false
17994 6105 Acked Read 10 Done Done Sent false
17995 6106 Acked Read 6 Done Done Sent false
17996 6107 Acked Read 2 Done Done Sent false
17997 6108 Acked Write 7 Done Done Sent false
17998 6109 Acked Read 6 Done Done Sent false
17999 6110 Acked Write 3 Done Done Sent false
18000 6111 Acked Flush 0 Done Done Sent false
18001 6112 Acked Read 8 Done Done Sent false
18002 6113 Acked Read 5 Done Done Sent false
18003 6114 Acked Write 7 Done Done Sent false
18004 6115 Acked Read 7 Done Done Sent false
18005 6116 Acked Read 7 Done Done Sent false
18006 6117 Acked Read 7 Done Done Sent false
18007 6118 Acked Write 4 Done Done Sent false
18008 6119 Acked Read 3 Done Done Sent false
18009 6120 Acked Read 7 Done Done Sent false
18010 6121 Acked Read 3 Done Done Sent false
18011 6122 Acked Write 10 Done Done Sent false
18012 6123 Acked Write 10 Done Done Sent false
18013 6124 Acked Write 7 Done Done Sent false
18014 6125 Acked Write 7 Done Done Sent false
18015 6126 Acked Write 2 Done Done Sent false
18016 6127 Acked Read 3 Done Done Sent false
18017 6128 Acked Read 3 Done Done Sent false
18018 6129 Acked Write 4 Done Done Sent false
18019 6130 Acked Write 4 Done Done Sent false
18020 6131 Acked Read 9 Done Done Sent false
18021 6132 Acked Flush 0 Done Done Sent false
18022 STATES DS:0 DS:1 DS:2 TOTAL
18023 Sent 0 0 302 302
18024 Done 302 302 0 604
18025 Skipped 0 0 0 0
18026 Error 0 0 0 0
18027 Last Flush: 6132 6132 5830
18028 Downstairs last five completed: 5830 5829 5828 5827 5826
18029 Upstairs last five completed: 6132 6131 6130 6129 6128
18030 CLIENT: Up:0 ds:302 act:2
180312024-11-26T22:08:18.317ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
180322024-11-26T22:08:18.317ZWARNcrucible: ds_connection connect to 127.0.0.1:8830 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
180332024-11-26T22:08:18.317ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
180342024-11-26T22:08:18.317ZWARNcrucible: downstairs task for 2 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
180352024-11-26T22:08:18.317ZINFOcrucible: client task is exiting = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
18036 ----------------------------------------------------------------
18037 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
18038 States: Active Active Offline
18039 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
18040 5831 Acked Write 3 Done Done Sent false
18041 5832 Acked Write 7 Done Done Sent false
18042 5833 Acked Read 10 Done Done Sent false
18043 5834 Acked Read 5 Done Done Sent false
18044 5835 Acked Write 6 Done Done Sent false
18045 5836 Acked Write 7 Done Done Sent false
18046 5837 Acked Read 1 Done Done Sent false
18047 5838 Acked Write 10 Done Done Sent false
18048 5839 Acked Read 1 Done Done Sent false
18049 5840 Acked Read 3 Done Done Sent false
18050 5841 Acked Read 9 Done Done Sent false
18051 5842 Acked Read 3 Done Done Sent false
18052 5843 Acked Read 1 Done Done Sent false
18053 5844 Acked Write 8 Done Done Sent false
18054 5845 Acked Read 10 Done Done Sent false
18055 5846 Acked Write 10 Done Done Sent false
18056 5847 Acked Read 3 Done Done Sent false
18057 5848 Acked Read 7 Done Done Sent false
18058 5849 Acked Read 10 Done Done Sent false
18059 5850 Acked Read 6 Done Done Sent false
18060 5851 Acked Write 1 Done Done Sent false
18061 5852 Acked Write 10 Done Done Sent false
18062 5853 Acked Read 5 Done Done Sent false
18063 5854 Acked Read 4 Done Done Sent false
18064 5855 Acked Write 3 Done Done Sent false
18065 5856 Acked Read 3 Done Done Sent false
18066 5857 Acked Read 5 Done Done Sent false
18067 5858 Acked Write 4 Done Done Sent false
18068 5859 Acked Read 6 Done Done Sent false
18069 5860 Acked Read 5 Done Done Sent false
18070 5861 Acked Read 10 Done Done Sent false
18071 5862 Acked Flush 0 Done Done Sent false
18072 5863 Acked Write 1 Done Done Sent false
18073 5864 Acked Read 8 Done Done Sent false
18074 5865 Acked Write 4 Done Done Sent false
18075 5866 Acked Read 6 Done Done Sent false
18076 5867 Acked Write 8 Done Done Sent false
18077 5868 Acked Read 9 Done Done Sent false
18078 5869 Acked Read 8 Done Done Sent false
18079 5870 Acked Flush 0 Done Done Sent false
18080 5871 Acked Read 1 Done Done Sent false
18081 5872 Acked Write 6 Done Done Sent false
18082 5873 Acked Read 1 Done Done Sent false
18083 5874 Acked Write 7 Done Done Sent false
18084 5875 Acked Read 3 Done Done Sent false
18085 5876 Acked Write 10 Done Done Sent false
18086 5877 Acked Read 3 Done Done Sent false
18087 5878 Acked Read 2 Done Done Sent false
18088 5879 Acked Write 10 Done Done Sent false
18089 5880 Acked Write 7 Done Done Sent false
18090 5881 Acked Read 8 Done Done Sent false
18091 5882 Acked Write 9 Done Done Sent false
18092 5883 Acked Read 5 Done Done Sent false
18093 5884 Acked Read 1 Done Done Sent false
18094 5885 Acked Write 1 Done Done Sent false
18095 5886 Acked Read 5 Done Done Sent false
18096 5887 Acked Read 8 Done Done Sent false
18097 5888 Acked Read 10 Done Done Sent false
18098 5889 Acked Write 6 Done Done Sent false
18099 5890 Acked Flush 0 Done Done Sent false
18100 5891 Acked Read 5 Done Done Sent false
18101 5892 Acked Read 3 Done Done Sent false
18102 5893 Acked Read 10 Done Done Sent false
18103 5894 Acked Write 7 Done Done Sent false
18104 5895 Acked Write 8 Done Done Sent false
18105 5896 Acked Write 10 Done Done Sent false
18106 5897 Acked Write 2 Done Done Sent false
18107 5898 Acked Write 2 Done Done Sent false
18108 5899 Acked Write 5 Done Done Sent false
18109 5900 Acked Write 6 Done Done Sent false
18110 5901 Acked Read 4 Done Done Sent false
18111 5902 Acked Read 6 Done Done Sent false
18112 5903 Acked Write 3 Done Done Sent false
18113 5904 Acked Write 9 Done Done Sent false
18114 5905 Acked Read 9 Done Done Sent false
18115 5906 Acked Read 7 Done Done Sent false
18116 5907 Acked Read 4 Done Done Sent false
18117 5908 Acked Read 10 Done Done Sent false
18118 5909 Acked Read 3 Done Done Sent false
18119 5910 Acked Read 4 Done Done Sent false
18120 5911 Acked Write 1 Done Done Sent false
18121 5912 Acked Write 10 Done Done Sent false
18122 5913 Acked Read 2 Done Done Sent false
18123 5914 Acked Read 8 Done Done Sent false
18124 5915 Acked Read 10 Done Done Sent false
18125 5916 Acked Flush 0 Done Done Sent false
18126 5917 Acked Read 9 Done Done Sent false
18127 5918 Acked Read 4 Done Done Sent false
18128 5919 Acked Read 2 Done Done Sent false
18129 5920 Acked Write 8 Done Done Sent false
18130 5921 Acked Read 3 Done Done Sent false
18131 5922 Acked Read 3 Done Done Sent false
18132 5923 Acked Read 5 Done Done Sent false
18133 5924 Acked Write 5 Done Done Sent false
18134 5925 Acked Flush 0 Done Done Sent false
18135 5926 Acked Read 4 Done Done Sent false
18136 5927 Acked Write 7 Done Done Sent false
18137 5928 Acked Flush 0 Done Done Sent false
18138 5929 Acked Read 8 Done Done Sent false
18139 5930 Acked Write 5 Done Done Sent false
18140 5931 Acked Read 5 Done Done Sent false
18141 5932 Acked Flush 0 Done Done Sent false
18142 5933 Acked Read 5 Done Done Sent false
18143 5934 Acked Flush 0 Done Done Sent false
18144 5935 Acked Read 5 Done Done Sent false
18145 5936 Acked Read 2 Done Done Sent false
18146 5937 Acked Read 5 Done Done Sent false
18147 5938 Acked Read 7 Done Done Sent false
18148 5939 Acked Flush 0 Done Done Sent false
18149 5940 Acked Read 3 Done Done Sent false
18150 5941 Acked Read 10 Done Done Sent false
18151 5942 Acked Write 9 Done Done Sent false
18152 5943 Acked Read 5 Done Done Sent false
18153 5944 Acked Write 8 Done Done Sent false
18154 5945 Acked Flush 0 Done Done Sent false
18155 5946 Acked Write 5 Done Done Sent false
18156 5947 Acked Read 5 Done Done Sent false
18157 5948 Acked Read 5 Done Done Sent false
18158 5949 Acked Read 1 Done Done Sent false
18159 5950 Acked Read 2 Done Done Sent false
18160 5951 Acked Write 5 Done Done Sent false
18161 5952 Acked Read 6 Done Done Sent false
18162 5953 Acked Read 10 Done Done Sent false
18163 5954 Acked Write 1 Done Done Sent false
18164 5955 Acked Read 3 Done Done Sent false
18165 5956 Acked Flush 0 Done Done Sent false
18166 5957 Acked Flush 0 Done Done Sent false
18167 5958 Acked Read 9 Done Done Sent false
18168 5959 Acked Write 3 Done Done Sent false
18169 5960 Acked Read 6 Done Done Sent false
18170 5961 Acked Read 7 Done Done Sent false
18171 5962 Acked Read 6 Done Done Sent false
18172 5963 Acked Write 6 Done Done Sent false
18173 5964 Acked Read 6 Done Done Sent false
18174 5965 Acked Read 1 Done Done Sent false
18175 5966 Acked Read 2 Done Done Sent false
18176 5967 Acked Write 2 Done Done Sent false
18177 5968 Acked Read 8 Done Done Sent false
18178 5969 Acked Write 8 Done Done Sent false
18179 5970 Acked Read 1 Done Done Sent false
18180 5971 Acked Read 1 Done Done Sent false
18181 5972 Acked Write 10 Done Done Sent false
18182 5973 Acked Read 7 Done Done Sent false
18183 5974 Acked Write 1 Done Done Sent false
18184 5975 Acked Write 7 Done Done Sent false
18185 5976 Acked Flush 0 Done Done Sent false
18186 5977 Acked Read 7 Done Done Sent false
18187 5978 Acked Flush 0 Done Done Sent false
18188 5979 Acked Read 3 Done Done Sent false
18189 5980 Acked Read 2 Done Done Sent false
18190 5981 Acked Write 5 Done Done Sent false
18191 5982 Acked Read 3 Done Done Sent false
18192 5983 Acked Read 1 Done Done Sent false
18193 5984 Acked Read 5 Done Done Sent false
18194 5985 Acked Write 3 Done Done Sent false
18195 5986 Acked Read 7 Done Done Sent false
18196 5987 Acked Write 9 Done Done Sent false
18197 5988 Acked Read 6 Done Done Sent false
18198 5989 Acked Write 1 Done Done Sent false
18199 5990 Acked Flush 0 Done Done Sent false
18200 5991 Acked Write 2 Done Done Sent false
18201 5992 Acked Read 10 Done Done Sent false
18202 5993 Acked Read 4 Done Done Sent false
18203 5994 Acked Write 1 Done Done Sent false
18204 5995 Acked Write 8 Done Done Sent false
18205 5996 Acked Read 10 Done Done Sent false
18206 5997 Acked Write 5 Done Done Sent false
18207 5998 Acked Write 6 Done Done Sent false
18208 5999 Acked Write 2 Done Done Sent false
18209 6000 Acked Write 6 Done Done Sent false
18210 6001 Acked Read 8 Done Done Sent false
18211 6002 Acked Read 7 Done Done Sent false
18212 6003 Acked Read 6 Done Done Sent false
18213 6004 Acked Read 3 Done Done Sent false
18214 6005 Acked Read 5 Done Done Sent false
18215 6006 Acked Read 3 Done Done Sent false
18216 6007 Acked Write 7 Done Done Sent false
18217 6008 Acked Read 7 Done Done Sent false
18218 6009 Acked Write 3 Done Done Sent false
18219 6010 Acked Write 8 Done Done Sent false
18220 6011 Acked Read 5 Done Done Sent false
18221 6012 Acked Write 4 Done Done Sent false
18222 6013 Acked Read 4 Done Done Sent false
18223 6014 Acked Flush 0 Done Done Sent false
18224 6015 Acked Read 7 Done Done Sent false
18225 6016 Acked Write 3 Done Done Sent false
18226 6017 Acked Read 10 Done Done Sent false
18227 6018 Acked Read 8 Done Done Sent false
18228 6019 Acked Read 7 Done Done Sent false
18229 6020 Acked Read 1 Done Done Sent false
18230 6021 Acked Read 7 Done Done Sent false
18231 6022 Acked Read 1 Done Done Sent false
18232 6023 Acked Write 5 Done Done Sent false
18233 6024 Acked Read 6 Done Done Sent false
18234 6025 Acked Write 1 Done Done Sent false
18235 6026 Acked Write 8 Done Done Sent false
18236 6027 Acked Flush 0 Done Done Sent false
18237 6028 Acked Read 9 Done Done Sent false
18238 6029 Acked Read 3 Done Done Sent false
18239 6030 Acked Read 7 Done Done Sent false
18240 6031 Acked Read 4 Done Done Sent false
18241 6032 Acked Write 6 Done Done Sent false
18242 6033 Acked Read 6 Done Done Sent false
18243 6034 Acked Write 7 Done Done Sent false
18244 6035 Acked Read 6 Done Done Sent false
18245 6036 Acked Read 7 Done Done Sent false
18246 6037 Acked Read 9 Done Done Sent false
18247 6038 Acked Read 1 Done Done Sent false
18248 6039 Acked Read 10 Done Done Sent false
18249 6040 Acked Read 8 Done Done Sent false
18250 6041 Acked Write 3 Done Done Sent false
18251 6042 Acked Read 6 Done Done Sent false
18252 6043 Acked Write 9 Done Done Sent false
18253 6044 Acked Read 3 Done Done Sent false
18254 6045 Acked Write 4 Done Done Sent false
18255 6046 Acked Write 9 Done Done Sent false
18256 6047 Acked Read 2 Done Done Sent false
18257 6048 Acked Read 9 Done Done Sent false
18258 6049 Acked Read 6 Done Done Sent false
18259 6050 Acked Read 8 Done Done Sent false
18260 6051 Acked Write 3 Done Done Sent false
18261 6052 Acked Write 4 Done Done Sent false
18262 6053 Acked Read 2 Done Done Sent false
18263 6054 Acked Write 6 Done Done Sent false
18264 6055 Acked Read 10 Done Done Sent false
18265 6056 Acked Write 8 Done Done Sent false
18266 6057 Acked Read 2 Done Done Sent false
18267 6058 Acked Write 4 Done Done Sent false
18268 6059 Acked Write 1 Done Done Sent false
18269 6060 Acked Read 10 Done Done Sent false
18270 6061 Acked Read 9 Done Done Sent false
18271 6062 Acked Write 9 Done Done Sent false
18272 6063 Acked Write 6 Done Done Sent false
18273 6064 Acked Write 10 Done Done Sent false
18274 6065 Acked Read 5 Done Done Sent false
18275 6066 Acked Write 3 Done Done Sent false
18276 6067 Acked Write 10 Done Done Sent false
18277 6068 Acked Read 6 Done Done Sent false
18278 6069 Acked Read 8 Done Done Sent false
18279 6070 Acked Read 2 Done Done Sent false
18280 6071 Acked Read 2 Done Done Sent false
18281 6072 Acked Read 7 Done Done Sent false
18282 6073 Acked Read 8 Done Done Sent false
18283 6074 Acked Write 2 Done Done Sent false
18284 6075 Acked Write 9 Done Done Sent false
18285 6076 Acked Read 3 Done Done Sent false
18286 6077 Acked Write 10 Done Done Sent false
18287 6078 Acked Write 8 Done Done Sent false
18288 6079 Acked Read 6 Done Done Sent false
18289 6080 Acked Write 3 Done Done Sent false
18290 6081 Acked Read 7 Done Done Sent false
18291 6082 Acked Write 4 Done Done Sent false
18292 6083 Acked Read 7 Done Done Sent false
18293 6084 Acked Write 1 Done Done Sent false
18294 6085 Acked Read 3 Done Done Sent false
18295 6086 Acked Write 4 Done Done Sent false
18296 6087 Acked Write 5 Done Done Sent false
18297 6088 Acked Read 7 Done Done Sent false
18298 6089 Acked Write 9 Done Done Sent false
18299 6090 Acked Read 9 Done Done Sent false
18300 6091 Acked Read 3 Done Done Sent false
18301 6092 Acked Read 10 Done Done Sent false
18302 6093 Acked Read 10 Done Done Sent false
18303 6094 Acked Write 8 Done Done Sent false
18304 6095 Acked Read 1 Done Done Sent false
18305 6096 Acked Read 6 Done Done Sent false
18306 6097 Acked Write 6 Done Done Sent false
18307 6098 Acked Read 3 Done Done Sent false
18308 6099 Acked Flush 0 Done Done Sent false
18309 6100 Acked Write 2 Done Done Sent false
18310 6101 Acked Read 8 Done Done Sent false
18311 6102 Acked Write 8 Done Done Sent false
18312 6103 Acked Read 3 Done Done Sent false
18313 6104 Acked Write 9 Done Done Sent false
18314 6105 Acked Read 10 Done Done Sent false
18315 6106 Acked Read 6 Done Done Sent false
18316 6107 Acked Read 2 Done Done Sent false
18317 6108 Acked Write 7 Done Done Sent false
18318 6109 Acked Read 6 Done Done Sent false
18319 6110 Acked Write 3 Done Done Sent false
18320 6111 Acked Flush 0 Done Done Sent false
18321 6112 Acked Read 8 Done Done Sent false
18322 6113 Acked Read 5 Done Done Sent false
18323 6114 Acked Write 7 Done Done Sent false
18324 6115 Acked Read 7 Done Done Sent false
18325 6116 Acked Read 7 Done Done Sent false
18326 6117 Acked Read 7 Done Done Sent false
18327 6118 Acked Write 4 Done Done Sent false
18328 6119 Acked Read 3 Done Done Sent false
18329 6120 Acked Read 7 Done Done Sent false
18330 6121 Acked Read 3 Done Done Sent false
18331 6122 Acked Write 10 Done Done Sent false
18332 6123 Acked Write 10 Done Done Sent false
18333 6124 Acked Write 7 Done Done Sent false
18334 6125 Acked Write 7 Done Done Sent false
18335 6126 Acked Write 2 Done Done Sent false
18336 6127 Acked Read 3 Done Done Sent false
18337 6128 Acked Read 3 Done Done Sent false
18338 6129 Acked Write 4 Done Done Sent false
18339 6130 Acked Write 4 Done Done Sent false
18340 6131 Acked Read 9 Done Done Sent false
18341 6132 Acked Flush 0 Done Done Sent false
18342 STATES DS:0 DS:1 DS:2 TOTAL
18343 Sent 0 0 302 302
18344 Done 302 302 0 604
18345 Skipped 0 0 0 0
18346 Error 0 0 0 0
18347 Last Flush: 6132 6132 5830
18348 Downstairs last five completed: 5830 5829 5828 5827 5826
18349 Upstairs last five completed: 6132 6131 6130 6129 6128
18350 CLIENT: Up:0 ds:302 act:2
18351 ----------------------------------------------------------------
18352 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
18353 States: Active Active Offline
18354 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
18355 5831 Acked Write 3 Done Done Sent false
18356 5832 Acked Write 7 Done Done Sent false
18357 5833 Acked Read 10 Done Done Sent false
18358 5834 Acked Read 5 Done Done Sent false
18359 5835 Acked Write 6 Done Done Sent false
18360 5836 Acked Write 7 Done Done Sent false
18361 5837 Acked Read 1 Done Done Sent false
18362 5838 Acked Write 10 Done Done Sent false
18363 5839 Acked Read 1 Done Done Sent false
18364 5840 Acked Read 3 Done Done Sent false
18365 5841 Acked Read 9 Done Done Sent false
18366 5842 Acked Read 3 Done Done Sent false
18367 5843 Acked Read 1 Done Done Sent false
18368 5844 Acked Write 8 Done Done Sent false
18369 5845 Acked Read 10 Done Done Sent false
18370 5846 Acked Write 10 Done Done Sent false
18371 5847 Acked Read 3 Done Done Sent false
18372 5848 Acked Read 7 Done Done Sent false
18373 5849 Acked Read 10 Done Done Sent false
18374 5850 Acked Read 6 Done Done Sent false
18375 5851 Acked Write 1 Done Done Sent false
18376 5852 Acked Write 10 Done Done Sent false
18377 5853 Acked Read 5 Done Done Sent false
18378 5854 Acked Read 4 Done Done Sent false
18379 5855 Acked Write 3 Done Done Sent false
18380 5856 Acked Read 3 Done Done Sent false
18381 5857 Acked Read 5 Done Done Sent false
18382 5858 Acked Write 4 Done Done Sent false
18383 5859 Acked Read 6 Done Done Sent false
18384 5860 Acked Read 5 Done Done Sent false
18385 5861 Acked Read 10 Done Done Sent false
18386 5862 Acked Flush 0 Done Done Sent false
18387 5863 Acked Write 1 Done Done Sent false
18388 5864 Acked Read 8 Done Done Sent false
18389 5865 Acked Write 4 Done Done Sent false
18390 5866 Acked Read 6 Done Done Sent false
18391 5867 Acked Write 8 Done Done Sent false
18392 5868 Acked Read 9 Done Done Sent false
18393 5869 Acked Read 8 Done Done Sent false
18394 5870 Acked Flush 0 Done Done Sent false
18395 5871 Acked Read 1 Done Done Sent false
18396 5872 Acked Write 6 Done Done Sent false
18397 5873 Acked Read 1 Done Done Sent false
18398 5874 Acked Write 7 Done Done Sent false
18399 5875 Acked Read 3 Done Done Sent false
18400 5876 Acked Write 10 Done Done Sent false
18401 5877 Acked Read 3 Done Done Sent false
18402 5878 Acked Read 2 Done Done Sent false
18403 5879 Acked Write 10 Done Done Sent false
18404 5880 Acked Write 7 Done Done Sent false
18405 5881 Acked Read 8 Done Done Sent false
18406 5882 Acked Write 9 Done Done Sent false
18407 5883 Acked Read 5 Done Done Sent false
18408 5884 Acked Read 1 Done Done Sent false
18409 5885 Acked Write 1 Done Done Sent false
18410 5886 Acked Read 5 Done Done Sent false
18411 5887 Acked Read 8 Done Done Sent false
18412 5888 Acked Read 10 Done Done Sent false
18413 5889 Acked Write 6 Done Done Sent false
18414 5890 Acked Flush 0 Done Done Sent false
18415 5891 Acked Read 5 Done Done Sent false
18416 5892 Acked Read 3 Done Done Sent false
18417 5893 Acked Read 10 Done Done Sent false
18418 5894 Acked Write 7 Done Done Sent false
18419 5895 Acked Write 8 Done Done Sent false
18420 5896 Acked Write 10 Done Done Sent false
18421 5897 Acked Write 2 Done Done Sent false
18422 5898 Acked Write 2 Done Done Sent false
18423 5899 Acked Write 5 Done Done Sent false
18424 5900 Acked Write 6 Done Done Sent false
18425 5901 Acked Read 4 Done Done Sent false
18426 5902 Acked Read 6 Done Done Sent false
18427 5903 Acked Write 3 Done Done Sent false
18428 5904 Acked Write 9 Done Done Sent false
18429 5905 Acked Read 9 Done Done Sent false
18430 5906 Acked Read 7 Done Done Sent false
18431 5907 Acked Read 4 Done Done Sent false
18432 5908 Acked Read 10 Done Done Sent false
18433 5909 Acked Read 3 Done Done Sent false
18434 5910 Acked Read 4 Done Done Sent false
18435 5911 Acked Write 1 Done Done Sent false
18436 5912 Acked Write 10 Done Done Sent false
18437 5913 Acked Read 2 Done Done Sent false
18438 5914 Acked Read 8 Done Done Sent false
18439 5915 Acked Read 10 Done Done Sent false
18440 5916 Acked Flush 0 Done Done Sent false
18441 5917 Acked Read 9 Done Done Sent false
18442 5918 Acked Read 4 Done Done Sent false
18443 5919 Acked Read 2 Done Done Sent false
18444 5920 Acked Write 8 Done Done Sent false
18445 5921 Acked Read 3 Done Done Sent false
18446 5922 Acked Read 3 Done Done Sent false
18447 5923 Acked Read 5 Done Done Sent false
18448 5924 Acked Write 5 Done Done Sent false
18449 5925 Acked Flush 0 Done Done Sent false
18450 5926 Acked Read 4 Done Done Sent false
18451 5927 Acked Write 7 Done Done Sent false
18452 5928 Acked Flush 0 Done Done Sent false
18453 5929 Acked Read 8 Done Done Sent false
18454 5930 Acked Write 5 Done Done Sent false
18455 5931 Acked Read 5 Done Done Sent false
18456 5932 Acked Flush 0 Done Done Sent false
18457 5933 Acked Read 5 Done Done Sent false
18458 5934 Acked Flush 0 Done Done Sent false
18459 5935 Acked Read 5 Done Done Sent false
18460 5936 Acked Read 2 Done Done Sent false
18461 5937 Acked Read 5 Done Done Sent false
18462 5938 Acked Read 7 Done Done Sent false
18463 5939 Acked Flush 0 Done Done Sent false
18464 5940 Acked Read 3 Done Done Sent false
18465 5941 Acked Read 10 Done Done Sent false
18466 5942 Acked Write 9 Done Done Sent false
18467 5943 Acked Read 5 Done Done Sent false
18468 5944 Acked Write 8 Done Done Sent false
18469 5945 Acked Flush 0 Done Done Sent false
18470 5946 Acked Write 5 Done Done Sent false
18471 5947 Acked Read 5 Done Done Sent false
18472 5948 Acked Read 5 Done Done Sent false
18473 5949 Acked Read 1 Done Done Sent false
18474 5950 Acked Read 2 Done Done Sent false
18475 5951 Acked Write 5 Done Done Sent false
18476 5952 Acked Read 6 Done Done Sent false
18477 5953 Acked Read 10 Done Done Sent false
18478 5954 Acked Write 1 Done Done Sent false
18479 5955 Acked Read 3 Done Done Sent false
18480 5956 Acked Flush 0 Done Done Sent false
18481 5957 Acked Flush 0 Done Done Sent false
18482 5958 Acked Read 9 Done Done Sent false
18483 5959 Acked Write 3 Done Done Sent false
18484 5960 Acked Read 6 Done Done Sent false
18485 5961 Acked Read 7 Done Done Sent false
18486 5962 Acked Read 6 Done Done Sent false
18487 5963 Acked Write 6 Done Done Sent false
18488 5964 Acked Read 6 Done Done Sent false
18489 5965 Acked Read 1 Done Done Sent false
18490 5966 Acked Read 2 Done Done Sent false
18491 5967 Acked Write 2 Done Done Sent false
18492 5968 Acked Read 8 Done Done Sent false
18493 5969 Acked Write 8 Done Done Sent false
18494 5970 Acked Read 1 Done Done Sent false
18495 5971 Acked Read 1 Done Done Sent false
18496 5972 Acked Write 10 Done Done Sent false
18497 5973 Acked Read 7 Done Done Sent false
18498 5974 Acked Write 1 Done Done Sent false
18499 5975 Acked Write 7 Done Done Sent false
18500 5976 Acked Flush 0 Done Done Sent false
18501 5977 Acked Read 7 Done Done Sent false
18502 5978 Acked Flush 0 Done Done Sent false
18503 5979 Acked Read 3 Done Done Sent false
18504 5980 Acked Read 2 Done Done Sent false
18505 5981 Acked Write 5 Done Done Sent false
18506 5982 Acked Read 3 Done Done Sent false
18507 5983 Acked Read 1 Done Done Sent false
18508 5984 Acked Read 5 Done Done Sent false
18509 5985 Acked Write 3 Done Done Sent false
18510 5986 Acked Read 7 Done Done Sent false
18511 5987 Acked Write 9 Done Done Sent false
18512 5988 Acked Read 6 Done Done Sent false
18513 5989 Acked Write 1 Done Done Sent false
18514 5990 Acked Flush 0 Done Done Sent false
18515 5991 Acked Write 2 Done Done Sent false
18516 5992 Acked Read 10 Done Done Sent false
18517 5993 Acked Read 4 Done Done Sent false
18518 5994 Acked Write 1 Done Done Sent false
18519 5995 Acked Write 8 Done Done Sent false
18520 5996 Acked Read 10 Done Done Sent false
18521 5997 Acked Write 5 Done Done Sent false
18522 5998 Acked Write 6 Done Done Sent false
18523 5999 Acked Write 2 Done Done Sent false
18524 6000 Acked Write 6 Done Done Sent false
18525 6001 Acked Read 8 Done Done Sent false
18526 6002 Acked Read 7 Done Done Sent false
18527 6003 Acked Read 6 Done Done Sent false
18528 6004 Acked Read 3 Done Done Sent false
18529 6005 Acked Read 5 Done Done Sent false
18530 6006 Acked Read 3 Done Done Sent false
18531 6007 Acked Write 7 Done Done Sent false
18532 6008 Acked Read 7 Done Done Sent false
18533 6009 Acked Write 3 Done Done Sent false
18534 6010 Acked Write 8 Done Done Sent false
18535 6011 Acked Read 5 Done Done Sent false
18536 6012 Acked Write 4 Done Done Sent false
18537 6013 Acked Read 4 Done Done Sent false
18538 6014 Acked Flush 0 Done Done Sent false
18539 6015 Acked Read 7 Done Done Sent false
18540 6016 Acked Write 3 Done Done Sent false
18541 6017 Acked Read 10 Done Done Sent false
18542 6018 Acked Read 8 Done Done Sent false
18543 6019 Acked Read 7 Done Done Sent false
18544 6020 Acked Read 1 Done Done Sent false
18545 6021 Acked Read 7 Done Done Sent false
18546 6022 Acked Read 1 Done Done Sent false
18547 6023 Acked Write 5 Done Done Sent false
18548 6024 Acked Read 6 Done Done Sent false
18549 6025 Acked Write 1 Done Done Sent false
18550 6026 Acked Write 8 Done Done Sent false
18551 6027 Acked Flush 0 Done Done Sent false
18552 6028 Acked Read 9 Done Done Sent false
18553 6029 Acked Read 3 Done Done Sent false
18554 6030 Acked Read 7 Done Done Sent false
18555 6031 Acked Read 4 Done Done Sent false
18556 6032 Acked Write 6 Done Done Sent false
18557 6033 Acked Read 6 Done Done Sent false
18558 6034 Acked Write 7 Done Done Sent false
18559 6035 Acked Read 6 Done Done Sent false
18560 6036 Acked Read 7 Done Done Sent false
18561 6037 Acked Read 9 Done Done Sent false
18562 6038 Acked Read 1 Done Done Sent false
18563 6039 Acked Read 10 Done Done Sent false
18564 6040 Acked Read 8 Done Done Sent false
18565 6041 Acked Write 3 Done Done Sent false
18566 6042 Acked Read 6 Done Done Sent false
18567 6043 Acked Write 9 Done Done Sent false
18568 6044 Acked Read 3 Done Done Sent false
18569 6045 Acked Write 4 Done Done Sent false
18570 6046 Acked Write 9 Done Done Sent false
18571 6047 Acked Read 2 Done Done Sent false
18572 6048 Acked Read 9 Done Done Sent false
18573 6049 Acked Read 6 Done Done Sent false
18574 6050 Acked Read 8 Done Done Sent false
18575 6051 Acked Write 3 Done Done Sent false
18576 6052 Acked Write 4 Done Done Sent false
18577 6053 Acked Read 2 Done Done Sent false
18578 6054 Acked Write 6 Done Done Sent false
18579 6055 Acked Read 10 Done Done Sent false
18580 6056 Acked Write 8 Done Done Sent false
18581 6057 Acked Read 2 Done Done Sent false
18582 6058 Acked Write 4 Done Done Sent false
18583 6059 Acked Write 1 Done Done Sent false
18584 6060 Acked Read 10 Done Done Sent false
18585 6061 Acked Read 9 Done Done Sent false
18586 6062 Acked Write 9 Done Done Sent false
18587 6063 Acked Write 6 Done Done Sent false
18588 6064 Acked Write 10 Done Done Sent false
18589 6065 Acked Read 5 Done Done Sent false
18590 6066 Acked Write 3 Done Done Sent false
18591 6067 Acked Write 10 Done Done Sent false
18592 6068 Acked Read 6 Done Done Sent false
18593 6069 Acked Read 8 Done Done Sent false
18594 6070 Acked Read 2 Done Done Sent false
18595 6071 Acked Read 2 Done Done Sent false
18596 6072 Acked Read 7 Done Done Sent false
18597 6073 Acked Read 8 Done Done Sent false
18598 6074 Acked Write 2 Done Done Sent false
18599 6075 Acked Write 9 Done Done Sent false
18600 6076 Acked Read 3 Done Done Sent false
18601 6077 Acked Write 10 Done Done Sent false
18602 6078 Acked Write 8 Done Done Sent false
18603 6079 Acked Read 6 Done Done Sent false
18604 6080 Acked Write 3 Done Done Sent false
18605 6081 Acked Read 7 Done Done Sent false
18606 6082 Acked Write 4 Done Done Sent false
18607 6083 Acked Read 7 Done Done Sent false
18608 6084 Acked Write 1 Done Done Sent false
18609 6085 Acked Read 3 Done Done Sent false
18610 6086 Acked Write 4 Done Done Sent false
18611 6087 Acked Write 5 Done Done Sent false
18612 6088 Acked Read 7 Done Done Sent false
18613 6089 Acked Write 9 Done Done Sent false
18614 6090 Acked Read 9 Done Done Sent false
18615 6091 Acked Read 3 Done Done Sent false
18616 6092 Acked Read 10 Done Done Sent false
18617 6093 Acked Read 10 Done Done Sent false
18618 6094 Acked Write 8 Done Done Sent false
18619 6095 Acked Read 1 Done Done Sent false
18620 6096 Acked Read 6 Done Done Sent false
18621 6097 Acked Write 6 Done Done Sent false
18622 6098 Acked Read 3 Done Done Sent false
18623 6099 Acked Flush 0 Done Done Sent false
18624 6100 Acked Write 2 Done Done Sent false
18625 6101 Acked Read 8 Done Done Sent false
18626 6102 Acked Write 8 Done Done Sent false
18627 6103 Acked Read 3 Done Done Sent false
18628 6104 Acked Write 9 Done Done Sent false
18629 6105 Acked Read 10 Done Done Sent false
18630 6106 Acked Read 6 Done Done Sent false
18631 6107 Acked Read 2 Done Done Sent false
18632 6108 Acked Write 7 Done Done Sent false
18633 6109 Acked Read 6 Done Done Sent false
18634 6110 Acked Write 3 Done Done Sent false
18635 6111 Acked Flush 0 Done Done Sent false
18636 6112 Acked Read 8 Done Done Sent false
18637 6113 Acked Read 5 Done Done Sent false
18638 6114 Acked Write 7 Done Done Sent false
18639 6115 Acked Read 7 Done Done Sent false
18640 6116 Acked Read 7 Done Done Sent false
18641 6117 Acked Read 7 Done Done Sent false
18642 6118 Acked Write 4 Done Done Sent false
18643 6119 Acked Read 3 Done Done Sent false
18644 6120 Acked Read 7 Done Done Sent false
18645 6121 Acked Read 3 Done Done Sent false
18646 6122 Acked Write 10 Done Done Sent false
18647 6123 Acked Write 10 Done Done Sent false
18648 6124 Acked Write 7 Done Done Sent false
18649 6125 Acked Write 7 Done Done Sent false
18650 6126 Acked Write 2 Done Done Sent false
18651 6127 Acked Read 3 Done Done Sent false
18652 6128 Acked Read 3 Done Done Sent false
18653 6129 Acked Write 4 Done Done Sent false
18654 6130 Acked Write 4 Done Done Sent false
18655 6131 Acked Read 9 Done Done Sent false
18656 6132 Acked Flush 0 Done Done Sent false
18657 STATES DS:0 DS:1 DS:2 TOTAL
18658 Sent 0 0 302 302
18659 Done 302 302 0 604
18660 Skipped 0 0 0 0
18661 Error 0 0 0 0
18662 Last Flush: 6132 6132 5830
18663 Downstairs last five completed: 5830 5829 5828 5827 5826
18664 Upstairs last five completed: 6132 6131 6130 6129 6128
18665 CLIENT: Up:0 ds:302 act:2
186662024-11-26T22:08:28.319ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
186672024-11-26T22:08:28.319ZWARNcrucible: ds_connection connect to 127.0.0.1:8830 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
186682024-11-26T22:08:28.319ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
186692024-11-26T22:08:28.319ZWARNcrucible: downstairs task for 2 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
186702024-11-26T22:08:28.319ZINFOcrucible: client task is exiting = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
18671 ----------------------------------------------------------------
18672 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
18673 States: Active Active Offline
18674 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
18675 5831 Acked Write 3 Done Done Sent false
18676 5832 Acked Write 7 Done Done Sent false
18677 5833 Acked Read 10 Done Done Sent false
18678 5834 Acked Read 5 Done Done Sent false
18679 5835 Acked Write 6 Done Done Sent false
18680 5836 Acked Write 7 Done Done Sent false
18681 5837 Acked Read 1 Done Done Sent false
18682 5838 Acked Write 10 Done Done Sent false
18683 5839 Acked Read 1 Done Done Sent false
18684 5840 Acked Read 3 Done Done Sent false
18685 5841 Acked Read 9 Done Done Sent false
18686 5842 Acked Read 3 Done Done Sent false
18687 5843 Acked Read 1 Done Done Sent false
18688 5844 Acked Write 8 Done Done Sent false
18689 5845 Acked Read 10 Done Done Sent false
18690 5846 Acked Write 10 Done Done Sent false
18691 5847 Acked Read 3 Done Done Sent false
18692 5848 Acked Read 7 Done Done Sent false
18693 5849 Acked Read 10 Done Done Sent false
18694 5850 Acked Read 6 Done Done Sent false
18695 5851 Acked Write 1 Done Done Sent false
18696 5852 Acked Write 10 Done Done Sent false
18697 5853 Acked Read 5 Done Done Sent false
18698 5854 Acked Read 4 Done Done Sent false
18699 5855 Acked Write 3 Done Done Sent false
18700 5856 Acked Read 3 Done Done Sent false
18701 5857 Acked Read 5 Done Done Sent false
18702 5858 Acked Write 4 Done Done Sent false
18703 5859 Acked Read 6 Done Done Sent false
18704 5860 Acked Read 5 Done Done Sent false
18705 5861 Acked Read 10 Done Done Sent false
18706 5862 Acked Flush 0 Done Done Sent false
18707 5863 Acked Write 1 Done Done Sent false
18708 5864 Acked Read 8 Done Done Sent false
18709 5865 Acked Write 4 Done Done Sent false
18710 5866 Acked Read 6 Done Done Sent false
18711 5867 Acked Write 8 Done Done Sent false
18712 5868 Acked Read 9 Done Done Sent false
18713 5869 Acked Read 8 Done Done Sent false
18714 5870 Acked Flush 0 Done Done Sent false
18715 5871 Acked Read 1 Done Done Sent false
18716 5872 Acked Write 6 Done Done Sent false
18717 5873 Acked Read 1 Done Done Sent false
18718 5874 Acked Write 7 Done Done Sent false
18719 5875 Acked Read 3 Done Done Sent false
18720 5876 Acked Write 10 Done Done Sent false
18721 5877 Acked Read 3 Done Done Sent false
18722 5878 Acked Read 2 Done Done Sent false
18723 5879 Acked Write 10 Done Done Sent false
18724 5880 Acked Write 7 Done Done Sent false
18725 5881 Acked Read 8 Done Done Sent false
18726 5882 Acked Write 9 Done Done Sent false
18727 5883 Acked Read 5 Done Done Sent false
18728 5884 Acked Read 1 Done Done Sent false
18729 5885 Acked Write 1 Done Done Sent false
18730 5886 Acked Read 5 Done Done Sent false
18731 5887 Acked Read 8 Done Done Sent false
18732 5888 Acked Read 10 Done Done Sent false
18733 5889 Acked Write 6 Done Done Sent false
18734 5890 Acked Flush 0 Done Done Sent false
18735 5891 Acked Read 5 Done Done Sent false
18736 5892 Acked Read 3 Done Done Sent false
18737 5893 Acked Read 10 Done Done Sent false
18738 5894 Acked Write 7 Done Done Sent false
18739 5895 Acked Write 8 Done Done Sent false
18740 5896 Acked Write 10 Done Done Sent false
18741 5897 Acked Write 2 Done Done Sent false
18742 5898 Acked Write 2 Done Done Sent false
18743 5899 Acked Write 5 Done Done Sent false
18744 5900 Acked Write 6 Done Done Sent false
18745 5901 Acked Read 4 Done Done Sent false
18746 5902 Acked Read 6 Done Done Sent false
18747 5903 Acked Write 3 Done Done Sent false
18748 5904 Acked Write 9 Done Done Sent false
18749 5905 Acked Read 9 Done Done Sent false
18750 5906 Acked Read 7 Done Done Sent false
18751 5907 Acked Read 4 Done Done Sent false
18752 5908 Acked Read 10 Done Done Sent false
18753 5909 Acked Read 3 Done Done Sent false
18754 5910 Acked Read 4 Done Done Sent false
18755 5911 Acked Write 1 Done Done Sent false
18756 5912 Acked Write 10 Done Done Sent false
18757 5913 Acked Read 2 Done Done Sent false
18758 5914 Acked Read 8 Done Done Sent false
18759 5915 Acked Read 10 Done Done Sent false
18760 5916 Acked Flush 0 Done Done Sent false
18761 5917 Acked Read 9 Done Done Sent false
18762 5918 Acked Read 4 Done Done Sent false
18763 5919 Acked Read 2 Done Done Sent false
18764 5920 Acked Write 8 Done Done Sent false
18765 5921 Acked Read 3 Done Done Sent false
18766 5922 Acked Read 3 Done Done Sent false
18767 5923 Acked Read 5 Done Done Sent false
18768 5924 Acked Write 5 Done Done Sent false
18769 5925 Acked Flush 0 Done Done Sent false
18770 5926 Acked Read 4 Done Done Sent false
18771 5927 Acked Write 7 Done Done Sent false
18772 5928 Acked Flush 0 Done Done Sent false
18773 5929 Acked Read 8 Done Done Sent false
18774 5930 Acked Write 5 Done Done Sent false
18775 5931 Acked Read 5 Done Done Sent false
18776 5932 Acked Flush 0 Done Done Sent false
18777 5933 Acked Read 5 Done Done Sent false
18778 5934 Acked Flush 0 Done Done Sent false
18779 5935 Acked Read 5 Done Done Sent false
18780 5936 Acked Read 2 Done Done Sent false
18781 5937 Acked Read 5 Done Done Sent false
18782 5938 Acked Read 7 Done Done Sent false
18783 5939 Acked Flush 0 Done Done Sent false
18784 5940 Acked Read 3 Done Done Sent false
18785 5941 Acked Read 10 Done Done Sent false
18786 5942 Acked Write 9 Done Done Sent false
18787 5943 Acked Read 5 Done Done Sent false
18788 5944 Acked Write 8 Done Done Sent false
18789 5945 Acked Flush 0 Done Done Sent false
18790 5946 Acked Write 5 Done Done Sent false
18791 5947 Acked Read 5 Done Done Sent false
18792 5948 Acked Read 5 Done Done Sent false
18793 5949 Acked Read 1 Done Done Sent false
18794 5950 Acked Read 2 Done Done Sent false
18795 5951 Acked Write 5 Done Done Sent false
18796 5952 Acked Read 6 Done Done Sent false
18797 5953 Acked Read 10 Done Done Sent false
18798 5954 Acked Write 1 Done Done Sent false
18799 5955 Acked Read 3 Done Done Sent false
18800 5956 Acked Flush 0 Done Done Sent false
18801 5957 Acked Flush 0 Done Done Sent false
18802 5958 Acked Read 9 Done Done Sent false
18803 5959 Acked Write 3 Done Done Sent false
18804 5960 Acked Read 6 Done Done Sent false
18805 5961 Acked Read 7 Done Done Sent false
18806 5962 Acked Read 6 Done Done Sent false
18807 5963 Acked Write 6 Done Done Sent false
18808 5964 Acked Read 6 Done Done Sent false
18809 5965 Acked Read 1 Done Done Sent false
18810 5966 Acked Read 2 Done Done Sent false
18811 5967 Acked Write 2 Done Done Sent false
18812 5968 Acked Read 8 Done Done Sent false
18813 5969 Acked Write 8 Done Done Sent false
18814 5970 Acked Read 1 Done Done Sent false
18815 5971 Acked Read 1 Done Done Sent false
18816 5972 Acked Write 10 Done Done Sent false
18817 5973 Acked Read 7 Done Done Sent false
18818 5974 Acked Write 1 Done Done Sent false
18819 5975 Acked Write 7 Done Done Sent false
18820 5976 Acked Flush 0 Done Done Sent false
18821 5977 Acked Read 7 Done Done Sent false
18822 5978 Acked Flush 0 Done Done Sent false
18823 5979 Acked Read 3 Done Done Sent false
18824 5980 Acked Read 2 Done Done Sent false
18825 5981 Acked Write 5 Done Done Sent false
18826 5982 Acked Read 3 Done Done Sent false
18827 5983 Acked Read 1 Done Done Sent false
18828 5984 Acked Read 5 Done Done Sent false
18829 5985 Acked Write 3 Done Done Sent false
18830 5986 Acked Read 7 Done Done Sent false
18831 5987 Acked Write 9 Done Done Sent false
18832 5988 Acked Read 6 Done Done Sent false
18833 5989 Acked Write 1 Done Done Sent false
18834 5990 Acked Flush 0 Done Done Sent false
18835 5991 Acked Write 2 Done Done Sent false
18836 5992 Acked Read 10 Done Done Sent false
18837 5993 Acked Read 4 Done Done Sent false
18838 5994 Acked Write 1 Done Done Sent false
18839 5995 Acked Write 8 Done Done Sent false
18840 5996 Acked Read 10 Done Done Sent false
18841 5997 Acked Write 5 Done Done Sent false
18842 5998 Acked Write 6 Done Done Sent false
18843 5999 Acked Write 2 Done Done Sent false
18844 6000 Acked Write 6 Done Done Sent false
18845 6001 Acked Read 8 Done Done Sent false
18846 6002 Acked Read 7 Done Done Sent false
18847 6003 Acked Read 6 Done Done Sent false
18848 6004 Acked Read 3 Done Done Sent false
18849 6005 Acked Read 5 Done Done Sent false
18850 6006 Acked Read 3 Done Done Sent false
18851 6007 Acked Write 7 Done Done Sent false
18852 6008 Acked Read 7 Done Done Sent false
18853 6009 Acked Write 3 Done Done Sent false
18854 6010 Acked Write 8 Done Done Sent false
18855 6011 Acked Read 5 Done Done Sent false
18856 6012 Acked Write 4 Done Done Sent false
18857 6013 Acked Read 4 Done Done Sent false
18858 6014 Acked Flush 0 Done Done Sent false
18859 6015 Acked Read 7 Done Done Sent false
18860 6016 Acked Write 3 Done Done Sent false
18861 6017 Acked Read 10 Done Done Sent false
18862 6018 Acked Read 8 Done Done Sent false
18863 6019 Acked Read 7 Done Done Sent false
18864 6020 Acked Read 1 Done Done Sent false
18865 6021 Acked Read 7 Done Done Sent false
18866 6022 Acked Read 1 Done Done Sent false
18867 6023 Acked Write 5 Done Done Sent false
18868 6024 Acked Read 6 Done Done Sent false
18869 6025 Acked Write 1 Done Done Sent false
18870 6026 Acked Write 8 Done Done Sent false
18871 6027 Acked Flush 0 Done Done Sent false
18872 6028 Acked Read 9 Done Done Sent false
18873 6029 Acked Read 3 Done Done Sent false
18874 6030 Acked Read 7 Done Done Sent false
18875 6031 Acked Read 4 Done Done Sent false
18876 6032 Acked Write 6 Done Done Sent false
18877 6033 Acked Read 6 Done Done Sent false
18878 6034 Acked Write 7 Done Done Sent false
18879 6035 Acked Read 6 Done Done Sent false
18880 6036 Acked Read 7 Done Done Sent false
18881 6037 Acked Read 9 Done Done Sent false
18882 6038 Acked Read 1 Done Done Sent false
18883 6039 Acked Read 10 Done Done Sent false
18884 6040 Acked Read 8 Done Done Sent false
18885 6041 Acked Write 3 Done Done Sent false
18886 6042 Acked Read 6 Done Done Sent false
18887 6043 Acked Write 9 Done Done Sent false
18888 6044 Acked Read 3 Done Done Sent false
18889 6045 Acked Write 4 Done Done Sent false
18890 6046 Acked Write 9 Done Done Sent false
18891 6047 Acked Read 2 Done Done Sent false
18892 6048 Acked Read 9 Done Done Sent false
18893 6049 Acked Read 6 Done Done Sent false
18894 6050 Acked Read 8 Done Done Sent false
18895 6051 Acked Write 3 Done Done Sent false
18896 6052 Acked Write 4 Done Done Sent false
18897 6053 Acked Read 2 Done Done Sent false
18898 6054 Acked Write 6 Done Done Sent false
18899 6055 Acked Read 10 Done Done Sent false
18900 6056 Acked Write 8 Done Done Sent false
18901 6057 Acked Read 2 Done Done Sent false
18902 6058 Acked Write 4 Done Done Sent false
18903 6059 Acked Write 1 Done Done Sent false
18904 6060 Acked Read 10 Done Done Sent false
18905 6061 Acked Read 9 Done Done Sent false
18906 6062 Acked Write 9 Done Done Sent false
18907 6063 Acked Write 6 Done Done Sent false
18908 6064 Acked Write 10 Done Done Sent false
18909 6065 Acked Read 5 Done Done Sent false
18910 6066 Acked Write 3 Done Done Sent false
18911 6067 Acked Write 10 Done Done Sent false
18912 6068 Acked Read 6 Done Done Sent false
18913 6069 Acked Read 8 Done Done Sent false
18914 6070 Acked Read 2 Done Done Sent false
18915 6071 Acked Read 2 Done Done Sent false
18916 6072 Acked Read 7 Done Done Sent false
18917 6073 Acked Read 8 Done Done Sent false
18918 6074 Acked Write 2 Done Done Sent false
18919 6075 Acked Write 9 Done Done Sent false
18920 6076 Acked Read 3 Done Done Sent false
18921 6077 Acked Write 10 Done Done Sent false
18922 6078 Acked Write 8 Done Done Sent false
18923 6079 Acked Read 6 Done Done Sent false
18924 6080 Acked Write 3 Done Done Sent false
18925 6081 Acked Read 7 Done Done Sent false
18926 6082 Acked Write 4 Done Done Sent false
18927 6083 Acked Read 7 Done Done Sent false
18928 6084 Acked Write 1 Done Done Sent false
18929 6085 Acked Read 3 Done Done Sent false
18930 6086 Acked Write 4 Done Done Sent false
18931 6087 Acked Write 5 Done Done Sent false
18932 6088 Acked Read 7 Done Done Sent false
18933 6089 Acked Write 9 Done Done Sent false
18934 6090 Acked Read 9 Done Done Sent false
18935 6091 Acked Read 3 Done Done Sent false
18936 6092 Acked Read 10 Done Done Sent false
18937 6093 Acked Read 10 Done Done Sent false
18938 6094 Acked Write 8 Done Done Sent false
18939 6095 Acked Read 1 Done Done Sent false
18940 6096 Acked Read 6 Done Done Sent false
18941 6097 Acked Write 6 Done Done Sent false
18942 6098 Acked Read 3 Done Done Sent false
18943 6099 Acked Flush 0 Done Done Sent false
18944 6100 Acked Write 2 Done Done Sent false
18945 6101 Acked Read 8 Done Done Sent false
18946 6102 Acked Write 8 Done Done Sent false
18947 6103 Acked Read 3 Done Done Sent false
18948 6104 Acked Write 9 Done Done Sent false
18949 6105 Acked Read 10 Done Done Sent false
18950 6106 Acked Read 6 Done Done Sent false
18951 6107 Acked Read 2 Done Done Sent false
18952 6108 Acked Write 7 Done Done Sent false
18953 6109 Acked Read 6 Done Done Sent false
18954 6110 Acked Write 3 Done Done Sent false
18955 6111 Acked Flush 0 Done Done Sent false
18956 6112 Acked Read 8 Done Done Sent false
18957 6113 Acked Read 5 Done Done Sent false
18958 6114 Acked Write 7 Done Done Sent false
18959 6115 Acked Read 7 Done Done Sent false
18960 6116 Acked Read 7 Done Done Sent false
18961 6117 Acked Read 7 Done Done Sent false
18962 6118 Acked Write 4 Done Done Sent false
18963 6119 Acked Read 3 Done Done Sent false
18964 6120 Acked Read 7 Done Done Sent false
18965 6121 Acked Read 3 Done Done Sent false
18966 6122 Acked Write 10 Done Done Sent false
18967 6123 Acked Write 10 Done Done Sent false
18968 6124 Acked Write 7 Done Done Sent false
18969 6125 Acked Write 7 Done Done Sent false
18970 6126 Acked Write 2 Done Done Sent false
18971 6127 Acked Read 3 Done Done Sent false
18972 6128 Acked Read 3 Done Done Sent false
18973 6129 Acked Write 4 Done Done Sent false
18974 6130 Acked Write 4 Done Done Sent false
18975 6131 Acked Read 9 Done Done Sent false
18976 6132 Acked Flush 0 Done Done Sent false
18977 STATES DS:0 DS:1 DS:2 TOTAL
18978 Sent 0 0 302 302
18979 Done 302 302 0 604
18980 Skipped 0 0 0 0
18981 Error 0 0 0 0
18982 Last Flush: 6132 6132 5830
18983 Downstairs last five completed: 5830 5829 5828 5827 5826
18984 Upstairs last five completed: 6132 6131 6130 6129 6128
18985 CLIENT: Up:0 ds:302 act:2
18986 [8830] got start action
18987 [8830] I am going to restart
18988 Starting downstairs at port 8830
18989 Make output file at "/tmp/dsc/downstairs-8830.txt"
18990 [8830][2] reports Starting
18991 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1220)
18992 [8830][2] reports Running
18993 ----------------------------------------------------------------
18994 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
18995 States: Active Active Offline
18996 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
18997 5831 Acked Write 3 Done Done Sent false
18998 5832 Acked Write 7 Done Done Sent false
18999 5833 Acked Read 10 Done Done Sent false
19000 5834 Acked Read 5 Done Done Sent false
19001 5835 Acked Write 6 Done Done Sent false
19002 5836 Acked Write 7 Done Done Sent false
19003 5837 Acked Read 1 Done Done Sent false
19004 5838 Acked Write 10 Done Done Sent false
19005 5839 Acked Read 1 Done Done Sent false
19006 5840 Acked Read 3 Done Done Sent false
19007 5841 Acked Read 9 Done Done Sent false
19008 5842 Acked Read 3 Done Done Sent false
19009 5843 Acked Read 1 Done Done Sent false
19010 5844 Acked Write 8 Done Done Sent false
19011 5845 Acked Read 10 Done Done Sent false
19012 5846 Acked Write 10 Done Done Sent false
19013 5847 Acked Read 3 Done Done Sent false
19014 5848 Acked Read 7 Done Done Sent false
19015 5849 Acked Read 10 Done Done Sent false
19016 5850 Acked Read 6 Done Done Sent false
19017 5851 Acked Write 1 Done Done Sent false
19018 5852 Acked Write 10 Done Done Sent false
19019 5853 Acked Read 5 Done Done Sent false
19020 5854 Acked Read 4 Done Done Sent false
19021 5855 Acked Write 3 Done Done Sent false
19022 5856 Acked Read 3 Done Done Sent false
19023 5857 Acked Read 5 Done Done Sent false
19024 5858 Acked Write 4 Done Done Sent false
19025 5859 Acked Read 6 Done Done Sent false
19026 5860 Acked Read 5 Done Done Sent false
19027 5861 Acked Read 10 Done Done Sent false
19028 5862 Acked Flush 0 Done Done Sent false
19029 5863 Acked Write 1 Done Done Sent false
19030 5864 Acked Read 8 Done Done Sent false
19031 5865 Acked Write 4 Done Done Sent false
19032 5866 Acked Read 6 Done Done Sent false
19033 5867 Acked Write 8 Done Done Sent false
19034 5868 Acked Read 9 Done Done Sent false
19035 5869 Acked Read 8 Done Done Sent false
19036 5870 Acked Flush 0 Done Done Sent false
19037 5871 Acked Read 1 Done Done Sent false
19038 5872 Acked Write 6 Done Done Sent false
19039 5873 Acked Read 1 Done Done Sent false
19040 5874 Acked Write 7 Done Done Sent false
19041 5875 Acked Read 3 Done Done Sent false
19042 5876 Acked Write 10 Done Done Sent false
19043 5877 Acked Read 3 Done Done Sent false
19044 5878 Acked Read 2 Done Done Sent false
19045 5879 Acked Write 10 Done Done Sent false
19046 5880 Acked Write 7 Done Done Sent false
19047 5881 Acked Read 8 Done Done Sent false
19048 5882 Acked Write 9 Done Done Sent false
19049 5883 Acked Read 5 Done Done Sent false
19050 5884 Acked Read 1 Done Done Sent false
19051 5885 Acked Write 1 Done Done Sent false
19052 5886 Acked Read 5 Done Done Sent false
19053 5887 Acked Read 8 Done Done Sent false
19054 5888 Acked Read 10 Done Done Sent false
19055 5889 Acked Write 6 Done Done Sent false
19056 5890 Acked Flush 0 Done Done Sent false
19057 5891 Acked Read 5 Done Done Sent false
19058 5892 Acked Read 3 Done Done Sent false
19059 5893 Acked Read 10 Done Done Sent false
19060 5894 Acked Write 7 Done Done Sent false
19061 5895 Acked Write 8 Done Done Sent false
19062 5896 Acked Write 10 Done Done Sent false
19063 5897 Acked Write 2 Done Done Sent false
19064 5898 Acked Write 2 Done Done Sent false
19065 5899 Acked Write 5 Done Done Sent false
19066 5900 Acked Write 6 Done Done Sent false
19067 5901 Acked Read 4 Done Done Sent false
19068 5902 Acked Read 6 Done Done Sent false
19069 5903 Acked Write 3 Done Done Sent false
19070 5904 Acked Write 9 Done Done Sent false
19071 5905 Acked Read 9 Done Done Sent false
19072 5906 Acked Read 7 Done Done Sent false
19073 5907 Acked Read 4 Done Done Sent false
19074 5908 Acked Read 10 Done Done Sent false
19075 5909 Acked Read 3 Done Done Sent false
19076 5910 Acked Read 4 Done Done Sent false
19077 5911 Acked Write 1 Done Done Sent false
19078 5912 Acked Write 10 Done Done Sent false
19079 5913 Acked Read 2 Done Done Sent false
19080 5914 Acked Read 8 Done Done Sent false
19081 5915 Acked Read 10 Done Done Sent false
19082 5916 Acked Flush 0 Done Done Sent false
19083 5917 Acked Read 9 Done Done Sent false
19084 5918 Acked Read 4 Done Done Sent false
19085 5919 Acked Read 2 Done Done Sent false
19086 5920 Acked Write 8 Done Done Sent false
19087 5921 Acked Read 3 Done Done Sent false
19088 5922 Acked Read 3 Done Done Sent false
19089 5923 Acked Read 5 Done Done Sent false
19090 5924 Acked Write 5 Done Done Sent false
19091 5925 Acked Flush 0 Done Done Sent false
19092 5926 Acked Read 4 Done Done Sent false
19093 5927 Acked Write 7 Done Done Sent false
19094 5928 Acked Flush 0 Done Done Sent false
19095 5929 Acked Read 8 Done Done Sent false
19096 5930 Acked Write 5 Done Done Sent false
19097 5931 Acked Read 5 Done Done Sent false
19098 5932 Acked Flush 0 Done Done Sent false
19099 5933 Acked Read 5 Done Done Sent false
19100 5934 Acked Flush 0 Done Done Sent false
19101 5935 Acked Read 5 Done Done Sent false
19102 5936 Acked Read 2 Done Done Sent false
19103 5937 Acked Read 5 Done Done Sent false
19104 5938 Acked Read 7 Done Done Sent false
19105 5939 Acked Flush 0 Done Done Sent false
19106 5940 Acked Read 3 Done Done Sent false
19107 5941 Acked Read 10 Done Done Sent false
19108 5942 Acked Write 9 Done Done Sent false
19109 5943 Acked Read 5 Done Done Sent false
19110 5944 Acked Write 8 Done Done Sent false
19111 5945 Acked Flush 0 Done Done Sent false
19112 5946 Acked Write 5 Done Done Sent false
19113 5947 Acked Read 5 Done Done Sent false
19114 5948 Acked Read 5 Done Done Sent false
19115 5949 Acked Read 1 Done Done Sent false
19116 5950 Acked Read 2 Done Done Sent false
19117 5951 Acked Write 5 Done Done Sent false
19118 5952 Acked Read 6 Done Done Sent false
19119 5953 Acked Read 10 Done Done Sent false
19120 5954 Acked Write 1 Done Done Sent false
19121 5955 Acked Read 3 Done Done Sent false
19122 5956 Acked Flush 0 Done Done Sent false
19123 5957 Acked Flush 0 Done Done Sent false
19124 5958 Acked Read 9 Done Done Sent false
19125 5959 Acked Write 3 Done Done Sent false
19126 5960 Acked Read 6 Done Done Sent false
19127 5961 Acked Read 7 Done Done Sent false
19128 5962 Acked Read 6 Done Done Sent false
19129 5963 Acked Write 6 Done Done Sent false
19130 5964 Acked Read 6 Done Done Sent false
19131 5965 Acked Read 1 Done Done Sent false
19132 5966 Acked Read 2 Done Done Sent false
19133 5967 Acked Write 2 Done Done Sent false
19134 5968 Acked Read 8 Done Done Sent false
19135 5969 Acked Write 8 Done Done Sent false
19136 5970 Acked Read 1 Done Done Sent false
19137 5971 Acked Read 1 Done Done Sent false
19138 5972 Acked Write 10 Done Done Sent false
19139 5973 Acked Read 7 Done Done Sent false
19140 5974 Acked Write 1 Done Done Sent false
19141 5975 Acked Write 7 Done Done Sent false
19142 5976 Acked Flush 0 Done Done Sent false
19143 5977 Acked Read 7 Done Done Sent false
19144 5978 Acked Flush 0 Done Done Sent false
19145 5979 Acked Read 3 Done Done Sent false
19146 5980 Acked Read 2 Done Done Sent false
19147 5981 Acked Write 5 Done Done Sent false
19148 5982 Acked Read 3 Done Done Sent false
19149 5983 Acked Read 1 Done Done Sent false
19150 5984 Acked Read 5 Done Done Sent false
19151 5985 Acked Write 3 Done Done Sent false
19152 5986 Acked Read 7 Done Done Sent false
19153 5987 Acked Write 9 Done Done Sent false
19154 5988 Acked Read 6 Done Done Sent false
19155 5989 Acked Write 1 Done Done Sent false
19156 5990 Acked Flush 0 Done Done Sent false
19157 5991 Acked Write 2 Done Done Sent false
19158 5992 Acked Read 10 Done Done Sent false
19159 5993 Acked Read 4 Done Done Sent false
19160 5994 Acked Write 1 Done Done Sent false
19161 5995 Acked Write 8 Done Done Sent false
19162 5996 Acked Read 10 Done Done Sent false
19163 5997 Acked Write 5 Done Done Sent false
19164 5998 Acked Write 6 Done Done Sent false
19165 5999 Acked Write 2 Done Done Sent false
19166 6000 Acked Write 6 Done Done Sent false
19167 6001 Acked Read 8 Done Done Sent false
19168 6002 Acked Read 7 Done Done Sent false
19169 6003 Acked Read 6 Done Done Sent false
19170 6004 Acked Read 3 Done Done Sent false
19171 6005 Acked Read 5 Done Done Sent false
19172 6006 Acked Read 3 Done Done Sent false
19173 6007 Acked Write 7 Done Done Sent false
19174 6008 Acked Read 7 Done Done Sent false
19175 6009 Acked Write 3 Done Done Sent false
19176 6010 Acked Write 8 Done Done Sent false
19177 6011 Acked Read 5 Done Done Sent false
19178 6012 Acked Write 4 Done Done Sent false
19179 6013 Acked Read 4 Done Done Sent false
19180 6014 Acked Flush 0 Done Done Sent false
19181 6015 Acked Read 7 Done Done Sent false
19182 6016 Acked Write 3 Done Done Sent false
19183 6017 Acked Read 10 Done Done Sent false
19184 6018 Acked Read 8 Done Done Sent false
19185 6019 Acked Read 7 Done Done Sent false
19186 6020 Acked Read 1 Done Done Sent false
19187 6021 Acked Read 7 Done Done Sent false
19188 6022 Acked Read 1 Done Done Sent false
19189 6023 Acked Write 5 Done Done Sent false
19190 6024 Acked Read 6 Done Done Sent false
19191 6025 Acked Write 1 Done Done Sent false
19192 6026 Acked Write 8 Done Done Sent false
19193 6027 Acked Flush 0 Done Done Sent false
19194 6028 Acked Read 9 Done Done Sent false
19195 6029 Acked Read 3 Done Done Sent false
19196 6030 Acked Read 7 Done Done Sent false
19197 6031 Acked Read 4 Done Done Sent false
19198 6032 Acked Write 6 Done Done Sent false
19199 6033 Acked Read 6 Done Done Sent false
19200 6034 Acked Write 7 Done Done Sent false
19201 6035 Acked Read 6 Done Done Sent false
19202 6036 Acked Read 7 Done Done Sent false
19203 6037 Acked Read 9 Done Done Sent false
19204 6038 Acked Read 1 Done Done Sent false
19205 6039 Acked Read 10 Done Done Sent false
19206 6040 Acked Read 8 Done Done Sent false
19207 6041 Acked Write 3 Done Done Sent false
19208 6042 Acked Read 6 Done Done Sent false
19209 6043 Acked Write 9 Done Done Sent false
19210 6044 Acked Read 3 Done Done Sent false
19211 6045 Acked Write 4 Done Done Sent false
19212 6046 Acked Write 9 Done Done Sent false
19213 6047 Acked Read 2 Done Done Sent false
19214 6048 Acked Read 9 Done Done Sent false
19215 6049 Acked Read 6 Done Done Sent false
19216 6050 Acked Read 8 Done Done Sent false
19217 6051 Acked Write 3 Done Done Sent false
19218 6052 Acked Write 4 Done Done Sent false
19219 6053 Acked Read 2 Done Done Sent false
19220 6054 Acked Write 6 Done Done Sent false
19221 6055 Acked Read 10 Done Done Sent false
19222 6056 Acked Write 8 Done Done Sent false
19223 6057 Acked Read 2 Done Done Sent false
19224 6058 Acked Write 4 Done Done Sent false
19225 6059 Acked Write 1 Done Done Sent false
19226 6060 Acked Read 10 Done Done Sent false
19227 6061 Acked Read 9 Done Done Sent false
19228 6062 Acked Write 9 Done Done Sent false
19229 6063 Acked Write 6 Done Done Sent false
19230 6064 Acked Write 10 Done Done Sent false
19231 6065 Acked Read 5 Done Done Sent false
19232 6066 Acked Write 3 Done Done Sent false
19233 6067 Acked Write 10 Done Done Sent false
19234 6068 Acked Read 6 Done Done Sent false
19235 6069 Acked Read 8 Done Done Sent false
19236 6070 Acked Read 2 Done Done Sent false
19237 6071 Acked Read 2 Done Done Sent false
19238 6072 Acked Read 7 Done Done Sent false
19239 6073 Acked Read 8 Done Done Sent false
19240 6074 Acked Write 2 Done Done Sent false
19241 6075 Acked Write 9 Done Done Sent false
19242 6076 Acked Read 3 Done Done Sent false
19243 6077 Acked Write 10 Done Done Sent false
19244 6078 Acked Write 8 Done Done Sent false
19245 6079 Acked Read 6 Done Done Sent false
19246 6080 Acked Write 3 Done Done Sent false
19247 6081 Acked Read 7 Done Done Sent false
19248 6082 Acked Write 4 Done Done Sent false
19249 6083 Acked Read 7 Done Done Sent false
19250 6084 Acked Write 1 Done Done Sent false
19251 6085 Acked Read 3 Done Done Sent false
19252 6086 Acked Write 4 Done Done Sent false
19253 6087 Acked Write 5 Done Done Sent false
19254 6088 Acked Read 7 Done Done Sent false
19255 6089 Acked Write 9 Done Done Sent false
19256 6090 Acked Read 9 Done Done Sent false
19257 6091 Acked Read 3 Done Done Sent false
19258 6092 Acked Read 10 Done Done Sent false
19259 6093 Acked Read 10 Done Done Sent false
19260 6094 Acked Write 8 Done Done Sent false
19261 6095 Acked Read 1 Done Done Sent false
19262 6096 Acked Read 6 Done Done Sent false
19263 6097 Acked Write 6 Done Done Sent false
19264 6098 Acked Read 3 Done Done Sent false
19265 6099 Acked Flush 0 Done Done Sent false
19266 6100 Acked Write 2 Done Done Sent false
19267 6101 Acked Read 8 Done Done Sent false
19268 6102 Acked Write 8 Done Done Sent false
19269 6103 Acked Read 3 Done Done Sent false
19270 6104 Acked Write 9 Done Done Sent false
19271 6105 Acked Read 10 Done Done Sent false
19272 6106 Acked Read 6 Done Done Sent false
19273 6107 Acked Read 2 Done Done Sent false
19274 6108 Acked Write 7 Done Done Sent false
19275 6109 Acked Read 6 Done Done Sent false
19276 6110 Acked Write 3 Done Done Sent false
19277 6111 Acked Flush 0 Done Done Sent false
19278 6112 Acked Read 8 Done Done Sent false
19279 6113 Acked Read 5 Done Done Sent false
19280 6114 Acked Write 7 Done Done Sent false
19281 6115 Acked Read 7 Done Done Sent false
19282 6116 Acked Read 7 Done Done Sent false
19283 6117 Acked Read 7 Done Done Sent false
19284 6118 Acked Write 4 Done Done Sent false
19285 6119 Acked Read 3 Done Done Sent false
19286 6120 Acked Read 7 Done Done Sent false
19287 6121 Acked Read 3 Done Done Sent false
19288 6122 Acked Write 10 Done Done Sent false
19289 6123 Acked Write 10 Done Done Sent false
19290 6124 Acked Write 7 Done Done Sent false
19291 6125 Acked Write 7 Done Done Sent false
19292 6126 Acked Write 2 Done Done Sent false
19293 6127 Acked Read 3 Done Done Sent false
19294 6128 Acked Read 3 Done Done Sent false
19295 6129 Acked Write 4 Done Done Sent false
19296 6130 Acked Write 4 Done Done Sent false
19297 6131 Acked Read 9 Done Done Sent false
19298 6132 Acked Flush 0 Done Done Sent false
19299 STATES DS:0 DS:1 DS:2 TOTAL
19300 Sent 0 0 302 302
19301 Done 302 302 0 604
19302 Skipped 0 0 0 0
19303 Error 0 0 0 0
19304 Last Flush: 6132 6132 5830
19305 Downstairs last five completed: 5830 5829 5828 5827 5826
19306 Upstairs last five completed: 6132 6131 6130 6129 6128
19307 CLIENT: Up:0 ds:302 act:2
19308 ----------------------------------------------------------------
19309 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
19310 States: Active Active Offline
19311 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
19312 5831 Acked Write 3 Done Done Sent false
19313 5832 Acked Write 7 Done Done Sent false
19314 5833 Acked Read 10 Done Done Sent false
19315 5834 Acked Read 5 Done Done Sent false
19316 5835 Acked Write 6 Done Done Sent false
19317 5836 Acked Write 7 Done Done Sent false
19318 5837 Acked Read 1 Done Done Sent false
19319 5838 Acked Write 10 Done Done Sent false
19320 5839 Acked Read 1 Done Done Sent false
19321 5840 Acked Read 3 Done Done Sent false
19322 5841 Acked Read 9 Done Done Sent false
19323 5842 Acked Read 3 Done Done Sent false
19324 5843 Acked Read 1 Done Done Sent false
19325 5844 Acked Write 8 Done Done Sent false
19326 5845 Acked Read 10 Done Done Sent false
19327 5846 Acked Write 10 Done Done Sent false
19328 5847 Acked Read 3 Done Done Sent false
19329 5848 Acked Read 7 Done Done Sent false
19330 5849 Acked Read 10 Done Done Sent false
19331 5850 Acked Read 6 Done Done Sent false
19332 5851 Acked Write 1 Done Done Sent false
19333 5852 Acked Write 10 Done Done Sent false
19334 5853 Acked Read 5 Done Done Sent false
19335 5854 Acked Read 4 Done Done Sent false
19336 5855 Acked Write 3 Done Done Sent false
19337 5856 Acked Read 3 Done Done Sent false
19338 5857 Acked Read 5 Done Done Sent false
19339 5858 Acked Write 4 Done Done Sent false
19340 5859 Acked Read 6 Done Done Sent false
19341 5860 Acked Read 5 Done Done Sent false
19342 5861 Acked Read 10 Done Done Sent false
19343 5862 Acked Flush 0 Done Done Sent false
19344 5863 Acked Write 1 Done Done Sent false
19345 5864 Acked Read 8 Done Done Sent false
19346 5865 Acked Write 4 Done Done Sent false
19347 5866 Acked Read 6 Done Done Sent false
19348 5867 Acked Write 8 Done Done Sent false
19349 5868 Acked Read 9 Done Done Sent false
19350 5869 Acked Read 8 Done Done Sent false
19351 5870 Acked Flush 0 Done Done Sent false
19352 5871 Acked Read 1 Done Done Sent false
19353 5872 Acked Write 6 Done Done Sent false
19354 5873 Acked Read 1 Done Done Sent false
19355 5874 Acked Write 7 Done Done Sent false
19356 5875 Acked Read 3 Done Done Sent false
19357 5876 Acked Write 10 Done Done Sent false
19358 5877 Acked Read 3 Done Done Sent false
19359 5878 Acked Read 2 Done Done Sent false
19360 5879 Acked Write 10 Done Done Sent false
19361 5880 Acked Write 7 Done Done Sent false
19362 5881 Acked Read 8 Done Done Sent false
19363 5882 Acked Write 9 Done Done Sent false
19364 5883 Acked Read 5 Done Done Sent false
19365 5884 Acked Read 1 Done Done Sent false
19366 5885 Acked Write 1 Done Done Sent false
19367 5886 Acked Read 5 Done Done Sent false
19368 5887 Acked Read 8 Done Done Sent false
19369 5888 Acked Read 10 Done Done Sent false
19370 5889 Acked Write 6 Done Done Sent false
19371 5890 Acked Flush 0 Done Done Sent false
19372 5891 Acked Read 5 Done Done Sent false
19373 5892 Acked Read 3 Done Done Sent false
19374 5893 Acked Read 10 Done Done Sent false
19375 5894 Acked Write 7 Done Done Sent false
19376 5895 Acked Write 8 Done Done Sent false
19377 5896 Acked Write 10 Done Done Sent false
19378 5897 Acked Write 2 Done Done Sent false
19379 5898 Acked Write 2 Done Done Sent false
19380 5899 Acked Write 5 Done Done Sent false
19381 5900 Acked Write 6 Done Done Sent false
19382 5901 Acked Read 4 Done Done Sent false
19383 5902 Acked Read 6 Done Done Sent false
19384 5903 Acked Write 3 Done Done Sent false
19385 5904 Acked Write 9 Done Done Sent false
19386 5905 Acked Read 9 Done Done Sent false
19387 5906 Acked Read 7 Done Done Sent false
19388 5907 Acked Read 4 Done Done Sent false
19389 5908 Acked Read 10 Done Done Sent false
19390 5909 Acked Read 3 Done Done Sent false
19391 5910 Acked Read 4 Done Done Sent false
19392 5911 Acked Write 1 Done Done Sent false
19393 5912 Acked Write 10 Done Done Sent false
19394 5913 Acked Read 2 Done Done Sent false
19395 5914 Acked Read 8 Done Done Sent false
19396 5915 Acked Read 10 Done Done Sent false
19397 5916 Acked Flush 0 Done Done Sent false
19398 5917 Acked Read 9 Done Done Sent false
19399 5918 Acked Read 4 Done Done Sent false
19400 5919 Acked Read 2 Done Done Sent false
19401 5920 Acked Write 8 Done Done Sent false
19402 5921 Acked Read 3 Done Done Sent false
19403 5922 Acked Read 3 Done Done Sent false
19404 5923 Acked Read 5 Done Done Sent false
19405 5924 Acked Write 5 Done Done Sent false
19406 5925 Acked Flush 0 Done Done Sent false
19407 5926 Acked Read 4 Done Done Sent false
19408 5927 Acked Write 7 Done Done Sent false
19409 5928 Acked Flush 0 Done Done Sent false
19410 5929 Acked Read 8 Done Done Sent false
19411 5930 Acked Write 5 Done Done Sent false
19412 5931 Acked Read 5 Done Done Sent false
19413 5932 Acked Flush 0 Done Done Sent false
19414 5933 Acked Read 5 Done Done Sent false
19415 5934 Acked Flush 0 Done Done Sent false
19416 5935 Acked Read 5 Done Done Sent false
19417 5936 Acked Read 2 Done Done Sent false
19418 5937 Acked Read 5 Done Done Sent false
19419 5938 Acked Read 7 Done Done Sent false
19420 5939 Acked Flush 0 Done Done Sent false
19421 5940 Acked Read 3 Done Done Sent false
19422 5941 Acked Read 10 Done Done Sent false
19423 5942 Acked Write 9 Done Done Sent false
19424 5943 Acked Read 5 Done Done Sent false
19425 5944 Acked Write 8 Done Done Sent false
19426 5945 Acked Flush 0 Done Done Sent false
19427 5946 Acked Write 5 Done Done Sent false
19428 5947 Acked Read 5 Done Done Sent false
19429 5948 Acked Read 5 Done Done Sent false
19430 5949 Acked Read 1 Done Done Sent false
19431 5950 Acked Read 2 Done Done Sent false
19432 5951 Acked Write 5 Done Done Sent false
19433 5952 Acked Read 6 Done Done Sent false
19434 5953 Acked Read 10 Done Done Sent false
19435 5954 Acked Write 1 Done Done Sent false
19436 5955 Acked Read 3 Done Done Sent false
19437 5956 Acked Flush 0 Done Done Sent false
19438 5957 Acked Flush 0 Done Done Sent false
19439 5958 Acked Read 9 Done Done Sent false
19440 5959 Acked Write 3 Done Done Sent false
19441 5960 Acked Read 6 Done Done Sent false
19442 5961 Acked Read 7 Done Done Sent false
19443 5962 Acked Read 6 Done Done Sent false
19444 5963 Acked Write 6 Done Done Sent false
19445 5964 Acked Read 6 Done Done Sent false
19446 5965 Acked Read 1 Done Done Sent false
19447 5966 Acked Read 2 Done Done Sent false
19448 5967 Acked Write 2 Done Done Sent false
19449 5968 Acked Read 8 Done Done Sent false
19450 5969 Acked Write 8 Done Done Sent false
19451 5970 Acked Read 1 Done Done Sent false
19452 5971 Acked Read 1 Done Done Sent false
19453 5972 Acked Write 10 Done Done Sent false
19454 5973 Acked Read 7 Done Done Sent false
19455 5974 Acked Write 1 Done Done Sent false
19456 5975 Acked Write 7 Done Done Sent false
19457 5976 Acked Flush 0 Done Done Sent false
19458 5977 Acked Read 7 Done Done Sent false
19459 5978 Acked Flush 0 Done Done Sent false
19460 5979 Acked Read 3 Done Done Sent false
19461 5980 Acked Read 2 Done Done Sent false
19462 5981 Acked Write 5 Done Done Sent false
19463 5982 Acked Read 3 Done Done Sent false
19464 5983 Acked Read 1 Done Done Sent false
19465 5984 Acked Read 5 Done Done Sent false
19466 5985 Acked Write 3 Done Done Sent false
19467 5986 Acked Read 7 Done Done Sent false
19468 5987 Acked Write 9 Done Done Sent false
19469 5988 Acked Read 6 Done Done Sent false
19470 5989 Acked Write 1 Done Done Sent false
19471 5990 Acked Flush 0 Done Done Sent false
19472 5991 Acked Write 2 Done Done Sent false
19473 5992 Acked Read 10 Done Done Sent false
19474 5993 Acked Read 4 Done Done Sent false
19475 5994 Acked Write 1 Done Done Sent false
19476 5995 Acked Write 8 Done Done Sent false
19477 5996 Acked Read 10 Done Done Sent false
19478 5997 Acked Write 5 Done Done Sent false
19479 5998 Acked Write 6 Done Done Sent false
19480 5999 Acked Write 2 Done Done Sent false
19481 6000 Acked Write 6 Done Done Sent false
19482 6001 Acked Read 8 Done Done Sent false
19483 6002 Acked Read 7 Done Done Sent false
19484 6003 Acked Read 6 Done Done Sent false
19485 6004 Acked Read 3 Done Done Sent false
19486 6005 Acked Read 5 Done Done Sent false
19487 6006 Acked Read 3 Done Done Sent false
19488 6007 Acked Write 7 Done Done Sent false
19489 6008 Acked Read 7 Done Done Sent false
19490 6009 Acked Write 3 Done Done Sent false
19491 6010 Acked Write 8 Done Done Sent false
19492 6011 Acked Read 5 Done Done Sent false
19493 6012 Acked Write 4 Done Done Sent false
19494 6013 Acked Read 4 Done Done Sent false
19495 6014 Acked Flush 0 Done Done Sent false
19496 6015 Acked Read 7 Done Done Sent false
19497 6016 Acked Write 3 Done Done Sent false
19498 6017 Acked Read 10 Done Done Sent false
19499 6018 Acked Read 8 Done Done Sent false
19500 6019 Acked Read 7 Done Done Sent false
19501 6020 Acked Read 1 Done Done Sent false
19502 6021 Acked Read 7 Done Done Sent false
19503 6022 Acked Read 1 Done Done Sent false
19504 6023 Acked Write 5 Done Done Sent false
19505 6024 Acked Read 6 Done Done Sent false
19506 6025 Acked Write 1 Done Done Sent false
19507 6026 Acked Write 8 Done Done Sent false
19508 6027 Acked Flush 0 Done Done Sent false
19509 6028 Acked Read 9 Done Done Sent false
19510 6029 Acked Read 3 Done Done Sent false
19511 6030 Acked Read 7 Done Done Sent false
19512 6031 Acked Read 4 Done Done Sent false
19513 6032 Acked Write 6 Done Done Sent false
19514 6033 Acked Read 6 Done Done Sent false
19515 6034 Acked Write 7 Done Done Sent false
19516 6035 Acked Read 6 Done Done Sent false
19517 6036 Acked Read 7 Done Done Sent false
19518 6037 Acked Read 9 Done Done Sent false
19519 6038 Acked Read 1 Done Done Sent false
19520 6039 Acked Read 10 Done Done Sent false
19521 6040 Acked Read 8 Done Done Sent false
19522 6041 Acked Write 3 Done Done Sent false
19523 6042 Acked Read 6 Done Done Sent false
19524 6043 Acked Write 9 Done Done Sent false
19525 6044 Acked Read 3 Done Done Sent false
19526 6045 Acked Write 4 Done Done Sent false
19527 6046 Acked Write 9 Done Done Sent false
19528 6047 Acked Read 2 Done Done Sent false
19529 6048 Acked Read 9 Done Done Sent false
19530 6049 Acked Read 6 Done Done Sent false
19531 6050 Acked Read 8 Done Done Sent false
19532 6051 Acked Write 3 Done Done Sent false
19533 6052 Acked Write 4 Done Done Sent false
19534 6053 Acked Read 2 Done Done Sent false
19535 6054 Acked Write 6 Done Done Sent false
19536 6055 Acked Read 10 Done Done Sent false
19537 6056 Acked Write 8 Done Done Sent false
19538 6057 Acked Read 2 Done Done Sent false
19539 6058 Acked Write 4 Done Done Sent false
19540 6059 Acked Write 1 Done Done Sent false
19541 6060 Acked Read 10 Done Done Sent false
19542 6061 Acked Read 9 Done Done Sent false
19543 6062 Acked Write 9 Done Done Sent false
19544 6063 Acked Write 6 Done Done Sent false
19545 6064 Acked Write 10 Done Done Sent false
19546 6065 Acked Read 5 Done Done Sent false
19547 6066 Acked Write 3 Done Done Sent false
19548 6067 Acked Write 10 Done Done Sent false
19549 6068 Acked Read 6 Done Done Sent false
19550 6069 Acked Read 8 Done Done Sent false
19551 6070 Acked Read 2 Done Done Sent false
19552 6071 Acked Read 2 Done Done Sent false
19553 6072 Acked Read 7 Done Done Sent false
19554 6073 Acked Read 8 Done Done Sent false
19555 6074 Acked Write 2 Done Done Sent false
19556 6075 Acked Write 9 Done Done Sent false
19557 6076 Acked Read 3 Done Done Sent false
19558 6077 Acked Write 10 Done Done Sent false
19559 6078 Acked Write 8 Done Done Sent false
19560 6079 Acked Read 6 Done Done Sent false
19561 6080 Acked Write 3 Done Done Sent false
19562 6081 Acked Read 7 Done Done Sent false
19563 6082 Acked Write 4 Done Done Sent false
19564 6083 Acked Read 7 Done Done Sent false
19565 6084 Acked Write 1 Done Done Sent false
19566 6085 Acked Read 3 Done Done Sent false
19567 6086 Acked Write 4 Done Done Sent false
19568 6087 Acked Write 5 Done Done Sent false
19569 6088 Acked Read 7 Done Done Sent false
19570 6089 Acked Write 9 Done Done Sent false
19571 6090 Acked Read 9 Done Done Sent false
19572 6091 Acked Read 3 Done Done Sent false
19573 6092 Acked Read 10 Done Done Sent false
19574 6093 Acked Read 10 Done Done Sent false
19575 6094 Acked Write 8 Done Done Sent false
19576 6095 Acked Read 1 Done Done Sent false
19577 6096 Acked Read 6 Done Done Sent false
19578 6097 Acked Write 6 Done Done Sent false
19579 6098 Acked Read 3 Done Done Sent false
19580 6099 Acked Flush 0 Done Done Sent false
19581 6100 Acked Write 2 Done Done Sent false
19582 6101 Acked Read 8 Done Done Sent false
19583 6102 Acked Write 8 Done Done Sent false
19584 6103 Acked Read 3 Done Done Sent false
19585 6104 Acked Write 9 Done Done Sent false
19586 6105 Acked Read 10 Done Done Sent false
19587 6106 Acked Read 6 Done Done Sent false
19588 6107 Acked Read 2 Done Done Sent false
19589 6108 Acked Write 7 Done Done Sent false
19590 6109 Acked Read 6 Done Done Sent false
19591 6110 Acked Write 3 Done Done Sent false
19592 6111 Acked Flush 0 Done Done Sent false
19593 6112 Acked Read 8 Done Done Sent false
19594 6113 Acked Read 5 Done Done Sent false
19595 6114 Acked Write 7 Done Done Sent false
19596 6115 Acked Read 7 Done Done Sent false
19597 6116 Acked Read 7 Done Done Sent false
19598 6117 Acked Read 7 Done Done Sent false
19599 6118 Acked Write 4 Done Done Sent false
19600 6119 Acked Read 3 Done Done Sent false
19601 6120 Acked Read 7 Done Done Sent false
19602 6121 Acked Read 3 Done Done Sent false
19603 6122 Acked Write 10 Done Done Sent false
19604 6123 Acked Write 10 Done Done Sent false
19605 6124 Acked Write 7 Done Done Sent false
19606 6125 Acked Write 7 Done Done Sent false
19607 6126 Acked Write 2 Done Done Sent false
19608 6127 Acked Read 3 Done Done Sent false
19609 6128 Acked Read 3 Done Done Sent false
19610 6129 Acked Write 4 Done Done Sent false
19611 6130 Acked Write 4 Done Done Sent false
19612 6131 Acked Read 9 Done Done Sent false
19613 6132 Acked Flush 0 Done Done Sent false
19614 STATES DS:0 DS:1 DS:2 TOTAL
19615 Sent 0 0 302 302
19616 Done 302 302 0 604
19617 Skipped 0 0 0 0
19618 Error 0 0 0 0
19619 Last Flush: 6132 6132 5830
19620 Downstairs last five completed: 5830 5829 5828 5827 5826
19621 Upstairs last five completed: 6132 6131 6130 6129 6128
19622 CLIENT: Up:0 ds:302 act:2
196232024-11-26T22:08:38.321ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
196242024-11-26T22:08:38.321ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:59484) = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
196252024-11-26T22:08:38.322ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
196262024-11-26T22:08:38.323ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
196272024-11-26T22:08:38.323ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
196282024-11-26T22:08:38.323ZINFOcrucible: send last flush ID to this DS: 5830 = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
196292024-11-26T22:08:38.323ZINFOcrucible: Replied this last flush ID: 5830 = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
196302024-11-26T22:08:38.323ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
196312024-11-26T22:08:38.323ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
196322024-11-26T22:08:38.323ZINFOcrucible: [2] client re-new 302 jobs since flush 5830 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
196332024-11-26T22:08:38.323ZINFOcrucible: [2] Replayed 302 jobs since flush: 5830 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
19634 ----------------------------------------------------------------
19635 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
19636 Downstairs last five completed: 6132 6131 6130 6129 6128
19637 Upstairs last five completed: 6132 6131 6130 6129 6128
19638 CLIENT: Up:0 ds:0 act:3
19639 Replay: All jobs finished
19640 Main task has work to do, go find it
19641 got dsc Stop(1)
19642 stop 1
19643 Nov 26 22:08:40.583 INFO request completed, latency_us: 337, response_code: 204, [8820] Got stop action so:false kr:false
19644 uri: /stop/cid/1, method: POST, req_id: b0b48806-3501-40e2-b03a-3c20e9a62f12, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
196452024-11-26T22:08:40.584ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
196462024-11-26T22:08:40.584ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
19647 Nov 26 22:08:40.584 INFO request completed, latency_us: {"msg":"downstairs task for 1 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2024-11-26T22:08:40.584793514Z","hostname":"w-01JDN7ZQ17ABQG15KHKN3F2D26","pid":1203,"session_id":"86a36c17-3718-484c-87fd-2b8a1dc2ab18"}
19648 112, response_code: 200, uri: /state/cid/1, method: GET, req_id:{"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-26T22:08:40.584820921Z","hostname":"w-01JDN7ZQ17ABQG15KHKN3F2D26","pid":1203,"client":"1","":"downstairs","session_id":"86a36c17-3718-484c-87fd-2b8a1dc2ab18"}
19649 7391a77c-5430-4341-9b53-93e1e2df22a3, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998{"msg":"[1] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-26T22:08:40.584886213Z","hostname":"w-01JDN7ZQ17ABQG15KHKN3F2D26","pid":1203,"client":"1","":"downstairs","session_id":"86a36c17-3718-484c-87fd-2b8a1dc2ab18"}
19650 
196512024-11-26T22:08:40.584ZINFOcrucible: client task is exiting = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
19652 [8820] Exited with: ExitStatus(unix_wait_status(9))
19653 [8820][1] reports Exit
19654 Nov 26 22:08:44.586 INFO request completed, latency_us: 202, response_code: 200, uri: /state/cid/1, method: GET, req_id: c90fae59-6b0b-46ff-b4ed-89750ba882d6, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
19655 001/300 Write block 800 len 16384 data: 4 5 4 4
19656 002/300 Read block 727 len 28672
19657 003/300 Write block 2612 len 16384 data: 6 4 4 4
19658 004/300 Write block 2164 len 40960 data: 5 7 7 6 6 6 5 2 2 3
19659 005/300 Write block 2309 len 40960 data: 2 2 2 3 3 4 4 3 3 2
19660 006/300 Read block 4926 len 24576
19661 007/300 Write block 429 len 8192 data: 4 3
19662 008/300 Write block 213 len 28672 data: 3 4 5 7 6 7 7
19663 009/300 Read block 3552 len 12288
19664 010/300 Write block 1760 len 12288 data: 6 5 5
19665 011/300 Write block 1099 len 36864 data: 5 5 4 5 6 4 4 5 5
19666 012/300 Read block 2556 len 12288
19667 013/300 Write block 2738 len 28672 data: 3 4 5 6 6 7 8
19668 014/300 Write block 4344 len 24576 data: 9 8 6 7 7 5
19669 015/300 Flush
19670 016/300 Write block 3951 len 16384 data: 6 6 5 6
19671 017/300 Read block 65 len 28672
19672 018/300 Read block 3616 len 36864
19673 019/300 Read block 3435 len 24576
19674 020/300 Flush
19675 021/300 Read block 4668 len 40960
19676 022/300 Read block 743 len 40960
19677 023/300 Write block 3761 len 12288 data: 3 4 5
19678 024/300 Write block 325 len 28672 data: 4 3 2 3 3 3 3
19679 025/300 Read block 2898 len 8192
19680 026/300 Write block 1035 len 36864 data: 4 4 4 4 3 3 2 2 3
19681 027/300 Read block 4291 len 28672
19682 028/300 Write block 2511 len 28672 data: 3 4 4 5 5 8 8
19683 029/300 Read block 765 len 40960
19684 030/300 Read block 1536 len 8192
19685 031/300 Read block 2844 len 24576
19686 032/300 Read block 3147 len 24576
19687 033/300 Read block 3343 len 16384
19688 034/300 Flush
19689 035/300 Write block 4825 len 32768 data: 5 3 4 5 6 6 6 5
19690 036/300 Read block 1183 len 36864
19691 037/300 Flush
19692 038/300 Read block 4295 len 24576
19693 039/300 Write block 4628 len 8192 data: 3 3
19694 040/300 Flush
19695 041/300 Write block 875 len 20480 data: 5 4 3 3 3
19696 042/300 Write block 3665 len 16384 data: 6 7 7 7
19697 043/300 Flush
19698 044/300 Read block 4577 len 8192
19699 045/300 Write block 1349 len 16384 data: 3 5 6 6
19700 046/300 Flush
19701 047/300 Read block 4175 len 28672
19702 048/300 Write block 1649 len 4096 data: 5
19703 049/300 Read block 4766 len 24576
19704 050/300 Read block 216 len 12288
19705 051/300 Write block 4219 len 40960 data: 3 3 3 2 3 3 3 3 3 2
19706 052/300 Read block 2265 len 24576
19707 053/300 Write block 4428 len 24576 data: 2 2 2 4 3 3
19708 054/300 Read block 1884 len 8192
19709 055/300 Read block 2622 len 28672
19710 056/300 Write block 2749 len 16384 data: 7 4 2 2
19711 057/300 Write block 2463 len 12288 data: 3 2 4
19712 058/300 Read block 4312 len 12288
19713 059/300 Read block 4250 len 12288
19714 060/300 Read block 4495 len 4096
19715 061/300 Read block 3382 len 40960
19716 062/300 Write block 2781 len 36864 data: 2 2 2 2 2 2 2 2 2
19717 063/300 Read block 3032 len 16384
19718 064/300 Write block 43 len 36864 data: 4 5 5 6 6 6 5 4 4
19719 065/300 Flush
19720 066/300 Read block 530 len 16384
19721 067/300 Write block 580 len 4096 data: 3
19722 068/300 Read block 4908 len 32768
19723 069/300 Write block 403 len 20480 data: 7 6 4 3 5
19724 070/300 Read block 2984 len 36864
19725 071/300 Read block 3525 len 24576
19726 072/300 Write block 2827 len 32768 data: 2 4 6 6 6 6 5 4
19727 073/300 Read block 3727 len 12288
19728 074/300 Write block 1519 len 28672 data: 2 3 3 4 5 6 7
19729 075/300 Read block 3633 len 12288
19730 076/300 Read block 1631 len 16384
19731 077/300 Write block 4664 len 12288 data: 6 4 5
19732 078/300 Read block 427 len 8192
19733 079/300 Write block 1822 len 16384 data: 4 4 5 6
19734 080/300 Read block 3973 len 36864
19735 081/300 Write block 1372 len 24576 data: 4 4 4 2 3 3
19736 082/300 Write block 161 len 40960 data: 6 4 5 4 4 3 3 3 4 5
19737 083/300 Write block 3066 len 4096 data: 4
19738 084/300 Write block 4739 len 16384 data: 2 2 3 2
19739 085/300 Write block 3045 len 28672 data: 3 3 2 3 4 4 4
19740 086/300 Write block 436 len 20480 data: 4 3 3 3 3
19741 087/300 Read block 879 len 20480
19742 088/300 Write block 34 len 24576 data: 3 4 5 5 5 5
19743 089/300 Write block 3896 len 8192 data: 7 6
19744 090/300 Flush
19745 091/300 Read block 3792 len 8192
19746 092/300 Flush
19747 093/300 Read block 175 len 24576
19748 094/300 Read block 1913 len 16384
19749 095/300 Write block 3011 len 24576 data: 6 5 5 4 5 4
19750 096/300 Write block 3890 len 20480 data: 4 4 3 4 5
19751 097/300 Read block 4710 len 8192
19752 098/300 Read block 2742 len 20480
19753 099/300 Read block 3253 len 4096
19754 100/300 Read block 623 len 12288
19755 101/300 Read block 2524 len 4096
19756 102/300 Read block 4487 len 32768
19757 103/300 Write block 2504 len 20480 data: 5 4 3 3 3
19758 104/300 Read block 2190 len 24576
19759 105/300 Write block 2308 len 32768 data: 2 3 3 3 4 4 5 5
19760 106/300 Read block 268 len 40960
19761 107/300 Read block 1133 len 28672
19762 108/300 Flush
19763 109/300 Read block 810 len 8192
19764 110/300 Read block 4756 len 20480
19765 111/300 Read block 3503 len 36864
19766 112/300 Read block 2337 len 8192
19767 113/300 Write block 2434 len 24576 data: 4 3 4 3 3 3
19768 114/300 Write block 1276 len 12288 data: 5 6 6
19769 115/300 Read block 1080 len 4096
19770 116/300 Write block 869 len 8192 data: 4 6
19771 117/300 Read block 2488 len 40960
19772 118/300 Read block 2700 len 24576
19773 119/300 Write block 3624 len 24576 data: 4 4 3 3 3 4
19774 120/300 Flush
19775 121/300 Read block 4532 len 20480
19776 122/300 Flush
19777 123/300 Read block 4580 len 36864
19778 124/300 Flush
19779 125/300 Read block 1318 len 12288
19780 126/300 Write block 4797 len 4096 data: 2
19781 127/300 Write block 1086 len 28672 data: 4 5 4 3 3 4 5
19782 128/300 Flush
19783 129/300 Write block 3287 len 36864 data: 4 4 3 4 4 3 4 4 3
19784 130/300 Write block 2918 len 8192 data: 6 5
19785 131/300 Write block 4327 len 36864 data: 5 5 4 4 6 6 6 6 6
19786 132/300 Write block 1776 len 36864 data: 6 6 6 7 6 4 4 3 3
19787 133/300 Read block 3227 len 24576
19788 134/300 Write block 572 len 24576 data: 6 5 5 6 5 4
19789 135/300 Read block 2864 len 20480
19790 136/300 Read block 4780 len 20480
19791 137/300 Write block 3856 len 8192 data: 5 7
19792 138/300 Read block 1155 len 28672
19793 139/300 Read block 4298 len 32768
19794 140/300 Write block 2150 len 28672 data: 2 2 3 2 2 2 2
19795 141/300 Read block 3235 len 36864
19796 142/300 Read block 13 len 16384
19797 143/300 Read block 4389 len 12288
19798 144/300 Read block 1276 len 24576
19799 145/300 Write block 2533 len 32768 data: 3 3 3 3 3 2 3 3
19800 146/300 Write block 1480 len 24576 data: 6 7 7 7 6 4
19801 147/300 Write block 2736 len 4096 data: 4
19802 148/300 Write block 3729 len 28672 data: 5 6 5 4 4 3 2
19803 149/300 Read block 4157 len 4096
19804 150/300 Write block 172 len 24576 data: 6 6 5 5 5 5
19805 151/300 Write block 2557 len 28672 data: 5 7 7 6 6 6 5
19806 152/300 Read block 4573 len 4096
19807 153/300 Read block 1525 len 20480
19808 154/300 Write block 3302 len 40960 data: 4 5 5 4 4 4 4 4 3 3
19809 155/300 Read block 624 len 40960
19810 156/300 Read block 2962 len 40960
19811 157/300 Read block 2837 len 40960
19812 158/300 Read block 2616 len 16384
19813 159/300 Read block 2447 len 8192
19814 160/300 Write block 2388 len 12288 data: 5 6 7
19815 161/300 Write block 3382 len 40960 data: 4 4 4 4 3 3 3 2 3 4
19816 162/300 Read block 2062 len 32768
19817 163/300 Write block 1270 len 12288 data: 3 2 2
19818 164/300 Read block 4260 len 4096
19819 165/300 Write block 2590 len 32768 data: 5 4 4 4 3 4 5 5
19820 166/300 Write block 4061 len 28672 data: 5 6 6 6 5 5 5
19821 167/300 Flush
19822 168/300 Read block 875 len 8192
19823 169/300 Read block 2216 len 20480
19824 170/300 Write block 4234 len 32768 data: 4 3 3 3 3 4 4 4
19825 171/300 Write block 927 len 32768 data: 6 5 7 8 6 7 7 5
19826 172/300 Read block 2079 len 4096
19827 173/300 Read block 2059 len 12288
19828 174/300 Write block 4494 len 36864 data: 4 5 4 3 3 2 2 2 2
19829 175/300 Read block 2371 len 8192
19830 176/300 Read block 4953 len 12288
19831 177/300 Flush
19832 178/300 Read block 7 len 16384
19833 179/300 Read block 807 len 20480
19834 180/300 Read block 1792 len 40960
19835 181/300 Read block 217 len 8192
19836 182/300 Write block 2348 len 32768 data: 3 4 4 6 6 7 6 4
19837 183/300 Write block 3344 len 36864 data: 4 3 4 4 4 4 4 6 5
19838 184/300 Read block 2094 len 24576
19839 185/300 Write block 997 len 32768 data: 2 2 2 2 2 3 3 4
19840 186/300 Write block 4574 len 16384 data: 3 5 4 3
19841 187/300 Flush
19842 188/300 Read block 3467 len 16384
19843 189/300 Read block 4320 len 12288
19844 190/300 Read block 3901 len 36864
19845 191/300 Write block 1914 len 32768 data: 2 2 3 3 3 5 5 6
19846 192/300 Read block 1143 len 8192
19847 193/300 Read block 2850 len 40960
19848 194/300 Write block 4244 len 32768 data: 3 2 4 4 5 5 4 4
19849 195/300 Write block 1194 len 16384 data: 7 6 7 8
19850 196/300 Flush
19851 197/300 Read block 4783 len 20480
19852 198/300 Write block 4955 len 20480 data: 3 3 4 3 4
19853 199/300 Read block 3755 len 40960
19854 200/300 Read block 3616 len 16384
19855 201/300 Write block 2096 len 40960 data: 2 2 2 2 3 4 6 6 6 5
19856 202/300 Read block 1953 len 4096
19857 203/300 Read block 1634 len 28672
19858 204/300 Read block 4496 len 36864
19859 205/300 Read block 2938 len 16384
19860 206/300 Write block 3729 len 12288 data: 6 7 6
19861 207/300 Write block 4494 len 4096 data: 5
19862 208/300 Write block 4158 len 12288 data: 5 6 5
19863 209/300 Write block 2446 len 40960 data: 4 3 3 3 3 3 3 2 3 4
19864 210/300 Read block 3617 len 40960
19865 211/300 Read block 1938 len 24576
19866 212/300 Write block 1078 len 8192 data: 4 5
19867 213/300 Read block 2037 len 40960
19868 214/300 Write block 2736 len 16384 data: 5 3 4 5
19869 215/300 Read block 1623 len 16384
19870 216/300 Write block 4592 len 28672 data: 4 5 5 4 5 5 5
19871 217/300 Write block 4521 len 8192 data: 3 3
19872 218/300 Write block 2047 len 4096 data: 3
19873 219/300 Write block 1477 len 32768 data: 6 7 7 7 8 8 8 7
19874 220/300 Write block 1598 len 24576 data: 5 7 5 4 4 4
19875 221/300 Read block 3559 len 4096
19876 222/300 Read block 3026 len 40960
19877 223/300 Write block 2454 len 8192 data: 4 5
19878 224/300 Write block 3615 len 28672 data: 4 4 4 4 4 3 3
19879 225/300 Read block 4041 len 40960
19880 226/300 Write block 448 len 20480 data: 7 6 6 6 7
19881 227/300 Read block 3118 len 20480
19882 228/300 Flush
19883 229/300 Read block 4216 len 40960
19884 230/300 Read block 3966 len 8192
19885 231/300 Read block 4349 len 40960
19886 232/300 Write block 1175 len 32768 data: 4 6 4 4 5 5 6 5
19887 233/300 Write block 4473 len 24576 data: 5 6 7 6 6 5
19888 234/300 Write block 893 len 12288 data: 4 3 2
19889 235/300 Write block 2218 len 20480 data: 3 4 3 3 2
19890 236/300 Flush
19891 237/300 Read block 4208 len 12288
19892 238/300 Write block 1706 len 28672 data: 2 2 2 2 3 3 3
19893 239/300 Write block 2068 len 12288 data: 7 6 6
19894 240/300 Read block 4172 len 20480
19895 241/300 Read block 3484 len 24576
19896 242/300 Write block 765 len 36864 data: 5 5 5 4 3 4 5 4 4
19897 243/300 Write block 2066 len 12288 data: 7 7 8
19898 244/300 Read block 2404 len 4096
19899 245/300 Write block 1716 len 20480 data: 7 7 6 4 4
19900 246/300 Write block 1988 len 12288 data: 6 6 5
19901 247/300 Read block 4003 len 4096
19902 248/300 Read block 4101 len 32768
19903 249/300 Flush
19904 250/300 Flush
19905 251/300 Read block 4583 len 12288
19906 252/300 Flush
19907 253/300 Read block 89 len 36864
19908 254/300 Read block 961 len 28672
19909 255/300 Flush
19910 256/300 Read block 1227 len 20480
19911 257/300 Write block 4946 len 16384 data: 5 5 6 4
19912 258/300 Read block 3251 len 24576
19913 259/300 Write block 3447 len 24576 data: 3 3 4 4 4 4
19914 260/300 Read block 2876 len 40960
19915 261/300 Read block 2156 len 32768
19916 262/300 Read block 2593 len 24576
19917 263/300 Write block 1127 len 4096 data: 5
19918 264/300 Read block 373 len 28672
19919 265/300 Read block 1232 len 28672
19920 266/300 Read block 179 len 20480
19921 267/300 Write block 1093 len 8192 data: 5 4
19922 268/300 Write block 2225 len 40960 data: 3 3 3 3 3 4 4 3 2 3
19923 269/300 Read block 4358 len 4096
19924 270/300 Write block 1249 len 16384 data: 5 6 4 4
19925 271/300 Write block 4781 len 8192 data: 2 2
19926 272/300 Read block 862 len 24576
19927 273/300 Write block 4943 len 20480 data: 3 3 3 6 6
19928 274/300 Write block 2915 len 4096 data: 6
19929 275/300 Read block 646 len 16384
19930 276/300 Read block 2985 len 24576
19931 277/300 Write block 1231 len 40960 data: 5 5 6 7 7 6 5 5 5 5
19932 278/300 Read block 523 len 12288
19933 279/300 Read block 1649 len 16384
19934 280/300 Write block 961 len 40960 data: 2 2 3 3 4 5 5 5 4 3
19935 281/300 Read block 2780 len 12288
19936 282/300 Read block 3377 len 24576
19937 283/300 Read block 489 len 28672
19938 284/300 Write block 2689 len 36864 data: 5 5 5 5 5 4 4 4 4
19939 285/300 Read block 581 len 40960
19940 286/300 Read block 433 len 4096
19941 287/300 Read block 3634 len 16384
19942 288/300 Write block 1173 len 36864 data: 3 4 5 7 5 5 6 6 7
19943 289/300 Read block 3020 len 24576
19944 290/300 Read block 997 len 28672
19945 291/300 Write block 3193 len 20480 data: 3 3 3 3 3
19946 292/300 Read block 1675 len 28672
19947 293/300 Read block 603 len 4096
19948 294/300 Flush
19949 295/300 Read block 3388 len 8192
19950 296/300 Read block 213 len 40960
19951 297/300 Read block 529 len 16384
19952 298/300 Read block 416 len 20480
19953 299/300 Read block 52 len 24576
19954 300/300 Read block 69 len 16384
19955 301/300 Write block 1048 len 40960 data: 6 5 4 4 4 4 5 5 4 4
19956 Main task has work to do, go find it
19957 got dsc Start(1)
19958 start 1
19959 Nov 26 22:08:44.823 INFO request completed, latency_us: 235, response_code: 204, uri: /start/cid/1, method: POST, req_id: d51936f0-d40a-4b25-a1ec-06a409ada7b8, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
19960 Replay: started 1, returned:Ok(())
19961 ----------------------------------------------------------------
19962 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
19963 States: Active Offline Active
19964 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
19965 6133 Acked Write 4 Done Sent Done false
19966 6134 Acked Read 7 Done Sent Done false
19967 6135 Acked Write 4 Done Sent Done false
19968 6136 Acked Write 10 Done Sent Done false
19969 6137 Acked Write 10 Done Sent Done false
19970 6138 Acked Read 6 Done Sent Done false
19971 6139 Acked Write 2 Done Sent Done false
19972 6140 Acked Write 7 Done Sent Done false
19973 6141 Acked Read 3 Done Sent Done false
19974 6142 Acked Write 3 Done Sent Done false
19975 6143 Acked Write 9 Done Sent Done false
19976 6144 Acked Read 3 Done Sent Done false
19977 6145 Acked Write 7 Done Sent Done false
19978 6146 Acked Write 6 Done Sent Done false
19979 6147 Acked Flush 0 Done Sent Done false
19980 6148 Acked Write 4 Done Sent Done false
19981 6149 Acked Read 7 Done Sent Done false
19982 6150 Acked Read 9 Done Sent Done false
19983 6151 Acked Read 6 Done Sent Done false
19984 6152 Acked Flush 0 Done Sent Done false
19985 6153 Acked Read 10 Done Sent Done false
19986 6154 Acked Read 10 Done Sent Done false
19987 6155 Acked Write 3 Done Sent Done false
19988 6156 Acked Write 7 Done Sent Done false
19989 6157 Acked Read 2 Done Sent Done false
19990 6158 Acked Write 9 Done Sent Done false
19991 6159 Acked Read 7 Done Sent Done false
19992 6160 Acked Write 7 Done Sent Done false
19993 6161 Acked Read 10 Done Sent Done false
19994 6162 Acked Read 2 Done Sent Done false
19995 6163 Acked Read 6 Done Sent Done false
19996 6164 Acked Read 6 Done Sent Done false
19997 6165 Acked Read 4 Done Sent Done false
19998 6166 Acked Flush 0 Done Sent Done false
19999 6167 Acked Write 8 Done Sent Done false
20000 6168 Acked Read 9 Done Sent Done false
20001 6169 Acked Flush 0 Done Sent Done false
20002 6170 Acked Read 6 Done Sent Done false
20003 6171 Acked Write 2 Done Sent Done false
20004 6172 Acked Flush 0 Done Sent Done false
20005 6173 Acked Write 5 Done Sent Done false
20006 6174 Acked Write 4 Done Sent Done false
20007 6175 Acked Flush 0 Done Sent Done false
20008 6176 Acked Read 2 Done Sent Done false
20009 6177 Acked Write 4 Done Sent Done false
20010 6178 Acked Flush 0 Done Sent Done false
20011 6179 Acked Read 7 Done Sent Done false
20012 6180 Acked Write 1 Done Sent Done false
20013 6181 Acked Read 6 Done Sent Done false
20014 6182 Acked Read 3 Done Sent Done false
20015 6183 Acked Write 10 Done Sent Done false
20016 6184 Acked Read 6 Done Sent Done false
20017 6185 Acked Write 6 Done Sent Done false
20018 6186 Acked Read 2 Done Sent Done false
20019 6187 Acked Read 7 Done Sent Done false
20020 6188 Acked Write 4 Done Sent Done false
20021 6189 Acked Write 3 Done Sent Done false
20022 6190 Acked Read 3 Done Sent Done false
20023 6191 Acked Read 3 Done Sent Done false
20024 6192 Acked Read 1 Done Sent Done false
20025 6193 Acked Read 10 Done Sent Done false
20026 6194 Acked Write 9 Done Sent Done false
20027 6195 Acked Read 4 Done Sent Done false
20028 6196 Acked Write 9 Done Sent Done false
20029 6197 Acked Flush 0 Done Sent Done false
20030 6198 Acked Read 4 Done Sent Done false
20031 6199 Acked Write 1 Done Sent Done false
20032 6200 Acked Read 8 Done Sent Done false
20033 6201 Acked Write 5 Done Sent Done false
20034 6202 Acked Read 9 Done Sent Done false
20035 6203 Acked Read 6 Done Sent Done false
20036 6204 Acked Write 8 Done Sent Done false
20037 6205 Acked Read 3 Done Sent Done false
20038 6206 Acked Write 7 Done Sent Done false
20039 6207 Acked Read 3 Done Sent Done false
20040 6208 Acked Read 4 Done Sent Done false
20041 6209 Acked Write 3 Done Sent Done false
20042 6210 Acked Read 2 Done Sent Done false
20043 6211 Acked Write 4 Done Sent Done false
20044 6212 Acked Read 9 Done Sent Done false
20045 6213 Acked Write 6 Done Sent Done false
20046 6214 Acked Write 10 Done Sent Done false
20047 6215 Acked Write 1 Done Sent Done false
20048 6216 Acked Write 4 Done Sent Done false
20049 6217 Acked Write 7 Done Sent Done false
20050 6218 Acked Write 5 Done Sent Done false
20051 6219 Acked Read 5 Done Sent Done false
20052 6220 Acked Write 6 Done Sent Done false
20053 6221 Acked Write 2 Done Sent Done false
20054 6222 Acked Flush 0 Done Sent Done false
20055 6223 Acked Read 2 Done Sent Done false
20056 6224 Acked Flush 0 Done Sent Done false
20057 6225 Acked Read 6 Done Sent Done false
20058 6226 Acked Read 4 Done Sent Done false
20059 6227 Acked Write 6 Done Sent Done false
20060 6228 Acked Write 5 Done Sent Done false
20061 6229 Acked Read 2 Done Sent Done false
20062 6230 Acked Read 5 Done Sent Done false
20063 6231 Acked Read 1 Done Sent Done false
20064 6232 Acked Read 3 Done Sent Done false
20065 6233 Acked Read 1 Done Sent Done false
20066 6234 Acked Read 8 Done Sent Done false
20067 6235 Acked Write 5 Done Sent Done false
20068 6236 Acked Read 6 Done Sent Done false
20069 6237 Acked Write 8 Done Sent Done false
20070 6238 Acked Read 10 Done Sent Done false
20071 6239 Acked Read 7 Done Sent Done false
20072 6240 Acked Flush 0 Done Sent Done false
20073 6241 Acked Read 2 Done Sent Done false
20074 6242 Acked Read 5 Done Sent Done false
20075 6243 Acked Read 9 Done Sent Done false
20076 6244 Acked Read 2 Done Sent Done false
20077 6245 Acked Write 6 Done Sent Done false
20078 6246 Acked Write 3 Done Sent Done false
20079 6247 Acked Read 1 Done Sent Done false
20080 6248 Acked Write 2 Done Sent Done false
20081 6249 Acked Read 10 Done Sent Done false
20082 6250 Acked Read 6 Done Sent Done false
20083 6251 Acked Write 6 Done Sent Done false
20084 6252 Acked Flush 0 Done Sent Done false
20085 6253 Acked Read 5 Done Sent Done false
20086 6254 Acked Flush 0 Done Sent Done false
20087 6255 Acked Read 9 Done Sent Done false
20088 6256 Acked Flush 0 Done Sent Done false
20089 6257 Acked Read 3 Done Sent Done false
20090 6258 Acked Write 1 Done Sent Done false
20091 6259 Acked Write 7 Done Sent Done false
20092 6260 Acked Flush 0 Done Sent Done false
20093 6261 Acked Write 9 Done Sent Done false
20094 6262 Acked Write 2 Done Sent Done false
20095 6263 Acked Write 9 Done Sent Done false
20096 6264 Acked Write 9 Done Sent Done false
20097 6265 Acked Read 6 Done Sent Done false
20098 6266 Acked Write 6 Done Sent Done false
20099 6267 Acked Read 5 Done Sent Done false
20100 6268 Acked Read 5 Done Sent Done false
20101 6269 Acked Write 2 Done Sent Done false
20102 6270 Acked Read 7 Done Sent Done false
20103 6271 Acked Read 8 Done Sent Done false
20104 6272 Acked Write 7 Done Sent Done false
20105 6273 Acked Read 9 Done Sent Done false
20106 6274 Acked Read 4 Done Sent Done false
20107 6275 Acked Read 3 Done Sent Done false
20108 6276 Acked Read 6 Done Sent Done false
20109 6277 Acked Write 8 Done Sent Done false
20110 6278 Acked Write 6 Done Sent Done false
20111 6279 Acked Write 1 Done Sent Done false
20112 6280 Acked Write 7 Done Sent Done false
20113 6281 Acked Read 1 Done Sent Done false
20114 6282 Acked Write 6 Done Sent Done false
20115 6283 Acked Write 7 Done Sent Done false
20116 6284 Acked Read 1 Done Sent Done false
20117 6285 Acked Read 5 Done Sent Done false
20118 6286 Acked Write 10 Done Sent Done false
20119 6287 Acked Read 10 Done Sent Done false
20120 6288 Acked Read 10 Done Sent Done false
20121 6289 Acked Read 10 Done Sent Done false
20122 6290 Acked Read 4 Done Sent Done false
20123 6291 Acked Read 2 Done Sent Done false
20124 6292 Acked Write 3 Done Sent Done false
20125 6293 Acked Write 10 Done Sent Done false
20126 6294 Acked Read 8 Done Sent Done false
20127 6295 Acked Write 3 Done Sent Done false
20128 6296 Acked Read 1 Done Sent Done false
20129 6297 Acked Write 8 Done Sent Done false
20130 6298 Acked Write 7 Done Sent Done false
20131 6299 Acked Flush 0 Done Sent Done false
20132 6300 Acked Read 2 Done Sent Done false
20133 6301 Acked Read 5 Done Sent Done false
20134 6302 Acked Write 8 Done Sent Done false
20135 6303 Acked Write 8 Done Sent Done false
20136 6304 Acked Read 1 Done Sent Done false
20137 6305 Acked Read 3 Done Sent Done false
20138 6306 Acked Write 9 Done Sent Done false
20139 6307 Acked Read 2 Done Sent Done false
20140 6308 Acked Read 3 Done Sent Done false
20141 6309 Acked Flush 0 Done Sent Done false
20142 6310 Acked Read 4 Done Sent Done false
20143 6311 Acked Read 5 Done Sent Done false
20144 6312 Acked Read 10 Done Sent Done false
20145 6313 Acked Read 2 Done Sent Done false
20146 6314 Acked Write 8 Done Sent Done false
20147 6315 Acked Write 9 Done Sent Done false
20148 6316 Acked Read 6 Done Sent Done false
20149 6317 Acked Write 8 Done Sent Done false
20150 6318 Acked Write 4 Done Sent Done false
20151 6319 Acked Flush 0 Done Sent Done false
20152 6320 Acked Read 4 Done Sent Done false
20153 6321 Acked Read 3 Done Sent Done false
20154 6322 Acked Read 9 Done Sent Done false
20155 6323 Acked Write 8 Done Sent Done false
20156 6324 Acked Read 2 Done Sent Done false
20157 6325 Acked Read 10 Done Sent Done false
20158 6326 Acked Write 8 Done Sent Done false
20159 6327 Acked Write 4 Done Sent Done false
20160 6328 Acked Flush 0 Done Sent Done false
20161 6329 Acked Read 5 Done Sent Done false
20162 6330 Acked Write 5 Done Sent Done false
20163 6331 Acked Read 10 Done Sent Done false
20164 6332 Acked Read 4 Done Sent Done false
20165 6333 Acked Write 10 Done Sent Done false
20166 6334 Acked Read 1 Done Sent Done false
20167 6335 Acked Read 7 Done Sent Done false
20168 6336 Acked Read 9 Done Sent Done false
20169 6337 Acked Read 4 Done Sent Done false
20170 6338 Acked Write 3 Done Sent Done false
20171 6339 Acked Write 1 Done Sent Done false
20172 6340 Acked Write 3 Done Sent Done false
20173 6341 Acked Write 10 Done Sent Done false
20174 6342 Acked Read 10 Done Sent Done false
20175 6343 Acked Read 6 Done Sent Done false
20176 6344 Acked Write 2 Done Sent Done false
20177 6345 Acked Read 10 Done Sent Done false
20178 6346 Acked Write 4 Done Sent Done false
20179 6347 Acked Read 4 Done Sent Done false
20180 6348 Acked Write 7 Done Sent Done false
20181 6349 Acked Write 2 Done Sent Done false
20182 6350 Acked Write 1 Done Sent Done false
20183 6351 Acked Write 8 Done Sent Done false
20184 6352 Acked Write 6 Done Sent Done false
20185 6353 Acked Read 1 Done Sent Done false
20186 6354 Acked Read 10 Done Sent Done false
20187 6355 Acked Write 2 Done Sent Done false
20188 6356 Acked Write 7 Done Sent Done false
20189 6357 Acked Read 10 Done Sent Done false
20190 6358 Acked Write 5 Done Sent Done false
20191 6359 Acked Read 5 Done Sent Done false
20192 6360 Acked Flush 0 Done Sent Done false
20193 6361 Acked Read 10 Done Sent Done false
20194 6362 Acked Read 2 Done Sent Done false
20195 6363 Acked Read 10 Done Sent Done false
20196 6364 Acked Write 8 Done Sent Done false
20197 6365 Acked Write 6 Done Sent Done false
20198 6366 Acked Write 3 Done Sent Done false
20199 6367 Acked Write 5 Done Sent Done false
20200 6368 Acked Flush 0 Done Sent Done false
20201 6369 Acked Read 3 Done Sent Done false
20202 6370 Acked Write 7 Done Sent Done false
20203 6371 Acked Write 3 Done Sent Done false
20204 6372 Acked Read 5 Done Sent Done false
20205 6373 Acked Read 6 Done Sent Done false
20206 6374 Acked Write 9 Done Sent Done false
20207 6375 Acked Write 3 Done Sent Done false
20208 6376 Acked Read 1 Done Sent Done false
20209 6377 Acked Write 5 Done Sent Done false
20210 6378 Acked Write 3 Done Sent Done false
20211 6379 Acked Read 1 Done Sent Done false
20212 6380 Acked Read 8 Done Sent Done false
20213 6381 Acked Flush 0 Done Sent Done false
20214 6382 Acked Flush 0 Done Sent Done false
20215 6383 Acked Read 3 Done Sent Done false
20216 6384 Acked Flush 0 Done Sent Done false
20217 6385 Acked Read 9 Done Sent Done false
20218 6386 Acked Read 7 Done Sent Done false
20219 6387 Acked Flush 0 Done Sent Done false
20220 6388 Acked Read 5 Done Sent Done false
20221 6389 Acked Write 4 Done Sent Done false
20222 6390 Acked Read 6 Done Sent Done false
20223 6391 Acked Write 6 Done Sent Done false
20224 6392 Acked Read 10 Done Sent Done false
20225 6393 Acked Read 8 Done Sent Done false
20226 6394 Acked Read 6 Done Sent Done false
20227 6395 Acked Write 1 Done Sent Done false
20228 6396 Acked Read 7 Done Sent Done false
20229 6397 Acked Read 7 Done Sent Done false
20230 6398 Acked Read 5 Done Sent Done false
20231 6399 Acked Write 2 Done Sent Done false
20232 6400 Acked Write 10 Done Sent Done false
20233 6401 Acked Read 1 Done Sent Done false
20234 6402 Acked Write 4 Done Sent Done false
20235 6403 Acked Write 2 Done Sent Done false
20236 6404 Acked Read 6 Done Sent Done false
20237 6405 Acked Write 5 Done Sent Done false
20238 6406 Acked Write 1 Done Sent Done false
20239 6407 Acked Read 4 Done Sent Done false
20240 6408 Acked Read 6 Done Sent Done false
20241 6409 Acked Write 10 Done Sent Done false
20242 6410 Acked Read 3 Done Sent Done false
20243 6411 Acked Read 4 Done Sent Done false
20244 6412 Acked Write 10 Done Sent Done false
20245 6413 Acked Read 3 Done Sent Done false
20246 6414 Acked Read 6 Done Sent Done false
20247 6415 Acked Read 7 Done Sent Done false
20248 6416 Acked Write 9 Done Sent Done false
20249 6417 Acked Read 10 Done Sent Done false
20250 6418 Acked Read 1 Done Sent Done false
20251 6419 Acked Read 4 Done Sent Done false
20252 6420 Acked Write 9 Done Sent Done false
20253 6421 Acked Read 6 Done Sent Done false
20254 6422 Acked Read 7 Done Sent Done false
20255 6423 Acked Write 5 Done Sent Done false
20256 6424 Acked Read 7 Done Sent Done false
20257 6425 Acked Read 1 Done Sent Done false
20258 6426 Acked Flush 0 Done Sent Done false
20259 6427 Acked Read 2 Done Sent Done false
20260 6428 Acked Read 10 Done Sent Done false
20261 6429 Acked Read 4 Done Sent Done false
20262 6430 Acked Read 5 Done Sent Done false
20263 6431 Acked Read 6 Done Sent Done false
20264 6432 Acked Read 4 Done Sent Done false
20265 6433 Acked Write 10 Done Sent Done false
20266 STATES DS:0 DS:1 DS:2 TOTAL
20267 Sent 0 301 0 301
20268 Done 301 0 301 602
20269 Skipped 0 0 0 0
20270 Error 0 0 0 0
20271 Last Flush: 6426 6132 6426
20272 Downstairs last five completed: 6132 6131 6130 6129 6128
20273 Upstairs last five completed: 6433 6432 6431 6430 6429
20274 CLIENT: Up:0 ds:301 act:2
20275 [8820] got start action
20276 [8820] I am going to restart
20277 Starting downstairs at port 8820
20278 Make output file at "/tmp/dsc/downstairs-8820.txt"
20279 [8820][1] reports Starting
20280 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1221)
20281 [8820][1] reports Running
20282 ----------------------------------------------------------------
20283 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
20284 States: Active Offline Active
20285 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
20286 6133 Acked Write 4 Done Sent Done false
20287 6134 Acked Read 7 Done Sent Done false
20288 6135 Acked Write 4 Done Sent Done false
20289 6136 Acked Write 10 Done Sent Done false
20290 6137 Acked Write 10 Done Sent Done false
20291 6138 Acked Read 6 Done Sent Done false
20292 6139 Acked Write 2 Done Sent Done false
20293 6140 Acked Write 7 Done Sent Done false
20294 6141 Acked Read 3 Done Sent Done false
20295 6142 Acked Write 3 Done Sent Done false
20296 6143 Acked Write 9 Done Sent Done false
20297 6144 Acked Read 3 Done Sent Done false
20298 6145 Acked Write 7 Done Sent Done false
20299 6146 Acked Write 6 Done Sent Done false
20300 6147 Acked Flush 0 Done Sent Done false
20301 6148 Acked Write 4 Done Sent Done false
20302 6149 Acked Read 7 Done Sent Done false
20303 6150 Acked Read 9 Done Sent Done false
20304 6151 Acked Read 6 Done Sent Done false
20305 6152 Acked Flush 0 Done Sent Done false
20306 6153 Acked Read 10 Done Sent Done false
20307 6154 Acked Read 10 Done Sent Done false
20308 6155 Acked Write 3 Done Sent Done false
20309 6156 Acked Write 7 Done Sent Done false
20310 6157 Acked Read 2 Done Sent Done false
20311 6158 Acked Write 9 Done Sent Done false
20312 6159 Acked Read 7 Done Sent Done false
20313 6160 Acked Write 7 Done Sent Done false
20314 6161 Acked Read 10 Done Sent Done false
20315 6162 Acked Read 2 Done Sent Done false
20316 6163 Acked Read 6 Done Sent Done false
20317 6164 Acked Read 6 Done Sent Done false
20318 6165 Acked Read 4 Done Sent Done false
20319 6166 Acked Flush 0 Done Sent Done false
20320 6167 Acked Write 8 Done Sent Done false
20321 6168 Acked Read 9 Done Sent Done false
20322 6169 Acked Flush 0 Done Sent Done false
20323 6170 Acked Read 6 Done Sent Done false
20324 6171 Acked Write 2 Done Sent Done false
20325 6172 Acked Flush 0 Done Sent Done false
20326 6173 Acked Write 5 Done Sent Done false
20327 6174 Acked Write 4 Done Sent Done false
20328 6175 Acked Flush 0 Done Sent Done false
20329 6176 Acked Read 2 Done Sent Done false
20330 6177 Acked Write 4 Done Sent Done false
20331 6178 Acked Flush 0 Done Sent Done false
20332 6179 Acked Read 7 Done Sent Done false
20333 6180 Acked Write 1 Done Sent Done false
20334 6181 Acked Read 6 Done Sent Done false
20335 6182 Acked Read 3 Done Sent Done false
20336 6183 Acked Write 10 Done Sent Done false
20337 6184 Acked Read 6 Done Sent Done false
20338 6185 Acked Write 6 Done Sent Done false
20339 6186 Acked Read 2 Done Sent Done false
20340 6187 Acked Read 7 Done Sent Done false
20341 6188 Acked Write 4 Done Sent Done false
20342 6189 Acked Write 3 Done Sent Done false
20343 6190 Acked Read 3 Done Sent Done false
20344 6191 Acked Read 3 Done Sent Done false
20345 6192 Acked Read 1 Done Sent Done false
20346 6193 Acked Read 10 Done Sent Done false
20347 6194 Acked Write 9 Done Sent Done false
20348 6195 Acked Read 4 Done Sent Done false
20349 6196 Acked Write 9 Done Sent Done false
20350 6197 Acked Flush 0 Done Sent Done false
20351 6198 Acked Read 4 Done Sent Done false
20352 6199 Acked Write 1 Done Sent Done false
20353 6200 Acked Read 8 Done Sent Done false
20354 6201 Acked Write 5 Done Sent Done false
20355 6202 Acked Read 9 Done Sent Done false
20356 6203 Acked Read 6 Done Sent Done false
20357 6204 Acked Write 8 Done Sent Done false
20358 6205 Acked Read 3 Done Sent Done false
20359 6206 Acked Write 7 Done Sent Done false
20360 6207 Acked Read 3 Done Sent Done false
20361 6208 Acked Read 4 Done Sent Done false
20362 6209 Acked Write 3 Done Sent Done false
20363 6210 Acked Read 2 Done Sent Done false
20364 6211 Acked Write 4 Done Sent Done false
20365 6212 Acked Read 9 Done Sent Done false
20366 6213 Acked Write 6 Done Sent Done false
20367 6214 Acked Write 10 Done Sent Done false
20368 6215 Acked Write 1 Done Sent Done false
20369 6216 Acked Write 4 Done Sent Done false
20370 6217 Acked Write 7 Done Sent Done false
20371 6218 Acked Write 5 Done Sent Done false
20372 6219 Acked Read 5 Done Sent Done false
20373 6220 Acked Write 6 Done Sent Done false
20374 6221 Acked Write 2 Done Sent Done false
20375 6222 Acked Flush 0 Done Sent Done false
20376 6223 Acked Read 2 Done Sent Done false
20377 6224 Acked Flush 0 Done Sent Done false
20378 6225 Acked Read 6 Done Sent Done false
20379 6226 Acked Read 4 Done Sent Done false
20380 6227 Acked Write 6 Done Sent Done false
20381 6228 Acked Write 5 Done Sent Done false
20382 6229 Acked Read 2 Done Sent Done false
20383 6230 Acked Read 5 Done Sent Done false
20384 6231 Acked Read 1 Done Sent Done false
20385 6232 Acked Read 3 Done Sent Done false
20386 6233 Acked Read 1 Done Sent Done false
20387 6234 Acked Read 8 Done Sent Done false
20388 6235 Acked Write 5 Done Sent Done false
20389 6236 Acked Read 6 Done Sent Done false
20390 6237 Acked Write 8 Done Sent Done false
20391 6238 Acked Read 10 Done Sent Done false
20392 6239 Acked Read 7 Done Sent Done false
20393 6240 Acked Flush 0 Done Sent Done false
20394 6241 Acked Read 2 Done Sent Done false
20395 6242 Acked Read 5 Done Sent Done false
20396 6243 Acked Read 9 Done Sent Done false
20397 6244 Acked Read 2 Done Sent Done false
20398 6245 Acked Write 6 Done Sent Done false
20399 6246 Acked Write 3 Done Sent Done false
20400 6247 Acked Read 1 Done Sent Done false
20401 6248 Acked Write 2 Done Sent Done false
20402 6249 Acked Read 10 Done Sent Done false
20403 6250 Acked Read 6 Done Sent Done false
20404 6251 Acked Write 6 Done Sent Done false
20405 6252 Acked Flush 0 Done Sent Done false
20406 6253 Acked Read 5 Done Sent Done false
20407 6254 Acked Flush 0 Done Sent Done false
20408 6255 Acked Read 9 Done Sent Done false
20409 6256 Acked Flush 0 Done Sent Done false
20410 6257 Acked Read 3 Done Sent Done false
20411 6258 Acked Write 1 Done Sent Done false
20412 6259 Acked Write 7 Done Sent Done false
20413 6260 Acked Flush 0 Done Sent Done false
20414 6261 Acked Write 9 Done Sent Done false
20415 6262 Acked Write 2 Done Sent Done false
20416 6263 Acked Write 9 Done Sent Done false
20417 6264 Acked Write 9 Done Sent Done false
20418 6265 Acked Read 6 Done Sent Done false
20419 6266 Acked Write 6 Done Sent Done false
20420 6267 Acked Read 5 Done Sent Done false
20421 6268 Acked Read 5 Done Sent Done false
20422 6269 Acked Write 2 Done Sent Done false
20423 6270 Acked Read 7 Done Sent Done false
20424 6271 Acked Read 8 Done Sent Done false
20425 6272 Acked Write 7 Done Sent Done false
20426 6273 Acked Read 9 Done Sent Done false
20427 6274 Acked Read 4 Done Sent Done false
20428 6275 Acked Read 3 Done Sent Done false
20429 6276 Acked Read 6 Done Sent Done false
20430 6277 Acked Write 8 Done Sent Done false
20431 6278 Acked Write 6 Done Sent Done false
20432 6279 Acked Write 1 Done Sent Done false
20433 6280 Acked Write 7 Done Sent Done false
20434 6281 Acked Read 1 Done Sent Done false
20435 6282 Acked Write 6 Done Sent Done false
20436 6283 Acked Write 7 Done Sent Done false
20437 6284 Acked Read 1 Done Sent Done false
20438 6285 Acked Read 5 Done Sent Done false
20439 6286 Acked Write 10 Done Sent Done false
20440 6287 Acked Read 10 Done Sent Done false
20441 6288 Acked Read 10 Done Sent Done false
20442 6289 Acked Read 10 Done Sent Done false
20443 6290 Acked Read 4 Done Sent Done false
20444 6291 Acked Read 2 Done Sent Done false
20445 6292 Acked Write 3 Done Sent Done false
20446 6293 Acked Write 10 Done Sent Done false
20447 6294 Acked Read 8 Done Sent Done false
20448 6295 Acked Write 3 Done Sent Done false
20449 6296 Acked Read 1 Done Sent Done false
20450 6297 Acked Write 8 Done Sent Done false
20451 6298 Acked Write 7 Done Sent Done false
20452 6299 Acked Flush 0 Done Sent Done false
20453 6300 Acked Read 2 Done Sent Done false
20454 6301 Acked Read 5 Done Sent Done false
20455 6302 Acked Write 8 Done Sent Done false
20456 6303 Acked Write 8 Done Sent Done false
20457 6304 Acked Read 1 Done Sent Done false
20458 6305 Acked Read 3 Done Sent Done false
20459 6306 Acked Write 9 Done Sent Done false
20460 6307 Acked Read 2 Done Sent Done false
20461 6308 Acked Read 3 Done Sent Done false
20462 6309 Acked Flush 0 Done Sent Done false
20463 6310 Acked Read 4 Done Sent Done false
20464 6311 Acked Read 5 Done Sent Done false
20465 6312 Acked Read 10 Done Sent Done false
20466 6313 Acked Read 2 Done Sent Done false
20467 6314 Acked Write 8 Done Sent Done false
20468 6315 Acked Write 9 Done Sent Done false
20469 6316 Acked Read 6 Done Sent Done false
20470 6317 Acked Write 8 Done Sent Done false
20471 6318 Acked Write 4 Done Sent Done false
20472 6319 Acked Flush 0 Done Sent Done false
20473 6320 Acked Read 4 Done Sent Done false
20474 6321 Acked Read 3 Done Sent Done false
20475 6322 Acked Read 9 Done Sent Done false
20476 6323 Acked Write 8 Done Sent Done false
20477 6324 Acked Read 2 Done Sent Done false
20478 6325 Acked Read 10 Done Sent Done false
20479 6326 Acked Write 8 Done Sent Done false
20480 6327 Acked Write 4 Done Sent Done false
20481 6328 Acked Flush 0 Done Sent Done false
20482 6329 Acked Read 5 Done Sent Done false
20483 6330 Acked Write 5 Done Sent Done false
20484 6331 Acked Read 10 Done Sent Done false
20485 6332 Acked Read 4 Done Sent Done false
20486 6333 Acked Write 10 Done Sent Done false
20487 6334 Acked Read 1 Done Sent Done false
20488 6335 Acked Read 7 Done Sent Done false
20489 6336 Acked Read 9 Done Sent Done false
20490 6337 Acked Read 4 Done Sent Done false
20491 6338 Acked Write 3 Done Sent Done false
20492 6339 Acked Write 1 Done Sent Done false
20493 6340 Acked Write 3 Done Sent Done false
20494 6341 Acked Write 10 Done Sent Done false
20495 6342 Acked Read 10 Done Sent Done false
20496 6343 Acked Read 6 Done Sent Done false
20497 6344 Acked Write 2 Done Sent Done false
20498 6345 Acked Read 10 Done Sent Done false
20499 6346 Acked Write 4 Done Sent Done false
20500 6347 Acked Read 4 Done Sent Done false
20501 6348 Acked Write 7 Done Sent Done false
20502 6349 Acked Write 2 Done Sent Done false
20503 6350 Acked Write 1 Done Sent Done false
20504 6351 Acked Write 8 Done Sent Done false
20505 6352 Acked Write 6 Done Sent Done false
20506 6353 Acked Read 1 Done Sent Done false
20507 6354 Acked Read 10 Done Sent Done false
20508 6355 Acked Write 2 Done Sent Done false
20509 6356 Acked Write 7 Done Sent Done false
20510 6357 Acked Read 10 Done Sent Done false
20511 6358 Acked Write 5 Done Sent Done false
20512 6359 Acked Read 5 Done Sent Done false
20513 6360 Acked Flush 0 Done Sent Done false
20514 6361 Acked Read 10 Done Sent Done false
20515 6362 Acked Read 2 Done Sent Done false
20516 6363 Acked Read 10 Done Sent Done false
20517 6364 Acked Write 8 Done Sent Done false
20518 6365 Acked Write 6 Done Sent Done false
20519 6366 Acked Write 3 Done Sent Done false
20520 6367 Acked Write 5 Done Sent Done false
20521 6368 Acked Flush 0 Done Sent Done false
20522 6369 Acked Read 3 Done Sent Done false
20523 6370 Acked Write 7 Done Sent Done false
20524 6371 Acked Write 3 Done Sent Done false
20525 6372 Acked Read 5 Done Sent Done false
20526 6373 Acked Read 6 Done Sent Done false
20527 6374 Acked Write 9 Done Sent Done false
20528 6375 Acked Write 3 Done Sent Done false
20529 6376 Acked Read 1 Done Sent Done false
20530 6377 Acked Write 5 Done Sent Done false
20531 6378 Acked Write 3 Done Sent Done false
20532 6379 Acked Read 1 Done Sent Done false
20533 6380 Acked Read 8 Done Sent Done false
20534 6381 Acked Flush 0 Done Sent Done false
20535 6382 Acked Flush 0 Done Sent Done false
20536 6383 Acked Read 3 Done Sent Done false
20537 6384 Acked Flush 0 Done Sent Done false
20538 6385 Acked Read 9 Done Sent Done false
20539 6386 Acked Read 7 Done Sent Done false
20540 6387 Acked Flush 0 Done Sent Done false
20541 6388 Acked Read 5 Done Sent Done false
20542 6389 Acked Write 4 Done Sent Done false
20543 6390 Acked Read 6 Done Sent Done false
20544 6391 Acked Write 6 Done Sent Done false
20545 6392 Acked Read 10 Done Sent Done false
20546 6393 Acked Read 8 Done Sent Done false
20547 6394 Acked Read 6 Done Sent Done false
20548 6395 Acked Write 1 Done Sent Done false
20549 6396 Acked Read 7 Done Sent Done false
20550 6397 Acked Read 7 Done Sent Done false
20551 6398 Acked Read 5 Done Sent Done false
20552 6399 Acked Write 2 Done Sent Done false
20553 6400 Acked Write 10 Done Sent Done false
20554 6401 Acked Read 1 Done Sent Done false
20555 6402 Acked Write 4 Done Sent Done false
20556 6403 Acked Write 2 Done Sent Done false
20557 6404 Acked Read 6 Done Sent Done false
20558 6405 Acked Write 5 Done Sent Done false
20559 6406 Acked Write 1 Done Sent Done false
20560 6407 Acked Read 4 Done Sent Done false
20561 6408 Acked Read 6 Done Sent Done false
20562 6409 Acked Write 10 Done Sent Done false
20563 6410 Acked Read 3 Done Sent Done false
20564 6411 Acked Read 4 Done Sent Done false
20565 6412 Acked Write 10 Done Sent Done false
20566 6413 Acked Read 3 Done Sent Done false
20567 6414 Acked Read 6 Done Sent Done false
20568 6415 Acked Read 7 Done Sent Done false
20569 6416 Acked Write 9 Done Sent Done false
20570 6417 Acked Read 10 Done Sent Done false
20571 6418 Acked Read 1 Done Sent Done false
20572 6419 Acked Read 4 Done Sent Done false
20573 6420 Acked Write 9 Done Sent Done false
20574 6421 Acked Read 6 Done Sent Done false
20575 6422 Acked Read 7 Done Sent Done false
20576 6423 Acked Write 5 Done Sent Done false
20577 6424 Acked Read 7 Done Sent Done false
20578 6425 Acked Read 1 Done Sent Done false
20579 6426 Acked Flush 0 Done Sent Done false
20580 6427 Acked Read 2 Done Sent Done false
20581 6428 Acked Read 10 Done Sent Done false
20582 6429 Acked Read 4 Done Sent Done false
20583 6430 Acked Read 5 Done Sent Done false
20584 6431 Acked Read 6 Done Sent Done false
20585 6432 Acked Read 4 Done Sent Done false
20586 6433 Acked Write 10 Done Sent Done false
20587 6434 Acked Flush 0 Done Sent Done false
20588 STATES DS:0 DS:1 DS:2 TOTAL
20589 Sent 0 302 0 302
20590 Done 302 0 302 604
20591 Skipped 0 0 0 0
20592 Error 0 0 0 0
20593 Last Flush: 6434 6132 6434
20594 Downstairs last five completed: 6132 6131 6130 6129 6128
20595 Upstairs last five completed: 6434 6433 6432 6431 6430
20596 CLIENT: Up:0 ds:302 act:2
205972024-11-26T22:08:50.586ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
205982024-11-26T22:08:50.586ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:41567) = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
205992024-11-26T22:08:50.587ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
206002024-11-26T22:08:50.588ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
206012024-11-26T22:08:50.588ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
206022024-11-26T22:08:50.588ZINFOcrucible: send last flush ID to this DS: 6132 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
206032024-11-26T22:08:50.588ZINFOcrucible: Replied this last flush ID: 6132 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
206042024-11-26T22:08:50.588ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
206052024-11-26T22:08:50.588ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
206062024-11-26T22:08:50.588ZINFOcrucible: [1] client re-new 302 jobs since flush 6132 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
206072024-11-26T22:08:50.588ZINFOcrucible: [1] Replayed 302 jobs since flush: 6132 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
20608 ----------------------------------------------------------------
20609 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
20610 Downstairs last five completed: 6434 6433 6432 6431 6430
20611 Upstairs last five completed: 6434 6433 6432 6431 6430
20612 CLIENT: Up:0 ds:0 act:3
20613 Replay: All jobs finished
20614 Main task has work to do, go find it
20615 got dsc Stop(2)
20616 stop 2
20617 Nov 26 22:08:52.828 INFO request completed, latency_us: 376, [8830] Got stop action so:false kr:false
20618 response_code: 204, uri: /stop/cid/2, method: POST, req_id: d1fd0ee3-5d15-498e-a9e5-3a57f726ae21, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
206192024-11-26T22:08:52.829ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
206202024-11-26T22:08:52.829ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
206212024-11-26T22:08:52.829ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
206222024-11-26T22:08:52.829ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
20623 Nov 26 22:08:52.829 INFO request completed, latency_us: 123{"msg":"[2] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-26T22:08:52.829367668Z","hostname":"w-01JDN7ZQ17ABQG15KHKN3F2D26","pid":1203,"client":"2","":"downstairs","session_id":"86a36c17-3718-484c-87fd-2b8a1dc2ab18"}
20624 , response_code: 200, uri: /state/cid/2, method: GET, req_id: f7a2b720-87cd-4242-b906-e3ac6d57a826, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
206252024-11-26T22:08:52.829ZINFOcrucible: client task is exiting = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
20626 [8830] Exited with: ExitStatus(unix_wait_status(9))
20627 [8830][2] reports Exit
20628 Nov 26 22:08:56.831 INFO request completed, latency_us: 214, response_code: 200, uri: /state/cid/2, method: GET, req_id: b687363a-a376-4ef1-80b8-57c9c3b83f3e, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
20629 001/300 Read block 3460 len 28672
20630 002/300 Write block 3279 len 28672 data: 4 5 4 4 4 4 5
20631 003/300 Write block 3285 len 16384 data: 6 5 5 5
20632 004/300 Write block 2772 len 8192 data: 6 6
20633 005/300 Read block 3026 len 40960
20634 006/300 Read block 4136 len 8192
20635 007/300 Read block 3227 len 16384
20636 008/300 Read block 3634 len 36864
20637 009/300 Read block 4633 len 32768
20638 010/300 Read block 2022 len 16384
20639 011/300 Write block 4641 len 8192 data: 5 4
20640 012/300 Read block 2325 len 12288
20641 013/300 Read block 4122 len 12288
20642 014/300 Read block 337 len 32768
20643 015/300 Write block 311 len 8192 data: 3 3
20644 016/300 Read block 3537 len 24576
20645 017/300 Write block 4710 len 28672 data: 5 5 4 4 4 4 4
20646 018/300 Write block 681 len 16384 data: 3 3 2 3
20647 019/300 Read block 382 len 40960
20648 020/300 Flush
20649 021/300 Write block 529 len 28672 data: 4 3 3 2 3 3 4
20650 022/300 Read block 2467 len 24576
20651 023/300 Write block 4500 len 36864 data: 3 3 3 2 3 3 3 4 4
20652 024/300 Write block 3769 len 36864 data: 5 4 4 4 4 3 4 3 5
20653 025/300 Read block 1978 len 28672
20654 026/300 Read block 3509 len 32768
20655 027/300 Write block 4919 len 36864 data: 2 4 4 4 4 5 6 6 6
20656 028/300 Write block 1833 len 4096 data: 6
20657 029/300 Read block 2615 len 12288
20658 030/300 Read block 518 len 36864
20659 031/300 Write block 1806 len 16384 data: 5 6 5 5
20660 032/300 Read block 2625 len 28672
20661 033/300 Read block 3970 len 40960
20662 034/300 Write block 498 len 40960 data: 4 4 4 3 3 3 4 4 3 3
20663 035/300 Write block 1987 len 16384 data: 6 7 7 6
20664 036/300 Read block 509 len 32768
20665 037/300 Write block 3487 len 40960 data: 3 3 3 5 6 5 6 7 7 7
20666 038/300 Read block 432 len 36864
20667 039/300 Read block 3952 len 16384
20668 040/300 Flush
20669 041/300 Read block 4960 len 32768
20670 042/300 Flush
20671 043/300 Write block 4676 len 40960 data: 4 7 7 7 9 8 6 5 5 4
20672 044/300 Write block 1539 len 8192 data: 5 7
20673 045/300 Write block 1135 len 40960 data: 6 4 3 3 3 4 4 3 3 5
20674 046/300 Read block 637 len 28672
20675 047/300 Read block 2078 len 20480
20676 048/300 Write block 2455 len 24576 data: 6 3 3 3 4 3
20677 049/300 Write block 3584 len 8192 data: 4 4
20678 050/300 Write block 4882 len 28672 data: 2 2 2 3 3 3 2
20679 051/300 Flush
20680 052/300 Write block 4089 len 8192 data: 5 5
20681 053/300 Write block 1952 len 16384 data: 2 3 3 4
20682 054/300 Read block 1467 len 8192
20683 055/300 Write block 6 len 28672 data: 6 6 6 5 5 4 4
20684 056/300 Read block 3523 len 32768
20685 057/300 Read block 1163 len 4096
20686 058/300 Write block 8 len 8192 data: 7 6
20687 059/300 Write block 4801 len 36864 data: 4 5 4 4 3 5 5 6 7
20688 060/300 Read block 3094 len 32768
20689 061/300 Write block 76 len 20480 data: 3 3 4 4 3
20690 062/300 Write block 38 len 24576 data: 6 6 6 5 5 5
20691 063/300 Read block 1568 len 28672
20692 064/300 Flush
20693 065/300 Write block 2201 len 8192 data: 4 4
20694 066/300 Write block 937 len 36864 data: 3 3 2 2 3 3 3 4 4
20695 067/300 Read block 77 len 28672
20696 068/300 Write block 554 len 16384 data: 3 3 3 4
20697 069/300 Read block 1397 len 36864
20698 070/300 Write block 745 len 4096 data: 4
20699 071/300 Write block 3713 len 32768 data: 4 4 4 5 5 7 7 5
20700 072/300 Read block 3420 len 36864
20701 073/300 Read block 1785 len 4096
20702 074/300 Write block 4275 len 16384 data: 3 3 3 4
20703 075/300 Read block 4788 len 24576
20704 076/300 Flush
20705 077/300 Read block 436 len 4096
20706 078/300 Read block 1468 len 8192
20707 079/300 Write block 1679 len 4096 data: 4
20708 080/300 Write block 3312 len 32768 data: 5 4 5 6 6 6 6 6
20709 081/300 Read block 84 len 36864
20710 082/300 Read block 4684 len 20480
20711 083/300 Write block 2373 len 32768 data: 6 4 4 3 3 4 3 3
20712 084/300 Read block 1168 len 24576
20713 085/300 Flush
20714 086/300 Write block 4538 len 40960 data: 5 5 5 3 2 2 3 4 4 5
20715 087/300 Write block 1461 len 8192 data: 3 4
20716 088/300 Read block 1482 len 32768
20717 089/300 Read block 4037 len 16384
20718 090/300 Write block 694 len 36864 data: 2 2 2 2 3 3 4 3 4
20719 091/300 Read block 4583 len 8192
20720 092/300 Read block 4881 len 32768
20721 093/300 Flush
20722 094/300 Flush
20723 095/300 Read block 981 len 28672
20724 096/300 Read block 604 len 8192
20725 097/300 Write block 1997 len 32768 data: 3 4 5 4 5 3 3 3
20726 098/300 Write block 3517 len 12288 data: 2 2 2
20727 099/300 Read block 1354 len 16384
20728 100/300 Read block 2540 len 16384
20729 101/300 Read block 4142 len 4096
20730 102/300 Write block 1102 len 12288 data: 6 7 5
20731 103/300 Write block 502 len 20480 data: 4 4 5 5 4
20732 104/300 Read block 1433 len 32768
20733 105/300 Read block 2517 len 40960
20734 106/300 Read block 3083 len 24576
20735 107/300 Flush
20736 108/300 Read block 3649 len 4096
20737 109/300 Write block 4264 len 8192 data: 3 3
20738 110/300 Write block 2442 len 32768 data: 4 5 5 5 5 4 4 4
20739 111/300 Read block 2094 len 16384
20740 112/300 Read block 1023 len 20480
20741 113/300 Read block 263 len 28672
20742 114/300 Write block 2499 len 28672 data: 3 5 5 5 5 6 5
20743 115/300 Read block 541 len 40960
20744 116/300 Write block 3696 len 12288 data: 4 4 5
20745 117/300 Read block 3817 len 32768
20746 118/300 Flush
20747 119/300 Read block 3668 len 40960
20748 120/300 Write block 2925 len 36864 data: 4 3 3 3 3 4 3 4 4
20749 121/300 Write block 857 len 24576 data: 4 4 4 5 5 5
20750 122/300 Read block 4512 len 36864
20751 123/300 Read block 2282 len 32768
20752 124/300 Read block 3660 len 20480
20753 125/300 Read block 2825 len 12288
20754 126/300 Read block 4444 len 24576
20755 127/300 Flush
20756 128/300 Read block 4862 len 16384
20757 129/300 Write block 2606 len 24576 data: 5 7 6 5 6 7
20758 130/300 Read block 3224 len 12288
20759 131/300 Flush
20760 132/300 Read block 3874 len 28672
20761 133/300 Write block 433 len 24576 data: 4 5 4 5 4 4
20762 134/300 Write block 873 len 20480 data: 3 5 6 5 4
20763 135/300 Read block 3981 len 8192
20764 136/300 Read block 2740 len 24576
20765 137/300 Write block 2061 len 4096 data: 6
20766 138/300 Read block 2843 len 32768
20767 139/300 Flush
20768 140/300 Read block 2907 len 36864
20769 141/300 Read block 1073 len 32768
20770 142/300 Read block 2251 len 8192
20771 143/300 Write block 1269 len 28672 data: 3 4 3 3 4 5 5
20772 144/300 Flush
20773 145/300 Flush
20774 146/300 Read block 267 len 40960
20775 147/300 Write block 2341 len 28672 data: 5 5 4 4 4 3 2
20776 148/300 Read block 2302 len 20480
20777 149/300 Write block 3663 len 16384 data: 5 6 7 8
20778 150/300 Write block 2983 len 16384 data: 3 4 5 5
20779 151/300 Read block 4467 len 20480
20780 152/300 Write block 4086 len 36864 data: 3 3 5 6 6 5 4 3 3
20781 153/300 Write block 2968 len 20480 data: 6 6 5 4 4
20782 154/300 Write block 2708 len 32768 data: 5 7 6 6 6 7 7 8
20783 155/300 Flush
20784 156/300 Write block 2374 len 4096 data: 5
20785 157/300 Write block 801 len 16384 data: 6 5 5 4
20786 158/300 Read block 3506 len 8192
20787 159/300 Flush
20788 160/300 Flush
20789 161/300 Read block 782 len 8192
20790 162/300 Read block 218 len 4096
20791 163/300 Flush
20792 164/300 Flush
20793 165/300 Flush
20794 166/300 Write block 834 len 28672 data: 4 3 3 2 2 2 2
20795 167/300 Read block 2522 len 4096
20796 168/300 Write block 3635 len 4096 data: 3
20797 169/300 Write block 3309 len 20480 data: 5 4 4 6 5
20798 170/300 Read block 191 len 36864
20799 171/300 Write block 3066 len 4096 data: 5
20800 172/300 Read block 3139 len 28672
20801 173/300 Read block 4901 len 32768
20802 174/300 Read block 4113 len 40960
20803 175/300 Read block 1642 len 40960
20804 176/300 Read block 15 len 12288
20805 177/300 Read block 2981 len 20480
20806 178/300 Write block 4040 len 32768 data: 5 6 7 7 6 7 6 6
20807 179/300 Write block 991 len 8192 data: 3 3
20808 180/300 Write block 4003 len 16384 data: 5 5 5 5
20809 181/300 Write block 4693 len 20480 data: 9 9 8 5 5
20810 182/300 Read block 769 len 16384
20811 183/300 Flush
20812 184/300 Write block 4893 len 36864 data: 5 6 5 6 5 4 4 3 3
20813 185/300 Read block 3203 len 36864
20814 186/300 Write block 104 len 16384 data: 5 3 3 3
20815 187/300 Write block 3585 len 12288 data: 5 4 4
20816 188/300 Flush
20817 189/300 Read block 339 len 20480
20818 190/300 Write block 1001 len 28672 data: 3 4 4 5 4 4 4
20819 191/300 Read block 4896 len 28672
20820 192/300 Read block 4106 len 40960
20821 193/300 Write block 2846 len 16384 data: 8 6 6 4
20822 194/300 Read block 82 len 36864
20823 195/300 Flush
20824 196/300 Read block 1405 len 4096
20825 197/300 Write block 4127 len 40960 data: 4 4 3 3 3 3 2 2 2 2
20826 198/300 Flush
20827 199/300 Read block 4168 len 36864
20828 200/300 Read block 354 len 8192
20829 201/300 Read block 4395 len 32768
20830 202/300 Write block 4197 len 12288 data: 7 7 6
20831 203/300 Flush
20832 204/300 Read block 1088 len 40960
20833 205/300 Read block 1881 len 24576
20834 206/300 Read block 4145 len 36864
20835 207/300 Read block 3296 len 36864
20836 208/300 Write block 4015 len 12288 data: 7 6 6
20837 209/300 Write block 1051 len 32768 data: 5 5 5 6 6 5 5 5
20838 210/300 Read block 3487 len 8192
20839 211/300 Write block 2824 len 16384 data: 3 3 3 3
20840 212/300 Write block 4142 len 12288 data: 4 5 7
20841 213/300 Read block 1459 len 28672
20842 214/300 Write block 3010 len 4096 data: 7
20843 215/300 Write block 1936 len 24576 data: 8 9 10 10 10 9
20844 216/300 Read block 4456 len 12288
20845 217/300 Write block 4516 len 28672 data: 4 3 3 3 3 4 4
20846 218/300 Read block 4563 len 20480
20847 219/300 Read block 4432 len 20480
20848 220/300 Read block 3430 len 4096
20849 221/300 Write block 3874 len 12288 data: 7 6 4
20850 222/300 Read block 2441 len 40960
20851 223/300 Write block 3278 len 28672 data: 3 5 6 5 5 5 5
20852 224/300 Write block 2727 len 40960 data: 3 4 4 5 6 5 4 5 5 6
20853 225/300 Write block 2431 len 32768 data: 3 4 4 5 4 5 4 4
20854 226/300 Read block 375 len 20480
20855 227/300 Read block 3514 len 16384
20856 228/300 Write block 3314 len 8192 data: 6 7
20857 229/300 Read block 1136 len 8192
20858 230/300 Flush
20859 231/300 Write block 1781 len 12288 data: 5 5 4
20860 232/300 Read block 2441 len 32768
20861 233/300 Write block 2443 len 40960 data: 6 6 6 6 5 5 5 4 4 4
20862 234/300 Read block 998 len 12288
20863 235/300 Read block 3467 len 4096
20864 236/300 Write block 1775 len 24576 data: 6 7 7 7 8 7
20865 237/300 Write block 3079 len 8192 data: 3 4
20866 238/300 Write block 1241 len 32768 data: 4 4 4 6 6 6 6 5
20867 239/300 Read block 380 len 8192
20868 240/300 Read block 4043 len 12288
20869 241/300 Read block 1451 len 28672
20870 242/300 Write block 1500 len 40960 data: 6 5 5 5 5 4 4 3 3 2
20871 243/300 Read block 4254 len 24576
20872 244/300 Write block 944 len 20480 data: 5 5 5 5 4
20873 245/300 Read block 350 len 28672
20874 246/300 Write block 2323 len 36864 data: 5 4 3 3 4 4 5 5 4
20875 247/300 Read block 1831 len 28672
20876 248/300 Write block 143 len 12288 data: 4 5 3
20877 249/300 Read block 2549 len 24576
20878 250/300 Write block 4595 len 32768 data: 5 6 6 6 5 5 5 6
20879 251/300 Read block 4783 len 28672
20880 252/300 Write block 2875 len 24576 data: 5 6 7 5 5 5
20881 253/300 Write block 3378 len 8192 data: 4 4
20882 254/300 Write block 4129 len 32768 data: 4 4 4 4 3 3 3 3
20883 255/300 Read block 1702 len 28672
20884 256/300 Write block 3223 len 32768 data: 4 5 4 6 7 9 10 11
20885 257/300 Read block 4762 len 16384
20886 258/300 Flush
20887 259/300 Write block 2442 len 16384 data: 5 7 7 7
20888 260/300 Read block 408 len 36864
20889 261/300 Read block 226 len 8192
20890 262/300 Write block 4292 len 20480 data: 4 4 3 3 3
20891 263/300 Write block 3191 len 24576 data: 2 3 4 4 4 4
20892 264/300 Flush
20893 265/300 Read block 1609 len 16384
20894 266/300 Write block 2177 len 40960 data: 3 3 3 3 3 4 4 4 4 3
20895 267/300 Flush
20896 268/300 Write block 3825 len 24576 data: 4 4 6 6 6 7
20897 269/300 Write block 3446 len 12288 data: 3 4 4
20898 270/300 Write block 1153 len 36864 data: 4 4 4 4 4 4 5 5 5
20899 271/300 Write block 339 len 8192 data: 5 6
20900 272/300 Write block 3958 len 32768 data: 3 3 4 3 2 2 2 3
20901 273/300 Read block 850 len 4096
20902 274/300 Read block 4361 len 28672
20903 275/300 Read block 4926 len 28672
20904 276/300 Write block 2808 len 40960 data: 4 5 4 5 6 7 5 3 4 4
20905 277/300 Write block 3791 len 12288 data: 4 5 5
20906 278/300 Write block 636 len 8192 data: 5 6
20907 279/300 Read block 1129 len 28672
20908 280/300 Write block 1195 len 16384 data: 7 8 9 6
20909 281/300 Write block 2092 len 8192 data: 3 3
20910 282/300 Read block 4682 len 8192
20911 283/300 Read block 4784 len 4096
20912 284/300 Read block 3967 len 12288
20913 285/300 Read block 3251 len 8192
20914 286/300 Read block 3172 len 24576
20915 287/300 Read block 735 len 28672
20916 288/300 Write block 3931 len 4096 data: 6
20917 289/300 Write block 4925 len 36864 data: 7 7 7 5 4 4 2 4 4
20918 290/300 Read block 1942 len 32768
20919 291/300 Write block 667 len 8192 data: 5 4
20920 292/300 Read block 2013 len 36864
20921 293/300 Write block 3286 len 40960 data: 6 6 6 4 5 5 4 5 5 4
20922 294/300 Write block 1745 len 8192 data: 3 3
20923 295/300 Write block 3270 len 12288 data: 2 2 4
20924 296/300 Flush
20925 297/300 Write block 3098 len 40960 data: 2 2 2 2 2 2 2 3 3 4
20926 298/300 Read block 362 len 32768
20927 299/300 Write block 1129 len 28672 data: 3 3 4 5 7 6 7
20928 300/300 Write block 4264 len 8192 data: 4 4
20929 301/300 Read block 2889 len 8192
20930 Main task has work to do, go find it
20931 got dsc Start(2)
20932 start 2
20933 Nov 26 22:08:57.064 INFO request completed, latency_us: 218, response_code: 204, uri: /start/cid/2, method: POST, req_id: d46d0682-ac86-4d15-99c6-80fd6b451226, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
20934 Replay: started 2, returned:Ok(())
20935 ----------------------------------------------------------------
20936 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
20937 States: Active Active Offline
20938 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
20939 6435 Acked Read 7 Done Done Sent false
20940 6436 Acked Write 7 Done Done Sent false
20941 6437 Acked Write 4 Done Done Sent false
20942 6438 Acked Write 2 Done Done Sent false
20943 6439 Acked Read 10 Done Done Sent false
20944 6440 Acked Read 2 Done Done Sent false
20945 6441 Acked Read 4 Done Done Sent false
20946 6442 Acked Read 9 Done Done Sent false
20947 6443 Acked Read 8 Done Done Sent false
20948 6444 Acked Read 4 Done Done Sent false
20949 6445 Acked Write 2 Done Done Sent false
20950 6446 Acked Read 3 Done Done Sent false
20951 6447 Acked Read 3 Done Done Sent false
20952 6448 Acked Read 8 Done Done Sent false
20953 6449 Acked Write 2 Done Done Sent false
20954 6450 Acked Read 6 Done Done Sent false
20955 6451 Acked Write 7 Done Done Sent false
20956 6452 Acked Write 4 Done Done Sent false
20957 6453 Acked Read 10 Done Done Sent false
20958 6454 Acked Flush 0 Done Done Sent false
20959 6455 Acked Write 7 Done Done Sent false
20960 6456 Acked Read 6 Done Done Sent false
20961 6457 Acked Write 9 Done Done Sent false
20962 6458 Acked Write 9 Done Done Sent false
20963 6459 Acked Read 7 Done Done Sent false
20964 6460 Acked Read 8 Done Done Sent false
20965 6461 Acked Write 9 Done Done Sent false
20966 6462 Acked Write 1 Done Done Sent false
20967 6463 Acked Read 3 Done Done Sent false
20968 6464 Acked Read 9 Done Done Sent false
20969 6465 Acked Write 4 Done Done Sent false
20970 6466 Acked Read 7 Done Done Sent false
20971 6467 Acked Read 10 Done Done Sent false
20972 6468 Acked Write 10 Done Done Sent false
20973 6469 Acked Write 4 Done Done Sent false
20974 6470 Acked Read 8 Done Done Sent false
20975 6471 Acked Write 10 Done Done Sent false
20976 6472 Acked Read 9 Done Done Sent false
20977 6473 Acked Read 4 Done Done Sent false
20978 6474 Acked Flush 0 Done Done Sent false
20979 6475 Acked Read 8 Done Done Sent false
20980 6476 Acked Flush 0 Done Done Sent false
20981 6477 Acked Write 10 Done Done Sent false
20982 6478 Acked Write 2 Done Done Sent false
20983 6479 Acked Write 10 Done Done Sent false
20984 6480 Acked Read 7 Done Done Sent false
20985 6481 Acked Read 5 Done Done Sent false
20986 6482 Acked Write 6 Done Done Sent false
20987 6483 Acked Write 2 Done Done Sent false
20988 6484 Acked Write 7 Done Done Sent false
20989 6485 Acked Flush 0 Done Done Sent false
20990 6486 Acked Write 2 Done Done Sent false
20991 6487 Acked Write 4 Done Done Sent false
20992 6488 Acked Read 2 Done Done Sent false
20993 6489 Acked Write 7 Done Done Sent false
20994 6490 Acked Read 8 Done Done Sent false
20995 6491 Acked Read 1 Done Done Sent false
20996 6492 Acked Write 2 Done Done Sent false
20997 6493 Acked Write 9 Done Done Sent false
20998 6494 Acked Read 8 Done Done Sent false
20999 6495 Acked Write 5 Done Done Sent false
21000 6496 Acked Write 6 Done Done Sent false
21001 6497 Acked Read 7 Done Done Sent false
21002 6498 Acked Flush 0 Done Done Sent false
21003 6499 Acked Write 2 Done Done Sent false
21004 6500 Acked Write 9 Done Done Sent false
21005 6501 Acked Read 7 Done Done Sent false
21006 6502 Acked Write 4 Done Done Sent false
21007 6503 Acked Read 9 Done Done Sent false
21008 6504 Acked Write 1 Done Done Sent false
21009 6505 Acked Write 8 Done Done Sent false
21010 6506 Acked Read 9 Done Done Sent false
21011 6507 Acked Read 1 Done Done Sent false
21012 6508 Acked Write 4 Done Done Sent false
21013 6509 Acked Read 6 Done Done Sent false
21014 6510 Acked Flush 0 Done Done Sent false
21015 6511 Acked Read 1 Done Done Sent false
21016 6512 Acked Read 2 Done Done Sent false
21017 6513 Acked Write 1 Done Done Sent false
21018 6514 Acked Write 8 Done Done Sent false
21019 6515 Acked Read 9 Done Done Sent false
21020 6516 Acked Read 5 Done Done Sent false
21021 6517 Acked Write 8 Done Done Sent false
21022 6518 Acked Read 6 Done Done Sent false
21023 6519 Acked Flush 0 Done Done Sent false
21024 6520 Acked Write 10 Done Done Sent false
21025 6521 Acked Write 2 Done Done Sent false
21026 6522 Acked Read 8 Done Done Sent false
21027 6523 Acked Read 4 Done Done Sent false
21028 6524 Acked Write 9 Done Done Sent false
21029 6525 Acked Read 2 Done Done Sent false
21030 6526 Acked Read 8 Done Done Sent false
21031 6527 Acked Flush 0 Done Done Sent false
21032 6528 Acked Flush 0 Done Done Sent false
21033 6529 Acked Read 7 Done Done Sent false
21034 6530 Acked Read 2 Done Done Sent false
21035 6531 Acked Write 8 Done Done Sent false
21036 6532 Acked Write 3 Done Done Sent false
21037 6533 Acked Read 4 Done Done Sent false
21038 6534 Acked Read 4 Done Done Sent false
21039 6535 Acked Read 1 Done Done Sent false
21040 6536 Acked Write 3 Done Done Sent false
21041 6537 Acked Write 5 Done Done Sent false
21042 6538 Acked Read 8 Done Done Sent false
21043 6539 Acked Read 10 Done Done Sent false
21044 6540 Acked Read 6 Done Done Sent false
21045 6541 Acked Flush 0 Done Done Sent false
21046 6542 Acked Read 1 Done Done Sent false
21047 6543 Acked Write 2 Done Done Sent false
21048 6544 Acked Write 8 Done Done Sent false
21049 6545 Acked Read 4 Done Done Sent false
21050 6546 Acked Read 5 Done Done Sent false
21051 6547 Acked Read 7 Done Done Sent false
21052 6548 Acked Write 7 Done Done Sent false
21053 6549 Acked Read 10 Done Done Sent false
21054 6550 Acked Write 3 Done Done Sent false
21055 6551 Acked Read 8 Done Done Sent false
21056 6552 Acked Flush 0 Done Done Sent false
21057 6553 Acked Read 10 Done Done Sent false
21058 6554 Acked Write 9 Done Done Sent false
21059 6555 Acked Write 6 Done Done Sent false
21060 6556 Acked Read 9 Done Done Sent false
21061 6557 Acked Read 8 Done Done Sent false
21062 6558 Acked Read 5 Done Done Sent false
21063 6559 Acked Read 3 Done Done Sent false
21064 6560 Acked Read 6 Done Done Sent false
21065 6561 Acked Flush 0 Done Done Sent false
21066 6562 Acked Read 4 Done Done Sent false
21067 6563 Acked Write 6 Done Done Sent false
21068 6564 Acked Read 3 Done Done Sent false
21069 6565 Acked Flush 0 Done Done Sent false
21070 6566 Acked Read 7 Done Done Sent false
21071 6567 Acked Write 6 Done Done Sent false
21072 6568 Acked Write 5 Done Done Sent false
21073 6569 Acked Read 2 Done Done Sent false
21074 6570 Acked Read 6 Done Done Sent false
21075 6571 Acked Write 1 Done Done Sent false
21076 6572 Acked Read 8 Done Done Sent false
21077 6573 Acked Flush 0 Done Done Sent false
21078 6574 Acked Read 9 Done Done Sent false
21079 6575 Acked Read 8 Done Done Sent false
21080 6576 Acked Read 2 Done Done Sent false
21081 6577 Acked Write 7 Done Done Sent false
21082 6578 Acked Flush 0 Done Done Sent false
21083 6579 Acked Flush 0 Done Done Sent false
21084 6580 Acked Read 10 Done Done Sent false
21085 6581 Acked Write 7 Done Done Sent false
21086 6582 Acked Read 5 Done Done Sent false
21087 6583 Acked Write 4 Done Done Sent false
21088 6584 Acked Write 4 Done Done Sent false
21089 6585 Acked Read 5 Done Done Sent false
21090 6586 Acked Write 9 Done Done Sent false
21091 6587 Acked Write 5 Done Done Sent false
21092 6588 Acked Write 8 Done Done Sent false
21093 6589 Acked Flush 0 Done Done Sent false
21094 6590 Acked Write 1 Done Done Sent false
21095 6591 Acked Write 4 Done Done Sent false
21096 6592 Acked Read 2 Done Done Sent false
21097 6593 Acked Flush 0 Done Done Sent false
21098 6594 Acked Flush 0 Done Done Sent false
21099 6595 Acked Read 2 Done Done Sent false
21100 6596 Acked Read 1 Done Done Sent false
21101 6597 Acked Flush 0 Done Done Sent false
21102 6598 Acked Flush 0 Done Done Sent false
21103 6599 Acked Flush 0 Done Done Sent false
21104 6600 Acked Write 7 Done Done Sent false
21105 6601 Acked Read 1 Done Done Sent false
21106 6602 Acked Write 1 Done Done Sent false
21107 6603 Acked Write 5 Done Done Sent false
21108 6604 Acked Read 9 Done Done Sent false
21109 6605 Acked Write 1 Done Done Sent false
21110 6606 Acked Read 7 Done Done Sent false
21111 6607 Acked Read 8 Done Done Sent false
21112 6608 Acked Read 10 Done Done Sent false
21113 6609 Acked Read 10 Done Done Sent false
21114 6610 Acked Read 3 Done Done Sent false
21115 6611 Acked Read 5 Done Done Sent false
21116 6612 Acked Write 8 Done Done Sent false
21117 6613 Acked Write 2 Done Done Sent false
21118 6614 Acked Write 4 Done Done Sent false
21119 6615 Acked Write 5 Done Done Sent false
21120 6616 Acked Read 4 Done Done Sent false
21121 6617 Acked Flush 0 Done Done Sent false
21122 6618 Acked Write 9 Done Done Sent false
21123 6619 Acked Read 9 Done Done Sent false
21124 6620 Acked Write 4 Done Done Sent false
21125 6621 Acked Write 3 Done Done Sent false
21126 6622 Acked Flush 0 Done Done Sent false
21127 6623 Acked Read 5 Done Done Sent false
21128 6624 Acked Write 7 Done Done Sent false
21129 6625 Acked Read 7 Done Done Sent false
21130 6626 Acked Read 10 Done Done Sent false
21131 6627 Acked Write 4 Done Done Sent false
21132 6628 Acked Read 9 Done Done Sent false
21133 6629 Acked Flush 0 Done Done Sent false
21134 6630 Acked Read 1 Done Done Sent false
21135 6631 Acked Write 10 Done Done Sent false
21136 6632 Acked Flush 0 Done Done Sent false
21137 6633 Acked Read 9 Done Done Sent false
21138 6634 Acked Read 2 Done Done Sent false
21139 6635 Acked Read 8 Done Done Sent false
21140 6636 Acked Write 3 Done Done Sent false
21141 6637 Acked Flush 0 Done Done Sent false
21142 6638 Acked Read 10 Done Done Sent false
21143 6639 Acked Read 6 Done Done Sent false
21144 6640 Acked Read 9 Done Done Sent false
21145 6641 Acked Read 9 Done Done Sent false
21146 6642 Acked Write 3 Done Done Sent false
21147 6643 Acked Write 8 Done Done Sent false
21148 6644 Acked Read 2 Done Done Sent false
21149 6645 Acked Write 4 Done Done Sent false
21150 6646 Acked Write 3 Done Done Sent false
21151 6647 Acked Read 7 Done Done Sent false
21152 6648 Acked Write 1 Done Done Sent false
21153 6649 Acked Write 6 Done Done Sent false
21154 6650 Acked Read 3 Done Done Sent false
21155 6651 Acked Write 7 Done Done Sent false
21156 6652 Acked Read 5 Done Done Sent false
21157 6653 Acked Read 5 Done Done Sent false
21158 6654 Acked Read 1 Done Done Sent false
21159 6655 Acked Write 3 Done Done Sent false
21160 6656 Acked Read 10 Done Done Sent false
21161 6657 Acked Write 7 Done Done Sent false
21162 6658 Acked Write 10 Done Done Sent false
21163 6659 Acked Write 8 Done Done Sent false
21164 6660 Acked Read 5 Done Done Sent false
21165 6661 Acked Read 4 Done Done Sent false
21166 6662 Acked Write 2 Done Done Sent false
21167 6663 Acked Read 2 Done Done Sent false
21168 6664 Acked Flush 0 Done Done Sent false
21169 6665 Acked Write 3 Done Done Sent false
21170 6666 Acked Read 8 Done Done Sent false
21171 6667 Acked Write 10 Done Done Sent false
21172 6668 Acked Read 3 Done Done Sent false
21173 6669 Acked Read 1 Done Done Sent false
21174 6670 Acked Write 6 Done Done Sent false
21175 6671 Acked Write 2 Done Done Sent false
21176 6672 Acked Write 8 Done Done Sent false
21177 6673 Acked Read 2 Done Done Sent false
21178 6674 Acked Read 3 Done Done Sent false
21179 6675 Acked Read 7 Done Done Sent false
21180 6676 Acked Write 10 Done Done Sent false
21181 6677 Acked Read 6 Done Done Sent false
21182 6678 Acked Write 5 Done Done Sent false
21183 6679 Acked Read 7 Done Done Sent false
21184 6680 Acked Write 9 Done Done Sent false
21185 6681 Acked Read 7 Done Done Sent false
21186 6682 Acked Write 3 Done Done Sent false
21187 6683 Acked Read 6 Done Done Sent false
21188 6684 Acked Write 8 Done Done Sent false
21189 6685 Acked Read 7 Done Done Sent false
21190 6686 Acked Write 6 Done Done Sent false
21191 6687 Acked Write 2 Done Done Sent false
21192 6688 Acked Write 8 Done Done Sent false
21193 6689 Acked Read 7 Done Done Sent false
21194 6690 Acked Write 8 Done Done Sent false
21195 6691 Acked Read 4 Done Done Sent false
21196 6692 Acked Flush 0 Done Done Sent false
21197 6693 Acked Write 4 Done Done Sent false
21198 6694 Acked Read 9 Done Done Sent false
21199 6695 Acked Read 2 Done Done Sent false
21200 6696 Acked Write 5 Done Done Sent false
21201 6697 Acked Write 6 Done Done Sent false
21202 6698 Acked Flush 0 Done Done Sent false
21203 6699 Acked Read 4 Done Done Sent false
21204 6700 Acked Write 10 Done Done Sent false
21205 6701 Acked Flush 0 Done Done Sent false
21206 6702 Acked Write 6 Done Done Sent false
21207 6703 Acked Write 3 Done Done Sent false
21208 6704 Acked Write 9 Done Done Sent false
21209 6705 Acked Write 2 Done Done Sent false
21210 6706 Acked Write 8 Done Done Sent false
21211 6707 Acked Read 1 Done Done Sent false
21212 6708 Acked Read 7 Done Done Sent false
21213 6709 Acked Read 7 Done Done Sent false
21214 6710 Acked Write 10 Done Done Sent false
21215 6711 Acked Write 3 Done Done Sent false
21216 6712 Acked Write 2 Done Done Sent false
21217 6713 Acked Read 7 Done Done Sent false
21218 6714 Acked Write 4 Done Done Sent false
21219 6715 Acked Write 2 Done Done Sent false
21220 6716 Acked Read 2 Done Done Sent false
21221 6717 Acked Read 1 Done Done Sent false
21222 6718 Acked Read 3 Done Done Sent false
21223 6719 Acked Read 2 Done Done Sent false
21224 6720 Acked Read 6 Done Done Sent false
21225 6721 Acked Read 7 Done Done Sent false
21226 6722 Acked Write 1 Done Done Sent false
21227 6723 Acked Write 9 Done Done Sent false
21228 6724 Acked Read 8 Done Done Sent false
21229 6725 Acked Write 2 Done Done Sent false
21230 6726 Acked Read 9 Done Done Sent false
21231 6727 Acked Write 10 Done Done Sent false
21232 6728 Acked Write 2 Done Done Sent false
21233 6729 Acked Write 3 Done Done Sent false
21234 6730 Acked Flush 0 Done Done Sent false
21235 6731 Acked Write 10 Done Done Sent false
21236 6732 Acked Read 8 Done Done Sent false
21237 6733 Acked Write 7 Done Done Sent false
21238 6734 Acked Write 2 Done Done Sent false
21239 6735 Acked Read 2 Done Done Sent false
21240 STATES DS:0 DS:1 DS:2 TOTAL
21241 Sent 0 0 301 301
21242 Done 301 301 0 602
21243 Skipped 0 0 0 0
21244 Error 0 0 0 0
21245 Last Flush: 6730 6730 6434
21246 Downstairs last five completed: 6434 6433 6432 6431 6430
21247 Upstairs last five completed: 6735 6734 6733 6732 6731
21248 CLIENT: Up:0 ds:301 act:2
21249 [8830] got start action
21250 [8830] I am going to restart
21251 Starting downstairs at port 8830
21252 Make output file at "/tmp/dsc/downstairs-8830.txt"
21253 [8830][2] reports Starting
21254 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1222)
21255 [8830][2] reports Running
21256 ----------------------------------------------------------------
21257 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
21258 States: Active Active Offline
21259 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
21260 6435 Acked Read 7 Done Done Sent false
21261 6436 Acked Write 7 Done Done Sent false
21262 6437 Acked Write 4 Done Done Sent false
21263 6438 Acked Write 2 Done Done Sent false
21264 6439 Acked Read 10 Done Done Sent false
21265 6440 Acked Read 2 Done Done Sent false
21266 6441 Acked Read 4 Done Done Sent false
21267 6442 Acked Read 9 Done Done Sent false
21268 6443 Acked Read 8 Done Done Sent false
21269 6444 Acked Read 4 Done Done Sent false
21270 6445 Acked Write 2 Done Done Sent false
21271 6446 Acked Read 3 Done Done Sent false
21272 6447 Acked Read 3 Done Done Sent false
21273 6448 Acked Read 8 Done Done Sent false
21274 6449 Acked Write 2 Done Done Sent false
21275 6450 Acked Read 6 Done Done Sent false
21276 6451 Acked Write 7 Done Done Sent false
21277 6452 Acked Write 4 Done Done Sent false
21278 6453 Acked Read 10 Done Done Sent false
21279 6454 Acked Flush 0 Done Done Sent false
21280 6455 Acked Write 7 Done Done Sent false
21281 6456 Acked Read 6 Done Done Sent false
21282 6457 Acked Write 9 Done Done Sent false
21283 6458 Acked Write 9 Done Done Sent false
21284 6459 Acked Read 7 Done Done Sent false
21285 6460 Acked Read 8 Done Done Sent false
21286 6461 Acked Write 9 Done Done Sent false
21287 6462 Acked Write 1 Done Done Sent false
21288 6463 Acked Read 3 Done Done Sent false
21289 6464 Acked Read 9 Done Done Sent false
21290 6465 Acked Write 4 Done Done Sent false
21291 6466 Acked Read 7 Done Done Sent false
21292 6467 Acked Read 10 Done Done Sent false
21293 6468 Acked Write 10 Done Done Sent false
21294 6469 Acked Write 4 Done Done Sent false
21295 6470 Acked Read 8 Done Done Sent false
21296 6471 Acked Write 10 Done Done Sent false
21297 6472 Acked Read 9 Done Done Sent false
21298 6473 Acked Read 4 Done Done Sent false
21299 6474 Acked Flush 0 Done Done Sent false
21300 6475 Acked Read 8 Done Done Sent false
21301 6476 Acked Flush 0 Done Done Sent false
21302 6477 Acked Write 10 Done Done Sent false
21303 6478 Acked Write 2 Done Done Sent false
21304 6479 Acked Write 10 Done Done Sent false
21305 6480 Acked Read 7 Done Done Sent false
21306 6481 Acked Read 5 Done Done Sent false
21307 6482 Acked Write 6 Done Done Sent false
21308 6483 Acked Write 2 Done Done Sent false
21309 6484 Acked Write 7 Done Done Sent false
21310 6485 Acked Flush 0 Done Done Sent false
21311 6486 Acked Write 2 Done Done Sent false
21312 6487 Acked Write 4 Done Done Sent false
21313 6488 Acked Read 2 Done Done Sent false
21314 6489 Acked Write 7 Done Done Sent false
21315 6490 Acked Read 8 Done Done Sent false
21316 6491 Acked Read 1 Done Done Sent false
21317 6492 Acked Write 2 Done Done Sent false
21318 6493 Acked Write 9 Done Done Sent false
21319 6494 Acked Read 8 Done Done Sent false
21320 6495 Acked Write 5 Done Done Sent false
21321 6496 Acked Write 6 Done Done Sent false
21322 6497 Acked Read 7 Done Done Sent false
21323 6498 Acked Flush 0 Done Done Sent false
21324 6499 Acked Write 2 Done Done Sent false
21325 6500 Acked Write 9 Done Done Sent false
21326 6501 Acked Read 7 Done Done Sent false
21327 6502 Acked Write 4 Done Done Sent false
21328 6503 Acked Read 9 Done Done Sent false
21329 6504 Acked Write 1 Done Done Sent false
21330 6505 Acked Write 8 Done Done Sent false
21331 6506 Acked Read 9 Done Done Sent false
21332 6507 Acked Read 1 Done Done Sent false
21333 6508 Acked Write 4 Done Done Sent false
21334 6509 Acked Read 6 Done Done Sent false
21335 6510 Acked Flush 0 Done Done Sent false
21336 6511 Acked Read 1 Done Done Sent false
21337 6512 Acked Read 2 Done Done Sent false
21338 6513 Acked Write 1 Done Done Sent false
21339 6514 Acked Write 8 Done Done Sent false
21340 6515 Acked Read 9 Done Done Sent false
21341 6516 Acked Read 5 Done Done Sent false
21342 6517 Acked Write 8 Done Done Sent false
21343 6518 Acked Read 6 Done Done Sent false
21344 6519 Acked Flush 0 Done Done Sent false
21345 6520 Acked Write 10 Done Done Sent false
21346 6521 Acked Write 2 Done Done Sent false
21347 6522 Acked Read 8 Done Done Sent false
21348 6523 Acked Read 4 Done Done Sent false
21349 6524 Acked Write 9 Done Done Sent false
21350 6525 Acked Read 2 Done Done Sent false
21351 6526 Acked Read 8 Done Done Sent false
21352 6527 Acked Flush 0 Done Done Sent false
21353 6528 Acked Flush 0 Done Done Sent false
21354 6529 Acked Read 7 Done Done Sent false
21355 6530 Acked Read 2 Done Done Sent false
21356 6531 Acked Write 8 Done Done Sent false
21357 6532 Acked Write 3 Done Done Sent false
21358 6533 Acked Read 4 Done Done Sent false
21359 6534 Acked Read 4 Done Done Sent false
21360 6535 Acked Read 1 Done Done Sent false
21361 6536 Acked Write 3 Done Done Sent false
21362 6537 Acked Write 5 Done Done Sent false
21363 6538 Acked Read 8 Done Done Sent false
21364 6539 Acked Read 10 Done Done Sent false
21365 6540 Acked Read 6 Done Done Sent false
21366 6541 Acked Flush 0 Done Done Sent false
21367 6542 Acked Read 1 Done Done Sent false
21368 6543 Acked Write 2 Done Done Sent false
21369 6544 Acked Write 8 Done Done Sent false
21370 6545 Acked Read 4 Done Done Sent false
21371 6546 Acked Read 5 Done Done Sent false
21372 6547 Acked Read 7 Done Done Sent false
21373 6548 Acked Write 7 Done Done Sent false
21374 6549 Acked Read 10 Done Done Sent false
21375 6550 Acked Write 3 Done Done Sent false
21376 6551 Acked Read 8 Done Done Sent false
21377 6552 Acked Flush 0 Done Done Sent false
21378 6553 Acked Read 10 Done Done Sent false
21379 6554 Acked Write 9 Done Done Sent false
21380 6555 Acked Write 6 Done Done Sent false
21381 6556 Acked Read 9 Done Done Sent false
21382 6557 Acked Read 8 Done Done Sent false
21383 6558 Acked Read 5 Done Done Sent false
21384 6559 Acked Read 3 Done Done Sent false
21385 6560 Acked Read 6 Done Done Sent false
21386 6561 Acked Flush 0 Done Done Sent false
21387 6562 Acked Read 4 Done Done Sent false
21388 6563 Acked Write 6 Done Done Sent false
21389 6564 Acked Read 3 Done Done Sent false
21390 6565 Acked Flush 0 Done Done Sent false
21391 6566 Acked Read 7 Done Done Sent false
21392 6567 Acked Write 6 Done Done Sent false
21393 6568 Acked Write 5 Done Done Sent false
21394 6569 Acked Read 2 Done Done Sent false
21395 6570 Acked Read 6 Done Done Sent false
21396 6571 Acked Write 1 Done Done Sent false
21397 6572 Acked Read 8 Done Done Sent false
21398 6573 Acked Flush 0 Done Done Sent false
21399 6574 Acked Read 9 Done Done Sent false
21400 6575 Acked Read 8 Done Done Sent false
21401 6576 Acked Read 2 Done Done Sent false
21402 6577 Acked Write 7 Done Done Sent false
21403 6578 Acked Flush 0 Done Done Sent false
21404 6579 Acked Flush 0 Done Done Sent false
21405 6580 Acked Read 10 Done Done Sent false
21406 6581 Acked Write 7 Done Done Sent false
21407 6582 Acked Read 5 Done Done Sent false
21408 6583 Acked Write 4 Done Done Sent false
21409 6584 Acked Write 4 Done Done Sent false
21410 6585 Acked Read 5 Done Done Sent false
21411 6586 Acked Write 9 Done Done Sent false
21412 6587 Acked Write 5 Done Done Sent false
21413 6588 Acked Write 8 Done Done Sent false
21414 6589 Acked Flush 0 Done Done Sent false
21415 6590 Acked Write 1 Done Done Sent false
21416 6591 Acked Write 4 Done Done Sent false
21417 6592 Acked Read 2 Done Done Sent false
21418 6593 Acked Flush 0 Done Done Sent false
21419 6594 Acked Flush 0 Done Done Sent false
21420 6595 Acked Read 2 Done Done Sent false
21421 6596 Acked Read 1 Done Done Sent false
21422 6597 Acked Flush 0 Done Done Sent false
21423 6598 Acked Flush 0 Done Done Sent false
21424 6599 Acked Flush 0 Done Done Sent false
21425 6600 Acked Write 7 Done Done Sent false
21426 6601 Acked Read 1 Done Done Sent false
21427 6602 Acked Write 1 Done Done Sent false
21428 6603 Acked Write 5 Done Done Sent false
21429 6604 Acked Read 9 Done Done Sent false
21430 6605 Acked Write 1 Done Done Sent false
21431 6606 Acked Read 7 Done Done Sent false
21432 6607 Acked Read 8 Done Done Sent false
21433 6608 Acked Read 10 Done Done Sent false
21434 6609 Acked Read 10 Done Done Sent false
21435 6610 Acked Read 3 Done Done Sent false
21436 6611 Acked Read 5 Done Done Sent false
21437 6612 Acked Write 8 Done Done Sent false
21438 6613 Acked Write 2 Done Done Sent false
21439 6614 Acked Write 4 Done Done Sent false
21440 6615 Acked Write 5 Done Done Sent false
21441 6616 Acked Read 4 Done Done Sent false
21442 6617 Acked Flush 0 Done Done Sent false
21443 6618 Acked Write 9 Done Done Sent false
21444 6619 Acked Read 9 Done Done Sent false
21445 6620 Acked Write 4 Done Done Sent false
21446 6621 Acked Write 3 Done Done Sent false
21447 6622 Acked Flush 0 Done Done Sent false
21448 6623 Acked Read 5 Done Done Sent false
21449 6624 Acked Write 7 Done Done Sent false
21450 6625 Acked Read 7 Done Done Sent false
21451 6626 Acked Read 10 Done Done Sent false
21452 6627 Acked Write 4 Done Done Sent false
21453 6628 Acked Read 9 Done Done Sent false
21454 6629 Acked Flush 0 Done Done Sent false
21455 6630 Acked Read 1 Done Done Sent false
21456 6631 Acked Write 10 Done Done Sent false
21457 6632 Acked Flush 0 Done Done Sent false
21458 6633 Acked Read 9 Done Done Sent false
21459 6634 Acked Read 2 Done Done Sent false
21460 6635 Acked Read 8 Done Done Sent false
21461 6636 Acked Write 3 Done Done Sent false
21462 6637 Acked Flush 0 Done Done Sent false
21463 6638 Acked Read 10 Done Done Sent false
21464 6639 Acked Read 6 Done Done Sent false
21465 6640 Acked Read 9 Done Done Sent false
21466 6641 Acked Read 9 Done Done Sent false
21467 6642 Acked Write 3 Done Done Sent false
21468 6643 Acked Write 8 Done Done Sent false
21469 6644 Acked Read 2 Done Done Sent false
21470 6645 Acked Write 4 Done Done Sent false
21471 6646 Acked Write 3 Done Done Sent false
21472 6647 Acked Read 7 Done Done Sent false
21473 6648 Acked Write 1 Done Done Sent false
21474 6649 Acked Write 6 Done Done Sent false
21475 6650 Acked Read 3 Done Done Sent false
21476 6651 Acked Write 7 Done Done Sent false
21477 6652 Acked Read 5 Done Done Sent false
21478 6653 Acked Read 5 Done Done Sent false
21479 6654 Acked Read 1 Done Done Sent false
21480 6655 Acked Write 3 Done Done Sent false
21481 6656 Acked Read 10 Done Done Sent false
21482 6657 Acked Write 7 Done Done Sent false
21483 6658 Acked Write 10 Done Done Sent false
21484 6659 Acked Write 8 Done Done Sent false
21485 6660 Acked Read 5 Done Done Sent false
21486 6661 Acked Read 4 Done Done Sent false
21487 6662 Acked Write 2 Done Done Sent false
21488 6663 Acked Read 2 Done Done Sent false
21489 6664 Acked Flush 0 Done Done Sent false
21490 6665 Acked Write 3 Done Done Sent false
21491 6666 Acked Read 8 Done Done Sent false
21492 6667 Acked Write 10 Done Done Sent false
21493 6668 Acked Read 3 Done Done Sent false
21494 6669 Acked Read 1 Done Done Sent false
21495 6670 Acked Write 6 Done Done Sent false
21496 6671 Acked Write 2 Done Done Sent false
21497 6672 Acked Write 8 Done Done Sent false
21498 6673 Acked Read 2 Done Done Sent false
21499 6674 Acked Read 3 Done Done Sent false
21500 6675 Acked Read 7 Done Done Sent false
21501 6676 Acked Write 10 Done Done Sent false
21502 6677 Acked Read 6 Done Done Sent false
21503 6678 Acked Write 5 Done Done Sent false
21504 6679 Acked Read 7 Done Done Sent false
21505 6680 Acked Write 9 Done Done Sent false
21506 6681 Acked Read 7 Done Done Sent false
21507 6682 Acked Write 3 Done Done Sent false
21508 6683 Acked Read 6 Done Done Sent false
21509 6684 Acked Write 8 Done Done Sent false
21510 6685 Acked Read 7 Done Done Sent false
21511 6686 Acked Write 6 Done Done Sent false
21512 6687 Acked Write 2 Done Done Sent false
21513 6688 Acked Write 8 Done Done Sent false
21514 6689 Acked Read 7 Done Done Sent false
21515 6690 Acked Write 8 Done Done Sent false
21516 6691 Acked Read 4 Done Done Sent false
21517 6692 Acked Flush 0 Done Done Sent false
21518 6693 Acked Write 4 Done Done Sent false
21519 6694 Acked Read 9 Done Done Sent false
21520 6695 Acked Read 2 Done Done Sent false
21521 6696 Acked Write 5 Done Done Sent false
21522 6697 Acked Write 6 Done Done Sent false
21523 6698 Acked Flush 0 Done Done Sent false
21524 6699 Acked Read 4 Done Done Sent false
21525 6700 Acked Write 10 Done Done Sent false
21526 6701 Acked Flush 0 Done Done Sent false
21527 6702 Acked Write 6 Done Done Sent false
21528 6703 Acked Write 3 Done Done Sent false
21529 6704 Acked Write 9 Done Done Sent false
21530 6705 Acked Write 2 Done Done Sent false
21531 6706 Acked Write 8 Done Done Sent false
21532 6707 Acked Read 1 Done Done Sent false
21533 6708 Acked Read 7 Done Done Sent false
21534 6709 Acked Read 7 Done Done Sent false
21535 6710 Acked Write 10 Done Done Sent false
21536 6711 Acked Write 3 Done Done Sent false
21537 6712 Acked Write 2 Done Done Sent false
21538 6713 Acked Read 7 Done Done Sent false
21539 6714 Acked Write 4 Done Done Sent false
21540 6715 Acked Write 2 Done Done Sent false
21541 6716 Acked Read 2 Done Done Sent false
21542 6717 Acked Read 1 Done Done Sent false
21543 6718 Acked Read 3 Done Done Sent false
21544 6719 Acked Read 2 Done Done Sent false
21545 6720 Acked Read 6 Done Done Sent false
21546 6721 Acked Read 7 Done Done Sent false
21547 6722 Acked Write 1 Done Done Sent false
21548 6723 Acked Write 9 Done Done Sent false
21549 6724 Acked Read 8 Done Done Sent false
21550 6725 Acked Write 2 Done Done Sent false
21551 6726 Acked Read 9 Done Done Sent false
21552 6727 Acked Write 10 Done Done Sent false
21553 6728 Acked Write 2 Done Done Sent false
21554 6729 Acked Write 3 Done Done Sent false
21555 6730 Acked Flush 0 Done Done Sent false
21556 6731 Acked Write 10 Done Done Sent false
21557 6732 Acked Read 8 Done Done Sent false
21558 6733 Acked Write 7 Done Done Sent false
21559 6734 Acked Write 2 Done Done Sent false
21560 6735 Acked Read 2 Done Done Sent false
21561 6736 Acked Flush 0 Done Done Sent false
21562 STATES DS:0 DS:1 DS:2 TOTAL
21563 Sent 0 0 302 302
21564 Done 302 302 0 604
21565 Skipped 0 0 0 0
21566 Error 0 0 0 0
21567 Last Flush: 6736 6736 6434
21568 Downstairs last five completed: 6434 6433 6432 6431 6430
21569 Upstairs last five completed: 6736 6735 6734 6733 6732
21570 CLIENT: Up:0 ds:302 act:2
215712024-11-26T22:09:02.830ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
215722024-11-26T22:09:02.830ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:57693) = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
215732024-11-26T22:09:02.831ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
215742024-11-26T22:09:02.832ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
215752024-11-26T22:09:02.832ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
215762024-11-26T22:09:02.832ZINFOcrucible: send last flush ID to this DS: 6434 = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
215772024-11-26T22:09:02.832ZINFOcrucible: Replied this last flush ID: 6434 = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
215782024-11-26T22:09:02.832ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
215792024-11-26T22:09:02.832ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
215802024-11-26T22:09:02.832ZINFOcrucible: [2] client re-new 302 jobs since flush 6434 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
215812024-11-26T22:09:02.832ZINFOcrucible: [2] Replayed 302 jobs since flush: 6434 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
21582 ----------------------------------------------------------------
21583 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
21584 Downstairs last five completed: 6736 6735 6734 6733 6732
21585 Upstairs last five completed: 6736 6735 6734 6733 6732
21586 CLIENT: Up:0 ds:0 act:3
21587 Replay: All jobs finished
21588 Main task has work to do, go find it
21589 got dsc Stop(1)
21590 stop 1
21591 [8820] Got stop action so:false kr:false
21592 Nov 26 22:09:05.069 INFO request completed, latency_us: 328, response_code: 204, uri: /stop/cid/1, method: POST, req_id: f7be9081-dc5d-4cd3-acde-505c114c231b, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
215932024-11-26T22:09:05.069ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
215942024-11-26T22:09:05.069ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
215952024-11-26T22:09:05.069ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
215962024-11-26T22:09:05.069ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
215972024-11-26T22:09:05.069ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
21598 Nov 26 22:09:05.070 INFO request completed, latency_us:{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2024-11-26T22:09:05.070003625Z","hostname":"w-01JDN7ZQ17ABQG15KHKN3F2D26","pid":1203,"":"io task","client":"1","":"downstairs","session_id":"86a36c17-3718-484c-87fd-2b8a1dc2ab18"}
21599 156, response_code: 200, uri: /state/cid/1, method: GET, req_id: 218d1697-27a5-4eaf-86a1-5e8c9f2f0e59, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
21600 [8820] Exited with: ExitStatus(unix_wait_status(9))
21601 [8820][1] reports Exit
21602 Nov 26 22:09:09.071 INFO request completed, latency_us: 223, response_code: 200, uri: /state/cid/1, method: GET, req_id: 516cca18-93c3-4bfa-8d47-1b8eb4928cfd, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
21603 001/300 Read block 2302 len 16384
21604 002/300 Read block 2116 len 20480
21605 003/300 Read block 3289 len 16384
21606 004/300 Read block 1844 len 36864
21607 005/300 Read block 90 len 8192
21608 006/300 Read block 1089 len 12288
21609 007/300 Write block 4475 len 16384 data: 8 7 7 6
21610 008/300 Flush
21611 009/300 Write block 4095 len 28672 data: 3 3 4 3 3 3 4
21612 010/300 Read block 4497 len 40960
21613 011/300 Write block 4892 len 4096 data: 4
21614 012/300 Flush
21615 013/300 Read block 597 len 28672
21616 014/300 Write block 3253 len 32768 data: 3 3 3 4 4 5 6 5
21617 015/300 Flush
21618 016/300 Write block 3330 len 40960 data: 3 4 2 2 2 2 2 2 2 3
21619 017/300 Read block 3718 len 16384
21620 018/300 Read block 704 len 36864
21621 019/300 Write block 3495 len 16384 data: 8 8 6 5
21622 020/300 Read block 1917 len 28672
21623 021/300 Write block 3699 len 28672 data: 6 6 6 5 4 4 3
21624 022/300 Write block 1546 len 40960 data: 3 4 2 2 3 3 7 8 6 7
21625 023/300 Read block 4553 len 28672
21626 024/300 Write block 2242 len 28672 data: 3 3 3 2 5 6 6
21627 025/300 Write block 3411 len 24576 data: 2 3 3 3 4 4
21628 026/300 Read block 1904 len 32768
21629 027/300 Read block 1053 len 24576
21630 028/300 Write block 3556 len 20480 data: 4 4 3 3 4
21631 029/300 Read block 189 len 36864
21632 030/300 Write block 892 len 12288 data: 5 5 4
21633 031/300 Read block 3114 len 36864
21634 032/300 Write block 670 len 32768 data: 5 5 5 5 5 5 4 4
21635 033/300 Read block 1589 len 20480
21636 034/300 Read block 2320 len 28672
21637 035/300 Write block 1413 len 12288 data: 3 4 3
21638 036/300 Read block 932 len 20480
21639 037/300 Read block 3298 len 12288
21640 038/300 Write block 4639 len 36864 data: 5 5 6 5 4 5 3 3 3
21641 039/300 Read block 3115 len 36864
21642 040/300 Write block 3430 len 28672 data: 3 3 2 2 2 3 3
21643 041/300 Read block 2145 len 8192
21644 042/300 Read block 1669 len 40960
21645 043/300 Write block 3445 len 24576 data: 3 4 5 5 5 5
21646 044/300 Read block 1032 len 8192
21647 045/300 Write block 474 len 8192 data: 4 4
21648 046/300 Write block 4344 len 32768 data: 10 9 7 8 8 6 4 4
21649 047/300 Write block 3439 len 28672 data: 3 3 3 3 3 3 4
21650 048/300 Write block 1355 len 40960 data: 6 6 5 6 5 3 3 4 4 6
21651 049/300 Write block 1641 len 12288 data: 3 4 4
21652 050/300 Read block 1831 len 40960
21653 051/300 Read block 1562 len 36864
21654 052/300 Write block 2914 len 16384 data: 5 7 5 5
21655 053/300 Read block 4527 len 36864
21656 054/300 Write block 2823 len 4096 data: 4
21657 055/300 Read block 683 len 12288
21658 056/300 Write block 3085 len 4096 data: 6
21659 057/300 Read block 1775 len 28672
21660 058/300 Read block 1867 len 36864
21661 059/300 Write block 1056 len 16384 data: 6 6 6 6
21662 060/300 Read block 4181 len 12288
21663 061/300 Write block 3144 len 24576 data: 5 6 5 5 4 3
21664 062/300 Read block 1043 len 4096
21665 063/300 Write block 3501 len 40960 data: 5 6 5 4 5 5 5 5 5 5
21666 064/300 Read block 4939 len 4096
21667 065/300 Write block 1828 len 28672 data: 6 6 6 6 6 7 5
21668 066/300 Read block 1945 len 24576
21669 067/300 Write block 1700 len 8192 data: 3 3
21670 068/300 Read block 1572 len 12288
21671 069/300 Write block 582 len 12288 data: 3 3 2
21672 070/300 Flush
21673 071/300 Read block 2507 len 12288
21674 072/300 Flush
21675 073/300 Write block 2352 len 4096 data: 7
21676 074/300 Read block 905 len 16384
21677 075/300 Write block 1926 len 36864 data: 5 6 5 4 4 4 5 5 6
21678 076/300 Write block 3158 len 20480 data: 4 4 8 7 6
21679 077/300 Write block 2500 len 4096 data: 6
21680 078/300 Read block 540 len 28672
21681 079/300 Read block 75 len 8192
21682 080/300 Read block 1171 len 16384
21683 081/300 Read block 2140 len 8192
21684 082/300 Read block 3848 len 24576
21685 083/300 Read block 2891 len 40960
21686 084/300 Write block 3877 len 16384 data: 4 4 4 3
21687 085/300 Read block 4508 len 28672
21688 086/300 Write block 219 len 4096 data: 8
21689 087/300 Write block 3813 len 16384 data: 3 3 3 3
21690 088/300 Read block 740 len 12288
21691 089/300 Write block 1050 len 40960 data: 5 6 6 6 7 7 7 7 7 7
21692 090/300 Read block 3889 len 16384
21693 091/300 Write block 1695 len 20480 data: 3 4 4 4 4
21694 092/300 Write block 948 len 8192 data: 5 4
21695 093/300 Write block 554 len 24576 data: 4 4 4 5 4 3
21696 094/300 Write block 751 len 20480 data: 4 4 3 3 4
21697 095/300 Read block 108 len 32768
21698 096/300 Read block 583 len 36864
21699 097/300 Read block 3580 len 8192
21700 098/300 Read block 2199 len 32768
21701 099/300 Flush
21702 100/300 Read block 133 len 28672
21703 101/300 Write block 3764 len 16384 data: 5 4 5 4
21704 102/300 Flush
21705 103/300 Read block 1735 len 20480
21706 104/300 Write block 3243 len 12288 data: 5 6 5
21707 105/300 Read block 4477 len 4096
21708 106/300 Read block 4733 len 28672
21709 107/300 Write block 98 len 28672 data: 6 6 6 6 6 5 6
21710 108/300 Read block 4646 len 28672
21711 109/300 Write block 3498 len 12288 data: 6 5 5
21712 110/300 Write block 4757 len 28672 data: 3 3 3 3 3 3 5
21713 111/300 Read block 1868 len 8192
21714 112/300 Read block 2424 len 20480
21715 113/300 Flush
21716 114/300 Write block 1711 len 24576 data: 4 4 3 4 6 8
21717 115/300 Write block 2045 len 36864 data: 4 4 4 5 4 5 5 5 3
21718 116/300 Write block 970 len 20480 data: 4 3 3 3 3
21719 117/300 Read block 1545 len 20480
21720 118/300 Read block 682 len 32768
21721 119/300 Write block 4061 len 28672 data: 6 7 7 7 6 6 6
21722 120/300 Write block 657 len 28672 data: 5 4 5 6 5 4 4
21723 121/300 Read block 1086 len 20480
21724 122/300 Write block 3131 len 40960 data: 2 2 3 4 5 5 6 5 5 6
21725 123/300 Write block 1442 len 20480 data: 2 2 3 3 4
21726 124/300 Read block 738 len 16384
21727 125/300 Read block 2811 len 28672
21728 126/300 Read block 1711 len 32768
21729 127/300 Read block 2075 len 24576
21730 128/300 Write block 3201 len 12288 data: 2 2 2
21731 129/300 Write block 771 len 36864 data: 6 5 5 3 3 3 3 3 3
21732 130/300 Write block 1926 len 24576 data: 6 7 6 5 5 5
21733 131/300 Write block 4916 len 16384 data: 3 3 3 3
21734 132/300 Write block 1468 len 8192 data: 3 2
21735 133/300 Write block 47 len 8192 data: 7 7
21736 134/300 Read block 2555 len 32768
21737 135/300 Read block 1812 len 4096
21738 136/300 Write block 285 len 40960 data: 3 4 6 7 9 8 8 9 8 7
21739 137/300 Write block 3800 len 8192 data: 3 4
21740 138/300 Write block 3764 len 16384 data: 6 5 6 5
21741 139/300 Write block 342 len 12288 data: 6 6 7
21742 140/300 Flush
21743 141/300 Write block 1661 len 4096 data: 5
21744 142/300 Write block 3082 len 4096 data: 8
21745 143/300 Write block 280 len 16384 data: 3 4 4 4
21746 144/300 Write block 4560 len 8192 data: 4 4
21747 145/300 Read block 995 len 12288
21748 146/300 Read block 2430 len 20480
21749 147/300 Read block 2799 len 8192
21750 148/300 Write block 2451 len 24576 data: 5 5 3 5 7 4
21751 149/300 Read block 4674 len 8192
21752 150/300 Write block 1216 len 36864 data: 5 3 3 2 2 2 2 2 2
21753 151/300 Flush
21754 152/300 Read block 2549 len 36864
21755 153/300 Write block 2928 len 24576 data: 4 4 5 4 5 5
21756 154/300 Write block 3699 len 20480 data: 7 7 7 6 5
21757 155/300 Read block 2100 len 24576
21758 156/300 Write block 2768 len 20480 data: 3 3 3 5 7
21759 157/300 Write block 1111 len 16384 data: 4 4 3 3
21760 158/300 Read block 4268 len 12288
21761 159/300 Read block 2778 len 4096
21762 160/300 Flush
21763 161/300 Write block 4598 len 32768 data: 7 6 6 6 7 7 7 7
21764 162/300 Read block 3536 len 24576
21765 163/300 Write block 3100 len 40960 data: 3 3 3 3 3 4 4 5 6 6
21766 164/300 Read block 1101 len 20480
21767 165/300 Flush
21768 166/300 Flush
21769 167/300 Write block 1227 len 36864 data: 2 3 3 3 6 6 7 8 8
21770 168/300 Read block 2082 len 8192
21771 169/300 Flush
21772 170/300 Write block 4025 len 32768 data: 8 8 7 8 7 6 4 4
21773 171/300 Read block 4267 len 36864
21774 172/300 Read block 480 len 20480
21775 173/300 Write block 4140 len 20480 data: 2 3 5 6 8
21776 174/300 Write block 3438 len 12288 data: 4 4 4
21777 175/300 Write block 4931 len 36864 data: 3 5 5 4 3 3 4 4 4
21778 176/300 Read block 3314 len 16384
21779 177/300 Flush
21780 178/300 Write block 3224 len 4096 data: 6
21781 179/300 Write block 524 len 20480 data: 6 6 4 4 4
21782 180/300 Write block 1044 len 12288 data: 3 4 4
21783 181/300 Write block 3253 len 36864 data: 4 4 4 5 5 6 7 6 5
21784 182/300 Read block 4112 len 16384
21785 183/300 Read block 652 len 40960
21786 184/300 Write block 1272 len 32768 data: 4 5 6 6 6 7 7 6
21787 185/300 Write block 1656 len 20480 data: 5 5 6 5 6
21788 186/300 Write block 972 len 8192 data: 4 4
21789 187/300 Write block 142 len 40960 data: 4 5 6 4 2 3 3 3 3 3
21790 188/300 Read block 1853 len 12288
21791 189/300 Write block 3667 len 4096 data: 8
21792 190/300 Read block 1032 len 4096
21793 191/300 Read block 912 len 36864
21794 192/300 Write block 2723 len 40960 data: 3 2 2 2 4 5 5 6 7 6
21795 193/300 Write block 1646 len 40960 data: 4 4 4 6 3 3 4 5 6 5
21796 194/300 Write block 4636 len 16384 data: 5 5 5 6
21797 195/300 Write block 4874 len 28672 data: 5 6 6 6 6 5 5
21798 196/300 Read block 4626 len 16384
21799 197/300 Flush
21800 198/300 Read block 1145 len 16384
21801 199/300 Read block 3876 len 28672
21802 200/300 Read block 2507 len 8192
21803 201/300 Read block 84 len 12288
21804 202/300 Flush
21805 203/300 Write block 2901 len 36864 data: 5 5 5 5 5 5 5 5 3
21806 204/300 Flush
21807 205/300 Flush
21808 206/300 Write block 2677 len 28672 data: 3 3 2 2 2 2 2
21809 207/300 Write block 289 len 8192 data: 10 9
21810 208/300 Read block 4320 len 40960
21811 209/300 Write block 3987 len 8192 data: 4 5
21812 210/300 Read block 1273 len 40960
21813 211/300 Write block 1275 len 4096 data: 7
21814 212/300 Write block 1166 len 36864 data: 3 3 5 4 4 4 3 4 5
21815 213/300 Write block 3704 len 40960 data: 5 4 3 3 3 4 4 3 3 5
21816 214/300 Write block 4029 len 4096 data: 8
21817 215/300 Read block 1485 len 4096
21818 216/300 Read block 1688 len 4096
21819 217/300 Write block 3537 len 12288 data: 4 3 4
21820 218/300 Write block 1592 len 8192 data: 5 5
21821 219/300 Read block 2382 len 20480
21822 220/300 Write block 3314 len 24576 data: 7 8 7 7 7 7
21823 221/300 Write block 1551 len 8192 data: 4 8
21824 222/300 Write block 4974 len 32768 data: 4 4 4 5 4 4 5 7
21825 223/300 Read block 2970 len 40960
21826 224/300 Read block 2825 len 24576
21827 225/300 Write block 1018 len 24576 data: 3 3 4 5 5 5
21828 226/300 Write block 1798 len 8192 data: 3 4
21829 227/300 Write block 3166 len 12288 data: 6 6 6
21830 228/300 Write block 1006 len 20480 data: 5 5 4 3 3
21831 229/300 Read block 151 len 12288
21832 230/300 Read block 2858 len 16384
21833 231/300 Write block 2803 len 20480 data: 3 3 2 3 3
21834 232/300 Read block 1729 len 16384
21835 233/300 Read block 771 len 12288
21836 234/300 Read block 2358 len 24576
21837 235/300 Read block 4284 len 20480
21838 236/300 Write block 1835 len 16384 data: 5 5 4 4
21839 237/300 Read block 835 len 4096
21840 238/300 Flush
21841 239/300 Write block 136 len 40960 data: 3 3 4 5 6 6 5 6 7 5
21842 240/300 Read block 3824 len 16384
21843 241/300 Write block 2614 len 4096 data: 5
21844 242/300 Flush
21845 243/300 Read block 2638 len 16384
21846 244/300 Read block 4317 len 28672
21847 245/300 Read block 3959 len 20480
21848 246/300 Read block 923 len 40960
21849 247/300 Write block 3839 len 12288 data: 3 3 3
21850 248/300 Write block 2479 len 16384 data: 4 7 6 6
21851 249/300 Read block 4499 len 4096
21852 250/300 Write block 4386 len 40960 data: 5 5 5 4 3 2 3 4 4 3
21853 251/300 Write block 241 len 16384 data: 6 5 5 4
21854 252/300 Write block 4199 len 20480 data: 7 6 5 5 5
21855 253/300 Read block 2967 len 20480
21856 254/300 Read block 1929 len 8192
21857 255/300 Read block 1701 len 32768
21858 256/300 Read block 3864 len 24576
21859 257/300 Write block 3035 len 12288 data: 3 3 4
21860 258/300 Read block 2530 len 32768
21861 259/300 Read block 451 len 16384
21862 260/300 Read block 1633 len 4096
21863 261/300 Write block 1843 len 24576 data: 4 5 5 6 8 8
21864 262/300 Write block 1226 len 40960 data: 2 3 4 4 4 7 7 8 9 9
21865 263/300 Write block 1019 len 12288 data: 4 5 6
21866 264/300 Read block 4425 len 20480
21867 265/300 Read block 1943 len 20480
21868 266/300 Write block 4440 len 40960 data: 4 5 6 7 7 5 5 5 6 7
21869 267/300 Read block 4784 len 20480
21870 268/300 Write block 4278 len 24576 data: 5 5 6 5 4 4
21871 269/300 Read block 2808 len 20480
21872 270/300 Read block 2923 len 36864
21873 271/300 Flush
21874 272/300 Write block 3981 len 4096 data: 4
21875 273/300 Read block 3737 len 16384
21876 274/300 Read block 2338 len 36864
21877 275/300 Write block 3595 len 24576 data: 7 8 9 9 9 8
21878 276/300 Read block 3340 len 20480
21879 277/300 Read block 4633 len 4096
21880 278/300 Write block 743 len 32768 data: 6 5 5 4 5 5 4 3
21881 279/300 Read block 590 len 8192
21882 280/300 Read block 4133 len 36864
21883 281/300 Flush
21884 282/300 Read block 2012 len 40960
21885 283/300 Read block 1331 len 28672
21886 284/300 Write block 561 len 4096 data: 4
21887 285/300 Read block 979 len 32768
21888 286/300 Read block 616 len 28672
21889 287/300 Read block 3501 len 36864
21890 288/300 Write block 1998 len 20480 data: 5 6 5 6 4
21891 289/300 Read block 222 len 40960
21892 290/300 Write block 1809 len 8192 data: 6 5
21893 291/300 Read block 439 len 32768
21894 292/300 Flush
21895 293/300 Write block 4767 len 20480 data: 5 5 5 6 6
21896 294/300 Write block 2801 len 40960 data: 3 4 4 4 3 4 4 5 6 5
21897 295/300 Read block 4368 len 4096
21898 296/300 Read block 4260 len 8192
21899 297/300 Read block 4742 len 32768
21900 298/300 Write block 4843 len 32768 data: 4 4 4 4 4 4 3 3
21901 299/300 Read block 3444 len 24576
21902 300/300 Write block 2438 len 24576 data: 5 4 3 4 6 8
21903 301/300 Read block 3911 len 16384
21904 Main task has work to do, go find it
21905 got dsc Start(1)
21906 start 1
21907 Nov 26 22:09:09.297 INFO request completed, latency_us: 193, response_code: 204, uri: /start/cid/1, method: POST, req_id: 925c92d5-0b23-45fc-a928-baf2949f76d1, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
21908 Replay: started 1, returned:Ok(())
21909 ----------------------------------------------------------------
21910 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
21911 States: Active Offline Active
21912 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
21913 6737 Acked Read 4 Done Sent Done false
21914 6738 Acked Read 5 Done Sent Done false
21915 6739 Acked Read 4 Done Sent Done false
21916 6740 Acked Read 9 Done Sent Done false
21917 6741 Acked Read 2 Done Sent Done false
21918 6742 Acked Read 3 Done Sent Done false
21919 6743 Acked Write 4 Done Sent Done false
21920 6744 Acked Flush 0 Done Sent Done false
21921 6745 Acked Write 7 Done Sent Done false
21922 6746 Acked Read 10 Done Sent Done false
21923 6747 Acked Write 1 Done Sent Done false
21924 6748 Acked Flush 0 Done Sent Done false
21925 6749 Acked Read 7 Done Sent Done false
21926 6750 Acked Write 8 Done Sent Done false
21927 6751 Acked Flush 0 Done Sent Done false
21928 6752 Acked Write 10 Done Sent Done false
21929 6753 Acked Read 4 Done Sent Done false
21930 6754 Acked Read 9 Done Sent Done false
21931 6755 Acked Write 4 Done Sent Done false
21932 6756 Acked Read 7 Done Sent Done false
21933 6757 Acked Write 7 Done Sent Done false
21934 6758 Acked Write 10 Done Sent Done false
21935 6759 Acked Read 7 Done Sent Done false
21936 6760 Acked Write 7 Done Sent Done false
21937 6761 Acked Write 6 Done Sent Done false
21938 6762 Acked Read 8 Done Sent Done false
21939 6763 Acked Read 6 Done Sent Done false
21940 6764 Acked Write 5 Done Sent Done false
21941 6765 Acked Read 9 Done Sent Done false
21942 6766 Acked Write 3 Done Sent Done false
21943 6767 Acked Read 9 Done Sent Done false
21944 6768 Acked Write 8 Done Sent Done false
21945 6769 Acked Read 5 Done Sent Done false
21946 6770 Acked Read 7 Done Sent Done false
21947 6771 Acked Write 3 Done Sent Done false
21948 6772 Acked Read 5 Done Sent Done false
21949 6773 Acked Read 3 Done Sent Done false
21950 6774 Acked Write 9 Done Sent Done false
21951 6775 Acked Read 9 Done Sent Done false
21952 6776 Acked Write 7 Done Sent Done false
21953 6777 Acked Read 2 Done Sent Done false
21954 6778 Acked Read 10 Done Sent Done false
21955 6779 Acked Write 6 Done Sent Done false
21956 6780 Acked Read 2 Done Sent Done false
21957 6781 Acked Write 2 Done Sent Done false
21958 6782 Acked Write 8 Done Sent Done false
21959 6783 Acked Write 7 Done Sent Done false
21960 6784 Acked Write 10 Done Sent Done false
21961 6785 Acked Write 3 Done Sent Done false
21962 6786 Acked Read 10 Done Sent Done false
21963 6787 Acked Read 9 Done Sent Done false
21964 6788 Acked Write 4 Done Sent Done false
21965 6789 Acked Read 9 Done Sent Done false
21966 6790 Acked Write 1 Done Sent Done false
21967 6791 Acked Read 3 Done Sent Done false
21968 6792 Acked Write 1 Done Sent Done false
21969 6793 Acked Read 7 Done Sent Done false
21970 6794 Acked Read 9 Done Sent Done false
21971 6795 Acked Write 4 Done Sent Done false
21972 6796 Acked Read 3 Done Sent Done false
21973 6797 Acked Write 6 Done Sent Done false
21974 6798 Acked Read 1 Done Sent Done false
21975 6799 Acked Write 10 Done Sent Done false
21976 6800 Acked Read 1 Done Sent Done false
21977 6801 Acked Write 7 Done Sent Done false
21978 6802 Acked Read 6 Done Sent Done false
21979 6803 Acked Write 2 Done Sent Done false
21980 6804 Acked Read 3 Done Sent Done false
21981 6805 Acked Write 3 Done Sent Done false
21982 6806 Acked Flush 0 Done Sent Done false
21983 6807 Acked Read 3 Done Sent Done false
21984 6808 Acked Flush 0 Done Sent Done false
21985 6809 Acked Write 1 Done Sent Done false
21986 6810 Acked Read 4 Done Sent Done false
21987 6811 Acked Write 9 Done Sent Done false
21988 6812 Acked Write 5 Done Sent Done false
21989 6813 Acked Write 1 Done Sent Done false
21990 6814 Acked Read 7 Done Sent Done false
21991 6815 Acked Read 2 Done Sent Done false
21992 6816 Acked Read 4 Done Sent Done false
21993 6817 Acked Read 2 Done Sent Done false
21994 6818 Acked Read 6 Done Sent Done false
21995 6819 Acked Read 10 Done Sent Done false
21996 6820 Acked Write 4 Done Sent Done false
21997 6821 Acked Read 7 Done Sent Done false
21998 6822 Acked Write 1 Done Sent Done false
21999 6823 Acked Write 4 Done Sent Done false
22000 6824 Acked Read 3 Done Sent Done false
22001 6825 Acked Write 10 Done Sent Done false
22002 6826 Acked Read 4 Done Sent Done false
22003 6827 Acked Write 5 Done Sent Done false
22004 6828 Acked Write 2 Done Sent Done false
22005 6829 Acked Write 6 Done Sent Done false
22006 6830 Acked Write 5 Done Sent Done false
22007 6831 Acked Read 8 Done Sent Done false
22008 6832 Acked Read 9 Done Sent Done false
22009 6833 Acked Read 2 Done Sent Done false
22010 6834 Acked Read 8 Done Sent Done false
22011 6835 Acked Flush 0 Done Sent Done false
22012 6836 Acked Read 7 Done Sent Done false
22013 6837 Acked Write 4 Done Sent Done false
22014 6838 Acked Flush 0 Done Sent Done false
22015 6839 Acked Read 5 Done Sent Done false
22016 6840 Acked Write 3 Done Sent Done false
22017 6841 Acked Read 1 Done Sent Done false
22018 6842 Acked Read 7 Done Sent Done false
22019 6843 Acked Write 7 Done Sent Done false
22020 6844 Acked Read 7 Done Sent Done false
22021 6845 Acked Write 3 Done Sent Done false
22022 6846 Acked Write 7 Done Sent Done false
22023 6847 Acked Read 2 Done Sent Done false
22024 6848 Acked Read 5 Done Sent Done false
22025 6849 Acked Flush 0 Done Sent Done false
22026 6850 Acked Write 6 Done Sent Done false
22027 6851 Acked Write 9 Done Sent Done false
22028 6852 Acked Write 5 Done Sent Done false
22029 6853 Acked Read 5 Done Sent Done false
22030 6854 Acked Read 8 Done Sent Done false
22031 6855 Acked Write 7 Done Sent Done false
22032 6856 Acked Write 7 Done Sent Done false
22033 6857 Acked Read 5 Done Sent Done false
22034 6858 Acked Write 10 Done Sent Done false
22035 6859 Acked Write 5 Done Sent Done false
22036 6860 Acked Read 4 Done Sent Done false
22037 6861 Acked Read 7 Done Sent Done false
22038 6862 Acked Read 8 Done Sent Done false
22039 6863 Acked Read 6 Done Sent Done false
22040 6864 Acked Write 3 Done Sent Done false
22041 6865 Acked Write 9 Done Sent Done false
22042 6866 Acked Write 6 Done Sent Done false
22043 6867 Acked Write 4 Done Sent Done false
22044 6868 Acked Write 2 Done Sent Done false
22045 6869 Acked Write 2 Done Sent Done false
22046 6870 Acked Read 8 Done Sent Done false
22047 6871 Acked Read 1 Done Sent Done false
22048 6872 Acked Write 10 Done Sent Done false
22049 6873 Acked Write 2 Done Sent Done false
22050 6874 Acked Write 4 Done Sent Done false
22051 6875 Acked Write 3 Done Sent Done false
22052 6876 Acked Flush 0 Done Sent Done false
22053 6877 Acked Write 1 Done Sent Done false
22054 6878 Acked Write 1 Done Sent Done false
22055 6879 Acked Write 4 Done Sent Done false
22056 6880 Acked Write 2 Done Sent Done false
22057 6881 Acked Read 3 Done Sent Done false
22058 6882 Acked Read 5 Done Sent Done false
22059 6883 Acked Read 2 Done Sent Done false
22060 6884 Acked Write 6 Done Sent Done false
22061 6885 Acked Read 2 Done Sent Done false
22062 6886 Acked Write 9 Done Sent Done false
22063 6887 Acked Flush 0 Done Sent Done false
22064 6888 Acked Read 9 Done Sent Done false
22065 6889 Acked Write 6 Done Sent Done false
22066 6890 Acked Write 5 Done Sent Done false
22067 6891 Acked Read 6 Done Sent Done false
22068 6892 Acked Write 5 Done Sent Done false
22069 6893 Acked Write 4 Done Sent Done false
22070 6894 Acked Read 3 Done Sent Done false
22071 6895 Acked Read 1 Done Sent Done false
22072 6896 Acked Flush 0 Done Sent Done false
22073 6897 Acked Write 8 Done Sent Done false
22074 6898 Acked Read 6 Done Sent Done false
22075 6899 Acked Write 10 Done Sent Done false
22076 6900 Acked Read 5 Done Sent Done false
22077 6901 Acked Flush 0 Done Sent Done false
22078 6902 Acked Flush 0 Done Sent Done false
22079 6903 Acked Write 9 Done Sent Done false
22080 6904 Acked Read 2 Done Sent Done false
22081 6905 Acked Flush 0 Done Sent Done false
22082 6906 Acked Write 8 Done Sent Done false
22083 6907 Acked Read 9 Done Sent Done false
22084 6908 Acked Read 5 Done Sent Done false
22085 6909 Acked Write 5 Done Sent Done false
22086 6910 Acked Write 3 Done Sent Done false
22087 6911 Acked Write 9 Done Sent Done false
22088 6912 Acked Read 4 Done Sent Done false
22089 6913 Acked Flush 0 Done Sent Done false
22090 6914 Acked Write 1 Done Sent Done false
22091 6915 Acked Write 5 Done Sent Done false
22092 6916 Acked Write 3 Done Sent Done false
22093 6917 Acked Write 9 Done Sent Done false
22094 6918 Acked Read 4 Done Sent Done false
22095 6919 Acked Read 10 Done Sent Done false
22096 6920 Acked Write 8 Done Sent Done false
22097 6921 Acked Write 5 Done Sent Done false
22098 6922 Acked Write 2 Done Sent Done false
22099 6923 Acked Write 10 Done Sent Done false
22100 6924 Acked Read 3 Done Sent Done false
22101 6925 Acked Write 1 Done Sent Done false
22102 6926 Acked Read 1 Done Sent Done false
22103 6927 Acked Read 9 Done Sent Done false
22104 6928 Acked Write 10 Done Sent Done false
22105 6929 Acked Write 10 Done Sent Done false
22106 6930 Acked Write 4 Done Sent Done false
22107 6931 Acked Write 7 Done Sent Done false
22108 6932 Acked Read 4 Done Sent Done false
22109 6933 Acked Flush 0 Done Sent Done false
22110 6934 Acked Read 4 Done Sent Done false
22111 6935 Acked Read 7 Done Sent Done false
22112 6936 Acked Read 2 Done Sent Done false
22113 6937 Acked Read 3 Done Sent Done false
22114 6938 Acked Flush 0 Done Sent Done false
22115 6939 Acked Write 9 Done Sent Done false
22116 6940 Acked Flush 0 Done Sent Done false
22117 6941 Acked Flush 0 Done Sent Done false
22118 6942 Acked Write 7 Done Sent Done false
22119 6943 Acked Write 2 Done Sent Done false
22120 6944 Acked Read 10 Done Sent Done false
22121 6945 Acked Write 2 Done Sent Done false
22122 6946 Acked Read 10 Done Sent Done false
22123 6947 Acked Write 1 Done Sent Done false
22124 6948 Acked Write 9 Done Sent Done false
22125 6949 Acked Write 10 Done Sent Done false
22126 6950 Acked Write 1 Done Sent Done false
22127 6951 Acked Read 1 Done Sent Done false
22128 6952 Acked Read 1 Done Sent Done false
22129 6953 Acked Write 3 Done Sent Done false
22130 6954 Acked Write 2 Done Sent Done false
22131 6955 Acked Read 5 Done Sent Done false
22132 6956 Acked Write 6 Done Sent Done false
22133 6957 Acked Write 2 Done Sent Done false
22134 6958 Acked Write 8 Done Sent Done false
22135 6959 Acked Read 10 Done Sent Done false
22136 6960 Acked Read 6 Done Sent Done false
22137 6961 Acked Write 6 Done Sent Done false
22138 6962 Acked Write 2 Done Sent Done false
22139 6963 Acked Write 3 Done Sent Done false
22140 6964 Acked Write 5 Done Sent Done false
22141 6965 Acked Read 3 Done Sent Done false
22142 6966 Acked Read 4 Done Sent Done false
22143 6967 Acked Write 5 Done Sent Done false
22144 6968 Acked Read 4 Done Sent Done false
22145 6969 Acked Read 3 Done Sent Done false
22146 6970 Acked Read 6 Done Sent Done false
22147 6971 Acked Read 5 Done Sent Done false
22148 6972 Acked Write 4 Done Sent Done false
22149 6973 Acked Read 1 Done Sent Done false
22150 6974 Acked Flush 0 Done Sent Done false
22151 6975 Acked Write 10 Done Sent Done false
22152 6976 Acked Read 4 Done Sent Done false
22153 6977 Acked Write 1 Done Sent Done false
22154 6978 Acked Flush 0 Done Sent Done false
22155 6979 Acked Read 4 Done Sent Done false
22156 6980 Acked Read 7 Done Sent Done false
22157 6981 Acked Read 5 Done Sent Done false
22158 6982 Acked Read 10 Done Sent Done false
22159 6983 Acked Write 3 Done Sent Done false
22160 6984 Acked Write 4 Done Sent Done false
22161 6985 Acked Read 1 Done Sent Done false
22162 6986 Acked Write 10 Done Sent Done false
22163 6987 Acked Write 4 Done Sent Done false
22164 6988 Acked Write 5 Done Sent Done false
22165 6989 Acked Read 5 Done Sent Done false
22166 6990 Acked Read 2 Done Sent Done false
22167 6991 Acked Read 8 Done Sent Done false
22168 6992 Acked Read 6 Done Sent Done false
22169 6993 Acked Write 3 Done Sent Done false
22170 6994 Acked Read 8 Done Sent Done false
22171 6995 Acked Read 4 Done Sent Done false
22172 6996 Acked Read 1 Done Sent Done false
22173 6997 Acked Write 6 Done Sent Done false
22174 6998 Acked Write 10 Done Sent Done false
22175 6999 Acked Write 3 Done Sent Done false
22176 7000 Acked Read 5 Done Sent Done false
22177 7001 Acked Read 5 Done Sent Done false
22178 7002 Acked Write 10 Done Sent Done false
22179 7003 Acked Read 5 Done Sent Done false
22180 7004 Acked Write 6 Done Sent Done false
22181 7005 Acked Read 5 Done Sent Done false
22182 7006 Acked Read 9 Done Sent Done false
22183 7007 Acked Flush 0 Done Sent Done false
22184 7008 Acked Write 1 Done Sent Done false
22185 7009 Acked Read 4 Done Sent Done false
22186 7010 Acked Read 9 Done Sent Done false
22187 7011 Acked Write 6 Done Sent Done false
22188 7012 Acked Read 5 Done Sent Done false
22189 7013 Acked Read 1 Done Sent Done false
22190 7014 Acked Write 8 Done Sent Done false
22191 7015 Acked Read 2 Done Sent Done false
22192 7016 Acked Read 9 Done Sent Done false
22193 7017 Acked Flush 0 Done Sent Done false
22194 7018 Acked Read 10 Done Sent Done false
22195 7019 Acked Read 7 Done Sent Done false
22196 7020 Acked Write 1 Done Sent Done false
22197 7021 Acked Read 8 Done Sent Done false
22198 7022 Acked Read 7 Done Sent Done false
22199 7023 Acked Read 9 Done Sent Done false
22200 7024 Acked Write 5 Done Sent Done false
22201 7025 Acked Read 10 Done Sent Done false
22202 7026 Acked Write 2 Done Sent Done false
22203 7027 Acked Read 8 Done Sent Done false
22204 7028 Acked Flush 0 Done Sent Done false
22205 7029 Acked Write 5 Done Sent Done false
22206 7030 Acked Write 10 Done Sent Done false
22207 7031 Acked Read 1 Done Sent Done false
22208 7032 Acked Read 2 Done Sent Done false
22209 7033 Acked Read 8 Done Sent Done false
22210 7034 Acked Write 8 Done Sent Done false
22211 7035 Acked Read 6 Done Sent Done false
22212 7036 Acked Write 6 Done Sent Done false
22213 7037 Acked Read 4 Done Sent Done false
22214 STATES DS:0 DS:1 DS:2 TOTAL
22215 Sent 0 301 0 301
22216 Done 301 0 301 602
22217 Skipped 0 0 0 0
22218 Error 0 0 0 0
22219 Last Flush: 7028 6736 7028
22220 Downstairs last five completed: 6736 6735 6734 6733 6732
22221 Upstairs last five completed: 7037 7036 7035 7034 7033
22222 CLIENT: Up:0 ds:301 act:2
22223 ----------------------------------------------------------------
22224 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
22225 States: Active Offline Active
22226 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
22227 6737 Acked Read 4 Done Sent Done false
22228 6738 Acked Read 5 Done Sent Done false
22229 6739 Acked Read 4 Done Sent Done false
22230 6740 Acked Read 9 Done Sent Done false
22231 6741 Acked Read 2 Done Sent Done false
22232 6742 Acked Read 3 Done Sent Done false
22233 6743 Acked Write 4 Done Sent Done false
22234 6744 Acked Flush 0 Done Sent Done false
22235 6745 Acked Write 7 Done Sent Done false
22236 6746 Acked Read 10 Done Sent Done false
22237 6747 Acked Write 1 Done Sent Done false
22238 6748 Acked Flush 0 Done Sent Done false
22239 6749 Acked Read 7 Done Sent Done false
22240 6750 Acked Write 8 Done Sent Done false
22241 6751 Acked Flush 0 Done Sent Done false
22242 6752 Acked Write 10 Done Sent Done false
22243 6753 Acked Read 4 Done Sent Done false
22244 6754 Acked Read 9 Done Sent Done false
22245 6755 Acked Write 4 Done Sent Done false
22246 6756 Acked Read 7 Done Sent Done false
22247 6757 Acked Write 7 Done Sent Done false
22248 6758 Acked Write 10 Done Sent Done false
22249 6759 Acked Read 7 Done Sent Done false
22250 6760 Acked Write 7 Done Sent Done false
22251 6761 Acked Write 6 Done Sent Done false
22252 6762 Acked Read 8 Done Sent Done false
22253 6763 Acked Read 6 Done Sent Done false
22254 6764 Acked Write 5 Done Sent Done false
22255 6765 Acked Read 9 Done Sent Done false
22256 6766 Acked Write 3 Done Sent Done false
22257 6767 Acked Read 9 Done Sent Done false
22258 6768 Acked Write 8 Done Sent Done false
22259 6769 Acked Read 5 Done Sent Done false
22260 6770 Acked Read 7 Done Sent Done false
22261 6771 Acked Write 3 Done Sent Done false
22262 6772 Acked Read 5 Done Sent Done false
22263 6773 Acked Read 3 Done Sent Done false
22264 6774 Acked Write 9 Done Sent Done false
22265 6775 Acked Read 9 Done Sent Done false
22266 6776 Acked Write 7 Done Sent Done false
22267 6777 Acked Read 2 Done Sent Done false
22268 6778 Acked Read 10 Done Sent Done false
22269 6779 Acked Write 6 Done Sent Done false
22270 6780 Acked Read 2 Done Sent Done false
22271 6781 Acked Write 2 Done Sent Done false
22272 6782 Acked Write 8 Done Sent Done false
22273 6783 Acked Write 7 Done Sent Done false
22274 6784 Acked Write 10 Done Sent Done false
22275 6785 Acked Write 3 Done Sent Done false
22276 6786 Acked Read 10 Done Sent Done false
22277 6787 Acked Read 9 Done Sent Done false
22278 6788 Acked Write 4 Done Sent Done false
22279 6789 Acked Read 9 Done Sent Done false
22280 6790 Acked Write 1 Done Sent Done false
22281 6791 Acked Read 3 Done Sent Done false
22282 6792 Acked Write 1 Done Sent Done false
22283 6793 Acked Read 7 Done Sent Done false
22284 6794 Acked Read 9 Done Sent Done false
22285 6795 Acked Write 4 Done Sent Done false
22286 6796 Acked Read 3 Done Sent Done false
22287 6797 Acked Write 6 Done Sent Done false
22288 6798 Acked Read 1 Done Sent Done false
22289 6799 Acked Write 10 Done Sent Done false
22290 6800 Acked Read 1 Done Sent Done false
22291 6801 Acked Write 7 Done Sent Done false
22292 6802 Acked Read 6 Done Sent Done false
22293 6803 Acked Write 2 Done Sent Done false
22294 6804 Acked Read 3 Done Sent Done false
22295 6805 Acked Write 3 Done Sent Done false
22296 6806 Acked Flush 0 Done Sent Done false
22297 6807 Acked Read 3 Done Sent Done false
22298 6808 Acked Flush 0 Done Sent Done false
22299 6809 Acked Write 1 Done Sent Done false
22300 6810 Acked Read 4 Done Sent Done false
22301 6811 Acked Write 9 Done Sent Done false
22302 6812 Acked Write 5 Done Sent Done false
22303 6813 Acked Write 1 Done Sent Done false
22304 6814 Acked Read 7 Done Sent Done false
22305 6815 Acked Read 2 Done Sent Done false
22306 6816 Acked Read 4 Done Sent Done false
22307 6817 Acked Read 2 Done Sent Done false
22308 6818 Acked Read 6 Done Sent Done false
22309 6819 Acked Read 10 Done Sent Done false
22310 6820 Acked Write 4 Done Sent Done false
22311 6821 Acked Read 7 Done Sent Done false
22312 6822 Acked Write 1 Done Sent Done false
22313 6823 Acked Write 4 Done Sent Done false
22314 6824 Acked Read 3 Done Sent Done false
22315 6825 Acked Write 10 Done Sent Done false
22316 6826 Acked Read 4 Done Sent Done false
22317 6827 Acked Write 5 Done Sent Done false
22318 6828 Acked Write 2 Done Sent Done false
22319 6829 Acked Write 6 Done Sent Done false
22320 6830 Acked Write 5 Done Sent Done false
22321 6831 Acked Read 8 Done Sent Done false
22322 6832 Acked Read 9 Done Sent Done false
22323 6833 Acked Read 2 Done Sent Done false
22324 6834 Acked Read 8 Done Sent Done false
22325 6835 Acked Flush 0 Done Sent Done false
22326 6836 Acked Read 7 Done Sent Done false
22327 6837 Acked Write 4 Done Sent Done false
22328 6838 Acked Flush 0 Done Sent Done false
22329 6839 Acked Read 5 Done Sent Done false
22330 6840 Acked Write 3 Done Sent Done false
22331 6841 Acked Read 1 Done Sent Done false
22332 6842 Acked Read 7 Done Sent Done false
22333 6843 Acked Write 7 Done Sent Done false
22334 6844 Acked Read 7 Done Sent Done false
22335 6845 Acked Write 3 Done Sent Done false
22336 6846 Acked Write 7 Done Sent Done false
22337 6847 Acked Read 2 Done Sent Done false
22338 6848 Acked Read 5 Done Sent Done false
22339 6849 Acked Flush 0 Done Sent Done false
22340 6850 Acked Write 6 Done Sent Done false
22341 6851 Acked Write 9 Done Sent Done false
22342 6852 Acked Write 5 Done Sent Done false
22343 6853 Acked Read 5 Done Sent Done false
22344 6854 Acked Read 8 Done Sent Done false
22345 6855 Acked Write 7 Done Sent Done false
22346 6856 Acked Write 7 Done Sent Done false
22347 6857 Acked Read 5 Done Sent Done false
22348 6858 Acked Write 10 Done Sent Done false
22349 6859 Acked Write 5 Done Sent Done false
22350 6860 Acked Read 4 Done Sent Done false
22351 6861 Acked Read 7 Done Sent Done false
22352 6862 Acked Read 8 Done Sent Done false
22353 6863 Acked Read 6 Done Sent Done false
22354 6864 Acked Write 3 Done Sent Done false
22355 6865 Acked Write 9 Done Sent Done false
22356 6866 Acked Write 6 Done Sent Done false
22357 6867 Acked Write 4 Done Sent Done false
22358 6868 Acked Write 2 Done Sent Done false
22359 6869 Acked Write 2 Done Sent Done false
22360 6870 Acked Read 8 Done Sent Done false
22361 6871 Acked Read 1 Done Sent Done false
22362 6872 Acked Write 10 Done Sent Done false
22363 6873 Acked Write 2 Done Sent Done false
22364 6874 Acked Write 4 Done Sent Done false
22365 6875 Acked Write 3 Done Sent Done false
22366 6876 Acked Flush 0 Done Sent Done false
22367 6877 Acked Write 1 Done Sent Done false
22368 6878 Acked Write 1 Done Sent Done false
22369 6879 Acked Write 4 Done Sent Done false
22370 6880 Acked Write 2 Done Sent Done false
22371 6881 Acked Read 3 Done Sent Done false
22372 6882 Acked Read 5 Done Sent Done false
22373 6883 Acked Read 2 Done Sent Done false
22374 6884 Acked Write 6 Done Sent Done false
22375 6885 Acked Read 2 Done Sent Done false
22376 6886 Acked Write 9 Done Sent Done false
22377 6887 Acked Flush 0 Done Sent Done false
22378 6888 Acked Read 9 Done Sent Done false
22379 6889 Acked Write 6 Done Sent Done false
22380 6890 Acked Write 5 Done Sent Done false
22381 6891 Acked Read 6 Done Sent Done false
22382 6892 Acked Write 5 Done Sent Done false
22383 6893 Acked Write 4 Done Sent Done false
22384 6894 Acked Read 3 Done Sent Done false
22385 6895 Acked Read 1 Done Sent Done false
22386 6896 Acked Flush 0 Done Sent Done false
22387 6897 Acked Write 8 Done Sent Done false
22388 6898 Acked Read 6 Done Sent Done false
22389 6899 Acked Write 10 Done Sent Done false
22390 6900 Acked Read 5 Done Sent Done false
22391 6901 Acked Flush 0 Done Sent Done false
22392 6902 Acked Flush 0 Done Sent Done false
22393 6903 Acked Write 9 Done Sent Done false
22394 6904 Acked Read 2 Done Sent Done false
22395 6905 Acked Flush 0 Done Sent Done false
22396 6906 Acked Write 8 Done Sent Done false
22397 6907 Acked Read 9 Done Sent Done false
22398 6908 Acked Read 5 Done Sent Done false
22399 6909 Acked Write 5 Done Sent Done false
22400 6910 Acked Write 3 Done Sent Done false
22401 6911 Acked Write 9 Done Sent Done false
22402 6912 Acked Read 4 Done Sent Done false
22403 6913 Acked Flush 0 Done Sent Done false
22404 6914 Acked Write 1 Done Sent Done false
22405 6915 Acked Write 5 Done Sent Done false
22406 6916 Acked Write 3 Done Sent Done false
22407 6917 Acked Write 9 Done Sent Done false
22408 6918 Acked Read 4 Done Sent Done false
22409 6919 Acked Read 10 Done Sent Done false
22410 6920 Acked Write 8 Done Sent Done false
22411 6921 Acked Write 5 Done Sent Done false
22412 6922 Acked Write 2 Done Sent Done false
22413 6923 Acked Write 10 Done Sent Done false
22414 6924 Acked Read 3 Done Sent Done false
22415 6925 Acked Write 1 Done Sent Done false
22416 6926 Acked Read 1 Done Sent Done false
22417 6927 Acked Read 9 Done Sent Done false
22418 6928 Acked Write 10 Done Sent Done false
22419 6929 Acked Write 10 Done Sent Done false
22420 6930 Acked Write 4 Done Sent Done false
22421 6931 Acked Write 7 Done Sent Done false
22422 6932 Acked Read 4 Done Sent Done false
22423 6933 Acked Flush 0 Done Sent Done false
22424 6934 Acked Read 4 Done Sent Done false
22425 6935 Acked Read 7 Done Sent Done false
22426 6936 Acked Read 2 Done Sent Done false
22427 6937 Acked Read 3 Done Sent Done false
22428 6938 Acked Flush 0 Done Sent Done false
22429 6939 Acked Write 9 Done Sent Done false
22430 6940 Acked Flush 0 Done Sent Done false
22431 6941 Acked Flush 0 Done Sent Done false
22432 6942 Acked Write 7 Done Sent Done false
22433 6943 Acked Write 2 Done Sent Done false
22434 6944 Acked Read 10 Done Sent Done false
22435 6945 Acked Write 2 Done Sent Done false
22436 6946 Acked Read 10 Done Sent Done false
22437 6947 Acked Write 1 Done Sent Done false
22438 6948 Acked Write 9 Done Sent Done false
22439 6949 Acked Write 10 Done Sent Done false
22440 6950 Acked Write 1 Done Sent Done false
22441 6951 Acked Read 1 Done Sent Done false
22442 6952 Acked Read 1 Done Sent Done false
22443 6953 Acked Write 3 Done Sent Done false
22444 6954 Acked Write 2 Done Sent Done false
22445 6955 Acked Read 5 Done Sent Done false
22446 6956 Acked Write 6 Done Sent Done false
22447 6957 Acked Write 2 Done Sent Done false
22448 6958 Acked Write 8 Done Sent Done false
22449 6959 Acked Read 10 Done Sent Done false
22450 6960 Acked Read 6 Done Sent Done false
22451 6961 Acked Write 6 Done Sent Done false
22452 6962 Acked Write 2 Done Sent Done false
22453 6963 Acked Write 3 Done Sent Done false
22454 6964 Acked Write 5 Done Sent Done false
22455 6965 Acked Read 3 Done Sent Done false
22456 6966 Acked Read 4 Done Sent Done false
22457 6967 Acked Write 5 Done Sent Done false
22458 6968 Acked Read 4 Done Sent Done false
22459 6969 Acked Read 3 Done Sent Done false
22460 6970 Acked Read 6 Done Sent Done false
22461 6971 Acked Read 5 Done Sent Done false
22462 6972 Acked Write 4 Done Sent Done false
22463 6973 Acked Read 1 Done Sent Done false
22464 6974 Acked Flush 0 Done Sent Done false
22465 6975 Acked Write 10 Done Sent Done false
22466 6976 Acked Read 4 Done Sent Done false
22467 6977 Acked Write 1 Done Sent Done false
22468 6978 Acked Flush 0 Done Sent Done false
22469 6979 Acked Read 4 Done Sent Done false
22470 6980 Acked Read 7 Done Sent Done false
22471 6981 Acked Read 5 Done Sent Done false
22472 6982 Acked Read 10 Done Sent Done false
22473 6983 Acked Write 3 Done Sent Done false
22474 6984 Acked Write 4 Done Sent Done false
22475 6985 Acked Read 1 Done Sent Done false
22476 6986 Acked Write 10 Done Sent Done false
22477 6987 Acked Write 4 Done Sent Done false
22478 6988 Acked Write 5 Done Sent Done false
22479 6989 Acked Read 5 Done Sent Done false
22480 6990 Acked Read 2 Done Sent Done false
22481 6991 Acked Read 8 Done Sent Done false
22482 6992 Acked Read 6 Done Sent Done false
22483 6993 Acked Write 3 Done Sent Done false
22484 6994 Acked Read 8 Done Sent Done false
22485 6995 Acked Read 4 Done Sent Done false
22486 6996 Acked Read 1 Done Sent Done false
22487 6997 Acked Write 6 Done Sent Done false
22488 6998 Acked Write 10 Done Sent Done false
22489 6999 Acked Write 3 Done Sent Done false
22490 7000 Acked Read 5 Done Sent Done false
22491 7001 Acked Read 5 Done Sent Done false
22492 7002 Acked Write 10 Done Sent Done false
22493 7003 Acked Read 5 Done Sent Done false
22494 7004 Acked Write 6 Done Sent Done false
22495 7005 Acked Read 5 Done Sent Done false
22496 7006 Acked Read 9 Done Sent Done false
22497 7007 Acked Flush 0 Done Sent Done false
22498 7008 Acked Write 1 Done Sent Done false
22499 7009 Acked Read 4 Done Sent Done false
22500 7010 Acked Read 9 Done Sent Done false
22501 7011 Acked Write 6 Done Sent Done false
22502 7012 Acked Read 5 Done Sent Done false
22503 7013 Acked Read 1 Done Sent Done false
22504 7014 Acked Write 8 Done Sent Done false
22505 7015 Acked Read 2 Done Sent Done false
22506 7016 Acked Read 9 Done Sent Done false
22507 7017 Acked Flush 0 Done Sent Done false
22508 7018 Acked Read 10 Done Sent Done false
22509 7019 Acked Read 7 Done Sent Done false
22510 7020 Acked Write 1 Done Sent Done false
22511 7021 Acked Read 8 Done Sent Done false
22512 7022 Acked Read 7 Done Sent Done false
22513 7023 Acked Read 9 Done Sent Done false
22514 7024 Acked Write 5 Done Sent Done false
22515 7025 Acked Read 10 Done Sent Done false
22516 7026 Acked Write 2 Done Sent Done false
22517 7027 Acked Read 8 Done Sent Done false
22518 7028 Acked Flush 0 Done Sent Done false
22519 7029 Acked Write 5 Done Sent Done false
22520 7030 Acked Write 10 Done Sent Done false
22521 7031 Acked Read 1 Done Sent Done false
22522 7032 Acked Read 2 Done Sent Done false
22523 7033 Acked Read 8 Done Sent Done false
22524 7034 Acked Write 8 Done Sent Done false
22525 7035 Acked Read 6 Done Sent Done false
22526 7036 Acked Write 6 Done Sent Done false
22527 7037 Acked Read 4 Done Sent Done false
22528 7038 Acked Flush 0 Done Sent Done false
22529 STATES DS:0 DS:1 DS:2 TOTAL
22530 Sent 0 302 0 302
22531 Done 302 0 302 604
22532 Skipped 0 0 0 0
22533 Error 0 0 0 0
22534 Last Flush: 7038 6736 7038
22535 Downstairs last five completed: 6736 6735 6734 6733 6732
22536 Upstairs last five completed: 7038 7037 7036 7035 7034
22537 CLIENT: Up:0 ds:302 act:2
225382024-11-26T22:09:15.071ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
225392024-11-26T22:09:15.071ZWARNcrucible: ds_connection connect to 127.0.0.1:8820 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
225402024-11-26T22:09:15.071ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
225412024-11-26T22:09:15.071ZWARNcrucible: downstairs task for 1 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
225422024-11-26T22:09:15.071ZINFOcrucible: client task is exiting = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
22543 ----------------------------------------------------------------
22544 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
22545 States: Active Offline Active
22546 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
22547 6737 Acked Read 4 Done Sent Done false
22548 6738 Acked Read 5 Done Sent Done false
22549 6739 Acked Read 4 Done Sent Done false
22550 6740 Acked Read 9 Done Sent Done false
22551 6741 Acked Read 2 Done Sent Done false
22552 6742 Acked Read 3 Done Sent Done false
22553 6743 Acked Write 4 Done Sent Done false
22554 6744 Acked Flush 0 Done Sent Done false
22555 6745 Acked Write 7 Done Sent Done false
22556 6746 Acked Read 10 Done Sent Done false
22557 6747 Acked Write 1 Done Sent Done false
22558 6748 Acked Flush 0 Done Sent Done false
22559 6749 Acked Read 7 Done Sent Done false
22560 6750 Acked Write 8 Done Sent Done false
22561 6751 Acked Flush 0 Done Sent Done false
22562 6752 Acked Write 10 Done Sent Done false
22563 6753 Acked Read 4 Done Sent Done false
22564 6754 Acked Read 9 Done Sent Done false
22565 6755 Acked Write 4 Done Sent Done false
22566 6756 Acked Read 7 Done Sent Done false
22567 6757 Acked Write 7 Done Sent Done false
22568 6758 Acked Write 10 Done Sent Done false
22569 6759 Acked Read 7 Done Sent Done false
22570 6760 Acked Write 7 Done Sent Done false
22571 6761 Acked Write 6 Done Sent Done false
22572 6762 Acked Read 8 Done Sent Done false
22573 6763 Acked Read 6 Done Sent Done false
22574 6764 Acked Write 5 Done Sent Done false
22575 6765 Acked Read 9 Done Sent Done false
22576 6766 Acked Write 3 Done Sent Done false
22577 6767 Acked Read 9 Done Sent Done false
22578 6768 Acked Write 8 Done Sent Done false
22579 6769 Acked Read 5 Done Sent Done false
22580 6770 Acked Read 7 Done Sent Done false
22581 6771 Acked Write 3 Done Sent Done false
22582 6772 Acked Read 5 Done Sent Done false
22583 6773 Acked Read 3 Done Sent Done false
22584 6774 Acked Write 9 Done Sent Done false
22585 6775 Acked Read 9 Done Sent Done false
22586 6776 Acked Write 7 Done Sent Done false
22587 6777 Acked Read 2 Done Sent Done false
22588 6778 Acked Read 10 Done Sent Done false
22589 6779 Acked Write 6 Done Sent Done false
22590 6780 Acked Read 2 Done Sent Done false
22591 6781 Acked Write 2 Done Sent Done false
22592 6782 Acked Write 8 Done Sent Done false
22593 6783 Acked Write 7 Done Sent Done false
22594 6784 Acked Write 10 Done Sent Done false
22595 6785 Acked Write 3 Done Sent Done false
22596 6786 Acked Read 10 Done Sent Done false
22597 6787 Acked Read 9 Done Sent Done false
22598 6788 Acked Write 4 Done Sent Done false
22599 6789 Acked Read 9 Done Sent Done false
22600 6790 Acked Write 1 Done Sent Done false
22601 6791 Acked Read 3 Done Sent Done false
22602 6792 Acked Write 1 Done Sent Done false
22603 6793 Acked Read 7 Done Sent Done false
22604 6794 Acked Read 9 Done Sent Done false
22605 6795 Acked Write 4 Done Sent Done false
22606 6796 Acked Read 3 Done Sent Done false
22607 6797 Acked Write 6 Done Sent Done false
22608 6798 Acked Read 1 Done Sent Done false
22609 6799 Acked Write 10 Done Sent Done false
22610 6800 Acked Read 1 Done Sent Done false
22611 6801 Acked Write 7 Done Sent Done false
22612 6802 Acked Read 6 Done Sent Done false
22613 6803 Acked Write 2 Done Sent Done false
22614 6804 Acked Read 3 Done Sent Done false
22615 6805 Acked Write 3 Done Sent Done false
22616 6806 Acked Flush 0 Done Sent Done false
22617 6807 Acked Read 3 Done Sent Done false
22618 6808 Acked Flush 0 Done Sent Done false
22619 6809 Acked Write 1 Done Sent Done false
22620 6810 Acked Read 4 Done Sent Done false
22621 6811 Acked Write 9 Done Sent Done false
22622 6812 Acked Write 5 Done Sent Done false
22623 6813 Acked Write 1 Done Sent Done false
22624 6814 Acked Read 7 Done Sent Done false
22625 6815 Acked Read 2 Done Sent Done false
22626 6816 Acked Read 4 Done Sent Done false
22627 6817 Acked Read 2 Done Sent Done false
22628 6818 Acked Read 6 Done Sent Done false
22629 6819 Acked Read 10 Done Sent Done false
22630 6820 Acked Write 4 Done Sent Done false
22631 6821 Acked Read 7 Done Sent Done false
22632 6822 Acked Write 1 Done Sent Done false
22633 6823 Acked Write 4 Done Sent Done false
22634 6824 Acked Read 3 Done Sent Done false
22635 6825 Acked Write 10 Done Sent Done false
22636 6826 Acked Read 4 Done Sent Done false
22637 6827 Acked Write 5 Done Sent Done false
22638 6828 Acked Write 2 Done Sent Done false
22639 6829 Acked Write 6 Done Sent Done false
22640 6830 Acked Write 5 Done Sent Done false
22641 6831 Acked Read 8 Done Sent Done false
22642 6832 Acked Read 9 Done Sent Done false
22643 6833 Acked Read 2 Done Sent Done false
22644 6834 Acked Read 8 Done Sent Done false
22645 6835 Acked Flush 0 Done Sent Done false
22646 6836 Acked Read 7 Done Sent Done false
22647 6837 Acked Write 4 Done Sent Done false
22648 6838 Acked Flush 0 Done Sent Done false
22649 6839 Acked Read 5 Done Sent Done false
22650 6840 Acked Write 3 Done Sent Done false
22651 6841 Acked Read 1 Done Sent Done false
22652 6842 Acked Read 7 Done Sent Done false
22653 6843 Acked Write 7 Done Sent Done false
22654 6844 Acked Read 7 Done Sent Done false
22655 6845 Acked Write 3 Done Sent Done false
22656 6846 Acked Write 7 Done Sent Done false
22657 6847 Acked Read 2 Done Sent Done false
22658 6848 Acked Read 5 Done Sent Done false
22659 6849 Acked Flush 0 Done Sent Done false
22660 6850 Acked Write 6 Done Sent Done false
22661 6851 Acked Write 9 Done Sent Done false
22662 6852 Acked Write 5 Done Sent Done false
22663 6853 Acked Read 5 Done Sent Done false
22664 6854 Acked Read 8 Done Sent Done false
22665 6855 Acked Write 7 Done Sent Done false
22666 6856 Acked Write 7 Done Sent Done false
22667 6857 Acked Read 5 Done Sent Done false
22668 6858 Acked Write 10 Done Sent Done false
22669 6859 Acked Write 5 Done Sent Done false
22670 6860 Acked Read 4 Done Sent Done false
22671 6861 Acked Read 7 Done Sent Done false
22672 6862 Acked Read 8 Done Sent Done false
22673 6863 Acked Read 6 Done Sent Done false
22674 6864 Acked Write 3 Done Sent Done false
22675 6865 Acked Write 9 Done Sent Done false
22676 6866 Acked Write 6 Done Sent Done false
22677 6867 Acked Write 4 Done Sent Done false
22678 6868 Acked Write 2 Done Sent Done false
22679 6869 Acked Write 2 Done Sent Done false
22680 6870 Acked Read 8 Done Sent Done false
22681 6871 Acked Read 1 Done Sent Done false
22682 6872 Acked Write 10 Done Sent Done false
22683 6873 Acked Write 2 Done Sent Done false
22684 6874 Acked Write 4 Done Sent Done false
22685 6875 Acked Write 3 Done Sent Done false
22686 6876 Acked Flush 0 Done Sent Done false
22687 6877 Acked Write 1 Done Sent Done false
22688 6878 Acked Write 1 Done Sent Done false
22689 6879 Acked Write 4 Done Sent Done false
22690 6880 Acked Write 2 Done Sent Done false
22691 6881 Acked Read 3 Done Sent Done false
22692 6882 Acked Read 5 Done Sent Done false
22693 6883 Acked Read 2 Done Sent Done false
22694 6884 Acked Write 6 Done Sent Done false
22695 6885 Acked Read 2 Done Sent Done false
22696 6886 Acked Write 9 Done Sent Done false
22697 6887 Acked Flush 0 Done Sent Done false
22698 6888 Acked Read 9 Done Sent Done false
22699 6889 Acked Write 6 Done Sent Done false
22700 6890 Acked Write 5 Done Sent Done false
22701 6891 Acked Read 6 Done Sent Done false
22702 6892 Acked Write 5 Done Sent Done false
22703 6893 Acked Write 4 Done Sent Done false
22704 6894 Acked Read 3 Done Sent Done false
22705 6895 Acked Read 1 Done Sent Done false
22706 6896 Acked Flush 0 Done Sent Done false
22707 6897 Acked Write 8 Done Sent Done false
22708 6898 Acked Read 6 Done Sent Done false
22709 6899 Acked Write 10 Done Sent Done false
22710 6900 Acked Read 5 Done Sent Done false
22711 6901 Acked Flush 0 Done Sent Done false
22712 6902 Acked Flush 0 Done Sent Done false
22713 6903 Acked Write 9 Done Sent Done false
22714 6904 Acked Read 2 Done Sent Done false
22715 6905 Acked Flush 0 Done Sent Done false
22716 6906 Acked Write 8 Done Sent Done false
22717 6907 Acked Read 9 Done Sent Done false
22718 6908 Acked Read 5 Done Sent Done false
22719 6909 Acked Write 5 Done Sent Done false
22720 6910 Acked Write 3 Done Sent Done false
22721 6911 Acked Write 9 Done Sent Done false
22722 6912 Acked Read 4 Done Sent Done false
22723 6913 Acked Flush 0 Done Sent Done false
22724 6914 Acked Write 1 Done Sent Done false
22725 6915 Acked Write 5 Done Sent Done false
22726 6916 Acked Write 3 Done Sent Done false
22727 6917 Acked Write 9 Done Sent Done false
22728 6918 Acked Read 4 Done Sent Done false
22729 6919 Acked Read 10 Done Sent Done false
22730 6920 Acked Write 8 Done Sent Done false
22731 6921 Acked Write 5 Done Sent Done false
22732 6922 Acked Write 2 Done Sent Done false
22733 6923 Acked Write 10 Done Sent Done false
22734 6924 Acked Read 3 Done Sent Done false
22735 6925 Acked Write 1 Done Sent Done false
22736 6926 Acked Read 1 Done Sent Done false
22737 6927 Acked Read 9 Done Sent Done false
22738 6928 Acked Write 10 Done Sent Done false
22739 6929 Acked Write 10 Done Sent Done false
22740 6930 Acked Write 4 Done Sent Done false
22741 6931 Acked Write 7 Done Sent Done false
22742 6932 Acked Read 4 Done Sent Done false
22743 6933 Acked Flush 0 Done Sent Done false
22744 6934 Acked Read 4 Done Sent Done false
22745 6935 Acked Read 7 Done Sent Done false
22746 6936 Acked Read 2 Done Sent Done false
22747 6937 Acked Read 3 Done Sent Done false
22748 6938 Acked Flush 0 Done Sent Done false
22749 6939 Acked Write 9 Done Sent Done false
22750 6940 Acked Flush 0 Done Sent Done false
22751 6941 Acked Flush 0 Done Sent Done false
22752 6942 Acked Write 7 Done Sent Done false
22753 6943 Acked Write 2 Done Sent Done false
22754 6944 Acked Read 10 Done Sent Done false
22755 6945 Acked Write 2 Done Sent Done false
22756 6946 Acked Read 10 Done Sent Done false
22757 6947 Acked Write 1 Done Sent Done false
22758 6948 Acked Write 9 Done Sent Done false
22759 6949 Acked Write 10 Done Sent Done false
22760 6950 Acked Write 1 Done Sent Done false
22761 6951 Acked Read 1 Done Sent Done false
22762 6952 Acked Read 1 Done Sent Done false
22763 6953 Acked Write 3 Done Sent Done false
22764 6954 Acked Write 2 Done Sent Done false
22765 6955 Acked Read 5 Done Sent Done false
22766 6956 Acked Write 6 Done Sent Done false
22767 6957 Acked Write 2 Done Sent Done false
22768 6958 Acked Write 8 Done Sent Done false
22769 6959 Acked Read 10 Done Sent Done false
22770 6960 Acked Read 6 Done Sent Done false
22771 6961 Acked Write 6 Done Sent Done false
22772 6962 Acked Write 2 Done Sent Done false
22773 6963 Acked Write 3 Done Sent Done false
22774 6964 Acked Write 5 Done Sent Done false
22775 6965 Acked Read 3 Done Sent Done false
22776 6966 Acked Read 4 Done Sent Done false
22777 6967 Acked Write 5 Done Sent Done false
22778 6968 Acked Read 4 Done Sent Done false
22779 6969 Acked Read 3 Done Sent Done false
22780 6970 Acked Read 6 Done Sent Done false
22781 6971 Acked Read 5 Done Sent Done false
22782 6972 Acked Write 4 Done Sent Done false
22783 6973 Acked Read 1 Done Sent Done false
22784 6974 Acked Flush 0 Done Sent Done false
22785 6975 Acked Write 10 Done Sent Done false
22786 6976 Acked Read 4 Done Sent Done false
22787 6977 Acked Write 1 Done Sent Done false
22788 6978 Acked Flush 0 Done Sent Done false
22789 6979 Acked Read 4 Done Sent Done false
22790 6980 Acked Read 7 Done Sent Done false
22791 6981 Acked Read 5 Done Sent Done false
22792 6982 Acked Read 10 Done Sent Done false
22793 6983 Acked Write 3 Done Sent Done false
22794 6984 Acked Write 4 Done Sent Done false
22795 6985 Acked Read 1 Done Sent Done false
22796 6986 Acked Write 10 Done Sent Done false
22797 6987 Acked Write 4 Done Sent Done false
22798 6988 Acked Write 5 Done Sent Done false
22799 6989 Acked Read 5 Done Sent Done false
22800 6990 Acked Read 2 Done Sent Done false
22801 6991 Acked Read 8 Done Sent Done false
22802 6992 Acked Read 6 Done Sent Done false
22803 6993 Acked Write 3 Done Sent Done false
22804 6994 Acked Read 8 Done Sent Done false
22805 6995 Acked Read 4 Done Sent Done false
22806 6996 Acked Read 1 Done Sent Done false
22807 6997 Acked Write 6 Done Sent Done false
22808 6998 Acked Write 10 Done Sent Done false
22809 6999 Acked Write 3 Done Sent Done false
22810 7000 Acked Read 5 Done Sent Done false
22811 7001 Acked Read 5 Done Sent Done false
22812 7002 Acked Write 10 Done Sent Done false
22813 7003 Acked Read 5 Done Sent Done false
22814 7004 Acked Write 6 Done Sent Done false
22815 7005 Acked Read 5 Done Sent Done false
22816 7006 Acked Read 9 Done Sent Done false
22817 7007 Acked Flush 0 Done Sent Done false
22818 7008 Acked Write 1 Done Sent Done false
22819 7009 Acked Read 4 Done Sent Done false
22820 7010 Acked Read 9 Done Sent Done false
22821 7011 Acked Write 6 Done Sent Done false
22822 7012 Acked Read 5 Done Sent Done false
22823 7013 Acked Read 1 Done Sent Done false
22824 7014 Acked Write 8 Done Sent Done false
22825 7015 Acked Read 2 Done Sent Done false
22826 7016 Acked Read 9 Done Sent Done false
22827 7017 Acked Flush 0 Done Sent Done false
22828 7018 Acked Read 10 Done Sent Done false
22829 7019 Acked Read 7 Done Sent Done false
22830 7020 Acked Write 1 Done Sent Done false
22831 7021 Acked Read 8 Done Sent Done false
22832 7022 Acked Read 7 Done Sent Done false
22833 7023 Acked Read 9 Done Sent Done false
22834 7024 Acked Write 5 Done Sent Done false
22835 7025 Acked Read 10 Done Sent Done false
22836 7026 Acked Write 2 Done Sent Done false
22837 7027 Acked Read 8 Done Sent Done false
22838 7028 Acked Flush 0 Done Sent Done false
22839 7029 Acked Write 5 Done Sent Done false
22840 7030 Acked Write 10 Done Sent Done false
22841 7031 Acked Read 1 Done Sent Done false
22842 7032 Acked Read 2 Done Sent Done false
22843 7033 Acked Read 8 Done Sent Done false
22844 7034 Acked Write 8 Done Sent Done false
22845 7035 Acked Read 6 Done Sent Done false
22846 7036 Acked Write 6 Done Sent Done false
22847 7037 Acked Read 4 Done Sent Done false
22848 7038 Acked Flush 0 Done Sent Done false
22849 STATES DS:0 DS:1 DS:2 TOTAL
22850 Sent 0 302 0 302
22851 Done 302 0 302 604
22852 Skipped 0 0 0 0
22853 Error 0 0 0 0
22854 Last Flush: 7038 6736 7038
22855 Downstairs last five completed: 6736 6735 6734 6733 6732
22856 Upstairs last five completed: 7038 7037 7036 7035 7034
22857 CLIENT: Up:0 ds:302 act:2
22858 [8820] got start action
22859 [8820] I am going to restart
22860 Starting downstairs at port 8820
22861 Make output file at "/tmp/dsc/downstairs-8820.txt"
22862 [8820][1] reports Starting
22863 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1223)
22864 [8820][1] reports Running
22865 ----------------------------------------------------------------
22866 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
22867 States: Active Offline Active
22868 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
22869 6737 Acked Read 4 Done Sent Done false
22870 6738 Acked Read 5 Done Sent Done false
22871 6739 Acked Read 4 Done Sent Done false
22872 6740 Acked Read 9 Done Sent Done false
22873 6741 Acked Read 2 Done Sent Done false
22874 6742 Acked Read 3 Done Sent Done false
22875 6743 Acked Write 4 Done Sent Done false
22876 6744 Acked Flush 0 Done Sent Done false
22877 6745 Acked Write 7 Done Sent Done false
22878 6746 Acked Read 10 Done Sent Done false
22879 6747 Acked Write 1 Done Sent Done false
22880 6748 Acked Flush 0 Done Sent Done false
22881 6749 Acked Read 7 Done Sent Done false
22882 6750 Acked Write 8 Done Sent Done false
22883 6751 Acked Flush 0 Done Sent Done false
22884 6752 Acked Write 10 Done Sent Done false
22885 6753 Acked Read 4 Done Sent Done false
22886 6754 Acked Read 9 Done Sent Done false
22887 6755 Acked Write 4 Done Sent Done false
22888 6756 Acked Read 7 Done Sent Done false
22889 6757 Acked Write 7 Done Sent Done false
22890 6758 Acked Write 10 Done Sent Done false
22891 6759 Acked Read 7 Done Sent Done false
22892 6760 Acked Write 7 Done Sent Done false
22893 6761 Acked Write 6 Done Sent Done false
22894 6762 Acked Read 8 Done Sent Done false
22895 6763 Acked Read 6 Done Sent Done false
22896 6764 Acked Write 5 Done Sent Done false
22897 6765 Acked Read 9 Done Sent Done false
22898 6766 Acked Write 3 Done Sent Done false
22899 6767 Acked Read 9 Done Sent Done false
22900 6768 Acked Write 8 Done Sent Done false
22901 6769 Acked Read 5 Done Sent Done false
22902 6770 Acked Read 7 Done Sent Done false
22903 6771 Acked Write 3 Done Sent Done false
22904 6772 Acked Read 5 Done Sent Done false
22905 6773 Acked Read 3 Done Sent Done false
22906 6774 Acked Write 9 Done Sent Done false
22907 6775 Acked Read 9 Done Sent Done false
22908 6776 Acked Write 7 Done Sent Done false
22909 6777 Acked Read 2 Done Sent Done false
22910 6778 Acked Read 10 Done Sent Done false
22911 6779 Acked Write 6 Done Sent Done false
22912 6780 Acked Read 2 Done Sent Done false
22913 6781 Acked Write 2 Done Sent Done false
22914 6782 Acked Write 8 Done Sent Done false
22915 6783 Acked Write 7 Done Sent Done false
22916 6784 Acked Write 10 Done Sent Done false
22917 6785 Acked Write 3 Done Sent Done false
22918 6786 Acked Read 10 Done Sent Done false
22919 6787 Acked Read 9 Done Sent Done false
22920 6788 Acked Write 4 Done Sent Done false
22921 6789 Acked Read 9 Done Sent Done false
22922 6790 Acked Write 1 Done Sent Done false
22923 6791 Acked Read 3 Done Sent Done false
22924 6792 Acked Write 1 Done Sent Done false
22925 6793 Acked Read 7 Done Sent Done false
22926 6794 Acked Read 9 Done Sent Done false
22927 6795 Acked Write 4 Done Sent Done false
22928 6796 Acked Read 3 Done Sent Done false
22929 6797 Acked Write 6 Done Sent Done false
22930 6798 Acked Read 1 Done Sent Done false
22931 6799 Acked Write 10 Done Sent Done false
22932 6800 Acked Read 1 Done Sent Done false
22933 6801 Acked Write 7 Done Sent Done false
22934 6802 Acked Read 6 Done Sent Done false
22935 6803 Acked Write 2 Done Sent Done false
22936 6804 Acked Read 3 Done Sent Done false
22937 6805 Acked Write 3 Done Sent Done false
22938 6806 Acked Flush 0 Done Sent Done false
22939 6807 Acked Read 3 Done Sent Done false
22940 6808 Acked Flush 0 Done Sent Done false
22941 6809 Acked Write 1 Done Sent Done false
22942 6810 Acked Read 4 Done Sent Done false
22943 6811 Acked Write 9 Done Sent Done false
22944 6812 Acked Write 5 Done Sent Done false
22945 6813 Acked Write 1 Done Sent Done false
22946 6814 Acked Read 7 Done Sent Done false
22947 6815 Acked Read 2 Done Sent Done false
22948 6816 Acked Read 4 Done Sent Done false
22949 6817 Acked Read 2 Done Sent Done false
22950 6818 Acked Read 6 Done Sent Done false
22951 6819 Acked Read 10 Done Sent Done false
22952 6820 Acked Write 4 Done Sent Done false
22953 6821 Acked Read 7 Done Sent Done false
22954 6822 Acked Write 1 Done Sent Done false
22955 6823 Acked Write 4 Done Sent Done false
22956 6824 Acked Read 3 Done Sent Done false
22957 6825 Acked Write 10 Done Sent Done false
22958 6826 Acked Read 4 Done Sent Done false
22959 6827 Acked Write 5 Done Sent Done false
22960 6828 Acked Write 2 Done Sent Done false
22961 6829 Acked Write 6 Done Sent Done false
22962 6830 Acked Write 5 Done Sent Done false
22963 6831 Acked Read 8 Done Sent Done false
22964 6832 Acked Read 9 Done Sent Done false
22965 6833 Acked Read 2 Done Sent Done false
22966 6834 Acked Read 8 Done Sent Done false
22967 6835 Acked Flush 0 Done Sent Done false
22968 6836 Acked Read 7 Done Sent Done false
22969 6837 Acked Write 4 Done Sent Done false
22970 6838 Acked Flush 0 Done Sent Done false
22971 6839 Acked Read 5 Done Sent Done false
22972 6840 Acked Write 3 Done Sent Done false
22973 6841 Acked Read 1 Done Sent Done false
22974 6842 Acked Read 7 Done Sent Done false
22975 6843 Acked Write 7 Done Sent Done false
22976 6844 Acked Read 7 Done Sent Done false
22977 6845 Acked Write 3 Done Sent Done false
22978 6846 Acked Write 7 Done Sent Done false
22979 6847 Acked Read 2 Done Sent Done false
22980 6848 Acked Read 5 Done Sent Done false
22981 6849 Acked Flush 0 Done Sent Done false
22982 6850 Acked Write 6 Done Sent Done false
22983 6851 Acked Write 9 Done Sent Done false
22984 6852 Acked Write 5 Done Sent Done false
22985 6853 Acked Read 5 Done Sent Done false
22986 6854 Acked Read 8 Done Sent Done false
22987 6855 Acked Write 7 Done Sent Done false
22988 6856 Acked Write 7 Done Sent Done false
22989 6857 Acked Read 5 Done Sent Done false
22990 6858 Acked Write 10 Done Sent Done false
22991 6859 Acked Write 5 Done Sent Done false
22992 6860 Acked Read 4 Done Sent Done false
22993 6861 Acked Read 7 Done Sent Done false
22994 6862 Acked Read 8 Done Sent Done false
22995 6863 Acked Read 6 Done Sent Done false
22996 6864 Acked Write 3 Done Sent Done false
22997 6865 Acked Write 9 Done Sent Done false
22998 6866 Acked Write 6 Done Sent Done false
22999 6867 Acked Write 4 Done Sent Done false
23000 6868 Acked Write 2 Done Sent Done false
23001 6869 Acked Write 2 Done Sent Done false
23002 6870 Acked Read 8 Done Sent Done false
23003 6871 Acked Read 1 Done Sent Done false
23004 6872 Acked Write 10 Done Sent Done false
23005 6873 Acked Write 2 Done Sent Done false
23006 6874 Acked Write 4 Done Sent Done false
23007 6875 Acked Write 3 Done Sent Done false
23008 6876 Acked Flush 0 Done Sent Done false
23009 6877 Acked Write 1 Done Sent Done false
23010 6878 Acked Write 1 Done Sent Done false
23011 6879 Acked Write 4 Done Sent Done false
23012 6880 Acked Write 2 Done Sent Done false
23013 6881 Acked Read 3 Done Sent Done false
23014 6882 Acked Read 5 Done Sent Done false
23015 6883 Acked Read 2 Done Sent Done false
23016 6884 Acked Write 6 Done Sent Done false
23017 6885 Acked Read 2 Done Sent Done false
23018 6886 Acked Write 9 Done Sent Done false
23019 6887 Acked Flush 0 Done Sent Done false
23020 6888 Acked Read 9 Done Sent Done false
23021 6889 Acked Write 6 Done Sent Done false
23022 6890 Acked Write 5 Done Sent Done false
23023 6891 Acked Read 6 Done Sent Done false
23024 6892 Acked Write 5 Done Sent Done false
23025 6893 Acked Write 4 Done Sent Done false
23026 6894 Acked Read 3 Done Sent Done false
23027 6895 Acked Read 1 Done Sent Done false
23028 6896 Acked Flush 0 Done Sent Done false
23029 6897 Acked Write 8 Done Sent Done false
23030 6898 Acked Read 6 Done Sent Done false
23031 6899 Acked Write 10 Done Sent Done false
23032 6900 Acked Read 5 Done Sent Done false
23033 6901 Acked Flush 0 Done Sent Done false
23034 6902 Acked Flush 0 Done Sent Done false
23035 6903 Acked Write 9 Done Sent Done false
23036 6904 Acked Read 2 Done Sent Done false
23037 6905 Acked Flush 0 Done Sent Done false
23038 6906 Acked Write 8 Done Sent Done false
23039 6907 Acked Read 9 Done Sent Done false
23040 6908 Acked Read 5 Done Sent Done false
23041 6909 Acked Write 5 Done Sent Done false
23042 6910 Acked Write 3 Done Sent Done false
23043 6911 Acked Write 9 Done Sent Done false
23044 6912 Acked Read 4 Done Sent Done false
23045 6913 Acked Flush 0 Done Sent Done false
23046 6914 Acked Write 1 Done Sent Done false
23047 6915 Acked Write 5 Done Sent Done false
23048 6916 Acked Write 3 Done Sent Done false
23049 6917 Acked Write 9 Done Sent Done false
23050 6918 Acked Read 4 Done Sent Done false
23051 6919 Acked Read 10 Done Sent Done false
23052 6920 Acked Write 8 Done Sent Done false
23053 6921 Acked Write 5 Done Sent Done false
23054 6922 Acked Write 2 Done Sent Done false
23055 6923 Acked Write 10 Done Sent Done false
23056 6924 Acked Read 3 Done Sent Done false
23057 6925 Acked Write 1 Done Sent Done false
23058 6926 Acked Read 1 Done Sent Done false
23059 6927 Acked Read 9 Done Sent Done false
23060 6928 Acked Write 10 Done Sent Done false
23061 6929 Acked Write 10 Done Sent Done false
23062 6930 Acked Write 4 Done Sent Done false
23063 6931 Acked Write 7 Done Sent Done false
23064 6932 Acked Read 4 Done Sent Done false
23065 6933 Acked Flush 0 Done Sent Done false
23066 6934 Acked Read 4 Done Sent Done false
23067 6935 Acked Read 7 Done Sent Done false
23068 6936 Acked Read 2 Done Sent Done false
23069 6937 Acked Read 3 Done Sent Done false
23070 6938 Acked Flush 0 Done Sent Done false
23071 6939 Acked Write 9 Done Sent Done false
23072 6940 Acked Flush 0 Done Sent Done false
23073 6941 Acked Flush 0 Done Sent Done false
23074 6942 Acked Write 7 Done Sent Done false
23075 6943 Acked Write 2 Done Sent Done false
23076 6944 Acked Read 10 Done Sent Done false
23077 6945 Acked Write 2 Done Sent Done false
23078 6946 Acked Read 10 Done Sent Done false
23079 6947 Acked Write 1 Done Sent Done false
23080 6948 Acked Write 9 Done Sent Done false
23081 6949 Acked Write 10 Done Sent Done false
23082 6950 Acked Write 1 Done Sent Done false
23083 6951 Acked Read 1 Done Sent Done false
23084 6952 Acked Read 1 Done Sent Done false
23085 6953 Acked Write 3 Done Sent Done false
23086 6954 Acked Write 2 Done Sent Done false
23087 6955 Acked Read 5 Done Sent Done false
23088 6956 Acked Write 6 Done Sent Done false
23089 6957 Acked Write 2 Done Sent Done false
23090 6958 Acked Write 8 Done Sent Done false
23091 6959 Acked Read 10 Done Sent Done false
23092 6960 Acked Read 6 Done Sent Done false
23093 6961 Acked Write 6 Done Sent Done false
23094 6962 Acked Write 2 Done Sent Done false
23095 6963 Acked Write 3 Done Sent Done false
23096 6964 Acked Write 5 Done Sent Done false
23097 6965 Acked Read 3 Done Sent Done false
23098 6966 Acked Read 4 Done Sent Done false
23099 6967 Acked Write 5 Done Sent Done false
23100 6968 Acked Read 4 Done Sent Done false
23101 6969 Acked Read 3 Done Sent Done false
23102 6970 Acked Read 6 Done Sent Done false
23103 6971 Acked Read 5 Done Sent Done false
23104 6972 Acked Write 4 Done Sent Done false
23105 6973 Acked Read 1 Done Sent Done false
23106 6974 Acked Flush 0 Done Sent Done false
23107 6975 Acked Write 10 Done Sent Done false
23108 6976 Acked Read 4 Done Sent Done false
23109 6977 Acked Write 1 Done Sent Done false
23110 6978 Acked Flush 0 Done Sent Done false
23111 6979 Acked Read 4 Done Sent Done false
23112 6980 Acked Read 7 Done Sent Done false
23113 6981 Acked Read 5 Done Sent Done false
23114 6982 Acked Read 10 Done Sent Done false
23115 6983 Acked Write 3 Done Sent Done false
23116 6984 Acked Write 4 Done Sent Done false
23117 6985 Acked Read 1 Done Sent Done false
23118 6986 Acked Write 10 Done Sent Done false
23119 6987 Acked Write 4 Done Sent Done false
23120 6988 Acked Write 5 Done Sent Done false
23121 6989 Acked Read 5 Done Sent Done false
23122 6990 Acked Read 2 Done Sent Done false
23123 6991 Acked Read 8 Done Sent Done false
23124 6992 Acked Read 6 Done Sent Done false
23125 6993 Acked Write 3 Done Sent Done false
23126 6994 Acked Read 8 Done Sent Done false
23127 6995 Acked Read 4 Done Sent Done false
23128 6996 Acked Read 1 Done Sent Done false
23129 6997 Acked Write 6 Done Sent Done false
23130 6998 Acked Write 10 Done Sent Done false
23131 6999 Acked Write 3 Done Sent Done false
23132 7000 Acked Read 5 Done Sent Done false
23133 7001 Acked Read 5 Done Sent Done false
23134 7002 Acked Write 10 Done Sent Done false
23135 7003 Acked Read 5 Done Sent Done false
23136 7004 Acked Write 6 Done Sent Done false
23137 7005 Acked Read 5 Done Sent Done false
23138 7006 Acked Read 9 Done Sent Done false
23139 7007 Acked Flush 0 Done Sent Done false
23140 7008 Acked Write 1 Done Sent Done false
23141 7009 Acked Read 4 Done Sent Done false
23142 7010 Acked Read 9 Done Sent Done false
23143 7011 Acked Write 6 Done Sent Done false
23144 7012 Acked Read 5 Done Sent Done false
23145 7013 Acked Read 1 Done Sent Done false
23146 7014 Acked Write 8 Done Sent Done false
23147 7015 Acked Read 2 Done Sent Done false
23148 7016 Acked Read 9 Done Sent Done false
23149 7017 Acked Flush 0 Done Sent Done false
23150 7018 Acked Read 10 Done Sent Done false
23151 7019 Acked Read 7 Done Sent Done false
23152 7020 Acked Write 1 Done Sent Done false
23153 7021 Acked Read 8 Done Sent Done false
23154 7022 Acked Read 7 Done Sent Done false
23155 7023 Acked Read 9 Done Sent Done false
23156 7024 Acked Write 5 Done Sent Done false
23157 7025 Acked Read 10 Done Sent Done false
23158 7026 Acked Write 2 Done Sent Done false
23159 7027 Acked Read 8 Done Sent Done false
23160 7028 Acked Flush 0 Done Sent Done false
23161 7029 Acked Write 5 Done Sent Done false
23162 7030 Acked Write 10 Done Sent Done false
23163 7031 Acked Read 1 Done Sent Done false
23164 7032 Acked Read 2 Done Sent Done false
23165 7033 Acked Read 8 Done Sent Done false
23166 7034 Acked Write 8 Done Sent Done false
23167 7035 Acked Read 6 Done Sent Done false
23168 7036 Acked Write 6 Done Sent Done false
23169 7037 Acked Read 4 Done Sent Done false
23170 7038 Acked Flush 0 Done Sent Done false
23171 STATES DS:0 DS:1 DS:2 TOTAL
23172 Sent 0 302 0 302
23173 Done 302 0 302 604
23174 Skipped 0 0 0 0
23175 Error 0 0 0 0
23176 Last Flush: 7038 6736 7038
23177 Downstairs last five completed: 6736 6735 6734 6733 6732
23178 Upstairs last five completed: 7038 7037 7036 7035 7034
23179 CLIENT: Up:0 ds:302 act:2
231802024-11-26T22:09:25.072ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
231812024-11-26T22:09:25.073ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:64381) = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
231822024-11-26T22:09:25.073ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
231832024-11-26T22:09:25.074ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
231842024-11-26T22:09:25.074ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
231852024-11-26T22:09:25.074ZINFOcrucible: send last flush ID to this DS: 6736 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
231862024-11-26T22:09:25.074ZINFOcrucible: Replied this last flush ID: 6736 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
231872024-11-26T22:09:25.074ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
231882024-11-26T22:09:25.074ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
231892024-11-26T22:09:25.074ZINFOcrucible: [1] client re-new 302 jobs since flush 6736 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
231902024-11-26T22:09:25.075ZINFOcrucible: [1] Replayed 302 jobs since flush: 6736 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
23191 ----------------------------------------------------------------
23192 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
23193 Downstairs last five completed: 7038 7037 7036 7035 7034
23194 Upstairs last five completed: 7038 7037 7036 7035 7034
23195 CLIENT: Up:0 ds:0 act:3
23196 Replay: All jobs finished
23197 Main task has work to do, go find it
23198 got dsc Stop(1)
23199 stop 1
23200 Nov 26 22:09:25.305 INFO request completed, [8820] Got stop action so:false kr:false
23201 latency_us: 340, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 709ffb0d-cd4e-4cd1-ae06-bac665c6ad4a, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
232022024-11-26T22:09:25.305ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
232032024-11-26T22:09:25.306ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
232042024-11-26T22:09:25.306ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
232052024-11-26T22:09:25.306ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
232062024-11-26T22:09:25.306ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
232072024-11-26T22:09:25.306ZINFOcrucible: client task is exiting = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
23208 Nov 26 22:09:25.306 INFO request completed, latency_us: 177, response_code: 200, uri: /state/cid/1, method: GET, req_id: 72fdac86-4d92-47b9-abe7-b575ee65823f, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
23209 [8820] Exited with: ExitStatus(unix_wait_status(9))
23210 [8820][1] reports Exit
23211 Nov 26 22:09:29.308 INFO request completed, latency_us: 199, response_code: 200, uri: /state/cid/1, method: GET, req_id: 33e81083-6f60-4083-b6e0-66097218e983, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
23212 001/300 Read block 4779 len 32768
23213 002/300 Read block 2828 len 24576
23214 003/300 Read block 478 len 24576
23215 004/300 Read block 2484 len 4096
23216 005/300 Write block 4057 len 24576 data: 4 4 5 6 7 8
23217 006/300 Read block 1316 len 16384
23218 007/300 Read block 4576 len 12288
23219 008/300 Flush
23220 009/300 Flush
23221 010/300 Write block 4966 len 4096 data: 6
23222 011/300 Write block 3771 len 8192 data: 5 5
23223 012/300 Read block 2514 len 16384
23224 013/300 Read block 4446 len 16384
23225 014/300 Read block 4979 len 20480
23226 015/300 Read block 3463 len 40960
23227 016/300 Read block 3210 len 20480
23228 017/300 Write block 2629 len 36864 data: 4 4 3 4 4 4 4 4 4
23229 018/300 Write block 1495 len 12288 data: 3 2 2
23230 019/300 Read block 2012 len 4096
23231 020/300 Read block 3847 len 36864
23232 021/300 Write block 4435 len 32768 data: 4 6 6 6 6 5 6 7
23233 022/300 Read block 1981 len 40960
23234 023/300 Write block 655 len 8192 data: 4 4
23235 024/300 Read block 2223 len 32768
23236 025/300 Read block 3826 len 40960
23237 026/300 Read block 1176 len 12288
23238 027/300 Flush
23239 028/300 Read block 2956 len 24576
23240 029/300 Read block 4016 len 8192
23241 030/300 Write block 4232 len 12288 data: 4 4 5
23242 031/300 Read block 3221 len 32768
23243 032/300 Read block 1536 len 20480
23244 033/300 Read block 1068 len 32768
23245 034/300 Write block 2548 len 16384 data: 3 3 3 4
23246 035/300 Read block 835 len 24576
23247 036/300 Read block 553 len 20480
23248 037/300 Read block 1537 len 28672
23249 038/300 Write block 3528 len 32768 data: 4 4 5 6 5 5 5 4
23250 039/300 Read block 2594 len 4096
23251 040/300 Write block 4293 len 32768 data: 5 4 4 4 3 3 3 3
23252 041/300 Read block 1811 len 8192
23253 042/300 Flush
23254 043/300 Read block 4372 len 4096
23255 044/300 Read block 3118 len 12288
23256 045/300 Flush
23257 046/300 Write block 466 len 12288 data: 5 4 4
23258 047/300 Read block 2466 len 28672
23259 048/300 Read block 1227 len 16384
23260 049/300 Flush
23261 050/300 Write block 75 len 40960 data: 3 4 4 5 5 4 3 3 3 3
23262 051/300 Write block 1395 len 20480 data: 3 4 4 3 2
23263 052/300 Read block 2885 len 36864
23264 053/300 Write block 2593 len 32768 data: 5 4 5 6 6 5 5 4
23265 054/300 Write block 2699 len 4096 data: 3
23266 055/300 Read block 2421 len 28672
23267 056/300 Flush
23268 057/300 Read block 883 len 16384
23269 058/300 Write block 3424 len 36864 data: 3 3 3 3 3 4 4 4 3
23270 059/300 Read block 1999 len 4096
23271 060/300 Read block 1310 len 32768
23272 061/300 Read block 4349 len 40960
23273 062/300 Read block 3597 len 20480
23274 063/300 Write block 3398 len 40960 data: 7 6 5 5 5 5 7 6 6 5
23275 064/300 Write block 4960 len 28672 data: 5 5 5 5 6 8 7
23276 065/300 Read block 2804 len 16384
23277 066/300 Write block 471 len 8192 data: 4 4
23278 067/300 Write block 4936 len 36864 data: 4 5 5 5 3 3 3 4 4
23279 068/300 Read block 674 len 4096
23280 069/300 Write block 711 len 32768 data: 6 6 5 5 5 6 7 6
23281 070/300 Write block 4030 len 16384 data: 7 5 5 4
23282 071/300 Write block 1812 len 20480 data: 6 6 5 5 4
23283 072/300 Read block 1794 len 36864
23284 073/300 Write block 577 len 32768 data: 5 4 3 4 3 4 4 3
23285 074/300 Read block 52 len 20480
23286 075/300 Flush
23287 076/300 Write block 711 len 16384 data: 7 7 6 6
23288 077/300 Flush
23289 078/300 Read block 3770 len 36864
23290 079/300 Read block 45 len 24576
23291 080/300 Write block 1076 len 36864 data: 4 4 5 6 6 4 4 5 5
23292 081/300 Read block 199 len 36864
23293 082/300 Write block 2490 len 4096 data: 6
23294 083/300 Read block 713 len 8192
23295 084/300 Read block 3113 len 24576
23296 085/300 Read block 2980 len 24576
23297 086/300 Flush
23298 087/300 Read block 1481 len 36864
23299 088/300 Write block 1785 len 40960 data: 4 4 5 5 2 5 5 4 5 3
23300 089/300 Write block 4658 len 12288 data: 3 4 4
23301 090/300 Read block 2746 len 20480
23302 091/300 Write block 4922 len 24576 data: 5 5 6 8 8 8
23303 092/300 Flush
23304 093/300 Write block 3931 len 4096 data: 7
23305 094/300 Write block 2076 len 36864 data: 4 5 6 5 5 5 5 4 5
23306 095/300 Write block 1956 len 24576 data: 5 4 5 3 4 4
23307 096/300 Read block 1688 len 20480
23308 097/300 Write block 1166 len 20480 data: 4 4 6 5 5
23309 098/300 Read block 4349 len 8192
23310 099/300 Write block 4314 len 24576 data: 7 5 5 4 4 4
23311 100/300 Read block 2518 len 12288
23312 101/300 Write block 1902 len 4096 data: 2
23313 102/300 Read block 3652 len 8192
23314 103/300 Read block 1273 len 20480
23315 104/300 Write block 4744 len 20480 data: 3 3 4 5 5
23316 105/300 Write block 4232 len 36864 data: 5 5 6 4 4 4 4 5 5
23317 106/300 Write block 3569 len 20480 data: 6 6 6 6 6
23318 107/300 Read block 2214 len 8192
23319 108/300 Write block 3405 len 16384 data: 7 7 6 5
23320 109/300 Write block 3582 len 28672 data: 4 4 5 6 5 5 3
23321 110/300 Write block 4524 len 28672 data: 2 2 2 3 4 4 4
23322 111/300 Read block 204 len 28672
23323 112/300 Flush
23324 113/300 Read block 1895 len 24576
23325 114/300 Read block 4617 len 24576
23326 115/300 Read block 203 len 40960
23327 116/300 Flush
23328 117/300 Read block 924 len 40960
23329 118/300 Write block 3169 len 40960 data: 5 5 5 4 4 3 3 4 4 4
23330 119/300 Write block 2483 len 36864 data: 6 7 8 9 8 8 7 7 7
23331 120/300 Read block 4142 len 12288
23332 121/300 Write block 4485 len 36864 data: 4 4 5 5 6 6 5 5 5
23333 122/300 Read block 4793 len 32768
23334 123/300 Read block 1365 len 24576
23335 124/300 Read block 2188 len 8192
23336 125/300 Read block 2335 len 36864
23337 126/300 Read block 4246 len 32768
23338 127/300 Write block 2704 len 36864 data: 4 6 5 5 6 8 7 7 7
23339 128/300 Read block 952 len 4096
23340 129/300 Write block 2353 len 36864 data: 8 7 5 6 5 4 3 3 2
23341 130/300 Read block 2425 len 16384
23342 131/300 Read block 222 len 12288
23343 132/300 Flush
23344 133/300 Read block 1606 len 40960
23345 134/300 Flush
23346 135/300 Read block 3446 len 28672
23347 136/300 Write block 2105 len 4096 data: 6
23348 137/300 Read block 667 len 24576
23349 138/300 Write block 479 len 32768 data: 4 3 5 6 4 3 4 4
23350 139/300 Write block 2631 len 4096 data: 4
23351 140/300 Write block 4473 len 40960 data: 6 7 9 8 8 7 4 4 3 4
23352 141/300 Write block 3635 len 36864 data: 4 3 3 2 2 3 5 5 5
23353 142/300 Write block 1911 len 4096 data: 5
23354 143/300 Write block 171 len 12288 data: 5 7 7
23355 144/300 Read block 2079 len 12288
23356 145/300 Write block 3404 len 16384 data: 8 8 8 7
23357 146/300 Write block 2749 len 40960 data: 8 5 3 3 2 3 3 3 3 6
23358 147/300 Read block 4050 len 20480
23359 148/300 Read block 1584 len 12288
23360 149/300 Write block 3866 len 20480 data: 3 3 3 2 3
23361 150/300 Read block 1118 len 16384
23362 151/300 Write block 1045 len 20480 data: 5 5 5 7 6
23363 152/300 Read block 4459 len 20480
23364 153/300 Read block 4534 len 32768
23365 154/300 Read block 4966 len 40960
23366 155/300 Read block 935 len 28672
23367 156/300 Write block 2766 len 24576 data: 3 3 4 4 4 6
23368 157/300 Write block 1103 len 8192 data: 8 6
23369 158/300 Write block 4485 len 36864 data: 5 5 6 6 7 7 6 6 6
23370 159/300 Write block 781 len 16384 data: 4 4 3 4
23371 160/300 Read block 283 len 24576
23372 161/300 Read block 356 len 16384
23373 162/300 Read block 3128 len 12288
23374 163/300 Read block 1589 len 12288
23375 164/300 Write block 4145 len 32768 data: 5 5 6 5 5 4 4 4
23376 165/300 Write block 2056 len 20480 data: 5 6 7 7 6
23377 166/300 Read block 3698 len 20480
23378 167/300 Write block 939 len 8192 data: 3 3
23379 168/300 Read block 892 len 4096
23380 169/300 Write block 354 len 4096 data: 8
23381 170/300 Write block 3539 len 16384 data: 5 6 6 6
23382 171/300 Write block 164 len 36864 data: 5 5 4 4 4 5 6 6 8
23383 172/300 Write block 166 len 24576 data: 5 5 5 6 7 7
23384 173/300 Write block 2113 len 28672 data: 3 3 3 4 4 4 4
23385 174/300 Read block 4738 len 4096
23386 175/300 Write block 2686 len 12288 data: 4 4 4
23387 176/300 Read block 4139 len 28672
23388 177/300 Read block 915 len 12288
23389 178/300 Write block 4752 len 36864 data: 5 5 5 5 3 4 4 4 4
23390 179/300 Read block 2800 len 28672
23391 180/300 Read block 2410 len 40960
23392 181/300 Write block 1866 len 32768 data: 3 3 4 4 6 5 5 5
23393 182/300 Write block 1281 len 36864 data: 5 7 6 7 7 6 5 5 6
23394 183/300 Write block 2032 len 20480 data: 5 5 7 6 6
23395 184/300 Read block 4501 len 16384
23396 185/300 Read block 2521 len 24576
23397 186/300 Write block 2254 len 12288 data: 8 5 5
23398 187/300 Read block 3799 len 32768
23399 188/300 Write block 85 len 16384 data: 3 4 4 5
23400 189/300 Write block 3598 len 32768 data: 10 10 9 7 5 3 3 4
23401 190/300 Flush
23402 191/300 Read block 4753 len 28672
23403 192/300 Write block 2734 len 28672 data: 6 6 7 4 5 6 6
23404 193/300 Read block 4056 len 12288
23405 194/300 Read block 216 len 32768
23406 195/300 Flush
23407 196/300 Write block 3221 len 32768 data: 5 4 5 7 5 7 8 10
23408 197/300 Flush
23409 198/300 Read block 743 len 24576
23410 199/300 Write block 4631 len 8192 data: 2 3
23411 200/300 Read block 3211 len 40960
23412 201/300 Write block 4614 len 40960 data: 4 4 5 5 6 7 5 4 4 4
23413 202/300 Write block 3281 len 24576 data: 6 6 6 6 7 7
23414 203/300 Write block 3745 len 16384 data: 7 5 4 4
23415 204/300 Write block 2581 len 36864 data: 3 2 2 2 3 3 4 4 5
23416 205/300 Write block 4166 len 12288 data: 3 3 3
23417 206/300 Flush
23418 207/300 Write block 3728 len 12288 data: 5 7 8
23419 208/300 Write block 269 len 8192 data: 4 4
23420 209/300 Write block 2497 len 20480 data: 4 4 4 7 6
23421 210/300 Flush
23422 211/300 Write block 1539 len 4096 data: 6
23423 212/300 Read block 2369 len 20480
23424 213/300 Read block 4630 len 4096
23425 214/300 Read block 2919 len 12288
23426 215/300 Write block 2859 len 20480 data: 5 6 6 4 5
23427 216/300 Read block 3614 len 36864
23428 217/300 Write block 1134 len 32768 data: 7 8 5 4 4 4 5 5
23429 218/300 Read block 3779 len 40960
23430 219/300 Write block 1392 len 16384 data: 2 3 3 4
23431 220/300 Write block 2566 len 28672 data: 4 5 3 3 3 3 3
23432 221/300 Read block 771 len 8192
23433 222/300 Read block 731 len 36864
23434 223/300 Read block 2564 len 8192
23435 224/300 Read block 4659 len 4096
23436 225/300 Read block 2280 len 32768
23437 226/300 Write block 3908 len 24576 data: 4 3 3 3 3 2
23438 227/300 Write block 761 len 4096 data: 4
23439 228/300 Write block 2110 len 16384 data: 4 4 3 4
23440 229/300 Write block 1749 len 8192 data: 5 5
23441 230/300 Write block 3642 len 32768 data: 6 6 5 5 5 7 7 5
23442 231/300 Write block 1385 len 32768 data: 4 4 4 3 2 2 2 3
23443 232/300 Read block 1673 len 36864
23444 233/300 Write block 4982 len 32768 data: 7 7 6 4 4 5 7 9
23445 234/300 Read block 2406 len 32768
23446 235/300 Write block 1082 len 20480 data: 5 6 6 4 5
23447 236/300 Write block 4627 len 20480 data: 4 4 4 2 3
23448 237/300 Write block 416 len 4096 data: 3
23449 238/300 Read block 3625 len 32768
23450 239/300 Write block 3680 len 12288 data: 6 7 8
23451 240/300 Write block 4419 len 40960 data: 2 2 4 4 4 4 4 3 3 3
23452 241/300 Read block 3554 len 8192
23453 242/300 Read block 280 len 32768
23454 243/300 Write block 74 len 28672 data: 3 4 5 5 6 6 5
23455 244/300 Read block 1457 len 40960
23456 245/300 Write block 1946 len 20480 data: 5 4 4 2 2
23457 246/300 Write block 1673 len 4096 data: 7
23458 247/300 Flush
23459 248/300 Read block 1555 len 40960
23460 249/300 Flush
23461 250/300 Read block 1589 len 12288
23462 251/300 Write block 3012 len 28672 data: 6 6 5 6 5 4 6
23463 252/300 Write block 4547 len 8192 data: 6 5
23464 253/300 Write block 4392 len 28672 data: 4 5 5 4 6 6 7
23465 254/300 Write block 1788 len 16384 data: 6 3 6 6
23466 255/300 Write block 1268 len 28672 data: 4 4 5 4 5 6 7
23467 256/300 Write block 4526 len 20480 data: 3 4 5 5 5
23468 257/300 Write block 3285 len 36864 data: 8 8 7 7 5 6 6 5 6
23469 258/300 Read block 535 len 4096
23470 259/300 Write block 1464 len 20480 data: 4 3 3 3 4
23471 260/300 Write block 4232 len 24576 data: 6 6 7 5 5 5
23472 261/300 Write block 2923 len 28672 data: 5 4 5 4 4 5 5
23473 262/300 Write block 3853 len 28672 data: 4 5 5 6 8 7 6
23474 263/300 Write block 2975 len 4096 data: 4
23475 264/300 Write block 3881 len 40960 data: 3 4 4 4 4 4 4 4 4 5
23476 265/300 Read block 4911 len 12288
23477 266/300 Read block 1378 len 28672
23478 267/300 Flush
23479 268/300 Read block 4640 len 12288
23480 269/300 Write block 3696 len 20480 data: 5 5 6 8 8
23481 270/300 Read block 3226 len 36864
23482 271/300 Read block 4799 len 4096
23483 272/300 Write block 3907 len 32768 data: 5 5 4 4 4 4 3 3
23484 273/300 Read block 1332 len 40960
23485 274/300 Read block 2549 len 16384
23486 275/300 Read block 4334 len 4096
23487 276/300 Write block 3609 len 16384 data: 4 5 5 5
23488 277/300 Read block 4851 len 12288
23489 278/300 Read block 723 len 20480
23490 279/300 Read block 3655 len 20480
23491 280/300 Write block 707 len 20480 data: 3 4 5 5 8
23492 281/300 Write block 4848 len 20480 data: 5 4 4 3 2
23493 282/300 Read block 37 len 16384
23494 283/300 Flush
23495 284/300 Read block 2763 len 12288
23496 285/300 Write block 1091 len 16384 data: 5 6 6 5
23497 286/300 Read block 1896 len 32768
23498 287/300 Read block 3349 len 40960
23499 288/300 Write block 4405 len 8192 data: 5 4
23500 289/300 Write block 4659 len 12288 data: 5 5 5
23501 290/300 Read block 4965 len 4096
23502 291/300 Read block 4875 len 32768
23503 292/300 Read block 1936 len 12288
23504 293/300 Flush
23505 294/300 Read block 4718 len 12288
23506 295/300 Write block 1948 len 36864 data: 5 3 3 2 3 4 4 5 6
23507 296/300 Write block 3328 len 20480 data: 4 4 4 5 3
23508 297/300 Read block 1046 len 20480
23509 298/300 Write block 1836 len 12288 data: 6 5 5
23510 299/300 Read block 4924 len 40960
23511 300/300 Write block 2494 len 4096 data: 4
23512 301/300 Read block 4106 len 24576
23513 Main task has work to do, go find it
23514 got dsc Start(1)
23515 start 1
23516 Nov 26 22:09:29.537 INFO request completed, latency_us: 191, response_code: 204, uri: /start/cid/1, method: POST, req_id: 00e07288-0191-41c4-86fe-a73516d541b8, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
23517 Replay: started 1, returned:Ok(())
23518 ----------------------------------------------------------------
23519 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
23520 States: Active Offline Active
23521 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
23522 7039 Acked Read 8 Done Sent Done false
23523 7040 Acked Read 6 Done Sent Done false
23524 7041 Acked Read 6 Done Sent Done false
23525 7042 Acked Read 1 Done Sent Done false
23526 7043 Acked Write 6 Done Sent Done false
23527 7044 Acked Read 4 Done Sent Done false
23528 7045 Acked Read 3 Done Sent Done false
23529 7046 Acked Flush 0 Done Sent Done false
23530 7047 Acked Flush 0 Done Sent Done false
23531 7048 Acked Write 1 Done Sent Done false
23532 7049 Acked Write 2 Done Sent Done false
23533 7050 Acked Read 4 Done Sent Done false
23534 7051 Acked Read 4 Done Sent Done false
23535 7052 Acked Read 5 Done Sent Done false
23536 7053 Acked Read 10 Done Sent Done false
23537 7054 Acked Read 5 Done Sent Done false
23538 7055 Acked Write 9 Done Sent Done false
23539 7056 Acked Write 3 Done Sent Done false
23540 7057 Acked Read 1 Done Sent Done false
23541 7058 Acked Read 9 Done Sent Done false
23542 7059 Acked Write 8 Done Sent Done false
23543 7060 Acked Read 10 Done Sent Done false
23544 7061 Acked Write 2 Done Sent Done false
23545 7062 Acked Read 8 Done Sent Done false
23546 7063 Acked Read 10 Done Sent Done false
23547 7064 Acked Read 3 Done Sent Done false
23548 7065 Acked Flush 0 Done Sent Done false
23549 7066 Acked Read 6 Done Sent Done false
23550 7067 Acked Read 2 Done Sent Done false
23551 7068 Acked Write 3 Done Sent Done false
23552 7069 Acked Read 8 Done Sent Done false
23553 7070 Acked Read 5 Done Sent Done false
23554 7071 Acked Read 8 Done Sent Done false
23555 7072 Acked Write 4 Done Sent Done false
23556 7073 Acked Read 6 Done Sent Done false
23557 7074 Acked Read 5 Done Sent Done false
23558 7075 Acked Read 7 Done Sent Done false
23559 7076 Acked Write 8 Done Sent Done false
23560 7077 Acked Read 1 Done Sent Done false
23561 7078 Acked Write 8 Done Sent Done false
23562 7079 Acked Read 2 Done Sent Done false
23563 7080 Acked Flush 0 Done Sent Done false
23564 7081 Acked Read 1 Done Sent Done false
23565 7082 Acked Read 3 Done Sent Done false
23566 7083 Acked Flush 0 Done Sent Done false
23567 7084 Acked Write 3 Done Sent Done false
23568 7085 Acked Read 7 Done Sent Done false
23569 7086 Acked Read 4 Done Sent Done false
23570 7087 Acked Flush 0 Done Sent Done false
23571 7088 Acked Write 10 Done Sent Done false
23572 7089 Acked Write 5 Done Sent Done false
23573 7090 Acked Read 9 Done Sent Done false
23574 7091 Acked Write 8 Done Sent Done false
23575 7092 Acked Write 1 Done Sent Done false
23576 7093 Acked Read 7 Done Sent Done false
23577 7094 Acked Flush 0 Done Sent Done false
23578 7095 Acked Read 4 Done Sent Done false
23579 7096 Acked Write 9 Done Sent Done false
23580 7097 Acked Read 1 Done Sent Done false
23581 7098 Acked Read 8 Done Sent Done false
23582 7099 Acked Read 10 Done Sent Done false
23583 7100 Acked Read 5 Done Sent Done false
23584 7101 Acked Write 10 Done Sent Done false
23585 7102 Acked Write 7 Done Sent Done false
23586 7103 Acked Read 4 Done Sent Done false
23587 7104 Acked Write 2 Done Sent Done false
23588 7105 Acked Write 9 Done Sent Done false
23589 7106 Acked Read 1 Done Sent Done false
23590 7107 Acked Write 8 Done Sent Done false
23591 7108 Acked Write 4 Done Sent Done false
23592 7109 Acked Write 5 Done Sent Done false
23593 7110 Acked Read 9 Done Sent Done false
23594 7111 Acked Write 8 Done Sent Done false
23595 7112 Acked Read 5 Done Sent Done false
23596 7113 Acked Flush 0 Done Sent Done false
23597 7114 Acked Write 4 Done Sent Done false
23598 7115 Acked Flush 0 Done Sent Done false
23599 7116 Acked Read 9 Done Sent Done false
23600 7117 Acked Read 6 Done Sent Done false
23601 7118 Acked Write 9 Done Sent Done false
23602 7119 Acked Read 9 Done Sent Done false
23603 7120 Acked Write 1 Done Sent Done false
23604 7121 Acked Read 2 Done Sent Done false
23605 7122 Acked Read 6 Done Sent Done false
23606 7123 Acked Read 6 Done Sent Done false
23607 7124 Acked Flush 0 Done Sent Done false
23608 7125 Acked Read 9 Done Sent Done false
23609 7126 Acked Write 10 Done Sent Done false
23610 7127 Acked Write 3 Done Sent Done false
23611 7128 Acked Read 5 Done Sent Done false
23612 7129 Acked Write 6 Done Sent Done false
23613 7130 Acked Flush 0 Done Sent Done false
23614 7131 Acked Write 1 Done Sent Done false
23615 7132 Acked Write 9 Done Sent Done false
23616 7133 Acked Write 6 Done Sent Done false
23617 7134 Acked Read 5 Done Sent Done false
23618 7135 Acked Write 5 Done Sent Done false
23619 7136 Acked Read 2 Done Sent Done false
23620 7137 Acked Write 6 Done Sent Done false
23621 7138 Acked Read 3 Done Sent Done false
23622 7139 Acked Write 1 Done Sent Done false
23623 7140 Acked Read 2 Done Sent Done false
23624 7141 Acked Read 5 Done Sent Done false
23625 7142 Acked Write 5 Done Sent Done false
23626 7143 Acked Write 9 Done Sent Done false
23627 7144 Acked Write 5 Done Sent Done false
23628 7145 Acked Read 2 Done Sent Done false
23629 7146 Acked Write 4 Done Sent Done false
23630 7147 Acked Write 7 Done Sent Done false
23631 7148 Acked Write 7 Done Sent Done false
23632 7149 Acked Read 7 Done Sent Done false
23633 7150 Acked Flush 0 Done Sent Done false
23634 7151 Acked Read 6 Done Sent Done false
23635 7152 Acked Read 6 Done Sent Done false
23636 7153 Acked Read 10 Done Sent Done false
23637 7154 Acked Flush 0 Done Sent Done false
23638 7155 Acked Read 10 Done Sent Done false
23639 7156 Acked Write 10 Done Sent Done false
23640 7157 Acked Write 9 Done Sent Done false
23641 7158 Acked Read 3 Done Sent Done false
23642 7159 Acked Write 9 Done Sent Done false
23643 7160 Acked Read 8 Done Sent Done false
23644 7161 Acked Read 6 Done Sent Done false
23645 7162 Acked Read 2 Done Sent Done false
23646 7163 Acked Read 9 Done Sent Done false
23647 7164 Acked Read 8 Done Sent Done false
23648 7165 Acked Write 9 Done Sent Done false
23649 7166 Acked Read 1 Done Sent Done false
23650 7167 Acked Write 9 Done Sent Done false
23651 7168 Acked Read 4 Done Sent Done false
23652 7169 Acked Read 3 Done Sent Done false
23653 7170 Acked Flush 0 Done Sent Done false
23654 7171 Acked Read 10 Done Sent Done false
23655 7172 Acked Flush 0 Done Sent Done false
23656 7173 Acked Read 7 Done Sent Done false
23657 7174 Acked Write 1 Done Sent Done false
23658 7175 Acked Read 6 Done Sent Done false
23659 7176 Acked Write 8 Done Sent Done false
23660 7177 Acked Write 1 Done Sent Done false
23661 7178 Acked Write 10 Done Sent Done false
23662 7179 Acked Write 9 Done Sent Done false
23663 7180 Acked Write 1 Done Sent Done false
23664 7181 Acked Write 3 Done Sent Done false
23665 7182 Acked Read 3 Done Sent Done false
23666 7183 Acked Write 4 Done Sent Done false
23667 7184 Acked Write 10 Done Sent Done false
23668 7185 Acked Read 5 Done Sent Done false
23669 7186 Acked Read 3 Done Sent Done false
23670 7187 Acked Write 5 Done Sent Done false
23671 7188 Acked Read 4 Done Sent Done false
23672 7189 Acked Write 5 Done Sent Done false
23673 7190 Acked Read 5 Done Sent Done false
23674 7191 Acked Read 8 Done Sent Done false
23675 7192 Acked Read 10 Done Sent Done false
23676 7193 Acked Read 7 Done Sent Done false
23677 7194 Acked Write 6 Done Sent Done false
23678 7195 Acked Write 2 Done Sent Done false
23679 7196 Acked Write 9 Done Sent Done false
23680 7197 Acked Write 4 Done Sent Done false
23681 7198 Acked Read 6 Done Sent Done false
23682 7199 Acked Read 4 Done Sent Done false
23683 7200 Acked Read 3 Done Sent Done false
23684 7201 Acked Read 3 Done Sent Done false
23685 7202 Acked Write 8 Done Sent Done false
23686 7203 Acked Write 5 Done Sent Done false
23687 7204 Acked Read 5 Done Sent Done false
23688 7205 Acked Write 2 Done Sent Done false
23689 7206 Acked Read 1 Done Sent Done false
23690 7207 Acked Write 1 Done Sent Done false
23691 7208 Acked Write 4 Done Sent Done false
23692 7209 Acked Write 9 Done Sent Done false
23693 7210 Acked Write 6 Done Sent Done false
23694 7211 Acked Write 7 Done Sent Done false
23695 7212 Acked Read 1 Done Sent Done false
23696 7213 Acked Write 3 Done Sent Done false
23697 7214 Acked Read 7 Done Sent Done false
23698 7215 Acked Read 3 Done Sent Done false
23699 7216 Acked Write 9 Done Sent Done false
23700 7217 Acked Read 7 Done Sent Done false
23701 7218 Acked Read 10 Done Sent Done false
23702 7219 Acked Write 8 Done Sent Done false
23703 7220 Acked Write 9 Done Sent Done false
23704 7221 Acked Write 5 Done Sent Done false
23705 7222 Acked Read 4 Done Sent Done false
23706 7223 Acked Read 6 Done Sent Done false
23707 7224 Acked Write 3 Done Sent Done false
23708 7225 Acked Read 8 Done Sent Done false
23709 7226 Acked Write 4 Done Sent Done false
23710 7227 Acked Write 8 Done Sent Done false
23711 7228 Acked Flush 0 Done Sent Done false
23712 7229 Acked Read 7 Done Sent Done false
23713 7230 Acked Write 7 Done Sent Done false
23714 7231 Acked Read 3 Done Sent Done false
23715 7232 Acked Read 8 Done Sent Done false
23716 7233 Acked Flush 0 Done Sent Done false
23717 7234 Acked Write 8 Done Sent Done false
23718 7235 Acked Flush 0 Done Sent Done false
23719 7236 Acked Read 6 Done Sent Done false
23720 7237 Acked Write 2 Done Sent Done false
23721 7238 Acked Read 10 Done Sent Done false
23722 7239 Acked Write 10 Done Sent Done false
23723 7240 Acked Write 6 Done Sent Done false
23724 7241 Acked Write 4 Done Sent Done false
23725 7242 Acked Write 9 Done Sent Done false
23726 7243 Acked Write 3 Done Sent Done false
23727 7244 Acked Flush 0 Done Sent Done false
23728 7245 Acked Write 3 Done Sent Done false
23729 7246 Acked Write 2 Done Sent Done false
23730 7247 Acked Write 5 Done Sent Done false
23731 7248 Acked Flush 0 Done Sent Done false
23732 7249 Acked Write 1 Done Sent Done false
23733 7250 Acked Read 5 Done Sent Done false
23734 7251 Acked Read 1 Done Sent Done false
23735 7252 Acked Read 3 Done Sent Done false
23736 7253 Acked Write 5 Done Sent Done false
23737 7254 Acked Read 9 Done Sent Done false
23738 7255 Acked Write 8 Done Sent Done false
23739 7256 Acked Read 10 Done Sent Done false
23740 7257 Acked Write 4 Done Sent Done false
23741 7258 Acked Write 7 Done Sent Done false
23742 7259 Acked Read 2 Done Sent Done false
23743 7260 Acked Read 9 Done Sent Done false
23744 7261 Acked Read 2 Done Sent Done false
23745 7262 Acked Read 1 Done Sent Done false
23746 7263 Acked Read 8 Done Sent Done false
23747 7264 Acked Write 6 Done Sent Done false
23748 7265 Acked Write 1 Done Sent Done false
23749 7266 Acked Write 4 Done Sent Done false
23750 7267 Acked Write 2 Done Sent Done false
23751 7268 Acked Write 8 Done Sent Done false
23752 7269 Acked Write 8 Done Sent Done false
23753 7270 Acked Read 9 Done Sent Done false
23754 7271 Acked Write 8 Done Sent Done false
23755 7272 Acked Read 8 Done Sent Done false
23756 7273 Acked Write 5 Done Sent Done false
23757 7274 Acked Write 5 Done Sent Done false
23758 7275 Acked Write 1 Done Sent Done false
23759 7276 Acked Read 8 Done Sent Done false
23760 7277 Acked Write 3 Done Sent Done false
23761 7278 Acked Write 10 Done Sent Done false
23762 7279 Acked Read 2 Done Sent Done false
23763 7280 Acked Read 8 Done Sent Done false
23764 7281 Acked Write 7 Done Sent Done false
23765 7282 Acked Read 10 Done Sent Done false
23766 7283 Acked Write 5 Done Sent Done false
23767 7284 Acked Write 1 Done Sent Done false
23768 7285 Acked Flush 0 Done Sent Done false
23769 7286 Acked Read 10 Done Sent Done false
23770 7287 Acked Flush 0 Done Sent Done false
23771 7288 Acked Read 3 Done Sent Done false
23772 7289 Acked Write 7 Done Sent Done false
23773 7290 Acked Write 2 Done Sent Done false
23774 7291 Acked Write 7 Done Sent Done false
23775 7292 Acked Write 4 Done Sent Done false
23776 7293 Acked Write 7 Done Sent Done false
23777 7294 Acked Write 5 Done Sent Done false
23778 7295 Acked Write 9 Done Sent Done false
23779 7296 Acked Read 1 Done Sent Done false
23780 7297 Acked Write 5 Done Sent Done false
23781 7298 Acked Write 6 Done Sent Done false
23782 7299 Acked Write 7 Done Sent Done false
23783 7300 Acked Write 7 Done Sent Done false
23784 7301 Acked Write 1 Done Sent Done false
23785 7302 Acked Write 10 Done Sent Done false
23786 7303 Acked Read 3 Done Sent Done false
23787 7304 Acked Read 7 Done Sent Done false
23788 7305 Acked Flush 0 Done Sent Done false
23789 7306 Acked Read 3 Done Sent Done false
23790 7307 Acked Write 5 Done Sent Done false
23791 7308 Acked Read 9 Done Sent Done false
23792 7309 Acked Read 1 Done Sent Done false
23793 7310 Acked Write 8 Done Sent Done false
23794 7311 Acked Read 10 Done Sent Done false
23795 7312 Acked Read 4 Done Sent Done false
23796 7313 Acked Read 1 Done Sent Done false
23797 7314 Acked Write 4 Done Sent Done false
23798 7315 Acked Read 3 Done Sent Done false
23799 7316 Acked Read 5 Done Sent Done false
23800 7317 Acked Read 5 Done Sent Done false
23801 7318 Acked Write 5 Done Sent Done false
23802 7319 Acked Write 5 Done Sent Done false
23803 7320 Acked Read 4 Done Sent Done false
23804 7321 Acked Flush 0 Done Sent Done false
23805 7322 Acked Read 3 Done Sent Done false
23806 7323 Acked Write 4 Done Sent Done false
23807 7324 Acked Read 8 Done Sent Done false
23808 7325 Acked Read 10 Done Sent Done false
23809 7326 Acked Write 2 Done Sent Done false
23810 7327 Acked Write 3 Done Sent Done false
23811 7328 Acked Read 1 Done Sent Done false
23812 7329 Acked Read 8 Done Sent Done false
23813 7330 Acked Read 3 Done Sent Done false
23814 7331 Acked Flush 0 Done Sent Done false
23815 7332 Acked Read 3 Done Sent Done false
23816 7333 Acked Write 9 Done Sent Done false
23817 7334 Acked Write 5 Done Sent Done false
23818 7335 Acked Read 5 Done Sent Done false
23819 7336 Acked Write 3 Done Sent Done false
23820 7337 Acked Read 10 Done Sent Done false
23821 7338 Acked Write 1 Done Sent Done false
23822 7339 Acked Read 6 Done Sent Done false
23823 STATES DS:0 DS:1 DS:2 TOTAL
23824 Sent 0 301 0 301
23825 Done 301 0 301 602
23826 Skipped 0 0 0 0
23827 Error 0 0 0 0
23828 Last Flush: 7331 7038 7331
23829 Downstairs last five completed: 7038 7037 7036 7035 7034
23830 Upstairs last five completed: 7339 7338 7337 7336 7335
23831 CLIENT: Up:0 ds:301 act:2
23832 [8820] got start action
23833 [8820] I am going to restart
23834 Starting downstairs at port 8820
23835 Make output file at "/tmp/dsc/downstairs-8820.txt"
23836 [8820][1] reports Starting
23837 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1224)
23838 [8820][1] reports Running
23839 ----------------------------------------------------------------
23840 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
23841 States: Active Offline Active
23842 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
23843 7039 Acked Read 8 Done Sent Done false
23844 7040 Acked Read 6 Done Sent Done false
23845 7041 Acked Read 6 Done Sent Done false
23846 7042 Acked Read 1 Done Sent Done false
23847 7043 Acked Write 6 Done Sent Done false
23848 7044 Acked Read 4 Done Sent Done false
23849 7045 Acked Read 3 Done Sent Done false
23850 7046 Acked Flush 0 Done Sent Done false
23851 7047 Acked Flush 0 Done Sent Done false
23852 7048 Acked Write 1 Done Sent Done false
23853 7049 Acked Write 2 Done Sent Done false
23854 7050 Acked Read 4 Done Sent Done false
23855 7051 Acked Read 4 Done Sent Done false
23856 7052 Acked Read 5 Done Sent Done false
23857 7053 Acked Read 10 Done Sent Done false
23858 7054 Acked Read 5 Done Sent Done false
23859 7055 Acked Write 9 Done Sent Done false
23860 7056 Acked Write 3 Done Sent Done false
23861 7057 Acked Read 1 Done Sent Done false
23862 7058 Acked Read 9 Done Sent Done false
23863 7059 Acked Write 8 Done Sent Done false
23864 7060 Acked Read 10 Done Sent Done false
23865 7061 Acked Write 2 Done Sent Done false
23866 7062 Acked Read 8 Done Sent Done false
23867 7063 Acked Read 10 Done Sent Done false
23868 7064 Acked Read 3 Done Sent Done false
23869 7065 Acked Flush 0 Done Sent Done false
23870 7066 Acked Read 6 Done Sent Done false
23871 7067 Acked Read 2 Done Sent Done false
23872 7068 Acked Write 3 Done Sent Done false
23873 7069 Acked Read 8 Done Sent Done false
23874 7070 Acked Read 5 Done Sent Done false
23875 7071 Acked Read 8 Done Sent Done false
23876 7072 Acked Write 4 Done Sent Done false
23877 7073 Acked Read 6 Done Sent Done false
23878 7074 Acked Read 5 Done Sent Done false
23879 7075 Acked Read 7 Done Sent Done false
23880 7076 Acked Write 8 Done Sent Done false
23881 7077 Acked Read 1 Done Sent Done false
23882 7078 Acked Write 8 Done Sent Done false
23883 7079 Acked Read 2 Done Sent Done false
23884 7080 Acked Flush 0 Done Sent Done false
23885 7081 Acked Read 1 Done Sent Done false
23886 7082 Acked Read 3 Done Sent Done false
23887 7083 Acked Flush 0 Done Sent Done false
23888 7084 Acked Write 3 Done Sent Done false
23889 7085 Acked Read 7 Done Sent Done false
23890 7086 Acked Read 4 Done Sent Done false
23891 7087 Acked Flush 0 Done Sent Done false
23892 7088 Acked Write 10 Done Sent Done false
23893 7089 Acked Write 5 Done Sent Done false
23894 7090 Acked Read 9 Done Sent Done false
23895 7091 Acked Write 8 Done Sent Done false
23896 7092 Acked Write 1 Done Sent Done false
23897 7093 Acked Read 7 Done Sent Done false
23898 7094 Acked Flush 0 Done Sent Done false
23899 7095 Acked Read 4 Done Sent Done false
23900 7096 Acked Write 9 Done Sent Done false
23901 7097 Acked Read 1 Done Sent Done false
23902 7098 Acked Read 8 Done Sent Done false
23903 7099 Acked Read 10 Done Sent Done false
23904 7100 Acked Read 5 Done Sent Done false
23905 7101 Acked Write 10 Done Sent Done false
23906 7102 Acked Write 7 Done Sent Done false
23907 7103 Acked Read 4 Done Sent Done false
23908 7104 Acked Write 2 Done Sent Done false
23909 7105 Acked Write 9 Done Sent Done false
23910 7106 Acked Read 1 Done Sent Done false
23911 7107 Acked Write 8 Done Sent Done false
23912 7108 Acked Write 4 Done Sent Done false
23913 7109 Acked Write 5 Done Sent Done false
23914 7110 Acked Read 9 Done Sent Done false
23915 7111 Acked Write 8 Done Sent Done false
23916 7112 Acked Read 5 Done Sent Done false
23917 7113 Acked Flush 0 Done Sent Done false
23918 7114 Acked Write 4 Done Sent Done false
23919 7115 Acked Flush 0 Done Sent Done false
23920 7116 Acked Read 9 Done Sent Done false
23921 7117 Acked Read 6 Done Sent Done false
23922 7118 Acked Write 9 Done Sent Done false
23923 7119 Acked Read 9 Done Sent Done false
23924 7120 Acked Write 1 Done Sent Done false
23925 7121 Acked Read 2 Done Sent Done false
23926 7122 Acked Read 6 Done Sent Done false
23927 7123 Acked Read 6 Done Sent Done false
23928 7124 Acked Flush 0 Done Sent Done false
23929 7125 Acked Read 9 Done Sent Done false
23930 7126 Acked Write 10 Done Sent Done false
23931 7127 Acked Write 3 Done Sent Done false
23932 7128 Acked Read 5 Done Sent Done false
23933 7129 Acked Write 6 Done Sent Done false
23934 7130 Acked Flush 0 Done Sent Done false
23935 7131 Acked Write 1 Done Sent Done false
23936 7132 Acked Write 9 Done Sent Done false
23937 7133 Acked Write 6 Done Sent Done false
23938 7134 Acked Read 5 Done Sent Done false
23939 7135 Acked Write 5 Done Sent Done false
23940 7136 Acked Read 2 Done Sent Done false
23941 7137 Acked Write 6 Done Sent Done false
23942 7138 Acked Read 3 Done Sent Done false
23943 7139 Acked Write 1 Done Sent Done false
23944 7140 Acked Read 2 Done Sent Done false
23945 7141 Acked Read 5 Done Sent Done false
23946 7142 Acked Write 5 Done Sent Done false
23947 7143 Acked Write 9 Done Sent Done false
23948 7144 Acked Write 5 Done Sent Done false
23949 7145 Acked Read 2 Done Sent Done false
23950 7146 Acked Write 4 Done Sent Done false
23951 7147 Acked Write 7 Done Sent Done false
23952 7148 Acked Write 7 Done Sent Done false
23953 7149 Acked Read 7 Done Sent Done false
23954 7150 Acked Flush 0 Done Sent Done false
23955 7151 Acked Read 6 Done Sent Done false
23956 7152 Acked Read 6 Done Sent Done false
23957 7153 Acked Read 10 Done Sent Done false
23958 7154 Acked Flush 0 Done Sent Done false
23959 7155 Acked Read 10 Done Sent Done false
23960 7156 Acked Write 10 Done Sent Done false
23961 7157 Acked Write 9 Done Sent Done false
23962 7158 Acked Read 3 Done Sent Done false
23963 7159 Acked Write 9 Done Sent Done false
23964 7160 Acked Read 8 Done Sent Done false
23965 7161 Acked Read 6 Done Sent Done false
23966 7162 Acked Read 2 Done Sent Done false
23967 7163 Acked Read 9 Done Sent Done false
23968 7164 Acked Read 8 Done Sent Done false
23969 7165 Acked Write 9 Done Sent Done false
23970 7166 Acked Read 1 Done Sent Done false
23971 7167 Acked Write 9 Done Sent Done false
23972 7168 Acked Read 4 Done Sent Done false
23973 7169 Acked Read 3 Done Sent Done false
23974 7170 Acked Flush 0 Done Sent Done false
23975 7171 Acked Read 10 Done Sent Done false
23976 7172 Acked Flush 0 Done Sent Done false
23977 7173 Acked Read 7 Done Sent Done false
23978 7174 Acked Write 1 Done Sent Done false
23979 7175 Acked Read 6 Done Sent Done false
23980 7176 Acked Write 8 Done Sent Done false
23981 7177 Acked Write 1 Done Sent Done false
23982 7178 Acked Write 10 Done Sent Done false
23983 7179 Acked Write 9 Done Sent Done false
23984 7180 Acked Write 1 Done Sent Done false
23985 7181 Acked Write 3 Done Sent Done false
23986 7182 Acked Read 3 Done Sent Done false
23987 7183 Acked Write 4 Done Sent Done false
23988 7184 Acked Write 10 Done Sent Done false
23989 7185 Acked Read 5 Done Sent Done false
23990 7186 Acked Read 3 Done Sent Done false
23991 7187 Acked Write 5 Done Sent Done false
23992 7188 Acked Read 4 Done Sent Done false
23993 7189 Acked Write 5 Done Sent Done false
23994 7190 Acked Read 5 Done Sent Done false
23995 7191 Acked Read 8 Done Sent Done false
23996 7192 Acked Read 10 Done Sent Done false
23997 7193 Acked Read 7 Done Sent Done false
23998 7194 Acked Write 6 Done Sent Done false
23999 7195 Acked Write 2 Done Sent Done false
24000 7196 Acked Write 9 Done Sent Done false
24001 7197 Acked Write 4 Done Sent Done false
24002 7198 Acked Read 6 Done Sent Done false
24003 7199 Acked Read 4 Done Sent Done false
24004 7200 Acked Read 3 Done Sent Done false
24005 7201 Acked Read 3 Done Sent Done false
24006 7202 Acked Write 8 Done Sent Done false
24007 7203 Acked Write 5 Done Sent Done false
24008 7204 Acked Read 5 Done Sent Done false
24009 7205 Acked Write 2 Done Sent Done false
24010 7206 Acked Read 1 Done Sent Done false
24011 7207 Acked Write 1 Done Sent Done false
24012 7208 Acked Write 4 Done Sent Done false
24013 7209 Acked Write 9 Done Sent Done false
24014 7210 Acked Write 6 Done Sent Done false
24015 7211 Acked Write 7 Done Sent Done false
24016 7212 Acked Read 1 Done Sent Done false
24017 7213 Acked Write 3 Done Sent Done false
24018 7214 Acked Read 7 Done Sent Done false
24019 7215 Acked Read 3 Done Sent Done false
24020 7216 Acked Write 9 Done Sent Done false
24021 7217 Acked Read 7 Done Sent Done false
24022 7218 Acked Read 10 Done Sent Done false
24023 7219 Acked Write 8 Done Sent Done false
24024 7220 Acked Write 9 Done Sent Done false
24025 7221 Acked Write 5 Done Sent Done false
24026 7222 Acked Read 4 Done Sent Done false
24027 7223 Acked Read 6 Done Sent Done false
24028 7224 Acked Write 3 Done Sent Done false
24029 7225 Acked Read 8 Done Sent Done false
24030 7226 Acked Write 4 Done Sent Done false
24031 7227 Acked Write 8 Done Sent Done false
24032 7228 Acked Flush 0 Done Sent Done false
24033 7229 Acked Read 7 Done Sent Done false
24034 7230 Acked Write 7 Done Sent Done false
24035 7231 Acked Read 3 Done Sent Done false
24036 7232 Acked Read 8 Done Sent Done false
24037 7233 Acked Flush 0 Done Sent Done false
24038 7234 Acked Write 8 Done Sent Done false
24039 7235 Acked Flush 0 Done Sent Done false
24040 7236 Acked Read 6 Done Sent Done false
24041 7237 Acked Write 2 Done Sent Done false
24042 7238 Acked Read 10 Done Sent Done false
24043 7239 Acked Write 10 Done Sent Done false
24044 7240 Acked Write 6 Done Sent Done false
24045 7241 Acked Write 4 Done Sent Done false
24046 7242 Acked Write 9 Done Sent Done false
24047 7243 Acked Write 3 Done Sent Done false
24048 7244 Acked Flush 0 Done Sent Done false
24049 7245 Acked Write 3 Done Sent Done false
24050 7246 Acked Write 2 Done Sent Done false
24051 7247 Acked Write 5 Done Sent Done false
24052 7248 Acked Flush 0 Done Sent Done false
24053 7249 Acked Write 1 Done Sent Done false
24054 7250 Acked Read 5 Done Sent Done false
24055 7251 Acked Read 1 Done Sent Done false
24056 7252 Acked Read 3 Done Sent Done false
24057 7253 Acked Write 5 Done Sent Done false
24058 7254 Acked Read 9 Done Sent Done false
24059 7255 Acked Write 8 Done Sent Done false
24060 7256 Acked Read 10 Done Sent Done false
24061 7257 Acked Write 4 Done Sent Done false
24062 7258 Acked Write 7 Done Sent Done false
24063 7259 Acked Read 2 Done Sent Done false
24064 7260 Acked Read 9 Done Sent Done false
24065 7261 Acked Read 2 Done Sent Done false
24066 7262 Acked Read 1 Done Sent Done false
24067 7263 Acked Read 8 Done Sent Done false
24068 7264 Acked Write 6 Done Sent Done false
24069 7265 Acked Write 1 Done Sent Done false
24070 7266 Acked Write 4 Done Sent Done false
24071 7267 Acked Write 2 Done Sent Done false
24072 7268 Acked Write 8 Done Sent Done false
24073 7269 Acked Write 8 Done Sent Done false
24074 7270 Acked Read 9 Done Sent Done false
24075 7271 Acked Write 8 Done Sent Done false
24076 7272 Acked Read 8 Done Sent Done false
24077 7273 Acked Write 5 Done Sent Done false
24078 7274 Acked Write 5 Done Sent Done false
24079 7275 Acked Write 1 Done Sent Done false
24080 7276 Acked Read 8 Done Sent Done false
24081 7277 Acked Write 3 Done Sent Done false
24082 7278 Acked Write 10 Done Sent Done false
24083 7279 Acked Read 2 Done Sent Done false
24084 7280 Acked Read 8 Done Sent Done false
24085 7281 Acked Write 7 Done Sent Done false
24086 7282 Acked Read 10 Done Sent Done false
24087 7283 Acked Write 5 Done Sent Done false
24088 7284 Acked Write 1 Done Sent Done false
24089 7285 Acked Flush 0 Done Sent Done false
24090 7286 Acked Read 10 Done Sent Done false
24091 7287 Acked Flush 0 Done Sent Done false
24092 7288 Acked Read 3 Done Sent Done false
24093 7289 Acked Write 7 Done Sent Done false
24094 7290 Acked Write 2 Done Sent Done false
24095 7291 Acked Write 7 Done Sent Done false
24096 7292 Acked Write 4 Done Sent Done false
24097 7293 Acked Write 7 Done Sent Done false
24098 7294 Acked Write 5 Done Sent Done false
24099 7295 Acked Write 9 Done Sent Done false
24100 7296 Acked Read 1 Done Sent Done false
24101 7297 Acked Write 5 Done Sent Done false
24102 7298 Acked Write 6 Done Sent Done false
24103 7299 Acked Write 7 Done Sent Done false
24104 7300 Acked Write 7 Done Sent Done false
24105 7301 Acked Write 1 Done Sent Done false
24106 7302 Acked Write 10 Done Sent Done false
24107 7303 Acked Read 3 Done Sent Done false
24108 7304 Acked Read 7 Done Sent Done false
24109 7305 Acked Flush 0 Done Sent Done false
24110 7306 Acked Read 3 Done Sent Done false
24111 7307 Acked Write 5 Done Sent Done false
24112 7308 Acked Read 9 Done Sent Done false
24113 7309 Acked Read 1 Done Sent Done false
24114 7310 Acked Write 8 Done Sent Done false
24115 7311 Acked Read 10 Done Sent Done false
24116 7312 Acked Read 4 Done Sent Done false
24117 7313 Acked Read 1 Done Sent Done false
24118 7314 Acked Write 4 Done Sent Done false
24119 7315 Acked Read 3 Done Sent Done false
24120 7316 Acked Read 5 Done Sent Done false
24121 7317 Acked Read 5 Done Sent Done false
24122 7318 Acked Write 5 Done Sent Done false
24123 7319 Acked Write 5 Done Sent Done false
24124 7320 Acked Read 4 Done Sent Done false
24125 7321 Acked Flush 0 Done Sent Done false
24126 7322 Acked Read 3 Done Sent Done false
24127 7323 Acked Write 4 Done Sent Done false
24128 7324 Acked Read 8 Done Sent Done false
24129 7325 Acked Read 10 Done Sent Done false
24130 7326 Acked Write 2 Done Sent Done false
24131 7327 Acked Write 3 Done Sent Done false
24132 7328 Acked Read 1 Done Sent Done false
24133 7329 Acked Read 8 Done Sent Done false
24134 7330 Acked Read 3 Done Sent Done false
24135 7331 Acked Flush 0 Done Sent Done false
24136 7332 Acked Read 3 Done Sent Done false
24137 7333 Acked Write 9 Done Sent Done false
24138 7334 Acked Write 5 Done Sent Done false
24139 7335 Acked Read 5 Done Sent Done false
24140 7336 Acked Write 3 Done Sent Done false
24141 7337 Acked Read 10 Done Sent Done false
24142 7338 Acked Write 1 Done Sent Done false
24143 7339 Acked Read 6 Done Sent Done false
24144 7340 Acked Flush 0 Done Sent Done false
24145 STATES DS:0 DS:1 DS:2 TOTAL
24146 Sent 0 302 0 302
24147 Done 302 0 302 604
24148 Skipped 0 0 0 0
24149 Error 0 0 0 0
24150 Last Flush: 7340 7038 7340
24151 Downstairs last five completed: 7038 7037 7036 7035 7034
24152 Upstairs last five completed: 7340 7339 7338 7337 7336
24153 CLIENT: Up:0 ds:302 act:2
241542024-11-26T22:09:35.306ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
241552024-11-26T22:09:35.307ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:60217) = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
241562024-11-26T22:09:35.307ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
241572024-11-26T22:09:35.308ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
241582024-11-26T22:09:35.308ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
241592024-11-26T22:09:35.308ZINFOcrucible: send last flush ID to this DS: 7038 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
241602024-11-26T22:09:35.308ZINFOcrucible: Replied this last flush ID: 7038 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
241612024-11-26T22:09:35.308ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
241622024-11-26T22:09:35.308ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
241632024-11-26T22:09:35.308ZINFOcrucible: [1] client re-new 302 jobs since flush 7038 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
241642024-11-26T22:09:35.308ZINFOcrucible: [1] Replayed 302 jobs since flush: 7038 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
24165 ----------------------------------------------------------------
24166 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
24167 Downstairs last five completed: 7340 7339 7338 7337 7336
24168 Upstairs last five completed: 7340 7339 7338 7337 7336
24169 CLIENT: Up:0 ds:0 act:3
24170 Replay: All jobs finished
24171 Main task has work to do, go find it
24172 got dsc Stop(1)
24173 stop 1
24174 Nov 26 22:09:37.542 INFO request completed, latency_us: 370, response_code: 204[8820] Got stop action so:false kr:false
24175 , uri: /stop/cid/1, method: POST, req_id: bffdfcce-48f6-4af4-b99b-b82605f9f01a, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
241762024-11-26T22:09:37.543ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
241772024-11-26T22:09:37.543ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
241782024-11-26T22:09:37.543ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
241792024-11-26T22:09:37.543ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
24180 Nov 26 22:09:37.543 INFO request completed, latency_us: 129, response_code: 200, uri: /state/cid/1, method: {"msg":"[1] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-26T22:09:37.543706699Z","hostname":"w-01JDN7ZQ17ABQG15KHKN3F2D26","pid":1203,"client":"1","":"downstairs","session_id":"86a36c17-3718-484c-87fd-2b8a1dc2ab18"}
24181 GET, req_id: 1b421478-6f25-4d2a-ad02-92fb72000200, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
241822024-11-26T22:09:37.543ZINFOcrucible: client task is exiting = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
24183 [8820] Exited with: ExitStatus(unix_wait_status(9))
24184 [8820][1] reports Exit
24185 Nov 26 22:09:41.546 INFO request completed, latency_us: 206, response_code: 200, uri: /state/cid/1, method: GET, req_id: 504fd4bc-e92c-4c00-9ab8-161dd3bb2687, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
24186 001/300 Read block 1874 len 12288
24187 002/300 Write block 2300 len 8192 data: 5 5
24188 003/300 Write block 1200 len 4096 data: 5
24189 004/300 Write block 3224 len 32768 data: 8 6 8 9 11 11 12 9
24190 005/300 Write block 4804 len 40960 data: 5 4 6 6 7 8 7 5 5 5
24191 006/300 Write block 3019 len 32768 data: 4 3 5 5 5 5 4 3
24192 007/300 Write block 4858 len 16384 data: 4 4 5 7
24193 008/300 Flush
24194 009/300 Write block 2587 len 32768 data: 5 5 6 6 5 5 6 5
24195 010/300 Read block 1539 len 28672
24196 011/300 Read block 2205 len 4096
24197 012/300 Read block 4946 len 4096
24198 013/300 Write block 1895 len 40960 data: 5 5 5 3 2 2 2 3 3 3
24199 014/300 Write block 1604 len 36864 data: 4 3 3 5 6 7 7 7 7
24200 015/300 Read block 135 len 12288
24201 016/300 Read block 478 len 16384
24202 017/300 Write block 1532 len 40960 data: 5 5 5 6 5 5 5 7 8 7
24203 018/300 Read block 1195 len 16384
24204 019/300 Read block 957 len 12288
24205 020/300 Write block 3003 len 32768 data: 4 4 4 5 5 5 5 8
24206 021/300 Write block 948 len 24576 data: 6 5 4 3 3 4
24207 022/300 Write block 1646 len 36864 data: 5 5 5 7 4 4 5 6 7
24208 023/300 Read block 1847 len 4096
24209 024/300 Write block 1415 len 32768 data: 4 3 3 4 3 3 3 3
24210 025/300 Read block 3069 len 20480
24211 026/300 Write block 824 len 36864 data: 2 2 2 2 3 3 3 3 3
24212 027/300 Read block 2357 len 12288
24213 028/300 Read block 1263 len 4096
24214 029/300 Write block 1998 len 40960 data: 6 7 6 7 5 4 4 3 3 3
24215 030/300 Flush
24216 031/300 Write block 4863 len 36864 data: 7 6 4 2 3 3 3 4 5
24217 032/300 Write block 1252 len 4096 data: 5
24218 033/300 Read block 4701 len 12288
24219 034/300 Write block 1153 len 16384 data: 5 5 5 5
24220 035/300 Write block 333 len 32768 data: 3 3 4 3 4 5 6 7
24221 036/300 Write block 1004 len 12288 data: 6 5 6
24222 037/300 Read block 2518 len 36864
24223 038/300 Read block 3332 len 4096
24224 039/300 Write block 3668 len 8192 data: 8 8
24225 040/300 Flush
24226 041/300 Write block 366 len 40960 data: 6 6 5 7 6 5 5 5 6 5
24227 042/300 Read block 1541 len 8192
24228 043/300 Read block 3849 len 32768
24229 044/300 Read block 1916 len 16384
24230 045/300 Read block 4581 len 24576
24231 046/300 Read block 3704 len 40960
24232 047/300 Read block 2474 len 24576
24233 048/300 Write block 788 len 8192 data: 5 6
24234 049/300 Write block 3641 len 24576 data: 6 7 7 6 6 6
24235 050/300 Write block 2358 len 40960 data: 5 4 4 3 2 3 3 3 4 4
24236 051/300 Read block 1189 len 20480
24237 052/300 Flush
24238 053/300 Read block 3244 len 8192
24239 054/300 Flush
24240 055/300 Write block 427 len 36864 data: 3 4 5 4 3 4 5 6 5
24241 056/300 Flush
24242 057/300 Write block 2592 len 16384 data: 6 7 6 6
24243 058/300 Flush
24244 059/300 Write block 34 len 12288 data: 4 5 6
24245 060/300 Flush
24246 061/300 Read block 1983 len 40960
24247 062/300 Read block 3260 len 20480
24248 063/300 Write block 500 len 4096 data: 5
24249 064/300 Write block 1729 len 36864 data: 5 5 4 4 4 4 4 3 3
24250 065/300 Write block 4547 len 16384 data: 7 6 6 5
24251 066/300 Write block 4747 len 32768 data: 6 6 4 6 5 6 6 6
24252 067/300 Read block 3318 len 16384
24253 068/300 Read block 4922 len 12288
24254 069/300 Write block 1096 len 12288 data: 6 6 7
24255 070/300 Read block 3152 len 8192
24256 071/300 Write block 1044 len 12288 data: 4 6 6
24257 072/300 Write block 1139 len 28672 data: 5 6 6 4 4 6 5
24258 073/300 Write block 297 len 28672 data: 5 3 2 2 2 3 2
24259 074/300 Read block 1141 len 24576
24260 075/300 Read block 345 len 40960
24261 076/300 Write block 4893 len 12288 data: 6 7 6
24262 077/300 Read block 983 len 12288
24263 078/300 Read block 637 len 24576
24264 079/300 Read block 2180 len 32768
24265 080/300 Flush
24266 081/300 Write block 3485 len 20480 data: 2 3 4 4 4
24267 082/300 Flush
24268 083/300 Read block 4636 len 20480
24269 084/300 Flush
24270 085/300 Write block 2598 len 36864 data: 6 6 5 4 4 6 6 5 6
24271 086/300 Write block 4658 len 12288 data: 4 6 6
24272 087/300 Flush
24273 088/300 Write block 3873 len 36864 data: 7 8 7 5 5 5 5 4 4
24274 089/300 Read block 4233 len 36864
24275 090/300 Read block 36 len 12288
24276 091/300 Write block 3812 len 4096 data: 4
24277 092/300 Read block 4896 len 32768
24278 093/300 Read block 4153 len 16384
24279 094/300 Flush
24280 095/300 Read block 1257 len 36864
24281 096/300 Read block 896 len 32768
24282 097/300 Read block 2779 len 16384
24283 098/300 Write block 2312 len 32768 data: 5 5 6 6 4 4 3 2
24284 099/300 Flush
24285 100/300 Write block 3912 len 32768 data: 5 4 4 4 4 5 5 4
24286 101/300 Read block 336 len 4096
24287 102/300 Read block 1952 len 20480
24288 103/300 Write block 4457 len 24576 data: 5 4 3 3 3 3
24289 104/300 Read block 750 len 28672
24290 105/300 Read block 4262 len 12288
24291 106/300 Write block 1005 len 36864 data: 6 7 6 5 4 4 4 3 3
24292 107/300 Write block 4434 len 4096 data: 4
24293 108/300 Read block 2903 len 36864
24294 109/300 Flush
24295 110/300 Read block 1612 len 20480
24296 111/300 Write block 1168 len 16384 data: 7 6 6 5
24297 112/300 Flush
24298 113/300 Write block 2872 len 36864 data: 2 2 3 6 7 8 6 6 6
24299 114/300 Write block 3303 len 20480 data: 6 6 5 5 5
24300 115/300 Read block 2332 len 20480
24301 116/300 Read block 3628 len 24576
24302 117/300 Read block 4261 len 12288
24303 118/300 Write block 660 len 4096 data: 7
24304 119/300 Flush
24305 120/300 Flush
24306 121/300 Read block 3072 len 32768
24307 122/300 Read block 106 len 24576
24308 123/300 Write block 4262 len 36864 data: 3 3 5 5 4 5 5 4 5
24309 124/300 Write block 4577 len 28672 data: 4 3 3 3 3 4 4
24310 125/300 Write block 4653 len 32768 data: 4 3 4 3 3 5 7 7
24311 126/300 Read block 1049 len 12288
24312 127/300 Read block 4899 len 16384
24313 128/300 Write block 1605 len 24576 data: 4 4 6 7 8 8
24314 129/300 Write block 1292 len 20480 data: 5 4 4 3 3
24315 130/300 Write block 4715 len 4096 data: 5
24316 131/300 Write block 4443 len 4096 data: 8
24317 132/300 Read block 3691 len 24576
24318 133/300 Write block 2112 len 28672 data: 4 5 4 4 5 5 5
24319 134/300 Read block 209 len 24576
24320 135/300 Read block 3847 len 20480
24321 136/300 Read block 4795 len 8192
24322 137/300 Read block 2739 len 12288
24323 138/300 Write block 3393 len 16384 data: 4 6 7 7
24324 139/300 Read block 3823 len 24576
24325 140/300 Read block 971 len 20480
24326 141/300 Write block 1783 len 4096 data: 5
24327 142/300 Write block 0 len 40960 data: 3 3 4 4 5 5 7 7 8 7
24328 143/300 Read block 4092 len 20480
24329 144/300 Read block 3494 len 24576
24330 145/300 Write block 3237 len 24576 data: 2 2 2 2 4 5
24331 146/300 Read block 1165 len 40960
24332 147/300 Write block 1498 len 24576 data: 3 6 7 6 6 6
24333 148/300 Read block 2826 len 8192
24334 149/300 Read block 2753 len 8192
24335 150/300 Write block 4485 len 36864 data: 6 6 7 7 8 8 7 7 7
24336 151/300 Read block 2204 len 20480
24337 152/300 Read block 946 len 28672
24338 153/300 Write block 2054 len 16384 data: 6 5 6 7
24339 154/300 Read block 3225 len 24576
24340 155/300 Read block 3502 len 12288
24341 156/300 Read block 41 len 12288
24342 157/300 Write block 4552 len 32768 data: 3 3 3 2 3 3 4 4
24343 158/300 Write block 1841 len 32768 data: 4 5 5 6 6 7 9 9
24344 159/300 Flush
24345 160/300 Write block 4647 len 16384 data: 4 4 4 5
24346 161/300 Write block 4051 len 12288 data: 2 2 3
24347 162/300 Write block 4636 len 28672 data: 6 6 6 7 6 7 6
24348 163/300 Write block 339 len 8192 data: 7 8
24349 164/300 Write block 2465 len 36864 data: 5 4 6 5 5 5 6 6 7
24350 165/300 Write block 668 len 32768 data: 5 5 6 6 6 6 6 6
24351 166/300 Write block 130 len 36864 data: 6 3 3 3 3 2 4 4 5
24352 167/300 Write block 4248 len 36864 data: 6 6 5 5 4 4 4 7 6
24353 168/300 Write block 847 len 32768 data: 3 3 3 3 3 4 4 3
24354 169/300 Read block 3325 len 8192
24355 170/300 Read block 1782 len 4096
24356 171/300 Write block 2145 len 36864 data: 5 3 2 3 3 3 3 4 3
24357 172/300 Read block 2877 len 8192
24358 173/300 Read block 3375 len 16384
24359 174/300 Write block 961 len 24576 data: 3 3 4 4 5 6
24360 175/300 Read block 717 len 40960
24361 176/300 Read block 4911 len 24576
24362 177/300 Write block 2637 len 32768 data: 5 5 5 4 4 4 4 4
24363 178/300 Read block 1727 len 28672
24364 179/300 Read block 2816 len 8192
24365 180/300 Write block 4984 len 12288 data: 7 5 5
24366 181/300 Write block 1836 len 36864 data: 7 6 6 4 3 5 6 6 7
24367 182/300 Write block 4524 len 36864 data: 3 3 4 5 6 6 6 4 4
24368 183/300 Read block 708 len 8192
24369 184/300 Read block 2815 len 12288
24370 185/300 Read block 2685 len 36864
24371 186/300 Read block 4667 len 28672
24372 187/300 Write block 1002 len 40960 data: 5 5 7 7 8 7 6 5 5 5
24373 188/300 Read block 3571 len 16384
24374 189/300 Flush
24375 190/300 Write block 1164 len 28672 data: 4 3 5 5 8 7 7
24376 191/300 Flush
24377 192/300 Read block 229 len 4096
24378 193/300 Write block 1169 len 20480 data: 8 8 6 4 5
24379 194/300 Read block 743 len 8192
24380 195/300 Write block 4727 len 4096 data: 5
24381 196/300 Read block 4858 len 28672
24382 197/300 Write block 4098 len 36864 data: 4 4 4 5 5 5 4 5 4
24383 198/300 Write block 2746 len 4096 data: 8
24384 199/300 Flush
24385 200/300 Write block 2609 len 16384 data: 6 7 8 7
24386 201/300 Write block 2288 len 20480 data: 4 3 2 2 2
24387 202/300 Read block 3585 len 28672
24388 203/300 Flush
24389 204/300 Read block 4486 len 16384
24390 205/300 Write block 4462 len 16384 data: 4 3 3 5
24391 206/300 Write block 2839 len 32768 data: 5 6 6 7 7 6 6 9
24392 207/300 Read block 4817 len 24576
24393 208/300 Read block 1794 len 4096
24394 209/300 Write block 88 len 4096 data: 6
24395 210/300 Read block 293 len 12288
24396 211/300 Read block 109 len 20480
24397 212/300 Flush
24398 213/300 Write block 2051 len 40960 data: 6 6 4 7 6 7 8 8 8 7
24399 214/300 Read block 3708 len 32768
24400 215/300 Write block 2484 len 20480 data: 8 9 10 9 9
24401 216/300 Read block 3705 len 28672
24402 217/300 Read block 4670 len 4096
24403 218/300 Read block 3809 len 32768
24404 219/300 Write block 2695 len 8192 data: 5 5
24405 220/300 Read block 2776 len 36864
24406 221/300 Read block 4303 len 16384
24407 222/300 Read block 421 len 12288
24408 223/300 Flush
24409 224/300 Read block 195 len 20480
24410 225/300 Read block 4331 len 24576
24411 226/300 Write block 4707 len 36864 data: 5 4 5 6 6 5 5 5 6
24412 227/300 Write block 1725 len 4096 data: 5
24413 228/300 Write block 3263 len 20480 data: 4 3 3 3 3
24414 229/300 Write block 2332 len 36864 data: 3 3 3 4 5 5 5 5 5
24415 230/300 Write block 2337 len 36864 data: 6 6 6 6 6 6 5 5 5
24416 231/300 Read block 1138 len 24576
24417 232/300 Write block 4123 len 12288 data: 2 2 3
24418 233/300 Read block 3235 len 24576
24419 234/300 Write block 1193 len 28672 data: 7 8 8 9 10 7 6
24420 235/300 Read block 337 len 28672
24421 236/300 Read block 2208 len 24576
24422 237/300 Read block 388 len 12288
24423 238/300 Read block 2682 len 24576
24424 239/300 Write block 822 len 36864 data: 2 2 3 3 3 3 4 4 4
24425 240/300 Read block 113 len 32768
24426 241/300 Write block 3192 len 40960 data: 4 5 5 5 5 4 3 3 2 3
24427 242/300 Read block 4843 len 36864
24428 243/300 Read block 1963 len 40960
24429 244/300 Read block 1705 len 8192
24430 245/300 Write block 536 len 28672 data: 5 6 6 8 8 8 9
24431 246/300 Read block 2890 len 24576
24432 247/300 Read block 2564 len 4096
24433 248/300 Read block 2595 len 28672
24434 249/300 Write block 3114 len 4096 data: 7
24435 250/300 Flush
24436 251/300 Write block 1456 len 36864 data: 4 4 3 3 4 4 5 4 5
24437 252/300 Read block 1216 len 24576
24438 253/300 Read block 4559 len 28672
24439 254/300 Write block 732 len 12288 data: 4 4 5
24440 255/300 Read block 3466 len 36864
24441 256/300 Read block 1172 len 24576
24442 257/300 Write block 753 len 32768 data: 4 4 5 3 3 4 3 3
24443 258/300 Write block 4509 len 24576 data: 5 4 4 3 4 3
24444 259/300 Write block 3785 len 12288 data: 6 5 4
24445 260/300 Flush
24446 261/300 Flush
24447 262/300 Write block 4799 len 4096 data: 3
24448 263/300 Read block 1827 len 40960
24449 264/300 Write block 3595 len 16384 data: 8 9 10 11
24450 265/300 Write block 1825 len 20480 data: 7 5 5 7 7
24451 266/300 Read block 3530 len 12288
24452 267/300 Flush
24453 268/300 Read block 1709 len 4096
24454 269/300 Flush
24455 270/300 Read block 2182 len 40960
24456 271/300 Flush
24457 272/300 Read block 4591 len 16384
24458 273/300 Flush
24459 274/300 Read block 1281 len 36864
24460 275/300 Read block 3854 len 28672
24461 276/300 Read block 3876 len 8192
24462 277/300 Write block 3166 len 28672 data: 7 7 7 6 6 6 5
24463 278/300 Read block 2431 len 36864
24464 279/300 Write block 3136 len 40960 data: 6 7 6 6 7 6 7 5 6 7
24465 280/300 Read block 409 len 16384
24466 281/300 Write block 1913 len 28672 data: 2 3 3 4 4 4 6
24467 282/300 Read block 923 len 12288
24468 283/300 Write block 3258 len 36864 data: 7 8 7 6 5 5 4 4 4
24469 284/300 Read block 4461 len 16384
24470 285/300 Write block 2082 len 16384 data: 6 5 6 5
24471 286/300 Flush
24472 287/300 Write block 655 len 8192 data: 5 5
24473 288/300 Write block 2564 len 4096 data: 5
24474 289/300 Write block 1796 len 4096 data: 3
24475 290/300 Read block 3261 len 36864
24476 291/300 Write block 3557 len 40960 data: 5 4 4 5 4 4 4 6 6 4
24477 292/300 Read block 4180 len 20480
24478 293/300 Read block 1803 len 8192
24479 294/300 Write block 4773 len 40960 data: 7 6 5 5 4 3 2 3 3 3
24480 295/300 Write block 199 len 24576 data: 3 3 3 3 2 2
24481 296/300 Read block 354 len 8192
24482 297/300 Write block 4175 len 8192 data: 3 3
24483 298/300 Flush
24484 299/300 Write block 174 len 28672 data: 6 6 6 6 5 3 4
24485 300/300 Write block 1198 len 12288 data: 8 7 6
24486 301/300 Flush
24487 Main task has work to do, go find it
24488 got dsc Start(1)
24489 start 1
24490 Nov 26 22:09:41.773 INFO request completed, latency_us: 186, response_code: 204, uri: /start/cid/1, method: POST, req_id: cc3612d1-7ac5-44c5-8709-1e805bf6048d, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
24491 Replay: started 1, returned:Ok(())
24492 ----------------------------------------------------------------
24493 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
24494 States: Active Offline Active
24495 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
24496 7341 Acked Read 3 Done Sent Done false
24497 7342 Acked Write 2 Done Sent Done false
24498 7343 Acked Write 1 Done Sent Done false
24499 7344 Acked Write 8 Done Sent Done false
24500 7345 Acked Write 10 Done Sent Done false
24501 7346 Acked Write 8 Done Sent Done false
24502 7347 Acked Write 4 Done Sent Done false
24503 7348 Acked Flush 0 Done Sent Done false
24504 7349 Acked Write 8 Done Sent Done false
24505 7350 Acked Read 7 Done Sent Done false
24506 7351 Acked Read 1 Done Sent Done false
24507 7352 Acked Read 1 Done Sent Done false
24508 7353 Acked Write 10 Done Sent Done false
24509 7354 Acked Write 9 Done Sent Done false
24510 7355 Acked Read 3 Done Sent Done false
24511 7356 Acked Read 4 Done Sent Done false
24512 7357 Acked Write 10 Done Sent Done false
24513 7358 Acked Read 4 Done Sent Done false
24514 7359 Acked Read 3 Done Sent Done false
24515 7360 Acked Write 8 Done Sent Done false
24516 7361 Acked Write 6 Done Sent Done false
24517 7362 Acked Write 9 Done Sent Done false
24518 7363 Acked Read 1 Done Sent Done false
24519 7364 Acked Write 8 Done Sent Done false
24520 7365 Acked Read 5 Done Sent Done false
24521 7366 Acked Write 9 Done Sent Done false
24522 7367 Acked Read 3 Done Sent Done false
24523 7368 Acked Read 1 Done Sent Done false
24524 7369 Acked Write 10 Done Sent Done false
24525 7370 Acked Flush 0 Done Sent Done false
24526 7371 Acked Write 9 Done Sent Done false
24527 7372 Acked Write 1 Done Sent Done false
24528 7373 Acked Read 3 Done Sent Done false
24529 7374 Acked Write 4 Done Sent Done false
24530 7375 Acked Write 8 Done Sent Done false
24531 7376 Acked Write 3 Done Sent Done false
24532 7377 Acked Read 9 Done Sent Done false
24533 7378 Acked Read 1 Done Sent Done false
24534 7379 Acked Write 2 Done Sent Done false
24535 7380 Acked Flush 0 Done Sent Done false
24536 7381 Acked Write 10 Done Sent Done false
24537 7382 Acked Read 2 Done Sent Done false
24538 7383 Acked Read 8 Done Sent Done false
24539 7384 Acked Read 4 Done Sent Done false
24540 7385 Acked Read 6 Done Sent Done false
24541 7386 Acked Read 10 Done Sent Done false
24542 7387 Acked Read 6 Done Sent Done false
24543 7388 Acked Write 2 Done Sent Done false
24544 7389 Acked Write 6 Done Sent Done false
24545 7390 Acked Write 10 Done Sent Done false
24546 7391 Acked Read 5 Done Sent Done false
24547 7392 Acked Flush 0 Done Sent Done false
24548 7393 Acked Read 2 Done Sent Done false
24549 7394 Acked Flush 0 Done Sent Done false
24550 7395 Acked Write 9 Done Sent Done false
24551 7396 Acked Flush 0 Done Sent Done false
24552 7397 Acked Write 4 Done Sent Done false
24553 7398 Acked Flush 0 Done Sent Done false
24554 7399 Acked Write 3 Done Sent Done false
24555 7400 Acked Flush 0 Done Sent Done false
24556 7401 Acked Read 10 Done Sent Done false
24557 7402 Acked Read 5 Done Sent Done false
24558 7403 Acked Write 1 Done Sent Done false
24559 7404 Acked Write 9 Done Sent Done false
24560 7405 Acked Write 4 Done Sent Done false
24561 7406 Acked Write 8 Done Sent Done false
24562 7407 Acked Read 4 Done Sent Done false
24563 7408 Acked Read 3 Done Sent Done false
24564 7409 Acked Write 3 Done Sent Done false
24565 7410 Acked Read 2 Done Sent Done false
24566 7411 Acked Write 3 Done Sent Done false
24567 7412 Acked Write 7 Done Sent Done false
24568 7413 Acked Write 7 Done Sent Done false
24569 7414 Acked Read 6 Done Sent Done false
24570 7415 Acked Read 10 Done Sent Done false
24571 7416 Acked Write 3 Done Sent Done false
24572 7417 Acked Read 3 Done Sent Done false
24573 7418 Acked Read 6 Done Sent Done false
24574 7419 Acked Read 8 Done Sent Done false
24575 7420 Acked Flush 0 Done Sent Done false
24576 7421 Acked Write 5 Done Sent Done false
24577 7422 Acked Flush 0 Done Sent Done false
24578 7423 Acked Read 5 Done Sent Done false
24579 7424 Acked Flush 0 Done Sent Done false
24580 7425 Acked Write 9 Done Sent Done false
24581 7426 Acked Write 3 Done Sent Done false
24582 7427 Acked Flush 0 Done Sent Done false
24583 7428 Acked Write 9 Done Sent Done false
24584 7429 Acked Read 9 Done Sent Done false
24585 7430 Acked Read 3 Done Sent Done false
24586 7431 Acked Write 1 Done Sent Done false
24587 7432 Acked Read 8 Done Sent Done false
24588 7433 Acked Read 4 Done Sent Done false
24589 7434 Acked Flush 0 Done Sent Done false
24590 7435 Acked Read 9 Done Sent Done false
24591 7436 Acked Read 8 Done Sent Done false
24592 7437 Acked Read 4 Done Sent Done false
24593 7438 Acked Write 8 Done Sent Done false
24594 7439 Acked Flush 0 Done Sent Done false
24595 7440 Acked Write 8 Done Sent Done false
24596 7441 Acked Read 1 Done Sent Done false
24597 7442 Acked Read 5 Done Sent Done false
24598 7443 Acked Write 6 Done Sent Done false
24599 7444 Acked Read 7 Done Sent Done false
24600 7445 Acked Read 3 Done Sent Done false
24601 7446 Acked Write 9 Done Sent Done false
24602 7447 Acked Write 1 Done Sent Done false
24603 7448 Acked Read 9 Done Sent Done false
24604 7449 Acked Flush 0 Done Sent Done false
24605 7450 Acked Read 5 Done Sent Done false
24606 7451 Acked Write 4 Done Sent Done false
24607 7452 Acked Flush 0 Done Sent Done false
24608 7453 Acked Write 9 Done Sent Done false
24609 7454 Acked Write 5 Done Sent Done false
24610 7455 Acked Read 5 Done Sent Done false
24611 7456 Acked Read 6 Done Sent Done false
24612 7457 Acked Read 3 Done Sent Done false
24613 7458 Acked Write 1 Done Sent Done false
24614 7459 Acked Flush 0 Done Sent Done false
24615 7460 Acked Flush 0 Done Sent Done false
24616 7461 Acked Read 8 Done Sent Done false
24617 7462 Acked Read 6 Done Sent Done false
24618 7463 Acked Write 9 Done Sent Done false
24619 7464 Acked Write 7 Done Sent Done false
24620 7465 Acked Write 8 Done Sent Done false
24621 7466 Acked Read 3 Done Sent Done false
24622 7467 Acked Read 4 Done Sent Done false
24623 7468 Acked Write 6 Done Sent Done false
24624 7469 Acked Write 5 Done Sent Done false
24625 7470 Acked Write 1 Done Sent Done false
24626 7471 Acked Write 1 Done Sent Done false
24627 7472 Acked Read 6 Done Sent Done false
24628 7473 Acked Write 7 Done Sent Done false
24629 7474 Acked Read 6 Done Sent Done false
24630 7475 Acked Read 5 Done Sent Done false
24631 7476 Acked Read 2 Done Sent Done false
24632 7477 Acked Read 3 Done Sent Done false
24633 7478 Acked Write 4 Done Sent Done false
24634 7479 Acked Read 6 Done Sent Done false
24635 7480 Acked Read 5 Done Sent Done false
24636 7481 Acked Write 1 Done Sent Done false
24637 7482 Acked Write 10 Done Sent Done false
24638 7483 Acked Read 5 Done Sent Done false
24639 7484 Acked Read 6 Done Sent Done false
24640 7485 Acked Write 6 Done Sent Done false
24641 7486 Acked Read 10 Done Sent Done false
24642 7487 Acked Write 6 Done Sent Done false
24643 7488 Acked Read 2 Done Sent Done false
24644 7489 Acked Read 2 Done Sent Done false
24645 7490 Acked Write 9 Done Sent Done false
24646 7491 Acked Read 5 Done Sent Done false
24647 7492 Acked Read 7 Done Sent Done false
24648 7493 Acked Write 4 Done Sent Done false
24649 7494 Acked Read 6 Done Sent Done false
24650 7495 Acked Read 3 Done Sent Done false
24651 7496 Acked Read 3 Done Sent Done false
24652 7497 Acked Write 8 Done Sent Done false
24653 7498 Acked Write 8 Done Sent Done false
24654 7499 Acked Flush 0 Done Sent Done false
24655 7500 Acked Write 4 Done Sent Done false
24656 7501 Acked Write 3 Done Sent Done false
24657 7502 Acked Write 7 Done Sent Done false
24658 7503 Acked Write 2 Done Sent Done false
24659 7504 Acked Write 9 Done Sent Done false
24660 7505 Acked Write 8 Done Sent Done false
24661 7506 Acked Write 9 Done Sent Done false
24662 7507 Acked Write 9 Done Sent Done false
24663 7508 Acked Write 8 Done Sent Done false
24664 7509 Acked Read 2 Done Sent Done false
24665 7510 Acked Read 1 Done Sent Done false
24666 7511 Acked Write 9 Done Sent Done false
24667 7512 Acked Read 2 Done Sent Done false
24668 7513 Acked Read 4 Done Sent Done false
24669 7514 Acked Write 6 Done Sent Done false
24670 7515 Acked Read 10 Done Sent Done false
24671 7516 Acked Read 6 Done Sent Done false
24672 7517 Acked Write 8 Done Sent Done false
24673 7518 Acked Read 7 Done Sent Done false
24674 7519 Acked Read 2 Done Sent Done false
24675 7520 Acked Write 3 Done Sent Done false
24676 7521 Acked Write 9 Done Sent Done false
24677 7522 Acked Write 9 Done Sent Done false
24678 7523 Acked Read 2 Done Sent Done false
24679 7524 Acked Read 3 Done Sent Done false
24680 7525 Acked Read 9 Done Sent Done false
24681 7526 Acked Read 7 Done Sent Done false
24682 7527 Acked Write 10 Done Sent Done false
24683 7528 Acked Read 4 Done Sent Done false
24684 7529 Acked Flush 0 Done Sent Done false
24685 7530 Acked Write 7 Done Sent Done false
24686 7531 Acked Flush 0 Done Sent Done false
24687 7532 Acked Read 1 Done Sent Done false
24688 7533 Acked Write 5 Done Sent Done false
24689 7534 Acked Read 2 Done Sent Done false
24690 7535 Acked Write 1 Done Sent Done false
24691 7536 Acked Read 7 Done Sent Done false
24692 7537 Acked Write 9 Done Sent Done false
24693 7538 Acked Write 1 Done Sent Done false
24694 7539 Acked Flush 0 Done Sent Done false
24695 7540 Acked Write 4 Done Sent Done false
24696 7541 Acked Write 5 Done Sent Done false
24697 7542 Acked Read 7 Done Sent Done false
24698 7543 Acked Flush 0 Done Sent Done false
24699 7544 Acked Read 4 Done Sent Done false
24700 7545 Acked Write 4 Done Sent Done false
24701 7546 Acked Write 8 Done Sent Done false
24702 7547 Acked Read 6 Done Sent Done false
24703 7548 Acked Read 1 Done Sent Done false
24704 7549 Acked Write 1 Done Sent Done false
24705 7550 Acked Read 3 Done Sent Done false
24706 7551 Acked Read 5 Done Sent Done false
24707 7552 Acked Flush 0 Done Sent Done false
24708 7553 Acked Write 10 Done Sent Done false
24709 7554 Acked Read 8 Done Sent Done false
24710 7555 Acked Write 5 Done Sent Done false
24711 7556 Acked Read 7 Done Sent Done false
24712 7557 Acked Read 1 Done Sent Done false
24713 7558 Acked Read 8 Done Sent Done false
24714 7559 Acked Write 2 Done Sent Done false
24715 7560 Acked Read 9 Done Sent Done false
24716 7561 Acked Read 4 Done Sent Done false
24717 7562 Acked Read 3 Done Sent Done false
24718 7563 Acked Flush 0 Done Sent Done false
24719 7564 Acked Read 5 Done Sent Done false
24720 7565 Acked Read 6 Done Sent Done false
24721 7566 Acked Write 9 Done Sent Done false
24722 7567 Acked Write 1 Done Sent Done false
24723 7568 Acked Write 5 Done Sent Done false
24724 7569 Acked Write 9 Done Sent Done false
24725 7570 Acked Write 9 Done Sent Done false
24726 7571 Acked Read 6 Done Sent Done false
24727 7572 Acked Write 3 Done Sent Done false
24728 7573 Acked Read 6 Done Sent Done false
24729 7574 Acked Write 7 Done Sent Done false
24730 7575 Acked Read 7 Done Sent Done false
24731 7576 Acked Read 6 Done Sent Done false
24732 7577 Acked Read 3 Done Sent Done false
24733 7578 Acked Read 6 Done Sent Done false
24734 7579 Acked Write 9 Done Sent Done false
24735 7580 Acked Read 8 Done Sent Done false
24736 7581 Acked Write 10 Done Sent Done false
24737 7582 Acked Read 9 Done Sent Done false
24738 7583 Acked Read 10 Done Sent Done false
24739 7584 Acked Read 2 Done Sent Done false
24740 7585 Acked Write 7 Done Sent Done false
24741 7586 Acked Read 6 Done Sent Done false
24742 7587 Acked Read 1 Done Sent Done false
24743 7588 Acked Read 7 Done Sent Done false
24744 7589 Acked Write 1 Done Sent Done false
24745 7590 Acked Flush 0 Done Sent Done false
24746 7591 Acked Write 9 Done Sent Done false
24747 7592 Acked Read 6 Done Sent Done false
24748 7593 Acked Read 7 Done Sent Done false
24749 7594 Acked Write 3 Done Sent Done false
24750 7595 Acked Read 9 Done Sent Done false
24751 7596 Acked Read 6 Done Sent Done false
24752 7597 Acked Write 8 Done Sent Done false
24753 7598 Acked Write 6 Done Sent Done false
24754 7599 Acked Write 3 Done Sent Done false
24755 7600 Acked Flush 0 Done Sent Done false
24756 7601 Acked Flush 0 Done Sent Done false
24757 7602 Acked Write 1 Done Sent Done false
24758 7603 Acked Read 10 Done Sent Done false
24759 7604 Acked Write 4 Done Sent Done false
24760 7605 Acked Write 5 Done Sent Done false
24761 7606 Acked Read 3 Done Sent Done false
24762 7607 Acked Flush 0 Done Sent Done false
24763 7608 Acked Read 1 Done Sent Done false
24764 7609 Acked Flush 0 Done Sent Done false
24765 7610 Acked Read 10 Done Sent Done false
24766 7611 Acked Flush 0 Done Sent Done false
24767 7612 Acked Read 4 Done Sent Done false
24768 7613 Acked Flush 0 Done Sent Done false
24769 7614 Acked Read 9 Done Sent Done false
24770 7615 Acked Read 7 Done Sent Done false
24771 7616 Acked Read 2 Done Sent Done false
24772 7617 Acked Write 7 Done Sent Done false
24773 7618 Acked Read 9 Done Sent Done false
24774 7619 Acked Write 10 Done Sent Done false
24775 7620 Acked Read 4 Done Sent Done false
24776 7621 Acked Write 7 Done Sent Done false
24777 7622 Acked Read 3 Done Sent Done false
24778 7623 Acked Write 9 Done Sent Done false
24779 7624 Acked Read 4 Done Sent Done false
24780 7625 Acked Write 4 Done Sent Done false
24781 7626 Acked Flush 0 Done Sent Done false
24782 7627 Acked Write 2 Done Sent Done false
24783 7628 Acked Write 1 Done Sent Done false
24784 7629 Acked Write 1 Done Sent Done false
24785 7630 Acked Read 9 Done Sent Done false
24786 7631 Acked Write 10 Done Sent Done false
24787 7632 Acked Read 5 Done Sent Done false
24788 7633 Acked Read 2 Done Sent Done false
24789 7634 Acked Write 10 Done Sent Done false
24790 7635 Acked Write 6 Done Sent Done false
24791 7636 Acked Read 2 Done Sent Done false
24792 7637 Acked Write 2 Done Sent Done false
24793 7638 Acked Flush 0 Done Sent Done false
24794 7639 Acked Write 7 Done Sent Done false
24795 7640 Acked Write 3 Done Sent Done false
24796 7641 Acked Flush 0 Done Sent Done false
24797 STATES DS:0 DS:1 DS:2 TOTAL
24798 Sent 0 301 0 301
24799 Done 301 0 301 602
24800 Skipped 0 0 0 0
24801 Error 0 0 0 0
24802 Last Flush: 7641 7340 7641
24803 Downstairs last five completed: 7340 7339 7338 7337 7336
24804 Upstairs last five completed: 7641 7640 7639 7638 7637
24805 CLIENT: Up:0 ds:301 act:2
24806 [8820] got start action
24807 [8820] I am going to restart
24808 Starting downstairs at port 8820
24809 Make output file at "/tmp/dsc/downstairs-8820.txt"
24810 [8820][1] reports Starting
24811 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1225)
24812 [8820][1] reports Running
24813 ----------------------------------------------------------------
24814 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
24815 States: Active Offline Active
24816 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
24817 7341 Acked Read 3 Done Sent Done false
24818 7342 Acked Write 2 Done Sent Done false
24819 7343 Acked Write 1 Done Sent Done false
24820 7344 Acked Write 8 Done Sent Done false
24821 7345 Acked Write 10 Done Sent Done false
24822 7346 Acked Write 8 Done Sent Done false
24823 7347 Acked Write 4 Done Sent Done false
24824 7348 Acked Flush 0 Done Sent Done false
24825 7349 Acked Write 8 Done Sent Done false
24826 7350 Acked Read 7 Done Sent Done false
24827 7351 Acked Read 1 Done Sent Done false
24828 7352 Acked Read 1 Done Sent Done false
24829 7353 Acked Write 10 Done Sent Done false
24830 7354 Acked Write 9 Done Sent Done false
24831 7355 Acked Read 3 Done Sent Done false
24832 7356 Acked Read 4 Done Sent Done false
24833 7357 Acked Write 10 Done Sent Done false
24834 7358 Acked Read 4 Done Sent Done false
24835 7359 Acked Read 3 Done Sent Done false
24836 7360 Acked Write 8 Done Sent Done false
24837 7361 Acked Write 6 Done Sent Done false
24838 7362 Acked Write 9 Done Sent Done false
24839 7363 Acked Read 1 Done Sent Done false
24840 7364 Acked Write 8 Done Sent Done false
24841 7365 Acked Read 5 Done Sent Done false
24842 7366 Acked Write 9 Done Sent Done false
24843 7367 Acked Read 3 Done Sent Done false
24844 7368 Acked Read 1 Done Sent Done false
24845 7369 Acked Write 10 Done Sent Done false
24846 7370 Acked Flush 0 Done Sent Done false
24847 7371 Acked Write 9 Done Sent Done false
24848 7372 Acked Write 1 Done Sent Done false
24849 7373 Acked Read 3 Done Sent Done false
24850 7374 Acked Write 4 Done Sent Done false
24851 7375 Acked Write 8 Done Sent Done false
24852 7376 Acked Write 3 Done Sent Done false
24853 7377 Acked Read 9 Done Sent Done false
24854 7378 Acked Read 1 Done Sent Done false
24855 7379 Acked Write 2 Done Sent Done false
24856 7380 Acked Flush 0 Done Sent Done false
24857 7381 Acked Write 10 Done Sent Done false
24858 7382 Acked Read 2 Done Sent Done false
24859 7383 Acked Read 8 Done Sent Done false
24860 7384 Acked Read 4 Done Sent Done false
24861 7385 Acked Read 6 Done Sent Done false
24862 7386 Acked Read 10 Done Sent Done false
24863 7387 Acked Read 6 Done Sent Done false
24864 7388 Acked Write 2 Done Sent Done false
24865 7389 Acked Write 6 Done Sent Done false
24866 7390 Acked Write 10 Done Sent Done false
24867 7391 Acked Read 5 Done Sent Done false
24868 7392 Acked Flush 0 Done Sent Done false
24869 7393 Acked Read 2 Done Sent Done false
24870 7394 Acked Flush 0 Done Sent Done false
24871 7395 Acked Write 9 Done Sent Done false
24872 7396 Acked Flush 0 Done Sent Done false
24873 7397 Acked Write 4 Done Sent Done false
24874 7398 Acked Flush 0 Done Sent Done false
24875 7399 Acked Write 3 Done Sent Done false
24876 7400 Acked Flush 0 Done Sent Done false
24877 7401 Acked Read 10 Done Sent Done false
24878 7402 Acked Read 5 Done Sent Done false
24879 7403 Acked Write 1 Done Sent Done false
24880 7404 Acked Write 9 Done Sent Done false
24881 7405 Acked Write 4 Done Sent Done false
24882 7406 Acked Write 8 Done Sent Done false
24883 7407 Acked Read 4 Done Sent Done false
24884 7408 Acked Read 3 Done Sent Done false
24885 7409 Acked Write 3 Done Sent Done false
24886 7410 Acked Read 2 Done Sent Done false
24887 7411 Acked Write 3 Done Sent Done false
24888 7412 Acked Write 7 Done Sent Done false
24889 7413 Acked Write 7 Done Sent Done false
24890 7414 Acked Read 6 Done Sent Done false
24891 7415 Acked Read 10 Done Sent Done false
24892 7416 Acked Write 3 Done Sent Done false
24893 7417 Acked Read 3 Done Sent Done false
24894 7418 Acked Read 6 Done Sent Done false
24895 7419 Acked Read 8 Done Sent Done false
24896 7420 Acked Flush 0 Done Sent Done false
24897 7421 Acked Write 5 Done Sent Done false
24898 7422 Acked Flush 0 Done Sent Done false
24899 7423 Acked Read 5 Done Sent Done false
24900 7424 Acked Flush 0 Done Sent Done false
24901 7425 Acked Write 9 Done Sent Done false
24902 7426 Acked Write 3 Done Sent Done false
24903 7427 Acked Flush 0 Done Sent Done false
24904 7428 Acked Write 9 Done Sent Done false
24905 7429 Acked Read 9 Done Sent Done false
24906 7430 Acked Read 3 Done Sent Done false
24907 7431 Acked Write 1 Done Sent Done false
24908 7432 Acked Read 8 Done Sent Done false
24909 7433 Acked Read 4 Done Sent Done false
24910 7434 Acked Flush 0 Done Sent Done false
24911 7435 Acked Read 9 Done Sent Done false
24912 7436 Acked Read 8 Done Sent Done false
24913 7437 Acked Read 4 Done Sent Done false
24914 7438 Acked Write 8 Done Sent Done false
24915 7439 Acked Flush 0 Done Sent Done false
24916 7440 Acked Write 8 Done Sent Done false
24917 7441 Acked Read 1 Done Sent Done false
24918 7442 Acked Read 5 Done Sent Done false
24919 7443 Acked Write 6 Done Sent Done false
24920 7444 Acked Read 7 Done Sent Done false
24921 7445 Acked Read 3 Done Sent Done false
24922 7446 Acked Write 9 Done Sent Done false
24923 7447 Acked Write 1 Done Sent Done false
24924 7448 Acked Read 9 Done Sent Done false
24925 7449 Acked Flush 0 Done Sent Done false
24926 7450 Acked Read 5 Done Sent Done false
24927 7451 Acked Write 4 Done Sent Done false
24928 7452 Acked Flush 0 Done Sent Done false
24929 7453 Acked Write 9 Done Sent Done false
24930 7454 Acked Write 5 Done Sent Done false
24931 7455 Acked Read 5 Done Sent Done false
24932 7456 Acked Read 6 Done Sent Done false
24933 7457 Acked Read 3 Done Sent Done false
24934 7458 Acked Write 1 Done Sent Done false
24935 7459 Acked Flush 0 Done Sent Done false
24936 7460 Acked Flush 0 Done Sent Done false
24937 7461 Acked Read 8 Done Sent Done false
24938 7462 Acked Read 6 Done Sent Done false
24939 7463 Acked Write 9 Done Sent Done false
24940 7464 Acked Write 7 Done Sent Done false
24941 7465 Acked Write 8 Done Sent Done false
24942 7466 Acked Read 3 Done Sent Done false
24943 7467 Acked Read 4 Done Sent Done false
24944 7468 Acked Write 6 Done Sent Done false
24945 7469 Acked Write 5 Done Sent Done false
24946 7470 Acked Write 1 Done Sent Done false
24947 7471 Acked Write 1 Done Sent Done false
24948 7472 Acked Read 6 Done Sent Done false
24949 7473 Acked Write 7 Done Sent Done false
24950 7474 Acked Read 6 Done Sent Done false
24951 7475 Acked Read 5 Done Sent Done false
24952 7476 Acked Read 2 Done Sent Done false
24953 7477 Acked Read 3 Done Sent Done false
24954 7478 Acked Write 4 Done Sent Done false
24955 7479 Acked Read 6 Done Sent Done false
24956 7480 Acked Read 5 Done Sent Done false
24957 7481 Acked Write 1 Done Sent Done false
24958 7482 Acked Write 10 Done Sent Done false
24959 7483 Acked Read 5 Done Sent Done false
24960 7484 Acked Read 6 Done Sent Done false
24961 7485 Acked Write 6 Done Sent Done false
24962 7486 Acked Read 10 Done Sent Done false
24963 7487 Acked Write 6 Done Sent Done false
24964 7488 Acked Read 2 Done Sent Done false
24965 7489 Acked Read 2 Done Sent Done false
24966 7490 Acked Write 9 Done Sent Done false
24967 7491 Acked Read 5 Done Sent Done false
24968 7492 Acked Read 7 Done Sent Done false
24969 7493 Acked Write 4 Done Sent Done false
24970 7494 Acked Read 6 Done Sent Done false
24971 7495 Acked Read 3 Done Sent Done false
24972 7496 Acked Read 3 Done Sent Done false
24973 7497 Acked Write 8 Done Sent Done false
24974 7498 Acked Write 8 Done Sent Done false
24975 7499 Acked Flush 0 Done Sent Done false
24976 7500 Acked Write 4 Done Sent Done false
24977 7501 Acked Write 3 Done Sent Done false
24978 7502 Acked Write 7 Done Sent Done false
24979 7503 Acked Write 2 Done Sent Done false
24980 7504 Acked Write 9 Done Sent Done false
24981 7505 Acked Write 8 Done Sent Done false
24982 7506 Acked Write 9 Done Sent Done false
24983 7507 Acked Write 9 Done Sent Done false
24984 7508 Acked Write 8 Done Sent Done false
24985 7509 Acked Read 2 Done Sent Done false
24986 7510 Acked Read 1 Done Sent Done false
24987 7511 Acked Write 9 Done Sent Done false
24988 7512 Acked Read 2 Done Sent Done false
24989 7513 Acked Read 4 Done Sent Done false
24990 7514 Acked Write 6 Done Sent Done false
24991 7515 Acked Read 10 Done Sent Done false
24992 7516 Acked Read 6 Done Sent Done false
24993 7517 Acked Write 8 Done Sent Done false
24994 7518 Acked Read 7 Done Sent Done false
24995 7519 Acked Read 2 Done Sent Done false
24996 7520 Acked Write 3 Done Sent Done false
24997 7521 Acked Write 9 Done Sent Done false
24998 7522 Acked Write 9 Done Sent Done false
24999 7523 Acked Read 2 Done Sent Done false
25000 7524 Acked Read 3 Done Sent Done false
25001 7525 Acked Read 9 Done Sent Done false
25002 7526 Acked Read 7 Done Sent Done false
25003 7527 Acked Write 10 Done Sent Done false
25004 7528 Acked Read 4 Done Sent Done false
25005 7529 Acked Flush 0 Done Sent Done false
25006 7530 Acked Write 7 Done Sent Done false
25007 7531 Acked Flush 0 Done Sent Done false
25008 7532 Acked Read 1 Done Sent Done false
25009 7533 Acked Write 5 Done Sent Done false
25010 7534 Acked Read 2 Done Sent Done false
25011 7535 Acked Write 1 Done Sent Done false
25012 7536 Acked Read 7 Done Sent Done false
25013 7537 Acked Write 9 Done Sent Done false
25014 7538 Acked Write 1 Done Sent Done false
25015 7539 Acked Flush 0 Done Sent Done false
25016 7540 Acked Write 4 Done Sent Done false
25017 7541 Acked Write 5 Done Sent Done false
25018 7542 Acked Read 7 Done Sent Done false
25019 7543 Acked Flush 0 Done Sent Done false
25020 7544 Acked Read 4 Done Sent Done false
25021 7545 Acked Write 4 Done Sent Done false
25022 7546 Acked Write 8 Done Sent Done false
25023 7547 Acked Read 6 Done Sent Done false
25024 7548 Acked Read 1 Done Sent Done false
25025 7549 Acked Write 1 Done Sent Done false
25026 7550 Acked Read 3 Done Sent Done false
25027 7551 Acked Read 5 Done Sent Done false
25028 7552 Acked Flush 0 Done Sent Done false
25029 7553 Acked Write 10 Done Sent Done false
25030 7554 Acked Read 8 Done Sent Done false
25031 7555 Acked Write 5 Done Sent Done false
25032 7556 Acked Read 7 Done Sent Done false
25033 7557 Acked Read 1 Done Sent Done false
25034 7558 Acked Read 8 Done Sent Done false
25035 7559 Acked Write 2 Done Sent Done false
25036 7560 Acked Read 9 Done Sent Done false
25037 7561 Acked Read 4 Done Sent Done false
25038 7562 Acked Read 3 Done Sent Done false
25039 7563 Acked Flush 0 Done Sent Done false
25040 7564 Acked Read 5 Done Sent Done false
25041 7565 Acked Read 6 Done Sent Done false
25042 7566 Acked Write 9 Done Sent Done false
25043 7567 Acked Write 1 Done Sent Done false
25044 7568 Acked Write 5 Done Sent Done false
25045 7569 Acked Write 9 Done Sent Done false
25046 7570 Acked Write 9 Done Sent Done false
25047 7571 Acked Read 6 Done Sent Done false
25048 7572 Acked Write 3 Done Sent Done false
25049 7573 Acked Read 6 Done Sent Done false
25050 7574 Acked Write 7 Done Sent Done false
25051 7575 Acked Read 7 Done Sent Done false
25052 7576 Acked Read 6 Done Sent Done false
25053 7577 Acked Read 3 Done Sent Done false
25054 7578 Acked Read 6 Done Sent Done false
25055 7579 Acked Write 9 Done Sent Done false
25056 7580 Acked Read 8 Done Sent Done false
25057 7581 Acked Write 10 Done Sent Done false
25058 7582 Acked Read 9 Done Sent Done false
25059 7583 Acked Read 10 Done Sent Done false
25060 7584 Acked Read 2 Done Sent Done false
25061 7585 Acked Write 7 Done Sent Done false
25062 7586 Acked Read 6 Done Sent Done false
25063 7587 Acked Read 1 Done Sent Done false
25064 7588 Acked Read 7 Done Sent Done false
25065 7589 Acked Write 1 Done Sent Done false
25066 7590 Acked Flush 0 Done Sent Done false
25067 7591 Acked Write 9 Done Sent Done false
25068 7592 Acked Read 6 Done Sent Done false
25069 7593 Acked Read 7 Done Sent Done false
25070 7594 Acked Write 3 Done Sent Done false
25071 7595 Acked Read 9 Done Sent Done false
25072 7596 Acked Read 6 Done Sent Done false
25073 7597 Acked Write 8 Done Sent Done false
25074 7598 Acked Write 6 Done Sent Done false
25075 7599 Acked Write 3 Done Sent Done false
25076 7600 Acked Flush 0 Done Sent Done false
25077 7601 Acked Flush 0 Done Sent Done false
25078 7602 Acked Write 1 Done Sent Done false
25079 7603 Acked Read 10 Done Sent Done false
25080 7604 Acked Write 4 Done Sent Done false
25081 7605 Acked Write 5 Done Sent Done false
25082 7606 Acked Read 3 Done Sent Done false
25083 7607 Acked Flush 0 Done Sent Done false
25084 7608 Acked Read 1 Done Sent Done false
25085 7609 Acked Flush 0 Done Sent Done false
25086 7610 Acked Read 10 Done Sent Done false
25087 7611 Acked Flush 0 Done Sent Done false
25088 7612 Acked Read 4 Done Sent Done false
25089 7613 Acked Flush 0 Done Sent Done false
25090 7614 Acked Read 9 Done Sent Done false
25091 7615 Acked Read 7 Done Sent Done false
25092 7616 Acked Read 2 Done Sent Done false
25093 7617 Acked Write 7 Done Sent Done false
25094 7618 Acked Read 9 Done Sent Done false
25095 7619 Acked Write 10 Done Sent Done false
25096 7620 Acked Read 4 Done Sent Done false
25097 7621 Acked Write 7 Done Sent Done false
25098 7622 Acked Read 3 Done Sent Done false
25099 7623 Acked Write 9 Done Sent Done false
25100 7624 Acked Read 4 Done Sent Done false
25101 7625 Acked Write 4 Done Sent Done false
25102 7626 Acked Flush 0 Done Sent Done false
25103 7627 Acked Write 2 Done Sent Done false
25104 7628 Acked Write 1 Done Sent Done false
25105 7629 Acked Write 1 Done Sent Done false
25106 7630 Acked Read 9 Done Sent Done false
25107 7631 Acked Write 10 Done Sent Done false
25108 7632 Acked Read 5 Done Sent Done false
25109 7633 Acked Read 2 Done Sent Done false
25110 7634 Acked Write 10 Done Sent Done false
25111 7635 Acked Write 6 Done Sent Done false
25112 7636 Acked Read 2 Done Sent Done false
25113 7637 Acked Write 2 Done Sent Done false
25114 7638 Acked Flush 0 Done Sent Done false
25115 7639 Acked Write 7 Done Sent Done false
25116 7640 Acked Write 3 Done Sent Done false
25117 7641 Acked Flush 0 Done Sent Done false
25118 STATES DS:0 DS:1 DS:2 TOTAL
25119 Sent 0 301 0 301
25120 Done 301 0 301 602
25121 Skipped 0 0 0 0
25122 Error 0 0 0 0
25123 Last Flush: 7641 7340 7641
25124 Downstairs last five completed: 7340 7339 7338 7337 7336
25125 Upstairs last five completed: 7641 7640 7639 7638 7637
25126 CLIENT: Up:0 ds:301 act:2
251272024-11-26T22:09:47.544ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
251282024-11-26T22:09:47.544ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:42710) = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
251292024-11-26T22:09:47.545ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
251302024-11-26T22:09:47.546ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
251312024-11-26T22:09:47.546ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
251322024-11-26T22:09:47.546ZINFOcrucible: send last flush ID to this DS: 7340 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
251332024-11-26T22:09:47.546ZINFOcrucible: Replied this last flush ID: 7340 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
251342024-11-26T22:09:47.546ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
251352024-11-26T22:09:47.546ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
251362024-11-26T22:09:47.546ZINFOcrucible: [1] client re-new 301 jobs since flush 7340 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
251372024-11-26T22:09:47.546ZINFOcrucible: [1] Replayed 301 jobs since flush: 7340 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
25138 ----------------------------------------------------------------
25139 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
25140 Downstairs last five completed: 7641 7640 7639 7638 7637
25141 Upstairs last five completed: 7641 7640 7639 7638 7637
25142 CLIENT: Up:0 ds:0 act:3
25143 Replay: All jobs finished
25144 Main task has work to do, go find it
25145 got dsc Stop(2)
25146 stop 2
25147 [8830] Got stop action so:false kr:false
25148 Nov 26 22:09:49.778 INFO request completed, latency_us: 355, response_code: 204, uri: /stop/cid/2, method: POST, req_id: c2114815-0e63-4e53-b70a-9b914006f18e, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
251492024-11-26T22:09:49.778ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
251502024-11-26T22:09:49.778ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
251512024-11-26T22:09:49.778ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
251522024-11-26T22:09:49.778ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
25153 Nov 26 22:09:49.778 INFO request completed, latency_us: 115{"msg":"[2] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-26T22:09:49.778891119Z","hostname":"w-01JDN7ZQ17ABQG15KHKN3F2D26","pid":1203,"client":"2","":"downstairs","session_id":"86a36c17-3718-484c-87fd-2b8a1dc2ab18"}
25154 , response_code: 200, uri: /state/cid/2, method: {"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2024-11-26T22:09:49.778936474Z","hostname":"w-01JDN7ZQ17ABQG15KHKN3F2D26","pid":1203,"":"io task","client":"2","":"downstairs","session_id":"86a36c17-3718-484c-87fd-2b8a1dc2ab18"}
25155 GET, req_id: c804511d-d9a9-4bf0-9a29-d79f39bf2c6a, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
25156 [8830] Exited with: ExitStatus(unix_wait_status(9))
25157 [8830][2] reports Exit
25158 Nov 26 22:09:53.780 INFO request completed, latency_us: 220, response_code: 200, uri: /state/cid/2, method: GET, req_id: 9537ce0e-d37d-4b6d-958b-6c6b600a8c82, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
25159 001/300 Write block 971 len 12288 data: 4 5 5
25160 002/300 Write block 2809 len 32768 data: 7 6 6 7 8 6 4 5
25161 003/300 Read block 2019 len 40960
25162 004/300 Read block 3728 len 4096
25163 005/300 Read block 1180 len 20480
25164 006/300 Write block 1168 len 24576 data: 9 9 9 7 5 6
25165 007/300 Read block 1563 len 16384
25166 008/300 Read block 4692 len 12288
25167 009/300 Flush
25168 010/300 Read block 2309 len 40960
25169 011/300 Write block 1351 len 20480 data: 7 7 5 6 7
25170 012/300 Read block 3574 len 8192
25171 013/300 Read block 462 len 16384
25172 014/300 Flush
25173 015/300 Write block 3391 len 24576 data: 5 4 5 7 8 8
25174 016/300 Write block 137 len 4096 data: 5
25175 017/300 Flush
25176 018/300 Write block 2057 len 40960 data: 9 9 9 8 7 6 6 7 9 8
25177 019/300 Write block 3328 len 36864 data: 5 5 5 6 4 3 3 3 3
25178 020/300 Write block 786 len 20480 data: 5 5 6 7 6
25179 021/300 Write block 4478 len 24576 data: 8 5 5 4 5 4
25180 022/300 Write block 2253 len 36864 data: 8 9 6 6 4 4 5 4 4
25181 023/300 Read block 4531 len 36864
25182 024/300 Flush
25183 025/300 Read block 1610 len 32768
25184 026/300 Read block 143 len 16384
25185 027/300 Write block 4193 len 8192 data: 4 5
25186 028/300 Read block 47 len 12288
25187 029/300 Read block 2324 len 28672
25188 030/300 Write block 1200 len 20480 data: 7 6 6 6 6
25189 031/300 Write block 2887 len 32768 data: 9 8 6 6 4 5 4 4
25190 032/300 Read block 634 len 12288
25191 033/300 Read block 3080 len 24576
25192 034/300 Write block 4852 len 24576 data: 3 3 4 4 6 5
25193 035/300 Read block 4581 len 32768
25194 036/300 Flush
25195 037/300 Write block 1002 len 8192 data: 6 6
25196 038/300 Flush
25197 039/300 Write block 2772 len 20480 data: 8 7 5 5 5
25198 040/300 Read block 1918 len 8192
25199 041/300 Read block 877 len 16384
25200 042/300 Write block 4168 len 28672 data: 4 3 2 3 3 3 3
25201 043/300 Read block 2484 len 4096
25202 044/300 Write block 4522 len 4096 data: 5
25203 045/300 Write block 1521 len 16384 data: 4 5 6 7
25204 046/300 Write block 4970 len 40960 data: 6 5 4 4 5 5 5 6 5 5
25205 047/300 Write block 4001 len 40960 data: 2 4 6 6 6 6 5 5 5 4
25206 048/300 Write block 2136 len 4096 data: 5
25207 049/300 Flush
25208 050/300 Write block 1406 len 36864 data: 4 4 3 2 3 3 3 4 5
25209 051/300 Write block 1947 len 20480 data: 5 6 4 4 3
25210 052/300 Write block 1580 len 16384 data: 6 5 5 4
25211 053/300 Read block 1830 len 28672
25212 054/300 Write block 3190 len 36864 data: 2 3 5 6 6 6 6 5 4
25213 055/300 Read block 1461 len 24576
25214 056/300 Read block 3955 len 40960
25215 057/300 Write block 4298 len 24576 data: 4 4 4 3 2 2
25216 058/300 Write block 3291 len 16384 data: 7 6 7 6
25217 059/300 Read block 3686 len 4096
25218 060/300 Write block 698 len 28672 data: 4 4 5 4 5 4 4
25219 061/300 Write block 3437 len 20480 data: 3 5 5 5 4
25220 062/300 Read block 3343 len 8192
25221 063/300 Write block 3757 len 36864 data: 4 4 3 3 4 5 6 7 6
25222 064/300 Read block 716 len 16384
25223 065/300 Read block 3520 len 32768
25224 066/300 Read block 2329 len 40960
25225 067/300 Read block 4386 len 12288
25226 068/300 Flush
25227 069/300 Read block 3375 len 8192
25228 070/300 Read block 1703 len 20480
25229 071/300 Read block 2683 len 24576
25230 072/300 Read block 1475 len 40960
25231 073/300 Read block 84 len 16384
25232 074/300 Write block 43 len 4096 data: 6
25233 075/300 Flush
25234 076/300 Write block 2825 len 12288 data: 4 4 4
25235 077/300 Read block 2779 len 20480
25236 078/300 Read block 203 len 24576
25237 079/300 Write block 2318 len 4096 data: 4
25238 080/300 Read block 4842 len 24576
25239 081/300 Read block 2498 len 8192
25240 082/300 Read block 1456 len 16384
25241 083/300 Write block 199 len 40960 data: 4 4 4 4 3 3 2 2 2 2
25242 084/300 Write block 1659 len 8192 data: 6 7
25243 085/300 Read block 3252 len 32768
25244 086/300 Read block 4897 len 8192
25245 087/300 Read block 4232 len 8192
25246 088/300 Read block 988 len 24576
25247 089/300 Write block 1243 len 32768 data: 5 7 7 7 7 6 6 7
25248 090/300 Flush
25249 091/300 Read block 4574 len 12288
25250 092/300 Flush
25251 093/300 Write block 505 len 36864 data: 6 5 4 3 5 6 6 7 7
25252 094/300 Write block 1427 len 4096 data: 4
25253 095/300 Read block 4980 len 24576
25254 096/300 Read block 4826 len 32768
25255 097/300 Read block 1693 len 24576
25256 098/300 Write block 2720 len 16384 data: 3 3 3 4
25257 099/300 Read block 3295 len 12288
25258 100/300 Read block 4161 len 20480
25259 101/300 Write block 1145 len 8192 data: 6 5
25260 102/300 Read block 448 len 8192
25261 103/300 Write block 4680 len 24576 data: 10 9 7 6 6 5
25262 104/300 Read block 1268 len 40960
25263 105/300 Read block 4354 len 4096
25264 106/300 Read block 2568 len 36864
25265 107/300 Read block 1579 len 24576
25266 108/300 Read block 3186 len 16384
25267 109/300 Read block 364 len 40960
25268 110/300 Read block 3955 len 28672
25269 111/300 Flush
25270 112/300 Read block 3153 len 24576
25271 113/300 Write block 2970 len 24576 data: 6 5 5 4 4 5
25272 114/300 Read block 2083 len 24576
25273 115/300 Write block 340 len 32768 data: 9 6 7 7 8 8 6 7
25274 116/300 Flush
25275 117/300 Write block 1630 len 24576 data: 2 3 4 4 3 3
25276 118/300 Read block 4840 len 32768
25277 119/300 Write block 1849 len 28672 data: 7 7 8 9 8 8 10
25278 120/300 Write block 1231 len 24576 data: 8 8 9 10 10 7
25279 121/300 Write block 4932 len 28672 data: 6 6 5 4 5 6 6
25280 122/300 Write block 6 len 12288 data: 8 8 9
25281 123/300 Read block 221 len 12288
25282 124/300 Read block 2307 len 20480
25283 125/300 Write block 3165 len 28672 data: 5 8 8 8 7 7 7
25284 126/300 Write block 371 len 20480 data: 6 6 6 7 6
25285 127/300 Write block 4653 len 8192 data: 5 4
25286 128/300 Write block 2070 len 16384 data: 7 5 4 3
25287 129/300 Read block 4404 len 4096
25288 130/300 Read block 972 len 4096
25289 131/300 Read block 3034 len 36864
25290 132/300 Write block 3452 len 16384 data: 5 5 5 5
25291 133/300 Write block 1442 len 12288 data: 3 3 4
25292 134/300 Write block 127 len 36864 data: 7 6 6 7 4 4 4 4 3
25293 135/300 Write block 2762 len 8192 data: 4 5
25294 136/300 Flush
25295 137/300 Write block 3709 len 24576 data: 5 5 4 4 6 5
25296 138/300 Read block 2039 len 36864
25297 139/300 Write block 3090 len 4096 data: 7
25298 140/300 Write block 4323 len 36864 data: 4 5 5 5 6 6 5 5 7
25299 141/300 Write block 1025 len 12288 data: 4 5 4
25300 142/300 Read block 1929 len 36864
25301 143/300 Write block 4948 len 12288 data: 7 5 5
25302 144/300 Write block 1903 len 16384 data: 4 4 3 2
25303 145/300 Read block 3299 len 40960
25304 146/300 Read block 1561 len 4096
25305 147/300 Read block 4462 len 16384
25306 148/300 Write block 2782 len 40960 data: 3 3 3 3 3 3 3 3 3 3
25307 149/300 Write block 2871 len 8192 data: 3 3
25308 150/300 Flush
25309 151/300 Read block 1471 len 36864
25310 152/300 Flush
25311 153/300 Write block 137 len 28672 data: 6 6 6 7 7 6 7
25312 154/300 Read block 2674 len 4096
25313 155/300 Write block 935 len 24576 data: 3 3 4 4 4 4
25314 156/300 Read block 549 len 4096
25315 157/300 Read block 3643 len 20480
25316 158/300 Read block 4945 len 12288
25317 159/300 Flush
25318 160/300 Write block 4535 len 32768 data: 4 5 7 6 6 6 4 3
25319 161/300 Write block 2036 len 40960 data: 7 6 6 5 3 3 3 4 4 5
25320 162/300 Write block 1284 len 12288 data: 8 8 7
25321 163/300 Read block 1331 len 12288
25322 164/300 Flush
25323 165/300 Read block 2490 len 40960
25324 166/300 Read block 2114 len 12288
25325 167/300 Read block 1806 len 32768
25326 168/300 Write block 2577 len 36864 data: 3 3 3 3 4 3 3 3 4
25327 169/300 Read block 15 len 36864
25328 170/300 Flush
25329 171/300 Read block 3897 len 40960
25330 172/300 Write block 2888 len 8192 data: 9 7
25331 173/300 Read block 933 len 20480
25332 174/300 Read block 1753 len 36864
25333 175/300 Write block 2838 len 20480 data: 5 6 7 7 8
25334 176/300 Read block 3984 len 28672
25335 177/300 Write block 628 len 16384 data: 4 4 3 5
25336 178/300 Read block 3385 len 28672
25337 179/300 Write block 607 len 8192 data: 3 3
25338 180/300 Read block 3287 len 12288
25339 181/300 Read block 3359 len 8192
25340 182/300 Write block 4416 len 28672 data: 4 4 2 3 3 5 5
25341 183/300 Write block 1372 len 12288 data: 5 5 5
25342 184/300 Flush
25343 185/300 Write block 2127 len 40960 data: 2 2 2 2 2 2 2 2 3 6
25344 186/300 Write block 2685 len 36864 data: 3 5 5 5 6 6 6 6 6
25345 187/300 Write block 1263 len 32768 data: 6 5 4 4 4 5 5 6
25346 188/300 Read block 225 len 12288
25347 189/300 Write block 1909 len 28672 data: 3 3 6 2 3 4 4
25348 190/300 Read block 1038 len 16384
25349 191/300 Read block 3909 len 40960
25350 192/300 Read block 4661 len 12288
25351 193/300 Write block 1204 len 8192 data: 7 6
25352 194/300 Read block 2363 len 40960
25353 195/300 Write block 817 len 24576 data: 2 2 2 2 2 3
25354 196/300 Read block 4093 len 28672
25355 197/300 Write block 934 len 40960 data: 6 4 4 5 5 5 5 4 4 4
25356 198/300 Read block 2032 len 32768
25357 199/300 Read block 4709 len 32768
25358 200/300 Read block 3063 len 12288
25359 201/300 Write block 624 len 28672 data: 5 4 4 4 5 5 4
25360 202/300 Write block 4088 len 28672 data: 6 7 7 6 5 4 4
25361 203/300 Read block 4137 len 32768
25362 204/300 Write block 2189 len 28672 data: 3 2 3 3 3 3 3
25363 205/300 Read block 1147 len 4096
25364 206/300 Write block 2624 len 16384 data: 5 5 5 4
25365 207/300 Read block 4442 len 12288
25366 208/300 Read block 1813 len 8192
25367 209/300 Read block 4044 len 12288
25368 210/300 Write block 3554 len 8192 data: 5 4
25369 211/300 Read block 1593 len 36864
25370 212/300 Write block 161 len 24576 data: 7 5 6 6 6 6
25371 213/300 Write block 916 len 24576 data: 5 3 3 3 2 2
25372 214/300 Read block 3620 len 8192
25373 215/300 Write block 1427 len 36864 data: 5 3 3 3 2 2 2 2 2
25374 216/300 Read block 1957 len 24576
25375 217/300 Read block 912 len 8192
25376 218/300 Read block 1246 len 8192
25377 219/300 Read block 107 len 32768
25378 220/300 Read block 1907 len 32768
25379 221/300 Flush
25380 222/300 Read block 1314 len 8192
25381 223/300 Read block 2268 len 28672
25382 224/300 Write block 1788 len 4096 data: 7
25383 225/300 Flush
25384 226/300 Flush
25385 227/300 Read block 987 len 20480
25386 228/300 Read block 1488 len 40960
25387 229/300 Read block 3811 len 8192
25388 230/300 Read block 654 len 28672
25389 231/300 Write block 4411 len 32768 data: 4 4 2 2 3 5 5 3
25390 232/300 Read block 3008 len 36864
25391 233/300 Read block 1389 len 40960
25392 234/300 Read block 3192 len 20480
25393 235/300 Read block 927 len 16384
25394 236/300 Write block 4718 len 20480 data: 4 4 4 4 4
25395 237/300 Read block 2574 len 32768
25396 238/300 Read block 892 len 4096
25397 239/300 Write block 3149 len 16384 data: 4 3 3 3
25398 240/300 Read block 2410 len 40960
25399 241/300 Write block 2916 len 24576 data: 6 6 7 6 5 6
25400 242/300 Write block 4526 len 36864 data: 5 6 7 7 7 5 5 4 4
25401 243/300 Write block 2882 len 8192 data: 3 5
25402 244/300 Write block 872 len 20480 data: 4 4 6 7 6
25403 245/300 Write block 2808 len 8192 data: 6 8
25404 246/300 Read block 3160 len 4096
25405 247/300 Read block 2615 len 16384
25406 248/300 Write block 1696 len 8192 data: 5 5
25407 249/300 Write block 892 len 40960 data: 6 6 5 3 2 2 2 2 2 2
25408 250/300 Read block 566 len 28672
25409 251/300 Read block 4823 len 32768
25410 252/300 Read block 4901 len 12288
25411 253/300 Write block 1451 len 32768 data: 3 3 3 3 4 5 5 4
25412 254/300 Read block 2903 len 32768
25413 255/300 Read block 1619 len 12288
25414 256/300 Read block 837 len 28672
25415 257/300 Read block 1977 len 8192
25416 258/300 Read block 649 len 4096
25417 259/300 Flush
25418 260/300 Read block 4731 len 40960
25419 261/300 Read block 1101 len 4096
25420 262/300 Read block 264 len 8192
25421 263/300 Write block 44 len 4096 data: 6
25422 264/300 Read block 1516 len 8192
25423 265/300 Read block 2054 len 4096
25424 266/300 Read block 1742 len 32768
25425 267/300 Write block 1094 len 40960 data: 6 5 7 7 8 6 6 5 7 9
25426 268/300 Write block 498 len 24576 data: 5 5 6 4 5 5
25427 269/300 Write block 536 len 4096 data: 6
25428 270/300 Read block 3259 len 40960
25429 271/300 Write block 2318 len 20480 data: 5 3 4 5 5
25430 272/300 Write block 1631 len 12288 data: 4 5 5
25431 273/300 Write block 3349 len 40960 data: 5 5 7 6 5 4 4 3 3 4
25432 274/300 Write block 3564 len 28672 data: 7 7 5 4 6 7 7
25433 275/300 Read block 2747 len 40960
25434 276/300 Flush
25435 277/300 Read block 2158 len 20480
25436 278/300 Read block 1892 len 20480
25437 279/300 Read block 3667 len 28672
25438 280/300 Flush
25439 281/300 Write block 785 len 8192 data: 5 6
25440 282/300 Flush
25441 283/300 Read block 2983 len 8192
25442 284/300 Read block 839 len 32768
25443 285/300 Read block 1762 len 16384
25444 286/300 Read block 1022 len 12288
25445 287/300 Read block 4279 len 8192
25446 288/300 Read block 93 len 28672
25447 289/300 Write block 2552 len 12288 data: 4 3 3
25448 290/300 Read block 794 len 16384
25449 291/300 Write block 1729 len 40960 data: 6 6 5 5 5 5 5 4 4 3
25450 292/300 Flush
25451 293/300 Read block 663 len 28672
25452 294/300 Write block 3911 len 40960 data: 5 6 5 5 5 5 6 6 5 5
25453 295/300 Flush
25454 296/300 Flush
25455 297/300 Read block 1371 len 8192
25456 298/300 Read block 4234 len 20480
25457 299/300 Read block 1584 len 32768
25458 300/300 Read block 169 len 32768
25459 301/300 Write block 3330 len 28672 data: 6 7 5 4 4 4 4
25460 Main task has work to do, go find it
25461 got dsc Start(2)
25462 start 2
25463 Nov 26 22:09:54.016 INFO request completed, latency_us: 223, response_code: 204, uri: /start/cid/2, method: POST, req_id: 293c31ec-f745-472a-8d3a-01978ee21694, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
25464 Replay: started 2, returned:Ok(())
25465 ----------------------------------------------------------------
25466 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
25467 States: Active Active Offline
25468 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
25469 7642 Acked Write 3 Done Done Sent false
25470 7643 Acked Write 8 Done Done Sent false
25471 7644 Acked Read 10 Done Done Sent false
25472 7645 Acked Read 1 Done Done Sent false
25473 7646 Acked Read 5 Done Done Sent false
25474 7647 Acked Write 6 Done Done Sent false
25475 7648 Acked Read 4 Done Done Sent false
25476 7649 Acked Read 3 Done Done Sent false
25477 7650 Acked Flush 0 Done Done Sent false
25478 7651 Acked Read 10 Done Done Sent false
25479 7652 Acked Write 5 Done Done Sent false
25480 7653 Acked Read 2 Done Done Sent false
25481 7654 Acked Read 4 Done Done Sent false
25482 7655 Acked Flush 0 Done Done Sent false
25483 7656 Acked Write 6 Done Done Sent false
25484 7657 Acked Write 1 Done Done Sent false
25485 7658 Acked Flush 0 Done Done Sent false
25486 7659 Acked Write 10 Done Done Sent false
25487 7660 Acked Write 9 Done Done Sent false
25488 7661 Acked Write 5 Done Done Sent false
25489 7662 Acked Write 6 Done Done Sent false
25490 7663 Acked Write 9 Done Done Sent false
25491 7664 Acked Read 9 Done Done Sent false
25492 7665 Acked Flush 0 Done Done Sent false
25493 7666 Acked Read 8 Done Done Sent false
25494 7667 Acked Read 4 Done Done Sent false
25495 7668 Acked Write 2 Done Done Sent false
25496 7669 Acked Read 3 Done Done Sent false
25497 7670 Acked Read 7 Done Done Sent false
25498 7671 Acked Write 5 Done Done Sent false
25499 7672 Acked Write 8 Done Done Sent false
25500 7673 Acked Read 3 Done Done Sent false
25501 7674 Acked Read 6 Done Done Sent false
25502 7675 Acked Write 6 Done Done Sent false
25503 7676 Acked Read 8 Done Done Sent false
25504 7677 Acked Flush 0 Done Done Sent false
25505 7678 Acked Write 2 Done Done Sent false
25506 7679 Acked Flush 0 Done Done Sent false
25507 7680 Acked Write 5 Done Done Sent false
25508 7681 Acked Read 2 Done Done Sent false
25509 7682 Acked Read 4 Done Done Sent false
25510 7683 Acked Write 7 Done Done Sent false
25511 7684 Acked Read 1 Done Done Sent false
25512 7685 Acked Write 1 Done Done Sent false
25513 7686 Acked Write 4 Done Done Sent false
25514 7687 Acked Write 10 Done Done Sent false
25515 7688 Acked Write 10 Done Done Sent false
25516 7689 Acked Write 1 Done Done Sent false
25517 7690 Acked Flush 0 Done Done Sent false
25518 7691 Acked Write 9 Done Done Sent false
25519 7692 Acked Write 5 Done Done Sent false
25520 7693 Acked Write 4 Done Done Sent false
25521 7694 Acked Read 7 Done Done Sent false
25522 7695 Acked Write 9 Done Done Sent false
25523 7696 Acked Read 6 Done Done Sent false
25524 7697 Acked Read 10 Done Done Sent false
25525 7698 Acked Write 6 Done Done Sent false
25526 7699 Acked Write 4 Done Done Sent false
25527 7700 Acked Read 1 Done Done Sent false
25528 7701 Acked Write 7 Done Done Sent false
25529 7702 Acked Write 5 Done Done Sent false
25530 7703 Acked Read 2 Done Done Sent false
25531 7704 Acked Write 9 Done Done Sent false
25532 7705 Acked Read 4 Done Done Sent false
25533 7706 Acked Read 8 Done Done Sent false
25534 7707 Acked Read 10 Done Done Sent false
25535 7708 Acked Read 3 Done Done Sent false
25536 7709 Acked Flush 0 Done Done Sent false
25537 7710 Acked Read 2 Done Done Sent false
25538 7711 Acked Read 5 Done Done Sent false
25539 7712 Acked Read 6 Done Done Sent false
25540 7713 Acked Read 10 Done Done Sent false
25541 7714 Acked Read 4 Done Done Sent false
25542 7715 Acked Write 1 Done Done Sent false
25543 7716 Acked Flush 0 Done Done Sent false
25544 7717 Acked Write 3 Done Done Sent false
25545 7718 Acked Read 5 Done Done Sent false
25546 7719 Acked Read 6 Done Done Sent false
25547 7720 Acked Write 1 Done Done Sent false
25548 7721 Acked Read 6 Done Done Sent false
25549 7722 Acked Read 2 Done Done Sent false
25550 7723 Acked Read 4 Done Done Sent false
25551 7724 Acked Write 10 Done Done Sent false
25552 7725 Acked Write 2 Done Done Sent false
25553 7726 Acked Read 8 Done Done Sent false
25554 7727 Acked Read 2 Done Done Sent false
25555 7728 Acked Read 2 Done Done Sent false
25556 7729 Acked Read 6 Done Done Sent false
25557 7730 Acked Write 8 Done Done Sent false
25558 7731 Acked Flush 0 Done Done Sent false
25559 7732 Acked Read 3 Done Done Sent false
25560 7733 Acked Flush 0 Done Done Sent false
25561 7734 Acked Write 9 Done Done Sent false
25562 7735 Acked Write 1 Done Done Sent false
25563 7736 Acked Read 6 Done Done Sent false
25564 7737 Acked Read 8 Done Done Sent false
25565 7738 Acked Read 6 Done Done Sent false
25566 7739 Acked Write 4 Done Done Sent false
25567 7740 Acked Read 3 Done Done Sent false
25568 7741 Acked Read 5 Done Done Sent false
25569 7742 Acked Write 2 Done Done Sent false
25570 7743 Acked Read 2 Done Done Sent false
25571 7744 Acked Write 6 Done Done Sent false
25572 7745 Acked Read 10 Done Done Sent false
25573 7746 Acked Read 1 Done Done Sent false
25574 7747 Acked Read 9 Done Done Sent false
25575 7748 Acked Read 6 Done Done Sent false
25576 7749 Acked Read 4 Done Done Sent false
25577 7750 Acked Read 10 Done Done Sent false
25578 7751 Acked Read 7 Done Done Sent false
25579 7752 Acked Flush 0 Done Done Sent false
25580 7753 Acked Read 6 Done Done Sent false
25581 7754 Acked Write 6 Done Done Sent false
25582 7755 Acked Read 6 Done Done Sent false
25583 7756 Acked Write 8 Done Done Sent false
25584 7757 Acked Flush 0 Done Done Sent false
25585 7758 Acked Write 6 Done Done Sent false
25586 7759 Acked Read 8 Done Done Sent false
25587 7760 Acked Write 7 Done Done Sent false
25588 7761 Acked Write 6 Done Done Sent false
25589 7762 Acked Write 7 Done Done Sent false
25590 7763 Acked Write 3 Done Done Sent false
25591 7764 Acked Read 3 Done Done Sent false
25592 7765 Acked Read 5 Done Done Sent false
25593 7766 Acked Write 7 Done Done Sent false
25594 7767 Acked Write 5 Done Done Sent false
25595 7768 Acked Write 2 Done Done Sent false
25596 7769 Acked Write 4 Done Done Sent false
25597 7770 Acked Read 1 Done Done Sent false
25598 7771 Acked Read 1 Done Done Sent false
25599 7772 Acked Read 9 Done Done Sent false
25600 7773 Acked Write 4 Done Done Sent false
25601 7774 Acked Write 3 Done Done Sent false
25602 7775 Acked Write 9 Done Done Sent false
25603 7776 Acked Write 2 Done Done Sent false
25604 7777 Acked Flush 0 Done Done Sent false
25605 7778 Acked Write 6 Done Done Sent false
25606 7779 Acked Read 9 Done Done Sent false
25607 7780 Acked Write 1 Done Done Sent false
25608 7781 Acked Write 9 Done Done Sent false
25609 7782 Acked Write 3 Done Done Sent false
25610 7783 Acked Read 9 Done Done Sent false
25611 7784 Acked Write 3 Done Done Sent false
25612 7785 Acked Write 4 Done Done Sent false
25613 7786 Acked Read 10 Done Done Sent false
25614 7787 Acked Read 1 Done Done Sent false
25615 7788 Acked Read 4 Done Done Sent false
25616 7789 Acked Write 10 Done Done Sent false
25617 7790 Acked Write 2 Done Done Sent false
25618 7791 Acked Flush 0 Done Done Sent false
25619 7792 Acked Read 9 Done Done Sent false
25620 7793 Acked Flush 0 Done Done Sent false
25621 7794 Acked Write 7 Done Done Sent false
25622 7795 Acked Read 1 Done Done Sent false
25623 7796 Acked Write 6 Done Done Sent false
25624 7797 Acked Read 1 Done Done Sent false
25625 7798 Acked Read 5 Done Done Sent false
25626 7799 Acked Read 3 Done Done Sent false
25627 7800 Acked Flush 0 Done Done Sent false
25628 7801 Acked Write 8 Done Done Sent false
25629 7802 Acked Write 10 Done Done Sent false
25630 7803 Acked Write 3 Done Done Sent false
25631 7804 Acked Read 3 Done Done Sent false
25632 7805 Acked Flush 0 Done Done Sent false
25633 7806 Acked Read 10 Done Done Sent false
25634 7807 Acked Read 3 Done Done Sent false
25635 7808 Acked Read 8 Done Done Sent false
25636 7809 Acked Write 9 Done Done Sent false
25637 7810 Acked Read 9 Done Done Sent false
25638 7811 Acked Flush 0 Done Done Sent false
25639 7812 Acked Read 10 Done Done Sent false
25640 7813 Acked Write 2 Done Done Sent false
25641 7814 Acked Read 5 Done Done Sent false
25642 7815 Acked Read 9 Done Done Sent false
25643 7816 Acked Write 5 Done Done Sent false
25644 7817 Acked Read 7 Done Done Sent false
25645 7818 Acked Write 4 Done Done Sent false
25646 7819 Acked Read 7 Done Done Sent false
25647 7820 Acked Write 2 Done Done Sent false
25648 7821 Acked Read 3 Done Done Sent false
25649 7822 Acked Read 2 Done Done Sent false
25650 7823 Acked Write 7 Done Done Sent false
25651 7824 Acked Write 3 Done Done Sent false
25652 7825 Acked Flush 0 Done Done Sent false
25653 7826 Acked Write 10 Done Done Sent false
25654 7827 Acked Write 9 Done Done Sent false
25655 7828 Acked Write 8 Done Done Sent false
25656 7829 Acked Read 3 Done Done Sent false
25657 7830 Acked Write 7 Done Done Sent false
25658 7831 Acked Read 4 Done Done Sent false
25659 7832 Acked Read 10 Done Done Sent false
25660 7833 Acked Read 3 Done Done Sent false
25661 7834 Acked Write 2 Done Done Sent false
25662 7835 Acked Read 10 Done Done Sent false
25663 7836 Acked Write 6 Done Done Sent false
25664 7837 Acked Read 7 Done Done Sent false
25665 7838 Acked Write 10 Done Done Sent false
25666 7839 Acked Read 8 Done Done Sent false
25667 7840 Acked Read 8 Done Done Sent false
25668 7841 Acked Read 3 Done Done Sent false
25669 7842 Acked Write 7 Done Done Sent false
25670 7843 Acked Write 7 Done Done Sent false
25671 7844 Acked Read 8 Done Done Sent false
25672 7845 Acked Write 7 Done Done Sent false
25673 7846 Acked Read 1 Done Done Sent false
25674 7847 Acked Write 4 Done Done Sent false
25675 7848 Acked Read 3 Done Done Sent false
25676 7849 Acked Read 2 Done Done Sent false
25677 7850 Acked Read 3 Done Done Sent false
25678 7851 Acked Write 2 Done Done Sent false
25679 7852 Acked Read 9 Done Done Sent false
25680 7853 Acked Write 6 Done Done Sent false
25681 7854 Acked Write 6 Done Done Sent false
25682 7855 Acked Read 2 Done Done Sent false
25683 7856 Acked Write 9 Done Done Sent false
25684 7857 Acked Read 6 Done Done Sent false
25685 7858 Acked Read 2 Done Done Sent false
25686 7859 Acked Read 2 Done Done Sent false
25687 7860 Acked Read 8 Done Done Sent false
25688 7861 Acked Read 8 Done Done Sent false
25689 7862 Acked Flush 0 Done Done Sent false
25690 7863 Acked Read 2 Done Done Sent false
25691 7864 Acked Read 7 Done Done Sent false
25692 7865 Acked Write 1 Done Done Sent false
25693 7866 Acked Flush 0 Done Done Sent false
25694 7867 Acked Flush 0 Done Done Sent false
25695 7868 Acked Read 5 Done Done Sent false
25696 7869 Acked Read 10 Done Done Sent false
25697 7870 Acked Read 2 Done Done Sent false
25698 7871 Acked Read 7 Done Done Sent false
25699 7872 Acked Write 8 Done Done Sent false
25700 7873 Acked Read 9 Done Done Sent false
25701 7874 Acked Read 10 Done Done Sent false
25702 7875 Acked Read 5 Done Done Sent false
25703 7876 Acked Read 4 Done Done Sent false
25704 7877 Acked Write 5 Done Done Sent false
25705 7878 Acked Read 8 Done Done Sent false
25706 7879 Acked Read 1 Done Done Sent false
25707 7880 Acked Write 4 Done Done Sent false
25708 7881 Acked Read 10 Done Done Sent false
25709 7882 Acked Write 6 Done Done Sent false
25710 7883 Acked Write 9 Done Done Sent false
25711 7884 Acked Write 2 Done Done Sent false
25712 7885 Acked Write 5 Done Done Sent false
25713 7886 Acked Write 2 Done Done Sent false
25714 7887 Acked Read 1 Done Done Sent false
25715 7888 Acked Read 4 Done Done Sent false
25716 7889 Acked Write 2 Done Done Sent false
25717 7890 Acked Write 10 Done Done Sent false
25718 7891 Acked Read 7 Done Done Sent false
25719 7892 Acked Read 8 Done Done Sent false
25720 7893 Acked Read 3 Done Done Sent false
25721 7894 Acked Write 8 Done Done Sent false
25722 7895 Acked Read 8 Done Done Sent false
25723 7896 Acked Read 3 Done Done Sent false
25724 7897 Acked Read 7 Done Done Sent false
25725 7898 Acked Read 2 Done Done Sent false
25726 7899 Acked Read 1 Done Done Sent false
25727 7900 Acked Flush 0 Done Done Sent false
25728 7901 Acked Read 10 Done Done Sent false
25729 7902 Acked Read 1 Done Done Sent false
25730 7903 Acked Read 2 Done Done Sent false
25731 7904 Acked Write 1 Done Done Sent false
25732 7905 Acked Read 2 Done Done Sent false
25733 7906 Acked Read 1 Done Done Sent false
25734 7907 Acked Read 8 Done Done Sent false
25735 7908 Acked Write 10 Done Done Sent false
25736 7909 Acked Write 6 Done Done Sent false
25737 7910 Acked Write 1 Done Done Sent false
25738 7911 Acked Read 10 Done Done Sent false
25739 7912 Acked Write 5 Done Done Sent false
25740 7913 Acked Write 3 Done Done Sent false
25741 7914 Acked Write 10 Done Done Sent false
25742 7915 Acked Write 7 Done Done Sent false
25743 7916 Acked Read 10 Done Done Sent false
25744 7917 Acked Flush 0 Done Done Sent false
25745 7918 Acked Read 5 Done Done Sent false
25746 7919 Acked Read 5 Done Done Sent false
25747 7920 Acked Read 7 Done Done Sent false
25748 7921 Acked Flush 0 Done Done Sent false
25749 7922 Acked Write 2 Done Done Sent false
25750 7923 Acked Flush 0 Done Done Sent false
25751 7924 Acked Read 2 Done Done Sent false
25752 7925 Acked Read 8 Done Done Sent false
25753 7926 Acked Read 4 Done Done Sent false
25754 7927 Acked Read 3 Done Done Sent false
25755 7928 Acked Read 2 Done Done Sent false
25756 7929 Acked Read 7 Done Done Sent false
25757 7930 Acked Write 3 Done Done Sent false
25758 7931 Acked Read 4 Done Done Sent false
25759 7932 Acked Write 10 Done Done Sent false
25760 7933 Acked Flush 0 Done Done Sent false
25761 7934 Acked Read 7 Done Done Sent false
25762 7935 Acked Write 10 Done Done Sent false
25763 7936 Acked Flush 0 Done Done Sent false
25764 7937 Acked Flush 0 Done Done Sent false
25765 7938 Acked Read 2 Done Done Sent false
25766 7939 Acked Read 5 Done Done Sent false
25767 7940 Acked Read 8 Done Done Sent false
25768 7941 Acked Read 8 Done Done Sent false
25769 7942 Acked Write 7 Done Done Sent false
25770 STATES DS:0 DS:1 DS:2 TOTAL
25771 Sent 0 0 301 301
25772 Done 301 301 0 602
25773 Skipped 0 0 0 0
25774 Error 0 0 0 0
25775 Last Flush: 7937 7937 7641
25776 Downstairs last five completed: 7641 7640 7639 7638 7637
25777 Upstairs last five completed: 7942 7941 7940 7939 7938
25778 CLIENT: Up:0 ds:301 act:2
25779 ----------------------------------------------------------------
25780 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
25781 States: Active Active Offline
25782 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
25783 7642 Acked Write 3 Done Done Sent false
25784 7643 Acked Write 8 Done Done Sent false
25785 7644 Acked Read 10 Done Done Sent false
25786 7645 Acked Read 1 Done Done Sent false
25787 7646 Acked Read 5 Done Done Sent false
25788 7647 Acked Write 6 Done Done Sent false
25789 7648 Acked Read 4 Done Done Sent false
25790 7649 Acked Read 3 Done Done Sent false
25791 7650 Acked Flush 0 Done Done Sent false
25792 7651 Acked Read 10 Done Done Sent false
25793 7652 Acked Write 5 Done Done Sent false
25794 7653 Acked Read 2 Done Done Sent false
25795 7654 Acked Read 4 Done Done Sent false
25796 7655 Acked Flush 0 Done Done Sent false
25797 7656 Acked Write 6 Done Done Sent false
25798 7657 Acked Write 1 Done Done Sent false
25799 7658 Acked Flush 0 Done Done Sent false
25800 7659 Acked Write 10 Done Done Sent false
25801 7660 Acked Write 9 Done Done Sent false
25802 7661 Acked Write 5 Done Done Sent false
25803 7662 Acked Write 6 Done Done Sent false
25804 7663 Acked Write 9 Done Done Sent false
25805 7664 Acked Read 9 Done Done Sent false
25806 7665 Acked Flush 0 Done Done Sent false
25807 7666 Acked Read 8 Done Done Sent false
25808 7667 Acked Read 4 Done Done Sent false
25809 7668 Acked Write 2 Done Done Sent false
25810 7669 Acked Read 3 Done Done Sent false
25811 7670 Acked Read 7 Done Done Sent false
25812 7671 Acked Write 5 Done Done Sent false
25813 7672 Acked Write 8 Done Done Sent false
25814 7673 Acked Read 3 Done Done Sent false
25815 7674 Acked Read 6 Done Done Sent false
25816 7675 Acked Write 6 Done Done Sent false
25817 7676 Acked Read 8 Done Done Sent false
25818 7677 Acked Flush 0 Done Done Sent false
25819 7678 Acked Write 2 Done Done Sent false
25820 7679 Acked Flush 0 Done Done Sent false
25821 7680 Acked Write 5 Done Done Sent false
25822 7681 Acked Read 2 Done Done Sent false
25823 7682 Acked Read 4 Done Done Sent false
25824 7683 Acked Write 7 Done Done Sent false
25825 7684 Acked Read 1 Done Done Sent false
25826 7685 Acked Write 1 Done Done Sent false
25827 7686 Acked Write 4 Done Done Sent false
25828 7687 Acked Write 10 Done Done Sent false
25829 7688 Acked Write 10 Done Done Sent false
25830 7689 Acked Write 1 Done Done Sent false
25831 7690 Acked Flush 0 Done Done Sent false
25832 7691 Acked Write 9 Done Done Sent false
25833 7692 Acked Write 5 Done Done Sent false
25834 7693 Acked Write 4 Done Done Sent false
25835 7694 Acked Read 7 Done Done Sent false
25836 7695 Acked Write 9 Done Done Sent false
25837 7696 Acked Read 6 Done Done Sent false
25838 7697 Acked Read 10 Done Done Sent false
25839 7698 Acked Write 6 Done Done Sent false
25840 7699 Acked Write 4 Done Done Sent false
25841 7700 Acked Read 1 Done Done Sent false
25842 7701 Acked Write 7 Done Done Sent false
25843 7702 Acked Write 5 Done Done Sent false
25844 7703 Acked Read 2 Done Done Sent false
25845 7704 Acked Write 9 Done Done Sent false
25846 7705 Acked Read 4 Done Done Sent false
25847 7706 Acked Read 8 Done Done Sent false
25848 7707 Acked Read 10 Done Done Sent false
25849 7708 Acked Read 3 Done Done Sent false
25850 7709 Acked Flush 0 Done Done Sent false
25851 7710 Acked Read 2 Done Done Sent false
25852 7711 Acked Read 5 Done Done Sent false
25853 7712 Acked Read 6 Done Done Sent false
25854 7713 Acked Read 10 Done Done Sent false
25855 7714 Acked Read 4 Done Done Sent false
25856 7715 Acked Write 1 Done Done Sent false
25857 7716 Acked Flush 0 Done Done Sent false
25858 7717 Acked Write 3 Done Done Sent false
25859 7718 Acked Read 5 Done Done Sent false
25860 7719 Acked Read 6 Done Done Sent false
25861 7720 Acked Write 1 Done Done Sent false
25862 7721 Acked Read 6 Done Done Sent false
25863 7722 Acked Read 2 Done Done Sent false
25864 7723 Acked Read 4 Done Done Sent false
25865 7724 Acked Write 10 Done Done Sent false
25866 7725 Acked Write 2 Done Done Sent false
25867 7726 Acked Read 8 Done Done Sent false
25868 7727 Acked Read 2 Done Done Sent false
25869 7728 Acked Read 2 Done Done Sent false
25870 7729 Acked Read 6 Done Done Sent false
25871 7730 Acked Write 8 Done Done Sent false
25872 7731 Acked Flush 0 Done Done Sent false
25873 7732 Acked Read 3 Done Done Sent false
25874 7733 Acked Flush 0 Done Done Sent false
25875 7734 Acked Write 9 Done Done Sent false
25876 7735 Acked Write 1 Done Done Sent false
25877 7736 Acked Read 6 Done Done Sent false
25878 7737 Acked Read 8 Done Done Sent false
25879 7738 Acked Read 6 Done Done Sent false
25880 7739 Acked Write 4 Done Done Sent false
25881 7740 Acked Read 3 Done Done Sent false
25882 7741 Acked Read 5 Done Done Sent false
25883 7742 Acked Write 2 Done Done Sent false
25884 7743 Acked Read 2 Done Done Sent false
25885 7744 Acked Write 6 Done Done Sent false
25886 7745 Acked Read 10 Done Done Sent false
25887 7746 Acked Read 1 Done Done Sent false
25888 7747 Acked Read 9 Done Done Sent false
25889 7748 Acked Read 6 Done Done Sent false
25890 7749 Acked Read 4 Done Done Sent false
25891 7750 Acked Read 10 Done Done Sent false
25892 7751 Acked Read 7 Done Done Sent false
25893 7752 Acked Flush 0 Done Done Sent false
25894 7753 Acked Read 6 Done Done Sent false
25895 7754 Acked Write 6 Done Done Sent false
25896 7755 Acked Read 6 Done Done Sent false
25897 7756 Acked Write 8 Done Done Sent false
25898 7757 Acked Flush 0 Done Done Sent false
25899 7758 Acked Write 6 Done Done Sent false
25900 7759 Acked Read 8 Done Done Sent false
25901 7760 Acked Write 7 Done Done Sent false
25902 7761 Acked Write 6 Done Done Sent false
25903 7762 Acked Write 7 Done Done Sent false
25904 7763 Acked Write 3 Done Done Sent false
25905 7764 Acked Read 3 Done Done Sent false
25906 7765 Acked Read 5 Done Done Sent false
25907 7766 Acked Write 7 Done Done Sent false
25908 7767 Acked Write 5 Done Done Sent false
25909 7768 Acked Write 2 Done Done Sent false
25910 7769 Acked Write 4 Done Done Sent false
25911 7770 Acked Read 1 Done Done Sent false
25912 7771 Acked Read 1 Done Done Sent false
25913 7772 Acked Read 9 Done Done Sent false
25914 7773 Acked Write 4 Done Done Sent false
25915 7774 Acked Write 3 Done Done Sent false
25916 7775 Acked Write 9 Done Done Sent false
25917 7776 Acked Write 2 Done Done Sent false
25918 7777 Acked Flush 0 Done Done Sent false
25919 7778 Acked Write 6 Done Done Sent false
25920 7779 Acked Read 9 Done Done Sent false
25921 7780 Acked Write 1 Done Done Sent false
25922 7781 Acked Write 9 Done Done Sent false
25923 7782 Acked Write 3 Done Done Sent false
25924 7783 Acked Read 9 Done Done Sent false
25925 7784 Acked Write 3 Done Done Sent false
25926 7785 Acked Write 4 Done Done Sent false
25927 7786 Acked Read 10 Done Done Sent false
25928 7787 Acked Read 1 Done Done Sent false
25929 7788 Acked Read 4 Done Done Sent false
25930 7789 Acked Write 10 Done Done Sent false
25931 7790 Acked Write 2 Done Done Sent false
25932 7791 Acked Flush 0 Done Done Sent false
25933 7792 Acked Read 9 Done Done Sent false
25934 7793 Acked Flush 0 Done Done Sent false
25935 7794 Acked Write 7 Done Done Sent false
25936 7795 Acked Read 1 Done Done Sent false
25937 7796 Acked Write 6 Done Done Sent false
25938 7797 Acked Read 1 Done Done Sent false
25939 7798 Acked Read 5 Done Done Sent false
25940 7799 Acked Read 3 Done Done Sent false
25941 7800 Acked Flush 0 Done Done Sent false
25942 7801 Acked Write 8 Done Done Sent false
25943 7802 Acked Write 10 Done Done Sent false
25944 7803 Acked Write 3 Done Done Sent false
25945 7804 Acked Read 3 Done Done Sent false
25946 7805 Acked Flush 0 Done Done Sent false
25947 7806 Acked Read 10 Done Done Sent false
25948 7807 Acked Read 3 Done Done Sent false
25949 7808 Acked Read 8 Done Done Sent false
25950 7809 Acked Write 9 Done Done Sent false
25951 7810 Acked Read 9 Done Done Sent false
25952 7811 Acked Flush 0 Done Done Sent false
25953 7812 Acked Read 10 Done Done Sent false
25954 7813 Acked Write 2 Done Done Sent false
25955 7814 Acked Read 5 Done Done Sent false
25956 7815 Acked Read 9 Done Done Sent false
25957 7816 Acked Write 5 Done Done Sent false
25958 7817 Acked Read 7 Done Done Sent false
25959 7818 Acked Write 4 Done Done Sent false
25960 7819 Acked Read 7 Done Done Sent false
25961 7820 Acked Write 2 Done Done Sent false
25962 7821 Acked Read 3 Done Done Sent false
25963 7822 Acked Read 2 Done Done Sent false
25964 7823 Acked Write 7 Done Done Sent false
25965 7824 Acked Write 3 Done Done Sent false
25966 7825 Acked Flush 0 Done Done Sent false
25967 7826 Acked Write 10 Done Done Sent false
25968 7827 Acked Write 9 Done Done Sent false
25969 7828 Acked Write 8 Done Done Sent false
25970 7829 Acked Read 3 Done Done Sent false
25971 7830 Acked Write 7 Done Done Sent false
25972 7831 Acked Read 4 Done Done Sent false
25973 7832 Acked Read 10 Done Done Sent false
25974 7833 Acked Read 3 Done Done Sent false
25975 7834 Acked Write 2 Done Done Sent false
25976 7835 Acked Read 10 Done Done Sent false
25977 7836 Acked Write 6 Done Done Sent false
25978 7837 Acked Read 7 Done Done Sent false
25979 7838 Acked Write 10 Done Done Sent false
25980 7839 Acked Read 8 Done Done Sent false
25981 7840 Acked Read 8 Done Done Sent false
25982 7841 Acked Read 3 Done Done Sent false
25983 7842 Acked Write 7 Done Done Sent false
25984 7843 Acked Write 7 Done Done Sent false
25985 7844 Acked Read 8 Done Done Sent false
25986 7845 Acked Write 7 Done Done Sent false
25987 7846 Acked Read 1 Done Done Sent false
25988 7847 Acked Write 4 Done Done Sent false
25989 7848 Acked Read 3 Done Done Sent false
25990 7849 Acked Read 2 Done Done Sent false
25991 7850 Acked Read 3 Done Done Sent false
25992 7851 Acked Write 2 Done Done Sent false
25993 7852 Acked Read 9 Done Done Sent false
25994 7853 Acked Write 6 Done Done Sent false
25995 7854 Acked Write 6 Done Done Sent false
25996 7855 Acked Read 2 Done Done Sent false
25997 7856 Acked Write 9 Done Done Sent false
25998 7857 Acked Read 6 Done Done Sent false
25999 7858 Acked Read 2 Done Done Sent false
26000 7859 Acked Read 2 Done Done Sent false
26001 7860 Acked Read 8 Done Done Sent false
26002 7861 Acked Read 8 Done Done Sent false
26003 7862 Acked Flush 0 Done Done Sent false
26004 7863 Acked Read 2 Done Done Sent false
26005 7864 Acked Read 7 Done Done Sent false
26006 7865 Acked Write 1 Done Done Sent false
26007 7866 Acked Flush 0 Done Done Sent false
26008 7867 Acked Flush 0 Done Done Sent false
26009 7868 Acked Read 5 Done Done Sent false
26010 7869 Acked Read 10 Done Done Sent false
26011 7870 Acked Read 2 Done Done Sent false
26012 7871 Acked Read 7 Done Done Sent false
26013 7872 Acked Write 8 Done Done Sent false
26014 7873 Acked Read 9 Done Done Sent false
26015 7874 Acked Read 10 Done Done Sent false
26016 7875 Acked Read 5 Done Done Sent false
26017 7876 Acked Read 4 Done Done Sent false
26018 7877 Acked Write 5 Done Done Sent false
26019 7878 Acked Read 8 Done Done Sent false
26020 7879 Acked Read 1 Done Done Sent false
26021 7880 Acked Write 4 Done Done Sent false
26022 7881 Acked Read 10 Done Done Sent false
26023 7882 Acked Write 6 Done Done Sent false
26024 7883 Acked Write 9 Done Done Sent false
26025 7884 Acked Write 2 Done Done Sent false
26026 7885 Acked Write 5 Done Done Sent false
26027 7886 Acked Write 2 Done Done Sent false
26028 7887 Acked Read 1 Done Done Sent false
26029 7888 Acked Read 4 Done Done Sent false
26030 7889 Acked Write 2 Done Done Sent false
26031 7890 Acked Write 10 Done Done Sent false
26032 7891 Acked Read 7 Done Done Sent false
26033 7892 Acked Read 8 Done Done Sent false
26034 7893 Acked Read 3 Done Done Sent false
26035 7894 Acked Write 8 Done Done Sent false
26036 7895 Acked Read 8 Done Done Sent false
26037 7896 Acked Read 3 Done Done Sent false
26038 7897 Acked Read 7 Done Done Sent false
26039 7898 Acked Read 2 Done Done Sent false
26040 7899 Acked Read 1 Done Done Sent false
26041 7900 Acked Flush 0 Done Done Sent false
26042 7901 Acked Read 10 Done Done Sent false
26043 7902 Acked Read 1 Done Done Sent false
26044 7903 Acked Read 2 Done Done Sent false
26045 7904 Acked Write 1 Done Done Sent false
26046 7905 Acked Read 2 Done Done Sent false
26047 7906 Acked Read 1 Done Done Sent false
26048 7907 Acked Read 8 Done Done Sent false
26049 7908 Acked Write 10 Done Done Sent false
26050 7909 Acked Write 6 Done Done Sent false
26051 7910 Acked Write 1 Done Done Sent false
26052 7911 Acked Read 10 Done Done Sent false
26053 7912 Acked Write 5 Done Done Sent false
26054 7913 Acked Write 3 Done Done Sent false
26055 7914 Acked Write 10 Done Done Sent false
26056 7915 Acked Write 7 Done Done Sent false
26057 7916 Acked Read 10 Done Done Sent false
26058 7917 Acked Flush 0 Done Done Sent false
26059 7918 Acked Read 5 Done Done Sent false
26060 7919 Acked Read 5 Done Done Sent false
26061 7920 Acked Read 7 Done Done Sent false
26062 7921 Acked Flush 0 Done Done Sent false
26063 7922 Acked Write 2 Done Done Sent false
26064 7923 Acked Flush 0 Done Done Sent false
26065 7924 Acked Read 2 Done Done Sent false
26066 7925 Acked Read 8 Done Done Sent false
26067 7926 Acked Read 4 Done Done Sent false
26068 7927 Acked Read 3 Done Done Sent false
26069 7928 Acked Read 2 Done Done Sent false
26070 7929 Acked Read 7 Done Done Sent false
26071 7930 Acked Write 3 Done Done Sent false
26072 7931 Acked Read 4 Done Done Sent false
26073 7932 Acked Write 10 Done Done Sent false
26074 7933 Acked Flush 0 Done Done Sent false
26075 7934 Acked Read 7 Done Done Sent false
26076 7935 Acked Write 10 Done Done Sent false
26077 7936 Acked Flush 0 Done Done Sent false
26078 7937 Acked Flush 0 Done Done Sent false
26079 7938 Acked Read 2 Done Done Sent false
26080 7939 Acked Read 5 Done Done Sent false
26081 7940 Acked Read 8 Done Done Sent false
26082 7941 Acked Read 8 Done Done Sent false
26083 7942 Acked Write 7 Done Done Sent false
26084 7943 Acked Flush 0 Done Done Sent false
26085 STATES DS:0 DS:1 DS:2 TOTAL
26086 Sent 0 0 302 302
26087 Done 302 302 0 604
26088 Skipped 0 0 0 0
26089 Error 0 0 0 0
26090 Last Flush: 7943 7943 7641
26091 Downstairs last five completed: 7641 7640 7639 7638 7637
26092 Upstairs last five completed: 7943 7942 7941 7940 7939
26093 CLIENT: Up:0 ds:302 act:2
260942024-11-26T22:09:59.780ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
260952024-11-26T22:09:59.780ZWARNcrucible: ds_connection connect to 127.0.0.1:8830 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
260962024-11-26T22:09:59.780ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
260972024-11-26T22:09:59.780ZWARNcrucible: downstairs task for 2 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
260982024-11-26T22:09:59.780ZINFOcrucible: client task is exiting = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
26099 [8830] got start action
26100 [8830] I am going to restart
26101 Starting downstairs at port 8830
26102 Make output file at "/tmp/dsc/downstairs-8830.txt"
26103 [8830][2] reports Starting
26104 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1226)
26105 [8830][2] reports Running
26106 ----------------------------------------------------------------
26107 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
26108 States: Active Active Offline
26109 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
26110 7642 Acked Write 3 Done Done Sent false
26111 7643 Acked Write 8 Done Done Sent false
26112 7644 Acked Read 10 Done Done Sent false
26113 7645 Acked Read 1 Done Done Sent false
26114 7646 Acked Read 5 Done Done Sent false
26115 7647 Acked Write 6 Done Done Sent false
26116 7648 Acked Read 4 Done Done Sent false
26117 7649 Acked Read 3 Done Done Sent false
26118 7650 Acked Flush 0 Done Done Sent false
26119 7651 Acked Read 10 Done Done Sent false
26120 7652 Acked Write 5 Done Done Sent false
26121 7653 Acked Read 2 Done Done Sent false
26122 7654 Acked Read 4 Done Done Sent false
26123 7655 Acked Flush 0 Done Done Sent false
26124 7656 Acked Write 6 Done Done Sent false
26125 7657 Acked Write 1 Done Done Sent false
26126 7658 Acked Flush 0 Done Done Sent false
26127 7659 Acked Write 10 Done Done Sent false
26128 7660 Acked Write 9 Done Done Sent false
26129 7661 Acked Write 5 Done Done Sent false
26130 7662 Acked Write 6 Done Done Sent false
26131 7663 Acked Write 9 Done Done Sent false
26132 7664 Acked Read 9 Done Done Sent false
26133 7665 Acked Flush 0 Done Done Sent false
26134 7666 Acked Read 8 Done Done Sent false
26135 7667 Acked Read 4 Done Done Sent false
26136 7668 Acked Write 2 Done Done Sent false
26137 7669 Acked Read 3 Done Done Sent false
26138 7670 Acked Read 7 Done Done Sent false
26139 7671 Acked Write 5 Done Done Sent false
26140 7672 Acked Write 8 Done Done Sent false
26141 7673 Acked Read 3 Done Done Sent false
26142 7674 Acked Read 6 Done Done Sent false
26143 7675 Acked Write 6 Done Done Sent false
26144 7676 Acked Read 8 Done Done Sent false
26145 7677 Acked Flush 0 Done Done Sent false
26146 7678 Acked Write 2 Done Done Sent false
26147 7679 Acked Flush 0 Done Done Sent false
26148 7680 Acked Write 5 Done Done Sent false
26149 7681 Acked Read 2 Done Done Sent false
26150 7682 Acked Read 4 Done Done Sent false
26151 7683 Acked Write 7 Done Done Sent false
26152 7684 Acked Read 1 Done Done Sent false
26153 7685 Acked Write 1 Done Done Sent false
26154 7686 Acked Write 4 Done Done Sent false
26155 7687 Acked Write 10 Done Done Sent false
26156 7688 Acked Write 10 Done Done Sent false
26157 7689 Acked Write 1 Done Done Sent false
26158 7690 Acked Flush 0 Done Done Sent false
26159 7691 Acked Write 9 Done Done Sent false
26160 7692 Acked Write 5 Done Done Sent false
26161 7693 Acked Write 4 Done Done Sent false
26162 7694 Acked Read 7 Done Done Sent false
26163 7695 Acked Write 9 Done Done Sent false
26164 7696 Acked Read 6 Done Done Sent false
26165 7697 Acked Read 10 Done Done Sent false
26166 7698 Acked Write 6 Done Done Sent false
26167 7699 Acked Write 4 Done Done Sent false
26168 7700 Acked Read 1 Done Done Sent false
26169 7701 Acked Write 7 Done Done Sent false
26170 7702 Acked Write 5 Done Done Sent false
26171 7703 Acked Read 2 Done Done Sent false
26172 7704 Acked Write 9 Done Done Sent false
26173 7705 Acked Read 4 Done Done Sent false
26174 7706 Acked Read 8 Done Done Sent false
26175 7707 Acked Read 10 Done Done Sent false
26176 7708 Acked Read 3 Done Done Sent false
26177 7709 Acked Flush 0 Done Done Sent false
26178 7710 Acked Read 2 Done Done Sent false
26179 7711 Acked Read 5 Done Done Sent false
26180 7712 Acked Read 6 Done Done Sent false
26181 7713 Acked Read 10 Done Done Sent false
26182 7714 Acked Read 4 Done Done Sent false
26183 7715 Acked Write 1 Done Done Sent false
26184 7716 Acked Flush 0 Done Done Sent false
26185 7717 Acked Write 3 Done Done Sent false
26186 7718 Acked Read 5 Done Done Sent false
26187 7719 Acked Read 6 Done Done Sent false
26188 7720 Acked Write 1 Done Done Sent false
26189 7721 Acked Read 6 Done Done Sent false
26190 7722 Acked Read 2 Done Done Sent false
26191 7723 Acked Read 4 Done Done Sent false
26192 7724 Acked Write 10 Done Done Sent false
26193 7725 Acked Write 2 Done Done Sent false
26194 7726 Acked Read 8 Done Done Sent false
26195 7727 Acked Read 2 Done Done Sent false
26196 7728 Acked Read 2 Done Done Sent false
26197 7729 Acked Read 6 Done Done Sent false
26198 7730 Acked Write 8 Done Done Sent false
26199 7731 Acked Flush 0 Done Done Sent false
26200 7732 Acked Read 3 Done Done Sent false
26201 7733 Acked Flush 0 Done Done Sent false
26202 7734 Acked Write 9 Done Done Sent false
26203 7735 Acked Write 1 Done Done Sent false
26204 7736 Acked Read 6 Done Done Sent false
26205 7737 Acked Read 8 Done Done Sent false
26206 7738 Acked Read 6 Done Done Sent false
26207 7739 Acked Write 4 Done Done Sent false
26208 7740 Acked Read 3 Done Done Sent false
26209 7741 Acked Read 5 Done Done Sent false
26210 7742 Acked Write 2 Done Done Sent false
26211 7743 Acked Read 2 Done Done Sent false
26212 7744 Acked Write 6 Done Done Sent false
26213 7745 Acked Read 10 Done Done Sent false
26214 7746 Acked Read 1 Done Done Sent false
26215 7747 Acked Read 9 Done Done Sent false
26216 7748 Acked Read 6 Done Done Sent false
26217 7749 Acked Read 4 Done Done Sent false
26218 7750 Acked Read 10 Done Done Sent false
26219 7751 Acked Read 7 Done Done Sent false
26220 7752 Acked Flush 0 Done Done Sent false
26221 7753 Acked Read 6 Done Done Sent false
26222 7754 Acked Write 6 Done Done Sent false
26223 7755 Acked Read 6 Done Done Sent false
26224 7756 Acked Write 8 Done Done Sent false
26225 7757 Acked Flush 0 Done Done Sent false
26226 7758 Acked Write 6 Done Done Sent false
26227 7759 Acked Read 8 Done Done Sent false
26228 7760 Acked Write 7 Done Done Sent false
26229 7761 Acked Write 6 Done Done Sent false
26230 7762 Acked Write 7 Done Done Sent false
26231 7763 Acked Write 3 Done Done Sent false
26232 7764 Acked Read 3 Done Done Sent false
26233 7765 Acked Read 5 Done Done Sent false
26234 7766 Acked Write 7 Done Done Sent false
26235 7767 Acked Write 5 Done Done Sent false
26236 7768 Acked Write 2 Done Done Sent false
26237 7769 Acked Write 4 Done Done Sent false
26238 7770 Acked Read 1 Done Done Sent false
26239 7771 Acked Read 1 Done Done Sent false
26240 7772 Acked Read 9 Done Done Sent false
26241 7773 Acked Write 4 Done Done Sent false
26242 7774 Acked Write 3 Done Done Sent false
26243 7775 Acked Write 9 Done Done Sent false
26244 7776 Acked Write 2 Done Done Sent false
26245 7777 Acked Flush 0 Done Done Sent false
26246 7778 Acked Write 6 Done Done Sent false
26247 7779 Acked Read 9 Done Done Sent false
26248 7780 Acked Write 1 Done Done Sent false
26249 7781 Acked Write 9 Done Done Sent false
26250 7782 Acked Write 3 Done Done Sent false
26251 7783 Acked Read 9 Done Done Sent false
26252 7784 Acked Write 3 Done Done Sent false
26253 7785 Acked Write 4 Done Done Sent false
26254 7786 Acked Read 10 Done Done Sent false
26255 7787 Acked Read 1 Done Done Sent false
26256 7788 Acked Read 4 Done Done Sent false
26257 7789 Acked Write 10 Done Done Sent false
26258 7790 Acked Write 2 Done Done Sent false
26259 7791 Acked Flush 0 Done Done Sent false
26260 7792 Acked Read 9 Done Done Sent false
26261 7793 Acked Flush 0 Done Done Sent false
26262 7794 Acked Write 7 Done Done Sent false
26263 7795 Acked Read 1 Done Done Sent false
26264 7796 Acked Write 6 Done Done Sent false
26265 7797 Acked Read 1 Done Done Sent false
26266 7798 Acked Read 5 Done Done Sent false
26267 7799 Acked Read 3 Done Done Sent false
26268 7800 Acked Flush 0 Done Done Sent false
26269 7801 Acked Write 8 Done Done Sent false
26270 7802 Acked Write 10 Done Done Sent false
26271 7803 Acked Write 3 Done Done Sent false
26272 7804 Acked Read 3 Done Done Sent false
26273 7805 Acked Flush 0 Done Done Sent false
26274 7806 Acked Read 10 Done Done Sent false
26275 7807 Acked Read 3 Done Done Sent false
26276 7808 Acked Read 8 Done Done Sent false
26277 7809 Acked Write 9 Done Done Sent false
26278 7810 Acked Read 9 Done Done Sent false
26279 7811 Acked Flush 0 Done Done Sent false
26280 7812 Acked Read 10 Done Done Sent false
26281 7813 Acked Write 2 Done Done Sent false
26282 7814 Acked Read 5 Done Done Sent false
26283 7815 Acked Read 9 Done Done Sent false
26284 7816 Acked Write 5 Done Done Sent false
26285 7817 Acked Read 7 Done Done Sent false
26286 7818 Acked Write 4 Done Done Sent false
26287 7819 Acked Read 7 Done Done Sent false
26288 7820 Acked Write 2 Done Done Sent false
26289 7821 Acked Read 3 Done Done Sent false
26290 7822 Acked Read 2 Done Done Sent false
26291 7823 Acked Write 7 Done Done Sent false
26292 7824 Acked Write 3 Done Done Sent false
26293 7825 Acked Flush 0 Done Done Sent false
26294 7826 Acked Write 10 Done Done Sent false
26295 7827 Acked Write 9 Done Done Sent false
26296 7828 Acked Write 8 Done Done Sent false
26297 7829 Acked Read 3 Done Done Sent false
26298 7830 Acked Write 7 Done Done Sent false
26299 7831 Acked Read 4 Done Done Sent false
26300 7832 Acked Read 10 Done Done Sent false
26301 7833 Acked Read 3 Done Done Sent false
26302 7834 Acked Write 2 Done Done Sent false
26303 7835 Acked Read 10 Done Done Sent false
26304 7836 Acked Write 6 Done Done Sent false
26305 7837 Acked Read 7 Done Done Sent false
26306 7838 Acked Write 10 Done Done Sent false
26307 7839 Acked Read 8 Done Done Sent false
26308 7840 Acked Read 8 Done Done Sent false
26309 7841 Acked Read 3 Done Done Sent false
26310 7842 Acked Write 7 Done Done Sent false
26311 7843 Acked Write 7 Done Done Sent false
26312 7844 Acked Read 8 Done Done Sent false
26313 7845 Acked Write 7 Done Done Sent false
26314 7846 Acked Read 1 Done Done Sent false
26315 7847 Acked Write 4 Done Done Sent false
26316 7848 Acked Read 3 Done Done Sent false
26317 7849 Acked Read 2 Done Done Sent false
26318 7850 Acked Read 3 Done Done Sent false
26319 7851 Acked Write 2 Done Done Sent false
26320 7852 Acked Read 9 Done Done Sent false
26321 7853 Acked Write 6 Done Done Sent false
26322 7854 Acked Write 6 Done Done Sent false
26323 7855 Acked Read 2 Done Done Sent false
26324 7856 Acked Write 9 Done Done Sent false
26325 7857 Acked Read 6 Done Done Sent false
26326 7858 Acked Read 2 Done Done Sent false
26327 7859 Acked Read 2 Done Done Sent false
26328 7860 Acked Read 8 Done Done Sent false
26329 7861 Acked Read 8 Done Done Sent false
26330 7862 Acked Flush 0 Done Done Sent false
26331 7863 Acked Read 2 Done Done Sent false
26332 7864 Acked Read 7 Done Done Sent false
26333 7865 Acked Write 1 Done Done Sent false
26334 7866 Acked Flush 0 Done Done Sent false
26335 7867 Acked Flush 0 Done Done Sent false
26336 7868 Acked Read 5 Done Done Sent false
26337 7869 Acked Read 10 Done Done Sent false
26338 7870 Acked Read 2 Done Done Sent false
26339 7871 Acked Read 7 Done Done Sent false
26340 7872 Acked Write 8 Done Done Sent false
26341 7873 Acked Read 9 Done Done Sent false
26342 7874 Acked Read 10 Done Done Sent false
26343 7875 Acked Read 5 Done Done Sent false
26344 7876 Acked Read 4 Done Done Sent false
26345 7877 Acked Write 5 Done Done Sent false
26346 7878 Acked Read 8 Done Done Sent false
26347 7879 Acked Read 1 Done Done Sent false
26348 7880 Acked Write 4 Done Done Sent false
26349 7881 Acked Read 10 Done Done Sent false
26350 7882 Acked Write 6 Done Done Sent false
26351 7883 Acked Write 9 Done Done Sent false
26352 7884 Acked Write 2 Done Done Sent false
26353 7885 Acked Write 5 Done Done Sent false
26354 7886 Acked Write 2 Done Done Sent false
26355 7887 Acked Read 1 Done Done Sent false
26356 7888 Acked Read 4 Done Done Sent false
26357 7889 Acked Write 2 Done Done Sent false
26358 7890 Acked Write 10 Done Done Sent false
26359 7891 Acked Read 7 Done Done Sent false
26360 7892 Acked Read 8 Done Done Sent false
26361 7893 Acked Read 3 Done Done Sent false
26362 7894 Acked Write 8 Done Done Sent false
26363 7895 Acked Read 8 Done Done Sent false
26364 7896 Acked Read 3 Done Done Sent false
26365 7897 Acked Read 7 Done Done Sent false
26366 7898 Acked Read 2 Done Done Sent false
26367 7899 Acked Read 1 Done Done Sent false
26368 7900 Acked Flush 0 Done Done Sent false
26369 7901 Acked Read 10 Done Done Sent false
26370 7902 Acked Read 1 Done Done Sent false
26371 7903 Acked Read 2 Done Done Sent false
26372 7904 Acked Write 1 Done Done Sent false
26373 7905 Acked Read 2 Done Done Sent false
26374 7906 Acked Read 1 Done Done Sent false
26375 7907 Acked Read 8 Done Done Sent false
26376 7908 Acked Write 10 Done Done Sent false
26377 7909 Acked Write 6 Done Done Sent false
26378 7910 Acked Write 1 Done Done Sent false
26379 7911 Acked Read 10 Done Done Sent false
26380 7912 Acked Write 5 Done Done Sent false
26381 7913 Acked Write 3 Done Done Sent false
26382 7914 Acked Write 10 Done Done Sent false
26383 7915 Acked Write 7 Done Done Sent false
26384 7916 Acked Read 10 Done Done Sent false
26385 7917 Acked Flush 0 Done Done Sent false
26386 7918 Acked Read 5 Done Done Sent false
26387 7919 Acked Read 5 Done Done Sent false
26388 7920 Acked Read 7 Done Done Sent false
26389 7921 Acked Flush 0 Done Done Sent false
26390 7922 Acked Write 2 Done Done Sent false
26391 7923 Acked Flush 0 Done Done Sent false
26392 7924 Acked Read 2 Done Done Sent false
26393 7925 Acked Read 8 Done Done Sent false
26394 7926 Acked Read 4 Done Done Sent false
26395 7927 Acked Read 3 Done Done Sent false
26396 7928 Acked Read 2 Done Done Sent false
26397 7929 Acked Read 7 Done Done Sent false
26398 7930 Acked Write 3 Done Done Sent false
26399 7931 Acked Read 4 Done Done Sent false
26400 7932 Acked Write 10 Done Done Sent false
26401 7933 Acked Flush 0 Done Done Sent false
26402 7934 Acked Read 7 Done Done Sent false
26403 7935 Acked Write 10 Done Done Sent false
26404 7936 Acked Flush 0 Done Done Sent false
26405 7937 Acked Flush 0 Done Done Sent false
26406 7938 Acked Read 2 Done Done Sent false
26407 7939 Acked Read 5 Done Done Sent false
26408 7940 Acked Read 8 Done Done Sent false
26409 7941 Acked Read 8 Done Done Sent false
26410 7942 Acked Write 7 Done Done Sent false
26411 7943 Acked Flush 0 Done Done Sent false
26412 STATES DS:0 DS:1 DS:2 TOTAL
26413 Sent 0 0 302 302
26414 Done 302 302 0 604
26415 Skipped 0 0 0 0
26416 Error 0 0 0 0
26417 Last Flush: 7943 7943 7641
26418 Downstairs last five completed: 7641 7640 7639 7638 7637
26419 Upstairs last five completed: 7943 7942 7941 7940 7939
26420 CLIENT: Up:0 ds:302 act:2
26421 ----------------------------------------------------------------
26422 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
26423 States: Active Active Offline
26424 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
26425 7642 Acked Write 3 Done Done Sent false
26426 7643 Acked Write 8 Done Done Sent false
26427 7644 Acked Read 10 Done Done Sent false
26428 7645 Acked Read 1 Done Done Sent false
26429 7646 Acked Read 5 Done Done Sent false
26430 7647 Acked Write 6 Done Done Sent false
26431 7648 Acked Read 4 Done Done Sent false
26432 7649 Acked Read 3 Done Done Sent false
26433 7650 Acked Flush 0 Done Done Sent false
26434 7651 Acked Read 10 Done Done Sent false
26435 7652 Acked Write 5 Done Done Sent false
26436 7653 Acked Read 2 Done Done Sent false
26437 7654 Acked Read 4 Done Done Sent false
26438 7655 Acked Flush 0 Done Done Sent false
26439 7656 Acked Write 6 Done Done Sent false
26440 7657 Acked Write 1 Done Done Sent false
26441 7658 Acked Flush 0 Done Done Sent false
26442 7659 Acked Write 10 Done Done Sent false
26443 7660 Acked Write 9 Done Done Sent false
26444 7661 Acked Write 5 Done Done Sent false
26445 7662 Acked Write 6 Done Done Sent false
26446 7663 Acked Write 9 Done Done Sent false
26447 7664 Acked Read 9 Done Done Sent false
26448 7665 Acked Flush 0 Done Done Sent false
26449 7666 Acked Read 8 Done Done Sent false
26450 7667 Acked Read 4 Done Done Sent false
26451 7668 Acked Write 2 Done Done Sent false
26452 7669 Acked Read 3 Done Done Sent false
26453 7670 Acked Read 7 Done Done Sent false
26454 7671 Acked Write 5 Done Done Sent false
26455 7672 Acked Write 8 Done Done Sent false
26456 7673 Acked Read 3 Done Done Sent false
26457 7674 Acked Read 6 Done Done Sent false
26458 7675 Acked Write 6 Done Done Sent false
26459 7676 Acked Read 8 Done Done Sent false
26460 7677 Acked Flush 0 Done Done Sent false
26461 7678 Acked Write 2 Done Done Sent false
26462 7679 Acked Flush 0 Done Done Sent false
26463 7680 Acked Write 5 Done Done Sent false
26464 7681 Acked Read 2 Done Done Sent false
26465 7682 Acked Read 4 Done Done Sent false
26466 7683 Acked Write 7 Done Done Sent false
26467 7684 Acked Read 1 Done Done Sent false
26468 7685 Acked Write 1 Done Done Sent false
26469 7686 Acked Write 4 Done Done Sent false
26470 7687 Acked Write 10 Done Done Sent false
26471 7688 Acked Write 10 Done Done Sent false
26472 7689 Acked Write 1 Done Done Sent false
26473 7690 Acked Flush 0 Done Done Sent false
26474 7691 Acked Write 9 Done Done Sent false
26475 7692 Acked Write 5 Done Done Sent false
26476 7693 Acked Write 4 Done Done Sent false
26477 7694 Acked Read 7 Done Done Sent false
26478 7695 Acked Write 9 Done Done Sent false
26479 7696 Acked Read 6 Done Done Sent false
26480 7697 Acked Read 10 Done Done Sent false
26481 7698 Acked Write 6 Done Done Sent false
26482 7699 Acked Write 4 Done Done Sent false
26483 7700 Acked Read 1 Done Done Sent false
26484 7701 Acked Write 7 Done Done Sent false
26485 7702 Acked Write 5 Done Done Sent false
26486 7703 Acked Read 2 Done Done Sent false
26487 7704 Acked Write 9 Done Done Sent false
26488 7705 Acked Read 4 Done Done Sent false
26489 7706 Acked Read 8 Done Done Sent false
26490 7707 Acked Read 10 Done Done Sent false
26491 7708 Acked Read 3 Done Done Sent false
26492 7709 Acked Flush 0 Done Done Sent false
26493 7710 Acked Read 2 Done Done Sent false
26494 7711 Acked Read 5 Done Done Sent false
26495 7712 Acked Read 6 Done Done Sent false
26496 7713 Acked Read 10 Done Done Sent false
26497 7714 Acked Read 4 Done Done Sent false
26498 7715 Acked Write 1 Done Done Sent false
26499 7716 Acked Flush 0 Done Done Sent false
26500 7717 Acked Write 3 Done Done Sent false
26501 7718 Acked Read 5 Done Done Sent false
26502 7719 Acked Read 6 Done Done Sent false
26503 7720 Acked Write 1 Done Done Sent false
26504 7721 Acked Read 6 Done Done Sent false
26505 7722 Acked Read 2 Done Done Sent false
26506 7723 Acked Read 4 Done Done Sent false
26507 7724 Acked Write 10 Done Done Sent false
26508 7725 Acked Write 2 Done Done Sent false
26509 7726 Acked Read 8 Done Done Sent false
26510 7727 Acked Read 2 Done Done Sent false
26511 7728 Acked Read 2 Done Done Sent false
26512 7729 Acked Read 6 Done Done Sent false
26513 7730 Acked Write 8 Done Done Sent false
26514 7731 Acked Flush 0 Done Done Sent false
26515 7732 Acked Read 3 Done Done Sent false
26516 7733 Acked Flush 0 Done Done Sent false
26517 7734 Acked Write 9 Done Done Sent false
26518 7735 Acked Write 1 Done Done Sent false
26519 7736 Acked Read 6 Done Done Sent false
26520 7737 Acked Read 8 Done Done Sent false
26521 7738 Acked Read 6 Done Done Sent false
26522 7739 Acked Write 4 Done Done Sent false
26523 7740 Acked Read 3 Done Done Sent false
26524 7741 Acked Read 5 Done Done Sent false
26525 7742 Acked Write 2 Done Done Sent false
26526 7743 Acked Read 2 Done Done Sent false
26527 7744 Acked Write 6 Done Done Sent false
26528 7745 Acked Read 10 Done Done Sent false
26529 7746 Acked Read 1 Done Done Sent false
26530 7747 Acked Read 9 Done Done Sent false
26531 7748 Acked Read 6 Done Done Sent false
26532 7749 Acked Read 4 Done Done Sent false
26533 7750 Acked Read 10 Done Done Sent false
26534 7751 Acked Read 7 Done Done Sent false
26535 7752 Acked Flush 0 Done Done Sent false
26536 7753 Acked Read 6 Done Done Sent false
26537 7754 Acked Write 6 Done Done Sent false
26538 7755 Acked Read 6 Done Done Sent false
26539 7756 Acked Write 8 Done Done Sent false
26540 7757 Acked Flush 0 Done Done Sent false
26541 7758 Acked Write 6 Done Done Sent false
26542 7759 Acked Read 8 Done Done Sent false
26543 7760 Acked Write 7 Done Done Sent false
26544 7761 Acked Write 6 Done Done Sent false
26545 7762 Acked Write 7 Done Done Sent false
26546 7763 Acked Write 3 Done Done Sent false
26547 7764 Acked Read 3 Done Done Sent false
26548 7765 Acked Read 5 Done Done Sent false
26549 7766 Acked Write 7 Done Done Sent false
26550 7767 Acked Write 5 Done Done Sent false
26551 7768 Acked Write 2 Done Done Sent false
26552 7769 Acked Write 4 Done Done Sent false
26553 7770 Acked Read 1 Done Done Sent false
26554 7771 Acked Read 1 Done Done Sent false
26555 7772 Acked Read 9 Done Done Sent false
26556 7773 Acked Write 4 Done Done Sent false
26557 7774 Acked Write 3 Done Done Sent false
26558 7775 Acked Write 9 Done Done Sent false
26559 7776 Acked Write 2 Done Done Sent false
26560 7777 Acked Flush 0 Done Done Sent false
26561 7778 Acked Write 6 Done Done Sent false
26562 7779 Acked Read 9 Done Done Sent false
26563 7780 Acked Write 1 Done Done Sent false
26564 7781 Acked Write 9 Done Done Sent false
26565 7782 Acked Write 3 Done Done Sent false
26566 7783 Acked Read 9 Done Done Sent false
26567 7784 Acked Write 3 Done Done Sent false
26568 7785 Acked Write 4 Done Done Sent false
26569 7786 Acked Read 10 Done Done Sent false
26570 7787 Acked Read 1 Done Done Sent false
26571 7788 Acked Read 4 Done Done Sent false
26572 7789 Acked Write 10 Done Done Sent false
26573 7790 Acked Write 2 Done Done Sent false
26574 7791 Acked Flush 0 Done Done Sent false
26575 7792 Acked Read 9 Done Done Sent false
26576 7793 Acked Flush 0 Done Done Sent false
26577 7794 Acked Write 7 Done Done Sent false
26578 7795 Acked Read 1 Done Done Sent false
26579 7796 Acked Write 6 Done Done Sent false
26580 7797 Acked Read 1 Done Done Sent false
26581 7798 Acked Read 5 Done Done Sent false
26582 7799 Acked Read 3 Done Done Sent false
26583 7800 Acked Flush 0 Done Done Sent false
26584 7801 Acked Write 8 Done Done Sent false
26585 7802 Acked Write 10 Done Done Sent false
26586 7803 Acked Write 3 Done Done Sent false
26587 7804 Acked Read 3 Done Done Sent false
26588 7805 Acked Flush 0 Done Done Sent false
26589 7806 Acked Read 10 Done Done Sent false
26590 7807 Acked Read 3 Done Done Sent false
26591 7808 Acked Read 8 Done Done Sent false
26592 7809 Acked Write 9 Done Done Sent false
26593 7810 Acked Read 9 Done Done Sent false
26594 7811 Acked Flush 0 Done Done Sent false
26595 7812 Acked Read 10 Done Done Sent false
26596 7813 Acked Write 2 Done Done Sent false
26597 7814 Acked Read 5 Done Done Sent false
26598 7815 Acked Read 9 Done Done Sent false
26599 7816 Acked Write 5 Done Done Sent false
26600 7817 Acked Read 7 Done Done Sent false
26601 7818 Acked Write 4 Done Done Sent false
26602 7819 Acked Read 7 Done Done Sent false
26603 7820 Acked Write 2 Done Done Sent false
26604 7821 Acked Read 3 Done Done Sent false
26605 7822 Acked Read 2 Done Done Sent false
26606 7823 Acked Write 7 Done Done Sent false
26607 7824 Acked Write 3 Done Done Sent false
26608 7825 Acked Flush 0 Done Done Sent false
26609 7826 Acked Write 10 Done Done Sent false
26610 7827 Acked Write 9 Done Done Sent false
26611 7828 Acked Write 8 Done Done Sent false
26612 7829 Acked Read 3 Done Done Sent false
26613 7830 Acked Write 7 Done Done Sent false
26614 7831 Acked Read 4 Done Done Sent false
26615 7832 Acked Read 10 Done Done Sent false
26616 7833 Acked Read 3 Done Done Sent false
26617 7834 Acked Write 2 Done Done Sent false
26618 7835 Acked Read 10 Done Done Sent false
26619 7836 Acked Write 6 Done Done Sent false
26620 7837 Acked Read 7 Done Done Sent false
26621 7838 Acked Write 10 Done Done Sent false
26622 7839 Acked Read 8 Done Done Sent false
26623 7840 Acked Read 8 Done Done Sent false
26624 7841 Acked Read 3 Done Done Sent false
26625 7842 Acked Write 7 Done Done Sent false
26626 7843 Acked Write 7 Done Done Sent false
26627 7844 Acked Read 8 Done Done Sent false
26628 7845 Acked Write 7 Done Done Sent false
26629 7846 Acked Read 1 Done Done Sent false
26630 7847 Acked Write 4 Done Done Sent false
26631 7848 Acked Read 3 Done Done Sent false
26632 7849 Acked Read 2 Done Done Sent false
26633 7850 Acked Read 3 Done Done Sent false
26634 7851 Acked Write 2 Done Done Sent false
26635 7852 Acked Read 9 Done Done Sent false
26636 7853 Acked Write 6 Done Done Sent false
26637 7854 Acked Write 6 Done Done Sent false
26638 7855 Acked Read 2 Done Done Sent false
26639 7856 Acked Write 9 Done Done Sent false
26640 7857 Acked Read 6 Done Done Sent false
26641 7858 Acked Read 2 Done Done Sent false
26642 7859 Acked Read 2 Done Done Sent false
26643 7860 Acked Read 8 Done Done Sent false
26644 7861 Acked Read 8 Done Done Sent false
26645 7862 Acked Flush 0 Done Done Sent false
26646 7863 Acked Read 2 Done Done Sent false
26647 7864 Acked Read 7 Done Done Sent false
26648 7865 Acked Write 1 Done Done Sent false
26649 7866 Acked Flush 0 Done Done Sent false
26650 7867 Acked Flush 0 Done Done Sent false
26651 7868 Acked Read 5 Done Done Sent false
26652 7869 Acked Read 10 Done Done Sent false
26653 7870 Acked Read 2 Done Done Sent false
26654 7871 Acked Read 7 Done Done Sent false
26655 7872 Acked Write 8 Done Done Sent false
26656 7873 Acked Read 9 Done Done Sent false
26657 7874 Acked Read 10 Done Done Sent false
26658 7875 Acked Read 5 Done Done Sent false
26659 7876 Acked Read 4 Done Done Sent false
26660 7877 Acked Write 5 Done Done Sent false
26661 7878 Acked Read 8 Done Done Sent false
26662 7879 Acked Read 1 Done Done Sent false
26663 7880 Acked Write 4 Done Done Sent false
26664 7881 Acked Read 10 Done Done Sent false
26665 7882 Acked Write 6 Done Done Sent false
26666 7883 Acked Write 9 Done Done Sent false
26667 7884 Acked Write 2 Done Done Sent false
26668 7885 Acked Write 5 Done Done Sent false
26669 7886 Acked Write 2 Done Done Sent false
26670 7887 Acked Read 1 Done Done Sent false
26671 7888 Acked Read 4 Done Done Sent false
26672 7889 Acked Write 2 Done Done Sent false
26673 7890 Acked Write 10 Done Done Sent false
26674 7891 Acked Read 7 Done Done Sent false
26675 7892 Acked Read 8 Done Done Sent false
26676 7893 Acked Read 3 Done Done Sent false
26677 7894 Acked Write 8 Done Done Sent false
26678 7895 Acked Read 8 Done Done Sent false
26679 7896 Acked Read 3 Done Done Sent false
26680 7897 Acked Read 7 Done Done Sent false
26681 7898 Acked Read 2 Done Done Sent false
26682 7899 Acked Read 1 Done Done Sent false
26683 7900 Acked Flush 0 Done Done Sent false
26684 7901 Acked Read 10 Done Done Sent false
26685 7902 Acked Read 1 Done Done Sent false
26686 7903 Acked Read 2 Done Done Sent false
26687 7904 Acked Write 1 Done Done Sent false
26688 7905 Acked Read 2 Done Done Sent false
26689 7906 Acked Read 1 Done Done Sent false
26690 7907 Acked Read 8 Done Done Sent false
26691 7908 Acked Write 10 Done Done Sent false
26692 7909 Acked Write 6 Done Done Sent false
26693 7910 Acked Write 1 Done Done Sent false
26694 7911 Acked Read 10 Done Done Sent false
26695 7912 Acked Write 5 Done Done Sent false
26696 7913 Acked Write 3 Done Done Sent false
26697 7914 Acked Write 10 Done Done Sent false
26698 7915 Acked Write 7 Done Done Sent false
26699 7916 Acked Read 10 Done Done Sent false
26700 7917 Acked Flush 0 Done Done Sent false
26701 7918 Acked Read 5 Done Done Sent false
26702 7919 Acked Read 5 Done Done Sent false
26703 7920 Acked Read 7 Done Done Sent false
26704 7921 Acked Flush 0 Done Done Sent false
26705 7922 Acked Write 2 Done Done Sent false
26706 7923 Acked Flush 0 Done Done Sent false
26707 7924 Acked Read 2 Done Done Sent false
26708 7925 Acked Read 8 Done Done Sent false
26709 7926 Acked Read 4 Done Done Sent false
26710 7927 Acked Read 3 Done Done Sent false
26711 7928 Acked Read 2 Done Done Sent false
26712 7929 Acked Read 7 Done Done Sent false
26713 7930 Acked Write 3 Done Done Sent false
26714 7931 Acked Read 4 Done Done Sent false
26715 7932 Acked Write 10 Done Done Sent false
26716 7933 Acked Flush 0 Done Done Sent false
26717 7934 Acked Read 7 Done Done Sent false
26718 7935 Acked Write 10 Done Done Sent false
26719 7936 Acked Flush 0 Done Done Sent false
26720 7937 Acked Flush 0 Done Done Sent false
26721 7938 Acked Read 2 Done Done Sent false
26722 7939 Acked Read 5 Done Done Sent false
26723 7940 Acked Read 8 Done Done Sent false
26724 7941 Acked Read 8 Done Done Sent false
26725 7942 Acked Write 7 Done Done Sent false
26726 7943 Acked Flush 0 Done Done Sent false
26727 STATES DS:0 DS:1 DS:2 TOTAL
26728 Sent 0 0 302 302
26729 Done 302 302 0 604
26730 Skipped 0 0 0 0
26731 Error 0 0 0 0
26732 Last Flush: 7943 7943 7641
26733 Downstairs last five completed: 7641 7640 7639 7638 7637
26734 Upstairs last five completed: 7943 7942 7941 7940 7939
26735 CLIENT: Up:0 ds:302 act:2
267362024-11-26T22:10:09.781ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
267372024-11-26T22:10:09.781ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:46293) = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
267382024-11-26T22:10:09.781ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
267392024-11-26T22:10:09.782ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
267402024-11-26T22:10:09.782ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
267412024-11-26T22:10:09.782ZINFOcrucible: send last flush ID to this DS: 7641 = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
267422024-11-26T22:10:09.782ZINFOcrucible: Replied this last flush ID: 7641 = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
267432024-11-26T22:10:09.782ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
267442024-11-26T22:10:09.782ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
267452024-11-26T22:10:09.782ZINFOcrucible: [2] client re-new 302 jobs since flush 7641 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
267462024-11-26T22:10:09.783ZINFOcrucible: [2] Replayed 302 jobs since flush: 7641 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
26747 ----------------------------------------------------------------
26748 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
26749 Downstairs last five completed: 7943 7942 7941 7940 7939
26750 Upstairs last five completed: 7943 7942 7941 7940 7939
26751 CLIENT: Up:0 ds:0 act:3
26752 Replay: All jobs finished
26753 Main task has work to do, go find it
26754 got dsc Stop(2)
26755 stop 2
26756 [8830] Got stop action so:false kr:false
26757 Nov 26 22:10:10.025 INFO request completed, latency_us: 365, response_code: 204, uri: /stop/cid/2, method: POST, req_id: dfeec12f-7f33-4fc4-9a79-b8f2e221470a, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
267582024-11-26T22:10:10.026ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
267592024-11-26T22:10:10.026ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
267602024-11-26T22:10:10.026ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
26761 Nov 26 22:10:10.026 INFO request completed, latency_us: 167, response_code: 200, uri: /state/cid/2, method: GET, req_id: eed91352-95f6-45ce-9286-b9a48e169694, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
267622024-11-26T22:10:10.026ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
267632024-11-26T22:10:10.026ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
267642024-11-26T22:10:10.026ZINFOcrucible: client task is exiting = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
26765 [8830] Exited with: ExitStatus(unix_wait_status(9))
26766 [8830][2] reports Exit
26767 Nov 26 22:10:14.027 INFO request completed, latency_us: 215, response_code: 200, uri: /state/cid/2, method: GET, req_id: d847c646-1c1f-4ac8-9f7d-3e3c2b35e611, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
26768 001/300 Read block 4459 len 20480
26769 002/300 Write block 41 len 8192 data: 6 6
26770 003/300 Read block 4515 len 28672
26771 004/300 Read block 4369 len 20480
26772 005/300 Write block 3446 len 24576 data: 5 6 6 6 6 5
26773 006/300 Read block 4949 len 40960
26774 007/300 Write block 3024 len 28672 data: 6 5 4 2 3 3 4
26775 008/300 Write block 4984 len 12288 data: 8 6 6
26776 009/300 Write block 1298 len 40960 data: 4 3 3 3 4 5 6 6 6 6
26777 010/300 Read block 1284 len 8192
26778 011/300 Read block 3361 len 12288
26779 012/300 Read block 3427 len 40960
26780 013/300 Write block 1604 len 24576 data: 5 5 5 7 8 9
26781 014/300 Read block 4827 len 20480
26782 015/300 Write block 3780 len 20480 data: 6 6 7 6 7
26783 016/300 Read block 1999 len 24576
26784 017/300 Flush
26785 018/300 Read block 1162 len 40960
26786 019/300 Write block 260 len 24576 data: 2 4 4 4 3 3
26787 020/300 Flush
26788 021/300 Read block 37 len 28672
26789 022/300 Read block 763 len 4096
26790 023/300 Write block 902 len 20480 data: 2 3 3 3 4
26791 024/300 Write block 2719 len 28672 data: 5 4 4 4 5 3 3
26792 025/300 Read block 3390 len 8192
26793 026/300 Write block 857 len 16384 data: 5 5 5 6
26794 027/300 Write block 2592 len 40960 data: 7 8 7 7 7 7 7 7 6 5
26795 028/300 Read block 2776 len 4096
26796 029/300 Write block 2834 len 12288 data: 5 4 4
26797 030/300 Read block 4929 len 12288
26798 031/300 Read block 463 len 36864
26799 032/300 Write block 3939 len 12288 data: 2 3 6
26800 033/300 Write block 4913 len 8192 data: 2 2
26801 034/300 Read block 76 len 24576
26802 035/300 Read block 761 len 8192
26803 036/300 Read block 2620 len 28672
26804 037/300 Read block 4998 len 4096
26805 038/300 Read block 3775 len 36864
26806 039/300 Write block 242 len 16384 data: 6 6 5 3
26807 040/300 Read block 1540 len 32768
26808 041/300 Write block 440 len 28672 data: 4 4 4 4 5 5 5
26809 042/300 Read block 3721 len 8192
26810 043/300 Read block 1251 len 28672
26811 044/300 Read block 1211 len 12288
26812 045/300 Write block 2463 len 12288 data: 4 3 6
26813 046/300 Read block 3848 len 16384
26814 047/300 Write block 835 len 16384 data: 4 4 3 3
26815 048/300 Read block 556 len 20480
26816 049/300 Write block 945 len 4096 data: 6
26817 050/300 Flush
26818 051/300 Read block 844 len 28672
26819 052/300 Flush
26820 053/300 Read block 3762 len 40960
26821 054/300 Read block 1483 len 20480
26822 055/300 Read block 2886 len 28672
26823 056/300 Write block 2758 len 40960 data: 7 6 6 6 5 6 4 3 4 4
26824 057/300 Flush
26825 058/300 Read block 4465 len 24576
26826 059/300 Read block 4389 len 28672
26827 060/300 Write block 1143 len 24576 data: 5 7 7 6 6 5
26828 061/300 Flush
26829 062/300 Read block 4501 len 32768
26830 063/300 Flush
26831 064/300 Read block 2973 len 40960
26832 065/300 Write block 89 len 24576 data: 5 6 6 6 7 7
26833 066/300 Read block 1613 len 16384
26834 067/300 Read block 3452 len 20480
26835 068/300 Read block 2254 len 8192
26836 069/300 Write block 1431 len 32768 data: 3 3 3 3 3 2 2 2
26837 070/300 Flush
26838 071/300 Write block 1445 len 8192 data: 4 5
26839 072/300 Flush
26840 073/300 Read block 1275 len 40960
26841 074/300 Flush
26842 075/300 Read block 4024 len 36864
26843 076/300 Read block 4026 len 28672
26844 077/300 Read block 4916 len 8192
26845 078/300 Read block 3099 len 40960
26846 079/300 Read block 3990 len 36864
26847 080/300 Read block 3388 len 24576
26848 081/300 Flush
26849 082/300 Write block 1265 len 20480 data: 5 5 5 6 6
26850 083/300 Read block 1256 len 8192
26851 084/300 Write block 2652 len 40960 data: 2 2 3 3 3 3 4 4 3 3
26852 085/300 Read block 4184 len 24576
26853 086/300 Read block 2209 len 32768
26854 087/300 Write block 2023 len 28672 data: 3 4 5 6 4 4 4
26855 088/300 Write block 1508 len 40960 data: 4 3 2 2 2 2 2 2 3 2
26856 089/300 Read block 4579 len 8192
26857 090/300 Read block 1410 len 20480
26858 091/300 Write block 1570 len 28672 data: 5 5 6 6 6 6 6
26859 092/300 Write block 847 len 24576 data: 4 4 4 4 4 5
26860 093/300 Read block 2283 len 36864
26861 094/300 Write block 4823 len 8192 data: 5 5
26862 095/300 Read block 4590 len 32768
26863 096/300 Write block 4801 len 40960 data: 5 6 5 6 5 7 7 8 9 8
26864 097/300 Write block 1464 len 32768 data: 6 4 4 4 5 3 2 3
26865 098/300 Read block 797 len 32768
26866 099/300 Read block 3828 len 32768
26867 100/300 Write block 3316 len 24576 data: 8 8 8 8 6 6
26868 101/300 Read block 3148 len 40960
26869 102/300 Write block 1282 len 32768 data: 8 7 9 9 8 6 6 7
26870 103/300 Read block 2843 len 16384
26871 104/300 Read block 1421 len 12288
26872 105/300 Flush
26873 106/300 Write block 1643 len 4096 data: 5
26874 107/300 Read block 3184 len 8192
26875 108/300 Read block 2188 len 12288
26876 109/300 Flush
26877 110/300 Read block 977 len 40960
26878 111/300 Write block 2257 len 40960 data: 5 5 6 5 5 2 2 3 4 5
26879 112/300 Read block 2716 len 40960
26880 113/300 Read block 1082 len 8192
26881 114/300 Read block 4382 len 32768
26882 115/300 Write block 4863 len 8192 data: 8 7
26883 116/300 Read block 627 len 32768
26884 117/300 Write block 2398 len 8192 data: 7 6
26885 118/300 Read block 2105 len 20480
26886 119/300 Read block 2527 len 40960
26887 120/300 Read block 4811 len 4096
26888 121/300 Write block 3613 len 12288 data: 5 5 5
26889 122/300 Write block 1409 len 16384 data: 3 4 4 4
26890 123/300 Read block 3498 len 4096
26891 124/300 Read block 3884 len 32768
26892 125/300 Read block 4863 len 28672
26893 126/300 Flush
26894 127/300 Write block 2517 len 32768 data: 9 8 7 6 5 4 3 2
26895 128/300 Write block 2721 len 16384 data: 5 5 6 4
26896 129/300 Read block 1797 len 12288
26897 130/300 Write block 1245 len 12288 data: 8 8 8
26898 131/300 Write block 4026 len 8192 data: 9 8
26899 132/300 Write block 3642 len 8192 data: 8 8
26900 133/300 Write block 2035 len 12288 data: 7 8 7
26901 134/300 Write block 1838 len 32768 data: 7 5 4 6 7 7 8 7
26902 135/300 Read block 3474 len 20480
26903 136/300 Read block 3612 len 20480
26904 137/300 Write block 60 len 40960 data: 4 5 4 4 4 4 3 3 3 3
26905 138/300 Read block 2678 len 16384
26906 139/300 Read block 3718 len 16384
26907 140/300 Read block 1518 len 24576
26908 141/300 Read block 2163 len 28672
26909 142/300 Read block 3289 len 40960
26910 143/300 Read block 1842 len 12288
26911 144/300 Read block 1339 len 28672
26912 145/300 Read block 4615 len 28672
26913 146/300 Write block 1435 len 32768 data: 4 3 3 3 2 2 2 4
26914 147/300 Write block 2065 len 8192 data: 10 9
26915 148/300 Read block 2601 len 16384
26916 149/300 Read block 4846 len 20480
26917 150/300 Write block 923 len 32768 data: 4 5 5 7 7 6 8 9
26918 151/300 Read block 4837 len 20480
26919 152/300 Read block 2331 len 36864
26920 153/300 Read block 4430 len 4096
26921 154/300 Read block 3566 len 8192
26922 155/300 Write block 1219 len 8192 data: 3 3
26923 156/300 Write block 2528 len 12288 data: 3 3 4
26924 157/300 Read block 1038 len 28672
26925 158/300 Write block 4344 len 8192 data: 11 10
26926 159/300 Write block 1792 len 4096 data: 5
26927 160/300 Write block 2477 len 12288 data: 4 4 5
26928 161/300 Read block 1132 len 36864
26929 162/300 Read block 521 len 32768
26930 163/300 Write block 1943 len 8192 data: 8 8
26931 164/300 Read block 1402 len 12288
26932 165/300 Write block 1446 len 24576 data: 6 4 4 3 3 4
26933 166/300 Read block 30 len 16384
26934 167/300 Read block 4504 len 32768
26935 168/300 Write block 2305 len 20480 data: 2 2 2 3 4
26936 169/300 Write block 1668 len 16384 data: 4 3 4 6
26937 170/300 Read block 909 len 40960
26938 171/300 Flush
26939 172/300 Write block 4803 len 4096 data: 6
26940 173/300 Write block 2238 len 12288 data: 3 4 4
26941 174/300 Write block 4711 len 24576 data: 7 6 6 6 7 5
26942 175/300 Read block 2309 len 36864
26943 176/300 Read block 4917 len 4096
26944 177/300 Read block 3460 len 4096
26945 178/300 Write block 3626 len 4096 data: 4
26946 179/300 Read block 2955 len 8192
26947 180/300 Read block 4977 len 32768
26948 181/300 Read block 2650 len 20480
26949 182/300 Read block 3962 len 28672
26950 183/300 Read block 1734 len 8192
26951 184/300 Read block 3914 len 36864
26952 185/300 Read block 4184 len 4096
26953 186/300 Write block 639 len 32768 data: 7 7 8 8 8 8 6 6
26954 187/300 Read block 4661 len 8192
26955 188/300 Write block 4623 len 24576 data: 5 4 4 3 5 5
26956 189/300 Read block 256 len 40960
26957 190/300 Read block 3312 len 32768
26958 191/300 Write block 328 len 36864 data: 4 4 4 4 3 4 4 5 4
26959 192/300 Read block 2587 len 32768
26960 193/300 Write block 4570 len 28672 data: 4 4 4 4 4 6 5
26961 194/300 Read block 4287 len 24576
26962 195/300 Read block 2208 len 20480
26963 196/300 Read block 1213 len 28672
26964 197/300 Read block 871 len 40960
26965 198/300 Write block 1364 len 28672 data: 7 6 6 5 5 4 4
26966 199/300 Read block 1975 len 28672
26967 200/300 Read block 903 len 4096
26968 201/300 Write block 4350 len 32768 data: 5 5 3 3 3 3 5 7
26969 202/300 Read block 620 len 28672
26970 203/300 Read block 2264 len 8192
26971 204/300 Write block 1428 len 32768 data: 4 4 4 4 4 4 4 5
26972 205/300 Read block 180 len 4096
26973 206/300 Write block 421 len 4096 data: 4
26974 207/300 Read block 4240 len 8192
26975 208/300 Read block 4363 len 28672
26976 209/300 Write block 2149 len 36864 data: 4 4 4 5 4 3 3 3 2
26977 210/300 Read block 4337 len 40960
26978 211/300 Write block 1369 len 8192 data: 5 5
26979 212/300 Read block 3669 len 24576
26980 213/300 Write block 3721 len 32768 data: 3 3 3 3 4 5 4 6
26981 214/300 Write block 4657 len 16384 data: 4 6 8 8
26982 215/300 Write block 3455 len 4096 data: 6
26983 216/300 Write block 4644 len 4096 data: 6
26984 217/300 Read block 774 len 20480
26985 218/300 Write block 4041 len 36864 data: 7 8 8 7 8 7 7 4 4
26986 219/300 Read block 2954 len 40960
26987 220/300 Read block 965 len 36864
26988 221/300 Read block 1289 len 4096
26989 222/300 Write block 1325 len 36864 data: 4 5 4 3 2 2 2 2 2
26990 223/300 Flush
26991 224/300 Read block 3183 len 28672
26992 225/300 Flush
26993 226/300 Write block 3658 len 36864 data: 4 3 3 4 4 6 7 8 9
26994 227/300 Flush
26995 228/300 Read block 368 len 4096
26996 229/300 Write block 3385 len 8192 data: 5 4
26997 230/300 Write block 4160 len 40960 data: 6 4 5 4 4 4 4 4 5 4
26998 231/300 Flush
26999 232/300 Read block 582 len 16384
27000 233/300 Read block 2942 len 32768
27001 234/300 Read block 281 len 32768
27002 235/300 Flush
27003 236/300 Write block 854 len 20480 data: 4 4 4 6 6
27004 237/300 Write block 276 len 28672 data: 3 3 3 3 4 5 5
27005 238/300 Write block 3982 len 8192 data: 4 3
27006 239/300 Write block 2581 len 24576 data: 5 4 4 4 5 4
27007 240/300 Write block 639 len 4096 data: 8
27008 241/300 Write block 4142 len 4096 data: 6
27009 242/300 Read block 1955 len 16384
27010 243/300 Read block 4121 len 16384
27011 244/300 Read block 2806 len 16384
27012 245/300 Read block 395 len 20480
27013 246/300 Write block 3090 len 28672 data: 8 7 4 4 3 3 2
27014 247/300 Write block 940 len 12288 data: 6 5 5
27015 248/300 Read block 470 len 12288
27016 249/300 Read block 1165 len 32768
27017 250/300 Flush
27018 251/300 Read block 3896 len 32768
27019 252/300 Flush
27020 253/300 Read block 104 len 36864
27021 254/300 Write block 2043 len 40960 data: 5 5 6 5 5 6 5 6 7 7
27022 255/300 Read block 3746 len 12288
27023 256/300 Read block 2265 len 20480
27024 257/300 Read block 4765 len 12288
27025 258/300 Write block 1165 len 16384 data: 4 6 6 10
27026 259/300 Write block 4869 len 32768 data: 4 5 6 6 4 6 7 7
27027 260/300 Read block 1708 len 8192
27028 261/300 Write block 4507 len 4096 data: 5
27029 262/300 Read block 1962 len 40960
27030 263/300 Read block 2981 len 12288
27031 264/300 Write block 1684 len 32768 data: 5 4 4 4 3 4 4 4
27032 265/300 Read block 4991 len 4096
27033 266/300 Read block 755 len 32768
27034 267/300 Write block 1045 len 12288 data: 7 7 6
27035 268/300 Read block 1890 len 20480
27036 269/300 Write block 1044 len 20480 data: 5 8 8 7 8
27037 270/300 Read block 480 len 8192
27038 271/300 Read block 4341 len 24576
27039 272/300 Read block 511 len 16384
27040 273/300 Read block 625 len 4096
27041 274/300 Read block 3649 len 24576
27042 275/300 Read block 54 len 12288
27043 276/300 Flush
27044 277/300 Write block 4471 len 24576 data: 4 4 7 8 10 9
27045 278/300 Write block 1254 len 20480 data: 3 4 5 5 6
27046 279/300 Write block 3116 len 28672 data: 2 3 4 4 5 5 4
27047 280/300 Write block 689 len 8192 data: 2 3
27048 281/300 Write block 792 len 12288 data: 5 4 4
27049 282/300 Flush
27050 283/300 Write block 2118 len 12288 data: 6 5 4
27051 284/300 Write block 1425 len 8192 data: 4 4
27052 285/300 Flush
27053 286/300 Read block 2046 len 12288
27054 287/300 Read block 2787 len 4096
27055 288/300 Read block 869 len 32768
27056 289/300 Read block 3753 len 24576
27057 290/300 Write block 1012 len 8192 data: 4 4
27058 291/300 Read block 4408 len 32768
27059 292/300 Read block 4862 len 16384
27060 293/300 Read block 4975 len 24576
27061 294/300 Write block 2111 len 12288 data: 5 5 6
27062 295/300 Write block 1211 len 8192 data: 7 6
27063 296/300 Write block 21 len 8192 data: 4 4
27064 297/300 Write block 1391 len 40960 data: 3 4 4 4 5 5 5 4 3 2
27065 298/300 Read block 3913 len 24576
27066 299/300 Write block 475 len 12288 data: 5 4 3
27067 300/300 Write block 764 len 4096 data: 5
27068 301/300 Flush
27069 Main task has work to do, go find it
27070 got dsc Start(2)
27071 start 2
27072 Nov 26 22:10:14.267 INFO request completed, latency_us: 193, response_code: 204, uri: /start/cid/2, method: POST, req_id: 6f82197d-afa6-4706-8f97-c1dd61092035, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
27073 Replay: started 2, returned:Ok(())
27074 ----------------------------------------------------------------
27075 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
27076 States: Active Active Offline
27077 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
27078 7944 Acked Read 5 Done Done Sent false
27079 7945 Acked Write 2 Done Done Sent false
27080 7946 Acked Read 7 Done Done Sent false
27081 7947 Acked Read 5 Done Done Sent false
27082 7948 Acked Write 6 Done Done Sent false
27083 7949 Acked Read 10 Done Done Sent false
27084 7950 Acked Write 7 Done Done Sent false
27085 7951 Acked Write 3 Done Done Sent false
27086 7952 Acked Write 10 Done Done Sent false
27087 7953 Acked Read 2 Done Done Sent false
27088 7954 Acked Read 3 Done Done Sent false
27089 7955 Acked Read 10 Done Done Sent false
27090 7956 Acked Write 6 Done Done Sent false
27091 7957 Acked Read 5 Done Done Sent false
27092 7958 Acked Write 5 Done Done Sent false
27093 7959 Acked Read 6 Done Done Sent false
27094 7960 Acked Flush 0 Done Done Sent false
27095 7961 Acked Read 10 Done Done Sent false
27096 7962 Acked Write 6 Done Done Sent false
27097 7963 Acked Flush 0 Done Done Sent false
27098 7964 Acked Read 7 Done Done Sent false
27099 7965 Acked Read 1 Done Done Sent false
27100 7966 Acked Write 5 Done Done Sent false
27101 7967 Acked Write 7 Done Done Sent false
27102 7968 Acked Read 2 Done Done Sent false
27103 7969 Acked Write 4 Done Done Sent false
27104 7970 Acked Write 10 Done Done Sent false
27105 7971 Acked Read 1 Done Done Sent false
27106 7972 Acked Write 3 Done Done Sent false
27107 7973 Acked Read 3 Done Done Sent false
27108 7974 Acked Read 9 Done Done Sent false
27109 7975 Acked Write 3 Done Done Sent false
27110 7976 Acked Write 2 Done Done Sent false
27111 7977 Acked Read 6 Done Done Sent false
27112 7978 Acked Read 2 Done Done Sent false
27113 7979 Acked Read 7 Done Done Sent false
27114 7980 Acked Read 1 Done Done Sent false
27115 7981 Acked Read 9 Done Done Sent false
27116 7982 Acked Write 4 Done Done Sent false
27117 7983 Acked Read 8 Done Done Sent false
27118 7984 Acked Write 7 Done Done Sent false
27119 7985 Acked Read 2 Done Done Sent false
27120 7986 Acked Read 7 Done Done Sent false
27121 7987 Acked Read 3 Done Done Sent false
27122 7988 Acked Write 3 Done Done Sent false
27123 7989 Acked Read 4 Done Done Sent false
27124 7990 Acked Write 4 Done Done Sent false
27125 7991 Acked Read 5 Done Done Sent false
27126 7992 Acked Write 1 Done Done Sent false
27127 7993 Acked Flush 0 Done Done Sent false
27128 7994 Acked Read 7 Done Done Sent false
27129 7995 Acked Flush 0 Done Done Sent false
27130 7996 Acked Read 10 Done Done Sent false
27131 7997 Acked Read 5 Done Done Sent false
27132 7998 Acked Read 7 Done Done Sent false
27133 7999 Acked Write 10 Done Done Sent false
27134 8000 Acked Flush 0 Done Done Sent false
27135 8001 Acked Read 6 Done Done Sent false
27136 8002 Acked Read 7 Done Done Sent false
27137 8003 Acked Write 6 Done Done Sent false
27138 8004 Acked Flush 0 Done Done Sent false
27139 8005 Acked Read 8 Done Done Sent false
27140 8006 Acked Flush 0 Done Done Sent false
27141 8007 Acked Read 10 Done Done Sent false
27142 8008 Acked Write 6 Done Done Sent false
27143 8009 Acked Read 4 Done Done Sent false
27144 8010 Acked Read 5 Done Done Sent false
27145 8011 Acked Read 2 Done Done Sent false
27146 8012 Acked Write 8 Done Done Sent false
27147 8013 Acked Flush 0 Done Done Sent false
27148 8014 Acked Write 2 Done Done Sent false
27149 8015 Acked Flush 0 Done Done Sent false
27150 8016 Acked Read 10 Done Done Sent false
27151 8017 Acked Flush 0 Done Done Sent false
27152 8018 Acked Read 9 Done Done Sent false
27153 8019 Acked Read 7 Done Done Sent false
27154 8020 Acked Read 2 Done Done Sent false
27155 8021 Acked Read 10 Done Done Sent false
27156 8022 Acked Read 9 Done Done Sent false
27157 8023 Acked Read 6 Done Done Sent false
27158 8024 Acked Flush 0 Done Done Sent false
27159 8025 Acked Write 5 Done Done Sent false
27160 8026 Acked Read 2 Done Done Sent false
27161 8027 Acked Write 10 Done Done Sent false
27162 8028 Acked Read 6 Done Done Sent false
27163 8029 Acked Read 8 Done Done Sent false
27164 8030 Acked Write 7 Done Done Sent false
27165 8031 Acked Write 10 Done Done Sent false
27166 8032 Acked Read 2 Done Done Sent false
27167 8033 Acked Read 5 Done Done Sent false
27168 8034 Acked Write 7 Done Done Sent false
27169 8035 Acked Write 6 Done Done Sent false
27170 8036 Acked Read 9 Done Done Sent false
27171 8037 Acked Write 2 Done Done Sent false
27172 8038 Acked Read 8 Done Done Sent false
27173 8039 Acked Write 10 Done Done Sent false
27174 8040 Acked Write 8 Done Done Sent false
27175 8041 Acked Read 8 Done Done Sent false
27176 8042 Acked Read 8 Done Done Sent false
27177 8043 Acked Write 6 Done Done Sent false
27178 8044 Acked Read 10 Done Done Sent false
27179 8045 Acked Write 8 Done Done Sent false
27180 8046 Acked Read 4 Done Done Sent false
27181 8047 Acked Read 3 Done Done Sent false
27182 8048 Acked Flush 0 Done Done Sent false
27183 8049 Acked Write 1 Done Done Sent false
27184 8050 Acked Read 2 Done Done Sent false
27185 8051 Acked Read 3 Done Done Sent false
27186 8052 Acked Flush 0 Done Done Sent false
27187 8053 Acked Read 10 Done Done Sent false
27188 8054 Acked Write 10 Done Done Sent false
27189 8055 Acked Read 10 Done Done Sent false
27190 8056 Acked Read 2 Done Done Sent false
27191 8057 Acked Read 8 Done Done Sent false
27192 8058 Acked Write 2 Done Done Sent false
27193 8059 Acked Read 8 Done Done Sent false
27194 8060 Acked Write 2 Done Done Sent false
27195 8061 Acked Read 5 Done Done Sent false
27196 8062 Acked Read 10 Done Done Sent false
27197 8063 Acked Read 1 Done Done Sent false
27198 8064 Acked Write 3 Done Done Sent false
27199 8065 Acked Write 4 Done Done Sent false
27200 8066 Acked Read 1 Done Done Sent false
27201 8067 Acked Read 8 Done Done Sent false
27202 8068 Acked Read 7 Done Done Sent false
27203 8069 Acked Flush 0 Done Done Sent false
27204 8070 Acked Write 8 Done Done Sent false
27205 8071 Acked Write 4 Done Done Sent false
27206 8072 Acked Read 3 Done Done Sent false
27207 8073 Acked Write 3 Done Done Sent false
27208 8074 Acked Write 2 Done Done Sent false
27209 8075 Acked Write 2 Done Done Sent false
27210 8076 Acked Write 3 Done Done Sent false
27211 8077 Acked Write 8 Done Done Sent false
27212 8078 Acked Read 5 Done Done Sent false
27213 8079 Acked Read 5 Done Done Sent false
27214 8080 Acked Write 10 Done Done Sent false
27215 8081 Acked Read 4 Done Done Sent false
27216 8082 Acked Read 4 Done Done Sent false
27217 8083 Acked Read 6 Done Done Sent false
27218 8084 Acked Read 7 Done Done Sent false
27219 8085 Acked Read 10 Done Done Sent false
27220 8086 Acked Read 3 Done Done Sent false
27221 8087 Acked Read 7 Done Done Sent false
27222 8088 Acked Read 7 Done Done Sent false
27223 8089 Acked Write 8 Done Done Sent false
27224 8090 Acked Write 2 Done Done Sent false
27225 8091 Acked Read 4 Done Done Sent false
27226 8092 Acked Read 5 Done Done Sent false
27227 8093 Acked Write 8 Done Done Sent false
27228 8094 Acked Read 5 Done Done Sent false
27229 8095 Acked Read 9 Done Done Sent false
27230 8096 Acked Read 1 Done Done Sent false
27231 8097 Acked Read 2 Done Done Sent false
27232 8098 Acked Write 2 Done Done Sent false
27233 8099 Acked Write 3 Done Done Sent false
27234 8100 Acked Read 7 Done Done Sent false
27235 8101 Acked Write 2 Done Done Sent false
27236 8102 Acked Write 1 Done Done Sent false
27237 8103 Acked Write 3 Done Done Sent false
27238 8104 Acked Read 9 Done Done Sent false
27239 8105 Acked Read 8 Done Done Sent false
27240 8106 Acked Write 2 Done Done Sent false
27241 8107 Acked Read 3 Done Done Sent false
27242 8108 Acked Write 6 Done Done Sent false
27243 8109 Acked Read 4 Done Done Sent false
27244 8110 Acked Read 8 Done Done Sent false
27245 8111 Acked Write 5 Done Done Sent false
27246 8112 Acked Write 4 Done Done Sent false
27247 8113 Acked Read 10 Done Done Sent false
27248 8114 Acked Flush 0 Done Done Sent false
27249 8115 Acked Write 1 Done Done Sent false
27250 8116 Acked Write 3 Done Done Sent false
27251 8117 Acked Write 6 Done Done Sent false
27252 8118 Acked Read 9 Done Done Sent false
27253 8119 Acked Read 1 Done Done Sent false
27254 8120 Acked Read 1 Done Done Sent false
27255 8121 Acked Write 1 Done Done Sent false
27256 8122 Acked Read 2 Done Done Sent false
27257 8123 Acked Read 8 Done Done Sent false
27258 8124 Acked Read 5 Done Done Sent false
27259 8125 Acked Read 7 Done Done Sent false
27260 8126 Acked Read 2 Done Done Sent false
27261 8127 Acked Read 9 Done Done Sent false
27262 8128 Acked Read 1 Done Done Sent false
27263 8129 Acked Write 8 Done Done Sent false
27264 8130 Acked Read 2 Done Done Sent false
27265 8131 Acked Write 6 Done Done Sent false
27266 8132 Acked Read 10 Done Done Sent false
27267 8133 Acked Read 8 Done Done Sent false
27268 8134 Acked Write 9 Done Done Sent false
27269 8135 Acked Read 8 Done Done Sent false
27270 8136 Acked Write 7 Done Done Sent false
27271 8137 Acked Read 6 Done Done Sent false
27272 8138 Acked Read 5 Done Done Sent false
27273 8139 Acked Read 7 Done Done Sent false
27274 8140 Acked Read 10 Done Done Sent false
27275 8141 Acked Write 7 Done Done Sent false
27276 8142 Acked Read 7 Done Done Sent false
27277 8143 Acked Read 1 Done Done Sent false
27278 8144 Acked Write 8 Done Done Sent false
27279 8145 Acked Read 7 Done Done Sent false
27280 8146 Acked Read 2 Done Done Sent false
27281 8147 Acked Write 8 Done Done Sent false
27282 8148 Acked Read 1 Done Done Sent false
27283 8149 Acked Write 1 Done Done Sent false
27284 8150 Acked Read 2 Done Done Sent false
27285 8151 Acked Read 7 Done Done Sent false
27286 8152 Acked Write 9 Done Done Sent false
27287 8153 Acked Read 10 Done Done Sent false
27288 8154 Acked Write 2 Done Done Sent false
27289 8155 Acked Read 6 Done Done Sent false
27290 8156 Acked Write 8 Done Done Sent false
27291 8157 Acked Write 4 Done Done Sent false
27292 8158 Acked Write 1 Done Done Sent false
27293 8159 Acked Write 1 Done Done Sent false
27294 8160 Acked Read 5 Done Done Sent false
27295 8161 Acked Write 9 Done Done Sent false
27296 8162 Acked Read 10 Done Done Sent false
27297 8163 Acked Read 9 Done Done Sent false
27298 8164 Acked Read 1 Done Done Sent false
27299 8165 Acked Write 9 Done Done Sent false
27300 8166 Acked Flush 0 Done Done Sent false
27301 8167 Acked Read 7 Done Done Sent false
27302 8168 Acked Flush 0 Done Done Sent false
27303 8169 Acked Write 9 Done Done Sent false
27304 8170 Acked Flush 0 Done Done Sent false
27305 8171 Acked Read 1 Done Done Sent false
27306 8172 Acked Write 2 Done Done Sent false
27307 8173 Acked Write 10 Done Done Sent false
27308 8174 Acked Flush 0 Done Done Sent false
27309 8175 Acked Read 4 Done Done Sent false
27310 8176 Acked Read 8 Done Done Sent false
27311 8177 Acked Read 8 Done Done Sent false
27312 8178 Acked Flush 0 Done Done Sent false
27313 8179 Acked Write 5 Done Done Sent false
27314 8180 Acked Write 7 Done Done Sent false
27315 8181 Acked Write 2 Done Done Sent false
27316 8182 Acked Write 6 Done Done Sent false
27317 8183 Acked Write 1 Done Done Sent false
27318 8184 Acked Write 1 Done Done Sent false
27319 8185 Acked Read 4 Done Done Sent false
27320 8186 Acked Read 4 Done Done Sent false
27321 8187 Acked Read 4 Done Done Sent false
27322 8188 Acked Read 5 Done Done Sent false
27323 8189 Acked Write 7 Done Done Sent false
27324 8190 Acked Write 3 Done Done Sent false
27325 8191 Acked Read 3 Done Done Sent false
27326 8192 Acked Read 8 Done Done Sent false
27327 8193 Acked Flush 0 Done Done Sent false
27328 8194 Acked Read 8 Done Done Sent false
27329 8195 Acked Flush 0 Done Done Sent false
27330 8196 Acked Read 9 Done Done Sent false
27331 8197 Acked Write 10 Done Done Sent false
27332 8198 Acked Read 3 Done Done Sent false
27333 8199 Acked Read 5 Done Done Sent false
27334 8200 Acked Read 3 Done Done Sent false
27335 8201 Acked Write 4 Done Done Sent false
27336 8202 Acked Write 8 Done Done Sent false
27337 8203 Acked Read 2 Done Done Sent false
27338 8204 Acked Write 1 Done Done Sent false
27339 8205 Acked Read 10 Done Done Sent false
27340 8206 Acked Read 3 Done Done Sent false
27341 8207 Acked Write 8 Done Done Sent false
27342 8208 Acked Read 1 Done Done Sent false
27343 8209 Acked Read 8 Done Done Sent false
27344 8210 Acked Write 3 Done Done Sent false
27345 8211 Acked Read 5 Done Done Sent false
27346 8212 Acked Write 5 Done Done Sent false
27347 8213 Acked Read 2 Done Done Sent false
27348 8214 Acked Read 6 Done Done Sent false
27349 8215 Acked Read 4 Done Done Sent false
27350 8216 Acked Read 1 Done Done Sent false
27351 8217 Acked Read 6 Done Done Sent false
27352 8218 Acked Read 3 Done Done Sent false
27353 8219 Acked Flush 0 Done Done Sent false
27354 8220 Acked Write 6 Done Done Sent false
27355 8221 Acked Write 5 Done Done Sent false
27356 8222 Acked Write 7 Done Done Sent false
27357 8223 Acked Write 2 Done Done Sent false
27358 8224 Acked Write 3 Done Done Sent false
27359 8225 Acked Flush 0 Done Done Sent false
27360 8226 Acked Write 3 Done Done Sent false
27361 8227 Acked Write 2 Done Done Sent false
27362 8228 Acked Flush 0 Done Done Sent false
27363 8229 Acked Read 3 Done Done Sent false
27364 8230 Acked Read 1 Done Done Sent false
27365 8231 Acked Read 8 Done Done Sent false
27366 8232 Acked Read 6 Done Done Sent false
27367 8233 Acked Write 2 Done Done Sent false
27368 8234 Acked Read 8 Done Done Sent false
27369 8235 Acked Read 4 Done Done Sent false
27370 8236 Acked Read 6 Done Done Sent false
27371 8237 Acked Write 3 Done Done Sent false
27372 8238 Acked Write 2 Done Done Sent false
27373 8239 Acked Write 2 Done Done Sent false
27374 8240 Acked Write 10 Done Done Sent false
27375 8241 Acked Read 6 Done Done Sent false
27376 8242 Acked Write 3 Done Done Sent false
27377 8243 Acked Write 1 Done Done Sent false
27378 8244 Acked Flush 0 Done Done Sent false
27379 STATES DS:0 DS:1 DS:2 TOTAL
27380 Sent 0 0 301 301
27381 Done 301 301 0 602
27382 Skipped 0 0 0 0
27383 Error 0 0 0 0
27384 Last Flush: 8244 8244 7943
27385 Downstairs last five completed: 7943 7942 7941 7940 7939
27386 Upstairs last five completed: 8244 8243 8242 8241 8240
27387 CLIENT: Up:0 ds:301 act:2
27388 ----------------------------------------------------------------
27389 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
27390 States: Active Active Offline
27391 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
27392 7944 Acked Read 5 Done Done Sent false
27393 7945 Acked Write 2 Done Done Sent false
27394 7946 Acked Read 7 Done Done Sent false
27395 7947 Acked Read 5 Done Done Sent false
27396 7948 Acked Write 6 Done Done Sent false
27397 7949 Acked Read 10 Done Done Sent false
27398 7950 Acked Write 7 Done Done Sent false
27399 7951 Acked Write 3 Done Done Sent false
27400 7952 Acked Write 10 Done Done Sent false
27401 7953 Acked Read 2 Done Done Sent false
27402 7954 Acked Read 3 Done Done Sent false
27403 7955 Acked Read 10 Done Done Sent false
27404 7956 Acked Write 6 Done Done Sent false
27405 7957 Acked Read 5 Done Done Sent false
27406 7958 Acked Write 5 Done Done Sent false
27407 7959 Acked Read 6 Done Done Sent false
27408 7960 Acked Flush 0 Done Done Sent false
27409 7961 Acked Read 10 Done Done Sent false
27410 7962 Acked Write 6 Done Done Sent false
27411 7963 Acked Flush 0 Done Done Sent false
27412 7964 Acked Read 7 Done Done Sent false
27413 7965 Acked Read 1 Done Done Sent false
27414 7966 Acked Write 5 Done Done Sent false
27415 7967 Acked Write 7 Done Done Sent false
27416 7968 Acked Read 2 Done Done Sent false
27417 7969 Acked Write 4 Done Done Sent false
27418 7970 Acked Write 10 Done Done Sent false
27419 7971 Acked Read 1 Done Done Sent false
27420 7972 Acked Write 3 Done Done Sent false
27421 7973 Acked Read 3 Done Done Sent false
27422 7974 Acked Read 9 Done Done Sent false
27423 7975 Acked Write 3 Done Done Sent false
27424 7976 Acked Write 2 Done Done Sent false
27425 7977 Acked Read 6 Done Done Sent false
27426 7978 Acked Read 2 Done Done Sent false
27427 7979 Acked Read 7 Done Done Sent false
27428 7980 Acked Read 1 Done Done Sent false
27429 7981 Acked Read 9 Done Done Sent false
27430 7982 Acked Write 4 Done Done Sent false
27431 7983 Acked Read 8 Done Done Sent false
27432 7984 Acked Write 7 Done Done Sent false
27433 7985 Acked Read 2 Done Done Sent false
27434 7986 Acked Read 7 Done Done Sent false
27435 7987 Acked Read 3 Done Done Sent false
27436 7988 Acked Write 3 Done Done Sent false
27437 7989 Acked Read 4 Done Done Sent false
27438 7990 Acked Write 4 Done Done Sent false
27439 7991 Acked Read 5 Done Done Sent false
27440 7992 Acked Write 1 Done Done Sent false
27441 7993 Acked Flush 0 Done Done Sent false
27442 7994 Acked Read 7 Done Done Sent false
27443 7995 Acked Flush 0 Done Done Sent false
27444 7996 Acked Read 10 Done Done Sent false
27445 7997 Acked Read 5 Done Done Sent false
27446 7998 Acked Read 7 Done Done Sent false
27447 7999 Acked Write 10 Done Done Sent false
27448 8000 Acked Flush 0 Done Done Sent false
27449 8001 Acked Read 6 Done Done Sent false
27450 8002 Acked Read 7 Done Done Sent false
27451 8003 Acked Write 6 Done Done Sent false
27452 8004 Acked Flush 0 Done Done Sent false
27453 8005 Acked Read 8 Done Done Sent false
27454 8006 Acked Flush 0 Done Done Sent false
27455 8007 Acked Read 10 Done Done Sent false
27456 8008 Acked Write 6 Done Done Sent false
27457 8009 Acked Read 4 Done Done Sent false
27458 8010 Acked Read 5 Done Done Sent false
27459 8011 Acked Read 2 Done Done Sent false
27460 8012 Acked Write 8 Done Done Sent false
27461 8013 Acked Flush 0 Done Done Sent false
27462 8014 Acked Write 2 Done Done Sent false
27463 8015 Acked Flush 0 Done Done Sent false
27464 8016 Acked Read 10 Done Done Sent false
27465 8017 Acked Flush 0 Done Done Sent false
27466 8018 Acked Read 9 Done Done Sent false
27467 8019 Acked Read 7 Done Done Sent false
27468 8020 Acked Read 2 Done Done Sent false
27469 8021 Acked Read 10 Done Done Sent false
27470 8022 Acked Read 9 Done Done Sent false
27471 8023 Acked Read 6 Done Done Sent false
27472 8024 Acked Flush 0 Done Done Sent false
27473 8025 Acked Write 5 Done Done Sent false
27474 8026 Acked Read 2 Done Done Sent false
27475 8027 Acked Write 10 Done Done Sent false
27476 8028 Acked Read 6 Done Done Sent false
27477 8029 Acked Read 8 Done Done Sent false
27478 8030 Acked Write 7 Done Done Sent false
27479 8031 Acked Write 10 Done Done Sent false
27480 8032 Acked Read 2 Done Done Sent false
27481 8033 Acked Read 5 Done Done Sent false
27482 8034 Acked Write 7 Done Done Sent false
27483 8035 Acked Write 6 Done Done Sent false
27484 8036 Acked Read 9 Done Done Sent false
27485 8037 Acked Write 2 Done Done Sent false
27486 8038 Acked Read 8 Done Done Sent false
27487 8039 Acked Write 10 Done Done Sent false
27488 8040 Acked Write 8 Done Done Sent false
27489 8041 Acked Read 8 Done Done Sent false
27490 8042 Acked Read 8 Done Done Sent false
27491 8043 Acked Write 6 Done Done Sent false
27492 8044 Acked Read 10 Done Done Sent false
27493 8045 Acked Write 8 Done Done Sent false
27494 8046 Acked Read 4 Done Done Sent false
27495 8047 Acked Read 3 Done Done Sent false
27496 8048 Acked Flush 0 Done Done Sent false
27497 8049 Acked Write 1 Done Done Sent false
27498 8050 Acked Read 2 Done Done Sent false
27499 8051 Acked Read 3 Done Done Sent false
27500 8052 Acked Flush 0 Done Done Sent false
27501 8053 Acked Read 10 Done Done Sent false
27502 8054 Acked Write 10 Done Done Sent false
27503 8055 Acked Read 10 Done Done Sent false
27504 8056 Acked Read 2 Done Done Sent false
27505 8057 Acked Read 8 Done Done Sent false
27506 8058 Acked Write 2 Done Done Sent false
27507 8059 Acked Read 8 Done Done Sent false
27508 8060 Acked Write 2 Done Done Sent false
27509 8061 Acked Read 5 Done Done Sent false
27510 8062 Acked Read 10 Done Done Sent false
27511 8063 Acked Read 1 Done Done Sent false
27512 8064 Acked Write 3 Done Done Sent false
27513 8065 Acked Write 4 Done Done Sent false
27514 8066 Acked Read 1 Done Done Sent false
27515 8067 Acked Read 8 Done Done Sent false
27516 8068 Acked Read 7 Done Done Sent false
27517 8069 Acked Flush 0 Done Done Sent false
27518 8070 Acked Write 8 Done Done Sent false
27519 8071 Acked Write 4 Done Done Sent false
27520 8072 Acked Read 3 Done Done Sent false
27521 8073 Acked Write 3 Done Done Sent false
27522 8074 Acked Write 2 Done Done Sent false
27523 8075 Acked Write 2 Done Done Sent false
27524 8076 Acked Write 3 Done Done Sent false
27525 8077 Acked Write 8 Done Done Sent false
27526 8078 Acked Read 5 Done Done Sent false
27527 8079 Acked Read 5 Done Done Sent false
27528 8080 Acked Write 10 Done Done Sent false
27529 8081 Acked Read 4 Done Done Sent false
27530 8082 Acked Read 4 Done Done Sent false
27531 8083 Acked Read 6 Done Done Sent false
27532 8084 Acked Read 7 Done Done Sent false
27533 8085 Acked Read 10 Done Done Sent false
27534 8086 Acked Read 3 Done Done Sent false
27535 8087 Acked Read 7 Done Done Sent false
27536 8088 Acked Read 7 Done Done Sent false
27537 8089 Acked Write 8 Done Done Sent false
27538 8090 Acked Write 2 Done Done Sent false
27539 8091 Acked Read 4 Done Done Sent false
27540 8092 Acked Read 5 Done Done Sent false
27541 8093 Acked Write 8 Done Done Sent false
27542 8094 Acked Read 5 Done Done Sent false
27543 8095 Acked Read 9 Done Done Sent false
27544 8096 Acked Read 1 Done Done Sent false
27545 8097 Acked Read 2 Done Done Sent false
27546 8098 Acked Write 2 Done Done Sent false
27547 8099 Acked Write 3 Done Done Sent false
27548 8100 Acked Read 7 Done Done Sent false
27549 8101 Acked Write 2 Done Done Sent false
27550 8102 Acked Write 1 Done Done Sent false
27551 8103 Acked Write 3 Done Done Sent false
27552 8104 Acked Read 9 Done Done Sent false
27553 8105 Acked Read 8 Done Done Sent false
27554 8106 Acked Write 2 Done Done Sent false
27555 8107 Acked Read 3 Done Done Sent false
27556 8108 Acked Write 6 Done Done Sent false
27557 8109 Acked Read 4 Done Done Sent false
27558 8110 Acked Read 8 Done Done Sent false
27559 8111 Acked Write 5 Done Done Sent false
27560 8112 Acked Write 4 Done Done Sent false
27561 8113 Acked Read 10 Done Done Sent false
27562 8114 Acked Flush 0 Done Done Sent false
27563 8115 Acked Write 1 Done Done Sent false
27564 8116 Acked Write 3 Done Done Sent false
27565 8117 Acked Write 6 Done Done Sent false
27566 8118 Acked Read 9 Done Done Sent false
27567 8119 Acked Read 1 Done Done Sent false
27568 8120 Acked Read 1 Done Done Sent false
27569 8121 Acked Write 1 Done Done Sent false
27570 8122 Acked Read 2 Done Done Sent false
27571 8123 Acked Read 8 Done Done Sent false
27572 8124 Acked Read 5 Done Done Sent false
27573 8125 Acked Read 7 Done Done Sent false
27574 8126 Acked Read 2 Done Done Sent false
27575 8127 Acked Read 9 Done Done Sent false
27576 8128 Acked Read 1 Done Done Sent false
27577 8129 Acked Write 8 Done Done Sent false
27578 8130 Acked Read 2 Done Done Sent false
27579 8131 Acked Write 6 Done Done Sent false
27580 8132 Acked Read 10 Done Done Sent false
27581 8133 Acked Read 8 Done Done Sent false
27582 8134 Acked Write 9 Done Done Sent false
27583 8135 Acked Read 8 Done Done Sent false
27584 8136 Acked Write 7 Done Done Sent false
27585 8137 Acked Read 6 Done Done Sent false
27586 8138 Acked Read 5 Done Done Sent false
27587 8139 Acked Read 7 Done Done Sent false
27588 8140 Acked Read 10 Done Done Sent false
27589 8141 Acked Write 7 Done Done Sent false
27590 8142 Acked Read 7 Done Done Sent false
27591 8143 Acked Read 1 Done Done Sent false
27592 8144 Acked Write 8 Done Done Sent false
27593 8145 Acked Read 7 Done Done Sent false
27594 8146 Acked Read 2 Done Done Sent false
27595 8147 Acked Write 8 Done Done Sent false
27596 8148 Acked Read 1 Done Done Sent false
27597 8149 Acked Write 1 Done Done Sent false
27598 8150 Acked Read 2 Done Done Sent false
27599 8151 Acked Read 7 Done Done Sent false
27600 8152 Acked Write 9 Done Done Sent false
27601 8153 Acked Read 10 Done Done Sent false
27602 8154 Acked Write 2 Done Done Sent false
27603 8155 Acked Read 6 Done Done Sent false
27604 8156 Acked Write 8 Done Done Sent false
27605 8157 Acked Write 4 Done Done Sent false
27606 8158 Acked Write 1 Done Done Sent false
27607 8159 Acked Write 1 Done Done Sent false
27608 8160 Acked Read 5 Done Done Sent false
27609 8161 Acked Write 9 Done Done Sent false
27610 8162 Acked Read 10 Done Done Sent false
27611 8163 Acked Read 9 Done Done Sent false
27612 8164 Acked Read 1 Done Done Sent false
27613 8165 Acked Write 9 Done Done Sent false
27614 8166 Acked Flush 0 Done Done Sent false
27615 8167 Acked Read 7 Done Done Sent false
27616 8168 Acked Flush 0 Done Done Sent false
27617 8169 Acked Write 9 Done Done Sent false
27618 8170 Acked Flush 0 Done Done Sent false
27619 8171 Acked Read 1 Done Done Sent false
27620 8172 Acked Write 2 Done Done Sent false
27621 8173 Acked Write 10 Done Done Sent false
27622 8174 Acked Flush 0 Done Done Sent false
27623 8175 Acked Read 4 Done Done Sent false
27624 8176 Acked Read 8 Done Done Sent false
27625 8177 Acked Read 8 Done Done Sent false
27626 8178 Acked Flush 0 Done Done Sent false
27627 8179 Acked Write 5 Done Done Sent false
27628 8180 Acked Write 7 Done Done Sent false
27629 8181 Acked Write 2 Done Done Sent false
27630 8182 Acked Write 6 Done Done Sent false
27631 8183 Acked Write 1 Done Done Sent false
27632 8184 Acked Write 1 Done Done Sent false
27633 8185 Acked Read 4 Done Done Sent false
27634 8186 Acked Read 4 Done Done Sent false
27635 8187 Acked Read 4 Done Done Sent false
27636 8188 Acked Read 5 Done Done Sent false
27637 8189 Acked Write 7 Done Done Sent false
27638 8190 Acked Write 3 Done Done Sent false
27639 8191 Acked Read 3 Done Done Sent false
27640 8192 Acked Read 8 Done Done Sent false
27641 8193 Acked Flush 0 Done Done Sent false
27642 8194 Acked Read 8 Done Done Sent false
27643 8195 Acked Flush 0 Done Done Sent false
27644 8196 Acked Read 9 Done Done Sent false
27645 8197 Acked Write 10 Done Done Sent false
27646 8198 Acked Read 3 Done Done Sent false
27647 8199 Acked Read 5 Done Done Sent false
27648 8200 Acked Read 3 Done Done Sent false
27649 8201 Acked Write 4 Done Done Sent false
27650 8202 Acked Write 8 Done Done Sent false
27651 8203 Acked Read 2 Done Done Sent false
27652 8204 Acked Write 1 Done Done Sent false
27653 8205 Acked Read 10 Done Done Sent false
27654 8206 Acked Read 3 Done Done Sent false
27655 8207 Acked Write 8 Done Done Sent false
27656 8208 Acked Read 1 Done Done Sent false
27657 8209 Acked Read 8 Done Done Sent false
27658 8210 Acked Write 3 Done Done Sent false
27659 8211 Acked Read 5 Done Done Sent false
27660 8212 Acked Write 5 Done Done Sent false
27661 8213 Acked Read 2 Done Done Sent false
27662 8214 Acked Read 6 Done Done Sent false
27663 8215 Acked Read 4 Done Done Sent false
27664 8216 Acked Read 1 Done Done Sent false
27665 8217 Acked Read 6 Done Done Sent false
27666 8218 Acked Read 3 Done Done Sent false
27667 8219 Acked Flush 0 Done Done Sent false
27668 8220 Acked Write 6 Done Done Sent false
27669 8221 Acked Write 5 Done Done Sent false
27670 8222 Acked Write 7 Done Done Sent false
27671 8223 Acked Write 2 Done Done Sent false
27672 8224 Acked Write 3 Done Done Sent false
27673 8225 Acked Flush 0 Done Done Sent false
27674 8226 Acked Write 3 Done Done Sent false
27675 8227 Acked Write 2 Done Done Sent false
27676 8228 Acked Flush 0 Done Done Sent false
27677 8229 Acked Read 3 Done Done Sent false
27678 8230 Acked Read 1 Done Done Sent false
27679 8231 Acked Read 8 Done Done Sent false
27680 8232 Acked Read 6 Done Done Sent false
27681 8233 Acked Write 2 Done Done Sent false
27682 8234 Acked Read 8 Done Done Sent false
27683 8235 Acked Read 4 Done Done Sent false
27684 8236 Acked Read 6 Done Done Sent false
27685 8237 Acked Write 3 Done Done Sent false
27686 8238 Acked Write 2 Done Done Sent false
27687 8239 Acked Write 2 Done Done Sent false
27688 8240 Acked Write 10 Done Done Sent false
27689 8241 Acked Read 6 Done Done Sent false
27690 8242 Acked Write 3 Done Done Sent false
27691 8243 Acked Write 1 Done Done Sent false
27692 8244 Acked Flush 0 Done Done Sent false
27693 STATES DS:0 DS:1 DS:2 TOTAL
27694 Sent 0 0 301 301
27695 Done 301 301 0 602
27696 Skipped 0 0 0 0
27697 Error 0 0 0 0
27698 Last Flush: 8244 8244 7943
27699 Downstairs last five completed: 7943 7942 7941 7940 7939
27700 Upstairs last five completed: 8244 8243 8242 8241 8240
27701 CLIENT: Up:0 ds:301 act:2
277022024-11-26T22:10:20.027ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
277032024-11-26T22:10:20.027ZWARNcrucible: ds_connection connect to 127.0.0.1:8830 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
277042024-11-26T22:10:20.027ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
277052024-11-26T22:10:20.027ZWARNcrucible: downstairs task for 2 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
277062024-11-26T22:10:20.027ZINFOcrucible: client task is exiting = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
27707 [8830] got start action
27708 [8830] I am going to restart
27709 Starting downstairs at port 8830
27710 Make output file at "/tmp/dsc/downstairs-8830.txt"
27711 [8830][2] reports Starting
27712 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1227)
27713 [8830][2] reports Running
27714 ----------------------------------------------------------------
27715 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
27716 States: Active Active Offline
27717 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
27718 7944 Acked Read 5 Done Done Sent false
27719 7945 Acked Write 2 Done Done Sent false
27720 7946 Acked Read 7 Done Done Sent false
27721 7947 Acked Read 5 Done Done Sent false
27722 7948 Acked Write 6 Done Done Sent false
27723 7949 Acked Read 10 Done Done Sent false
27724 7950 Acked Write 7 Done Done Sent false
27725 7951 Acked Write 3 Done Done Sent false
27726 7952 Acked Write 10 Done Done Sent false
27727 7953 Acked Read 2 Done Done Sent false
27728 7954 Acked Read 3 Done Done Sent false
27729 7955 Acked Read 10 Done Done Sent false
27730 7956 Acked Write 6 Done Done Sent false
27731 7957 Acked Read 5 Done Done Sent false
27732 7958 Acked Write 5 Done Done Sent false
27733 7959 Acked Read 6 Done Done Sent false
27734 7960 Acked Flush 0 Done Done Sent false
27735 7961 Acked Read 10 Done Done Sent false
27736 7962 Acked Write 6 Done Done Sent false
27737 7963 Acked Flush 0 Done Done Sent false
27738 7964 Acked Read 7 Done Done Sent false
27739 7965 Acked Read 1 Done Done Sent false
27740 7966 Acked Write 5 Done Done Sent false
27741 7967 Acked Write 7 Done Done Sent false
27742 7968 Acked Read 2 Done Done Sent false
27743 7969 Acked Write 4 Done Done Sent false
27744 7970 Acked Write 10 Done Done Sent false
27745 7971 Acked Read 1 Done Done Sent false
27746 7972 Acked Write 3 Done Done Sent false
27747 7973 Acked Read 3 Done Done Sent false
27748 7974 Acked Read 9 Done Done Sent false
27749 7975 Acked Write 3 Done Done Sent false
27750 7976 Acked Write 2 Done Done Sent false
27751 7977 Acked Read 6 Done Done Sent false
27752 7978 Acked Read 2 Done Done Sent false
27753 7979 Acked Read 7 Done Done Sent false
27754 7980 Acked Read 1 Done Done Sent false
27755 7981 Acked Read 9 Done Done Sent false
27756 7982 Acked Write 4 Done Done Sent false
27757 7983 Acked Read 8 Done Done Sent false
27758 7984 Acked Write 7 Done Done Sent false
27759 7985 Acked Read 2 Done Done Sent false
27760 7986 Acked Read 7 Done Done Sent false
27761 7987 Acked Read 3 Done Done Sent false
27762 7988 Acked Write 3 Done Done Sent false
27763 7989 Acked Read 4 Done Done Sent false
27764 7990 Acked Write 4 Done Done Sent false
27765 7991 Acked Read 5 Done Done Sent false
27766 7992 Acked Write 1 Done Done Sent false
27767 7993 Acked Flush 0 Done Done Sent false
27768 7994 Acked Read 7 Done Done Sent false
27769 7995 Acked Flush 0 Done Done Sent false
27770 7996 Acked Read 10 Done Done Sent false
27771 7997 Acked Read 5 Done Done Sent false
27772 7998 Acked Read 7 Done Done Sent false
27773 7999 Acked Write 10 Done Done Sent false
27774 8000 Acked Flush 0 Done Done Sent false
27775 8001 Acked Read 6 Done Done Sent false
27776 8002 Acked Read 7 Done Done Sent false
27777 8003 Acked Write 6 Done Done Sent false
27778 8004 Acked Flush 0 Done Done Sent false
27779 8005 Acked Read 8 Done Done Sent false
27780 8006 Acked Flush 0 Done Done Sent false
27781 8007 Acked Read 10 Done Done Sent false
27782 8008 Acked Write 6 Done Done Sent false
27783 8009 Acked Read 4 Done Done Sent false
27784 8010 Acked Read 5 Done Done Sent false
27785 8011 Acked Read 2 Done Done Sent false
27786 8012 Acked Write 8 Done Done Sent false
27787 8013 Acked Flush 0 Done Done Sent false
27788 8014 Acked Write 2 Done Done Sent false
27789 8015 Acked Flush 0 Done Done Sent false
27790 8016 Acked Read 10 Done Done Sent false
27791 8017 Acked Flush 0 Done Done Sent false
27792 8018 Acked Read 9 Done Done Sent false
27793 8019 Acked Read 7 Done Done Sent false
27794 8020 Acked Read 2 Done Done Sent false
27795 8021 Acked Read 10 Done Done Sent false
27796 8022 Acked Read 9 Done Done Sent false
27797 8023 Acked Read 6 Done Done Sent false
27798 8024 Acked Flush 0 Done Done Sent false
27799 8025 Acked Write 5 Done Done Sent false
27800 8026 Acked Read 2 Done Done Sent false
27801 8027 Acked Write 10 Done Done Sent false
27802 8028 Acked Read 6 Done Done Sent false
27803 8029 Acked Read 8 Done Done Sent false
27804 8030 Acked Write 7 Done Done Sent false
27805 8031 Acked Write 10 Done Done Sent false
27806 8032 Acked Read 2 Done Done Sent false
27807 8033 Acked Read 5 Done Done Sent false
27808 8034 Acked Write 7 Done Done Sent false
27809 8035 Acked Write 6 Done Done Sent false
27810 8036 Acked Read 9 Done Done Sent false
27811 8037 Acked Write 2 Done Done Sent false
27812 8038 Acked Read 8 Done Done Sent false
27813 8039 Acked Write 10 Done Done Sent false
27814 8040 Acked Write 8 Done Done Sent false
27815 8041 Acked Read 8 Done Done Sent false
27816 8042 Acked Read 8 Done Done Sent false
27817 8043 Acked Write 6 Done Done Sent false
27818 8044 Acked Read 10 Done Done Sent false
27819 8045 Acked Write 8 Done Done Sent false
27820 8046 Acked Read 4 Done Done Sent false
27821 8047 Acked Read 3 Done Done Sent false
27822 8048 Acked Flush 0 Done Done Sent false
27823 8049 Acked Write 1 Done Done Sent false
27824 8050 Acked Read 2 Done Done Sent false
27825 8051 Acked Read 3 Done Done Sent false
27826 8052 Acked Flush 0 Done Done Sent false
27827 8053 Acked Read 10 Done Done Sent false
27828 8054 Acked Write 10 Done Done Sent false
27829 8055 Acked Read 10 Done Done Sent false
27830 8056 Acked Read 2 Done Done Sent false
27831 8057 Acked Read 8 Done Done Sent false
27832 8058 Acked Write 2 Done Done Sent false
27833 8059 Acked Read 8 Done Done Sent false
27834 8060 Acked Write 2 Done Done Sent false
27835 8061 Acked Read 5 Done Done Sent false
27836 8062 Acked Read 10 Done Done Sent false
27837 8063 Acked Read 1 Done Done Sent false
27838 8064 Acked Write 3 Done Done Sent false
27839 8065 Acked Write 4 Done Done Sent false
27840 8066 Acked Read 1 Done Done Sent false
27841 8067 Acked Read 8 Done Done Sent false
27842 8068 Acked Read 7 Done Done Sent false
27843 8069 Acked Flush 0 Done Done Sent false
27844 8070 Acked Write 8 Done Done Sent false
27845 8071 Acked Write 4 Done Done Sent false
27846 8072 Acked Read 3 Done Done Sent false
27847 8073 Acked Write 3 Done Done Sent false
27848 8074 Acked Write 2 Done Done Sent false
27849 8075 Acked Write 2 Done Done Sent false
27850 8076 Acked Write 3 Done Done Sent false
27851 8077 Acked Write 8 Done Done Sent false
27852 8078 Acked Read 5 Done Done Sent false
27853 8079 Acked Read 5 Done Done Sent false
27854 8080 Acked Write 10 Done Done Sent false
27855 8081 Acked Read 4 Done Done Sent false
27856 8082 Acked Read 4 Done Done Sent false
27857 8083 Acked Read 6 Done Done Sent false
27858 8084 Acked Read 7 Done Done Sent false
27859 8085 Acked Read 10 Done Done Sent false
27860 8086 Acked Read 3 Done Done Sent false
27861 8087 Acked Read 7 Done Done Sent false
27862 8088 Acked Read 7 Done Done Sent false
27863 8089 Acked Write 8 Done Done Sent false
27864 8090 Acked Write 2 Done Done Sent false
27865 8091 Acked Read 4 Done Done Sent false
27866 8092 Acked Read 5 Done Done Sent false
27867 8093 Acked Write 8 Done Done Sent false
27868 8094 Acked Read 5 Done Done Sent false
27869 8095 Acked Read 9 Done Done Sent false
27870 8096 Acked Read 1 Done Done Sent false
27871 8097 Acked Read 2 Done Done Sent false
27872 8098 Acked Write 2 Done Done Sent false
27873 8099 Acked Write 3 Done Done Sent false
27874 8100 Acked Read 7 Done Done Sent false
27875 8101 Acked Write 2 Done Done Sent false
27876 8102 Acked Write 1 Done Done Sent false
27877 8103 Acked Write 3 Done Done Sent false
27878 8104 Acked Read 9 Done Done Sent false
27879 8105 Acked Read 8 Done Done Sent false
27880 8106 Acked Write 2 Done Done Sent false
27881 8107 Acked Read 3 Done Done Sent false
27882 8108 Acked Write 6 Done Done Sent false
27883 8109 Acked Read 4 Done Done Sent false
27884 8110 Acked Read 8 Done Done Sent false
27885 8111 Acked Write 5 Done Done Sent false
27886 8112 Acked Write 4 Done Done Sent false
27887 8113 Acked Read 10 Done Done Sent false
27888 8114 Acked Flush 0 Done Done Sent false
27889 8115 Acked Write 1 Done Done Sent false
27890 8116 Acked Write 3 Done Done Sent false
27891 8117 Acked Write 6 Done Done Sent false
27892 8118 Acked Read 9 Done Done Sent false
27893 8119 Acked Read 1 Done Done Sent false
27894 8120 Acked Read 1 Done Done Sent false
27895 8121 Acked Write 1 Done Done Sent false
27896 8122 Acked Read 2 Done Done Sent false
27897 8123 Acked Read 8 Done Done Sent false
27898 8124 Acked Read 5 Done Done Sent false
27899 8125 Acked Read 7 Done Done Sent false
27900 8126 Acked Read 2 Done Done Sent false
27901 8127 Acked Read 9 Done Done Sent false
27902 8128 Acked Read 1 Done Done Sent false
27903 8129 Acked Write 8 Done Done Sent false
27904 8130 Acked Read 2 Done Done Sent false
27905 8131 Acked Write 6 Done Done Sent false
27906 8132 Acked Read 10 Done Done Sent false
27907 8133 Acked Read 8 Done Done Sent false
27908 8134 Acked Write 9 Done Done Sent false
27909 8135 Acked Read 8 Done Done Sent false
27910 8136 Acked Write 7 Done Done Sent false
27911 8137 Acked Read 6 Done Done Sent false
27912 8138 Acked Read 5 Done Done Sent false
27913 8139 Acked Read 7 Done Done Sent false
27914 8140 Acked Read 10 Done Done Sent false
27915 8141 Acked Write 7 Done Done Sent false
27916 8142 Acked Read 7 Done Done Sent false
27917 8143 Acked Read 1 Done Done Sent false
27918 8144 Acked Write 8 Done Done Sent false
27919 8145 Acked Read 7 Done Done Sent false
27920 8146 Acked Read 2 Done Done Sent false
27921 8147 Acked Write 8 Done Done Sent false
27922 8148 Acked Read 1 Done Done Sent false
27923 8149 Acked Write 1 Done Done Sent false
27924 8150 Acked Read 2 Done Done Sent false
27925 8151 Acked Read 7 Done Done Sent false
27926 8152 Acked Write 9 Done Done Sent false
27927 8153 Acked Read 10 Done Done Sent false
27928 8154 Acked Write 2 Done Done Sent false
27929 8155 Acked Read 6 Done Done Sent false
27930 8156 Acked Write 8 Done Done Sent false
27931 8157 Acked Write 4 Done Done Sent false
27932 8158 Acked Write 1 Done Done Sent false
27933 8159 Acked Write 1 Done Done Sent false
27934 8160 Acked Read 5 Done Done Sent false
27935 8161 Acked Write 9 Done Done Sent false
27936 8162 Acked Read 10 Done Done Sent false
27937 8163 Acked Read 9 Done Done Sent false
27938 8164 Acked Read 1 Done Done Sent false
27939 8165 Acked Write 9 Done Done Sent false
27940 8166 Acked Flush 0 Done Done Sent false
27941 8167 Acked Read 7 Done Done Sent false
27942 8168 Acked Flush 0 Done Done Sent false
27943 8169 Acked Write 9 Done Done Sent false
27944 8170 Acked Flush 0 Done Done Sent false
27945 8171 Acked Read 1 Done Done Sent false
27946 8172 Acked Write 2 Done Done Sent false
27947 8173 Acked Write 10 Done Done Sent false
27948 8174 Acked Flush 0 Done Done Sent false
27949 8175 Acked Read 4 Done Done Sent false
27950 8176 Acked Read 8 Done Done Sent false
27951 8177 Acked Read 8 Done Done Sent false
27952 8178 Acked Flush 0 Done Done Sent false
27953 8179 Acked Write 5 Done Done Sent false
27954 8180 Acked Write 7 Done Done Sent false
27955 8181 Acked Write 2 Done Done Sent false
27956 8182 Acked Write 6 Done Done Sent false
27957 8183 Acked Write 1 Done Done Sent false
27958 8184 Acked Write 1 Done Done Sent false
27959 8185 Acked Read 4 Done Done Sent false
27960 8186 Acked Read 4 Done Done Sent false
27961 8187 Acked Read 4 Done Done Sent false
27962 8188 Acked Read 5 Done Done Sent false
27963 8189 Acked Write 7 Done Done Sent false
27964 8190 Acked Write 3 Done Done Sent false
27965 8191 Acked Read 3 Done Done Sent false
27966 8192 Acked Read 8 Done Done Sent false
27967 8193 Acked Flush 0 Done Done Sent false
27968 8194 Acked Read 8 Done Done Sent false
27969 8195 Acked Flush 0 Done Done Sent false
27970 8196 Acked Read 9 Done Done Sent false
27971 8197 Acked Write 10 Done Done Sent false
27972 8198 Acked Read 3 Done Done Sent false
27973 8199 Acked Read 5 Done Done Sent false
27974 8200 Acked Read 3 Done Done Sent false
27975 8201 Acked Write 4 Done Done Sent false
27976 8202 Acked Write 8 Done Done Sent false
27977 8203 Acked Read 2 Done Done Sent false
27978 8204 Acked Write 1 Done Done Sent false
27979 8205 Acked Read 10 Done Done Sent false
27980 8206 Acked Read 3 Done Done Sent false
27981 8207 Acked Write 8 Done Done Sent false
27982 8208 Acked Read 1 Done Done Sent false
27983 8209 Acked Read 8 Done Done Sent false
27984 8210 Acked Write 3 Done Done Sent false
27985 8211 Acked Read 5 Done Done Sent false
27986 8212 Acked Write 5 Done Done Sent false
27987 8213 Acked Read 2 Done Done Sent false
27988 8214 Acked Read 6 Done Done Sent false
27989 8215 Acked Read 4 Done Done Sent false
27990 8216 Acked Read 1 Done Done Sent false
27991 8217 Acked Read 6 Done Done Sent false
27992 8218 Acked Read 3 Done Done Sent false
27993 8219 Acked Flush 0 Done Done Sent false
27994 8220 Acked Write 6 Done Done Sent false
27995 8221 Acked Write 5 Done Done Sent false
27996 8222 Acked Write 7 Done Done Sent false
27997 8223 Acked Write 2 Done Done Sent false
27998 8224 Acked Write 3 Done Done Sent false
27999 8225 Acked Flush 0 Done Done Sent false
28000 8226 Acked Write 3 Done Done Sent false
28001 8227 Acked Write 2 Done Done Sent false
28002 8228 Acked Flush 0 Done Done Sent false
28003 8229 Acked Read 3 Done Done Sent false
28004 8230 Acked Read 1 Done Done Sent false
28005 8231 Acked Read 8 Done Done Sent false
28006 8232 Acked Read 6 Done Done Sent false
28007 8233 Acked Write 2 Done Done Sent false
28008 8234 Acked Read 8 Done Done Sent false
28009 8235 Acked Read 4 Done Done Sent false
28010 8236 Acked Read 6 Done Done Sent false
28011 8237 Acked Write 3 Done Done Sent false
28012 8238 Acked Write 2 Done Done Sent false
28013 8239 Acked Write 2 Done Done Sent false
28014 8240 Acked Write 10 Done Done Sent false
28015 8241 Acked Read 6 Done Done Sent false
28016 8242 Acked Write 3 Done Done Sent false
28017 8243 Acked Write 1 Done Done Sent false
28018 8244 Acked Flush 0 Done Done Sent false
28019 STATES DS:0 DS:1 DS:2 TOTAL
28020 Sent 0 0 301 301
28021 Done 301 301 0 602
28022 Skipped 0 0 0 0
28023 Error 0 0 0 0
28024 Last Flush: 8244 8244 7943
28025 Downstairs last five completed: 7943 7942 7941 7940 7939
28026 Upstairs last five completed: 8244 8243 8242 8241 8240
28027 CLIENT: Up:0 ds:301 act:2
28028 ----------------------------------------------------------------
28029 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
28030 States: Active Active Offline
28031 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
28032 7944 Acked Read 5 Done Done Sent false
28033 7945 Acked Write 2 Done Done Sent false
28034 7946 Acked Read 7 Done Done Sent false
28035 7947 Acked Read 5 Done Done Sent false
28036 7948 Acked Write 6 Done Done Sent false
28037 7949 Acked Read 10 Done Done Sent false
28038 7950 Acked Write 7 Done Done Sent false
28039 7951 Acked Write 3 Done Done Sent false
28040 7952 Acked Write 10 Done Done Sent false
28041 7953 Acked Read 2 Done Done Sent false
28042 7954 Acked Read 3 Done Done Sent false
28043 7955 Acked Read 10 Done Done Sent false
28044 7956 Acked Write 6 Done Done Sent false
28045 7957 Acked Read 5 Done Done Sent false
28046 7958 Acked Write 5 Done Done Sent false
28047 7959 Acked Read 6 Done Done Sent false
28048 7960 Acked Flush 0 Done Done Sent false
28049 7961 Acked Read 10 Done Done Sent false
28050 7962 Acked Write 6 Done Done Sent false
28051 7963 Acked Flush 0 Done Done Sent false
28052 7964 Acked Read 7 Done Done Sent false
28053 7965 Acked Read 1 Done Done Sent false
28054 7966 Acked Write 5 Done Done Sent false
28055 7967 Acked Write 7 Done Done Sent false
28056 7968 Acked Read 2 Done Done Sent false
28057 7969 Acked Write 4 Done Done Sent false
28058 7970 Acked Write 10 Done Done Sent false
28059 7971 Acked Read 1 Done Done Sent false
28060 7972 Acked Write 3 Done Done Sent false
28061 7973 Acked Read 3 Done Done Sent false
28062 7974 Acked Read 9 Done Done Sent false
28063 7975 Acked Write 3 Done Done Sent false
28064 7976 Acked Write 2 Done Done Sent false
28065 7977 Acked Read 6 Done Done Sent false
28066 7978 Acked Read 2 Done Done Sent false
28067 7979 Acked Read 7 Done Done Sent false
28068 7980 Acked Read 1 Done Done Sent false
28069 7981 Acked Read 9 Done Done Sent false
28070 7982 Acked Write 4 Done Done Sent false
28071 7983 Acked Read 8 Done Done Sent false
28072 7984 Acked Write 7 Done Done Sent false
28073 7985 Acked Read 2 Done Done Sent false
28074 7986 Acked Read 7 Done Done Sent false
28075 7987 Acked Read 3 Done Done Sent false
28076 7988 Acked Write 3 Done Done Sent false
28077 7989 Acked Read 4 Done Done Sent false
28078 7990 Acked Write 4 Done Done Sent false
28079 7991 Acked Read 5 Done Done Sent false
28080 7992 Acked Write 1 Done Done Sent false
28081 7993 Acked Flush 0 Done Done Sent false
28082 7994 Acked Read 7 Done Done Sent false
28083 7995 Acked Flush 0 Done Done Sent false
28084 7996 Acked Read 10 Done Done Sent false
28085 7997 Acked Read 5 Done Done Sent false
28086 7998 Acked Read 7 Done Done Sent false
28087 7999 Acked Write 10 Done Done Sent false
28088 8000 Acked Flush 0 Done Done Sent false
28089 8001 Acked Read 6 Done Done Sent false
28090 8002 Acked Read 7 Done Done Sent false
28091 8003 Acked Write 6 Done Done Sent false
28092 8004 Acked Flush 0 Done Done Sent false
28093 8005 Acked Read 8 Done Done Sent false
28094 8006 Acked Flush 0 Done Done Sent false
28095 8007 Acked Read 10 Done Done Sent false
28096 8008 Acked Write 6 Done Done Sent false
28097 8009 Acked Read 4 Done Done Sent false
28098 8010 Acked Read 5 Done Done Sent false
28099 8011 Acked Read 2 Done Done Sent false
28100 8012 Acked Write 8 Done Done Sent false
28101 8013 Acked Flush 0 Done Done Sent false
28102 8014 Acked Write 2 Done Done Sent false
28103 8015 Acked Flush 0 Done Done Sent false
28104 8016 Acked Read 10 Done Done Sent false
28105 8017 Acked Flush 0 Done Done Sent false
28106 8018 Acked Read 9 Done Done Sent false
28107 8019 Acked Read 7 Done Done Sent false
28108 8020 Acked Read 2 Done Done Sent false
28109 8021 Acked Read 10 Done Done Sent false
28110 8022 Acked Read 9 Done Done Sent false
28111 8023 Acked Read 6 Done Done Sent false
28112 8024 Acked Flush 0 Done Done Sent false
28113 8025 Acked Write 5 Done Done Sent false
28114 8026 Acked Read 2 Done Done Sent false
28115 8027 Acked Write 10 Done Done Sent false
28116 8028 Acked Read 6 Done Done Sent false
28117 8029 Acked Read 8 Done Done Sent false
28118 8030 Acked Write 7 Done Done Sent false
28119 8031 Acked Write 10 Done Done Sent false
28120 8032 Acked Read 2 Done Done Sent false
28121 8033 Acked Read 5 Done Done Sent false
28122 8034 Acked Write 7 Done Done Sent false
28123 8035 Acked Write 6 Done Done Sent false
28124 8036 Acked Read 9 Done Done Sent false
28125 8037 Acked Write 2 Done Done Sent false
28126 8038 Acked Read 8 Done Done Sent false
28127 8039 Acked Write 10 Done Done Sent false
28128 8040 Acked Write 8 Done Done Sent false
28129 8041 Acked Read 8 Done Done Sent false
28130 8042 Acked Read 8 Done Done Sent false
28131 8043 Acked Write 6 Done Done Sent false
28132 8044 Acked Read 10 Done Done Sent false
28133 8045 Acked Write 8 Done Done Sent false
28134 8046 Acked Read 4 Done Done Sent false
28135 8047 Acked Read 3 Done Done Sent false
28136 8048 Acked Flush 0 Done Done Sent false
28137 8049 Acked Write 1 Done Done Sent false
28138 8050 Acked Read 2 Done Done Sent false
28139 8051 Acked Read 3 Done Done Sent false
28140 8052 Acked Flush 0 Done Done Sent false
28141 8053 Acked Read 10 Done Done Sent false
28142 8054 Acked Write 10 Done Done Sent false
28143 8055 Acked Read 10 Done Done Sent false
28144 8056 Acked Read 2 Done Done Sent false
28145 8057 Acked Read 8 Done Done Sent false
28146 8058 Acked Write 2 Done Done Sent false
28147 8059 Acked Read 8 Done Done Sent false
28148 8060 Acked Write 2 Done Done Sent false
28149 8061 Acked Read 5 Done Done Sent false
28150 8062 Acked Read 10 Done Done Sent false
28151 8063 Acked Read 1 Done Done Sent false
28152 8064 Acked Write 3 Done Done Sent false
28153 8065 Acked Write 4 Done Done Sent false
28154 8066 Acked Read 1 Done Done Sent false
28155 8067 Acked Read 8 Done Done Sent false
28156 8068 Acked Read 7 Done Done Sent false
28157 8069 Acked Flush 0 Done Done Sent false
28158 8070 Acked Write 8 Done Done Sent false
28159 8071 Acked Write 4 Done Done Sent false
28160 8072 Acked Read 3 Done Done Sent false
28161 8073 Acked Write 3 Done Done Sent false
28162 8074 Acked Write 2 Done Done Sent false
28163 8075 Acked Write 2 Done Done Sent false
28164 8076 Acked Write 3 Done Done Sent false
28165 8077 Acked Write 8 Done Done Sent false
28166 8078 Acked Read 5 Done Done Sent false
28167 8079 Acked Read 5 Done Done Sent false
28168 8080 Acked Write 10 Done Done Sent false
28169 8081 Acked Read 4 Done Done Sent false
28170 8082 Acked Read 4 Done Done Sent false
28171 8083 Acked Read 6 Done Done Sent false
28172 8084 Acked Read 7 Done Done Sent false
28173 8085 Acked Read 10 Done Done Sent false
28174 8086 Acked Read 3 Done Done Sent false
28175 8087 Acked Read 7 Done Done Sent false
28176 8088 Acked Read 7 Done Done Sent false
28177 8089 Acked Write 8 Done Done Sent false
28178 8090 Acked Write 2 Done Done Sent false
28179 8091 Acked Read 4 Done Done Sent false
28180 8092 Acked Read 5 Done Done Sent false
28181 8093 Acked Write 8 Done Done Sent false
28182 8094 Acked Read 5 Done Done Sent false
28183 8095 Acked Read 9 Done Done Sent false
28184 8096 Acked Read 1 Done Done Sent false
28185 8097 Acked Read 2 Done Done Sent false
28186 8098 Acked Write 2 Done Done Sent false
28187 8099 Acked Write 3 Done Done Sent false
28188 8100 Acked Read 7 Done Done Sent false
28189 8101 Acked Write 2 Done Done Sent false
28190 8102 Acked Write 1 Done Done Sent false
28191 8103 Acked Write 3 Done Done Sent false
28192 8104 Acked Read 9 Done Done Sent false
28193 8105 Acked Read 8 Done Done Sent false
28194 8106 Acked Write 2 Done Done Sent false
28195 8107 Acked Read 3 Done Done Sent false
28196 8108 Acked Write 6 Done Done Sent false
28197 8109 Acked Read 4 Done Done Sent false
28198 8110 Acked Read 8 Done Done Sent false
28199 8111 Acked Write 5 Done Done Sent false
28200 8112 Acked Write 4 Done Done Sent false
28201 8113 Acked Read 10 Done Done Sent false
28202 8114 Acked Flush 0 Done Done Sent false
28203 8115 Acked Write 1 Done Done Sent false
28204 8116 Acked Write 3 Done Done Sent false
28205 8117 Acked Write 6 Done Done Sent false
28206 8118 Acked Read 9 Done Done Sent false
28207 8119 Acked Read 1 Done Done Sent false
28208 8120 Acked Read 1 Done Done Sent false
28209 8121 Acked Write 1 Done Done Sent false
28210 8122 Acked Read 2 Done Done Sent false
28211 8123 Acked Read 8 Done Done Sent false
28212 8124 Acked Read 5 Done Done Sent false
28213 8125 Acked Read 7 Done Done Sent false
28214 8126 Acked Read 2 Done Done Sent false
28215 8127 Acked Read 9 Done Done Sent false
28216 8128 Acked Read 1 Done Done Sent false
28217 8129 Acked Write 8 Done Done Sent false
28218 8130 Acked Read 2 Done Done Sent false
28219 8131 Acked Write 6 Done Done Sent false
28220 8132 Acked Read 10 Done Done Sent false
28221 8133 Acked Read 8 Done Done Sent false
28222 8134 Acked Write 9 Done Done Sent false
28223 8135 Acked Read 8 Done Done Sent false
28224 8136 Acked Write 7 Done Done Sent false
28225 8137 Acked Read 6 Done Done Sent false
28226 8138 Acked Read 5 Done Done Sent false
28227 8139 Acked Read 7 Done Done Sent false
28228 8140 Acked Read 10 Done Done Sent false
28229 8141 Acked Write 7 Done Done Sent false
28230 8142 Acked Read 7 Done Done Sent false
28231 8143 Acked Read 1 Done Done Sent false
28232 8144 Acked Write 8 Done Done Sent false
28233 8145 Acked Read 7 Done Done Sent false
28234 8146 Acked Read 2 Done Done Sent false
28235 8147 Acked Write 8 Done Done Sent false
28236 8148 Acked Read 1 Done Done Sent false
28237 8149 Acked Write 1 Done Done Sent false
28238 8150 Acked Read 2 Done Done Sent false
28239 8151 Acked Read 7 Done Done Sent false
28240 8152 Acked Write 9 Done Done Sent false
28241 8153 Acked Read 10 Done Done Sent false
28242 8154 Acked Write 2 Done Done Sent false
28243 8155 Acked Read 6 Done Done Sent false
28244 8156 Acked Write 8 Done Done Sent false
28245 8157 Acked Write 4 Done Done Sent false
28246 8158 Acked Write 1 Done Done Sent false
28247 8159 Acked Write 1 Done Done Sent false
28248 8160 Acked Read 5 Done Done Sent false
28249 8161 Acked Write 9 Done Done Sent false
28250 8162 Acked Read 10 Done Done Sent false
28251 8163 Acked Read 9 Done Done Sent false
28252 8164 Acked Read 1 Done Done Sent false
28253 8165 Acked Write 9 Done Done Sent false
28254 8166 Acked Flush 0 Done Done Sent false
28255 8167 Acked Read 7 Done Done Sent false
28256 8168 Acked Flush 0 Done Done Sent false
28257 8169 Acked Write 9 Done Done Sent false
28258 8170 Acked Flush 0 Done Done Sent false
28259 8171 Acked Read 1 Done Done Sent false
28260 8172 Acked Write 2 Done Done Sent false
28261 8173 Acked Write 10 Done Done Sent false
28262 8174 Acked Flush 0 Done Done Sent false
28263 8175 Acked Read 4 Done Done Sent false
28264 8176 Acked Read 8 Done Done Sent false
28265 8177 Acked Read 8 Done Done Sent false
28266 8178 Acked Flush 0 Done Done Sent false
28267 8179 Acked Write 5 Done Done Sent false
28268 8180 Acked Write 7 Done Done Sent false
28269 8181 Acked Write 2 Done Done Sent false
28270 8182 Acked Write 6 Done Done Sent false
28271 8183 Acked Write 1 Done Done Sent false
28272 8184 Acked Write 1 Done Done Sent false
28273 8185 Acked Read 4 Done Done Sent false
28274 8186 Acked Read 4 Done Done Sent false
28275 8187 Acked Read 4 Done Done Sent false
28276 8188 Acked Read 5 Done Done Sent false
28277 8189 Acked Write 7 Done Done Sent false
28278 8190 Acked Write 3 Done Done Sent false
28279 8191 Acked Read 3 Done Done Sent false
28280 8192 Acked Read 8 Done Done Sent false
28281 8193 Acked Flush 0 Done Done Sent false
28282 8194 Acked Read 8 Done Done Sent false
28283 8195 Acked Flush 0 Done Done Sent false
28284 8196 Acked Read 9 Done Done Sent false
28285 8197 Acked Write 10 Done Done Sent false
28286 8198 Acked Read 3 Done Done Sent false
28287 8199 Acked Read 5 Done Done Sent false
28288 8200 Acked Read 3 Done Done Sent false
28289 8201 Acked Write 4 Done Done Sent false
28290 8202 Acked Write 8 Done Done Sent false
28291 8203 Acked Read 2 Done Done Sent false
28292 8204 Acked Write 1 Done Done Sent false
28293 8205 Acked Read 10 Done Done Sent false
28294 8206 Acked Read 3 Done Done Sent false
28295 8207 Acked Write 8 Done Done Sent false
28296 8208 Acked Read 1 Done Done Sent false
28297 8209 Acked Read 8 Done Done Sent false
28298 8210 Acked Write 3 Done Done Sent false
28299 8211 Acked Read 5 Done Done Sent false
28300 8212 Acked Write 5 Done Done Sent false
28301 8213 Acked Read 2 Done Done Sent false
28302 8214 Acked Read 6 Done Done Sent false
28303 8215 Acked Read 4 Done Done Sent false
28304 8216 Acked Read 1 Done Done Sent false
28305 8217 Acked Read 6 Done Done Sent false
28306 8218 Acked Read 3 Done Done Sent false
28307 8219 Acked Flush 0 Done Done Sent false
28308 8220 Acked Write 6 Done Done Sent false
28309 8221 Acked Write 5 Done Done Sent false
28310 8222 Acked Write 7 Done Done Sent false
28311 8223 Acked Write 2 Done Done Sent false
28312 8224 Acked Write 3 Done Done Sent false
28313 8225 Acked Flush 0 Done Done Sent false
28314 8226 Acked Write 3 Done Done Sent false
28315 8227 Acked Write 2 Done Done Sent false
28316 8228 Acked Flush 0 Done Done Sent false
28317 8229 Acked Read 3 Done Done Sent false
28318 8230 Acked Read 1 Done Done Sent false
28319 8231 Acked Read 8 Done Done Sent false
28320 8232 Acked Read 6 Done Done Sent false
28321 8233 Acked Write 2 Done Done Sent false
28322 8234 Acked Read 8 Done Done Sent false
28323 8235 Acked Read 4 Done Done Sent false
28324 8236 Acked Read 6 Done Done Sent false
28325 8237 Acked Write 3 Done Done Sent false
28326 8238 Acked Write 2 Done Done Sent false
28327 8239 Acked Write 2 Done Done Sent false
28328 8240 Acked Write 10 Done Done Sent false
28329 8241 Acked Read 6 Done Done Sent false
28330 8242 Acked Write 3 Done Done Sent false
28331 8243 Acked Write 1 Done Done Sent false
28332 8244 Acked Flush 0 Done Done Sent false
28333 STATES DS:0 DS:1 DS:2 TOTAL
28334 Sent 0 0 301 301
28335 Done 301 301 0 602
28336 Skipped 0 0 0 0
28337 Error 0 0 0 0
28338 Last Flush: 8244 8244 7943
28339 Downstairs last five completed: 7943 7942 7941 7940 7939
28340 Upstairs last five completed: 8244 8243 8242 8241 8240
28341 CLIENT: Up:0 ds:301 act:2
283422024-11-26T22:10:30.029ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
283432024-11-26T22:10:30.029ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:65121) = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
283442024-11-26T22:10:30.030ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
283452024-11-26T22:10:30.030ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
283462024-11-26T22:10:30.030ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
283472024-11-26T22:10:30.030ZINFOcrucible: send last flush ID to this DS: 7943 = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
283482024-11-26T22:10:30.031ZINFOcrucible: Replied this last flush ID: 7943 = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
283492024-11-26T22:10:30.031ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
283502024-11-26T22:10:30.031ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
283512024-11-26T22:10:30.031ZINFOcrucible: [2] client re-new 301 jobs since flush 7943 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
283522024-11-26T22:10:30.031ZINFOcrucible: [2] Replayed 301 jobs since flush: 7943 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
28353 ----------------------------------------------------------------
28354 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
28355 Downstairs last five completed: 8244 8243 8242 8241 8240
28356 Upstairs last five completed: 8244 8243 8242 8241 8240
28357 CLIENT: Up:0 ds:0 act:3
28358 Replay: All jobs finished
28359 Main task has work to do, go find it
28360 got dsc Stop(1)
28361 stop 1
28362 [8820] Got stop action so:false kr:false
28363 Nov 26 22:10:30.275 INFO request completed, latency_us: 333, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 919de87d-6e37-4a4c-8ace-effaa2386023, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
283642024-11-26T22:10:30.276ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
283652024-11-26T22:10:30.276ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
283662024-11-26T22:10:30.276ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
283672024-11-26T22:10:30.276ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
283682024-11-26T22:10:30.276ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
283692024-11-26T22:10:30.276ZINFOcrucible: client task is exiting = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
28370 Nov 26 22:10:30.276 INFO request completed, latency_us: 112, response_code: 200, uri: /state/cid/1, method: GET, req_id: daa4ade8-f5e2-437d-bece-8ecb5f2b9aac, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
28371 [8820] Exited with: ExitStatus(unix_wait_status(9))
28372 [8820][1] reports Exit
28373 Nov 26 22:10:34.278 INFO request completed, latency_us: 213, response_code: 200, uri: /state/cid/1, method: GET, req_id: d2ab20ac-7773-429f-b497-74767da6addf, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
28374 001/300 Write block 3177 len 24576 data: 5 5 4 5 4 5
28375 002/300 Write block 4636 len 20480 data: 7 7 7 8 7
28376 003/300 Read block 2008 len 24576
28377 004/300 Read block 771 len 20480
28378 005/300 Read block 3249 len 12288
28379 006/300 Write block 127 len 4096 data: 8
28380 007/300 Read block 1036 len 12288
28381 008/300 Read block 1066 len 16384
28382 009/300 Read block 772 len 36864
28383 010/300 Write block 3566 len 40960 data: 6 5 7 8 8 7 7 7 6 5
28384 011/300 Write block 975 len 16384 data: 3 3 3 3
28385 012/300 Read block 4457 len 36864
28386 013/300 Read block 86 len 16384
28387 014/300 Read block 522 len 20480
28388 015/300 Read block 402 len 12288
28389 016/300 Read block 4749 len 32768
28390 017/300 Read block 935 len 4096
28391 018/300 Read block 4372 len 32768
28392 019/300 Write block 1923 len 12288 data: 6 5 5
28393 020/300 Read block 4367 len 8192
28394 021/300 Read block 1086 len 20480
28395 022/300 Write block 4084 len 8192 data: 4 3
28396 023/300 Write block 4966 len 8192 data: 8 5
28397 024/300 Write block 4457 len 40960 data: 6 5 4 4 4 5 4 4 6 5
28398 025/300 Flush
28399 026/300 Read block 531 len 20480
28400 027/300 Flush
28401 028/300 Read block 4278 len 8192
28402 029/300 Read block 3451 len 40960
28403 030/300 Write block 4726 len 24576 data: 5 6 4 5 5 6
28404 031/300 Read block 2210 len 28672
28405 032/300 Read block 1741 len 12288
28406 033/300 Write block 2090 len 32768 data: 5 4 4 4 2 2 3 3
28407 034/300 Write block 1023 len 20480 data: 6 4 5 6 5
28408 035/300 Read block 4169 len 28672
28409 036/300 Read block 3151 len 8192
28410 037/300 Write block 4778 len 24576 data: 4 3 4 4 4 2
28411 038/300 Read block 2127 len 24576
28412 039/300 Flush
28413 040/300 Read block 1599 len 40960
28414 041/300 Read block 3220 len 36864
28415 042/300 Write block 4527 len 12288 data: 7 8 8
28416 043/300 Write block 3141 len 12288 data: 7 8 6
28417 044/300 Read block 1846 len 36864
28418 045/300 Read block 1693 len 40960
28419 046/300 Read block 138 len 24576
28420 047/300 Write block 3309 len 12288 data: 6 5 5
28421 048/300 Read block 4708 len 12288
28422 049/300 Read block 1657 len 4096
28423 050/300 Write block 3778 len 28672 data: 6 6 7 7 8 7 8
28424 051/300 Read block 1266 len 16384
28425 052/300 Read block 1019 len 16384
28426 053/300 Flush
28427 054/300 Read block 883 len 28672
28428 055/300 Write block 4523 len 24576 data: 3 4 4 6 8 9
28429 056/300 Read block 348 len 32768
28430 057/300 Read block 508 len 32768
28431 058/300 Write block 507 len 8192 data: 5 4
28432 059/300 Write block 1358 len 8192 data: 7 6
28433 060/300 Write block 2598 len 24576 data: 8 8 7 6 5 7
28434 061/300 Write block 761 len 12288 data: 5 4 4
28435 062/300 Flush
28436 063/300 Read block 417 len 20480
28437 064/300 Write block 757 len 32768 data: 4 5 4 4 6 5 5 6
28438 065/300 Read block 874 len 8192
28439 066/300 Read block 1854 len 12288
28440 067/300 Read block 4301 len 40960
28441 068/300 Read block 3325 len 36864
28442 069/300 Write block 3857 len 28672 data: 9 8 7 6 7 5 3
28443 070/300 Write block 185 len 28672 data: 5 4 4 5 4 6 5
28444 071/300 Read block 2371 len 40960
28445 072/300 Read block 2040 len 24576
28446 073/300 Read block 19 len 24576
28447 074/300 Read block 4815 len 32768
28448 075/300 Write block 1735 len 16384 data: 6 5 5 4
28449 076/300 Write block 1577 len 20480 data: 5 4 4 7 6
28450 077/300 Flush
28451 078/300 Read block 2676 len 12288
28452 079/300 Read block 835 len 12288
28453 080/300 Read block 2076 len 8192
28454 081/300 Read block 3589 len 16384
28455 082/300 Read block 929 len 16384
28456 083/300 Read block 474 len 36864
28457 084/300 Write block 1432 len 32768 data: 5 5 5 6 4 4 4 3
28458 085/300 Write block 1864 len 28672 data: 3 3 4 4 5 5 7
28459 086/300 Flush
28460 087/300 Flush
28461 088/300 Read block 2167 len 24576
28462 089/300 Read block 3286 len 36864
28463 090/300 Write block 3144 len 32768 data: 7 8 6 6 5 5 4 4
28464 091/300 Write block 1170 len 32768 data: 10 8 6 7 6 6 8 6
28465 092/300 Read block 774 len 40960
28466 093/300 Read block 3840 len 36864
28467 094/300 Read block 2277 len 28672
28468 095/300 Read block 2368 len 8192
28469 096/300 Read block 3572 len 40960
28470 097/300 Write block 2722 len 4096 data: 6
28471 098/300 Write block 4760 len 20480 data: 5 4 4 6 5
28472 099/300 Write block 4444 len 40960 data: 8 6 6 6 7 8 7 8 8 7
28473 100/300 Read block 3642 len 16384
28474 101/300 Flush
28475 102/300 Write block 3132 len 36864 data: 3 4 5 6 7 8 7 7 8
28476 103/300 Flush
28477 104/300 Write block 398 len 4096 data: 5
28478 105/300 Read block 3288 len 40960
28479 106/300 Write block 3417 len 20480 data: 4 4 4 3 4
28480 107/300 Read block 3560 len 36864
28481 108/300 Write block 4133 len 40960 data: 4 4 4 4 2 2 2 3 4 7
28482 109/300 Read block 3385 len 24576
28483 110/300 Write block 1748 len 40960 data: 5 6 6 6 5 5 5 5 5 6
28484 111/300 Write block 4196 len 40960 data: 6 8 8 8 7 6 6 6 4 3
28485 112/300 Write block 1744 len 8192 data: 2 4
28486 113/300 Write block 4563 len 40960 data: 5 6 5 3 3 3 3 5 5 5
28487 114/300 Read block 4301 len 4096
28488 115/300 Read block 2891 len 20480
28489 116/300 Flush
28490 117/300 Write block 1406 len 4096 data: 5
28491 118/300 Read block 1516 len 32768
28492 119/300 Write block 4152 len 12288 data: 5 3 4
28493 120/300 Read block 2351 len 4096
28494 121/300 Write block 4158 len 12288 data: 6 7 7
28495 122/300 Write block 923 len 28672 data: 5 6 6 8 8 7 9
28496 123/300 Read block 1228 len 36864
28497 124/300 Write block 518 len 24576 data: 4 4 4 5 5 5
28498 125/300 Read block 1023 len 8192
28499 126/300 Write block 3959 len 4096 data: 4
28500 127/300 Flush
28501 128/300 Write block 3788 len 16384 data: 3 3 4 5
28502 129/300 Write block 833 len 8192 data: 3 5
28503 130/300 Write block 2903 len 12288 data: 6 6 6
28504 131/300 Read block 1242 len 20480
28505 132/300 Read block 817 len 32768
28506 133/300 Write block 2027 len 24576 data: 5 5 5 4 4 6
28507 134/300 Read block 3303 len 4096
28508 135/300 Flush
28509 136/300 Read block 3343 len 8192
28510 137/300 Write block 679 len 40960 data: 4 4 4 4 3 4 3 3 2 2
28511 138/300 Read block 3297 len 28672
28512 139/300 Read block 1697 len 8192
28513 140/300 Write block 4201 len 24576 data: 7 7 7 5 4 4
28514 141/300 Write block 347 len 4096 data: 8
28515 142/300 Read block 2579 len 40960
28516 143/300 Read block 2037 len 12288
28517 144/300 Read block 4496 len 4096
28518 145/300 Write block 4269 len 28672 data: 5 6 5 4 4 3 4
28519 146/300 Read block 1282 len 36864
28520 147/300 Read block 142 len 16384
28521 148/300 Read block 2132 len 20480
28522 149/300 Flush
28523 150/300 Read block 3331 len 12288
28524 151/300 Flush
28525 152/300 Read block 3934 len 24576
28526 153/300 Flush
28527 154/300 Read block 1995 len 32768
28528 155/300 Read block 13 len 24576
28529 156/300 Read block 2346 len 36864
28530 157/300 Write block 1676 len 32768 data: 5 4 3 5 4 3 5 5
28531 158/300 Write block 2541 len 36864 data: 4 4 3 3 3 4 3 4 4
28532 159/300 Read block 937 len 4096
28533 160/300 Write block 778 len 12288 data: 4 4 2
28534 161/300 Read block 2168 len 32768
28535 162/300 Read block 2384 len 32768
28536 163/300 Read block 2165 len 28672
28537 164/300 Write block 1168 len 32768 data: 11 10 11 9 7 8 7 7
28538 165/300 Read block 2917 len 36864
28539 166/300 Read block 4655 len 36864
28540 167/300 Read block 4845 len 32768
28541 168/300 Write block 4582 len 32768 data: 5 5 3 3 4 5 5 4
28542 169/300 Read block 4832 len 36864
28543 170/300 Read block 2162 len 16384
28544 171/300 Read block 2535 len 32768
28545 172/300 Write block 1473 len 40960 data: 5 5 5 5 7 8 8 8 9 9
28546 173/300 Write block 4685 len 16384 data: 6 4 4 4
28547 174/300 Read block 670 len 32768
28548 175/300 Write block 4511 len 40960 data: 5 4 5 4 5 5 4 4 4 4
28549 176/300 Write block 2742 len 8192 data: 7 8
28550 177/300 Write block 3191 len 12288 data: 4 6 7
28551 178/300 Read block 3710 len 8192
28552 179/300 Read block 1314 len 12288
28553 180/300 Write block 3589 len 12288 data: 3 4 5
28554 181/300 Write block 3015 len 8192 data: 7 6
28555 182/300 Write block 3 len 16384 data: 5 6 6 9
28556 183/300 Read block 2106 len 4096
28557 184/300 Read block 3357 len 36864
28558 185/300 Write block 3065 len 28672 data: 2 6 4 4 4 3 3
28559 186/300 Read block 3919 len 20480
28560 187/300 Read block 363 len 20480
28561 188/300 Read block 1841 len 36864
28562 189/300 Write block 854 len 4096 data: 5
28563 190/300 Read block 1645 len 16384
28564 191/300 Read block 1625 len 40960
28565 192/300 Write block 1138 len 16384 data: 5 6 7 7
28566 193/300 Read block 4481 len 24576
28567 194/300 Read block 4588 len 16384
28568 195/300 Read block 653 len 4096
28569 196/300 Read block 2560 len 40960
28570 197/300 Write block 3045 len 4096 data: 4
28571 198/300 Read block 2990 len 4096
28572 199/300 Read block 4293 len 16384
28573 200/300 Flush
28574 201/300 Write block 3463 len 16384 data: 6 5 2 2
28575 202/300 Read block 3239 len 20480
28576 203/300 Write block 1201 len 12288 data: 7 7 7
28577 204/300 Write block 1721 len 4096 data: 5
28578 205/300 Write block 3082 len 40960 data: 9 8 6 7 6 8 7 6 9 8
28579 206/300 Write block 299 len 32768 data: 3 3 3 4 3 2 2 2
28580 207/300 Read block 2426 len 40960
28581 208/300 Write block 550 len 28672 data: 3 3 3 3 5 5 5
28582 209/300 Read block 4442 len 36864
28583 210/300 Write block 461 len 12288 data: 4 5 4
28584 211/300 Read block 1459 len 24576
28585 212/300 Write block 4185 len 8192 data: 4 4
28586 213/300 Read block 4916 len 8192
28587 214/300 Write block 2708 len 8192 data: 7 9
28588 215/300 Flush
28589 216/300 Write block 1098 len 32768 data: 9 7 7 6 8 10 7 5
28590 217/300 Write block 178 len 4096 data: 6
28591 218/300 Write block 1189 len 4096 data: 4
28592 219/300 Write block 154 len 20480 data: 3 3 3 3 4
28593 220/300 Write block 986 len 8192 data: 3 3
28594 221/300 Write block 1424 len 36864 data: 4 5 5 6 5 5 5 5 6
28595 222/300 Read block 2366 len 16384
28596 223/300 Write block 3756 len 32768 data: 3 5 5 4 4 5 6 7
28597 224/300 Read block 3320 len 36864
28598 225/300 Flush
28599 226/300 Write block 3544 len 40960 data: 4 3 4 3 2 2 3 5 6 6
28600 227/300 Read block 4649 len 28672
28601 228/300 Read block 1353 len 36864
28602 229/300 Read block 4236 len 24576
28603 230/300 Write block 343 len 4096 data: 8
28604 231/300 Read block 1000 len 28672
28605 232/300 Read block 66 len 12288
28606 233/300 Write block 1556 len 24576 data: 6 5 5 5 5 5
28607 234/300 Write block 883 len 28672 data: 3 3 5 5 5 5 6
28608 235/300 Read block 4765 len 40960
28609 236/300 Write block 4464 len 4096 data: 5
28610 237/300 Read block 1321 len 28672
28611 238/300 Read block 1847 len 40960
28612 239/300 Write block 1615 len 36864 data: 5 3 3 4 5 5 6 6 6
28613 240/300 Read block 1551 len 4096
28614 241/300 Flush
28615 242/300 Read block 4792 len 20480
28616 243/300 Read block 399 len 36864
28617 244/300 Read block 3310 len 12288
28618 245/300 Read block 2093 len 4096
28619 246/300 Read block 3954 len 4096
28620 247/300 Write block 79 len 20480 data: 7 6 4 4 4
28621 248/300 Read block 2240 len 8192
28622 249/300 Read block 400 len 36864
28623 250/300 Write block 1844 len 4096 data: 9
28624 251/300 Read block 1765 len 36864
28625 252/300 Read block 2200 len 40960
28626 253/300 Read block 3507 len 40960
28627 254/300 Read block 4158 len 12288
28628 255/300 Write block 2962 len 4096 data: 5
28629 256/300 Read block 846 len 40960
28630 257/300 Write block 1043 len 32768 data: 4 6 9 9 8 9 7 6
28631 258/300 Flush
28632 259/300 Read block 884 len 40960
28633 260/300 Read block 1516 len 24576
28634 261/300 Flush
28635 262/300 Read block 2022 len 20480
28636 263/300 Write block 728 len 20480 data: 5 4 4 4 5
28637 264/300 Write block 1734 len 12288 data: 6 7 6
28638 265/300 Read block 3154 len 16384
28639 266/300 Write block 3627 len 24576 data: 4 4 5 4 6 6
28640 267/300 Read block 582 len 4096
28641 268/300 Read block 3862 len 28672
28642 269/300 Write block 574 len 32768 data: 6 7 6 6 5 4 5 4
28643 270/300 Read block 990 len 20480
28644 271/300 Read block 1109 len 4096
28645 272/300 Flush
28646 273/300 Flush
28647 274/300 Flush
28648 275/300 Write block 826 len 36864 data: 4 4 5 5 5 4 4 4 6
28649 276/300 Read block 2973 len 4096
28650 277/300 Read block 4279 len 36864
28651 278/300 Flush
28652 279/300 Write block 1017 len 36864 data: 3 4 5 6 7 6 7 5 6
28653 280/300 Read block 4401 len 8192
28654 281/300 Read block 1704 len 24576
28655 282/300 Read block 1660 len 20480
28656 283/300 Read block 4321 len 8192
28657 284/300 Read block 178 len 36864
28658 285/300 Read block 250 len 8192
28659 286/300 Read block 4988 len 20480
28660 287/300 Write block 104 len 28672 data: 7 4 4 4 3 3 3
28661 288/300 Write block 891 len 36864 data: 5 7 7 6 4 3 3 3 3
28662 289/300 Read block 259 len 20480
28663 290/300 Read block 43 len 36864
28664 291/300 Write block 4236 len 36864 data: 6 6 5 6 6 5 4 4 4
28665 292/300 Write block 1454 len 32768 data: 4 5 6 6 5 4 5 5
28666 293/300 Flush
28667 294/300 Read block 2552 len 32768
28668 295/300 Read block 4100 len 32768
28669 296/300 Read block 434 len 12288
28670 297/300 Write block 612 len 16384 data: 4 4 5 5
28671 298/300 Read block 1479 len 8192
28672 299/300 Read block 3718 len 28672
28673 300/300 Read block 3290 len 12288
28674 301/300 Read block 4225 len 8192
28675 Main task has work to do, go find it
28676 got dsc Start(1)
28677 start 1
28678 Nov 26 22:10:34.523 INFO request completed, latency_us: 244, response_code: 204, uri: /start/cid/1, method: POST, req_id: 64eccc8a-596d-4c61-a386-2068817e53cd, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
28679 Replay: started 1, returned:Ok(())
28680 ----------------------------------------------------------------
28681 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
28682 States: Active Offline Active
28683 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
28684 8245 Acked Write 6 Done Sent Done false
28685 8246 Acked Write 5 Done Sent Done false
28686 8247 Acked Read 6 Done Sent Done false
28687 8248 Acked Read 5 Done Sent Done false
28688 8249 Acked Read 3 Done Sent Done false
28689 8250 Acked Write 1 Done Sent Done false
28690 8251 Acked Read 3 Done Sent Done false
28691 8252 Acked Read 4 Done Sent Done false
28692 8253 Acked Read 9 Done Sent Done false
28693 8254 Acked Write 10 Done Sent Done false
28694 8255 Acked Write 4 Done Sent Done false
28695 8256 Acked Read 9 Done Sent Done false
28696 8257 Acked Read 4 Done Sent Done false
28697 8258 Acked Read 5 Done Sent Done false
28698 8259 Acked Read 3 Done Sent Done false
28699 8260 Acked Read 8 Done Sent Done false
28700 8261 Acked Read 1 Done Sent Done false
28701 8262 Acked Read 8 Done Sent Done false
28702 8263 Acked Write 3 Done Sent Done false
28703 8264 Acked Read 2 Done Sent Done false
28704 8265 Acked Read 5 Done Sent Done false
28705 8266 Acked Write 2 Done Sent Done false
28706 8267 Acked Write 2 Done Sent Done false
28707 8268 Acked Write 10 Done Sent Done false
28708 8269 Acked Flush 0 Done Sent Done false
28709 8270 Acked Read 5 Done Sent Done false
28710 8271 Acked Flush 0 Done Sent Done false
28711 8272 Acked Read 2 Done Sent Done false
28712 8273 Acked Read 10 Done Sent Done false
28713 8274 Acked Write 6 Done Sent Done false
28714 8275 Acked Read 7 Done Sent Done false
28715 8276 Acked Read 3 Done Sent Done false
28716 8277 Acked Write 8 Done Sent Done false
28717 8278 Acked Write 5 Done Sent Done false
28718 8279 Acked Read 7 Done Sent Done false
28719 8280 Acked Read 2 Done Sent Done false
28720 8281 Acked Write 6 Done Sent Done false
28721 8282 Acked Read 6 Done Sent Done false
28722 8283 Acked Flush 0 Done Sent Done false
28723 8284 Acked Read 10 Done Sent Done false
28724 8285 Acked Read 9 Done Sent Done false
28725 8286 Acked Write 3 Done Sent Done false
28726 8287 Acked Write 3 Done Sent Done false
28727 8288 Acked Read 9 Done Sent Done false
28728 8289 Acked Read 10 Done Sent Done false
28729 8290 Acked Read 6 Done Sent Done false
28730 8291 Acked Write 3 Done Sent Done false
28731 8292 Acked Read 3 Done Sent Done false
28732 8293 Acked Read 1 Done Sent Done false
28733 8294 Acked Write 7 Done Sent Done false
28734 8295 Acked Read 4 Done Sent Done false
28735 8296 Acked Read 4 Done Sent Done false
28736 8297 Acked Flush 0 Done Sent Done false
28737 8298 Acked Read 7 Done Sent Done false
28738 8299 Acked Write 6 Done Sent Done false
28739 8300 Acked Read 8 Done Sent Done false
28740 8301 Acked Read 8 Done Sent Done false
28741 8302 Acked Write 2 Done Sent Done false
28742 8303 Acked Write 2 Done Sent Done false
28743 8304 Acked Write 6 Done Sent Done false
28744 8305 Acked Write 3 Done Sent Done false
28745 8306 Acked Flush 0 Done Sent Done false
28746 8307 Acked Read 5 Done Sent Done false
28747 8308 Acked Write 8 Done Sent Done false
28748 8309 Acked Read 2 Done Sent Done false
28749 8310 Acked Read 3 Done Sent Done false
28750 8311 Acked Read 10 Done Sent Done false
28751 8312 Acked Read 9 Done Sent Done false
28752 8313 Acked Write 7 Done Sent Done false
28753 8314 Acked Write 7 Done Sent Done false
28754 8315 Acked Read 10 Done Sent Done false
28755 8316 Acked Read 6 Done Sent Done false
28756 8317 Acked Read 6 Done Sent Done false
28757 8318 Acked Read 8 Done Sent Done false
28758 8319 Acked Write 4 Done Sent Done false
28759 8320 Acked Write 5 Done Sent Done false
28760 8321 Acked Flush 0 Done Sent Done false
28761 8322 Acked Read 3 Done Sent Done false
28762 8323 Acked Read 3 Done Sent Done false
28763 8324 Acked Read 2 Done Sent Done false
28764 8325 Acked Read 4 Done Sent Done false
28765 8326 Acked Read 4 Done Sent Done false
28766 8327 Acked Read 9 Done Sent Done false
28767 8328 Acked Write 8 Done Sent Done false
28768 8329 Acked Write 7 Done Sent Done false
28769 8330 Acked Flush 0 Done Sent Done false
28770 8331 Acked Flush 0 Done Sent Done false
28771 8332 Acked Read 6 Done Sent Done false
28772 8333 Acked Read 9 Done Sent Done false
28773 8334 Acked Write 8 Done Sent Done false
28774 8335 Acked Write 8 Done Sent Done false
28775 8336 Acked Read 10 Done Sent Done false
28776 8337 Acked Read 9 Done Sent Done false
28777 8338 Acked Read 7 Done Sent Done false
28778 8339 Acked Read 2 Done Sent Done false
28779 8340 Acked Read 10 Done Sent Done false
28780 8341 Acked Write 1 Done Sent Done false
28781 8342 Acked Write 5 Done Sent Done false
28782 8343 Acked Write 10 Done Sent Done false
28783 8344 Acked Read 4 Done Sent Done false
28784 8345 Acked Flush 0 Done Sent Done false
28785 8346 Acked Write 9 Done Sent Done false
28786 8347 Acked Flush 0 Done Sent Done false
28787 8348 Acked Write 1 Done Sent Done false
28788 8349 Acked Read 10 Done Sent Done false
28789 8350 Acked Write 5 Done Sent Done false
28790 8351 Acked Read 9 Done Sent Done false
28791 8352 Acked Write 10 Done Sent Done false
28792 8353 Acked Read 6 Done Sent Done false
28793 8354 Acked Write 10 Done Sent Done false
28794 8355 Acked Write 10 Done Sent Done false
28795 8356 Acked Write 2 Done Sent Done false
28796 8357 Acked Write 10 Done Sent Done false
28797 8358 Acked Read 1 Done Sent Done false
28798 8359 Acked Read 5 Done Sent Done false
28799 8360 Acked Flush 0 Done Sent Done false
28800 8361 Acked Write 1 Done Sent Done false
28801 8362 Acked Read 8 Done Sent Done false
28802 8363 Acked Write 3 Done Sent Done false
28803 8364 Acked Read 1 Done Sent Done false
28804 8365 Acked Write 3 Done Sent Done false
28805 8366 Acked Write 7 Done Sent Done false
28806 8367 Acked Read 9 Done Sent Done false
28807 8368 Acked Write 6 Done Sent Done false
28808 8369 Acked Read 2 Done Sent Done false
28809 8370 Acked Write 1 Done Sent Done false
28810 8371 Acked Flush 0 Done Sent Done false
28811 8372 Acked Write 4 Done Sent Done false
28812 8373 Acked Write 2 Done Sent Done false
28813 8374 Acked Write 3 Done Sent Done false
28814 8375 Acked Read 5 Done Sent Done false
28815 8376 Acked Read 8 Done Sent Done false
28816 8377 Acked Write 6 Done Sent Done false
28817 8378 Acked Read 1 Done Sent Done false
28818 8379 Acked Flush 0 Done Sent Done false
28819 8380 Acked Read 2 Done Sent Done false
28820 8381 Acked Write 10 Done Sent Done false
28821 8382 Acked Read 7 Done Sent Done false
28822 8383 Acked Read 2 Done Sent Done false
28823 8384 Acked Write 6 Done Sent Done false
28824 8385 Acked Write 1 Done Sent Done false
28825 8386 Acked Read 10 Done Sent Done false
28826 8387 Acked Read 3 Done Sent Done false
28827 8388 Acked Read 1 Done Sent Done false
28828 8389 Acked Write 7 Done Sent Done false
28829 8390 Acked Read 9 Done Sent Done false
28830 8391 Acked Read 4 Done Sent Done false
28831 8392 Acked Read 5 Done Sent Done false
28832 8393 Acked Flush 0 Done Sent Done false
28833 8394 Acked Read 3 Done Sent Done false
28834 8395 Acked Flush 0 Done Sent Done false
28835 8396 Acked Read 6 Done Sent Done false
28836 8397 Acked Flush 0 Done Sent Done false
28837 8398 Acked Read 8 Done Sent Done false
28838 8399 Acked Read 6 Done Sent Done false
28839 8400 Acked Read 9 Done Sent Done false
28840 8401 Acked Write 8 Done Sent Done false
28841 8402 Acked Write 9 Done Sent Done false
28842 8403 Acked Read 1 Done Sent Done false
28843 8404 Acked Write 3 Done Sent Done false
28844 8405 Acked Read 8 Done Sent Done false
28845 8406 Acked Read 8 Done Sent Done false
28846 8407 Acked Read 7 Done Sent Done false
28847 8408 Acked Write 8 Done Sent Done false
28848 8409 Acked Read 9 Done Sent Done false
28849 8410 Acked Read 9 Done Sent Done false
28850 8411 Acked Read 8 Done Sent Done false
28851 8412 Acked Write 8 Done Sent Done false
28852 8413 Acked Read 9 Done Sent Done false
28853 8414 Acked Read 4 Done Sent Done false
28854 8415 Acked Read 8 Done Sent Done false
28855 8416 Acked Write 10 Done Sent Done false
28856 8417 Acked Write 4 Done Sent Done false
28857 8418 Acked Read 8 Done Sent Done false
28858 8419 Acked Write 10 Done Sent Done false
28859 8420 Acked Write 2 Done Sent Done false
28860 8421 Acked Write 3 Done Sent Done false
28861 8422 Acked Read 2 Done Sent Done false
28862 8423 Acked Read 3 Done Sent Done false
28863 8424 Acked Write 3 Done Sent Done false
28864 8425 Acked Write 2 Done Sent Done false
28865 8426 Acked Write 4 Done Sent Done false
28866 8427 Acked Read 1 Done Sent Done false
28867 8428 Acked Read 9 Done Sent Done false
28868 8429 Acked Write 7 Done Sent Done false
28869 8430 Acked Read 5 Done Sent Done false
28870 8431 Acked Read 5 Done Sent Done false
28871 8432 Acked Read 9 Done Sent Done false
28872 8433 Acked Write 1 Done Sent Done false
28873 8434 Acked Read 4 Done Sent Done false
28874 8435 Acked Read 10 Done Sent Done false
28875 8436 Acked Write 4 Done Sent Done false
28876 8437 Acked Read 6 Done Sent Done false
28877 8438 Acked Read 4 Done Sent Done false
28878 8439 Acked Read 1 Done Sent Done false
28879 8440 Acked Read 10 Done Sent Done false
28880 8441 Acked Write 1 Done Sent Done false
28881 8442 Acked Read 1 Done Sent Done false
28882 8443 Acked Read 4 Done Sent Done false
28883 8444 Acked Flush 0 Done Sent Done false
28884 8445 Acked Write 4 Done Sent Done false
28885 8446 Acked Read 5 Done Sent Done false
28886 8447 Acked Write 3 Done Sent Done false
28887 8448 Acked Write 1 Done Sent Done false
28888 8449 Acked Write 10 Done Sent Done false
28889 8450 Acked Write 8 Done Sent Done false
28890 8451 Acked Read 10 Done Sent Done false
28891 8452 Acked Write 7 Done Sent Done false
28892 8453 Acked Read 9 Done Sent Done false
28893 8454 Acked Write 3 Done Sent Done false
28894 8455 Acked Read 6 Done Sent Done false
28895 8456 Acked Write 2 Done Sent Done false
28896 8457 Acked Read 2 Done Sent Done false
28897 8458 Acked Write 2 Done Sent Done false
28898 8459 Acked Flush 0 Done Sent Done false
28899 8460 Acked Write 8 Done Sent Done false
28900 8461 Acked Write 1 Done Sent Done false
28901 8462 Acked Write 1 Done Sent Done false
28902 8463 Acked Write 5 Done Sent Done false
28903 8464 Acked Write 2 Done Sent Done false
28904 8465 Acked Write 9 Done Sent Done false
28905 8466 Acked Read 4 Done Sent Done false
28906 8467 Acked Write 8 Done Sent Done false
28907 8468 Acked Read 9 Done Sent Done false
28908 8469 Acked Flush 0 Done Sent Done false
28909 8470 Acked Write 10 Done Sent Done false
28910 8471 Acked Read 7 Done Sent Done false
28911 8472 Acked Read 9 Done Sent Done false
28912 8473 Acked Read 6 Done Sent Done false
28913 8474 Acked Write 1 Done Sent Done false
28914 8475 Acked Read 7 Done Sent Done false
28915 8476 Acked Read 3 Done Sent Done false
28916 8477 Acked Write 6 Done Sent Done false
28917 8478 Acked Write 7 Done Sent Done false
28918 8479 Acked Read 10 Done Sent Done false
28919 8480 Acked Write 1 Done Sent Done false
28920 8481 Acked Read 7 Done Sent Done false
28921 8482 Acked Read 10 Done Sent Done false
28922 8483 Acked Write 9 Done Sent Done false
28923 8484 Acked Read 1 Done Sent Done false
28924 8485 Acked Flush 0 Done Sent Done false
28925 8486 Acked Read 5 Done Sent Done false
28926 8487 Acked Read 9 Done Sent Done false
28927 8488 Acked Read 3 Done Sent Done false
28928 8489 Acked Read 1 Done Sent Done false
28929 8490 Acked Read 1 Done Sent Done false
28930 8491 Acked Write 5 Done Sent Done false
28931 8492 Acked Read 2 Done Sent Done false
28932 8493 Acked Read 9 Done Sent Done false
28933 8494 Acked Write 1 Done Sent Done false
28934 8495 Acked Read 9 Done Sent Done false
28935 8496 Acked Read 10 Done Sent Done false
28936 8497 Acked Read 10 Done Sent Done false
28937 8498 Acked Read 3 Done Sent Done false
28938 8499 Acked Write 1 Done Sent Done false
28939 8500 Acked Read 10 Done Sent Done false
28940 8501 Acked Write 8 Done Sent Done false
28941 8502 Acked Flush 0 Done Sent Done false
28942 8503 Acked Read 10 Done Sent Done false
28943 8504 Acked Read 6 Done Sent Done false
28944 8505 Acked Flush 0 Done Sent Done false
28945 8506 Acked Read 5 Done Sent Done false
28946 8507 Acked Write 5 Done Sent Done false
28947 8508 Acked Write 3 Done Sent Done false
28948 8509 Acked Read 4 Done Sent Done false
28949 8510 Acked Write 6 Done Sent Done false
28950 8511 Acked Read 1 Done Sent Done false
28951 8512 Acked Read 7 Done Sent Done false
28952 8513 Acked Write 8 Done Sent Done false
28953 8514 Acked Read 5 Done Sent Done false
28954 8515 Acked Read 1 Done Sent Done false
28955 8516 Acked Flush 0 Done Sent Done false
28956 8517 Acked Flush 0 Done Sent Done false
28957 8518 Acked Flush 0 Done Sent Done false
28958 8519 Acked Write 9 Done Sent Done false
28959 8520 Acked Read 1 Done Sent Done false
28960 8521 Acked Read 9 Done Sent Done false
28961 8522 Acked Flush 0 Done Sent Done false
28962 8523 Acked Write 9 Done Sent Done false
28963 8524 Acked Read 2 Done Sent Done false
28964 8525 Acked Read 6 Done Sent Done false
28965 8526 Acked Read 5 Done Sent Done false
28966 8527 Acked Read 2 Done Sent Done false
28967 8528 Acked Read 9 Done Sent Done false
28968 8529 Acked Read 2 Done Sent Done false
28969 8530 Acked Read 5 Done Sent Done false
28970 8531 Acked Write 7 Done Sent Done false
28971 8532 Acked Write 9 Done Sent Done false
28972 8533 Acked Read 5 Done Sent Done false
28973 8534 Acked Read 9 Done Sent Done false
28974 8535 Acked Write 9 Done Sent Done false
28975 8536 Acked Write 8 Done Sent Done false
28976 8537 Acked Flush 0 Done Sent Done false
28977 8538 Acked Read 8 Done Sent Done false
28978 8539 Acked Read 8 Done Sent Done false
28979 8540 Acked Read 3 Done Sent Done false
28980 8541 Acked Write 4 Done Sent Done false
28981 8542 Acked Read 2 Done Sent Done false
28982 8543 Acked Read 7 Done Sent Done false
28983 8544 Acked Read 3 Done Sent Done false
28984 8545 Acked Read 2 Done Sent Done false
28985 STATES DS:0 DS:1 DS:2 TOTAL
28986 Sent 0 301 0 301
28987 Done 301 0 301 602
28988 Skipped 0 0 0 0
28989 Error 0 0 0 0
28990 Last Flush: 8537 8244 8537
28991 Downstairs last five completed: 8244 8243 8242 8241 8240
28992 Upstairs last five completed: 8545 8544 8543 8542 8541
28993 CLIENT: Up:0 ds:301 act:2
28994 [8820] got start action
28995 [8820] I am going to restart
28996 Starting downstairs at port 8820
28997 Make output file at "/tmp/dsc/downstairs-8820.txt"
28998 [8820][1] reports Starting
28999 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1228)
29000 [8820][1] reports Running
29001 ----------------------------------------------------------------
29002 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
29003 States: Active Offline Active
29004 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
29005 8245 Acked Write 6 Done Sent Done false
29006 8246 Acked Write 5 Done Sent Done false
29007 8247 Acked Read 6 Done Sent Done false
29008 8248 Acked Read 5 Done Sent Done false
29009 8249 Acked Read 3 Done Sent Done false
29010 8250 Acked Write 1 Done Sent Done false
29011 8251 Acked Read 3 Done Sent Done false
29012 8252 Acked Read 4 Done Sent Done false
29013 8253 Acked Read 9 Done Sent Done false
29014 8254 Acked Write 10 Done Sent Done false
29015 8255 Acked Write 4 Done Sent Done false
29016 8256 Acked Read 9 Done Sent Done false
29017 8257 Acked Read 4 Done Sent Done false
29018 8258 Acked Read 5 Done Sent Done false
29019 8259 Acked Read 3 Done Sent Done false
29020 8260 Acked Read 8 Done Sent Done false
29021 8261 Acked Read 1 Done Sent Done false
29022 8262 Acked Read 8 Done Sent Done false
29023 8263 Acked Write 3 Done Sent Done false
29024 8264 Acked Read 2 Done Sent Done false
29025 8265 Acked Read 5 Done Sent Done false
29026 8266 Acked Write 2 Done Sent Done false
29027 8267 Acked Write 2 Done Sent Done false
29028 8268 Acked Write 10 Done Sent Done false
29029 8269 Acked Flush 0 Done Sent Done false
29030 8270 Acked Read 5 Done Sent Done false
29031 8271 Acked Flush 0 Done Sent Done false
29032 8272 Acked Read 2 Done Sent Done false
29033 8273 Acked Read 10 Done Sent Done false
29034 8274 Acked Write 6 Done Sent Done false
29035 8275 Acked Read 7 Done Sent Done false
29036 8276 Acked Read 3 Done Sent Done false
29037 8277 Acked Write 8 Done Sent Done false
29038 8278 Acked Write 5 Done Sent Done false
29039 8279 Acked Read 7 Done Sent Done false
29040 8280 Acked Read 2 Done Sent Done false
29041 8281 Acked Write 6 Done Sent Done false
29042 8282 Acked Read 6 Done Sent Done false
29043 8283 Acked Flush 0 Done Sent Done false
29044 8284 Acked Read 10 Done Sent Done false
29045 8285 Acked Read 9 Done Sent Done false
29046 8286 Acked Write 3 Done Sent Done false
29047 8287 Acked Write 3 Done Sent Done false
29048 8288 Acked Read 9 Done Sent Done false
29049 8289 Acked Read 10 Done Sent Done false
29050 8290 Acked Read 6 Done Sent Done false
29051 8291 Acked Write 3 Done Sent Done false
29052 8292 Acked Read 3 Done Sent Done false
29053 8293 Acked Read 1 Done Sent Done false
29054 8294 Acked Write 7 Done Sent Done false
29055 8295 Acked Read 4 Done Sent Done false
29056 8296 Acked Read 4 Done Sent Done false
29057 8297 Acked Flush 0 Done Sent Done false
29058 8298 Acked Read 7 Done Sent Done false
29059 8299 Acked Write 6 Done Sent Done false
29060 8300 Acked Read 8 Done Sent Done false
29061 8301 Acked Read 8 Done Sent Done false
29062 8302 Acked Write 2 Done Sent Done false
29063 8303 Acked Write 2 Done Sent Done false
29064 8304 Acked Write 6 Done Sent Done false
29065 8305 Acked Write 3 Done Sent Done false
29066 8306 Acked Flush 0 Done Sent Done false
29067 8307 Acked Read 5 Done Sent Done false
29068 8308 Acked Write 8 Done Sent Done false
29069 8309 Acked Read 2 Done Sent Done false
29070 8310 Acked Read 3 Done Sent Done false
29071 8311 Acked Read 10 Done Sent Done false
29072 8312 Acked Read 9 Done Sent Done false
29073 8313 Acked Write 7 Done Sent Done false
29074 8314 Acked Write 7 Done Sent Done false
29075 8315 Acked Read 10 Done Sent Done false
29076 8316 Acked Read 6 Done Sent Done false
29077 8317 Acked Read 6 Done Sent Done false
29078 8318 Acked Read 8 Done Sent Done false
29079 8319 Acked Write 4 Done Sent Done false
29080 8320 Acked Write 5 Done Sent Done false
29081 8321 Acked Flush 0 Done Sent Done false
29082 8322 Acked Read 3 Done Sent Done false
29083 8323 Acked Read 3 Done Sent Done false
29084 8324 Acked Read 2 Done Sent Done false
29085 8325 Acked Read 4 Done Sent Done false
29086 8326 Acked Read 4 Done Sent Done false
29087 8327 Acked Read 9 Done Sent Done false
29088 8328 Acked Write 8 Done Sent Done false
29089 8329 Acked Write 7 Done Sent Done false
29090 8330 Acked Flush 0 Done Sent Done false
29091 8331 Acked Flush 0 Done Sent Done false
29092 8332 Acked Read 6 Done Sent Done false
29093 8333 Acked Read 9 Done Sent Done false
29094 8334 Acked Write 8 Done Sent Done false
29095 8335 Acked Write 8 Done Sent Done false
29096 8336 Acked Read 10 Done Sent Done false
29097 8337 Acked Read 9 Done Sent Done false
29098 8338 Acked Read 7 Done Sent Done false
29099 8339 Acked Read 2 Done Sent Done false
29100 8340 Acked Read 10 Done Sent Done false
29101 8341 Acked Write 1 Done Sent Done false
29102 8342 Acked Write 5 Done Sent Done false
29103 8343 Acked Write 10 Done Sent Done false
29104 8344 Acked Read 4 Done Sent Done false
29105 8345 Acked Flush 0 Done Sent Done false
29106 8346 Acked Write 9 Done Sent Done false
29107 8347 Acked Flush 0 Done Sent Done false
29108 8348 Acked Write 1 Done Sent Done false
29109 8349 Acked Read 10 Done Sent Done false
29110 8350 Acked Write 5 Done Sent Done false
29111 8351 Acked Read 9 Done Sent Done false
29112 8352 Acked Write 10 Done Sent Done false
29113 8353 Acked Read 6 Done Sent Done false
29114 8354 Acked Write 10 Done Sent Done false
29115 8355 Acked Write 10 Done Sent Done false
29116 8356 Acked Write 2 Done Sent Done false
29117 8357 Acked Write 10 Done Sent Done false
29118 8358 Acked Read 1 Done Sent Done false
29119 8359 Acked Read 5 Done Sent Done false
29120 8360 Acked Flush 0 Done Sent Done false
29121 8361 Acked Write 1 Done Sent Done false
29122 8362 Acked Read 8 Done Sent Done false
29123 8363 Acked Write 3 Done Sent Done false
29124 8364 Acked Read 1 Done Sent Done false
29125 8365 Acked Write 3 Done Sent Done false
29126 8366 Acked Write 7 Done Sent Done false
29127 8367 Acked Read 9 Done Sent Done false
29128 8368 Acked Write 6 Done Sent Done false
29129 8369 Acked Read 2 Done Sent Done false
29130 8370 Acked Write 1 Done Sent Done false
29131 8371 Acked Flush 0 Done Sent Done false
29132 8372 Acked Write 4 Done Sent Done false
29133 8373 Acked Write 2 Done Sent Done false
29134 8374 Acked Write 3 Done Sent Done false
29135 8375 Acked Read 5 Done Sent Done false
29136 8376 Acked Read 8 Done Sent Done false
29137 8377 Acked Write 6 Done Sent Done false
29138 8378 Acked Read 1 Done Sent Done false
29139 8379 Acked Flush 0 Done Sent Done false
29140 8380 Acked Read 2 Done Sent Done false
29141 8381 Acked Write 10 Done Sent Done false
29142 8382 Acked Read 7 Done Sent Done false
29143 8383 Acked Read 2 Done Sent Done false
29144 8384 Acked Write 6 Done Sent Done false
29145 8385 Acked Write 1 Done Sent Done false
29146 8386 Acked Read 10 Done Sent Done false
29147 8387 Acked Read 3 Done Sent Done false
29148 8388 Acked Read 1 Done Sent Done false
29149 8389 Acked Write 7 Done Sent Done false
29150 8390 Acked Read 9 Done Sent Done false
29151 8391 Acked Read 4 Done Sent Done false
29152 8392 Acked Read 5 Done Sent Done false
29153 8393 Acked Flush 0 Done Sent Done false
29154 8394 Acked Read 3 Done Sent Done false
29155 8395 Acked Flush 0 Done Sent Done false
29156 8396 Acked Read 6 Done Sent Done false
29157 8397 Acked Flush 0 Done Sent Done false
29158 8398 Acked Read 8 Done Sent Done false
29159 8399 Acked Read 6 Done Sent Done false
29160 8400 Acked Read 9 Done Sent Done false
29161 8401 Acked Write 8 Done Sent Done false
29162 8402 Acked Write 9 Done Sent Done false
29163 8403 Acked Read 1 Done Sent Done false
29164 8404 Acked Write 3 Done Sent Done false
29165 8405 Acked Read 8 Done Sent Done false
29166 8406 Acked Read 8 Done Sent Done false
29167 8407 Acked Read 7 Done Sent Done false
29168 8408 Acked Write 8 Done Sent Done false
29169 8409 Acked Read 9 Done Sent Done false
29170 8410 Acked Read 9 Done Sent Done false
29171 8411 Acked Read 8 Done Sent Done false
29172 8412 Acked Write 8 Done Sent Done false
29173 8413 Acked Read 9 Done Sent Done false
29174 8414 Acked Read 4 Done Sent Done false
29175 8415 Acked Read 8 Done Sent Done false
29176 8416 Acked Write 10 Done Sent Done false
29177 8417 Acked Write 4 Done Sent Done false
29178 8418 Acked Read 8 Done Sent Done false
29179 8419 Acked Write 10 Done Sent Done false
29180 8420 Acked Write 2 Done Sent Done false
29181 8421 Acked Write 3 Done Sent Done false
29182 8422 Acked Read 2 Done Sent Done false
29183 8423 Acked Read 3 Done Sent Done false
29184 8424 Acked Write 3 Done Sent Done false
29185 8425 Acked Write 2 Done Sent Done false
29186 8426 Acked Write 4 Done Sent Done false
29187 8427 Acked Read 1 Done Sent Done false
29188 8428 Acked Read 9 Done Sent Done false
29189 8429 Acked Write 7 Done Sent Done false
29190 8430 Acked Read 5 Done Sent Done false
29191 8431 Acked Read 5 Done Sent Done false
29192 8432 Acked Read 9 Done Sent Done false
29193 8433 Acked Write 1 Done Sent Done false
29194 8434 Acked Read 4 Done Sent Done false
29195 8435 Acked Read 10 Done Sent Done false
29196 8436 Acked Write 4 Done Sent Done false
29197 8437 Acked Read 6 Done Sent Done false
29198 8438 Acked Read 4 Done Sent Done false
29199 8439 Acked Read 1 Done Sent Done false
29200 8440 Acked Read 10 Done Sent Done false
29201 8441 Acked Write 1 Done Sent Done false
29202 8442 Acked Read 1 Done Sent Done false
29203 8443 Acked Read 4 Done Sent Done false
29204 8444 Acked Flush 0 Done Sent Done false
29205 8445 Acked Write 4 Done Sent Done false
29206 8446 Acked Read 5 Done Sent Done false
29207 8447 Acked Write 3 Done Sent Done false
29208 8448 Acked Write 1 Done Sent Done false
29209 8449 Acked Write 10 Done Sent Done false
29210 8450 Acked Write 8 Done Sent Done false
29211 8451 Acked Read 10 Done Sent Done false
29212 8452 Acked Write 7 Done Sent Done false
29213 8453 Acked Read 9 Done Sent Done false
29214 8454 Acked Write 3 Done Sent Done false
29215 8455 Acked Read 6 Done Sent Done false
29216 8456 Acked Write 2 Done Sent Done false
29217 8457 Acked Read 2 Done Sent Done false
29218 8458 Acked Write 2 Done Sent Done false
29219 8459 Acked Flush 0 Done Sent Done false
29220 8460 Acked Write 8 Done Sent Done false
29221 8461 Acked Write 1 Done Sent Done false
29222 8462 Acked Write 1 Done Sent Done false
29223 8463 Acked Write 5 Done Sent Done false
29224 8464 Acked Write 2 Done Sent Done false
29225 8465 Acked Write 9 Done Sent Done false
29226 8466 Acked Read 4 Done Sent Done false
29227 8467 Acked Write 8 Done Sent Done false
29228 8468 Acked Read 9 Done Sent Done false
29229 8469 Acked Flush 0 Done Sent Done false
29230 8470 Acked Write 10 Done Sent Done false
29231 8471 Acked Read 7 Done Sent Done false
29232 8472 Acked Read 9 Done Sent Done false
29233 8473 Acked Read 6 Done Sent Done false
29234 8474 Acked Write 1 Done Sent Done false
29235 8475 Acked Read 7 Done Sent Done false
29236 8476 Acked Read 3 Done Sent Done false
29237 8477 Acked Write 6 Done Sent Done false
29238 8478 Acked Write 7 Done Sent Done false
29239 8479 Acked Read 10 Done Sent Done false
29240 8480 Acked Write 1 Done Sent Done false
29241 8481 Acked Read 7 Done Sent Done false
29242 8482 Acked Read 10 Done Sent Done false
29243 8483 Acked Write 9 Done Sent Done false
29244 8484 Acked Read 1 Done Sent Done false
29245 8485 Acked Flush 0 Done Sent Done false
29246 8486 Acked Read 5 Done Sent Done false
29247 8487 Acked Read 9 Done Sent Done false
29248 8488 Acked Read 3 Done Sent Done false
29249 8489 Acked Read 1 Done Sent Done false
29250 8490 Acked Read 1 Done Sent Done false
29251 8491 Acked Write 5 Done Sent Done false
29252 8492 Acked Read 2 Done Sent Done false
29253 8493 Acked Read 9 Done Sent Done false
29254 8494 Acked Write 1 Done Sent Done false
29255 8495 Acked Read 9 Done Sent Done false
29256 8496 Acked Read 10 Done Sent Done false
29257 8497 Acked Read 10 Done Sent Done false
29258 8498 Acked Read 3 Done Sent Done false
29259 8499 Acked Write 1 Done Sent Done false
29260 8500 Acked Read 10 Done Sent Done false
29261 8501 Acked Write 8 Done Sent Done false
29262 8502 Acked Flush 0 Done Sent Done false
29263 8503 Acked Read 10 Done Sent Done false
29264 8504 Acked Read 6 Done Sent Done false
29265 8505 Acked Flush 0 Done Sent Done false
29266 8506 Acked Read 5 Done Sent Done false
29267 8507 Acked Write 5 Done Sent Done false
29268 8508 Acked Write 3 Done Sent Done false
29269 8509 Acked Read 4 Done Sent Done false
29270 8510 Acked Write 6 Done Sent Done false
29271 8511 Acked Read 1 Done Sent Done false
29272 8512 Acked Read 7 Done Sent Done false
29273 8513 Acked Write 8 Done Sent Done false
29274 8514 Acked Read 5 Done Sent Done false
29275 8515 Acked Read 1 Done Sent Done false
29276 8516 Acked Flush 0 Done Sent Done false
29277 8517 Acked Flush 0 Done Sent Done false
29278 8518 Acked Flush 0 Done Sent Done false
29279 8519 Acked Write 9 Done Sent Done false
29280 8520 Acked Read 1 Done Sent Done false
29281 8521 Acked Read 9 Done Sent Done false
29282 8522 Acked Flush 0 Done Sent Done false
29283 8523 Acked Write 9 Done Sent Done false
29284 8524 Acked Read 2 Done Sent Done false
29285 8525 Acked Read 6 Done Sent Done false
29286 8526 Acked Read 5 Done Sent Done false
29287 8527 Acked Read 2 Done Sent Done false
29288 8528 Acked Read 9 Done Sent Done false
29289 8529 Acked Read 2 Done Sent Done false
29290 8530 Acked Read 5 Done Sent Done false
29291 8531 Acked Write 7 Done Sent Done false
29292 8532 Acked Write 9 Done Sent Done false
29293 8533 Acked Read 5 Done Sent Done false
29294 8534 Acked Read 9 Done Sent Done false
29295 8535 Acked Write 9 Done Sent Done false
29296 8536 Acked Write 8 Done Sent Done false
29297 8537 Acked Flush 0 Done Sent Done false
29298 8538 Acked Read 8 Done Sent Done false
29299 8539 Acked Read 8 Done Sent Done false
29300 8540 Acked Read 3 Done Sent Done false
29301 8541 Acked Write 4 Done Sent Done false
29302 8542 Acked Read 2 Done Sent Done false
29303 8543 Acked Read 7 Done Sent Done false
29304 8544 Acked Read 3 Done Sent Done false
29305 8545 Acked Read 2 Done Sent Done false
29306 8546 Acked Flush 0 Done Sent Done false
29307 STATES DS:0 DS:1 DS:2 TOTAL
29308 Sent 0 302 0 302
29309 Done 302 0 302 604
29310 Skipped 0 0 0 0
29311 Error 0 0 0 0
29312 Last Flush: 8546 8244 8546
29313 Downstairs last five completed: 8244 8243 8242 8241 8240
29314 Upstairs last five completed: 8546 8545 8544 8543 8542
29315 CLIENT: Up:0 ds:302 act:2
293162024-11-26T22:10:40.277ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
293172024-11-26T22:10:40.277ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:41825) = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
293182024-11-26T22:10:40.278ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
293192024-11-26T22:10:40.278ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
293202024-11-26T22:10:40.278ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
293212024-11-26T22:10:40.278ZINFOcrucible: send last flush ID to this DS: 8244 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
293222024-11-26T22:10:40.279ZINFOcrucible: Replied this last flush ID: 8244 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
293232024-11-26T22:10:40.279ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
293242024-11-26T22:10:40.279ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
293252024-11-26T22:10:40.279ZINFOcrucible: [1] client re-new 302 jobs since flush 8244 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
293262024-11-26T22:10:40.279ZINFOcrucible: [1] Replayed 302 jobs since flush: 8244 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
29327 ----------------------------------------------------------------
29328 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
29329 Downstairs last five completed: 8546 8545 8544 8543 8542
29330 Upstairs last five completed: 8546 8545 8544 8543 8542
29331 CLIENT: Up:0 ds:0 act:3
29332 Replay: All jobs finished
29333 Main task has work to do, go find it
29334 got dsc Stop(1)
29335 stop 1
29336 Nov 26 22:10:42.528 INFO request completed, latency_us: 330, response_code: 204, uri: /stop/cid/1, method: POST, req_id: efdc75e0-641a-46cf-828d-f09573b93373, remote_addr: 127.0.0.1:36829, local_addr: 127.0.[8820] Got stop action so:false kr:false
29337 0.1:9998
29338 Nov 26 22:10:42.528 INFO request completed, latency_us: 120, response_code: 200, uri: /state/cid/1, method: GET, req_id: 26fdb5f4-cb5c-4e2a-9155-53bad9a4ec60, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
293392024-11-26T22:10:42.528ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
293402024-11-26T22:10:42.528ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
293412024-11-26T22:10:42.528ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
293422024-11-26T22:10:42.528ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
293432024-11-26T22:10:42.528ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
293442024-11-26T22:10:42.529ZINFOcrucible: client task is exiting = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
29345 [8820] Exited with: ExitStatus(unix_wait_status(9))
29346 [8820][1] reports Exit
29347 Nov 26 22:10:46.530 INFO request completed, latency_us: 180, response_code: 200, uri: /state/cid/1, method: GET, req_id: b2399001-e4cb-4986-8c59-911e990666d3, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
29348 001/300 Read block 1198 len 28672
29349 002/300 Flush
29350 003/300 Read block 1513 len 40960
29351 004/300 Flush
29352 005/300 Write block 4962 len 40960 data: 6 6 7 9 9 6 7 8 7 6
29353 006/300 Write block 724 len 24576 data: 6 5 6 5 6 5
29354 007/300 Write block 4320 len 4096 data: 4
29355 008/300 Read block 261 len 8192
29356 009/300 Read block 2806 len 20480
29357 010/300 Read block 3042 len 16384
29358 011/300 Read block 668 len 16384
29359 012/300 Read block 4019 len 40960
29360 013/300 Read block 4837 len 40960
29361 014/300 Read block 2131 len 12288
29362 015/300 Read block 143 len 8192
29363 016/300 Write block 4243 len 28672 data: 5 5 3 5 5 7 7
29364 017/300 Write block 4911 len 24576 data: 2 2 3 3 3 4
29365 018/300 Flush
29366 019/300 Read block 2700 len 20480
29367 020/300 Write block 4641 len 24576 data: 8 7 5 7 4 4
29368 021/300 Read block 169 len 16384
29369 022/300 Read block 3140 len 36864
29370 023/300 Write block 410 len 28672 data: 5 5 5 4 4 2 4
29371 024/300 Read block 226 len 20480
29372 025/300 Read block 253 len 20480
29373 026/300 Read block 1165 len 24576
29374 027/300 Write block 2287 len 36864 data: 5 5 4 3 3 3 2 2 2
29375 028/300 Write block 3189 len 12288 data: 2 3 5
29376 029/300 Write block 4145 len 24576 data: 6 6 7 6 6 5
29377 030/300 Flush
29378 031/300 Write block 1162 len 16384 data: 4 5 5 5
29379 032/300 Read block 304 len 36864
29380 033/300 Read block 4667 len 36864
29381 034/300 Read block 2947 len 28672
29382 035/300 Write block 3862 len 36864 data: 6 4 3 3 4 4 4 3 4
29383 036/300 Read block 3732 len 24576
29384 037/300 Write block 1137 len 28672 data: 5 6 7 8 8 5 6
29385 038/300 Write block 4934 len 12288 data: 6 5 6
29386 039/300 Read block 4636 len 16384
29387 040/300 Write block 896 len 32768 data: 4 4 4 4 3 3 3 4
29388 041/300 Flush
29389 042/300 Read block 3249 len 8192
29390 043/300 Read block 705 len 8192
29391 044/300 Read block 137 len 20480
29392 045/300 Read block 2880 len 8192
29393 046/300 Write block 1795 len 8192 data: 3 4
29394 047/300 Read block 4087 len 4096
29395 048/300 Write block 4 len 32768 data: 7 7 10 9 10 8 6 5
29396 049/300 Read block 4201 len 12288
29397 050/300 Write block 2448 len 24576 data: 6 6 5 6 6 4
29398 051/300 Write block 1736 len 28672 data: 7 6 5 2 2 2 2
29399 052/300 Read block 4191 len 8192
29400 053/300 Write block 3571 len 24576 data: 8 8 8 7 6 3
29401 054/300 Write block 1388 len 4096 data: 4
29402 055/300 Write block 2600 len 36864 data: 8 7 6 8 7 6 7 8 7
29403 056/300 Read block 4102 len 36864
29404 057/300 Read block 1345 len 8192
29405 058/300 Write block 940 len 28672 data: 7 6 6 5 6 7 6
29406 059/300 Read block 1434 len 16384
29407 060/300 Flush
29408 061/300 Read block 268 len 40960
29409 062/300 Write block 1440 len 36864 data: 3 3 5 4 5 5 7 5 5
29410 063/300 Write block 1980 len 32768 data: 3 5 6 6 6 6 8 7
29411 064/300 Write block 3495 len 40960 data: 9 9 7 7 6 6 6 7 6 5
29412 065/300 Read block 3625 len 36864
29413 066/300 Write block 3923 len 12288 data: 4 4 4
29414 067/300 Write block 4031 len 36864 data: 6 6 5 4 3 3 6 5 5
29415 068/300 Write block 256 len 4096 data: 4
29416 069/300 Read block 1518 len 40960
29417 070/300 Write block 4166 len 12288 data: 5 5 6
29418 071/300 Flush
29419 072/300 Read block 1887 len 8192
29420 073/300 Read block 1001 len 32768
29421 074/300 Flush
29422 075/300 Flush
29423 076/300 Write block 1746 len 4096 data: 4
29424 077/300 Read block 1175 len 40960
29425 078/300 Read block 3680 len 12288
29426 079/300 Write block 4603 len 24576 data: 8 8 8 5 4 5
29427 080/300 Read block 3197 len 32768
29428 081/300 Read block 4697 len 8192
29429 082/300 Write block 3432 len 36864 data: 4 3 3 4 4 4 6 6 6
29430 083/300 Read block 4056 len 40960
29431 084/300 Write block 704 len 28672 data: 5 2 2 4 5 6 6
29432 085/300 Read block 1464 len 24576
29433 086/300 Write block 157 len 8192 data: 4 5
29434 087/300 Read block 1298 len 32768
29435 088/300 Read block 4290 len 12288
29436 089/300 Read block 4728 len 12288
29437 090/300 Flush
29438 091/300 Write block 4150 len 12288 data: 6 5 6
29439 092/300 Write block 630 len 16384 data: 5 6 5 4
29440 093/300 Read block 2113 len 36864
29441 094/300 Write block 2591 len 40960 data: 6 8 9 8 8 8 8 9 9 9
29442 095/300 Write block 4121 len 16384 data: 2 2 3 3
29443 096/300 Read block 3572 len 32768
29444 097/300 Read block 535 len 4096
29445 098/300 Read block 1950 len 8192
29446 099/300 Write block 328 len 24576 data: 5 5 5 5 4 5
29447 100/300 Read block 486 len 12288
29448 101/300 Write block 3402 len 16384 data: 6 6 9 9
29449 102/300 Read block 3987 len 20480
29450 103/300 Read block 4336 len 8192
29451 104/300 Read block 43 len 40960
29452 105/300 Read block 2662 len 20480
29453 106/300 Read block 1415 len 12288
29454 107/300 Flush
29455 108/300 Write block 379 len 36864 data: 5 3 3 4 4 4 3 3 3
29456 109/300 Read block 4578 len 8192
29457 110/300 Write block 2085 len 32768 data: 6 4 6 5 5 6 5 5
29458 111/300 Read block 2389 len 4096
29459 112/300 Read block 3698 len 16384
29460 113/300 Read block 756 len 12288
29461 114/300 Read block 106 len 12288
29462 115/300 Read block 2980 len 8192
29463 116/300 Write block 4795 len 16384 data: 3 2 3 2
29464 117/300 Write block 3931 len 4096 data: 8
29465 118/300 Write block 305 len 36864 data: 3 3 2 2 2 3 4 4 3
29466 119/300 Read block 3689 len 40960
29467 120/300 Write block 807 len 4096 data: 4
29468 121/300 Flush
29469 122/300 Write block 4551 len 28672 data: 4 4 4 4 3 4 4
29470 123/300 Read block 4978 len 28672
29471 124/300 Read block 2593 len 40960
29472 125/300 Write block 2904 len 28672 data: 7 7 6 6 6 4 3
29473 126/300 Read block 1592 len 8192
29474 127/300 Write block 1769 len 12288 data: 3 3 5
29475 128/300 Write block 1491 len 20480 data: 5 4 4 4 4
29476 129/300 Read block 4757 len 36864
29477 130/300 Read block 3535 len 12288
29478 131/300 Read block 4658 len 24576
29479 132/300 Read block 2582 len 36864
29480 133/300 Read block 1728 len 12288
29481 134/300 Read block 827 len 8192
29482 135/300 Read block 2209 len 8192
29483 136/300 Read block 4052 len 24576
29484 137/300 Write block 3010 len 4096 data: 9
29485 138/300 Read block 190 len 32768
29486 139/300 Read block 1810 len 28672
29487 140/300 Read block 2510 len 16384
29488 141/300 Read block 3075 len 16384
29489 142/300 Flush
29490 143/300 Write block 3211 len 12288 data: 2 2 3
29491 144/300 Read block 3000 len 40960
29492 145/300 Write block 2886 len 24576 data: 7 10 10 8 7 5
29493 146/300 Read block 595 len 36864
29494 147/300 Read block 4255 len 40960
29495 148/300 Write block 2844 len 8192 data: 7 7
29496 149/300 Write block 4034 len 24576 data: 5 4 4 7 6 6
29497 150/300 Write block 1590 len 40960 data: 5 5 6 6 7 5 5 4 6 8
29498 151/300 Write block 2269 len 28672 data: 6 5 5 6 5 5 3
29499 152/300 Read block 3320 len 20480
29500 153/300 Write block 136 len 32768 data: 5 7 7 7 8 8 7 8
29501 154/300 Read block 3410 len 40960
29502 155/300 Write block 2097 len 32768 data: 4 3 3 4 5 7 7 7
29503 156/300 Read block 3874 len 40960
29504 157/300 Write block 1676 len 28672 data: 6 5 4 6 5 4 6
29505 158/300 Write block 4750 len 4096 data: 7
29506 159/300 Write block 2284 len 32768 data: 3 4 5 6 6 5 4 4
29507 160/300 Read block 1748 len 36864
29508 161/300 Read block 823 len 4096
29509 162/300 Write block 22 len 20480 data: 5 4 4 4 4
29510 163/300 Write block 1512 len 16384 data: 3 3 3 3
29511 164/300 Read block 4171 len 8192
29512 165/300 Write block 1116 len 36864 data: 3 4 4 4 4 4 5 5 4
29513 166/300 Read block 3614 len 12288
29514 167/300 Read block 425 len 12288
29515 168/300 Write block 4834 len 24576 data: 6 5 5 4 4 4
29516 169/300 Write block 1902 len 8192 data: 4 5
29517 170/300 Write block 3946 len 4096 data: 5
29518 171/300 Read block 4940 len 12288
29519 172/300 Write block 3002 len 4096 data: 4
29520 173/300 Read block 1238 len 16384
29521 174/300 Read block 4906 len 12288
29522 175/300 Write block 2874 len 8192 data: 4 7
29523 176/300 Write block 2724 len 36864 data: 5 4 3 5 6 6 7 8 7
29524 177/300 Read block 163 len 24576
29525 178/300 Write block 4884 len 24576 data: 3 4 4 4 3 2
29526 179/300 Read block 3395 len 36864
29527 180/300 Read block 1591 len 4096
29528 181/300 Write block 491 len 20480 data: 4 3 4 4 4
29529 182/300 Write block 2960 len 28672 data: 5 5 6 5 5 4 4
29530 183/300 Flush
29531 184/300 Write block 1918 len 20480 data: 5 7 6 7 6
29532 185/300 Write block 2122 len 16384 data: 4 4 3 3
29533 186/300 Read block 1391 len 36864
29534 187/300 Read block 3732 len 8192
29535 188/300 Read block 1139 len 24576
29536 189/300 Read block 1496 len 32768
29537 190/300 Read block 848 len 32768
29538 191/300 Read block 4258 len 28672
29539 192/300 Write block 898 len 4096 data: 5
29540 193/300 Read block 1684 len 32768
29541 194/300 Read block 1094 len 36864
29542 195/300 Write block 4863 len 32768 data: 9 8 5 3 4 4 5 6
29543 196/300 Read block 3019 len 4096
29544 197/300 Read block 1460 len 8192
29545 198/300 Write block 822 len 4096 data: 4
29546 199/300 Read block 3484 len 8192
29547 200/300 Read block 138 len 36864
29548 201/300 Write block 3362 len 20480 data: 4 5 4 4 4
29549 202/300 Read block 932 len 32768
29550 203/300 Flush
29551 204/300 Write block 2575 len 40960 data: 3 3 4 4 4 4 6 5 5 5
29552 205/300 Read block 3938 len 36864
29553 206/300 Read block 1247 len 32768
29554 207/300 Read block 3094 len 24576
29555 208/300 Read block 3797 len 32768
29556 209/300 Read block 3612 len 36864
29557 210/300 Read block 988 len 28672
29558 211/300 Read block 3736 len 36864
29559 212/300 Read block 1867 len 12288
29560 213/300 Write block 967 len 20480 data: 6 6 5 5 5
29561 214/300 Read block 384 len 32768
29562 215/300 Flush
29563 216/300 Read block 2340 len 32768
29564 217/300 Read block 616 len 28672
29565 218/300 Write block 498 len 36864 data: 6 6 7 5 6 6 6 7 6
29566 219/300 Write block 3583 len 40960 data: 5 6 7 6 6 4 4 5 6 6
29567 220/300 Flush
29568 221/300 Read block 3304 len 28672
29569 222/300 Read block 2359 len 24576
29570 223/300 Write block 754 len 16384 data: 5 6 4 5
29571 224/300 Read block 1389 len 36864
29572 225/300 Read block 1585 len 24576
29573 226/300 Read block 734 len 36864
29574 227/300 Read block 3438 len 32768
29575 228/300 Write block 3423 len 24576 data: 3 4 4 4 4 4
29576 229/300 Read block 4434 len 16384
29577 230/300 Read block 1494 len 20480
29578 231/300 Write block 2189 len 32768 data: 4 3 4 4 4 4 4 4
29579 232/300 Read block 2765 len 36864
29580 233/300 Read block 4475 len 40960
29581 234/300 Read block 551 len 36864
29582 235/300 Read block 2767 len 12288
29583 236/300 Read block 4769 len 36864
29584 237/300 Write block 2280 len 8192 data: 3 3
29585 238/300 Write block 3465 len 32768 data: 3 3 2 2 3 4 4 4
29586 239/300 Read block 1076 len 8192
29587 240/300 Write block 1395 len 32768 data: 6 6 6 5 4 3 2 3
29588 241/300 Write block 212 len 16384 data: 3 4 5 6
29589 242/300 Write block 3358 len 4096 data: 5
29590 243/300 Write block 4438 len 36864 data: 7 7 6 7 8 9 9 7 7
29591 244/300 Read block 4144 len 32768
29592 245/300 Read block 3304 len 20480
29593 246/300 Write block 4503 len 4096 data: 3
29594 247/300 Read block 1390 len 8192
29595 248/300 Write block 2354 len 8192 data: 8 6
29596 249/300 Read block 418 len 40960
29597 250/300 Write block 579 len 16384 data: 5 6 5 5
29598 251/300 Write block 477 len 12288 data: 4 3 5
29599 252/300 Write block 3831 len 12288 data: 7 5 4
29600 253/300 Read block 211 len 28672
29601 254/300 Read block 4041 len 40960
29602 255/300 Read block 1893 len 40960
29603 256/300 Write block 4758 len 16384 data: 5 5 6 5
29604 257/300 Write block 1591 len 8192 data: 6 7
29605 258/300 Write block 3449 len 28672 data: 7 7 6 6 6 6 7
29606 259/300 Write block 2555 len 40960 data: 3 5 6 8 8 7 7 7 6 6
29607 260/300 Write block 26 len 4096 data: 5
29608 261/300 Read block 2327 len 20480
29609 262/300 Read block 3959 len 32768
29610 263/300 Read block 3980 len 4096
29611 264/300 Flush
29612 265/300 Read block 2816 len 32768
29613 266/300 Write block 3134 len 8192 data: 6 7
29614 267/300 Write block 4380 len 24576 data: 3 3 3 3 3 5
29615 268/300 Flush
29616 269/300 Flush
29617 270/300 Read block 1004 len 32768
29618 271/300 Read block 445 len 16384
29619 272/300 Flush
29620 273/300 Write block 4455 len 8192 data: 5 5
29621 274/300 Write block 2994 len 4096 data: 3
29622 275/300 Read block 297 len 24576
29623 276/300 Write block 2573 len 16384 data: 3 3 4 4
29624 277/300 Read block 307 len 16384
29625 278/300 Write block 129 len 20480 data: 7 8 5 5 5
29626 279/300 Read block 4463 len 16384
29627 280/300 Read block 1415 len 24576
29628 281/300 Write block 2557 len 28672 data: 7 9 9 8 8 8 7
29629 282/300 Write block 4615 len 20480 data: 5 6 6 7 8
29630 283/300 Flush
29631 284/300 Write block 4923 len 12288 data: 6 7 9
29632 285/300 Write block 844 len 28672 data: 3 3 3 5 5 5 5
29633 286/300 Write block 1154 len 28672 data: 6 6 6 5 5 6 6
29634 287/300 Read block 1082 len 20480
29635 288/300 Write block 3170 len 28672 data: 8 8 6 5 4 4 5
29636 289/300 Flush
29637 290/300 Read block 1936 len 40960
29638 291/300 Write block 4653 len 32768 data: 6 5 5 4 5 7 9 9
29639 292/300 Read block 3895 len 16384
29640 293/300 Write block 3516 len 8192 data: 2 3
29641 294/300 Read block 3383 len 8192
29642 295/300 Read block 4305 len 36864
29643 296/300 Write block 3875 len 16384 data: 8 6 6 6
29644 297/300 Read block 4664 len 16384
29645 298/300 Read block 4123 len 36864
29646 299/300 Read block 3818 len 8192
29647 300/300 Write block 1524 len 16384 data: 8 8 7 6
29648 301/300 Read block 2207 len 8192
29649 Main task has work to do, go find it
29650 got dsc Start(1)
29651 start 1
29652 Nov 26 22:10:46.772 INFO request completed, latency_us: 197, response_code: 204, uri: /start/cid/1, method: POST, req_id: f72e62a1-6775-4680-a0e0-475493f977e0, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
29653 Replay: started 1, returned:Ok(())
29654 ----------------------------------------------------------------
29655 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
29656 States: Active Offline Active
29657 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
29658 8547 Acked Read 7 Done Sent Done false
29659 8548 Acked Flush 0 Done Sent Done false
29660 8549 Acked Read 10 Done Sent Done false
29661 8550 Acked Flush 0 Done Sent Done false
29662 8551 Acked Write 10 Done Sent Done false
29663 8552 Acked Write 6 Done Sent Done false
29664 8553 Acked Write 1 Done Sent Done false
29665 8554 Acked Read 2 Done Sent Done false
29666 8555 Acked Read 5 Done Sent Done false
29667 8556 Acked Read 4 Done Sent Done false
29668 8557 Acked Read 4 Done Sent Done false
29669 8558 Acked Read 10 Done Sent Done false
29670 8559 Acked Read 10 Done Sent Done false
29671 8560 Acked Read 3 Done Sent Done false
29672 8561 Acked Read 2 Done Sent Done false
29673 8562 Acked Write 7 Done Sent Done false
29674 8563 Acked Write 6 Done Sent Done false
29675 8564 Acked Flush 0 Done Sent Done false
29676 8565 Acked Read 5 Done Sent Done false
29677 8566 Acked Write 6 Done Sent Done false
29678 8567 Acked Read 4 Done Sent Done false
29679 8568 Acked Read 9 Done Sent Done false
29680 8569 Acked Write 7 Done Sent Done false
29681 8570 Acked Read 5 Done Sent Done false
29682 8571 Acked Read 5 Done Sent Done false
29683 8572 Acked Read 6 Done Sent Done false
29684 8573 Acked Write 9 Done Sent Done false
29685 8574 Acked Write 3 Done Sent Done false
29686 8575 Acked Write 6 Done Sent Done false
29687 8576 Acked Flush 0 Done Sent Done false
29688 8577 Acked Write 4 Done Sent Done false
29689 8578 Acked Read 9 Done Sent Done false
29690 8579 Acked Read 9 Done Sent Done false
29691 8580 Acked Read 7 Done Sent Done false
29692 8581 Acked Write 9 Done Sent Done false
29693 8582 Acked Read 6 Done Sent Done false
29694 8583 Acked Write 7 Done Sent Done false
29695 8584 Acked Write 3 Done Sent Done false
29696 8585 Acked Read 4 Done Sent Done false
29697 8586 Acked Write 8 Done Sent Done false
29698 8587 Acked Flush 0 Done Sent Done false
29699 8588 Acked Read 2 Done Sent Done false
29700 8589 Acked Read 2 Done Sent Done false
29701 8590 Acked Read 5 Done Sent Done false
29702 8591 Acked Read 2 Done Sent Done false
29703 8592 Acked Write 2 Done Sent Done false
29704 8593 Acked Read 1 Done Sent Done false
29705 8594 Acked Write 8 Done Sent Done false
29706 8595 Acked Read 3 Done Sent Done false
29707 8596 Acked Write 6 Done Sent Done false
29708 8597 Acked Write 7 Done Sent Done false
29709 8598 Acked Read 2 Done Sent Done false
29710 8599 Acked Write 6 Done Sent Done false
29711 8600 Acked Write 1 Done Sent Done false
29712 8601 Acked Write 9 Done Sent Done false
29713 8602 Acked Read 9 Done Sent Done false
29714 8603 Acked Read 2 Done Sent Done false
29715 8604 Acked Write 7 Done Sent Done false
29716 8605 Acked Read 4 Done Sent Done false
29717 8606 Acked Flush 0 Done Sent Done false
29718 8607 Acked Read 10 Done Sent Done false
29719 8608 Acked Write 9 Done Sent Done false
29720 8609 Acked Write 8 Done Sent Done false
29721 8610 Acked Write 10 Done Sent Done false
29722 8611 Acked Read 9 Done Sent Done false
29723 8612 Acked Write 3 Done Sent Done false
29724 8613 Acked Write 9 Done Sent Done false
29725 8614 Acked Write 1 Done Sent Done false
29726 8615 Acked Read 10 Done Sent Done false
29727 8616 Acked Write 3 Done Sent Done false
29728 8617 Acked Flush 0 Done Sent Done false
29729 8618 Acked Read 2 Done Sent Done false
29730 8619 Acked Read 8 Done Sent Done false
29731 8620 Acked Flush 0 Done Sent Done false
29732 8621 Acked Flush 0 Done Sent Done false
29733 8622 Acked Write 1 Done Sent Done false
29734 8623 Acked Read 10 Done Sent Done false
29735 8624 Acked Read 3 Done Sent Done false
29736 8625 Acked Write 6 Done Sent Done false
29737 8626 Acked Read 8 Done Sent Done false
29738 8627 Acked Read 2 Done Sent Done false
29739 8628 Acked Write 9 Done Sent Done false
29740 8629 Acked Read 10 Done Sent Done false
29741 8630 Acked Write 7 Done Sent Done false
29742 8631 Acked Read 6 Done Sent Done false
29743 8632 Acked Write 2 Done Sent Done false
29744 8633 Acked Read 8 Done Sent Done false
29745 8634 Acked Read 3 Done Sent Done false
29746 8635 Acked Read 3 Done Sent Done false
29747 8636 Acked Flush 0 Done Sent Done false
29748 8637 Acked Write 3 Done Sent Done false
29749 8638 Acked Write 4 Done Sent Done false
29750 8639 Acked Read 9 Done Sent Done false
29751 8640 Acked Write 10 Done Sent Done false
29752 8641 Acked Write 4 Done Sent Done false
29753 8642 Acked Read 8 Done Sent Done false
29754 8643 Acked Read 1 Done Sent Done false
29755 8644 Acked Read 2 Done Sent Done false
29756 8645 Acked Write 6 Done Sent Done false
29757 8646 Acked Read 3 Done Sent Done false
29758 8647 Acked Write 4 Done Sent Done false
29759 8648 Acked Read 5 Done Sent Done false
29760 8649 Acked Read 2 Done Sent Done false
29761 8650 Acked Read 10 Done Sent Done false
29762 8651 Acked Read 5 Done Sent Done false
29763 8652 Acked Read 3 Done Sent Done false
29764 8653 Acked Flush 0 Done Sent Done false
29765 8654 Acked Write 9 Done Sent Done false
29766 8655 Acked Read 2 Done Sent Done false
29767 8656 Acked Write 8 Done Sent Done false
29768 8657 Acked Read 1 Done Sent Done false
29769 8658 Acked Read 4 Done Sent Done false
29770 8659 Acked Read 3 Done Sent Done false
29771 8660 Acked Read 3 Done Sent Done false
29772 8661 Acked Read 2 Done Sent Done false
29773 8662 Acked Write 4 Done Sent Done false
29774 8663 Acked Write 1 Done Sent Done false
29775 8664 Acked Write 9 Done Sent Done false
29776 8665 Acked Read 10 Done Sent Done false
29777 8666 Acked Write 1 Done Sent Done false
29778 8667 Acked Flush 0 Done Sent Done false
29779 8668 Acked Write 7 Done Sent Done false
29780 8669 Acked Read 7 Done Sent Done false
29781 8670 Acked Read 10 Done Sent Done false
29782 8671 Acked Write 7 Done Sent Done false
29783 8672 Acked Read 2 Done Sent Done false
29784 8673 Acked Write 3 Done Sent Done false
29785 8674 Acked Write 5 Done Sent Done false
29786 8675 Acked Read 9 Done Sent Done false
29787 8676 Acked Read 3 Done Sent Done false
29788 8677 Acked Read 6 Done Sent Done false
29789 8678 Acked Read 9 Done Sent Done false
29790 8679 Acked Read 3 Done Sent Done false
29791 8680 Acked Read 2 Done Sent Done false
29792 8681 Acked Read 2 Done Sent Done false
29793 8682 Acked Read 6 Done Sent Done false
29794 8683 Acked Write 1 Done Sent Done false
29795 8684 Acked Read 8 Done Sent Done false
29796 8685 Acked Read 7 Done Sent Done false
29797 8686 Acked Read 4 Done Sent Done false
29798 8687 Acked Read 4 Done Sent Done false
29799 8688 Acked Flush 0 Done Sent Done false
29800 8689 Acked Write 3 Done Sent Done false
29801 8690 Acked Read 10 Done Sent Done false
29802 8691 Acked Write 6 Done Sent Done false
29803 8692 Acked Read 9 Done Sent Done false
29804 8693 Acked Read 10 Done Sent Done false
29805 8694 Acked Write 2 Done Sent Done false
29806 8695 Acked Write 6 Done Sent Done false
29807 8696 Acked Write 10 Done Sent Done false
29808 8697 Acked Write 7 Done Sent Done false
29809 8698 Acked Read 5 Done Sent Done false
29810 8699 Acked Write 8 Done Sent Done false
29811 8700 Acked Read 10 Done Sent Done false
29812 8701 Acked Write 8 Done Sent Done false
29813 8702 Acked Read 10 Done Sent Done false
29814 8703 Acked Write 7 Done Sent Done false
29815 8704 Acked Write 1 Done Sent Done false
29816 8705 Acked Write 8 Done Sent Done false
29817 8706 Acked Read 9 Done Sent Done false
29818 8707 Acked Read 1 Done Sent Done false
29819 8708 Acked Write 5 Done Sent Done false
29820 8709 Acked Write 4 Done Sent Done false
29821 8710 Acked Read 2 Done Sent Done false
29822 8711 Acked Write 9 Done Sent Done false
29823 8712 Acked Read 3 Done Sent Done false
29824 8713 Acked Read 3 Done Sent Done false
29825 8714 Acked Write 6 Done Sent Done false
29826 8715 Acked Write 2 Done Sent Done false
29827 8716 Acked Write 1 Done Sent Done false
29828 8717 Acked Read 3 Done Sent Done false
29829 8718 Acked Write 1 Done Sent Done false
29830 8719 Acked Read 4 Done Sent Done false
29831 8720 Acked Read 3 Done Sent Done false
29832 8721 Acked Write 2 Done Sent Done false
29833 8722 Acked Write 9 Done Sent Done false
29834 8723 Acked Read 6 Done Sent Done false
29835 8724 Acked Write 6 Done Sent Done false
29836 8725 Acked Read 9 Done Sent Done false
29837 8726 Acked Read 1 Done Sent Done false
29838 8727 Acked Write 5 Done Sent Done false
29839 8728 Acked Write 7 Done Sent Done false
29840 8729 Acked Flush 0 Done Sent Done false
29841 8730 Acked Write 5 Done Sent Done false
29842 8731 Acked Write 4 Done Sent Done false
29843 8732 Acked Read 9 Done Sent Done false
29844 8733 Acked Read 2 Done Sent Done false
29845 8734 Acked Read 6 Done Sent Done false
29846 8735 Acked Read 8 Done Sent Done false
29847 8736 Acked Read 8 Done Sent Done false
29848 8737 Acked Read 7 Done Sent Done false
29849 8738 Acked Write 1 Done Sent Done false
29850 8739 Acked Read 8 Done Sent Done false
29851 8740 Acked Read 9 Done Sent Done false
29852 8741 Acked Write 8 Done Sent Done false
29853 8742 Acked Read 1 Done Sent Done false
29854 8743 Acked Read 2 Done Sent Done false
29855 8744 Acked Write 1 Done Sent Done false
29856 8745 Acked Read 2 Done Sent Done false
29857 8746 Acked Read 9 Done Sent Done false
29858 8747 Acked Write 5 Done Sent Done false
29859 8748 Acked Read 8 Done Sent Done false
29860 8749 Acked Flush 0 Done Sent Done false
29861 8750 Acked Write 10 Done Sent Done false
29862 8751 Acked Read 9 Done Sent Done false
29863 8752 Acked Read 8 Done Sent Done false
29864 8753 Acked Read 6 Done Sent Done false
29865 8754 Acked Read 8 Done Sent Done false
29866 8755 Acked Read 9 Done Sent Done false
29867 8756 Acked Read 7 Done Sent Done false
29868 8757 Acked Read 9 Done Sent Done false
29869 8758 Acked Read 3 Done Sent Done false
29870 8759 Acked Write 5 Done Sent Done false
29871 8760 Acked Read 8 Done Sent Done false
29872 8761 Acked Flush 0 Done Sent Done false
29873 8762 Acked Read 8 Done Sent Done false
29874 8763 Acked Read 7 Done Sent Done false
29875 8764 Acked Write 9 Done Sent Done false
29876 8765 Acked Write 10 Done Sent Done false
29877 8766 Acked Flush 0 Done Sent Done false
29878 8767 Acked Read 7 Done Sent Done false
29879 8768 Acked Read 6 Done Sent Done false
29880 8769 Acked Write 4 Done Sent Done false
29881 8770 Acked Read 9 Done Sent Done false
29882 8771 Acked Read 6 Done Sent Done false
29883 8772 Acked Read 9 Done Sent Done false
29884 8773 Acked Read 8 Done Sent Done false
29885 8774 Acked Write 6 Done Sent Done false
29886 8775 Acked Read 4 Done Sent Done false
29887 8776 Acked Read 5 Done Sent Done false
29888 8777 Acked Write 8 Done Sent Done false
29889 8778 Acked Read 9 Done Sent Done false
29890 8779 Acked Read 10 Done Sent Done false
29891 8780 Acked Read 9 Done Sent Done false
29892 8781 Acked Read 3 Done Sent Done false
29893 8782 Acked Read 9 Done Sent Done false
29894 8783 Acked Write 2 Done Sent Done false
29895 8784 Acked Write 8 Done Sent Done false
29896 8785 Acked Read 2 Done Sent Done false
29897 8786 Acked Write 8 Done Sent Done false
29898 8787 Acked Write 4 Done Sent Done false
29899 8788 Acked Write 1 Done Sent Done false
29900 8789 Acked Write 9 Done Sent Done false
29901 8790 Acked Read 8 Done Sent Done false
29902 8791 Acked Read 5 Done Sent Done false
29903 8792 Acked Write 1 Done Sent Done false
29904 8793 Acked Read 2 Done Sent Done false
29905 8794 Acked Write 2 Done Sent Done false
29906 8795 Acked Read 10 Done Sent Done false
29907 8796 Acked Write 4 Done Sent Done false
29908 8797 Acked Write 3 Done Sent Done false
29909 8798 Acked Write 3 Done Sent Done false
29910 8799 Acked Read 7 Done Sent Done false
29911 8800 Acked Read 10 Done Sent Done false
29912 8801 Acked Read 10 Done Sent Done false
29913 8802 Acked Write 4 Done Sent Done false
29914 8803 Acked Write 2 Done Sent Done false
29915 8804 Acked Write 7 Done Sent Done false
29916 8805 Acked Write 10 Done Sent Done false
29917 8806 Acked Write 1 Done Sent Done false
29918 8807 Acked Read 5 Done Sent Done false
29919 8808 Acked Read 8 Done Sent Done false
29920 8809 Acked Read 1 Done Sent Done false
29921 8810 Acked Flush 0 Done Sent Done false
29922 8811 Acked Read 8 Done Sent Done false
29923 8812 Acked Write 2 Done Sent Done false
29924 8813 Acked Write 6 Done Sent Done false
29925 8814 Acked Flush 0 Done Sent Done false
29926 8815 Acked Flush 0 Done Sent Done false
29927 8816 Acked Read 8 Done Sent Done false
29928 8817 Acked Read 4 Done Sent Done false
29929 8818 Acked Flush 0 Done Sent Done false
29930 8819 Acked Write 2 Done Sent Done false
29931 8820 Acked Write 1 Done Sent Done false
29932 8821 Acked Read 6 Done Sent Done false
29933 8822 Acked Write 4 Done Sent Done false
29934 8823 Acked Read 4 Done Sent Done false
29935 8824 Acked Write 5 Done Sent Done false
29936 8825 Acked Read 4 Done Sent Done false
29937 8826 Acked Read 6 Done Sent Done false
29938 8827 Acked Write 7 Done Sent Done false
29939 8828 Acked Write 5 Done Sent Done false
29940 8829 Acked Flush 0 Done Sent Done false
29941 8830 Acked Write 3 Done Sent Done false
29942 8831 Acked Write 7 Done Sent Done false
29943 8832 Acked Write 7 Done Sent Done false
29944 8833 Acked Read 5 Done Sent Done false
29945 8834 Acked Write 7 Done Sent Done false
29946 8835 Acked Flush 0 Done Sent Done false
29947 8836 Acked Read 10 Done Sent Done false
29948 8837 Acked Write 8 Done Sent Done false
29949 8838 Acked Read 4 Done Sent Done false
29950 8839 Acked Write 2 Done Sent Done false
29951 8840 Acked Read 2 Done Sent Done false
29952 8841 Acked Read 9 Done Sent Done false
29953 8842 Acked Write 4 Done Sent Done false
29954 8843 Acked Read 4 Done Sent Done false
29955 8844 Acked Read 9 Done Sent Done false
29956 8845 Acked Read 2 Done Sent Done false
29957 8846 Acked Write 4 Done Sent Done false
29958 8847 Acked Read 2 Done Sent Done false
29959 STATES DS:0 DS:1 DS:2 TOTAL
29960 Sent 0 301 0 301
29961 Done 301 0 301 602
29962 Skipped 0 0 0 0
29963 Error 0 0 0 0
29964 Last Flush: 8835 8546 8835
29965 Downstairs last five completed: 8546 8545 8544 8543 8542
29966 Upstairs last five completed: 8847 8846 8845 8844 8843
29967 CLIENT: Up:0 ds:301 act:2
29968 ----------------------------------------------------------------
29969 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
29970 States: Active Offline Active
29971 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
29972 8547 Acked Read 7 Done Sent Done false
29973 8548 Acked Flush 0 Done Sent Done false
29974 8549 Acked Read 10 Done Sent Done false
29975 8550 Acked Flush 0 Done Sent Done false
29976 8551 Acked Write 10 Done Sent Done false
29977 8552 Acked Write 6 Done Sent Done false
29978 8553 Acked Write 1 Done Sent Done false
29979 8554 Acked Read 2 Done Sent Done false
29980 8555 Acked Read 5 Done Sent Done false
29981 8556 Acked Read 4 Done Sent Done false
29982 8557 Acked Read 4 Done Sent Done false
29983 8558 Acked Read 10 Done Sent Done false
29984 8559 Acked Read 10 Done Sent Done false
29985 8560 Acked Read 3 Done Sent Done false
29986 8561 Acked Read 2 Done Sent Done false
29987 8562 Acked Write 7 Done Sent Done false
29988 8563 Acked Write 6 Done Sent Done false
29989 8564 Acked Flush 0 Done Sent Done false
29990 8565 Acked Read 5 Done Sent Done false
29991 8566 Acked Write 6 Done Sent Done false
29992 8567 Acked Read 4 Done Sent Done false
29993 8568 Acked Read 9 Done Sent Done false
29994 8569 Acked Write 7 Done Sent Done false
29995 8570 Acked Read 5 Done Sent Done false
29996 8571 Acked Read 5 Done Sent Done false
29997 8572 Acked Read 6 Done Sent Done false
29998 8573 Acked Write 9 Done Sent Done false
29999 8574 Acked Write 3 Done Sent Done false
30000 8575 Acked Write 6 Done Sent Done false
30001 8576 Acked Flush 0 Done Sent Done false
30002 8577 Acked Write 4 Done Sent Done false
30003 8578 Acked Read 9 Done Sent Done false
30004 8579 Acked Read 9 Done Sent Done false
30005 8580 Acked Read 7 Done Sent Done false
30006 8581 Acked Write 9 Done Sent Done false
30007 8582 Acked Read 6 Done Sent Done false
30008 8583 Acked Write 7 Done Sent Done false
30009 8584 Acked Write 3 Done Sent Done false
30010 8585 Acked Read 4 Done Sent Done false
30011 8586 Acked Write 8 Done Sent Done false
30012 8587 Acked Flush 0 Done Sent Done false
30013 8588 Acked Read 2 Done Sent Done false
30014 8589 Acked Read 2 Done Sent Done false
30015 8590 Acked Read 5 Done Sent Done false
30016 8591 Acked Read 2 Done Sent Done false
30017 8592 Acked Write 2 Done Sent Done false
30018 8593 Acked Read 1 Done Sent Done false
30019 8594 Acked Write 8 Done Sent Done false
30020 8595 Acked Read 3 Done Sent Done false
30021 8596 Acked Write 6 Done Sent Done false
30022 8597 Acked Write 7 Done Sent Done false
30023 8598 Acked Read 2 Done Sent Done false
30024 8599 Acked Write 6 Done Sent Done false
30025 8600 Acked Write 1 Done Sent Done false
30026 8601 Acked Write 9 Done Sent Done false
30027 8602 Acked Read 9 Done Sent Done false
30028 8603 Acked Read 2 Done Sent Done false
30029 8604 Acked Write 7 Done Sent Done false
30030 8605 Acked Read 4 Done Sent Done false
30031 8606 Acked Flush 0 Done Sent Done false
30032 8607 Acked Read 10 Done Sent Done false
30033 8608 Acked Write 9 Done Sent Done false
30034 8609 Acked Write 8 Done Sent Done false
30035 8610 Acked Write 10 Done Sent Done false
30036 8611 Acked Read 9 Done Sent Done false
30037 8612 Acked Write 3 Done Sent Done false
30038 8613 Acked Write 9 Done Sent Done false
30039 8614 Acked Write 1 Done Sent Done false
30040 8615 Acked Read 10 Done Sent Done false
30041 8616 Acked Write 3 Done Sent Done false
30042 8617 Acked Flush 0 Done Sent Done false
30043 8618 Acked Read 2 Done Sent Done false
30044 8619 Acked Read 8 Done Sent Done false
30045 8620 Acked Flush 0 Done Sent Done false
30046 8621 Acked Flush 0 Done Sent Done false
30047 8622 Acked Write 1 Done Sent Done false
30048 8623 Acked Read 10 Done Sent Done false
30049 8624 Acked Read 3 Done Sent Done false
30050 8625 Acked Write 6 Done Sent Done false
30051 8626 Acked Read 8 Done Sent Done false
30052 8627 Acked Read 2 Done Sent Done false
30053 8628 Acked Write 9 Done Sent Done false
30054 8629 Acked Read 10 Done Sent Done false
30055 8630 Acked Write 7 Done Sent Done false
30056 8631 Acked Read 6 Done Sent Done false
30057 8632 Acked Write 2 Done Sent Done false
30058 8633 Acked Read 8 Done Sent Done false
30059 8634 Acked Read 3 Done Sent Done false
30060 8635 Acked Read 3 Done Sent Done false
30061 8636 Acked Flush 0 Done Sent Done false
30062 8637 Acked Write 3 Done Sent Done false
30063 8638 Acked Write 4 Done Sent Done false
30064 8639 Acked Read 9 Done Sent Done false
30065 8640 Acked Write 10 Done Sent Done false
30066 8641 Acked Write 4 Done Sent Done false
30067 8642 Acked Read 8 Done Sent Done false
30068 8643 Acked Read 1 Done Sent Done false
30069 8644 Acked Read 2 Done Sent Done false
30070 8645 Acked Write 6 Done Sent Done false
30071 8646 Acked Read 3 Done Sent Done false
30072 8647 Acked Write 4 Done Sent Done false
30073 8648 Acked Read 5 Done Sent Done false
30074 8649 Acked Read 2 Done Sent Done false
30075 8650 Acked Read 10 Done Sent Done false
30076 8651 Acked Read 5 Done Sent Done false
30077 8652 Acked Read 3 Done Sent Done false
30078 8653 Acked Flush 0 Done Sent Done false
30079 8654 Acked Write 9 Done Sent Done false
30080 8655 Acked Read 2 Done Sent Done false
30081 8656 Acked Write 8 Done Sent Done false
30082 8657 Acked Read 1 Done Sent Done false
30083 8658 Acked Read 4 Done Sent Done false
30084 8659 Acked Read 3 Done Sent Done false
30085 8660 Acked Read 3 Done Sent Done false
30086 8661 Acked Read 2 Done Sent Done false
30087 8662 Acked Write 4 Done Sent Done false
30088 8663 Acked Write 1 Done Sent Done false
30089 8664 Acked Write 9 Done Sent Done false
30090 8665 Acked Read 10 Done Sent Done false
30091 8666 Acked Write 1 Done Sent Done false
30092 8667 Acked Flush 0 Done Sent Done false
30093 8668 Acked Write 7 Done Sent Done false
30094 8669 Acked Read 7 Done Sent Done false
30095 8670 Acked Read 10 Done Sent Done false
30096 8671 Acked Write 7 Done Sent Done false
30097 8672 Acked Read 2 Done Sent Done false
30098 8673 Acked Write 3 Done Sent Done false
30099 8674 Acked Write 5 Done Sent Done false
30100 8675 Acked Read 9 Done Sent Done false
30101 8676 Acked Read 3 Done Sent Done false
30102 8677 Acked Read 6 Done Sent Done false
30103 8678 Acked Read 9 Done Sent Done false
30104 8679 Acked Read 3 Done Sent Done false
30105 8680 Acked Read 2 Done Sent Done false
30106 8681 Acked Read 2 Done Sent Done false
30107 8682 Acked Read 6 Done Sent Done false
30108 8683 Acked Write 1 Done Sent Done false
30109 8684 Acked Read 8 Done Sent Done false
30110 8685 Acked Read 7 Done Sent Done false
30111 8686 Acked Read 4 Done Sent Done false
30112 8687 Acked Read 4 Done Sent Done false
30113 8688 Acked Flush 0 Done Sent Done false
30114 8689 Acked Write 3 Done Sent Done false
30115 8690 Acked Read 10 Done Sent Done false
30116 8691 Acked Write 6 Done Sent Done false
30117 8692 Acked Read 9 Done Sent Done false
30118 8693 Acked Read 10 Done Sent Done false
30119 8694 Acked Write 2 Done Sent Done false
30120 8695 Acked Write 6 Done Sent Done false
30121 8696 Acked Write 10 Done Sent Done false
30122 8697 Acked Write 7 Done Sent Done false
30123 8698 Acked Read 5 Done Sent Done false
30124 8699 Acked Write 8 Done Sent Done false
30125 8700 Acked Read 10 Done Sent Done false
30126 8701 Acked Write 8 Done Sent Done false
30127 8702 Acked Read 10 Done Sent Done false
30128 8703 Acked Write 7 Done Sent Done false
30129 8704 Acked Write 1 Done Sent Done false
30130 8705 Acked Write 8 Done Sent Done false
30131 8706 Acked Read 9 Done Sent Done false
30132 8707 Acked Read 1 Done Sent Done false
30133 8708 Acked Write 5 Done Sent Done false
30134 8709 Acked Write 4 Done Sent Done false
30135 8710 Acked Read 2 Done Sent Done false
30136 8711 Acked Write 9 Done Sent Done false
30137 8712 Acked Read 3 Done Sent Done false
30138 8713 Acked Read 3 Done Sent Done false
30139 8714 Acked Write 6 Done Sent Done false
30140 8715 Acked Write 2 Done Sent Done false
30141 8716 Acked Write 1 Done Sent Done false
30142 8717 Acked Read 3 Done Sent Done false
30143 8718 Acked Write 1 Done Sent Done false
30144 8719 Acked Read 4 Done Sent Done false
30145 8720 Acked Read 3 Done Sent Done false
30146 8721 Acked Write 2 Done Sent Done false
30147 8722 Acked Write 9 Done Sent Done false
30148 8723 Acked Read 6 Done Sent Done false
30149 8724 Acked Write 6 Done Sent Done false
30150 8725 Acked Read 9 Done Sent Done false
30151 8726 Acked Read 1 Done Sent Done false
30152 8727 Acked Write 5 Done Sent Done false
30153 8728 Acked Write 7 Done Sent Done false
30154 8729 Acked Flush 0 Done Sent Done false
30155 8730 Acked Write 5 Done Sent Done false
30156 8731 Acked Write 4 Done Sent Done false
30157 8732 Acked Read 9 Done Sent Done false
30158 8733 Acked Read 2 Done Sent Done false
30159 8734 Acked Read 6 Done Sent Done false
30160 8735 Acked Read 8 Done Sent Done false
30161 8736 Acked Read 8 Done Sent Done false
30162 8737 Acked Read 7 Done Sent Done false
30163 8738 Acked Write 1 Done Sent Done false
30164 8739 Acked Read 8 Done Sent Done false
30165 8740 Acked Read 9 Done Sent Done false
30166 8741 Acked Write 8 Done Sent Done false
30167 8742 Acked Read 1 Done Sent Done false
30168 8743 Acked Read 2 Done Sent Done false
30169 8744 Acked Write 1 Done Sent Done false
30170 8745 Acked Read 2 Done Sent Done false
30171 8746 Acked Read 9 Done Sent Done false
30172 8747 Acked Write 5 Done Sent Done false
30173 8748 Acked Read 8 Done Sent Done false
30174 8749 Acked Flush 0 Done Sent Done false
30175 8750 Acked Write 10 Done Sent Done false
30176 8751 Acked Read 9 Done Sent Done false
30177 8752 Acked Read 8 Done Sent Done false
30178 8753 Acked Read 6 Done Sent Done false
30179 8754 Acked Read 8 Done Sent Done false
30180 8755 Acked Read 9 Done Sent Done false
30181 8756 Acked Read 7 Done Sent Done false
30182 8757 Acked Read 9 Done Sent Done false
30183 8758 Acked Read 3 Done Sent Done false
30184 8759 Acked Write 5 Done Sent Done false
30185 8760 Acked Read 8 Done Sent Done false
30186 8761 Acked Flush 0 Done Sent Done false
30187 8762 Acked Read 8 Done Sent Done false
30188 8763 Acked Read 7 Done Sent Done false
30189 8764 Acked Write 9 Done Sent Done false
30190 8765 Acked Write 10 Done Sent Done false
30191 8766 Acked Flush 0 Done Sent Done false
30192 8767 Acked Read 7 Done Sent Done false
30193 8768 Acked Read 6 Done Sent Done false
30194 8769 Acked Write 4 Done Sent Done false
30195 8770 Acked Read 9 Done Sent Done false
30196 8771 Acked Read 6 Done Sent Done false
30197 8772 Acked Read 9 Done Sent Done false
30198 8773 Acked Read 8 Done Sent Done false
30199 8774 Acked Write 6 Done Sent Done false
30200 8775 Acked Read 4 Done Sent Done false
30201 8776 Acked Read 5 Done Sent Done false
30202 8777 Acked Write 8 Done Sent Done false
30203 8778 Acked Read 9 Done Sent Done false
30204 8779 Acked Read 10 Done Sent Done false
30205 8780 Acked Read 9 Done Sent Done false
30206 8781 Acked Read 3 Done Sent Done false
30207 8782 Acked Read 9 Done Sent Done false
30208 8783 Acked Write 2 Done Sent Done false
30209 8784 Acked Write 8 Done Sent Done false
30210 8785 Acked Read 2 Done Sent Done false
30211 8786 Acked Write 8 Done Sent Done false
30212 8787 Acked Write 4 Done Sent Done false
30213 8788 Acked Write 1 Done Sent Done false
30214 8789 Acked Write 9 Done Sent Done false
30215 8790 Acked Read 8 Done Sent Done false
30216 8791 Acked Read 5 Done Sent Done false
30217 8792 Acked Write 1 Done Sent Done false
30218 8793 Acked Read 2 Done Sent Done false
30219 8794 Acked Write 2 Done Sent Done false
30220 8795 Acked Read 10 Done Sent Done false
30221 8796 Acked Write 4 Done Sent Done false
30222 8797 Acked Write 3 Done Sent Done false
30223 8798 Acked Write 3 Done Sent Done false
30224 8799 Acked Read 7 Done Sent Done false
30225 8800 Acked Read 10 Done Sent Done false
30226 8801 Acked Read 10 Done Sent Done false
30227 8802 Acked Write 4 Done Sent Done false
30228 8803 Acked Write 2 Done Sent Done false
30229 8804 Acked Write 7 Done Sent Done false
30230 8805 Acked Write 10 Done Sent Done false
30231 8806 Acked Write 1 Done Sent Done false
30232 8807 Acked Read 5 Done Sent Done false
30233 8808 Acked Read 8 Done Sent Done false
30234 8809 Acked Read 1 Done Sent Done false
30235 8810 Acked Flush 0 Done Sent Done false
30236 8811 Acked Read 8 Done Sent Done false
30237 8812 Acked Write 2 Done Sent Done false
30238 8813 Acked Write 6 Done Sent Done false
30239 8814 Acked Flush 0 Done Sent Done false
30240 8815 Acked Flush 0 Done Sent Done false
30241 8816 Acked Read 8 Done Sent Done false
30242 8817 Acked Read 4 Done Sent Done false
30243 8818 Acked Flush 0 Done Sent Done false
30244 8819 Acked Write 2 Done Sent Done false
30245 8820 Acked Write 1 Done Sent Done false
30246 8821 Acked Read 6 Done Sent Done false
30247 8822 Acked Write 4 Done Sent Done false
30248 8823 Acked Read 4 Done Sent Done false
30249 8824 Acked Write 5 Done Sent Done false
30250 8825 Acked Read 4 Done Sent Done false
30251 8826 Acked Read 6 Done Sent Done false
30252 8827 Acked Write 7 Done Sent Done false
30253 8828 Acked Write 5 Done Sent Done false
30254 8829 Acked Flush 0 Done Sent Done false
30255 8830 Acked Write 3 Done Sent Done false
30256 8831 Acked Write 7 Done Sent Done false
30257 8832 Acked Write 7 Done Sent Done false
30258 8833 Acked Read 5 Done Sent Done false
30259 8834 Acked Write 7 Done Sent Done false
30260 8835 Acked Flush 0 Done Sent Done false
30261 8836 Acked Read 10 Done Sent Done false
30262 8837 Acked Write 8 Done Sent Done false
30263 8838 Acked Read 4 Done Sent Done false
30264 8839 Acked Write 2 Done Sent Done false
30265 8840 Acked Read 2 Done Sent Done false
30266 8841 Acked Read 9 Done Sent Done false
30267 8842 Acked Write 4 Done Sent Done false
30268 8843 Acked Read 4 Done Sent Done false
30269 8844 Acked Read 9 Done Sent Done false
30270 8845 Acked Read 2 Done Sent Done false
30271 8846 Acked Write 4 Done Sent Done false
30272 8847 Acked Read 2 Done Sent Done false
30273 8848 Acked Flush 0 Done Sent Done false
30274 STATES DS:0 DS:1 DS:2 TOTAL
30275 Sent 0 302 0 302
30276 Done 302 0 302 604
30277 Skipped 0 0 0 0
30278 Error 0 0 0 0
30279 Last Flush: 8848 8546 8848
30280 Downstairs last five completed: 8546 8545 8544 8543 8542
30281 Upstairs last five completed: 8848 8847 8846 8845 8844
30282 CLIENT: Up:0 ds:302 act:2
302832024-11-26T22:10:52.530ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
302842024-11-26T22:10:52.530ZWARNcrucible: ds_connection connect to 127.0.0.1:8820 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
302852024-11-26T22:10:52.530ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
302862024-11-26T22:10:52.530ZWARNcrucible: downstairs task for 1 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
302872024-11-26T22:10:52.530ZINFOcrucible: client task is exiting = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
30288 [8820] got start action
30289 [8820] I am going to restart
30290 Starting downstairs at port 8820
30291 Make output file at "/tmp/dsc/downstairs-8820.txt"
30292 [8820][1] reports Starting
30293 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1229)
30294 [8820][1] reports Running
30295 ----------------------------------------------------------------
30296 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
30297 States: Active Offline Active
30298 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
30299 8547 Acked Read 7 Done Sent Done false
30300 8548 Acked Flush 0 Done Sent Done false
30301 8549 Acked Read 10 Done Sent Done false
30302 8550 Acked Flush 0 Done Sent Done false
30303 8551 Acked Write 10 Done Sent Done false
30304 8552 Acked Write 6 Done Sent Done false
30305 8553 Acked Write 1 Done Sent Done false
30306 8554 Acked Read 2 Done Sent Done false
30307 8555 Acked Read 5 Done Sent Done false
30308 8556 Acked Read 4 Done Sent Done false
30309 8557 Acked Read 4 Done Sent Done false
30310 8558 Acked Read 10 Done Sent Done false
30311 8559 Acked Read 10 Done Sent Done false
30312 8560 Acked Read 3 Done Sent Done false
30313 8561 Acked Read 2 Done Sent Done false
30314 8562 Acked Write 7 Done Sent Done false
30315 8563 Acked Write 6 Done Sent Done false
30316 8564 Acked Flush 0 Done Sent Done false
30317 8565 Acked Read 5 Done Sent Done false
30318 8566 Acked Write 6 Done Sent Done false
30319 8567 Acked Read 4 Done Sent Done false
30320 8568 Acked Read 9 Done Sent Done false
30321 8569 Acked Write 7 Done Sent Done false
30322 8570 Acked Read 5 Done Sent Done false
30323 8571 Acked Read 5 Done Sent Done false
30324 8572 Acked Read 6 Done Sent Done false
30325 8573 Acked Write 9 Done Sent Done false
30326 8574 Acked Write 3 Done Sent Done false
30327 8575 Acked Write 6 Done Sent Done false
30328 8576 Acked Flush 0 Done Sent Done false
30329 8577 Acked Write 4 Done Sent Done false
30330 8578 Acked Read 9 Done Sent Done false
30331 8579 Acked Read 9 Done Sent Done false
30332 8580 Acked Read 7 Done Sent Done false
30333 8581 Acked Write 9 Done Sent Done false
30334 8582 Acked Read 6 Done Sent Done false
30335 8583 Acked Write 7 Done Sent Done false
30336 8584 Acked Write 3 Done Sent Done false
30337 8585 Acked Read 4 Done Sent Done false
30338 8586 Acked Write 8 Done Sent Done false
30339 8587 Acked Flush 0 Done Sent Done false
30340 8588 Acked Read 2 Done Sent Done false
30341 8589 Acked Read 2 Done Sent Done false
30342 8590 Acked Read 5 Done Sent Done false
30343 8591 Acked Read 2 Done Sent Done false
30344 8592 Acked Write 2 Done Sent Done false
30345 8593 Acked Read 1 Done Sent Done false
30346 8594 Acked Write 8 Done Sent Done false
30347 8595 Acked Read 3 Done Sent Done false
30348 8596 Acked Write 6 Done Sent Done false
30349 8597 Acked Write 7 Done Sent Done false
30350 8598 Acked Read 2 Done Sent Done false
30351 8599 Acked Write 6 Done Sent Done false
30352 8600 Acked Write 1 Done Sent Done false
30353 8601 Acked Write 9 Done Sent Done false
30354 8602 Acked Read 9 Done Sent Done false
30355 8603 Acked Read 2 Done Sent Done false
30356 8604 Acked Write 7 Done Sent Done false
30357 8605 Acked Read 4 Done Sent Done false
30358 8606 Acked Flush 0 Done Sent Done false
30359 8607 Acked Read 10 Done Sent Done false
30360 8608 Acked Write 9 Done Sent Done false
30361 8609 Acked Write 8 Done Sent Done false
30362 8610 Acked Write 10 Done Sent Done false
30363 8611 Acked Read 9 Done Sent Done false
30364 8612 Acked Write 3 Done Sent Done false
30365 8613 Acked Write 9 Done Sent Done false
30366 8614 Acked Write 1 Done Sent Done false
30367 8615 Acked Read 10 Done Sent Done false
30368 8616 Acked Write 3 Done Sent Done false
30369 8617 Acked Flush 0 Done Sent Done false
30370 8618 Acked Read 2 Done Sent Done false
30371 8619 Acked Read 8 Done Sent Done false
30372 8620 Acked Flush 0 Done Sent Done false
30373 8621 Acked Flush 0 Done Sent Done false
30374 8622 Acked Write 1 Done Sent Done false
30375 8623 Acked Read 10 Done Sent Done false
30376 8624 Acked Read 3 Done Sent Done false
30377 8625 Acked Write 6 Done Sent Done false
30378 8626 Acked Read 8 Done Sent Done false
30379 8627 Acked Read 2 Done Sent Done false
30380 8628 Acked Write 9 Done Sent Done false
30381 8629 Acked Read 10 Done Sent Done false
30382 8630 Acked Write 7 Done Sent Done false
30383 8631 Acked Read 6 Done Sent Done false
30384 8632 Acked Write 2 Done Sent Done false
30385 8633 Acked Read 8 Done Sent Done false
30386 8634 Acked Read 3 Done Sent Done false
30387 8635 Acked Read 3 Done Sent Done false
30388 8636 Acked Flush 0 Done Sent Done false
30389 8637 Acked Write 3 Done Sent Done false
30390 8638 Acked Write 4 Done Sent Done false
30391 8639 Acked Read 9 Done Sent Done false
30392 8640 Acked Write 10 Done Sent Done false
30393 8641 Acked Write 4 Done Sent Done false
30394 8642 Acked Read 8 Done Sent Done false
30395 8643 Acked Read 1 Done Sent Done false
30396 8644 Acked Read 2 Done Sent Done false
30397 8645 Acked Write 6 Done Sent Done false
30398 8646 Acked Read 3 Done Sent Done false
30399 8647 Acked Write 4 Done Sent Done false
30400 8648 Acked Read 5 Done Sent Done false
30401 8649 Acked Read 2 Done Sent Done false
30402 8650 Acked Read 10 Done Sent Done false
30403 8651 Acked Read 5 Done Sent Done false
30404 8652 Acked Read 3 Done Sent Done false
30405 8653 Acked Flush 0 Done Sent Done false
30406 8654 Acked Write 9 Done Sent Done false
30407 8655 Acked Read 2 Done Sent Done false
30408 8656 Acked Write 8 Done Sent Done false
30409 8657 Acked Read 1 Done Sent Done false
30410 8658 Acked Read 4 Done Sent Done false
30411 8659 Acked Read 3 Done Sent Done false
30412 8660 Acked Read 3 Done Sent Done false
30413 8661 Acked Read 2 Done Sent Done false
30414 8662 Acked Write 4 Done Sent Done false
30415 8663 Acked Write 1 Done Sent Done false
30416 8664 Acked Write 9 Done Sent Done false
30417 8665 Acked Read 10 Done Sent Done false
30418 8666 Acked Write 1 Done Sent Done false
30419 8667 Acked Flush 0 Done Sent Done false
30420 8668 Acked Write 7 Done Sent Done false
30421 8669 Acked Read 7 Done Sent Done false
30422 8670 Acked Read 10 Done Sent Done false
30423 8671 Acked Write 7 Done Sent Done false
30424 8672 Acked Read 2 Done Sent Done false
30425 8673 Acked Write 3 Done Sent Done false
30426 8674 Acked Write 5 Done Sent Done false
30427 8675 Acked Read 9 Done Sent Done false
30428 8676 Acked Read 3 Done Sent Done false
30429 8677 Acked Read 6 Done Sent Done false
30430 8678 Acked Read 9 Done Sent Done false
30431 8679 Acked Read 3 Done Sent Done false
30432 8680 Acked Read 2 Done Sent Done false
30433 8681 Acked Read 2 Done Sent Done false
30434 8682 Acked Read 6 Done Sent Done false
30435 8683 Acked Write 1 Done Sent Done false
30436 8684 Acked Read 8 Done Sent Done false
30437 8685 Acked Read 7 Done Sent Done false
30438 8686 Acked Read 4 Done Sent Done false
30439 8687 Acked Read 4 Done Sent Done false
30440 8688 Acked Flush 0 Done Sent Done false
30441 8689 Acked Write 3 Done Sent Done false
30442 8690 Acked Read 10 Done Sent Done false
30443 8691 Acked Write 6 Done Sent Done false
30444 8692 Acked Read 9 Done Sent Done false
30445 8693 Acked Read 10 Done Sent Done false
30446 8694 Acked Write 2 Done Sent Done false
30447 8695 Acked Write 6 Done Sent Done false
30448 8696 Acked Write 10 Done Sent Done false
30449 8697 Acked Write 7 Done Sent Done false
30450 8698 Acked Read 5 Done Sent Done false
30451 8699 Acked Write 8 Done Sent Done false
30452 8700 Acked Read 10 Done Sent Done false
30453 8701 Acked Write 8 Done Sent Done false
30454 8702 Acked Read 10 Done Sent Done false
30455 8703 Acked Write 7 Done Sent Done false
30456 8704 Acked Write 1 Done Sent Done false
30457 8705 Acked Write 8 Done Sent Done false
30458 8706 Acked Read 9 Done Sent Done false
30459 8707 Acked Read 1 Done Sent Done false
30460 8708 Acked Write 5 Done Sent Done false
30461 8709 Acked Write 4 Done Sent Done false
30462 8710 Acked Read 2 Done Sent Done false
30463 8711 Acked Write 9 Done Sent Done false
30464 8712 Acked Read 3 Done Sent Done false
30465 8713 Acked Read 3 Done Sent Done false
30466 8714 Acked Write 6 Done Sent Done false
30467 8715 Acked Write 2 Done Sent Done false
30468 8716 Acked Write 1 Done Sent Done false
30469 8717 Acked Read 3 Done Sent Done false
30470 8718 Acked Write 1 Done Sent Done false
30471 8719 Acked Read 4 Done Sent Done false
30472 8720 Acked Read 3 Done Sent Done false
30473 8721 Acked Write 2 Done Sent Done false
30474 8722 Acked Write 9 Done Sent Done false
30475 8723 Acked Read 6 Done Sent Done false
30476 8724 Acked Write 6 Done Sent Done false
30477 8725 Acked Read 9 Done Sent Done false
30478 8726 Acked Read 1 Done Sent Done false
30479 8727 Acked Write 5 Done Sent Done false
30480 8728 Acked Write 7 Done Sent Done false
30481 8729 Acked Flush 0 Done Sent Done false
30482 8730 Acked Write 5 Done Sent Done false
30483 8731 Acked Write 4 Done Sent Done false
30484 8732 Acked Read 9 Done Sent Done false
30485 8733 Acked Read 2 Done Sent Done false
30486 8734 Acked Read 6 Done Sent Done false
30487 8735 Acked Read 8 Done Sent Done false
30488 8736 Acked Read 8 Done Sent Done false
30489 8737 Acked Read 7 Done Sent Done false
30490 8738 Acked Write 1 Done Sent Done false
30491 8739 Acked Read 8 Done Sent Done false
30492 8740 Acked Read 9 Done Sent Done false
30493 8741 Acked Write 8 Done Sent Done false
30494 8742 Acked Read 1 Done Sent Done false
30495 8743 Acked Read 2 Done Sent Done false
30496 8744 Acked Write 1 Done Sent Done false
30497 8745 Acked Read 2 Done Sent Done false
30498 8746 Acked Read 9 Done Sent Done false
30499 8747 Acked Write 5 Done Sent Done false
30500 8748 Acked Read 8 Done Sent Done false
30501 8749 Acked Flush 0 Done Sent Done false
30502 8750 Acked Write 10 Done Sent Done false
30503 8751 Acked Read 9 Done Sent Done false
30504 8752 Acked Read 8 Done Sent Done false
30505 8753 Acked Read 6 Done Sent Done false
30506 8754 Acked Read 8 Done Sent Done false
30507 8755 Acked Read 9 Done Sent Done false
30508 8756 Acked Read 7 Done Sent Done false
30509 8757 Acked Read 9 Done Sent Done false
30510 8758 Acked Read 3 Done Sent Done false
30511 8759 Acked Write 5 Done Sent Done false
30512 8760 Acked Read 8 Done Sent Done false
30513 8761 Acked Flush 0 Done Sent Done false
30514 8762 Acked Read 8 Done Sent Done false
30515 8763 Acked Read 7 Done Sent Done false
30516 8764 Acked Write 9 Done Sent Done false
30517 8765 Acked Write 10 Done Sent Done false
30518 8766 Acked Flush 0 Done Sent Done false
30519 8767 Acked Read 7 Done Sent Done false
30520 8768 Acked Read 6 Done Sent Done false
30521 8769 Acked Write 4 Done Sent Done false
30522 8770 Acked Read 9 Done Sent Done false
30523 8771 Acked Read 6 Done Sent Done false
30524 8772 Acked Read 9 Done Sent Done false
30525 8773 Acked Read 8 Done Sent Done false
30526 8774 Acked Write 6 Done Sent Done false
30527 8775 Acked Read 4 Done Sent Done false
30528 8776 Acked Read 5 Done Sent Done false
30529 8777 Acked Write 8 Done Sent Done false
30530 8778 Acked Read 9 Done Sent Done false
30531 8779 Acked Read 10 Done Sent Done false
30532 8780 Acked Read 9 Done Sent Done false
30533 8781 Acked Read 3 Done Sent Done false
30534 8782 Acked Read 9 Done Sent Done false
30535 8783 Acked Write 2 Done Sent Done false
30536 8784 Acked Write 8 Done Sent Done false
30537 8785 Acked Read 2 Done Sent Done false
30538 8786 Acked Write 8 Done Sent Done false
30539 8787 Acked Write 4 Done Sent Done false
30540 8788 Acked Write 1 Done Sent Done false
30541 8789 Acked Write 9 Done Sent Done false
30542 8790 Acked Read 8 Done Sent Done false
30543 8791 Acked Read 5 Done Sent Done false
30544 8792 Acked Write 1 Done Sent Done false
30545 8793 Acked Read 2 Done Sent Done false
30546 8794 Acked Write 2 Done Sent Done false
30547 8795 Acked Read 10 Done Sent Done false
30548 8796 Acked Write 4 Done Sent Done false
30549 8797 Acked Write 3 Done Sent Done false
30550 8798 Acked Write 3 Done Sent Done false
30551 8799 Acked Read 7 Done Sent Done false
30552 8800 Acked Read 10 Done Sent Done false
30553 8801 Acked Read 10 Done Sent Done false
30554 8802 Acked Write 4 Done Sent Done false
30555 8803 Acked Write 2 Done Sent Done false
30556 8804 Acked Write 7 Done Sent Done false
30557 8805 Acked Write 10 Done Sent Done false
30558 8806 Acked Write 1 Done Sent Done false
30559 8807 Acked Read 5 Done Sent Done false
30560 8808 Acked Read 8 Done Sent Done false
30561 8809 Acked Read 1 Done Sent Done false
30562 8810 Acked Flush 0 Done Sent Done false
30563 8811 Acked Read 8 Done Sent Done false
30564 8812 Acked Write 2 Done Sent Done false
30565 8813 Acked Write 6 Done Sent Done false
30566 8814 Acked Flush 0 Done Sent Done false
30567 8815 Acked Flush 0 Done Sent Done false
30568 8816 Acked Read 8 Done Sent Done false
30569 8817 Acked Read 4 Done Sent Done false
30570 8818 Acked Flush 0 Done Sent Done false
30571 8819 Acked Write 2 Done Sent Done false
30572 8820 Acked Write 1 Done Sent Done false
30573 8821 Acked Read 6 Done Sent Done false
30574 8822 Acked Write 4 Done Sent Done false
30575 8823 Acked Read 4 Done Sent Done false
30576 8824 Acked Write 5 Done Sent Done false
30577 8825 Acked Read 4 Done Sent Done false
30578 8826 Acked Read 6 Done Sent Done false
30579 8827 Acked Write 7 Done Sent Done false
30580 8828 Acked Write 5 Done Sent Done false
30581 8829 Acked Flush 0 Done Sent Done false
30582 8830 Acked Write 3 Done Sent Done false
30583 8831 Acked Write 7 Done Sent Done false
30584 8832 Acked Write 7 Done Sent Done false
30585 8833 Acked Read 5 Done Sent Done false
30586 8834 Acked Write 7 Done Sent Done false
30587 8835 Acked Flush 0 Done Sent Done false
30588 8836 Acked Read 10 Done Sent Done false
30589 8837 Acked Write 8 Done Sent Done false
30590 8838 Acked Read 4 Done Sent Done false
30591 8839 Acked Write 2 Done Sent Done false
30592 8840 Acked Read 2 Done Sent Done false
30593 8841 Acked Read 9 Done Sent Done false
30594 8842 Acked Write 4 Done Sent Done false
30595 8843 Acked Read 4 Done Sent Done false
30596 8844 Acked Read 9 Done Sent Done false
30597 8845 Acked Read 2 Done Sent Done false
30598 8846 Acked Write 4 Done Sent Done false
30599 8847 Acked Read 2 Done Sent Done false
30600 8848 Acked Flush 0 Done Sent Done false
30601 STATES DS:0 DS:1 DS:2 TOTAL
30602 Sent 0 302 0 302
30603 Done 302 0 302 604
30604 Skipped 0 0 0 0
30605 Error 0 0 0 0
30606 Last Flush: 8848 8546 8848
30607 Downstairs last five completed: 8546 8545 8544 8543 8542
30608 Upstairs last five completed: 8848 8847 8846 8845 8844
30609 CLIENT: Up:0 ds:302 act:2
30610 ----------------------------------------------------------------
30611 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
30612 States: Active Offline Active
30613 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
30614 8547 Acked Read 7 Done Sent Done false
30615 8548 Acked Flush 0 Done Sent Done false
30616 8549 Acked Read 10 Done Sent Done false
30617 8550 Acked Flush 0 Done Sent Done false
30618 8551 Acked Write 10 Done Sent Done false
30619 8552 Acked Write 6 Done Sent Done false
30620 8553 Acked Write 1 Done Sent Done false
30621 8554 Acked Read 2 Done Sent Done false
30622 8555 Acked Read 5 Done Sent Done false
30623 8556 Acked Read 4 Done Sent Done false
30624 8557 Acked Read 4 Done Sent Done false
30625 8558 Acked Read 10 Done Sent Done false
30626 8559 Acked Read 10 Done Sent Done false
30627 8560 Acked Read 3 Done Sent Done false
30628 8561 Acked Read 2 Done Sent Done false
30629 8562 Acked Write 7 Done Sent Done false
30630 8563 Acked Write 6 Done Sent Done false
30631 8564 Acked Flush 0 Done Sent Done false
30632 8565 Acked Read 5 Done Sent Done false
30633 8566 Acked Write 6 Done Sent Done false
30634 8567 Acked Read 4 Done Sent Done false
30635 8568 Acked Read 9 Done Sent Done false
30636 8569 Acked Write 7 Done Sent Done false
30637 8570 Acked Read 5 Done Sent Done false
30638 8571 Acked Read 5 Done Sent Done false
30639 8572 Acked Read 6 Done Sent Done false
30640 8573 Acked Write 9 Done Sent Done false
30641 8574 Acked Write 3 Done Sent Done false
30642 8575 Acked Write 6 Done Sent Done false
30643 8576 Acked Flush 0 Done Sent Done false
30644 8577 Acked Write 4 Done Sent Done false
30645 8578 Acked Read 9 Done Sent Done false
30646 8579 Acked Read 9 Done Sent Done false
30647 8580 Acked Read 7 Done Sent Done false
30648 8581 Acked Write 9 Done Sent Done false
30649 8582 Acked Read 6 Done Sent Done false
30650 8583 Acked Write 7 Done Sent Done false
30651 8584 Acked Write 3 Done Sent Done false
30652 8585 Acked Read 4 Done Sent Done false
30653 8586 Acked Write 8 Done Sent Done false
30654 8587 Acked Flush 0 Done Sent Done false
30655 8588 Acked Read 2 Done Sent Done false
30656 8589 Acked Read 2 Done Sent Done false
30657 8590 Acked Read 5 Done Sent Done false
30658 8591 Acked Read 2 Done Sent Done false
30659 8592 Acked Write 2 Done Sent Done false
30660 8593 Acked Read 1 Done Sent Done false
30661 8594 Acked Write 8 Done Sent Done false
30662 8595 Acked Read 3 Done Sent Done false
30663 8596 Acked Write 6 Done Sent Done false
30664 8597 Acked Write 7 Done Sent Done false
30665 8598 Acked Read 2 Done Sent Done false
30666 8599 Acked Write 6 Done Sent Done false
30667 8600 Acked Write 1 Done Sent Done false
30668 8601 Acked Write 9 Done Sent Done false
30669 8602 Acked Read 9 Done Sent Done false
30670 8603 Acked Read 2 Done Sent Done false
30671 8604 Acked Write 7 Done Sent Done false
30672 8605 Acked Read 4 Done Sent Done false
30673 8606 Acked Flush 0 Done Sent Done false
30674 8607 Acked Read 10 Done Sent Done false
30675 8608 Acked Write 9 Done Sent Done false
30676 8609 Acked Write 8 Done Sent Done false
30677 8610 Acked Write 10 Done Sent Done false
30678 8611 Acked Read 9 Done Sent Done false
30679 8612 Acked Write 3 Done Sent Done false
30680 8613 Acked Write 9 Done Sent Done false
30681 8614 Acked Write 1 Done Sent Done false
30682 8615 Acked Read 10 Done Sent Done false
30683 8616 Acked Write 3 Done Sent Done false
30684 8617 Acked Flush 0 Done Sent Done false
30685 8618 Acked Read 2 Done Sent Done false
30686 8619 Acked Read 8 Done Sent Done false
30687 8620 Acked Flush 0 Done Sent Done false
30688 8621 Acked Flush 0 Done Sent Done false
30689 8622 Acked Write 1 Done Sent Done false
30690 8623 Acked Read 10 Done Sent Done false
30691 8624 Acked Read 3 Done Sent Done false
30692 8625 Acked Write 6 Done Sent Done false
30693 8626 Acked Read 8 Done Sent Done false
30694 8627 Acked Read 2 Done Sent Done false
30695 8628 Acked Write 9 Done Sent Done false
30696 8629 Acked Read 10 Done Sent Done false
30697 8630 Acked Write 7 Done Sent Done false
30698 8631 Acked Read 6 Done Sent Done false
30699 8632 Acked Write 2 Done Sent Done false
30700 8633 Acked Read 8 Done Sent Done false
30701 8634 Acked Read 3 Done Sent Done false
30702 8635 Acked Read 3 Done Sent Done false
30703 8636 Acked Flush 0 Done Sent Done false
30704 8637 Acked Write 3 Done Sent Done false
30705 8638 Acked Write 4 Done Sent Done false
30706 8639 Acked Read 9 Done Sent Done false
30707 8640 Acked Write 10 Done Sent Done false
30708 8641 Acked Write 4 Done Sent Done false
30709 8642 Acked Read 8 Done Sent Done false
30710 8643 Acked Read 1 Done Sent Done false
30711 8644 Acked Read 2 Done Sent Done false
30712 8645 Acked Write 6 Done Sent Done false
30713 8646 Acked Read 3 Done Sent Done false
30714 8647 Acked Write 4 Done Sent Done false
30715 8648 Acked Read 5 Done Sent Done false
30716 8649 Acked Read 2 Done Sent Done false
30717 8650 Acked Read 10 Done Sent Done false
30718 8651 Acked Read 5 Done Sent Done false
30719 8652 Acked Read 3 Done Sent Done false
30720 8653 Acked Flush 0 Done Sent Done false
30721 8654 Acked Write 9 Done Sent Done false
30722 8655 Acked Read 2 Done Sent Done false
30723 8656 Acked Write 8 Done Sent Done false
30724 8657 Acked Read 1 Done Sent Done false
30725 8658 Acked Read 4 Done Sent Done false
30726 8659 Acked Read 3 Done Sent Done false
30727 8660 Acked Read 3 Done Sent Done false
30728 8661 Acked Read 2 Done Sent Done false
30729 8662 Acked Write 4 Done Sent Done false
30730 8663 Acked Write 1 Done Sent Done false
30731 8664 Acked Write 9 Done Sent Done false
30732 8665 Acked Read 10 Done Sent Done false
30733 8666 Acked Write 1 Done Sent Done false
30734 8667 Acked Flush 0 Done Sent Done false
30735 8668 Acked Write 7 Done Sent Done false
30736 8669 Acked Read 7 Done Sent Done false
30737 8670 Acked Read 10 Done Sent Done false
30738 8671 Acked Write 7 Done Sent Done false
30739 8672 Acked Read 2 Done Sent Done false
30740 8673 Acked Write 3 Done Sent Done false
30741 8674 Acked Write 5 Done Sent Done false
30742 8675 Acked Read 9 Done Sent Done false
30743 8676 Acked Read 3 Done Sent Done false
30744 8677 Acked Read 6 Done Sent Done false
30745 8678 Acked Read 9 Done Sent Done false
30746 8679 Acked Read 3 Done Sent Done false
30747 8680 Acked Read 2 Done Sent Done false
30748 8681 Acked Read 2 Done Sent Done false
30749 8682 Acked Read 6 Done Sent Done false
30750 8683 Acked Write 1 Done Sent Done false
30751 8684 Acked Read 8 Done Sent Done false
30752 8685 Acked Read 7 Done Sent Done false
30753 8686 Acked Read 4 Done Sent Done false
30754 8687 Acked Read 4 Done Sent Done false
30755 8688 Acked Flush 0 Done Sent Done false
30756 8689 Acked Write 3 Done Sent Done false
30757 8690 Acked Read 10 Done Sent Done false
30758 8691 Acked Write 6 Done Sent Done false
30759 8692 Acked Read 9 Done Sent Done false
30760 8693 Acked Read 10 Done Sent Done false
30761 8694 Acked Write 2 Done Sent Done false
30762 8695 Acked Write 6 Done Sent Done false
30763 8696 Acked Write 10 Done Sent Done false
30764 8697 Acked Write 7 Done Sent Done false
30765 8698 Acked Read 5 Done Sent Done false
30766 8699 Acked Write 8 Done Sent Done false
30767 8700 Acked Read 10 Done Sent Done false
30768 8701 Acked Write 8 Done Sent Done false
30769 8702 Acked Read 10 Done Sent Done false
30770 8703 Acked Write 7 Done Sent Done false
30771 8704 Acked Write 1 Done Sent Done false
30772 8705 Acked Write 8 Done Sent Done false
30773 8706 Acked Read 9 Done Sent Done false
30774 8707 Acked Read 1 Done Sent Done false
30775 8708 Acked Write 5 Done Sent Done false
30776 8709 Acked Write 4 Done Sent Done false
30777 8710 Acked Read 2 Done Sent Done false
30778 8711 Acked Write 9 Done Sent Done false
30779 8712 Acked Read 3 Done Sent Done false
30780 8713 Acked Read 3 Done Sent Done false
30781 8714 Acked Write 6 Done Sent Done false
30782 8715 Acked Write 2 Done Sent Done false
30783 8716 Acked Write 1 Done Sent Done false
30784 8717 Acked Read 3 Done Sent Done false
30785 8718 Acked Write 1 Done Sent Done false
30786 8719 Acked Read 4 Done Sent Done false
30787 8720 Acked Read 3 Done Sent Done false
30788 8721 Acked Write 2 Done Sent Done false
30789 8722 Acked Write 9 Done Sent Done false
30790 8723 Acked Read 6 Done Sent Done false
30791 8724 Acked Write 6 Done Sent Done false
30792 8725 Acked Read 9 Done Sent Done false
30793 8726 Acked Read 1 Done Sent Done false
30794 8727 Acked Write 5 Done Sent Done false
30795 8728 Acked Write 7 Done Sent Done false
30796 8729 Acked Flush 0 Done Sent Done false
30797 8730 Acked Write 5 Done Sent Done false
30798 8731 Acked Write 4 Done Sent Done false
30799 8732 Acked Read 9 Done Sent Done false
30800 8733 Acked Read 2 Done Sent Done false
30801 8734 Acked Read 6 Done Sent Done false
30802 8735 Acked Read 8 Done Sent Done false
30803 8736 Acked Read 8 Done Sent Done false
30804 8737 Acked Read 7 Done Sent Done false
30805 8738 Acked Write 1 Done Sent Done false
30806 8739 Acked Read 8 Done Sent Done false
30807 8740 Acked Read 9 Done Sent Done false
30808 8741 Acked Write 8 Done Sent Done false
30809 8742 Acked Read 1 Done Sent Done false
30810 8743 Acked Read 2 Done Sent Done false
30811 8744 Acked Write 1 Done Sent Done false
30812 8745 Acked Read 2 Done Sent Done false
30813 8746 Acked Read 9 Done Sent Done false
30814 8747 Acked Write 5 Done Sent Done false
30815 8748 Acked Read 8 Done Sent Done false
30816 8749 Acked Flush 0 Done Sent Done false
30817 8750 Acked Write 10 Done Sent Done false
30818 8751 Acked Read 9 Done Sent Done false
30819 8752 Acked Read 8 Done Sent Done false
30820 8753 Acked Read 6 Done Sent Done false
30821 8754 Acked Read 8 Done Sent Done false
30822 8755 Acked Read 9 Done Sent Done false
30823 8756 Acked Read 7 Done Sent Done false
30824 8757 Acked Read 9 Done Sent Done false
30825 8758 Acked Read 3 Done Sent Done false
30826 8759 Acked Write 5 Done Sent Done false
30827 8760 Acked Read 8 Done Sent Done false
30828 8761 Acked Flush 0 Done Sent Done false
30829 8762 Acked Read 8 Done Sent Done false
30830 8763 Acked Read 7 Done Sent Done false
30831 8764 Acked Write 9 Done Sent Done false
30832 8765 Acked Write 10 Done Sent Done false
30833 8766 Acked Flush 0 Done Sent Done false
30834 8767 Acked Read 7 Done Sent Done false
30835 8768 Acked Read 6 Done Sent Done false
30836 8769 Acked Write 4 Done Sent Done false
30837 8770 Acked Read 9 Done Sent Done false
30838 8771 Acked Read 6 Done Sent Done false
30839 8772 Acked Read 9 Done Sent Done false
30840 8773 Acked Read 8 Done Sent Done false
30841 8774 Acked Write 6 Done Sent Done false
30842 8775 Acked Read 4 Done Sent Done false
30843 8776 Acked Read 5 Done Sent Done false
30844 8777 Acked Write 8 Done Sent Done false
30845 8778 Acked Read 9 Done Sent Done false
30846 8779 Acked Read 10 Done Sent Done false
30847 8780 Acked Read 9 Done Sent Done false
30848 8781 Acked Read 3 Done Sent Done false
30849 8782 Acked Read 9 Done Sent Done false
30850 8783 Acked Write 2 Done Sent Done false
30851 8784 Acked Write 8 Done Sent Done false
30852 8785 Acked Read 2 Done Sent Done false
30853 8786 Acked Write 8 Done Sent Done false
30854 8787 Acked Write 4 Done Sent Done false
30855 8788 Acked Write 1 Done Sent Done false
30856 8789 Acked Write 9 Done Sent Done false
30857 8790 Acked Read 8 Done Sent Done false
30858 8791 Acked Read 5 Done Sent Done false
30859 8792 Acked Write 1 Done Sent Done false
30860 8793 Acked Read 2 Done Sent Done false
30861 8794 Acked Write 2 Done Sent Done false
30862 8795 Acked Read 10 Done Sent Done false
30863 8796 Acked Write 4 Done Sent Done false
30864 8797 Acked Write 3 Done Sent Done false
30865 8798 Acked Write 3 Done Sent Done false
30866 8799 Acked Read 7 Done Sent Done false
30867 8800 Acked Read 10 Done Sent Done false
30868 8801 Acked Read 10 Done Sent Done false
30869 8802 Acked Write 4 Done Sent Done false
30870 8803 Acked Write 2 Done Sent Done false
30871 8804 Acked Write 7 Done Sent Done false
30872 8805 Acked Write 10 Done Sent Done false
30873 8806 Acked Write 1 Done Sent Done false
30874 8807 Acked Read 5 Done Sent Done false
30875 8808 Acked Read 8 Done Sent Done false
30876 8809 Acked Read 1 Done Sent Done false
30877 8810 Acked Flush 0 Done Sent Done false
30878 8811 Acked Read 8 Done Sent Done false
30879 8812 Acked Write 2 Done Sent Done false
30880 8813 Acked Write 6 Done Sent Done false
30881 8814 Acked Flush 0 Done Sent Done false
30882 8815 Acked Flush 0 Done Sent Done false
30883 8816 Acked Read 8 Done Sent Done false
30884 8817 Acked Read 4 Done Sent Done false
30885 8818 Acked Flush 0 Done Sent Done false
30886 8819 Acked Write 2 Done Sent Done false
30887 8820 Acked Write 1 Done Sent Done false
30888 8821 Acked Read 6 Done Sent Done false
30889 8822 Acked Write 4 Done Sent Done false
30890 8823 Acked Read 4 Done Sent Done false
30891 8824 Acked Write 5 Done Sent Done false
30892 8825 Acked Read 4 Done Sent Done false
30893 8826 Acked Read 6 Done Sent Done false
30894 8827 Acked Write 7 Done Sent Done false
30895 8828 Acked Write 5 Done Sent Done false
30896 8829 Acked Flush 0 Done Sent Done false
30897 8830 Acked Write 3 Done Sent Done false
30898 8831 Acked Write 7 Done Sent Done false
30899 8832 Acked Write 7 Done Sent Done false
30900 8833 Acked Read 5 Done Sent Done false
30901 8834 Acked Write 7 Done Sent Done false
30902 8835 Acked Flush 0 Done Sent Done false
30903 8836 Acked Read 10 Done Sent Done false
30904 8837 Acked Write 8 Done Sent Done false
30905 8838 Acked Read 4 Done Sent Done false
30906 8839 Acked Write 2 Done Sent Done false
30907 8840 Acked Read 2 Done Sent Done false
30908 8841 Acked Read 9 Done Sent Done false
30909 8842 Acked Write 4 Done Sent Done false
30910 8843 Acked Read 4 Done Sent Done false
30911 8844 Acked Read 9 Done Sent Done false
30912 8845 Acked Read 2 Done Sent Done false
30913 8846 Acked Write 4 Done Sent Done false
30914 8847 Acked Read 2 Done Sent Done false
30915 8848 Acked Flush 0 Done Sent Done false
30916 STATES DS:0 DS:1 DS:2 TOTAL
30917 Sent 0 302 0 302
30918 Done 302 0 302 604
30919 Skipped 0 0 0 0
30920 Error 0 0 0 0
30921 Last Flush: 8848 8546 8848
30922 Downstairs last five completed: 8546 8545 8544 8543 8542
30923 Upstairs last five completed: 8848 8847 8846 8845 8844
30924 CLIENT: Up:0 ds:302 act:2
309252024-11-26T22:11:02.531ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
309262024-11-26T22:11:02.531ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:38742) = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
309272024-11-26T22:11:02.532ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
309282024-11-26T22:11:02.533ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
309292024-11-26T22:11:02.533ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
309302024-11-26T22:11:02.533ZINFOcrucible: send last flush ID to this DS: 8546 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
309312024-11-26T22:11:02.533ZINFOcrucible: Replied this last flush ID: 8546 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
309322024-11-26T22:11:02.533ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
309332024-11-26T22:11:02.533ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
309342024-11-26T22:11:02.533ZINFOcrucible: [1] client re-new 302 jobs since flush 8546 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
309352024-11-26T22:11:02.533ZINFOcrucible: [1] Replayed 302 jobs since flush: 8546 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
30936 ----------------------------------------------------------------
30937 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
30938 Downstairs last five completed: 8848 8847 8846 8845 8844
30939 Upstairs last five completed: 8848 8847 8846 8845 8844
30940 CLIENT: Up:0 ds:0 act:3
30941 Replay: All jobs finished
30942 Main task has work to do, go find it
30943 got dsc Stop(1)
30944 stop 1
30945 Nov 26 22:11:02.781 INFO request completed, latency_us: 370, response_code: 204, uri: /stop/cid/1[8820] Got stop action so:false kr:false
30946 , method: POST, req_id: 8c7592eb-6726-41ee-be90-8acafe151082, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
309472024-11-26T22:11:02.781ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
309482024-11-26T22:11:02.781ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
309492024-11-26T22:11:02.781ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
309502024-11-26T22:11:02.781ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
309512024-11-26T22:11:02.781ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
309522024-11-26T22:11:02.781ZINFOcrucible: client task is exiting = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
30953 Nov 26 22:11:02.781 INFO request completed, latency_us: 148, response_code: 200, uri: /state/cid/1, method: GET, req_id: 31695b9f-510e-448b-b4e2-505578c6fd2f, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
30954 [8820] Exited with: ExitStatus(unix_wait_status(9))
30955 [8820][1] reports Exit
30956 Nov 26 22:11:06.784 INFO request completed, latency_us: 224, response_code: 200, uri: /state/cid/1, method: GET, req_id: a8058474-2816-4794-8f5c-0e733b20e004, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
30957 001/300 Read block 319 len 40960
30958 002/300 Read block 3114 len 28672
30959 003/300 Read block 1445 len 28672
30960 004/300 Flush
30961 005/300 Flush
30962 006/300 Read block 1704 len 36864
30963 007/300 Write block 1719 len 24576 data: 5 5 6 5 6 6
30964 008/300 Write block 3868 len 32768 data: 5 4 5 3 5 8 9 9
30965 009/300 Flush
30966 010/300 Read block 1312 len 4096
30967 011/300 Write block 2615 len 32768 data: 5 4 2 3 4 4 4 4
30968 012/300 Read block 2970 len 40960
30969 013/300 Read block 239 len 32768
30970 014/300 Read block 3592 len 12288
30971 015/300 Write block 3629 len 24576 data: 6 5 7 7 6 3
30972 016/300 Flush
30973 017/300 Flush
30974 018/300 Flush
30975 019/300 Flush
30976 020/300 Read block 4065 len 20480
30977 021/300 Write block 1832 len 40960 data: 7 8 6 6 8 7 8 6 5 7
30978 022/300 Write block 3893 len 24576 data: 5 6 7 8 7 4
30979 023/300 Read block 1078 len 24576
30980 024/300 Read block 963 len 40960
30981 025/300 Flush
30982 026/300 Read block 691 len 20480
30983 027/300 Flush
30984 028/300 Write block 892 len 8192 data: 8 8
30985 029/300 Read block 314 len 32768
30986 030/300 Write block 427 len 32768 data: 4 5 6 5 4 5 6 7
30987 031/300 Read block 4029 len 40960
30988 032/300 Write block 1032 len 4096 data: 5
30989 033/300 Write block 1226 len 8192 data: 3 4
30990 034/300 Write block 1631 len 12288 data: 5 6 6
30991 035/300 Write block 3496 len 4096 data: 10
30992 036/300 Read block 3990 len 12288
30993 037/300 Read block 623 len 32768
30994 038/300 Read block 1700 len 28672
30995 039/300 Write block 3368 len 40960 data: 4 2 2 3 3 4 5 5 4 3
30996 040/300 Read block 1441 len 4096
30997 041/300 Read block 3018 len 40960
30998 042/300 Write block 470 len 40960 data: 4 5 5 4 5 6 5 5 4 6
30999 043/300 Flush
31000 044/300 Read block 2094 len 28672
31001 045/300 Write block 4265 len 28672 data: 6 5 6 6 6 7 6
31002 046/300 Read block 4481 len 28672
31003 047/300 Read block 613 len 4096
31004 048/300 Read block 1945 len 4096
31005 049/300 Read block 919 len 40960
31006 050/300 Flush
31007 051/300 Flush
31008 052/300 Flush
31009 053/300 Read block 1297 len 12288
31010 054/300 Write block 1298 len 24576 data: 5 4 4 4 5 6
31011 055/300 Write block 3803 len 8192 data: 5 4
31012 056/300 Flush
31013 057/300 Read block 3890 len 16384
31014 058/300 Write block 2290 len 28672 data: 5 5 4 3 3 3 4
31015 059/300 Write block 1193 len 20480 data: 8 9 9 10 11
31016 060/300 Read block 3210 len 8192
31017 061/300 Read block 1502 len 36864
31018 062/300 Write block 4795 len 28672 data: 4 3 4 3 4 3 6
31019 063/300 Read block 1580 len 36864
31020 064/300 Read block 551 len 28672
31021 065/300 Read block 4248 len 20480
31022 066/300 Flush
31023 067/300 Write block 1681 len 24576 data: 5 7 6 6 5 5
31024 068/300 Read block 3936 len 20480
31025 069/300 Read block 3387 len 12288
31026 070/300 Read block 2732 len 16384
31027 071/300 Write block 728 len 28672 data: 7 6 5 5 6 5 6
31028 072/300 Read block 4937 len 16384
31029 073/300 Flush
31030 074/300 Read block 4245 len 12288
31031 075/300 Read block 1295 len 20480
31032 076/300 Read block 779 len 36864
31033 077/300 Read block 4764 len 4096
31034 078/300 Flush
31035 079/300 Write block 4426 len 36864 data: 4 4 4 3 3 5 4 4 5
31036 080/300 Write block 72 len 20480 data: 2 2 4 5 6
31037 081/300 Read block 4101 len 32768
31038 082/300 Read block 901 len 20480
31039 083/300 Read block 1518 len 36864
31040 084/300 Write block 4051 len 12288 data: 3 3 4
31041 085/300 Write block 2530 len 20480 data: 5 4 3 4 4
31042 086/300 Read block 4530 len 4096
31043 087/300 Write block 3913 len 16384 data: 6 6 6 6
31044 088/300 Write block 2060 len 28672 data: 9 8 7 7 8 11 10
31045 089/300 Read block 606 len 40960
31046 090/300 Write block 3714 len 16384 data: 6 5 6 6
31047 091/300 Write block 102 len 12288 data: 7 6 8
31048 092/300 Write block 3587 len 16384 data: 7 5 5 6
31049 093/300 Read block 4946 len 40960
31050 094/300 Read block 2252 len 8192
31051 095/300 Write block 3541 len 36864 data: 7 7 5 5 4 5 4 3 3
31052 096/300 Flush
31053 097/300 Read block 4172 len 4096
31054 098/300 Read block 404 len 4096
31055 099/300 Write block 3574 len 28672 data: 8 7 4 3 2 3 3
31056 100/300 Read block 1960 len 32768
31057 101/300 Write block 3502 len 24576 data: 8 7 6 6 6 6
31058 102/300 Read block 1062 len 40960
31059 103/300 Read block 1369 len 28672
31060 104/300 Read block 3477 len 40960
31061 105/300 Write block 1399 len 4096 data: 5
31062 106/300 Write block 401 len 16384 data: 5 7 8 7
31063 107/300 Write block 4346 len 36864 data: 8 9 9 7 6 6 4 4 4
31064 108/300 Write block 4968 len 20480 data: 8 9 8 7 5
31065 109/300 Write block 90 len 16384 data: 7 7 7 8
31066 110/300 Read block 1110 len 12288
31067 111/300 Write block 4170 len 12288 data: 3 4 4
31068 112/300 Flush
31069 113/300 Write block 316 len 28672 data: 3 5 5 5 5 5 5
31070 114/300 Write block 3833 len 36864 data: 5 3 3 3 2 3 4 4 4
31071 115/300 Read block 1046 len 4096
31072 116/300 Write block 3258 len 36864 data: 8 9 8 7 6 6 5 5 5
31073 117/300 Read block 1753 len 4096
31074 118/300 Flush
31075 119/300 Read block 2733 len 20480
31076 120/300 Read block 2240 len 36864
31077 121/300 Write block 767 len 20480 data: 6 5 4 5 7
31078 122/300 Read block 2025 len 36864
31079 123/300 Write block 4648 len 28672 data: 5 5 6 5 4 7 6
31080 124/300 Write block 51 len 24576 data: 5 4 4 4 3 4
31081 125/300 Flush
31082 126/300 Write block 3053 len 12288 data: 4 6 5
31083 127/300 Write block 4710 len 8192 data: 7 8
31084 128/300 Read block 4816 len 24576
31085 129/300 Write block 2767 len 20480 data: 5 5 5 5 7
31086 130/300 Write block 1529 len 40960 data: 7 6 6 6 6 6 7 6 6 6
31087 131/300 Write block 348 len 40960 data: 6 8 7 8 8 8 9 7 7 5
31088 132/300 Write block 936 len 8192 data: 5 6
31089 133/300 Write block 2479 len 36864 data: 6 8 7 7 7 9 10 11 10
31090 134/300 Read block 3314 len 32768
31091 135/300 Read block 4639 len 12288
31092 136/300 Write block 1830 len 24576 data: 7 7 8 9 7 7
31093 137/300 Write block 4941 len 8192 data: 4 4
31094 138/300 Read block 745 len 36864
31095 139/300 Flush
31096 140/300 Read block 2240 len 28672
31097 141/300 Write block 1195 len 12288 data: 10 11 12
31098 142/300 Read block 2153 len 8192
31099 143/300 Write block 1500 len 8192 data: 8 7
31100 144/300 Write block 212 len 36864 data: 4 5 6 7 8 7 8 9 6
31101 145/300 Read block 3312 len 12288
31102 146/300 Write block 2946 len 32768 data: 5 5 3 3 3 5 6 6
31103 147/300 Read block 1575 len 20480
31104 148/300 Flush
31105 149/300 Write block 3775 len 36864 data: 5 4 6 7 7 8 8 9 8
31106 150/300 Write block 869 len 32768 data: 5 7 4 5 5 7 8 7
31107 151/300 Write block 2193 len 32768 data: 5 5 5 5 3 4 4 4
31108 152/300 Write block 2084 len 20480 data: 7 7 5 7 6
31109 153/300 Read block 428 len 28672
31110 154/300 Flush
31111 155/300 Read block 1508 len 36864
31112 156/300 Write block 3180 len 16384 data: 6 5 6 3
31113 157/300 Write block 484 len 20480 data: 4 5 5 4 5
31114 158/300 Write block 1882 len 40960 data: 4 4 4 5 4 5 4 4 5 5
31115 159/300 Read block 4258 len 16384
31116 160/300 Write block 1299 len 28672 data: 5 5 5 6 7 7 7
31117 161/300 Flush
31118 162/300 Write block 1259 len 16384 data: 6 5 5 5
31119 163/300 Write block 1471 len 32768 data: 4 5 6 6 6 6 8 9
31120 164/300 Write block 694 len 16384 data: 3 3 3 3
31121 165/300 Read block 808 len 28672
31122 166/300 Write block 1184 len 4096 data: 4
31123 167/300 Flush
31124 168/300 Write block 4294 len 4096 data: 5
31125 169/300 Flush
31126 170/300 Write block 4120 len 36864 data: 2 3 3 4 4 4 4 5 5
31127 171/300 Write block 2952 len 24576 data: 7 7 5 4 5 5
31128 172/300 Read block 3629 len 32768
31129 173/300 Read block 3346 len 20480
31130 174/300 Read block 3704 len 28672
31131 175/300 Read block 512 len 16384
31132 176/300 Write block 4278 len 12288 data: 6 6 7
31133 177/300 Write block 1439 len 8192 data: 4 4
31134 178/300 Read block 1818 len 28672
31135 179/300 Write block 1926 len 4096 data: 7
31136 180/300 Write block 1265 len 32768 data: 6 6 6 7 7 7 5 6
31137 181/300 Read block 31 len 28672
31138 182/300 Write block 1517 len 28672 data: 3 3 3 4 5 6 7
31139 183/300 Write block 2965 len 8192 data: 5 5
31140 184/300 Read block 1407 len 8192
31141 185/300 Read block 308 len 24576
31142 186/300 Read block 2231 len 20480
31143 187/300 Read block 3866 len 32768
31144 188/300 Read block 17 len 32768
31145 189/300 Read block 732 len 28672
31146 190/300 Read block 2286 len 40960
31147 191/300 Read block 1862 len 40960
31148 192/300 Read block 4086 len 4096
31149 193/300 Read block 4426 len 16384
31150 194/300 Flush
31151 195/300 Read block 329 len 12288
31152 196/300 Read block 4652 len 12288
31153 197/300 Read block 2329 len 20480
31154 198/300 Read block 4136 len 36864
31155 199/300 Read block 2308 len 36864
31156 200/300 Write block 1441 len 36864 data: 4 6 5 6 6 8 6 6 4
31157 201/300 Read block 2242 len 8192
31158 202/300 Write block 3112 len 16384 data: 6 6 8 4
31159 203/300 Write block 4593 len 12288 data: 6 6 6
31160 204/300 Write block 1585 len 32768 data: 3 3 4 4 4 6 7 8
31161 205/300 Write block 1509 len 8192 data: 4 3
31162 206/300 Read block 4569 len 32768
31163 207/300 Read block 3044 len 36864
31164 208/300 Write block 1782 len 16384 data: 6 6 4 5
31165 209/300 Read block 1150 len 28672
31166 210/300 Read block 4277 len 8192
31167 211/300 Read block 1220 len 16384
31168 212/300 Read block 3000 len 24576
31169 213/300 Write block 4941 len 4096 data: 5
31170 214/300 Read block 3027 len 36864
31171 215/300 Read block 1376 len 36864
31172 216/300 Read block 4905 len 36864
31173 217/300 Read block 1827 len 4096
31174 218/300 Write block 1027 len 20480 data: 6 4 5 4 5
31175 219/300 Flush
31176 220/300 Read block 342 len 8192
31177 221/300 Read block 223 len 20480
31178 222/300 Read block 363 len 32768
31179 223/300 Flush
31180 224/300 Read block 2864 len 32768
31181 225/300 Read block 2270 len 28672
31182 226/300 Read block 2216 len 8192
31183 227/300 Write block 2999 len 20480 data: 7 6 5 5 5
31184 228/300 Read block 219 len 36864
31185 229/300 Read block 135 len 16384
31186 230/300 Write block 4876 len 24576 data: 8 7 7 6 6 4
31187 231/300 Read block 3121 len 24576
31188 232/300 Read block 4055 len 12288
31189 233/300 Read block 3307 len 20480
31190 234/300 Read block 12 len 20480
31191 235/300 Read block 1809 len 32768
31192 236/300 Read block 2769 len 32768
31193 237/300 Read block 45 len 36864
31194 238/300 Write block 2067 len 40960 data: 8 9 7 8 6 5 4 3 3 5
31195 239/300 Write block 1711 len 12288 data: 5 5 4
31196 240/300 Read block 3865 len 28672
31197 241/300 Read block 3599 len 12288
31198 242/300 Read block 2924 len 36864
31199 243/300 Write block 3557 len 36864 data: 6 5 5 6 5 5 5 8 8
31200 244/300 Read block 1847 len 4096
31201 245/300 Read block 2594 len 20480
31202 246/300 Write block 4047 len 8192 data: 8 5
31203 247/300 Read block 4039 len 8192
31204 248/300 Read block 469 len 8192
31205 249/300 Read block 547 len 28672
31206 250/300 Read block 513 len 32768
31207 251/300 Read block 3441 len 4096
31208 252/300 Read block 1510 len 24576
31209 253/300 Write block 4015 len 28672 data: 8 7 7 7 5 4 4
31210 254/300 Write block 3958 len 20480 data: 4 5 5 4 3
31211 255/300 Write block 3320 len 32768 data: 7 7 4 3 3 4 4 4
31212 256/300 Write block 527 len 24576 data: 5 5 5 4 4 3
31213 257/300 Read block 2801 len 12288
31214 258/300 Flush
31215 259/300 Write block 2800 len 20480 data: 4 4 5 5 5
31216 260/300 Write block 1618 len 24576 data: 5 6 6 7 7 7
31217 261/300 Write block 2487 len 20480 data: 11 10 8 8 8
31218 262/300 Read block 4915 len 16384
31219 263/300 Write block 3618 len 8192 data: 5 5
31220 264/300 Read block 2692 len 32768
31221 265/300 Write block 62 len 12288 data: 5 5 5
31222 266/300 Write block 4092 len 20480 data: 6 5 5 4 4
31223 267/300 Write block 4340 len 12288 data: 4 5 6
31224 268/300 Read block 4819 len 8192
31225 269/300 Read block 2961 len 4096
31226 270/300 Read block 4209 len 40960
31227 271/300 Read block 4443 len 16384
31228 272/300 Write block 3228 len 12288 data: 12 12 13
31229 273/300 Flush
31230 274/300 Write block 1874 len 8192 data: 5 6
31231 275/300 Write block 2239 len 32768 data: 5 5 3 4 4 4 3 6
31232 276/300 Read block 869 len 32768
31233 277/300 Flush
31234 278/300 Write block 3831 len 32768 data: 8 6 6 4 4 4 3 4
31235 279/300 Read block 97 len 36864
31236 280/300 Read block 365 len 12288
31237 281/300 Flush
31238 282/300 Write block 4811 len 28672 data: 6 6 6 5 5 5 5
31239 283/300 Write block 3425 len 4096 data: 5
31240 284/300 Read block 686 len 36864
31241 285/300 Read block 1558 len 24576
31242 286/300 Read block 541 len 32768
31243 287/300 Read block 2039 len 24576
31244 288/300 Write block 208 len 28672 data: 3 2 3 3 5 6 7
31245 289/300 Write block 867 len 24576 data: 5 5 6 8 5 6
31246 290/300 Flush
31247 291/300 Read block 1582 len 4096
31248 292/300 Read block 3456 len 28672
31249 293/300 Read block 4113 len 36864
31250 294/300 Write block 2603 len 24576 data: 9 8 7 8 9 8
31251 295/300 Write block 59 len 32768 data: 4 5 6 6 6 6 5 4
31252 296/300 Read block 1627 len 28672
31253 297/300 Write block 1455 len 16384 data: 6 7 7 6
31254 298/300 Write block 539 len 12288 data: 9 9 9
31255 299/300 Read block 4140 len 28672
31256 300/300 Read block 2200 len 4096
31257 301/300 Read block 3543 len 32768
31258 Main task has work to do, go find it
31259 got dsc Start(1)
31260 start 1
31261 Nov 26 22:11:07.024 INFO request completed, latency_us: 214, response_code: 204, uri: /start/cid/1, method: POST, req_id: 382000c2-96a4-42b6-937f-cc1ae885126a, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
31262 Replay: started 1, returned:Ok(())
31263 ----------------------------------------------------------------
31264 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
31265 States: Active Offline Active
31266 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
31267 8849 Acked Read 10 Done Sent Done false
31268 8850 Acked Read 7 Done Sent Done false
31269 8851 Acked Read 7 Done Sent Done false
31270 8852 Acked Flush 0 Done Sent Done false
31271 8853 Acked Flush 0 Done Sent Done false
31272 8854 Acked Read 9 Done Sent Done false
31273 8855 Acked Write 6 Done Sent Done false
31274 8856 Acked Write 8 Done Sent Done false
31275 8857 Acked Flush 0 Done Sent Done false
31276 8858 Acked Read 1 Done Sent Done false
31277 8859 Acked Write 8 Done Sent Done false
31278 8860 Acked Read 10 Done Sent Done false
31279 8861 Acked Read 8 Done Sent Done false
31280 8862 Acked Read 3 Done Sent Done false
31281 8863 Acked Write 6 Done Sent Done false
31282 8864 Acked Flush 0 Done Sent Done false
31283 8865 Acked Flush 0 Done Sent Done false
31284 8866 Acked Flush 0 Done Sent Done false
31285 8867 Acked Flush 0 Done Sent Done false
31286 8868 Acked Read 5 Done Sent Done false
31287 8869 Acked Write 10 Done Sent Done false
31288 8870 Acked Write 6 Done Sent Done false
31289 8871 Acked Read 6 Done Sent Done false
31290 8872 Acked Read 10 Done Sent Done false
31291 8873 Acked Flush 0 Done Sent Done false
31292 8874 Acked Read 5 Done Sent Done false
31293 8875 Acked Flush 0 Done Sent Done false
31294 8876 Acked Write 2 Done Sent Done false
31295 8877 Acked Read 8 Done Sent Done false
31296 8878 Acked Write 8 Done Sent Done false
31297 8879 Acked Read 10 Done Sent Done false
31298 8880 Acked Write 1 Done Sent Done false
31299 8881 Acked Write 2 Done Sent Done false
31300 8882 Acked Write 3 Done Sent Done false
31301 8883 Acked Write 1 Done Sent Done false
31302 8884 Acked Read 3 Done Sent Done false
31303 8885 Acked Read 8 Done Sent Done false
31304 8886 Acked Read 7 Done Sent Done false
31305 8887 Acked Write 10 Done Sent Done false
31306 8888 Acked Read 1 Done Sent Done false
31307 8889 Acked Read 10 Done Sent Done false
31308 8890 Acked Write 10 Done Sent Done false
31309 8891 Acked Flush 0 Done Sent Done false
31310 8892 Acked Read 7 Done Sent Done false
31311 8893 Acked Write 7 Done Sent Done false
31312 8894 Acked Read 7 Done Sent Done false
31313 8895 Acked Read 1 Done Sent Done false
31314 8896 Acked Read 1 Done Sent Done false
31315 8897 Acked Read 10 Done Sent Done false
31316 8898 Acked Flush 0 Done Sent Done false
31317 8899 Acked Flush 0 Done Sent Done false
31318 8900 Acked Flush 0 Done Sent Done false
31319 8901 Acked Read 3 Done Sent Done false
31320 8902 Acked Write 6 Done Sent Done false
31321 8903 Acked Write 2 Done Sent Done false
31322 8904 Acked Flush 0 Done Sent Done false
31323 8905 Acked Read 4 Done Sent Done false
31324 8906 Acked Write 7 Done Sent Done false
31325 8907 Acked Write 5 Done Sent Done false
31326 8908 Acked Read 2 Done Sent Done false
31327 8909 Acked Read 9 Done Sent Done false
31328 8910 Acked Write 7 Done Sent Done false
31329 8911 Acked Read 9 Done Sent Done false
31330 8912 Acked Read 7 Done Sent Done false
31331 8913 Acked Read 5 Done Sent Done false
31332 8914 Acked Flush 0 Done Sent Done false
31333 8915 Acked Write 6 Done Sent Done false
31334 8916 Acked Read 5 Done Sent Done false
31335 8917 Acked Read 3 Done Sent Done false
31336 8918 Acked Read 4 Done Sent Done false
31337 8919 Acked Write 7 Done Sent Done false
31338 8920 Acked Read 4 Done Sent Done false
31339 8921 Acked Flush 0 Done Sent Done false
31340 8922 Acked Read 3 Done Sent Done false
31341 8923 Acked Read 5 Done Sent Done false
31342 8924 Acked Read 9 Done Sent Done false
31343 8925 Acked Read 1 Done Sent Done false
31344 8926 Acked Flush 0 Done Sent Done false
31345 8927 Acked Write 9 Done Sent Done false
31346 8928 Acked Write 5 Done Sent Done false
31347 8929 Acked Read 8 Done Sent Done false
31348 8930 Acked Read 5 Done Sent Done false
31349 8931 Acked Read 9 Done Sent Done false
31350 8932 Acked Write 3 Done Sent Done false
31351 8933 Acked Write 5 Done Sent Done false
31352 8934 Acked Read 1 Done Sent Done false
31353 8935 Acked Write 4 Done Sent Done false
31354 8936 Acked Write 7 Done Sent Done false
31355 8937 Acked Read 10 Done Sent Done false
31356 8938 Acked Write 4 Done Sent Done false
31357 8939 Acked Write 3 Done Sent Done false
31358 8940 Acked Write 4 Done Sent Done false
31359 8941 Acked Read 10 Done Sent Done false
31360 8942 Acked Read 2 Done Sent Done false
31361 8943 Acked Write 9 Done Sent Done false
31362 8944 Acked Flush 0 Done Sent Done false
31363 8945 Acked Read 1 Done Sent Done false
31364 8946 Acked Read 1 Done Sent Done false
31365 8947 Acked Write 7 Done Sent Done false
31366 8948 Acked Read 8 Done Sent Done false
31367 8949 Acked Write 6 Done Sent Done false
31368 8950 Acked Read 10 Done Sent Done false
31369 8951 Acked Read 7 Done Sent Done false
31370 8952 Acked Read 10 Done Sent Done false
31371 8953 Acked Write 1 Done Sent Done false
31372 8954 Acked Write 4 Done Sent Done false
31373 8955 Acked Write 9 Done Sent Done false
31374 8956 Acked Write 5 Done Sent Done false
31375 8957 Acked Write 4 Done Sent Done false
31376 8958 Acked Read 3 Done Sent Done false
31377 8959 Acked Write 3 Done Sent Done false
31378 8960 Acked Flush 0 Done Sent Done false
31379 8961 Acked Write 7 Done Sent Done false
31380 8962 Acked Write 9 Done Sent Done false
31381 8963 Acked Read 1 Done Sent Done false
31382 8964 Acked Write 9 Done Sent Done false
31383 8965 Acked Read 1 Done Sent Done false
31384 8966 Acked Flush 0 Done Sent Done false
31385 8967 Acked Read 5 Done Sent Done false
31386 8968 Acked Read 9 Done Sent Done false
31387 8969 Acked Write 5 Done Sent Done false
31388 8970 Acked Read 9 Done Sent Done false
31389 8971 Acked Write 7 Done Sent Done false
31390 8972 Acked Write 6 Done Sent Done false
31391 8973 Acked Flush 0 Done Sent Done false
31392 8974 Acked Write 3 Done Sent Done false
31393 8975 Acked Write 2 Done Sent Done false
31394 8976 Acked Read 6 Done Sent Done false
31395 8977 Acked Write 5 Done Sent Done false
31396 8978 Acked Write 10 Done Sent Done false
31397 8979 Acked Write 10 Done Sent Done false
31398 8980 Acked Write 2 Done Sent Done false
31399 8981 Acked Write 9 Done Sent Done false
31400 8982 Acked Read 8 Done Sent Done false
31401 8983 Acked Read 3 Done Sent Done false
31402 8984 Acked Write 6 Done Sent Done false
31403 8985 Acked Write 2 Done Sent Done false
31404 8986 Acked Read 9 Done Sent Done false
31405 8987 Acked Flush 0 Done Sent Done false
31406 8988 Acked Read 7 Done Sent Done false
31407 8989 Acked Write 3 Done Sent Done false
31408 8990 Acked Read 2 Done Sent Done false
31409 8991 Acked Write 2 Done Sent Done false
31410 8992 Acked Write 9 Done Sent Done false
31411 8993 Acked Read 3 Done Sent Done false
31412 8994 Acked Write 8 Done Sent Done false
31413 8995 Acked Read 5 Done Sent Done false
31414 8996 Acked Flush 0 Done Sent Done false
31415 8997 Acked Write 9 Done Sent Done false
31416 8998 Acked Write 8 Done Sent Done false
31417 8999 Acked Write 8 Done Sent Done false
31418 9000 Acked Write 5 Done Sent Done false
31419 9001 Acked Read 7 Done Sent Done false
31420 9002 Acked Flush 0 Done Sent Done false
31421 9003 Acked Read 9 Done Sent Done false
31422 9004 Acked Write 4 Done Sent Done false
31423 9005 Acked Write 5 Done Sent Done false
31424 9006 Acked Write 10 Done Sent Done false
31425 9007 Acked Read 4 Done Sent Done false
31426 9008 Acked Write 7 Done Sent Done false
31427 9009 Acked Flush 0 Done Sent Done false
31428 9010 Acked Write 4 Done Sent Done false
31429 9011 Acked Write 8 Done Sent Done false
31430 9012 Acked Write 4 Done Sent Done false
31431 9013 Acked Read 7 Done Sent Done false
31432 9014 Acked Write 1 Done Sent Done false
31433 9015 Acked Flush 0 Done Sent Done false
31434 9016 Acked Write 1 Done Sent Done false
31435 9017 Acked Flush 0 Done Sent Done false
31436 9018 Acked Write 9 Done Sent Done false
31437 9019 Acked Write 6 Done Sent Done false
31438 9020 Acked Read 8 Done Sent Done false
31439 9021 Acked Read 5 Done Sent Done false
31440 9022 Acked Read 7 Done Sent Done false
31441 9023 Acked Read 4 Done Sent Done false
31442 9024 Acked Write 3 Done Sent Done false
31443 9025 Acked Write 2 Done Sent Done false
31444 9026 Acked Read 7 Done Sent Done false
31445 9027 Acked Write 1 Done Sent Done false
31446 9028 Acked Write 8 Done Sent Done false
31447 9029 Acked Read 7 Done Sent Done false
31448 9030 Acked Write 7 Done Sent Done false
31449 9031 Acked Write 2 Done Sent Done false
31450 9032 Acked Read 2 Done Sent Done false
31451 9033 Acked Read 6 Done Sent Done false
31452 9034 Acked Read 5 Done Sent Done false
31453 9035 Acked Read 8 Done Sent Done false
31454 9036 Acked Read 8 Done Sent Done false
31455 9037 Acked Read 7 Done Sent Done false
31456 9038 Acked Read 10 Done Sent Done false
31457 9039 Acked Read 10 Done Sent Done false
31458 9040 Acked Read 1 Done Sent Done false
31459 9041 Acked Read 4 Done Sent Done false
31460 9042 Acked Flush 0 Done Sent Done false
31461 9043 Acked Read 3 Done Sent Done false
31462 9044 Acked Read 3 Done Sent Done false
31463 9045 Acked Read 5 Done Sent Done false
31464 9046 Acked Read 9 Done Sent Done false
31465 9047 Acked Read 9 Done Sent Done false
31466 9048 Acked Write 9 Done Sent Done false
31467 9049 Acked Read 2 Done Sent Done false
31468 9050 Acked Write 4 Done Sent Done false
31469 9051 Acked Write 3 Done Sent Done false
31470 9052 Acked Write 8 Done Sent Done false
31471 9053 Acked Write 2 Done Sent Done false
31472 9054 Acked Read 8 Done Sent Done false
31473 9055 Acked Read 9 Done Sent Done false
31474 9056 Acked Write 4 Done Sent Done false
31475 9057 Acked Read 7 Done Sent Done false
31476 9058 Acked Read 2 Done Sent Done false
31477 9059 Acked Read 4 Done Sent Done false
31478 9060 Acked Read 6 Done Sent Done false
31479 9061 Acked Write 1 Done Sent Done false
31480 9062 Acked Read 9 Done Sent Done false
31481 9063 Acked Read 9 Done Sent Done false
31482 9064 Acked Read 9 Done Sent Done false
31483 9065 Acked Read 1 Done Sent Done false
31484 9066 Acked Write 5 Done Sent Done false
31485 9067 Acked Flush 0 Done Sent Done false
31486 9068 Acked Read 2 Done Sent Done false
31487 9069 Acked Read 5 Done Sent Done false
31488 9070 Acked Read 8 Done Sent Done false
31489 9071 Acked Flush 0 Done Sent Done false
31490 9072 Acked Read 8 Done Sent Done false
31491 9073 Acked Read 7 Done Sent Done false
31492 9074 Acked Read 2 Done Sent Done false
31493 9075 Acked Write 5 Done Sent Done false
31494 9076 Acked Read 9 Done Sent Done false
31495 9077 Acked Read 4 Done Sent Done false
31496 9078 Acked Write 6 Done Sent Done false
31497 9079 Acked Read 6 Done Sent Done false
31498 9080 Acked Read 3 Done Sent Done false
31499 9081 Acked Read 5 Done Sent Done false
31500 9082 Acked Read 5 Done Sent Done false
31501 9083 Acked Read 8 Done Sent Done false
31502 9084 Acked Read 8 Done Sent Done false
31503 9085 Acked Read 9 Done Sent Done false
31504 9086 Acked Write 10 Done Sent Done false
31505 9087 Acked Write 3 Done Sent Done false
31506 9088 Acked Read 7 Done Sent Done false
31507 9089 Acked Read 3 Done Sent Done false
31508 9090 Acked Read 9 Done Sent Done false
31509 9091 Acked Write 9 Done Sent Done false
31510 9092 Acked Read 1 Done Sent Done false
31511 9093 Acked Read 5 Done Sent Done false
31512 9094 Acked Write 2 Done Sent Done false
31513 9095 Acked Read 2 Done Sent Done false
31514 9096 Acked Read 2 Done Sent Done false
31515 9097 Acked Read 7 Done Sent Done false
31516 9098 Acked Read 8 Done Sent Done false
31517 9099 Acked Read 1 Done Sent Done false
31518 9100 Acked Read 6 Done Sent Done false
31519 9101 Acked Write 7 Done Sent Done false
31520 9102 Acked Write 5 Done Sent Done false
31521 9103 Acked Write 8 Done Sent Done false
31522 9104 Acked Write 6 Done Sent Done false
31523 9105 Acked Read 3 Done Sent Done false
31524 9106 Acked Flush 0 Done Sent Done false
31525 9107 Acked Write 5 Done Sent Done false
31526 9108 Acked Write 6 Done Sent Done false
31527 9109 Acked Write 5 Done Sent Done false
31528 9110 Acked Read 4 Done Sent Done false
31529 9111 Acked Write 2 Done Sent Done false
31530 9112 Acked Read 8 Done Sent Done false
31531 9113 Acked Write 3 Done Sent Done false
31532 9114 Acked Write 5 Done Sent Done false
31533 9115 Acked Write 3 Done Sent Done false
31534 9116 Acked Read 2 Done Sent Done false
31535 9117 Acked Read 1 Done Sent Done false
31536 9118 Acked Read 10 Done Sent Done false
31537 9119 Acked Read 4 Done Sent Done false
31538 9120 Acked Write 3 Done Sent Done false
31539 9121 Acked Flush 0 Done Sent Done false
31540 9122 Acked Write 2 Done Sent Done false
31541 9123 Acked Write 8 Done Sent Done false
31542 9124 Acked Read 8 Done Sent Done false
31543 9125 Acked Flush 0 Done Sent Done false
31544 9126 Acked Write 8 Done Sent Done false
31545 9127 Acked Read 9 Done Sent Done false
31546 9128 Acked Read 3 Done Sent Done false
31547 9129 Acked Flush 0 Done Sent Done false
31548 9130 Acked Write 7 Done Sent Done false
31549 9131 Acked Write 1 Done Sent Done false
31550 9132 Acked Read 9 Done Sent Done false
31551 9133 Acked Read 6 Done Sent Done false
31552 9134 Acked Read 8 Done Sent Done false
31553 9135 Acked Read 6 Done Sent Done false
31554 9136 Acked Write 7 Done Sent Done false
31555 9137 Acked Write 6 Done Sent Done false
31556 9138 Acked Flush 0 Done Sent Done false
31557 9139 Acked Read 1 Done Sent Done false
31558 9140 Acked Read 7 Done Sent Done false
31559 9141 Acked Read 9 Done Sent Done false
31560 9142 Acked Write 6 Done Sent Done false
31561 9143 Acked Write 8 Done Sent Done false
31562 9144 Acked Read 7 Done Sent Done false
31563 9145 Acked Write 4 Done Sent Done false
31564 9146 Acked Write 3 Done Sent Done false
31565 9147 Acked Read 7 Done Sent Done false
31566 9148 Acked Read 1 Done Sent Done false
31567 9149 Acked Read 8 Done Sent Done false
31568 STATES DS:0 DS:1 DS:2 TOTAL
31569 Sent 0 301 0 301
31570 Done 301 0 301 602
31571 Skipped 0 0 0 0
31572 Error 0 0 0 0
31573 Last Flush: 9138 8848 9138
31574 Downstairs last five completed: 8848 8847 8846 8845 8844
31575 Upstairs last five completed: 9149 9148 9147 9146 9145
31576 CLIENT: Up:0 ds:301 act:2
31577 [8820] got start action
31578 [8820] I am going to restart
31579 Starting downstairs at port 8820
31580 Make output file at "/tmp/dsc/downstairs-8820.txt"
31581 [8820][1] reports Starting
31582 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1230)
31583 [8820][1] reports Running
31584 ----------------------------------------------------------------
31585 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
31586 States: Active Offline Active
31587 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
31588 8849 Acked Read 10 Done Sent Done false
31589 8850 Acked Read 7 Done Sent Done false
31590 8851 Acked Read 7 Done Sent Done false
31591 8852 Acked Flush 0 Done Sent Done false
31592 8853 Acked Flush 0 Done Sent Done false
31593 8854 Acked Read 9 Done Sent Done false
31594 8855 Acked Write 6 Done Sent Done false
31595 8856 Acked Write 8 Done Sent Done false
31596 8857 Acked Flush 0 Done Sent Done false
31597 8858 Acked Read 1 Done Sent Done false
31598 8859 Acked Write 8 Done Sent Done false
31599 8860 Acked Read 10 Done Sent Done false
31600 8861 Acked Read 8 Done Sent Done false
31601 8862 Acked Read 3 Done Sent Done false
31602 8863 Acked Write 6 Done Sent Done false
31603 8864 Acked Flush 0 Done Sent Done false
31604 8865 Acked Flush 0 Done Sent Done false
31605 8866 Acked Flush 0 Done Sent Done false
31606 8867 Acked Flush 0 Done Sent Done false
31607 8868 Acked Read 5 Done Sent Done false
31608 8869 Acked Write 10 Done Sent Done false
31609 8870 Acked Write 6 Done Sent Done false
31610 8871 Acked Read 6 Done Sent Done false
31611 8872 Acked Read 10 Done Sent Done false
31612 8873 Acked Flush 0 Done Sent Done false
31613 8874 Acked Read 5 Done Sent Done false
31614 8875 Acked Flush 0 Done Sent Done false
31615 8876 Acked Write 2 Done Sent Done false
31616 8877 Acked Read 8 Done Sent Done false
31617 8878 Acked Write 8 Done Sent Done false
31618 8879 Acked Read 10 Done Sent Done false
31619 8880 Acked Write 1 Done Sent Done false
31620 8881 Acked Write 2 Done Sent Done false
31621 8882 Acked Write 3 Done Sent Done false
31622 8883 Acked Write 1 Done Sent Done false
31623 8884 Acked Read 3 Done Sent Done false
31624 8885 Acked Read 8 Done Sent Done false
31625 8886 Acked Read 7 Done Sent Done false
31626 8887 Acked Write 10 Done Sent Done false
31627 8888 Acked Read 1 Done Sent Done false
31628 8889 Acked Read 10 Done Sent Done false
31629 8890 Acked Write 10 Done Sent Done false
31630 8891 Acked Flush 0 Done Sent Done false
31631 8892 Acked Read 7 Done Sent Done false
31632 8893 Acked Write 7 Done Sent Done false
31633 8894 Acked Read 7 Done Sent Done false
31634 8895 Acked Read 1 Done Sent Done false
31635 8896 Acked Read 1 Done Sent Done false
31636 8897 Acked Read 10 Done Sent Done false
31637 8898 Acked Flush 0 Done Sent Done false
31638 8899 Acked Flush 0 Done Sent Done false
31639 8900 Acked Flush 0 Done Sent Done false
31640 8901 Acked Read 3 Done Sent Done false
31641 8902 Acked Write 6 Done Sent Done false
31642 8903 Acked Write 2 Done Sent Done false
31643 8904 Acked Flush 0 Done Sent Done false
31644 8905 Acked Read 4 Done Sent Done false
31645 8906 Acked Write 7 Done Sent Done false
31646 8907 Acked Write 5 Done Sent Done false
31647 8908 Acked Read 2 Done Sent Done false
31648 8909 Acked Read 9 Done Sent Done false
31649 8910 Acked Write 7 Done Sent Done false
31650 8911 Acked Read 9 Done Sent Done false
31651 8912 Acked Read 7 Done Sent Done false
31652 8913 Acked Read 5 Done Sent Done false
31653 8914 Acked Flush 0 Done Sent Done false
31654 8915 Acked Write 6 Done Sent Done false
31655 8916 Acked Read 5 Done Sent Done false
31656 8917 Acked Read 3 Done Sent Done false
31657 8918 Acked Read 4 Done Sent Done false
31658 8919 Acked Write 7 Done Sent Done false
31659 8920 Acked Read 4 Done Sent Done false
31660 8921 Acked Flush 0 Done Sent Done false
31661 8922 Acked Read 3 Done Sent Done false
31662 8923 Acked Read 5 Done Sent Done false
31663 8924 Acked Read 9 Done Sent Done false
31664 8925 Acked Read 1 Done Sent Done false
31665 8926 Acked Flush 0 Done Sent Done false
31666 8927 Acked Write 9 Done Sent Done false
31667 8928 Acked Write 5 Done Sent Done false
31668 8929 Acked Read 8 Done Sent Done false
31669 8930 Acked Read 5 Done Sent Done false
31670 8931 Acked Read 9 Done Sent Done false
31671 8932 Acked Write 3 Done Sent Done false
31672 8933 Acked Write 5 Done Sent Done false
31673 8934 Acked Read 1 Done Sent Done false
31674 8935 Acked Write 4 Done Sent Done false
31675 8936 Acked Write 7 Done Sent Done false
31676 8937 Acked Read 10 Done Sent Done false
31677 8938 Acked Write 4 Done Sent Done false
31678 8939 Acked Write 3 Done Sent Done false
31679 8940 Acked Write 4 Done Sent Done false
31680 8941 Acked Read 10 Done Sent Done false
31681 8942 Acked Read 2 Done Sent Done false
31682 8943 Acked Write 9 Done Sent Done false
31683 8944 Acked Flush 0 Done Sent Done false
31684 8945 Acked Read 1 Done Sent Done false
31685 8946 Acked Read 1 Done Sent Done false
31686 8947 Acked Write 7 Done Sent Done false
31687 8948 Acked Read 8 Done Sent Done false
31688 8949 Acked Write 6 Done Sent Done false
31689 8950 Acked Read 10 Done Sent Done false
31690 8951 Acked Read 7 Done Sent Done false
31691 8952 Acked Read 10 Done Sent Done false
31692 8953 Acked Write 1 Done Sent Done false
31693 8954 Acked Write 4 Done Sent Done false
31694 8955 Acked Write 9 Done Sent Done false
31695 8956 Acked Write 5 Done Sent Done false
31696 8957 Acked Write 4 Done Sent Done false
31697 8958 Acked Read 3 Done Sent Done false
31698 8959 Acked Write 3 Done Sent Done false
31699 8960 Acked Flush 0 Done Sent Done false
31700 8961 Acked Write 7 Done Sent Done false
31701 8962 Acked Write 9 Done Sent Done false
31702 8963 Acked Read 1 Done Sent Done false
31703 8964 Acked Write 9 Done Sent Done false
31704 8965 Acked Read 1 Done Sent Done false
31705 8966 Acked Flush 0 Done Sent Done false
31706 8967 Acked Read 5 Done Sent Done false
31707 8968 Acked Read 9 Done Sent Done false
31708 8969 Acked Write 5 Done Sent Done false
31709 8970 Acked Read 9 Done Sent Done false
31710 8971 Acked Write 7 Done Sent Done false
31711 8972 Acked Write 6 Done Sent Done false
31712 8973 Acked Flush 0 Done Sent Done false
31713 8974 Acked Write 3 Done Sent Done false
31714 8975 Acked Write 2 Done Sent Done false
31715 8976 Acked Read 6 Done Sent Done false
31716 8977 Acked Write 5 Done Sent Done false
31717 8978 Acked Write 10 Done Sent Done false
31718 8979 Acked Write 10 Done Sent Done false
31719 8980 Acked Write 2 Done Sent Done false
31720 8981 Acked Write 9 Done Sent Done false
31721 8982 Acked Read 8 Done Sent Done false
31722 8983 Acked Read 3 Done Sent Done false
31723 8984 Acked Write 6 Done Sent Done false
31724 8985 Acked Write 2 Done Sent Done false
31725 8986 Acked Read 9 Done Sent Done false
31726 8987 Acked Flush 0 Done Sent Done false
31727 8988 Acked Read 7 Done Sent Done false
31728 8989 Acked Write 3 Done Sent Done false
31729 8990 Acked Read 2 Done Sent Done false
31730 8991 Acked Write 2 Done Sent Done false
31731 8992 Acked Write 9 Done Sent Done false
31732 8993 Acked Read 3 Done Sent Done false
31733 8994 Acked Write 8 Done Sent Done false
31734 8995 Acked Read 5 Done Sent Done false
31735 8996 Acked Flush 0 Done Sent Done false
31736 8997 Acked Write 9 Done Sent Done false
31737 8998 Acked Write 8 Done Sent Done false
31738 8999 Acked Write 8 Done Sent Done false
31739 9000 Acked Write 5 Done Sent Done false
31740 9001 Acked Read 7 Done Sent Done false
31741 9002 Acked Flush 0 Done Sent Done false
31742 9003 Acked Read 9 Done Sent Done false
31743 9004 Acked Write 4 Done Sent Done false
31744 9005 Acked Write 5 Done Sent Done false
31745 9006 Acked Write 10 Done Sent Done false
31746 9007 Acked Read 4 Done Sent Done false
31747 9008 Acked Write 7 Done Sent Done false
31748 9009 Acked Flush 0 Done Sent Done false
31749 9010 Acked Write 4 Done Sent Done false
31750 9011 Acked Write 8 Done Sent Done false
31751 9012 Acked Write 4 Done Sent Done false
31752 9013 Acked Read 7 Done Sent Done false
31753 9014 Acked Write 1 Done Sent Done false
31754 9015 Acked Flush 0 Done Sent Done false
31755 9016 Acked Write 1 Done Sent Done false
31756 9017 Acked Flush 0 Done Sent Done false
31757 9018 Acked Write 9 Done Sent Done false
31758 9019 Acked Write 6 Done Sent Done false
31759 9020 Acked Read 8 Done Sent Done false
31760 9021 Acked Read 5 Done Sent Done false
31761 9022 Acked Read 7 Done Sent Done false
31762 9023 Acked Read 4 Done Sent Done false
31763 9024 Acked Write 3 Done Sent Done false
31764 9025 Acked Write 2 Done Sent Done false
31765 9026 Acked Read 7 Done Sent Done false
31766 9027 Acked Write 1 Done Sent Done false
31767 9028 Acked Write 8 Done Sent Done false
31768 9029 Acked Read 7 Done Sent Done false
31769 9030 Acked Write 7 Done Sent Done false
31770 9031 Acked Write 2 Done Sent Done false
31771 9032 Acked Read 2 Done Sent Done false
31772 9033 Acked Read 6 Done Sent Done false
31773 9034 Acked Read 5 Done Sent Done false
31774 9035 Acked Read 8 Done Sent Done false
31775 9036 Acked Read 8 Done Sent Done false
31776 9037 Acked Read 7 Done Sent Done false
31777 9038 Acked Read 10 Done Sent Done false
31778 9039 Acked Read 10 Done Sent Done false
31779 9040 Acked Read 1 Done Sent Done false
31780 9041 Acked Read 4 Done Sent Done false
31781 9042 Acked Flush 0 Done Sent Done false
31782 9043 Acked Read 3 Done Sent Done false
31783 9044 Acked Read 3 Done Sent Done false
31784 9045 Acked Read 5 Done Sent Done false
31785 9046 Acked Read 9 Done Sent Done false
31786 9047 Acked Read 9 Done Sent Done false
31787 9048 Acked Write 9 Done Sent Done false
31788 9049 Acked Read 2 Done Sent Done false
31789 9050 Acked Write 4 Done Sent Done false
31790 9051 Acked Write 3 Done Sent Done false
31791 9052 Acked Write 8 Done Sent Done false
31792 9053 Acked Write 2 Done Sent Done false
31793 9054 Acked Read 8 Done Sent Done false
31794 9055 Acked Read 9 Done Sent Done false
31795 9056 Acked Write 4 Done Sent Done false
31796 9057 Acked Read 7 Done Sent Done false
31797 9058 Acked Read 2 Done Sent Done false
31798 9059 Acked Read 4 Done Sent Done false
31799 9060 Acked Read 6 Done Sent Done false
31800 9061 Acked Write 1 Done Sent Done false
31801 9062 Acked Read 9 Done Sent Done false
31802 9063 Acked Read 9 Done Sent Done false
31803 9064 Acked Read 9 Done Sent Done false
31804 9065 Acked Read 1 Done Sent Done false
31805 9066 Acked Write 5 Done Sent Done false
31806 9067 Acked Flush 0 Done Sent Done false
31807 9068 Acked Read 2 Done Sent Done false
31808 9069 Acked Read 5 Done Sent Done false
31809 9070 Acked Read 8 Done Sent Done false
31810 9071 Acked Flush 0 Done Sent Done false
31811 9072 Acked Read 8 Done Sent Done false
31812 9073 Acked Read 7 Done Sent Done false
31813 9074 Acked Read 2 Done Sent Done false
31814 9075 Acked Write 5 Done Sent Done false
31815 9076 Acked Read 9 Done Sent Done false
31816 9077 Acked Read 4 Done Sent Done false
31817 9078 Acked Write 6 Done Sent Done false
31818 9079 Acked Read 6 Done Sent Done false
31819 9080 Acked Read 3 Done Sent Done false
31820 9081 Acked Read 5 Done Sent Done false
31821 9082 Acked Read 5 Done Sent Done false
31822 9083 Acked Read 8 Done Sent Done false
31823 9084 Acked Read 8 Done Sent Done false
31824 9085 Acked Read 9 Done Sent Done false
31825 9086 Acked Write 10 Done Sent Done false
31826 9087 Acked Write 3 Done Sent Done false
31827 9088 Acked Read 7 Done Sent Done false
31828 9089 Acked Read 3 Done Sent Done false
31829 9090 Acked Read 9 Done Sent Done false
31830 9091 Acked Write 9 Done Sent Done false
31831 9092 Acked Read 1 Done Sent Done false
31832 9093 Acked Read 5 Done Sent Done false
31833 9094 Acked Write 2 Done Sent Done false
31834 9095 Acked Read 2 Done Sent Done false
31835 9096 Acked Read 2 Done Sent Done false
31836 9097 Acked Read 7 Done Sent Done false
31837 9098 Acked Read 8 Done Sent Done false
31838 9099 Acked Read 1 Done Sent Done false
31839 9100 Acked Read 6 Done Sent Done false
31840 9101 Acked Write 7 Done Sent Done false
31841 9102 Acked Write 5 Done Sent Done false
31842 9103 Acked Write 8 Done Sent Done false
31843 9104 Acked Write 6 Done Sent Done false
31844 9105 Acked Read 3 Done Sent Done false
31845 9106 Acked Flush 0 Done Sent Done false
31846 9107 Acked Write 5 Done Sent Done false
31847 9108 Acked Write 6 Done Sent Done false
31848 9109 Acked Write 5 Done Sent Done false
31849 9110 Acked Read 4 Done Sent Done false
31850 9111 Acked Write 2 Done Sent Done false
31851 9112 Acked Read 8 Done Sent Done false
31852 9113 Acked Write 3 Done Sent Done false
31853 9114 Acked Write 5 Done Sent Done false
31854 9115 Acked Write 3 Done Sent Done false
31855 9116 Acked Read 2 Done Sent Done false
31856 9117 Acked Read 1 Done Sent Done false
31857 9118 Acked Read 10 Done Sent Done false
31858 9119 Acked Read 4 Done Sent Done false
31859 9120 Acked Write 3 Done Sent Done false
31860 9121 Acked Flush 0 Done Sent Done false
31861 9122 Acked Write 2 Done Sent Done false
31862 9123 Acked Write 8 Done Sent Done false
31863 9124 Acked Read 8 Done Sent Done false
31864 9125 Acked Flush 0 Done Sent Done false
31865 9126 Acked Write 8 Done Sent Done false
31866 9127 Acked Read 9 Done Sent Done false
31867 9128 Acked Read 3 Done Sent Done false
31868 9129 Acked Flush 0 Done Sent Done false
31869 9130 Acked Write 7 Done Sent Done false
31870 9131 Acked Write 1 Done Sent Done false
31871 9132 Acked Read 9 Done Sent Done false
31872 9133 Acked Read 6 Done Sent Done false
31873 9134 Acked Read 8 Done Sent Done false
31874 9135 Acked Read 6 Done Sent Done false
31875 9136 Acked Write 7 Done Sent Done false
31876 9137 Acked Write 6 Done Sent Done false
31877 9138 Acked Flush 0 Done Sent Done false
31878 9139 Acked Read 1 Done Sent Done false
31879 9140 Acked Read 7 Done Sent Done false
31880 9141 Acked Read 9 Done Sent Done false
31881 9142 Acked Write 6 Done Sent Done false
31882 9143 Acked Write 8 Done Sent Done false
31883 9144 Acked Read 7 Done Sent Done false
31884 9145 Acked Write 4 Done Sent Done false
31885 9146 Acked Write 3 Done Sent Done false
31886 9147 Acked Read 7 Done Sent Done false
31887 9148 Acked Read 1 Done Sent Done false
31888 9149 Acked Read 8 Done Sent Done false
31889 9150 Acked Flush 0 Done Sent Done false
31890 STATES DS:0 DS:1 DS:2 TOTAL
31891 Sent 0 302 0 302
31892 Done 302 0 302 604
31893 Skipped 0 0 0 0
31894 Error 0 0 0 0
31895 Last Flush: 9150 8848 9150
31896 Downstairs last five completed: 8848 8847 8846 8845 8844
31897 Upstairs last five completed: 9150 9149 9148 9147 9146
31898 CLIENT: Up:0 ds:302 act:2
318992024-11-26T22:11:12.783ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
319002024-11-26T22:11:12.783ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:52464) = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
319012024-11-26T22:11:12.784ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
319022024-11-26T22:11:12.785ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
319032024-11-26T22:11:12.785ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
319042024-11-26T22:11:12.785ZINFOcrucible: send last flush ID to this DS: 8848 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
319052024-11-26T22:11:12.785ZINFOcrucible: Replied this last flush ID: 8848 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
319062024-11-26T22:11:12.785ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
319072024-11-26T22:11:12.785ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
319082024-11-26T22:11:12.785ZINFOcrucible: [1] client re-new 302 jobs since flush 8848 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
319092024-11-26T22:11:12.785ZINFOcrucible: [1] Replayed 302 jobs since flush: 8848 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
31910 ----------------------------------------------------------------
31911 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
31912 Downstairs last five completed: 9150 9149 9148 9147 9146
31913 Upstairs last five completed: 9150 9149 9148 9147 9146
31914 CLIENT: Up:0 ds:0 act:3
31915 Replay: All jobs finished
31916 Main task has work to do, go find it
31917 got dsc Stop(1)
31918 stop 1
31919 Nov 26 22:11:15.029 INFO request completed, latency_us:[8820] Got stop action so:false kr:false
31920 354, response_code: 204, uri: /stop/cid/1, method: POST, req_id: dd909a13-eff1-490e-8f5b-69f2dc720068, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
319212024-11-26T22:11:15.030ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
319222024-11-26T22:11:15.030ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
319232024-11-26T22:11:15.030ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
319242024-11-26T22:11:15.030ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
31925 Nov 26 22:11:15.030{"msg":"[1] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-26T22:11:15.030456813Z","hostname":"w-01JDN7ZQ17ABQG15KHKN3F2D26","pid":1203,"client":"1","":"downstairs","session_id":"86a36c17-3718-484c-87fd-2b8a1dc2ab18"}
31926 INFO request completed, latency_us: 153, response_code: 200, uri: /state/cid/1, method:{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2024-11-26T22:11:15.030509677Z","hostname":"w-01JDN7ZQ17ABQG15KHKN3F2D26","pid":1203,"":"io task","client":"1","":"downstairs","session_id":"86a36c17-3718-484c-87fd-2b8a1dc2ab18"}
31927 GET, req_id: b4afaf5e-1e58-4110-96d6-3c74d379da02, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
31928 [8820] Exited with: ExitStatus(unix_wait_status(9))
31929 [8820][1] reports Exit
31930 Nov 26 22:11:19.031 INFO request completed, latency_us: 188, response_code: 200, uri: /state/cid/1, method: GET, req_id: 1c2c598c-3542-4134-b330-9d546de7b1d0, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
31931 001/300 Read block 2779 len 8192
31932 002/300 Write block 2449 len 28672 data: 7 6 7 7 5 6 8
31933 003/300 Read block 577 len 12288
31934 004/300 Write block 2878 len 8192 data: 7 7
31935 005/300 Read block 4072 len 28672
31936 006/300 Read block 4599 len 16384
31937 007/300 Flush
31938 008/300 Read block 1722 len 24576
31939 009/300 Read block 351 len 8192
31940 010/300 Read block 4299 len 36864
31941 011/300 Read block 1728 len 16384
31942 012/300 Read block 4271 len 20480
31943 013/300 Write block 4921 len 20480 data: 5 6 7 8 10
31944 014/300 Write block 3902 len 8192 data: 2 2
31945 015/300 Read block 3531 len 36864
31946 016/300 Read block 4496 len 28672
31947 017/300 Write block 374 len 28672 data: 8 7 4 4 4 6 4
31948 018/300 Read block 2091 len 40960
31949 019/300 Write block 4401 len 40960 data: 3 4 4 5 6 5 4 4 4 4
31950 020/300 Read block 679 len 32768
31951 021/300 Read block 2826 len 28672
31952 022/300 Write block 4219 len 12288 data: 4 4 4
31953 023/300 Write block 2323 len 40960 data: 6 5 4 4 5 5 6 6 5 4
31954 024/300 Write block 1193 len 40960 data: 9 10 11 12 13 9 8 8 8 8
31955 025/300 Read block 1272 len 4096
31956 026/300 Read block 986 len 28672
31957 027/300 Write block 2101 len 40960 data: 6 8 8 8 7 5 5 5 4 5
31958 028/300 Read block 967 len 40960
31959 029/300 Read block 2310 len 16384
31960 030/300 Write block 531 len 12288 data: 5 4 4
31961 031/300 Read block 2557 len 8192
31962 032/300 Read block 4602 len 12288
31963 033/300 Write block 1773 len 4096 data: 5
31964 034/300 Read block 2118 len 40960
31965 035/300 Flush
31966 036/300 Write block 4223 len 4096 data: 4
31967 037/300 Flush
31968 038/300 Read block 4292 len 20480
31969 039/300 Write block 145 len 24576 data: 6 3 4 4 4 4
31970 040/300 Read block 608 len 28672
31971 041/300 Read block 4685 len 28672
31972 042/300 Flush
31973 043/300 Write block 3529 len 16384 data: 5 6 7 6
31974 044/300 Write block 4613 len 24576 data: 4 5 6 7 7 8
31975 045/300 Read block 4737 len 28672
31976 046/300 Read block 1146 len 16384
31977 047/300 Write block 2374 len 20480 data: 6 5 4 4 5
31978 048/300 Read block 3614 len 16384
31979 049/300 Flush
31980 050/300 Write block 4649 len 4096 data: 6
31981 051/300 Read block 4918 len 4096
31982 052/300 Flush
31983 053/300 Write block 4275 len 40960 data: 5 4 4 7 7 8 6 5 5 4
31984 054/300 Write block 1677 len 36864 data: 6 5 7 6 6 8 7 7 6
31985 055/300 Write block 4942 len 16384 data: 5 5 5 4
31986 056/300 Read block 1765 len 28672
31987 057/300 Write block 3156 len 12288 data: 5 4 5
31988 058/300 Write block 651 len 28672 data: 3 3 3 3 6 6 6
31989 059/300 Write block 4825 len 20480 data: 6 4 5 6 7
31990 060/300 Write block 3379 len 40960 data: 5 3 4 5 5 5 6 5 4 4
31991 061/300 Flush
31992 062/300 Read block 4341 len 12288
31993 063/300 Write block 3777 len 16384 data: 7 8 8 9
31994 064/300 Write block 1762 len 40960 data: 6 3 2 2 3 3 2 4 4 6
31995 065/300 Flush
31996 066/300 Read block 1887 len 16384
31997 067/300 Read block 3050 len 4096
31998 068/300 Read block 1094 len 32768
31999 069/300 Read block 4823 len 8192
32000 070/300 Write block 1896 len 4096 data: 6
32001 071/300 Write block 2124 len 28672 data: 4 4 2 3 3 3 3
32002 072/300 Read block 1356 len 8192
32003 073/300 Read block 10 len 4096
32004 074/300 Write block 1235 len 4096 data: 11
32005 075/300 Write block 3375 len 12288 data: 6 5 4
32006 076/300 Write block 4989 len 40960 data: 10 11 10 10 9 8 8 5 4 3
32007 077/300 Read block 816 len 36864
32008 078/300 Read block 1950 len 12288
32009 079/300 Read block 1503 len 40960
32010 080/300 Write block 1663 len 32768 data: 4 4 4 3 4 5 4 5
32011 081/300 Flush
32012 082/300 Read block 3141 len 20480
32013 083/300 Read block 740 len 4096
32014 084/300 Read block 2038 len 8192
32015 085/300 Write block 1595 len 40960 data: 6 6 5 7 9 6 5 5 5 6
32016 086/300 Read block 4299 len 24576
32017 087/300 Read block 1255 len 4096
32018 088/300 Flush
32019 089/300 Write block 3623 len 12288 data: 2 5 5
32020 090/300 Write block 2176 len 12288 data: 3 4 4
32021 091/300 Flush
32022 092/300 Read block 2402 len 32768
32023 093/300 Read block 1704 len 28672
32024 094/300 Read block 2728 len 40960
32025 095/300 Read block 4346 len 20480
32026 096/300 Write block 1422 len 12288 data: 4 4 5
32027 097/300 Read block 3745 len 8192
32028 098/300 Read block 2890 len 8192
32029 099/300 Read block 2268 len 4096
32030 100/300 Write block 118 len 16384 data: 6 5 5 6
32031 101/300 Flush
32032 102/300 Read block 2720 len 8192
32033 103/300 Write block 2773 len 36864 data: 8 6 6 6 4 4 4 3 3
32034 104/300 Write block 2661 len 40960 data: 4 3 5 5 6 5 6 5 4 5
32035 105/300 Write block 289 len 20480 data: 11 10 9 10 9
32036 106/300 Read block 2564 len 32768
32037 107/300 Read block 730 len 4096
32038 108/300 Read block 4199 len 12288
32039 109/300 Read block 1149 len 32768
32040 110/300 Read block 450 len 32768
32041 111/300 Read block 4302 len 28672
32042 112/300 Read block 4988 len 28672
32043 113/300 Read block 4033 len 4096
32044 114/300 Read block 3430 len 32768
32045 115/300 Write block 564 len 4096 data: 5
32046 116/300 Write block 692 len 4096 data: 2
32047 117/300 Read block 2657 len 16384
32048 118/300 Read block 2133 len 32768
32049 119/300 Read block 506 len 8192
32050 120/300 Flush
32051 121/300 Flush
32052 122/300 Read block 681 len 20480
32053 123/300 Write block 4307 len 40960 data: 4 5 5 4 5 6 7 8 6 6
32054 124/300 Read block 4542 len 12288
32055 125/300 Write block 242 len 36864 data: 7 7 6 4 5 5 4 4 3
32056 126/300 Read block 4727 len 12288
32057 127/300 Read block 4933 len 40960
32058 128/300 Read block 2158 len 32768
32059 129/300 Write block 2806 len 24576 data: 5 5 7 9 7 7
32060 130/300 Read block 2355 len 12288
32061 131/300 Write block 790 len 28672 data: 7 6 6 5 5 3 3
32062 132/300 Write block 3820 len 36864 data: 2 2 2 3 3 5 5 7 7
32063 133/300 Read block 1785 len 28672
32064 134/300 Read block 3065 len 4096
32065 135/300 Read block 2899 len 24576
32066 136/300 Flush
32067 137/300 Write block 2142 len 20480 data: 4 4 4 6 4
32068 138/300 Write block 2270 len 36864 data: 6 6 7 6 6 4 3 3 3
32069 139/300 Read block 3653 len 28672
32070 140/300 Read block 2175 len 32768
32071 141/300 Read block 2275 len 16384
32072 142/300 Write block 4508 len 8192 data: 5 6
32073 143/300 Read block 1640 len 28672
32074 144/300 Read block 4013 len 24576
32075 145/300 Flush
32076 146/300 Read block 1651 len 8192
32077 147/300 Read block 789 len 24576
32078 148/300 Read block 326 len 16384
32079 149/300 Write block 2873 len 20480 data: 3 5 8 8 9
32080 150/300 Read block 1502 len 20480
32081 151/300 Read block 2927 len 16384
32082 152/300 Write block 2578 len 8192 data: 5 5
32083 153/300 Write block 1115 len 28672 data: 3 4 5 5 5 5 5
32084 154/300 Read block 3439 len 4096
32085 155/300 Read block 806 len 24576
32086 156/300 Flush
32087 157/300 Write block 1380 len 24576 data: 4 4 5 5 5 5
32088 158/300 Read block 1618 len 28672
32089 159/300 Write block 801 len 28672 data: 7 6 6 5 3 3 5
32090 160/300 Write block 1556 len 12288 data: 7 6 6
32091 161/300 Read block 1664 len 24576
32092 162/300 Read block 4921 len 28672
32093 163/300 Write block 3670 len 20480 data: 10 9 10 11 10
32094 164/300 Write block 4867 len 24576 data: 5 5 6 7 7 7
32095 165/300 Write block 3300 len 4096 data: 3
32096 166/300 Write block 4056 len 24576 data: 4 5 5 6 7 8
32097 167/300 Read block 708 len 8192
32098 168/300 Read block 3754 len 12288
32099 169/300 Read block 1310 len 36864
32100 170/300 Read block 2344 len 32768
32101 171/300 Read block 1204 len 8192
32102 172/300 Read block 1468 len 8192
32103 173/300 Read block 3988 len 12288
32104 174/300 Read block 4413 len 8192
32105 175/300 Write block 2309 len 40960 data: 5 4 4 6 6 7 7 5 5 6
32106 176/300 Read block 4936 len 12288
32107 177/300 Read block 77 len 8192
32108 178/300 Read block 3656 len 8192
32109 179/300 Write block 371 len 32768 data: 7 7 7 9 8 5 5 5
32110 180/300 Write block 595 len 28672 data: 3 3 3 3 4 4 5
32111 181/300 Write block 4197 len 20480 data: 9 9 9 8 8
32112 182/300 Read block 922 len 36864
32113 183/300 Write block 4579 len 40960 data: 4 4 4 6 6 4 4 5 6 6
32114 184/300 Write block 3419 len 40960 data: 5 4 5 4 4 5 6 5 5 5
32115 185/300 Write block 394 len 8192 data: 3 4
32116 186/300 Write block 4186 len 24576 data: 5 3 3 3 3 4
32117 187/300 Read block 4718 len 8192
32118 188/300 Read block 487 len 16384
32119 189/300 Flush
32120 190/300 Read block 2858 len 8192
32121 191/300 Read block 4601 len 12288
32122 192/300 Read block 1220 len 20480
32123 193/300 Write block 2863 len 24576 data: 6 5 4 4 4 4
32124 194/300 Write block 470 len 20480 data: 5 6 6 5 6
32125 195/300 Write block 3848 len 32768 data: 4 2 3 4 4 5 6 6
32126 196/300 Write block 1770 len 12288 data: 5 7 5
32127 197/300 Write block 566 len 40960 data: 4 4 4 4 4 6 7 6 7 8
32128 198/300 Read block 876 len 12288
32129 199/300 Write block 4004 len 8192 data: 7 7
32130 200/300 Read block 2156 len 12288
32131 201/300 Flush
32132 202/300 Flush
32133 203/300 Write block 3157 len 36864 data: 5 6 5 9 8 7 5 5 6
32134 204/300 Write block 3001 len 16384 data: 6 6 6 5
32135 205/300 Write block 975 len 12288 data: 4 4 4
32136 206/300 Write block 2647 len 32768 data: 3 3 2 2 2 3 3 4
32137 207/300 Write block 2096 len 24576 data: 4 5 4 4 5 7
32138 208/300 Read block 4616 len 8192
32139 209/300 Read block 1861 len 40960
32140 210/300 Write block 3084 len 8192 data: 7 8
32141 211/300 Read block 368 len 8192
32142 212/300 Read block 124 len 40960
32143 213/300 Read block 2454 len 40960
32144 214/300 Read block 2710 len 8192
32145 215/300 Write block 1868 len 8192 data: 6 6
32146 216/300 Write block 4610 len 12288 data: 5 4 4
32147 217/300 Flush
32148 218/300 Read block 197 len 16384
32149 219/300 Read block 2840 len 24576
32150 220/300 Read block 3511 len 4096
32151 221/300 Read block 4371 len 32768
32152 222/300 Flush
32153 223/300 Write block 550 len 36864 data: 4 4 4 4 6 6 6 6 5
32154 224/300 Read block 2928 len 8192
32155 225/300 Write block 771 len 4096 data: 8
32156 226/300 Read block 2958 len 12288
32157 227/300 Write block 4569 len 24576 data: 4 6 6 6 5 5
32158 228/300 Write block 1159 len 36864 data: 7 7 6 5 6 6 6 7 7
32159 229/300 Read block 1171 len 32768
32160 230/300 Read block 1111 len 16384
32161 231/300 Read block 2186 len 24576
32162 232/300 Write block 2815 len 16384 data: 5 6 5 4
32163 233/300 Read block 2314 len 24576
32164 234/300 Write block 2489 len 40960 data: 9 9 9 7 5 5 4 4 5 5
32165 235/300 Write block 2363 len 4096 data: 4
32166 236/300 Read block 3319 len 12288
32167 237/300 Write block 1872 len 28672 data: 6 6 6 7 5 4 2
32168 238/300 Write block 4594 len 40960 data: 7 7 7 7 8 7 7 7 8 9
32169 239/300 Write block 2125 len 8192 data: 5 3
32170 240/300 Write block 345 len 8192 data: 9 7
32171 241/300 Flush
32172 242/300 Read block 774 len 8192
32173 243/300 Write block 1295 len 8192 data: 4 4
32174 244/300 Write block 774 len 8192 data: 4 4
32175 245/300 Read block 1335 len 28672
32176 246/300 Write block 2058 len 4096 data: 10
32177 247/300 Write block 649 len 4096 data: 4
32178 248/300 Write block 2178 len 36864 data: 5 4 4 4 5 5 5 5 4
32179 249/300 Write block 3759 len 8192 data: 5 5
32180 250/300 Read block 4065 len 20480
32181 251/300 Read block 3359 len 28672
32182 252/300 Write block 4844 len 16384 data: 5 5 5 5
32183 253/300 Write block 4383 len 28672 data: 4 4 6 6 6 6 5
32184 254/300 Write block 2268 len 40960 data: 6 7 7 7 8 7 7 5 4 4
32185 255/300 Read block 724 len 16384
32186 256/300 Write block 3280 len 16384 data: 7 7 7 7
32187 257/300 Write block 135 len 24576 data: 4 6 8 8 8 9
32188 258/300 Write block 2505 len 8192 data: 6 4
32189 259/300 Read block 1382 len 4096
32190 260/300 Read block 132 len 12288
32191 261/300 Read block 2202 len 36864
32192 262/300 Read block 4177 len 28672
32193 263/300 Flush
32194 264/300 Write block 3232 len 16384 data: 9 7 5 3
32195 265/300 Read block 4489 len 32768
32196 266/300 Write block 1888 len 8192 data: 5 5
32197 267/300 Read block 3898 len 24576
32198 268/300 Flush
32199 269/300 Read block 2487 len 28672
32200 270/300 Write block 2238 len 12288 data: 4 6 6
32201 271/300 Write block 186 len 32768 data: 5 5 6 5 7 6 5 5
32202 272/300 Write block 1305 len 12288 data: 8 7 7
32203 273/300 Flush
32204 274/300 Read block 664 len 4096
32205 275/300 Read block 121 len 8192
32206 276/300 Flush
32207 277/300 Read block 904 len 16384
32208 278/300 Read block 3000 len 24576
32209 279/300 Write block 1600 len 12288 data: 7 6 6
32210 280/300 Read block 3336 len 16384
32211 281/300 Write block 304 len 32768 data: 3 4 4 3 3 3 4 5
32212 282/300 Write block 2931 len 32768 data: 5 6 6 5 5 4 4 3
32213 283/300 Read block 3504 len 20480
32214 284/300 Write block 1969 len 32768 data: 4 3 3 3 2 2 2 3
32215 285/300 Write block 2655 len 28672 data: 4 4 4 5 5 4 5
32216 286/300 Write block 2974 len 12288 data: 5 6 3
32217 287/300 Read block 2685 len 28672
32218 288/300 Read block 2464 len 12288
32219 289/300 Flush
32220 290/300 Write block 154 len 36864 data: 4 4 4 5 6 5 6 8 6
32221 291/300 Write block 3348 len 32768 data: 5 6 6 8 7 6 5 5
32222 292/300 Read block 3159 len 40960
32223 293/300 Write block 2942 len 40960 data: 3 4 4 4 6 6 4 4 4 6
32224 294/300 Read block 3116 len 28672
32225 295/300 Read block 619 len 40960
32226 296/300 Flush
32227 297/300 Write block 923 len 20480 data: 6 7 7 9 9
32228 298/300 Write block 3158 len 28672 data: 7 6 10 9 8 6 6
32229 299/300 Read block 3393 len 12288
32230 300/300 Read block 3061 len 32768
32231 301/300 Write block 3997 len 36864 data: 5 4 3 2 3 5 7 8 8
32232 Main task has work to do, go find it
32233 got dsc Start(1)
32234 start 1
32235 Nov 26 22:11:19.259 INFO request completed, latency_us: 206, response_code: 204, uri: /start/cid/1, method: POST, req_id: 02460d71-6abb-4f02-8dfa-eaf71c37e063, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
32236 Replay: started 1, returned:Ok(())
32237 ----------------------------------------------------------------
32238 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
32239 States: Active Offline Active
32240 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
32241 9151 Acked Read 2 Done Sent Done false
32242 9152 Acked Write 7 Done Sent Done false
32243 9153 Acked Read 3 Done Sent Done false
32244 9154 Acked Write 2 Done Sent Done false
32245 9155 Acked Read 7 Done Sent Done false
32246 9156 Acked Read 4 Done Sent Done false
32247 9157 Acked Flush 0 Done Sent Done false
32248 9158 Acked Read 6 Done Sent Done false
32249 9159 Acked Read 2 Done Sent Done false
32250 9160 Acked Read 9 Done Sent Done false
32251 9161 Acked Read 4 Done Sent Done false
32252 9162 Acked Read 5 Done Sent Done false
32253 9163 Acked Write 5 Done Sent Done false
32254 9164 Acked Write 2 Done Sent Done false
32255 9165 Acked Read 9 Done Sent Done false
32256 9166 Acked Read 7 Done Sent Done false
32257 9167 Acked Write 7 Done Sent Done false
32258 9168 Acked Read 10 Done Sent Done false
32259 9169 Acked Write 10 Done Sent Done false
32260 9170 Acked Read 8 Done Sent Done false
32261 9171 Acked Read 7 Done Sent Done false
32262 9172 Acked Write 3 Done Sent Done false
32263 9173 Acked Write 10 Done Sent Done false
32264 9174 Acked Write 10 Done Sent Done false
32265 9175 Acked Read 1 Done Sent Done false
32266 9176 Acked Read 7 Done Sent Done false
32267 9177 Acked Write 10 Done Sent Done false
32268 9178 Acked Read 10 Done Sent Done false
32269 9179 Acked Read 4 Done Sent Done false
32270 9180 Acked Write 3 Done Sent Done false
32271 9181 Acked Read 2 Done Sent Done false
32272 9182 Acked Read 3 Done Sent Done false
32273 9183 Acked Write 1 Done Sent Done false
32274 9184 Acked Read 10 Done Sent Done false
32275 9185 Acked Flush 0 Done Sent Done false
32276 9186 Acked Write 1 Done Sent Done false
32277 9187 Acked Flush 0 Done Sent Done false
32278 9188 Acked Read 5 Done Sent Done false
32279 9189 Acked Write 6 Done Sent Done false
32280 9190 Acked Read 7 Done Sent Done false
32281 9191 Acked Read 7 Done Sent Done false
32282 9192 Acked Flush 0 Done Sent Done false
32283 9193 Acked Write 4 Done Sent Done false
32284 9194 Acked Write 6 Done Sent Done false
32285 9195 Acked Read 7 Done Sent Done false
32286 9196 Acked Read 4 Done Sent Done false
32287 9197 Acked Write 5 Done Sent Done false
32288 9198 Acked Read 4 Done Sent Done false
32289 9199 Acked Flush 0 Done Sent Done false
32290 9200 Acked Write 1 Done Sent Done false
32291 9201 Acked Read 1 Done Sent Done false
32292 9202 Acked Flush 0 Done Sent Done false
32293 9203 Acked Write 10 Done Sent Done false
32294 9204 Acked Write 9 Done Sent Done false
32295 9205 Acked Write 4 Done Sent Done false
32296 9206 Acked Read 7 Done Sent Done false
32297 9207 Acked Write 3 Done Sent Done false
32298 9208 Acked Write 7 Done Sent Done false
32299 9209 Acked Write 5 Done Sent Done false
32300 9210 Acked Write 10 Done Sent Done false
32301 9211 Acked Flush 0 Done Sent Done false
32302 9212 Acked Read 3 Done Sent Done false
32303 9213 Acked Write 4 Done Sent Done false
32304 9214 Acked Write 10 Done Sent Done false
32305 9215 Acked Flush 0 Done Sent Done false
32306 9216 Acked Read 4 Done Sent Done false
32307 9217 Acked Read 1 Done Sent Done false
32308 9218 Acked Read 8 Done Sent Done false
32309 9219 Acked Read 2 Done Sent Done false
32310 9220 Acked Write 1 Done Sent Done false
32311 9221 Acked Write 7 Done Sent Done false
32312 9222 Acked Read 2 Done Sent Done false
32313 9223 Acked Read 1 Done Sent Done false
32314 9224 Acked Write 1 Done Sent Done false
32315 9225 Acked Write 3 Done Sent Done false
32316 9226 Acked Write 10 Done Sent Done false
32317 9227 Acked Read 9 Done Sent Done false
32318 9228 Acked Read 3 Done Sent Done false
32319 9229 Acked Read 10 Done Sent Done false
32320 9230 Acked Write 8 Done Sent Done false
32321 9231 Acked Flush 0 Done Sent Done false
32322 9232 Acked Read 5 Done Sent Done false
32323 9233 Acked Read 1 Done Sent Done false
32324 9234 Acked Read 2 Done Sent Done false
32325 9235 Acked Write 10 Done Sent Done false
32326 9236 Acked Read 6 Done Sent Done false
32327 9237 Acked Read 1 Done Sent Done false
32328 9238 Acked Flush 0 Done Sent Done false
32329 9239 Acked Write 3 Done Sent Done false
32330 9240 Acked Write 3 Done Sent Done false
32331 9241 Acked Flush 0 Done Sent Done false
32332 9242 Acked Read 8 Done Sent Done false
32333 9243 Acked Read 7 Done Sent Done false
32334 9244 Acked Read 10 Done Sent Done false
32335 9245 Acked Read 5 Done Sent Done false
32336 9246 Acked Write 3 Done Sent Done false
32337 9247 Acked Read 2 Done Sent Done false
32338 9248 Acked Read 2 Done Sent Done false
32339 9249 Acked Read 1 Done Sent Done false
32340 9250 Acked Write 4 Done Sent Done false
32341 9251 Acked Flush 0 Done Sent Done false
32342 9252 Acked Read 2 Done Sent Done false
32343 9253 Acked Write 9 Done Sent Done false
32344 9254 Acked Write 10 Done Sent Done false
32345 9255 Acked Write 5 Done Sent Done false
32346 9256 Acked Read 8 Done Sent Done false
32347 9257 Acked Read 1 Done Sent Done false
32348 9258 Acked Read 3 Done Sent Done false
32349 9259 Acked Read 8 Done Sent Done false
32350 9260 Acked Read 8 Done Sent Done false
32351 9261 Acked Read 7 Done Sent Done false
32352 9262 Acked Read 7 Done Sent Done false
32353 9263 Acked Read 1 Done Sent Done false
32354 9264 Acked Read 8 Done Sent Done false
32355 9265 Acked Write 1 Done Sent Done false
32356 9266 Acked Write 1 Done Sent Done false
32357 9267 Acked Read 4 Done Sent Done false
32358 9268 Acked Read 8 Done Sent Done false
32359 9269 Acked Read 2 Done Sent Done false
32360 9270 Acked Flush 0 Done Sent Done false
32361 9271 Acked Flush 0 Done Sent Done false
32362 9272 Acked Read 5 Done Sent Done false
32363 9273 Acked Write 10 Done Sent Done false
32364 9274 Acked Read 3 Done Sent Done false
32365 9275 Acked Write 9 Done Sent Done false
32366 9276 Acked Read 3 Done Sent Done false
32367 9277 Acked Read 10 Done Sent Done false
32368 9278 Acked Read 8 Done Sent Done false
32369 9279 Acked Write 6 Done Sent Done false
32370 9280 Acked Read 3 Done Sent Done false
32371 9281 Acked Write 7 Done Sent Done false
32372 9282 Acked Write 9 Done Sent Done false
32373 9283 Acked Read 7 Done Sent Done false
32374 9284 Acked Read 1 Done Sent Done false
32375 9285 Acked Read 6 Done Sent Done false
32376 9286 Acked Flush 0 Done Sent Done false
32377 9287 Acked Write 5 Done Sent Done false
32378 9288 Acked Write 9 Done Sent Done false
32379 9289 Acked Read 7 Done Sent Done false
32380 9290 Acked Read 8 Done Sent Done false
32381 9291 Acked Read 4 Done Sent Done false
32382 9292 Acked Write 2 Done Sent Done false
32383 9293 Acked Read 7 Done Sent Done false
32384 9294 Acked Read 6 Done Sent Done false
32385 9295 Acked Flush 0 Done Sent Done false
32386 9296 Acked Read 2 Done Sent Done false
32387 9297 Acked Read 6 Done Sent Done false
32388 9298 Acked Read 4 Done Sent Done false
32389 9299 Acked Write 5 Done Sent Done false
32390 9300 Acked Read 5 Done Sent Done false
32391 9301 Acked Read 4 Done Sent Done false
32392 9302 Acked Write 2 Done Sent Done false
32393 9303 Acked Write 7 Done Sent Done false
32394 9304 Acked Read 1 Done Sent Done false
32395 9305 Acked Read 6 Done Sent Done false
32396 9306 Acked Flush 0 Done Sent Done false
32397 9307 Acked Write 6 Done Sent Done false
32398 9308 Acked Read 7 Done Sent Done false
32399 9309 Acked Write 7 Done Sent Done false
32400 9310 Acked Write 3 Done Sent Done false
32401 9311 Acked Read 6 Done Sent Done false
32402 9312 Acked Read 7 Done Sent Done false
32403 9313 Acked Write 5 Done Sent Done false
32404 9314 Acked Write 6 Done Sent Done false
32405 9315 Acked Write 1 Done Sent Done false
32406 9316 Acked Write 6 Done Sent Done false
32407 9317 Acked Read 2 Done Sent Done false
32408 9318 Acked Read 3 Done Sent Done false
32409 9319 Acked Read 9 Done Sent Done false
32410 9320 Acked Read 8 Done Sent Done false
32411 9321 Acked Read 2 Done Sent Done false
32412 9322 Acked Read 2 Done Sent Done false
32413 9323 Acked Read 3 Done Sent Done false
32414 9324 Acked Read 2 Done Sent Done false
32415 9325 Acked Write 10 Done Sent Done false
32416 9326 Acked Read 3 Done Sent Done false
32417 9327 Acked Read 2 Done Sent Done false
32418 9328 Acked Read 2 Done Sent Done false
32419 9329 Acked Write 8 Done Sent Done false
32420 9330 Acked Write 7 Done Sent Done false
32421 9331 Acked Write 5 Done Sent Done false
32422 9332 Acked Read 9 Done Sent Done false
32423 9333 Acked Write 10 Done Sent Done false
32424 9334 Acked Write 10 Done Sent Done false
32425 9335 Acked Write 2 Done Sent Done false
32426 9336 Acked Write 6 Done Sent Done false
32427 9337 Acked Read 2 Done Sent Done false
32428 9338 Acked Read 4 Done Sent Done false
32429 9339 Acked Flush 0 Done Sent Done false
32430 9340 Acked Read 2 Done Sent Done false
32431 9341 Acked Read 3 Done Sent Done false
32432 9342 Acked Read 5 Done Sent Done false
32433 9343 Acked Write 6 Done Sent Done false
32434 9344 Acked Write 5 Done Sent Done false
32435 9345 Acked Write 8 Done Sent Done false
32436 9346 Acked Write 3 Done Sent Done false
32437 9347 Acked Write 10 Done Sent Done false
32438 9348 Acked Read 3 Done Sent Done false
32439 9349 Acked Write 2 Done Sent Done false
32440 9350 Acked Read 3 Done Sent Done false
32441 9351 Acked Flush 0 Done Sent Done false
32442 9352 Acked Flush 0 Done Sent Done false
32443 9353 Acked Write 9 Done Sent Done false
32444 9354 Acked Write 4 Done Sent Done false
32445 9355 Acked Write 3 Done Sent Done false
32446 9356 Acked Write 8 Done Sent Done false
32447 9357 Acked Write 6 Done Sent Done false
32448 9358 Acked Read 2 Done Sent Done false
32449 9359 Acked Read 10 Done Sent Done false
32450 9360 Acked Write 2 Done Sent Done false
32451 9361 Acked Read 2 Done Sent Done false
32452 9362 Acked Read 10 Done Sent Done false
32453 9363 Acked Read 10 Done Sent Done false
32454 9364 Acked Read 2 Done Sent Done false
32455 9365 Acked Write 2 Done Sent Done false
32456 9366 Acked Write 3 Done Sent Done false
32457 9367 Acked Flush 0 Done Sent Done false
32458 9368 Acked Read 4 Done Sent Done false
32459 9369 Acked Read 6 Done Sent Done false
32460 9370 Acked Read 1 Done Sent Done false
32461 9371 Acked Read 8 Done Sent Done false
32462 9372 Acked Flush 0 Done Sent Done false
32463 9373 Acked Write 9 Done Sent Done false
32464 9374 Acked Read 2 Done Sent Done false
32465 9375 Acked Write 1 Done Sent Done false
32466 9376 Acked Read 3 Done Sent Done false
32467 9377 Acked Write 6 Done Sent Done false
32468 9378 Acked Write 9 Done Sent Done false
32469 9379 Acked Read 8 Done Sent Done false
32470 9380 Acked Read 4 Done Sent Done false
32471 9381 Acked Read 6 Done Sent Done false
32472 9382 Acked Write 4 Done Sent Done false
32473 9383 Acked Read 6 Done Sent Done false
32474 9384 Acked Write 10 Done Sent Done false
32475 9385 Acked Write 1 Done Sent Done false
32476 9386 Acked Read 3 Done Sent Done false
32477 9387 Acked Write 7 Done Sent Done false
32478 9388 Acked Write 10 Done Sent Done false
32479 9389 Acked Write 2 Done Sent Done false
32480 9390 Acked Write 2 Done Sent Done false
32481 9391 Acked Flush 0 Done Sent Done false
32482 9392 Acked Read 2 Done Sent Done false
32483 9393 Acked Write 2 Done Sent Done false
32484 9394 Acked Write 2 Done Sent Done false
32485 9395 Acked Read 7 Done Sent Done false
32486 9396 Acked Write 1 Done Sent Done false
32487 9397 Acked Write 1 Done Sent Done false
32488 9398 Acked Write 9 Done Sent Done false
32489 9399 Acked Write 2 Done Sent Done false
32490 9400 Acked Read 5 Done Sent Done false
32491 9401 Acked Read 7 Done Sent Done false
32492 9402 Acked Write 4 Done Sent Done false
32493 9403 Acked Write 7 Done Sent Done false
32494 9404 Acked Write 10 Done Sent Done false
32495 9405 Acked Read 4 Done Sent Done false
32496 9406 Acked Write 4 Done Sent Done false
32497 9407 Acked Write 6 Done Sent Done false
32498 9408 Acked Write 2 Done Sent Done false
32499 9409 Acked Read 1 Done Sent Done false
32500 9410 Acked Read 3 Done Sent Done false
32501 9411 Acked Read 9 Done Sent Done false
32502 9412 Acked Read 7 Done Sent Done false
32503 9413 Acked Flush 0 Done Sent Done false
32504 9414 Acked Write 4 Done Sent Done false
32505 9415 Acked Read 8 Done Sent Done false
32506 9416 Acked Write 2 Done Sent Done false
32507 9417 Acked Read 6 Done Sent Done false
32508 9418 Acked Flush 0 Done Sent Done false
32509 9419 Acked Read 7 Done Sent Done false
32510 9420 Acked Write 3 Done Sent Done false
32511 9421 Acked Write 8 Done Sent Done false
32512 9422 Acked Write 3 Done Sent Done false
32513 9423 Acked Flush 0 Done Sent Done false
32514 9424 Acked Read 1 Done Sent Done false
32515 9425 Acked Read 2 Done Sent Done false
32516 9426 Acked Flush 0 Done Sent Done false
32517 9427 Acked Read 4 Done Sent Done false
32518 9428 Acked Read 6 Done Sent Done false
32519 9429 Acked Write 3 Done Sent Done false
32520 9430 Acked Read 4 Done Sent Done false
32521 9431 Acked Write 8 Done Sent Done false
32522 9432 Acked Write 8 Done Sent Done false
32523 9433 Acked Read 5 Done Sent Done false
32524 9434 Acked Write 8 Done Sent Done false
32525 9435 Acked Write 7 Done Sent Done false
32526 9436 Acked Write 3 Done Sent Done false
32527 9437 Acked Read 7 Done Sent Done false
32528 9438 Acked Read 3 Done Sent Done false
32529 9439 Acked Flush 0 Done Sent Done false
32530 9440 Acked Write 9 Done Sent Done false
32531 9441 Acked Write 8 Done Sent Done false
32532 9442 Acked Read 10 Done Sent Done false
32533 9443 Acked Write 10 Done Sent Done false
32534 9444 Acked Read 7 Done Sent Done false
32535 9445 Acked Read 10 Done Sent Done false
32536 9446 Acked Flush 0 Done Sent Done false
32537 9447 Acked Write 5 Done Sent Done false
32538 9448 Acked Write 7 Done Sent Done false
32539 9449 Acked Read 3 Done Sent Done false
32540 9450 Acked Read 8 Done Sent Done false
32541 9451 Acked Write 9 Done Sent Done false
32542 STATES DS:0 DS:1 DS:2 TOTAL
32543 Sent 0 301 0 301
32544 Done 301 0 301 602
32545 Skipped 0 0 0 0
32546 Error 0 0 0 0
32547 Last Flush: 9446 9150 9446
32548 Downstairs last five completed: 9150 9149 9148 9147 9146
32549 Upstairs last five completed: 9451 9450 9449 9448 9447
32550 CLIENT: Up:0 ds:301 act:2
32551 [8820] got start action
32552 [8820] I am going to restart
32553 Starting downstairs at port 8820
32554 Make output file at "/tmp/dsc/downstairs-8820.txt"
32555 [8820][1] reports Starting
32556 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1231)
32557 [8820][1] reports Running
32558 ----------------------------------------------------------------
32559 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
32560 States: Active Offline Active
32561 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
32562 9151 Acked Read 2 Done Sent Done false
32563 9152 Acked Write 7 Done Sent Done false
32564 9153 Acked Read 3 Done Sent Done false
32565 9154 Acked Write 2 Done Sent Done false
32566 9155 Acked Read 7 Done Sent Done false
32567 9156 Acked Read 4 Done Sent Done false
32568 9157 Acked Flush 0 Done Sent Done false
32569 9158 Acked Read 6 Done Sent Done false
32570 9159 Acked Read 2 Done Sent Done false
32571 9160 Acked Read 9 Done Sent Done false
32572 9161 Acked Read 4 Done Sent Done false
32573 9162 Acked Read 5 Done Sent Done false
32574 9163 Acked Write 5 Done Sent Done false
32575 9164 Acked Write 2 Done Sent Done false
32576 9165 Acked Read 9 Done Sent Done false
32577 9166 Acked Read 7 Done Sent Done false
32578 9167 Acked Write 7 Done Sent Done false
32579 9168 Acked Read 10 Done Sent Done false
32580 9169 Acked Write 10 Done Sent Done false
32581 9170 Acked Read 8 Done Sent Done false
32582 9171 Acked Read 7 Done Sent Done false
32583 9172 Acked Write 3 Done Sent Done false
32584 9173 Acked Write 10 Done Sent Done false
32585 9174 Acked Write 10 Done Sent Done false
32586 9175 Acked Read 1 Done Sent Done false
32587 9176 Acked Read 7 Done Sent Done false
32588 9177 Acked Write 10 Done Sent Done false
32589 9178 Acked Read 10 Done Sent Done false
32590 9179 Acked Read 4 Done Sent Done false
32591 9180 Acked Write 3 Done Sent Done false
32592 9181 Acked Read 2 Done Sent Done false
32593 9182 Acked Read 3 Done Sent Done false
32594 9183 Acked Write 1 Done Sent Done false
32595 9184 Acked Read 10 Done Sent Done false
32596 9185 Acked Flush 0 Done Sent Done false
32597 9186 Acked Write 1 Done Sent Done false
32598 9187 Acked Flush 0 Done Sent Done false
32599 9188 Acked Read 5 Done Sent Done false
32600 9189 Acked Write 6 Done Sent Done false
32601 9190 Acked Read 7 Done Sent Done false
32602 9191 Acked Read 7 Done Sent Done false
32603 9192 Acked Flush 0 Done Sent Done false
32604 9193 Acked Write 4 Done Sent Done false
32605 9194 Acked Write 6 Done Sent Done false
32606 9195 Acked Read 7 Done Sent Done false
32607 9196 Acked Read 4 Done Sent Done false
32608 9197 Acked Write 5 Done Sent Done false
32609 9198 Acked Read 4 Done Sent Done false
32610 9199 Acked Flush 0 Done Sent Done false
32611 9200 Acked Write 1 Done Sent Done false
32612 9201 Acked Read 1 Done Sent Done false
32613 9202 Acked Flush 0 Done Sent Done false
32614 9203 Acked Write 10 Done Sent Done false
32615 9204 Acked Write 9 Done Sent Done false
32616 9205 Acked Write 4 Done Sent Done false
32617 9206 Acked Read 7 Done Sent Done false
32618 9207 Acked Write 3 Done Sent Done false
32619 9208 Acked Write 7 Done Sent Done false
32620 9209 Acked Write 5 Done Sent Done false
32621 9210 Acked Write 10 Done Sent Done false
32622 9211 Acked Flush 0 Done Sent Done false
32623 9212 Acked Read 3 Done Sent Done false
32624 9213 Acked Write 4 Done Sent Done false
32625 9214 Acked Write 10 Done Sent Done false
32626 9215 Acked Flush 0 Done Sent Done false
32627 9216 Acked Read 4 Done Sent Done false
32628 9217 Acked Read 1 Done Sent Done false
32629 9218 Acked Read 8 Done Sent Done false
32630 9219 Acked Read 2 Done Sent Done false
32631 9220 Acked Write 1 Done Sent Done false
32632 9221 Acked Write 7 Done Sent Done false
32633 9222 Acked Read 2 Done Sent Done false
32634 9223 Acked Read 1 Done Sent Done false
32635 9224 Acked Write 1 Done Sent Done false
32636 9225 Acked Write 3 Done Sent Done false
32637 9226 Acked Write 10 Done Sent Done false
32638 9227 Acked Read 9 Done Sent Done false
32639 9228 Acked Read 3 Done Sent Done false
32640 9229 Acked Read 10 Done Sent Done false
32641 9230 Acked Write 8 Done Sent Done false
32642 9231 Acked Flush 0 Done Sent Done false
32643 9232 Acked Read 5 Done Sent Done false
32644 9233 Acked Read 1 Done Sent Done false
32645 9234 Acked Read 2 Done Sent Done false
32646 9235 Acked Write 10 Done Sent Done false
32647 9236 Acked Read 6 Done Sent Done false
32648 9237 Acked Read 1 Done Sent Done false
32649 9238 Acked Flush 0 Done Sent Done false
32650 9239 Acked Write 3 Done Sent Done false
32651 9240 Acked Write 3 Done Sent Done false
32652 9241 Acked Flush 0 Done Sent Done false
32653 9242 Acked Read 8 Done Sent Done false
32654 9243 Acked Read 7 Done Sent Done false
32655 9244 Acked Read 10 Done Sent Done false
32656 9245 Acked Read 5 Done Sent Done false
32657 9246 Acked Write 3 Done Sent Done false
32658 9247 Acked Read 2 Done Sent Done false
32659 9248 Acked Read 2 Done Sent Done false
32660 9249 Acked Read 1 Done Sent Done false
32661 9250 Acked Write 4 Done Sent Done false
32662 9251 Acked Flush 0 Done Sent Done false
32663 9252 Acked Read 2 Done Sent Done false
32664 9253 Acked Write 9 Done Sent Done false
32665 9254 Acked Write 10 Done Sent Done false
32666 9255 Acked Write 5 Done Sent Done false
32667 9256 Acked Read 8 Done Sent Done false
32668 9257 Acked Read 1 Done Sent Done false
32669 9258 Acked Read 3 Done Sent Done false
32670 9259 Acked Read 8 Done Sent Done false
32671 9260 Acked Read 8 Done Sent Done false
32672 9261 Acked Read 7 Done Sent Done false
32673 9262 Acked Read 7 Done Sent Done false
32674 9263 Acked Read 1 Done Sent Done false
32675 9264 Acked Read 8 Done Sent Done false
32676 9265 Acked Write 1 Done Sent Done false
32677 9266 Acked Write 1 Done Sent Done false
32678 9267 Acked Read 4 Done Sent Done false
32679 9268 Acked Read 8 Done Sent Done false
32680 9269 Acked Read 2 Done Sent Done false
32681 9270 Acked Flush 0 Done Sent Done false
32682 9271 Acked Flush 0 Done Sent Done false
32683 9272 Acked Read 5 Done Sent Done false
32684 9273 Acked Write 10 Done Sent Done false
32685 9274 Acked Read 3 Done Sent Done false
32686 9275 Acked Write 9 Done Sent Done false
32687 9276 Acked Read 3 Done Sent Done false
32688 9277 Acked Read 10 Done Sent Done false
32689 9278 Acked Read 8 Done Sent Done false
32690 9279 Acked Write 6 Done Sent Done false
32691 9280 Acked Read 3 Done Sent Done false
32692 9281 Acked Write 7 Done Sent Done false
32693 9282 Acked Write 9 Done Sent Done false
32694 9283 Acked Read 7 Done Sent Done false
32695 9284 Acked Read 1 Done Sent Done false
32696 9285 Acked Read 6 Done Sent Done false
32697 9286 Acked Flush 0 Done Sent Done false
32698 9287 Acked Write 5 Done Sent Done false
32699 9288 Acked Write 9 Done Sent Done false
32700 9289 Acked Read 7 Done Sent Done false
32701 9290 Acked Read 8 Done Sent Done false
32702 9291 Acked Read 4 Done Sent Done false
32703 9292 Acked Write 2 Done Sent Done false
32704 9293 Acked Read 7 Done Sent Done false
32705 9294 Acked Read 6 Done Sent Done false
32706 9295 Acked Flush 0 Done Sent Done false
32707 9296 Acked Read 2 Done Sent Done false
32708 9297 Acked Read 6 Done Sent Done false
32709 9298 Acked Read 4 Done Sent Done false
32710 9299 Acked Write 5 Done Sent Done false
32711 9300 Acked Read 5 Done Sent Done false
32712 9301 Acked Read 4 Done Sent Done false
32713 9302 Acked Write 2 Done Sent Done false
32714 9303 Acked Write 7 Done Sent Done false
32715 9304 Acked Read 1 Done Sent Done false
32716 9305 Acked Read 6 Done Sent Done false
32717 9306 Acked Flush 0 Done Sent Done false
32718 9307 Acked Write 6 Done Sent Done false
32719 9308 Acked Read 7 Done Sent Done false
32720 9309 Acked Write 7 Done Sent Done false
32721 9310 Acked Write 3 Done Sent Done false
32722 9311 Acked Read 6 Done Sent Done false
32723 9312 Acked Read 7 Done Sent Done false
32724 9313 Acked Write 5 Done Sent Done false
32725 9314 Acked Write 6 Done Sent Done false
32726 9315 Acked Write 1 Done Sent Done false
32727 9316 Acked Write 6 Done Sent Done false
32728 9317 Acked Read 2 Done Sent Done false
32729 9318 Acked Read 3 Done Sent Done false
32730 9319 Acked Read 9 Done Sent Done false
32731 9320 Acked Read 8 Done Sent Done false
32732 9321 Acked Read 2 Done Sent Done false
32733 9322 Acked Read 2 Done Sent Done false
32734 9323 Acked Read 3 Done Sent Done false
32735 9324 Acked Read 2 Done Sent Done false
32736 9325 Acked Write 10 Done Sent Done false
32737 9326 Acked Read 3 Done Sent Done false
32738 9327 Acked Read 2 Done Sent Done false
32739 9328 Acked Read 2 Done Sent Done false
32740 9329 Acked Write 8 Done Sent Done false
32741 9330 Acked Write 7 Done Sent Done false
32742 9331 Acked Write 5 Done Sent Done false
32743 9332 Acked Read 9 Done Sent Done false
32744 9333 Acked Write 10 Done Sent Done false
32745 9334 Acked Write 10 Done Sent Done false
32746 9335 Acked Write 2 Done Sent Done false
32747 9336 Acked Write 6 Done Sent Done false
32748 9337 Acked Read 2 Done Sent Done false
32749 9338 Acked Read 4 Done Sent Done false
32750 9339 Acked Flush 0 Done Sent Done false
32751 9340 Acked Read 2 Done Sent Done false
32752 9341 Acked Read 3 Done Sent Done false
32753 9342 Acked Read 5 Done Sent Done false
32754 9343 Acked Write 6 Done Sent Done false
32755 9344 Acked Write 5 Done Sent Done false
32756 9345 Acked Write 8 Done Sent Done false
32757 9346 Acked Write 3 Done Sent Done false
32758 9347 Acked Write 10 Done Sent Done false
32759 9348 Acked Read 3 Done Sent Done false
32760 9349 Acked Write 2 Done Sent Done false
32761 9350 Acked Read 3 Done Sent Done false
32762 9351 Acked Flush 0 Done Sent Done false
32763 9352 Acked Flush 0 Done Sent Done false
32764 9353 Acked Write 9 Done Sent Done false
32765 9354 Acked Write 4 Done Sent Done false
32766 9355 Acked Write 3 Done Sent Done false
32767 9356 Acked Write 8 Done Sent Done false
32768 9357 Acked Write 6 Done Sent Done false
32769 9358 Acked Read 2 Done Sent Done false
32770 9359 Acked Read 10 Done Sent Done false
32771 9360 Acked Write 2 Done Sent Done false
32772 9361 Acked Read 2 Done Sent Done false
32773 9362 Acked Read 10 Done Sent Done false
32774 9363 Acked Read 10 Done Sent Done false
32775 9364 Acked Read 2 Done Sent Done false
32776 9365 Acked Write 2 Done Sent Done false
32777 9366 Acked Write 3 Done Sent Done false
32778 9367 Acked Flush 0 Done Sent Done false
32779 9368 Acked Read 4 Done Sent Done false
32780 9369 Acked Read 6 Done Sent Done false
32781 9370 Acked Read 1 Done Sent Done false
32782 9371 Acked Read 8 Done Sent Done false
32783 9372 Acked Flush 0 Done Sent Done false
32784 9373 Acked Write 9 Done Sent Done false
32785 9374 Acked Read 2 Done Sent Done false
32786 9375 Acked Write 1 Done Sent Done false
32787 9376 Acked Read 3 Done Sent Done false
32788 9377 Acked Write 6 Done Sent Done false
32789 9378 Acked Write 9 Done Sent Done false
32790 9379 Acked Read 8 Done Sent Done false
32791 9380 Acked Read 4 Done Sent Done false
32792 9381 Acked Read 6 Done Sent Done false
32793 9382 Acked Write 4 Done Sent Done false
32794 9383 Acked Read 6 Done Sent Done false
32795 9384 Acked Write 10 Done Sent Done false
32796 9385 Acked Write 1 Done Sent Done false
32797 9386 Acked Read 3 Done Sent Done false
32798 9387 Acked Write 7 Done Sent Done false
32799 9388 Acked Write 10 Done Sent Done false
32800 9389 Acked Write 2 Done Sent Done false
32801 9390 Acked Write 2 Done Sent Done false
32802 9391 Acked Flush 0 Done Sent Done false
32803 9392 Acked Read 2 Done Sent Done false
32804 9393 Acked Write 2 Done Sent Done false
32805 9394 Acked Write 2 Done Sent Done false
32806 9395 Acked Read 7 Done Sent Done false
32807 9396 Acked Write 1 Done Sent Done false
32808 9397 Acked Write 1 Done Sent Done false
32809 9398 Acked Write 9 Done Sent Done false
32810 9399 Acked Write 2 Done Sent Done false
32811 9400 Acked Read 5 Done Sent Done false
32812 9401 Acked Read 7 Done Sent Done false
32813 9402 Acked Write 4 Done Sent Done false
32814 9403 Acked Write 7 Done Sent Done false
32815 9404 Acked Write 10 Done Sent Done false
32816 9405 Acked Read 4 Done Sent Done false
32817 9406 Acked Write 4 Done Sent Done false
32818 9407 Acked Write 6 Done Sent Done false
32819 9408 Acked Write 2 Done Sent Done false
32820 9409 Acked Read 1 Done Sent Done false
32821 9410 Acked Read 3 Done Sent Done false
32822 9411 Acked Read 9 Done Sent Done false
32823 9412 Acked Read 7 Done Sent Done false
32824 9413 Acked Flush 0 Done Sent Done false
32825 9414 Acked Write 4 Done Sent Done false
32826 9415 Acked Read 8 Done Sent Done false
32827 9416 Acked Write 2 Done Sent Done false
32828 9417 Acked Read 6 Done Sent Done false
32829 9418 Acked Flush 0 Done Sent Done false
32830 9419 Acked Read 7 Done Sent Done false
32831 9420 Acked Write 3 Done Sent Done false
32832 9421 Acked Write 8 Done Sent Done false
32833 9422 Acked Write 3 Done Sent Done false
32834 9423 Acked Flush 0 Done Sent Done false
32835 9424 Acked Read 1 Done Sent Done false
32836 9425 Acked Read 2 Done Sent Done false
32837 9426 Acked Flush 0 Done Sent Done false
32838 9427 Acked Read 4 Done Sent Done false
32839 9428 Acked Read 6 Done Sent Done false
32840 9429 Acked Write 3 Done Sent Done false
32841 9430 Acked Read 4 Done Sent Done false
32842 9431 Acked Write 8 Done Sent Done false
32843 9432 Acked Write 8 Done Sent Done false
32844 9433 Acked Read 5 Done Sent Done false
32845 9434 Acked Write 8 Done Sent Done false
32846 9435 Acked Write 7 Done Sent Done false
32847 9436 Acked Write 3 Done Sent Done false
32848 9437 Acked Read 7 Done Sent Done false
32849 9438 Acked Read 3 Done Sent Done false
32850 9439 Acked Flush 0 Done Sent Done false
32851 9440 Acked Write 9 Done Sent Done false
32852 9441 Acked Write 8 Done Sent Done false
32853 9442 Acked Read 10 Done Sent Done false
32854 9443 Acked Write 10 Done Sent Done false
32855 9444 Acked Read 7 Done Sent Done false
32856 9445 Acked Read 10 Done Sent Done false
32857 9446 Acked Flush 0 Done Sent Done false
32858 9447 Acked Write 5 Done Sent Done false
32859 9448 Acked Write 7 Done Sent Done false
32860 9449 Acked Read 3 Done Sent Done false
32861 9450 Acked Read 8 Done Sent Done false
32862 9451 Acked Write 9 Done Sent Done false
32863 9452 Acked Flush 0 Done Sent Done false
32864 STATES DS:0 DS:1 DS:2 TOTAL
32865 Sent 0 302 0 302
32866 Done 302 0 302 604
32867 Skipped 0 0 0 0
32868 Error 0 0 0 0
32869 Last Flush: 9452 9150 9452
32870 Downstairs last five completed: 9150 9149 9148 9147 9146
32871 Upstairs last five completed: 9452 9451 9450 9449 9448
32872 CLIENT: Up:0 ds:302 act:2
328732024-11-26T22:11:25.030ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
328742024-11-26T22:11:25.031ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:41905) = io task client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
328752024-11-26T22:11:25.031ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
328762024-11-26T22:11:25.032ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
328772024-11-26T22:11:25.032ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
328782024-11-26T22:11:25.032ZINFOcrucible: send last flush ID to this DS: 9150 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
328792024-11-26T22:11:25.032ZINFOcrucible: Replied this last flush ID: 9150 = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
328802024-11-26T22:11:25.032ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
328812024-11-26T22:11:25.032ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
328822024-11-26T22:11:25.032ZINFOcrucible: [1] client re-new 302 jobs since flush 9150 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
328832024-11-26T22:11:25.032ZINFOcrucible: [1] Replayed 302 jobs since flush: 9150 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
32884 ----------------------------------------------------------------
32885 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
32886 Downstairs last five completed: 9452 9451 9450 9449 9448
32887 Upstairs last five completed: 9452 9451 9450 9449 9448
32888 CLIENT: Up:0 ds:0 act:3
32889 Replay: All jobs finished
32890 Main task has work to do, go find it
32891 got dsc Stop(2)
32892 stop 2
32893 [8830] Got stop action so:false kr:false
32894 Nov 26 22:11:27.264 INFO request completed, latency_us: 333, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 72ceb1a0-ffd8-4447-b28f-d0c5cea7a19c, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
328952024-11-26T22:11:27.265ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
328962024-11-26T22:11:27.265ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
328972024-11-26T22:11:27.265ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
328982024-11-26T22:11:27.265ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
328992024-11-26T22:11:27.265ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
32900 Nov 26 22:11:27.265 INFO request completed, latency_us: 178, response_code: 200{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2024-11-26T22:11:27.265854054Z","hostname":"w-01JDN7ZQ17ABQG15KHKN3F2D26","pid":1203,"":"io task","client":"2","":"downstairs","session_id":"86a36c17-3718-484c-87fd-2b8a1dc2ab18"}
32901 , uri: /state/cid/2, method: GET, req_id: 9b92f4a3-39ab-4ce1-bd4a-3f0d38b2aa1b, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
32902 [8830] Exited with: ExitStatus(unix_wait_status(9))
32903 [8830][2] reports Exit
32904 Nov 26 22:11:31.268 INFO request completed, latency_us: 222, response_code: 200, uri: /state/cid/2, method: GET, req_id: 8e7eb95a-d58c-43c2-bc90-b0981074994a, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
32905 001/300 Write block 3650 len 28672 data: 4 4 4 7 6 4 3
32906 002/300 Read block 3554 len 40960
32907 003/300 Write block 1322 len 16384 data: 3 3 3 5
32908 004/300 Flush
32909 005/300 Read block 753 len 36864
32910 006/300 Write block 2843 len 28672 data: 8 8 8 10 7 7 5
32911 007/300 Read block 553 len 8192
32912 008/300 Write block 3843 len 4096 data: 3
32913 009/300 Read block 4279 len 4096
32914 010/300 Read block 2217 len 24576
32915 011/300 Read block 2147 len 16384
32916 012/300 Read block 71 len 4096
32917 013/300 Read block 1093 len 16384
32918 014/300 Read block 2537 len 36864
32919 015/300 Write block 1215 len 4096 data: 6
32920 016/300 Flush
32921 017/300 Read block 1569 len 28672
32922 018/300 Write block 2844 len 4096 data: 9
32923 019/300 Write block 3564 len 24576 data: 9 9 7 6 8 9
32924 020/300 Write block 3179 len 28672 data: 5 7 6 7 4 4 3
32925 021/300 Write block 1047 len 24576 data: 9 10 8 7 7 7
32926 022/300 Read block 568 len 16384
32927 023/300 Write block 4642 len 28672 data: 8 6 8 5 5 5 6
32928 024/300 Write block 4633 len 32768 data: 3 4 5 8 8 8 9 8
32929 025/300 Write block 285 len 4096 data: 4
32930 026/300 Read block 4468 len 40960
32931 027/300 Write block 2851 len 4096 data: 2
32932 028/300 Write block 3790 len 20480 data: 5 6 6 6 5
32933 029/300 Write block 4780 len 16384 data: 5 5 5 3
32934 030/300 Read block 2313 len 36864
32935 031/300 Read block 1218 len 32768
32936 032/300 Write block 4891 len 16384 data: 4 5 7 8
32937 033/300 Write block 3746 len 36864 data: 6 5 5 4 5 6 5 4 3
32938 034/300 Write block 4940 len 40960 data: 4 6 6 6 6 5 7 7 8 6
32939 035/300 Write block 3699 len 4096 data: 9
32940 036/300 Write block 115 len 4096 data: 5
32941 037/300 Write block 2501 len 8192 data: 7 6
32942 038/300 Write block 666 len 20480 data: 4 6 6 6 7
32943 039/300 Write block 567 len 12288 data: 5 5 5
32944 040/300 Write block 1813 len 12288 data: 7 6 6
32945 041/300 Flush
32946 042/300 Read block 200 len 28672
32947 043/300 Write block 559 len 40960 data: 4 4 5 4 4 6 6 5 6 6
32948 044/300 Read block 2395 len 20480
32949 045/300 Write block 4617 len 40960 data: 8 9 9 6 5 5 6 5 5 4
32950 046/300 Read block 4944 len 24576
32951 047/300 Read block 4440 len 32768
32952 048/300 Read block 2955 len 20480
32953 049/300 Flush
32954 050/300 Read block 2583 len 24576
32955 051/300 Read block 2792 len 24576
32956 052/300 Write block 535 len 28672 data: 5 7 7 7 10 10 10
32957 053/300 Write block 3788 len 16384 data: 4 4 6 7
32958 054/300 Read block 1766 len 32768
32959 055/300 Read block 2162 len 4096
32960 056/300 Write block 757 len 8192 data: 6 6
32961 057/300 Write block 4676 len 8192 data: 5 8
32962 058/300 Write block 1826 len 8192 data: 6 6
32963 059/300 Read block 3983 len 8192
32964 060/300 Read block 3239 len 28672
32965 061/300 Write block 4503 len 16384 data: 4 4 4 4
32966 062/300 Read block 71 len 4096
32967 063/300 Read block 3228 len 28672
32968 064/300 Read block 4522 len 12288
32969 065/300 Read block 4502 len 4096
32970 066/300 Write block 1964 len 4096 data: 4
32971 067/300 Write block 261 len 20480 data: 5 5 5 4 4
32972 068/300 Read block 307 len 28672
32973 069/300 Read block 4366 len 12288
32974 070/300 Read block 914 len 40960
32975 071/300 Flush
32976 072/300 Flush
32977 073/300 Write block 2501 len 40960 data: 8 7 6 7 7 5 4 4 3 3
32978 074/300 Read block 1765 len 20480
32979 075/300 Read block 3121 len 4096
32980 076/300 Read block 4164 len 16384
32981 077/300 Flush
32982 078/300 Write block 3746 len 4096 data: 7
32983 079/300 Write block 1526 len 12288 data: 8 7 7
32984 080/300 Write block 252 len 40960 data: 3 4 4 4 5 3 2 2 3 6
32985 081/300 Read block 3512 len 40960
32986 082/300 Read block 4951 len 4096
32987 083/300 Write block 4301 len 16384 data: 4 3 3 2
32988 084/300 Read block 3212 len 4096
32989 085/300 Write block 3354 len 12288 data: 6 6 4
32990 086/300 Write block 2399 len 36864 data: 7 6 8 7 7 7 7 5 5
32991 087/300 Read block 467 len 12288
32992 088/300 Write block 1466 len 40960 data: 5 5 6 4 3 5 6 7 7 7
32993 089/300 Read block 7 len 4096
32994 090/300 Read block 4462 len 8192
32995 091/300 Read block 4258 len 8192
32996 092/300 Read block 2855 len 28672
32997 093/300 Write block 617 len 12288 data: 5 4 3
32998 094/300 Read block 3552 len 24576
32999 095/300 Read block 1352 len 40960
33000 096/300 Write block 1452 len 40960 data: 4 4 5 7 8 8 7 5 6 6
33001 097/300 Read block 1504 len 8192
33002 098/300 Read block 654 len 16384
33003 099/300 Write block 2431 len 36864 data: 4 5 5 6 5 6 5 6 5
33004 100/300 Read block 186 len 36864
33005 101/300 Write block 1662 len 32768 data: 4 5 5 5 4 5 6 5
33006 102/300 Write block 4931 len 32768 data: 4 7 7 7 6 7 7 7
33007 103/300 Write block 1157 len 40960 data: 6 6 8 8 7 6 7 7 7 8
33008 104/300 Flush
33009 105/300 Write block 1435 len 4096 data: 7
33010 106/300 Write block 147 len 16384 data: 5 5 5 5
33011 107/300 Read block 1137 len 24576
33012 108/300 Write block 3720 len 4096 data: 6
33013 109/300 Read block 1249 len 24576
33014 110/300 Read block 3133 len 4096
33015 111/300 Read block 3061 len 36864
33016 112/300 Read block 1398 len 40960
33017 113/300 Read block 3328 len 28672
33018 114/300 Write block 2665 len 36864 data: 7 6 7 6 5 6 5 3 3
33019 115/300 Read block 2489 len 28672
33020 116/300 Flush
33021 117/300 Write block 4930 len 32768 data: 5 5 8 8 8 7 8 8
33022 118/300 Read block 4668 len 28672
33023 119/300 Write block 2117 len 24576 data: 6 7 6 5 4 5
33024 120/300 Read block 3988 len 24576
33025 121/300 Read block 3730 len 28672
33026 122/300 Read block 1166 len 16384
33027 123/300 Flush
33028 124/300 Read block 1653 len 24576
33029 125/300 Write block 1892 len 4096 data: 5
33030 126/300 Write block 1879 len 8192 data: 2 3
33031 127/300 Read block 1553 len 20480
33032 128/300 Read block 3464 len 40960
33033 129/300 Read block 1146 len 4096
33034 130/300 Write block 2710 len 16384 data: 8 8 8 8
33035 131/300 Write block 575 len 32768 data: 9 7 7 6 6 7 6 6
33036 132/300 Read block 1092 len 20480
33037 133/300 Write block 155 len 40960 data: 5 5 6 7 6 7 9 7 7 7
33038 134/300 Read block 1653 len 28672
33039 135/300 Read block 1435 len 16384
33040 136/300 Write block 4509 len 28672 data: 7 5 6 5 6 5 6
33041 137/300 Write block 912 len 20480 data: 8 7 6 5 6
33042 138/300 Read block 1457 len 24576
33043 139/300 Read block 1890 len 24576
33044 140/300 Read block 238 len 28672
33045 141/300 Write block 1406 len 36864 data: 6 5 4 4 5 5 5 5 6
33046 142/300 Read block 3421 len 28672
33047 143/300 Write block 1855 len 32768 data: 11 10 8 8 3 2 3 3
33048 144/300 Read block 2307 len 28672
33049 145/300 Write block 3739 len 12288 data: 3 5 5
33050 146/300 Read block 4071 len 32768
33051 147/300 Write block 4004 len 4096 data: 9
33052 148/300 Read block 1719 len 28672
33053 149/300 Write block 4622 len 4096 data: 6
33054 150/300 Read block 4898 len 32768
33055 151/300 Read block 4299 len 16384
33056 152/300 Write block 3818 len 36864 data: 2 2 3 3 3 4 4 6 6
33057 153/300 Read block 2765 len 40960
33058 154/300 Read block 3657 len 32768
33059 155/300 Read block 569 len 8192
33060 156/300 Read block 1527 len 36864
33061 157/300 Write block 1161 len 20480 data: 8 7 8 8 8
33062 158/300 Flush
33063 159/300 Read block 2570 len 28672
33064 160/300 Read block 4313 len 16384
33065 161/300 Flush
33066 162/300 Write block 233 len 4096 data: 5
33067 163/300 Read block 4238 len 20480
33068 164/300 Flush
33069 165/300 Read block 4708 len 32768
33070 166/300 Read block 4400 len 16384
33071 167/300 Write block 4540 len 36864 data: 7 5 4 3 4 5 5 8 7
33072 168/300 Write block 4581 len 12288 data: 5 7 7
33073 169/300 Read block 33 len 36864
33074 170/300 Read block 4422 len 20480
33075 171/300 Write block 1542 len 24576 data: 7 6 5 3 4 5
33076 172/300 Write block 3246 len 28672 data: 5 6 4 4 3 2 2
33077 173/300 Read block 1515 len 4096
33078 174/300 Read block 2203 len 32768
33079 175/300 Write block 3598 len 40960 data: 12 11 10 8 6 4 4 5 4 4
33080 176/300 Write block 4206 len 28672 data: 5 5 4 4 4 4 2
33081 177/300 Read block 1904 len 8192
33082 178/300 Write block 2599 len 16384 data: 10 10 8 7
33083 179/300 Write block 2387 len 12288 data: 6 6 7
33084 180/300 Write block 2627 len 24576 data: 5 4 5 5 5 5
33085 181/300 Write block 1723 len 28672 data: 7 7 6 6 6 4 7
33086 182/300 Write block 2843 len 8192 data: 9 10
33087 183/300 Write block 4964 len 8192 data: 8 10
33088 184/300 Read block 4871 len 24576
33089 185/300 Write block 28 len 20480 data: 5 4 3 2 2
33090 186/300 Read block 1318 len 8192
33091 187/300 Read block 3110 len 4096
33092 188/300 Write block 3077 len 24576 data: 2 2 4 5 6 10
33093 189/300 Read block 3732 len 16384
33094 190/300 Flush
33095 191/300 Read block 3528 len 36864
33096 192/300 Read block 408 len 28672
33097 193/300 Write block 2654 len 12288 data: 5 5 5
33098 194/300 Write block 4696 len 24576 data: 6 6 5 4 5 4
33099 195/300 Write block 2955 len 36864 data: 5 6 6 7 6 6 6 7 6
33100 196/300 Read block 4732 len 24576
33101 197/300 Flush
33102 198/300 Read block 4622 len 40960
33103 199/300 Write block 1320 len 8192 data: 4 3
33104 200/300 Write block 4670 len 16384 data: 3 4 4 4
33105 201/300 Write block 1321 len 20480 data: 4 4 4 4 6
33106 202/300 Read block 994 len 36864
33107 203/300 Write block 4283 len 20480 data: 6 5 4 4 3
33108 204/300 Write block 85 len 28672 data: 4 5 5 7 6 8 8
33109 205/300 Read block 4897 len 12288
33110 206/300 Write block 652 len 24576 data: 4 4 4 7 7 7
33111 207/300 Flush
33112 208/300 Flush
33113 209/300 Read block 220 len 24576
33114 210/300 Read block 4171 len 16384
33115 211/300 Flush
33116 212/300 Read block 170 len 16384
33117 213/300 Read block 2467 len 24576
33118 214/300 Flush
33119 215/300 Read block 187 len 8192
33120 216/300 Write block 3786 len 4096 data: 6
33121 217/300 Write block 2599 len 36864 data: 11 11 9 8 10 9 8 9 10
33122 218/300 Write block 3671 len 32768 data: 10 11 12 11 10 11 11 10
33123 219/300 Flush
33124 220/300 Write block 4541 len 12288 data: 6 5 4
33125 221/300 Read block 1754 len 16384
33126 222/300 Read block 2894 len 12288
33127 223/300 Write block 3750 len 36864 data: 6 7 6 5 4 4 4 6 6
33128 224/300 Write block 311 len 24576 data: 6 5 4 3 3 4
33129 225/300 Flush
33130 226/300 Write block 472 len 28672 data: 7 6 7 7 6 6 5
33131 227/300 Read block 4318 len 40960
33132 228/300 Write block 2757 len 12288 data: 4 8 7
33133 229/300 Write block 3625 len 36864 data: 6 5 5 5 7 6 8 8 7
33134 230/300 Write block 9 len 28672 data: 9 7 6 5 4 4 3
33135 231/300 Read block 1975 len 20480
33136 232/300 Write block 2413 len 28672 data: 4 4 3 3 3 3 3
33137 233/300 Write block 3163 len 8192 data: 7 7
33138 234/300 Flush
33139 235/300 Flush
33140 236/300 Write block 1807 len 32768 data: 7 6 7 6 6 7 8 7
33141 237/300 Read block 4704 len 32768
33142 238/300 Write block 1766 len 36864 data: 4 4 3 5 6 8 6 6 6
33143 239/300 Write block 1097 len 28672 data: 8 10 8 8 7 9 11
33144 240/300 Read block 876 len 12288
33145 241/300 Write block 1400 len 28672 data: 4 3 4 2 2 4 7
33146 242/300 Read block 4447 len 12288
33147 243/300 Write block 1655 len 28672 data: 6 6 6 7 7 8 6
33148 244/300 Read block 1200 len 40960
33149 245/300 Write block 475 len 24576 data: 8 7 7 6 7 4
33150 246/300 Read block 754 len 36864
33151 247/300 Flush
33152 248/300 Flush
33153 249/300 Flush
33154 250/300 Read block 1174 len 28672
33155 251/300 Read block 4493 len 40960
33156 252/300 Read block 1523 len 4096
33157 253/300 Write block 146 len 32768 data: 4 6 6 6 6 4 2 2
33158 254/300 Read block 3369 len 28672
33159 255/300 Read block 484 len 8192
33160 256/300 Write block 4212 len 36864 data: 3 2 2 2 3 4 4 5 5
33161 257/300 Write block 654 len 36864 data: 5 8 8 8 5 6 8 6 5
33162 258/300 Write block 3083 len 24576 data: 9 8 9 7 9 8
33163 259/300 Read block 3911 len 20480
33164 260/300 Read block 4373 len 8192
33165 261/300 Flush
33166 262/300 Write block 3661 len 4096 data: 5
33167 263/300 Write block 715 len 8192 data: 6 7
33168 264/300 Flush
33169 265/300 Write block 3469 len 4096 data: 4
33170 266/300 Read block 381 len 12288
33171 267/300 Read block 4155 len 4096
33172 268/300 Read block 3557 len 40960
33173 269/300 Write block 609 len 24576 data: 4 3 3 5 5 6
33174 270/300 Read block 1924 len 4096
33175 271/300 Write block 2187 len 4096 data: 4
33176 272/300 Write block 263 len 4096 data: 6
33177 273/300 Read block 1201 len 40960
33178 274/300 Read block 2639 len 36864
33179 275/300 Read block 2762 len 12288
33180 276/300 Write block 2793 len 4096 data: 3
33181 277/300 Write block 3376 len 24576 data: 6 5 5 6 4 5
33182 278/300 Write block 4806 len 8192 data: 8 8
33183 279/300 Read block 831 len 36864
33184 280/300 Read block 3462 len 16384
33185 281/300 Read block 778 len 32768
33186 282/300 Write block 2825 len 28672 data: 5 5 5 5 7 7 7
33187 283/300 Flush
33188 284/300 Read block 3276 len 4096
33189 285/300 Read block 518 len 28672
33190 286/300 Read block 4160 len 28672
33191 287/300 Flush
33192 288/300 Flush
33193 289/300 Read block 3217 len 40960
33194 290/300 Read block 3590 len 40960
33195 291/300 Read block 4530 len 32768
33196 292/300 Read block 4989 len 12288
33197 293/300 Read block 1243 len 40960
33198 294/300 Write block 600 len 24576 data: 5 6 5 5 4 4
33199 295/300 Read block 1650 len 4096
33200 296/300 Write block 3714 len 20480 data: 7 6 7 7 8
33201 297/300 Write block 1197 len 24576 data: 14 10 9 9 9 9
33202 298/300 Write block 4728 len 36864 data: 5 6 6 7 5 5 3 2 2
33203 299/300 Read block 393 len 24576
33204 300/300 Read block 4857 len 36864
33205 301/300 Read block 4618 len 8192
33206 Main task has work to do, go find it
33207 got dsc Start(2)
33208 start 2
33209 Nov 26 22:11:31.499 INFO request completed, latency_us: 188, response_code: 204, uri: /start/cid/2, method: POST, req_id: 896d6eef-bb0e-4a4f-847d-fa8923cc5f10, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
33210 Replay: started 2, returned:Ok(())
33211 ----------------------------------------------------------------
33212 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
33213 States: Active Active Offline
33214 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
33215 9453 Acked Write 7 Done Done Sent false
33216 9454 Acked Read 10 Done Done Sent false
33217 9455 Acked Write 4 Done Done Sent false
33218 9456 Acked Flush 0 Done Done Sent false
33219 9457 Acked Read 9 Done Done Sent false
33220 9458 Acked Write 7 Done Done Sent false
33221 9459 Acked Read 2 Done Done Sent false
33222 9460 Acked Write 1 Done Done Sent false
33223 9461 Acked Read 1 Done Done Sent false
33224 9462 Acked Read 6 Done Done Sent false
33225 9463 Acked Read 4 Done Done Sent false
33226 9464 Acked Read 1 Done Done Sent false
33227 9465 Acked Read 4 Done Done Sent false
33228 9466 Acked Read 9 Done Done Sent false
33229 9467 Acked Write 1 Done Done Sent false
33230 9468 Acked Flush 0 Done Done Sent false
33231 9469 Acked Read 7 Done Done Sent false
33232 9470 Acked Write 1 Done Done Sent false
33233 9471 Acked Write 6 Done Done Sent false
33234 9472 Acked Write 7 Done Done Sent false
33235 9473 Acked Write 6 Done Done Sent false
33236 9474 Acked Read 4 Done Done Sent false
33237 9475 Acked Write 7 Done Done Sent false
33238 9476 Acked Write 8 Done Done Sent false
33239 9477 Acked Write 1 Done Done Sent false
33240 9478 Acked Read 10 Done Done Sent false
33241 9479 Acked Write 1 Done Done Sent false
33242 9480 Acked Write 5 Done Done Sent false
33243 9481 Acked Write 4 Done Done Sent false
33244 9482 Acked Read 9 Done Done Sent false
33245 9483 Acked Read 8 Done Done Sent false
33246 9484 Acked Write 4 Done Done Sent false
33247 9485 Acked Write 9 Done Done Sent false
33248 9486 Acked Write 10 Done Done Sent false
33249 9487 Acked Write 1 Done Done Sent false
33250 9488 Acked Write 1 Done Done Sent false
33251 9489 Acked Write 2 Done Done Sent false
33252 9490 Acked Write 5 Done Done Sent false
33253 9491 Acked Write 3 Done Done Sent false
33254 9492 Acked Write 3 Done Done Sent false
33255 9493 Acked Flush 0 Done Done Sent false
33256 9494 Acked Read 7 Done Done Sent false
33257 9495 Acked Write 10 Done Done Sent false
33258 9496 Acked Read 5 Done Done Sent false
33259 9497 Acked Write 10 Done Done Sent false
33260 9498 Acked Read 6 Done Done Sent false
33261 9499 Acked Read 8 Done Done Sent false
33262 9500 Acked Read 5 Done Done Sent false
33263 9501 Acked Flush 0 Done Done Sent false
33264 9502 Acked Read 6 Done Done Sent false
33265 9503 Acked Read 6 Done Done Sent false
33266 9504 Acked Write 7 Done Done Sent false
33267 9505 Acked Write 4 Done Done Sent false
33268 9506 Acked Read 8 Done Done Sent false
33269 9507 Acked Read 1 Done Done Sent false
33270 9508 Acked Write 2 Done Done Sent false
33271 9509 Acked Write 2 Done Done Sent false
33272 9510 Acked Write 2 Done Done Sent false
33273 9511 Acked Read 2 Done Done Sent false
33274 9512 Acked Read 7 Done Done Sent false
33275 9513 Acked Write 4 Done Done Sent false
33276 9514 Acked Read 1 Done Done Sent false
33277 9515 Acked Read 7 Done Done Sent false
33278 9516 Acked Read 3 Done Done Sent false
33279 9517 Acked Read 1 Done Done Sent false
33280 9518 Acked Write 1 Done Done Sent false
33281 9519 Acked Write 5 Done Done Sent false
33282 9520 Acked Read 7 Done Done Sent false
33283 9521 Acked Read 3 Done Done Sent false
33284 9522 Acked Read 10 Done Done Sent false
33285 9523 Acked Flush 0 Done Done Sent false
33286 9524 Acked Flush 0 Done Done Sent false
33287 9525 Acked Write 10 Done Done Sent false
33288 9526 Acked Read 5 Done Done Sent false
33289 9527 Acked Read 1 Done Done Sent false
33290 9528 Acked Read 4 Done Done Sent false
33291 9529 Acked Flush 0 Done Done Sent false
33292 9530 Acked Write 1 Done Done Sent false
33293 9531 Acked Write 3 Done Done Sent false
33294 9532 Acked Write 10 Done Done Sent false
33295 9533 Acked Read 10 Done Done Sent false
33296 9534 Acked Read 1 Done Done Sent false
33297 9535 Acked Write 4 Done Done Sent false
33298 9536 Acked Read 1 Done Done Sent false
33299 9537 Acked Write 3 Done Done Sent false
33300 9538 Acked Write 9 Done Done Sent false
33301 9539 Acked Read 3 Done Done Sent false
33302 9540 Acked Write 10 Done Done Sent false
33303 9541 Acked Read 1 Done Done Sent false
33304 9542 Acked Read 2 Done Done Sent false
33305 9543 Acked Read 2 Done Done Sent false
33306 9544 Acked Read 7 Done Done Sent false
33307 9545 Acked Write 3 Done Done Sent false
33308 9546 Acked Read 6 Done Done Sent false
33309 9547 Acked Read 10 Done Done Sent false
33310 9548 Acked Write 10 Done Done Sent false
33311 9549 Acked Read 2 Done Done Sent false
33312 9550 Acked Read 4 Done Done Sent false
33313 9551 Acked Write 9 Done Done Sent false
33314 9552 Acked Read 9 Done Done Sent false
33315 9553 Acked Write 8 Done Done Sent false
33316 9554 Acked Write 8 Done Done Sent false
33317 9555 Acked Write 10 Done Done Sent false
33318 9556 Acked Flush 0 Done Done Sent false
33319 9557 Acked Write 1 Done Done Sent false
33320 9558 Acked Write 4 Done Done Sent false
33321 9559 Acked Read 6 Done Done Sent false
33322 9560 Acked Write 1 Done Done Sent false
33323 9561 Acked Read 6 Done Done Sent false
33324 9562 Acked Read 1 Done Done Sent false
33325 9563 Acked Read 9 Done Done Sent false
33326 9564 Acked Read 10 Done Done Sent false
33327 9565 Acked Read 7 Done Done Sent false
33328 9566 Acked Write 9 Done Done Sent false
33329 9567 Acked Read 7 Done Done Sent false
33330 9568 Acked Flush 0 Done Done Sent false
33331 9569 Acked Write 8 Done Done Sent false
33332 9570 Acked Read 7 Done Done Sent false
33333 9571 Acked Write 6 Done Done Sent false
33334 9572 Acked Read 6 Done Done Sent false
33335 9573 Acked Read 7 Done Done Sent false
33336 9574 Acked Read 4 Done Done Sent false
33337 9575 Acked Flush 0 Done Done Sent false
33338 9576 Acked Read 6 Done Done Sent false
33339 9577 Acked Write 1 Done Done Sent false
33340 9578 Acked Write 2 Done Done Sent false
33341 9579 Acked Read 5 Done Done Sent false
33342 9580 Acked Read 10 Done Done Sent false
33343 9581 Acked Read 1 Done Done Sent false
33344 9582 Acked Write 4 Done Done Sent false
33345 9583 Acked Write 8 Done Done Sent false
33346 9584 Acked Read 5 Done Done Sent false
33347 9585 Acked Write 10 Done Done Sent false
33348 9586 Acked Read 7 Done Done Sent false
33349 9587 Acked Read 4 Done Done Sent false
33350 9588 Acked Write 7 Done Done Sent false
33351 9589 Acked Write 5 Done Done Sent false
33352 9590 Acked Read 6 Done Done Sent false
33353 9591 Acked Read 6 Done Done Sent false
33354 9592 Acked Read 7 Done Done Sent false
33355 9593 Acked Write 9 Done Done Sent false
33356 9594 Acked Read 7 Done Done Sent false
33357 9595 Acked Write 8 Done Done Sent false
33358 9596 Acked Read 7 Done Done Sent false
33359 9597 Acked Write 3 Done Done Sent false
33360 9598 Acked Read 8 Done Done Sent false
33361 9599 Acked Write 1 Done Done Sent false
33362 9600 Acked Read 7 Done Done Sent false
33363 9601 Acked Write 1 Done Done Sent false
33364 9602 Acked Read 8 Done Done Sent false
33365 9603 Acked Read 4 Done Done Sent false
33366 9604 Acked Write 9 Done Done Sent false
33367 9605 Acked Read 10 Done Done Sent false
33368 9606 Acked Read 8 Done Done Sent false
33369 9607 Acked Read 2 Done Done Sent false
33370 9608 Acked Read 9 Done Done Sent false
33371 9609 Acked Write 5 Done Done Sent false
33372 9610 Acked Flush 0 Done Done Sent false
33373 9611 Acked Read 7 Done Done Sent false
33374 9612 Acked Read 4 Done Done Sent false
33375 9613 Acked Flush 0 Done Done Sent false
33376 9614 Acked Write 1 Done Done Sent false
33377 9615 Acked Read 5 Done Done Sent false
33378 9616 Acked Flush 0 Done Done Sent false
33379 9617 Acked Read 8 Done Done Sent false
33380 9618 Acked Read 4 Done Done Sent false
33381 9619 Acked Write 9 Done Done Sent false
33382 9620 Acked Write 3 Done Done Sent false
33383 9621 Acked Read 9 Done Done Sent false
33384 9622 Acked Read 5 Done Done Sent false
33385 9623 Acked Write 6 Done Done Sent false
33386 9624 Acked Write 7 Done Done Sent false
33387 9625 Acked Read 1 Done Done Sent false
33388 9626 Acked Read 8 Done Done Sent false
33389 9627 Acked Write 10 Done Done Sent false
33390 9628 Acked Write 7 Done Done Sent false
33391 9629 Acked Read 2 Done Done Sent false
33392 9630 Acked Write 4 Done Done Sent false
33393 9631 Acked Write 3 Done Done Sent false
33394 9632 Acked Write 6 Done Done Sent false
33395 9633 Acked Write 7 Done Done Sent false
33396 9634 Acked Write 2 Done Done Sent false
33397 9635 Acked Write 2 Done Done Sent false
33398 9636 Acked Read 6 Done Done Sent false
33399 9637 Acked Write 5 Done Done Sent false
33400 9638 Acked Read 2 Done Done Sent false
33401 9639 Acked Read 1 Done Done Sent false
33402 9640 Acked Write 6 Done Done Sent false
33403 9641 Acked Read 4 Done Done Sent false
33404 9642 Acked Flush 0 Done Done Sent false
33405 9643 Acked Read 9 Done Done Sent false
33406 9644 Acked Read 7 Done Done Sent false
33407 9645 Acked Write 3 Done Done Sent false
33408 9646 Acked Write 6 Done Done Sent false
33409 9647 Acked Write 9 Done Done Sent false
33410 9648 Acked Read 6 Done Done Sent false
33411 9649 Acked Flush 0 Done Done Sent false
33412 9650 Acked Read 10 Done Done Sent false
33413 9651 Acked Write 2 Done Done Sent false
33414 9652 Acked Write 4 Done Done Sent false
33415 9653 Acked Write 5 Done Done Sent false
33416 9654 Acked Read 9 Done Done Sent false
33417 9655 Acked Write 5 Done Done Sent false
33418 9656 Acked Write 7 Done Done Sent false
33419 9657 Acked Read 3 Done Done Sent false
33420 9658 Acked Write 6 Done Done Sent false
33421 9659 Acked Flush 0 Done Done Sent false
33422 9660 Acked Flush 0 Done Done Sent false
33423 9661 Acked Read 6 Done Done Sent false
33424 9662 Acked Read 4 Done Done Sent false
33425 9663 Acked Flush 0 Done Done Sent false
33426 9664 Acked Read 4 Done Done Sent false
33427 9665 Acked Read 6 Done Done Sent false
33428 9666 Acked Flush 0 Done Done Sent false
33429 9667 Acked Read 2 Done Done Sent false
33430 9668 Acked Write 1 Done Done Sent false
33431 9669 Acked Write 9 Done Done Sent false
33432 9670 Acked Write 8 Done Done Sent false
33433 9671 Acked Flush 0 Done Done Sent false
33434 9672 Acked Write 3 Done Done Sent false
33435 9673 Acked Read 4 Done Done Sent false
33436 9674 Acked Read 3 Done Done Sent false
33437 9675 Acked Write 9 Done Done Sent false
33438 9676 Acked Write 6 Done Done Sent false
33439 9677 Acked Flush 0 Done Done Sent false
33440 9678 Acked Write 7 Done Done Sent false
33441 9679 Acked Read 10 Done Done Sent false
33442 9680 Acked Write 3 Done Done Sent false
33443 9681 Acked Write 9 Done Done Sent false
33444 9682 Acked Write 7 Done Done Sent false
33445 9683 Acked Read 5 Done Done Sent false
33446 9684 Acked Write 7 Done Done Sent false
33447 9685 Acked Write 2 Done Done Sent false
33448 9686 Acked Flush 0 Done Done Sent false
33449 9687 Acked Flush 0 Done Done Sent false
33450 9688 Acked Write 8 Done Done Sent false
33451 9689 Acked Read 8 Done Done Sent false
33452 9690 Acked Write 9 Done Done Sent false
33453 9691 Acked Write 7 Done Done Sent false
33454 9692 Acked Read 3 Done Done Sent false
33455 9693 Acked Write 7 Done Done Sent false
33456 9694 Acked Read 3 Done Done Sent false
33457 9695 Acked Write 7 Done Done Sent false
33458 9696 Acked Read 10 Done Done Sent false
33459 9697 Acked Write 6 Done Done Sent false
33460 9698 Acked Read 9 Done Done Sent false
33461 9699 Acked Flush 0 Done Done Sent false
33462 9700 Acked Flush 0 Done Done Sent false
33463 9701 Acked Flush 0 Done Done Sent false
33464 9702 Acked Read 7 Done Done Sent false
33465 9703 Acked Read 10 Done Done Sent false
33466 9704 Acked Read 1 Done Done Sent false
33467 9705 Acked Write 8 Done Done Sent false
33468 9706 Acked Read 7 Done Done Sent false
33469 9707 Acked Read 2 Done Done Sent false
33470 9708 Acked Write 9 Done Done Sent false
33471 9709 Acked Write 9 Done Done Sent false
33472 9710 Acked Write 6 Done Done Sent false
33473 9711 Acked Read 5 Done Done Sent false
33474 9712 Acked Read 2 Done Done Sent false
33475 9713 Acked Flush 0 Done Done Sent false
33476 9714 Acked Write 1 Done Done Sent false
33477 9715 Acked Write 2 Done Done Sent false
33478 9716 Acked Flush 0 Done Done Sent false
33479 9717 Acked Write 1 Done Done Sent false
33480 9718 Acked Read 3 Done Done Sent false
33481 9719 Acked Read 1 Done Done Sent false
33482 9720 Acked Read 10 Done Done Sent false
33483 9721 Acked Write 6 Done Done Sent false
33484 9722 Acked Read 1 Done Done Sent false
33485 9723 Acked Write 1 Done Done Sent false
33486 9724 Acked Write 1 Done Done Sent false
33487 9725 Acked Read 10 Done Done Sent false
33488 9726 Acked Read 9 Done Done Sent false
33489 9727 Acked Read 3 Done Done Sent false
33490 9728 Acked Write 1 Done Done Sent false
33491 9729 Acked Write 6 Done Done Sent false
33492 9730 Acked Write 2 Done Done Sent false
33493 9731 Acked Read 9 Done Done Sent false
33494 9732 Acked Read 4 Done Done Sent false
33495 9733 Acked Read 8 Done Done Sent false
33496 9734 Acked Write 7 Done Done Sent false
33497 9735 Acked Flush 0 Done Done Sent false
33498 9736 Acked Read 1 Done Done Sent false
33499 9737 Acked Read 7 Done Done Sent false
33500 9738 Acked Read 7 Done Done Sent false
33501 9739 Acked Flush 0 Done Done Sent false
33502 9740 Acked Flush 0 Done Done Sent false
33503 9741 Acked Read 10 Done Done Sent false
33504 9742 Acked Read 10 Done Done Sent false
33505 9743 Acked Read 8 Done Done Sent false
33506 9744 Acked Read 3 Done Done Sent false
33507 9745 Acked Read 10 Done Done Sent false
33508 9746 Acked Write 6 Done Done Sent false
33509 9747 Acked Read 1 Done Done Sent false
33510 9748 Acked Write 5 Done Done Sent false
33511 9749 Acked Write 6 Done Done Sent false
33512 9750 Acked Write 9 Done Done Sent false
33513 9751 Acked Read 6 Done Done Sent false
33514 9752 Acked Read 9 Done Done Sent false
33515 9753 Acked Read 2 Done Done Sent false
33516 STATES DS:0 DS:1 DS:2 TOTAL
33517 Sent 0 0 301 301
33518 Done 301 301 0 602
33519 Skipped 0 0 0 0
33520 Error 0 0 0 0
33521 Last Flush: 9740 9740 9452
33522 Downstairs last five completed: 9452 9451 9450 9449 9448
33523 Upstairs last five completed: 9753 9752 9751 9750 9749
33524 CLIENT: Up:0 ds:301 act:2
33525 ----------------------------------------------------------------
33526 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
33527 States: Active Active Offline
33528 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
33529 9453 Acked Write 7 Done Done Sent false
33530 9454 Acked Read 10 Done Done Sent false
33531 9455 Acked Write 4 Done Done Sent false
33532 9456 Acked Flush 0 Done Done Sent false
33533 9457 Acked Read 9 Done Done Sent false
33534 9458 Acked Write 7 Done Done Sent false
33535 9459 Acked Read 2 Done Done Sent false
33536 9460 Acked Write 1 Done Done Sent false
33537 9461 Acked Read 1 Done Done Sent false
33538 9462 Acked Read 6 Done Done Sent false
33539 9463 Acked Read 4 Done Done Sent false
33540 9464 Acked Read 1 Done Done Sent false
33541 9465 Acked Read 4 Done Done Sent false
33542 9466 Acked Read 9 Done Done Sent false
33543 9467 Acked Write 1 Done Done Sent false
33544 9468 Acked Flush 0 Done Done Sent false
33545 9469 Acked Read 7 Done Done Sent false
33546 9470 Acked Write 1 Done Done Sent false
33547 9471 Acked Write 6 Done Done Sent false
33548 9472 Acked Write 7 Done Done Sent false
33549 9473 Acked Write 6 Done Done Sent false
33550 9474 Acked Read 4 Done Done Sent false
33551 9475 Acked Write 7 Done Done Sent false
33552 9476 Acked Write 8 Done Done Sent false
33553 9477 Acked Write 1 Done Done Sent false
33554 9478 Acked Read 10 Done Done Sent false
33555 9479 Acked Write 1 Done Done Sent false
33556 9480 Acked Write 5 Done Done Sent false
33557 9481 Acked Write 4 Done Done Sent false
33558 9482 Acked Read 9 Done Done Sent false
33559 9483 Acked Read 8 Done Done Sent false
33560 9484 Acked Write 4 Done Done Sent false
33561 9485 Acked Write 9 Done Done Sent false
33562 9486 Acked Write 10 Done Done Sent false
33563 9487 Acked Write 1 Done Done Sent false
33564 9488 Acked Write 1 Done Done Sent false
33565 9489 Acked Write 2 Done Done Sent false
33566 9490 Acked Write 5 Done Done Sent false
33567 9491 Acked Write 3 Done Done Sent false
33568 9492 Acked Write 3 Done Done Sent false
33569 9493 Acked Flush 0 Done Done Sent false
33570 9494 Acked Read 7 Done Done Sent false
33571 9495 Acked Write 10 Done Done Sent false
33572 9496 Acked Read 5 Done Done Sent false
33573 9497 Acked Write 10 Done Done Sent false
33574 9498 Acked Read 6 Done Done Sent false
33575 9499 Acked Read 8 Done Done Sent false
33576 9500 Acked Read 5 Done Done Sent false
33577 9501 Acked Flush 0 Done Done Sent false
33578 9502 Acked Read 6 Done Done Sent false
33579 9503 Acked Read 6 Done Done Sent false
33580 9504 Acked Write 7 Done Done Sent false
33581 9505 Acked Write 4 Done Done Sent false
33582 9506 Acked Read 8 Done Done Sent false
33583 9507 Acked Read 1 Done Done Sent false
33584 9508 Acked Write 2 Done Done Sent false
33585 9509 Acked Write 2 Done Done Sent false
33586 9510 Acked Write 2 Done Done Sent false
33587 9511 Acked Read 2 Done Done Sent false
33588 9512 Acked Read 7 Done Done Sent false
33589 9513 Acked Write 4 Done Done Sent false
33590 9514 Acked Read 1 Done Done Sent false
33591 9515 Acked Read 7 Done Done Sent false
33592 9516 Acked Read 3 Done Done Sent false
33593 9517 Acked Read 1 Done Done Sent false
33594 9518 Acked Write 1 Done Done Sent false
33595 9519 Acked Write 5 Done Done Sent false
33596 9520 Acked Read 7 Done Done Sent false
33597 9521 Acked Read 3 Done Done Sent false
33598 9522 Acked Read 10 Done Done Sent false
33599 9523 Acked Flush 0 Done Done Sent false
33600 9524 Acked Flush 0 Done Done Sent false
33601 9525 Acked Write 10 Done Done Sent false
33602 9526 Acked Read 5 Done Done Sent false
33603 9527 Acked Read 1 Done Done Sent false
33604 9528 Acked Read 4 Done Done Sent false
33605 9529 Acked Flush 0 Done Done Sent false
33606 9530 Acked Write 1 Done Done Sent false
33607 9531 Acked Write 3 Done Done Sent false
33608 9532 Acked Write 10 Done Done Sent false
33609 9533 Acked Read 10 Done Done Sent false
33610 9534 Acked Read 1 Done Done Sent false
33611 9535 Acked Write 4 Done Done Sent false
33612 9536 Acked Read 1 Done Done Sent false
33613 9537 Acked Write 3 Done Done Sent false
33614 9538 Acked Write 9 Done Done Sent false
33615 9539 Acked Read 3 Done Done Sent false
33616 9540 Acked Write 10 Done Done Sent false
33617 9541 Acked Read 1 Done Done Sent false
33618 9542 Acked Read 2 Done Done Sent false
33619 9543 Acked Read 2 Done Done Sent false
33620 9544 Acked Read 7 Done Done Sent false
33621 9545 Acked Write 3 Done Done Sent false
33622 9546 Acked Read 6 Done Done Sent false
33623 9547 Acked Read 10 Done Done Sent false
33624 9548 Acked Write 10 Done Done Sent false
33625 9549 Acked Read 2 Done Done Sent false
33626 9550 Acked Read 4 Done Done Sent false
33627 9551 Acked Write 9 Done Done Sent false
33628 9552 Acked Read 9 Done Done Sent false
33629 9553 Acked Write 8 Done Done Sent false
33630 9554 Acked Write 8 Done Done Sent false
33631 9555 Acked Write 10 Done Done Sent false
33632 9556 Acked Flush 0 Done Done Sent false
33633 9557 Acked Write 1 Done Done Sent false
33634 9558 Acked Write 4 Done Done Sent false
33635 9559 Acked Read 6 Done Done Sent false
33636 9560 Acked Write 1 Done Done Sent false
33637 9561 Acked Read 6 Done Done Sent false
33638 9562 Acked Read 1 Done Done Sent false
33639 9563 Acked Read 9 Done Done Sent false
33640 9564 Acked Read 10 Done Done Sent false
33641 9565 Acked Read 7 Done Done Sent false
33642 9566 Acked Write 9 Done Done Sent false
33643 9567 Acked Read 7 Done Done Sent false
33644 9568 Acked Flush 0 Done Done Sent false
33645 9569 Acked Write 8 Done Done Sent false
33646 9570 Acked Read 7 Done Done Sent false
33647 9571 Acked Write 6 Done Done Sent false
33648 9572 Acked Read 6 Done Done Sent false
33649 9573 Acked Read 7 Done Done Sent false
33650 9574 Acked Read 4 Done Done Sent false
33651 9575 Acked Flush 0 Done Done Sent false
33652 9576 Acked Read 6 Done Done Sent false
33653 9577 Acked Write 1 Done Done Sent false
33654 9578 Acked Write 2 Done Done Sent false
33655 9579 Acked Read 5 Done Done Sent false
33656 9580 Acked Read 10 Done Done Sent false
33657 9581 Acked Read 1 Done Done Sent false
33658 9582 Acked Write 4 Done Done Sent false
33659 9583 Acked Write 8 Done Done Sent false
33660 9584 Acked Read 5 Done Done Sent false
33661 9585 Acked Write 10 Done Done Sent false
33662 9586 Acked Read 7 Done Done Sent false
33663 9587 Acked Read 4 Done Done Sent false
33664 9588 Acked Write 7 Done Done Sent false
33665 9589 Acked Write 5 Done Done Sent false
33666 9590 Acked Read 6 Done Done Sent false
33667 9591 Acked Read 6 Done Done Sent false
33668 9592 Acked Read 7 Done Done Sent false
33669 9593 Acked Write 9 Done Done Sent false
33670 9594 Acked Read 7 Done Done Sent false
33671 9595 Acked Write 8 Done Done Sent false
33672 9596 Acked Read 7 Done Done Sent false
33673 9597 Acked Write 3 Done Done Sent false
33674 9598 Acked Read 8 Done Done Sent false
33675 9599 Acked Write 1 Done Done Sent false
33676 9600 Acked Read 7 Done Done Sent false
33677 9601 Acked Write 1 Done Done Sent false
33678 9602 Acked Read 8 Done Done Sent false
33679 9603 Acked Read 4 Done Done Sent false
33680 9604 Acked Write 9 Done Done Sent false
33681 9605 Acked Read 10 Done Done Sent false
33682 9606 Acked Read 8 Done Done Sent false
33683 9607 Acked Read 2 Done Done Sent false
33684 9608 Acked Read 9 Done Done Sent false
33685 9609 Acked Write 5 Done Done Sent false
33686 9610 Acked Flush 0 Done Done Sent false
33687 9611 Acked Read 7 Done Done Sent false
33688 9612 Acked Read 4 Done Done Sent false
33689 9613 Acked Flush 0 Done Done Sent false
33690 9614 Acked Write 1 Done Done Sent false
33691 9615 Acked Read 5 Done Done Sent false
33692 9616 Acked Flush 0 Done Done Sent false
33693 9617 Acked Read 8 Done Done Sent false
33694 9618 Acked Read 4 Done Done Sent false
33695 9619 Acked Write 9 Done Done Sent false
33696 9620 Acked Write 3 Done Done Sent false
33697 9621 Acked Read 9 Done Done Sent false
33698 9622 Acked Read 5 Done Done Sent false
33699 9623 Acked Write 6 Done Done Sent false
33700 9624 Acked Write 7 Done Done Sent false
33701 9625 Acked Read 1 Done Done Sent false
33702 9626 Acked Read 8 Done Done Sent false
33703 9627 Acked Write 10 Done Done Sent false
33704 9628 Acked Write 7 Done Done Sent false
33705 9629 Acked Read 2 Done Done Sent false
33706 9630 Acked Write 4 Done Done Sent false
33707 9631 Acked Write 3 Done Done Sent false
33708 9632 Acked Write 6 Done Done Sent false
33709 9633 Acked Write 7 Done Done Sent false
33710 9634 Acked Write 2 Done Done Sent false
33711 9635 Acked Write 2 Done Done Sent false
33712 9636 Acked Read 6 Done Done Sent false
33713 9637 Acked Write 5 Done Done Sent false
33714 9638 Acked Read 2 Done Done Sent false
33715 9639 Acked Read 1 Done Done Sent false
33716 9640 Acked Write 6 Done Done Sent false
33717 9641 Acked Read 4 Done Done Sent false
33718 9642 Acked Flush 0 Done Done Sent false
33719 9643 Acked Read 9 Done Done Sent false
33720 9644 Acked Read 7 Done Done Sent false
33721 9645 Acked Write 3 Done Done Sent false
33722 9646 Acked Write 6 Done Done Sent false
33723 9647 Acked Write 9 Done Done Sent false
33724 9648 Acked Read 6 Done Done Sent false
33725 9649 Acked Flush 0 Done Done Sent false
33726 9650 Acked Read 10 Done Done Sent false
33727 9651 Acked Write 2 Done Done Sent false
33728 9652 Acked Write 4 Done Done Sent false
33729 9653 Acked Write 5 Done Done Sent false
33730 9654 Acked Read 9 Done Done Sent false
33731 9655 Acked Write 5 Done Done Sent false
33732 9656 Acked Write 7 Done Done Sent false
33733 9657 Acked Read 3 Done Done Sent false
33734 9658 Acked Write 6 Done Done Sent false
33735 9659 Acked Flush 0 Done Done Sent false
33736 9660 Acked Flush 0 Done Done Sent false
33737 9661 Acked Read 6 Done Done Sent false
33738 9662 Acked Read 4 Done Done Sent false
33739 9663 Acked Flush 0 Done Done Sent false
33740 9664 Acked Read 4 Done Done Sent false
33741 9665 Acked Read 6 Done Done Sent false
33742 9666 Acked Flush 0 Done Done Sent false
33743 9667 Acked Read 2 Done Done Sent false
33744 9668 Acked Write 1 Done Done Sent false
33745 9669 Acked Write 9 Done Done Sent false
33746 9670 Acked Write 8 Done Done Sent false
33747 9671 Acked Flush 0 Done Done Sent false
33748 9672 Acked Write 3 Done Done Sent false
33749 9673 Acked Read 4 Done Done Sent false
33750 9674 Acked Read 3 Done Done Sent false
33751 9675 Acked Write 9 Done Done Sent false
33752 9676 Acked Write 6 Done Done Sent false
33753 9677 Acked Flush 0 Done Done Sent false
33754 9678 Acked Write 7 Done Done Sent false
33755 9679 Acked Read 10 Done Done Sent false
33756 9680 Acked Write 3 Done Done Sent false
33757 9681 Acked Write 9 Done Done Sent false
33758 9682 Acked Write 7 Done Done Sent false
33759 9683 Acked Read 5 Done Done Sent false
33760 9684 Acked Write 7 Done Done Sent false
33761 9685 Acked Write 2 Done Done Sent false
33762 9686 Acked Flush 0 Done Done Sent false
33763 9687 Acked Flush 0 Done Done Sent false
33764 9688 Acked Write 8 Done Done Sent false
33765 9689 Acked Read 8 Done Done Sent false
33766 9690 Acked Write 9 Done Done Sent false
33767 9691 Acked Write 7 Done Done Sent false
33768 9692 Acked Read 3 Done Done Sent false
33769 9693 Acked Write 7 Done Done Sent false
33770 9694 Acked Read 3 Done Done Sent false
33771 9695 Acked Write 7 Done Done Sent false
33772 9696 Acked Read 10 Done Done Sent false
33773 9697 Acked Write 6 Done Done Sent false
33774 9698 Acked Read 9 Done Done Sent false
33775 9699 Acked Flush 0 Done Done Sent false
33776 9700 Acked Flush 0 Done Done Sent false
33777 9701 Acked Flush 0 Done Done Sent false
33778 9702 Acked Read 7 Done Done Sent false
33779 9703 Acked Read 10 Done Done Sent false
33780 9704 Acked Read 1 Done Done Sent false
33781 9705 Acked Write 8 Done Done Sent false
33782 9706 Acked Read 7 Done Done Sent false
33783 9707 Acked Read 2 Done Done Sent false
33784 9708 Acked Write 9 Done Done Sent false
33785 9709 Acked Write 9 Done Done Sent false
33786 9710 Acked Write 6 Done Done Sent false
33787 9711 Acked Read 5 Done Done Sent false
33788 9712 Acked Read 2 Done Done Sent false
33789 9713 Acked Flush 0 Done Done Sent false
33790 9714 Acked Write 1 Done Done Sent false
33791 9715 Acked Write 2 Done Done Sent false
33792 9716 Acked Flush 0 Done Done Sent false
33793 9717 Acked Write 1 Done Done Sent false
33794 9718 Acked Read 3 Done Done Sent false
33795 9719 Acked Read 1 Done Done Sent false
33796 9720 Acked Read 10 Done Done Sent false
33797 9721 Acked Write 6 Done Done Sent false
33798 9722 Acked Read 1 Done Done Sent false
33799 9723 Acked Write 1 Done Done Sent false
33800 9724 Acked Write 1 Done Done Sent false
33801 9725 Acked Read 10 Done Done Sent false
33802 9726 Acked Read 9 Done Done Sent false
33803 9727 Acked Read 3 Done Done Sent false
33804 9728 Acked Write 1 Done Done Sent false
33805 9729 Acked Write 6 Done Done Sent false
33806 9730 Acked Write 2 Done Done Sent false
33807 9731 Acked Read 9 Done Done Sent false
33808 9732 Acked Read 4 Done Done Sent false
33809 9733 Acked Read 8 Done Done Sent false
33810 9734 Acked Write 7 Done Done Sent false
33811 9735 Acked Flush 0 Done Done Sent false
33812 9736 Acked Read 1 Done Done Sent false
33813 9737 Acked Read 7 Done Done Sent false
33814 9738 Acked Read 7 Done Done Sent false
33815 9739 Acked Flush 0 Done Done Sent false
33816 9740 Acked Flush 0 Done Done Sent false
33817 9741 Acked Read 10 Done Done Sent false
33818 9742 Acked Read 10 Done Done Sent false
33819 9743 Acked Read 8 Done Done Sent false
33820 9744 Acked Read 3 Done Done Sent false
33821 9745 Acked Read 10 Done Done Sent false
33822 9746 Acked Write 6 Done Done Sent false
33823 9747 Acked Read 1 Done Done Sent false
33824 9748 Acked Write 5 Done Done Sent false
33825 9749 Acked Write 6 Done Done Sent false
33826 9750 Acked Write 9 Done Done Sent false
33827 9751 Acked Read 6 Done Done Sent false
33828 9752 Acked Read 9 Done Done Sent false
33829 9753 Acked Read 2 Done Done Sent false
33830 9754 Acked Flush 0 Done Done Sent false
33831 STATES DS:0 DS:1 DS:2 TOTAL
33832 Sent 0 0 302 302
33833 Done 302 302 0 604
33834 Skipped 0 0 0 0
33835 Error 0 0 0 0
33836 Last Flush: 9754 9754 9452
33837 Downstairs last five completed: 9452 9451 9450 9449 9448
33838 Upstairs last five completed: 9754 9753 9752 9751 9750
33839 CLIENT: Up:0 ds:302 act:2
338402024-11-26T22:11:37.266ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
338412024-11-26T22:11:37.267ZWARNcrucible: ds_connection connect to 127.0.0.1:8830 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
338422024-11-26T22:11:37.267ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
338432024-11-26T22:11:37.267ZWARNcrucible: downstairs task for 2 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
338442024-11-26T22:11:37.267ZINFOcrucible: client task is exiting = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
33845 ----------------------------------------------------------------
33846 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
33847 States: Active Active Offline
33848 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
33849 9453 Acked Write 7 Done Done Sent false
33850 9454 Acked Read 10 Done Done Sent false
33851 9455 Acked Write 4 Done Done Sent false
33852 9456 Acked Flush 0 Done Done Sent false
33853 9457 Acked Read 9 Done Done Sent false
33854 9458 Acked Write 7 Done Done Sent false
33855 9459 Acked Read 2 Done Done Sent false
33856 9460 Acked Write 1 Done Done Sent false
33857 9461 Acked Read 1 Done Done Sent false
33858 9462 Acked Read 6 Done Done Sent false
33859 9463 Acked Read 4 Done Done Sent false
33860 9464 Acked Read 1 Done Done Sent false
33861 9465 Acked Read 4 Done Done Sent false
33862 9466 Acked Read 9 Done Done Sent false
33863 9467 Acked Write 1 Done Done Sent false
33864 9468 Acked Flush 0 Done Done Sent false
33865 9469 Acked Read 7 Done Done Sent false
33866 9470 Acked Write 1 Done Done Sent false
33867 9471 Acked Write 6 Done Done Sent false
33868 9472 Acked Write 7 Done Done Sent false
33869 9473 Acked Write 6 Done Done Sent false
33870 9474 Acked Read 4 Done Done Sent false
33871 9475 Acked Write 7 Done Done Sent false
33872 9476 Acked Write 8 Done Done Sent false
33873 9477 Acked Write 1 Done Done Sent false
33874 9478 Acked Read 10 Done Done Sent false
33875 9479 Acked Write 1 Done Done Sent false
33876 9480 Acked Write 5 Done Done Sent false
33877 9481 Acked Write 4 Done Done Sent false
33878 9482 Acked Read 9 Done Done Sent false
33879 9483 Acked Read 8 Done Done Sent false
33880 9484 Acked Write 4 Done Done Sent false
33881 9485 Acked Write 9 Done Done Sent false
33882 9486 Acked Write 10 Done Done Sent false
33883 9487 Acked Write 1 Done Done Sent false
33884 9488 Acked Write 1 Done Done Sent false
33885 9489 Acked Write 2 Done Done Sent false
33886 9490 Acked Write 5 Done Done Sent false
33887 9491 Acked Write 3 Done Done Sent false
33888 9492 Acked Write 3 Done Done Sent false
33889 9493 Acked Flush 0 Done Done Sent false
33890 9494 Acked Read 7 Done Done Sent false
33891 9495 Acked Write 10 Done Done Sent false
33892 9496 Acked Read 5 Done Done Sent false
33893 9497 Acked Write 10 Done Done Sent false
33894 9498 Acked Read 6 Done Done Sent false
33895 9499 Acked Read 8 Done Done Sent false
33896 9500 Acked Read 5 Done Done Sent false
33897 9501 Acked Flush 0 Done Done Sent false
33898 9502 Acked Read 6 Done Done Sent false
33899 9503 Acked Read 6 Done Done Sent false
33900 9504 Acked Write 7 Done Done Sent false
33901 9505 Acked Write 4 Done Done Sent false
33902 9506 Acked Read 8 Done Done Sent false
33903 9507 Acked Read 1 Done Done Sent false
33904 9508 Acked Write 2 Done Done Sent false
33905 9509 Acked Write 2 Done Done Sent false
33906 9510 Acked Write 2 Done Done Sent false
33907 9511 Acked Read 2 Done Done Sent false
33908 9512 Acked Read 7 Done Done Sent false
33909 9513 Acked Write 4 Done Done Sent false
33910 9514 Acked Read 1 Done Done Sent false
33911 9515 Acked Read 7 Done Done Sent false
33912 9516 Acked Read 3 Done Done Sent false
33913 9517 Acked Read 1 Done Done Sent false
33914 9518 Acked Write 1 Done Done Sent false
33915 9519 Acked Write 5 Done Done Sent false
33916 9520 Acked Read 7 Done Done Sent false
33917 9521 Acked Read 3 Done Done Sent false
33918 9522 Acked Read 10 Done Done Sent false
33919 9523 Acked Flush 0 Done Done Sent false
33920 9524 Acked Flush 0 Done Done Sent false
33921 9525 Acked Write 10 Done Done Sent false
33922 9526 Acked Read 5 Done Done Sent false
33923 9527 Acked Read 1 Done Done Sent false
33924 9528 Acked Read 4 Done Done Sent false
33925 9529 Acked Flush 0 Done Done Sent false
33926 9530 Acked Write 1 Done Done Sent false
33927 9531 Acked Write 3 Done Done Sent false
33928 9532 Acked Write 10 Done Done Sent false
33929 9533 Acked Read 10 Done Done Sent false
33930 9534 Acked Read 1 Done Done Sent false
33931 9535 Acked Write 4 Done Done Sent false
33932 9536 Acked Read 1 Done Done Sent false
33933 9537 Acked Write 3 Done Done Sent false
33934 9538 Acked Write 9 Done Done Sent false
33935 9539 Acked Read 3 Done Done Sent false
33936 9540 Acked Write 10 Done Done Sent false
33937 9541 Acked Read 1 Done Done Sent false
33938 9542 Acked Read 2 Done Done Sent false
33939 9543 Acked Read 2 Done Done Sent false
33940 9544 Acked Read 7 Done Done Sent false
33941 9545 Acked Write 3 Done Done Sent false
33942 9546 Acked Read 6 Done Done Sent false
33943 9547 Acked Read 10 Done Done Sent false
33944 9548 Acked Write 10 Done Done Sent false
33945 9549 Acked Read 2 Done Done Sent false
33946 9550 Acked Read 4 Done Done Sent false
33947 9551 Acked Write 9 Done Done Sent false
33948 9552 Acked Read 9 Done Done Sent false
33949 9553 Acked Write 8 Done Done Sent false
33950 9554 Acked Write 8 Done Done Sent false
33951 9555 Acked Write 10 Done Done Sent false
33952 9556 Acked Flush 0 Done Done Sent false
33953 9557 Acked Write 1 Done Done Sent false
33954 9558 Acked Write 4 Done Done Sent false
33955 9559 Acked Read 6 Done Done Sent false
33956 9560 Acked Write 1 Done Done Sent false
33957 9561 Acked Read 6 Done Done Sent false
33958 9562 Acked Read 1 Done Done Sent false
33959 9563 Acked Read 9 Done Done Sent false
33960 9564 Acked Read 10 Done Done Sent false
33961 9565 Acked Read 7 Done Done Sent false
33962 9566 Acked Write 9 Done Done Sent false
33963 9567 Acked Read 7 Done Done Sent false
33964 9568 Acked Flush 0 Done Done Sent false
33965 9569 Acked Write 8 Done Done Sent false
33966 9570 Acked Read 7 Done Done Sent false
33967 9571 Acked Write 6 Done Done Sent false
33968 9572 Acked Read 6 Done Done Sent false
33969 9573 Acked Read 7 Done Done Sent false
33970 9574 Acked Read 4 Done Done Sent false
33971 9575 Acked Flush 0 Done Done Sent false
33972 9576 Acked Read 6 Done Done Sent false
33973 9577 Acked Write 1 Done Done Sent false
33974 9578 Acked Write 2 Done Done Sent false
33975 9579 Acked Read 5 Done Done Sent false
33976 9580 Acked Read 10 Done Done Sent false
33977 9581 Acked Read 1 Done Done Sent false
33978 9582 Acked Write 4 Done Done Sent false
33979 9583 Acked Write 8 Done Done Sent false
33980 9584 Acked Read 5 Done Done Sent false
33981 9585 Acked Write 10 Done Done Sent false
33982 9586 Acked Read 7 Done Done Sent false
33983 9587 Acked Read 4 Done Done Sent false
33984 9588 Acked Write 7 Done Done Sent false
33985 9589 Acked Write 5 Done Done Sent false
33986 9590 Acked Read 6 Done Done Sent false
33987 9591 Acked Read 6 Done Done Sent false
33988 9592 Acked Read 7 Done Done Sent false
33989 9593 Acked Write 9 Done Done Sent false
33990 9594 Acked Read 7 Done Done Sent false
33991 9595 Acked Write 8 Done Done Sent false
33992 9596 Acked Read 7 Done Done Sent false
33993 9597 Acked Write 3 Done Done Sent false
33994 9598 Acked Read 8 Done Done Sent false
33995 9599 Acked Write 1 Done Done Sent false
33996 9600 Acked Read 7 Done Done Sent false
33997 9601 Acked Write 1 Done Done Sent false
33998 9602 Acked Read 8 Done Done Sent false
33999 9603 Acked Read 4 Done Done Sent false
34000 9604 Acked Write 9 Done Done Sent false
34001 9605 Acked Read 10 Done Done Sent false
34002 9606 Acked Read 8 Done Done Sent false
34003 9607 Acked Read 2 Done Done Sent false
34004 9608 Acked Read 9 Done Done Sent false
34005 9609 Acked Write 5 Done Done Sent false
34006 9610 Acked Flush 0 Done Done Sent false
34007 9611 Acked Read 7 Done Done Sent false
34008 9612 Acked Read 4 Done Done Sent false
34009 9613 Acked Flush 0 Done Done Sent false
34010 9614 Acked Write 1 Done Done Sent false
34011 9615 Acked Read 5 Done Done Sent false
34012 9616 Acked Flush 0 Done Done Sent false
34013 9617 Acked Read 8 Done Done Sent false
34014 9618 Acked Read 4 Done Done Sent false
34015 9619 Acked Write 9 Done Done Sent false
34016 9620 Acked Write 3 Done Done Sent false
34017 9621 Acked Read 9 Done Done Sent false
34018 9622 Acked Read 5 Done Done Sent false
34019 9623 Acked Write 6 Done Done Sent false
34020 9624 Acked Write 7 Done Done Sent false
34021 9625 Acked Read 1 Done Done Sent false
34022 9626 Acked Read 8 Done Done Sent false
34023 9627 Acked Write 10 Done Done Sent false
34024 9628 Acked Write 7 Done Done Sent false
34025 9629 Acked Read 2 Done Done Sent false
34026 9630 Acked Write 4 Done Done Sent false
34027 9631 Acked Write 3 Done Done Sent false
34028 9632 Acked Write 6 Done Done Sent false
34029 9633 Acked Write 7 Done Done Sent false
34030 9634 Acked Write 2 Done Done Sent false
34031 9635 Acked Write 2 Done Done Sent false
34032 9636 Acked Read 6 Done Done Sent false
34033 9637 Acked Write 5 Done Done Sent false
34034 9638 Acked Read 2 Done Done Sent false
34035 9639 Acked Read 1 Done Done Sent false
34036 9640 Acked Write 6 Done Done Sent false
34037 9641 Acked Read 4 Done Done Sent false
34038 9642 Acked Flush 0 Done Done Sent false
34039 9643 Acked Read 9 Done Done Sent false
34040 9644 Acked Read 7 Done Done Sent false
34041 9645 Acked Write 3 Done Done Sent false
34042 9646 Acked Write 6 Done Done Sent false
34043 9647 Acked Write 9 Done Done Sent false
34044 9648 Acked Read 6 Done Done Sent false
34045 9649 Acked Flush 0 Done Done Sent false
34046 9650 Acked Read 10 Done Done Sent false
34047 9651 Acked Write 2 Done Done Sent false
34048 9652 Acked Write 4 Done Done Sent false
34049 9653 Acked Write 5 Done Done Sent false
34050 9654 Acked Read 9 Done Done Sent false
34051 9655 Acked Write 5 Done Done Sent false
34052 9656 Acked Write 7 Done Done Sent false
34053 9657 Acked Read 3 Done Done Sent false
34054 9658 Acked Write 6 Done Done Sent false
34055 9659 Acked Flush 0 Done Done Sent false
34056 9660 Acked Flush 0 Done Done Sent false
34057 9661 Acked Read 6 Done Done Sent false
34058 9662 Acked Read 4 Done Done Sent false
34059 9663 Acked Flush 0 Done Done Sent false
34060 9664 Acked Read 4 Done Done Sent false
34061 9665 Acked Read 6 Done Done Sent false
34062 9666 Acked Flush 0 Done Done Sent false
34063 9667 Acked Read 2 Done Done Sent false
34064 9668 Acked Write 1 Done Done Sent false
34065 9669 Acked Write 9 Done Done Sent false
34066 9670 Acked Write 8 Done Done Sent false
34067 9671 Acked Flush 0 Done Done Sent false
34068 9672 Acked Write 3 Done Done Sent false
34069 9673 Acked Read 4 Done Done Sent false
34070 9674 Acked Read 3 Done Done Sent false
34071 9675 Acked Write 9 Done Done Sent false
34072 9676 Acked Write 6 Done Done Sent false
34073 9677 Acked Flush 0 Done Done Sent false
34074 9678 Acked Write 7 Done Done Sent false
34075 9679 Acked Read 10 Done Done Sent false
34076 9680 Acked Write 3 Done Done Sent false
34077 9681 Acked Write 9 Done Done Sent false
34078 9682 Acked Write 7 Done Done Sent false
34079 9683 Acked Read 5 Done Done Sent false
34080 9684 Acked Write 7 Done Done Sent false
34081 9685 Acked Write 2 Done Done Sent false
34082 9686 Acked Flush 0 Done Done Sent false
34083 9687 Acked Flush 0 Done Done Sent false
34084 9688 Acked Write 8 Done Done Sent false
34085 9689 Acked Read 8 Done Done Sent false
34086 9690 Acked Write 9 Done Done Sent false
34087 9691 Acked Write 7 Done Done Sent false
34088 9692 Acked Read 3 Done Done Sent false
34089 9693 Acked Write 7 Done Done Sent false
34090 9694 Acked Read 3 Done Done Sent false
34091 9695 Acked Write 7 Done Done Sent false
34092 9696 Acked Read 10 Done Done Sent false
34093 9697 Acked Write 6 Done Done Sent false
34094 9698 Acked Read 9 Done Done Sent false
34095 9699 Acked Flush 0 Done Done Sent false
34096 9700 Acked Flush 0 Done Done Sent false
34097 9701 Acked Flush 0 Done Done Sent false
34098 9702 Acked Read 7 Done Done Sent false
34099 9703 Acked Read 10 Done Done Sent false
34100 9704 Acked Read 1 Done Done Sent false
34101 9705 Acked Write 8 Done Done Sent false
34102 9706 Acked Read 7 Done Done Sent false
34103 9707 Acked Read 2 Done Done Sent false
34104 9708 Acked Write 9 Done Done Sent false
34105 9709 Acked Write 9 Done Done Sent false
34106 9710 Acked Write 6 Done Done Sent false
34107 9711 Acked Read 5 Done Done Sent false
34108 9712 Acked Read 2 Done Done Sent false
34109 9713 Acked Flush 0 Done Done Sent false
34110 9714 Acked Write 1 Done Done Sent false
34111 9715 Acked Write 2 Done Done Sent false
34112 9716 Acked Flush 0 Done Done Sent false
34113 9717 Acked Write 1 Done Done Sent false
34114 9718 Acked Read 3 Done Done Sent false
34115 9719 Acked Read 1 Done Done Sent false
34116 9720 Acked Read 10 Done Done Sent false
34117 9721 Acked Write 6 Done Done Sent false
34118 9722 Acked Read 1 Done Done Sent false
34119 9723 Acked Write 1 Done Done Sent false
34120 9724 Acked Write 1 Done Done Sent false
34121 9725 Acked Read 10 Done Done Sent false
34122 9726 Acked Read 9 Done Done Sent false
34123 9727 Acked Read 3 Done Done Sent false
34124 9728 Acked Write 1 Done Done Sent false
34125 9729 Acked Write 6 Done Done Sent false
34126 9730 Acked Write 2 Done Done Sent false
34127 9731 Acked Read 9 Done Done Sent false
34128 9732 Acked Read 4 Done Done Sent false
34129 9733 Acked Read 8 Done Done Sent false
34130 9734 Acked Write 7 Done Done Sent false
34131 9735 Acked Flush 0 Done Done Sent false
34132 9736 Acked Read 1 Done Done Sent false
34133 9737 Acked Read 7 Done Done Sent false
34134 9738 Acked Read 7 Done Done Sent false
34135 9739 Acked Flush 0 Done Done Sent false
34136 9740 Acked Flush 0 Done Done Sent false
34137 9741 Acked Read 10 Done Done Sent false
34138 9742 Acked Read 10 Done Done Sent false
34139 9743 Acked Read 8 Done Done Sent false
34140 9744 Acked Read 3 Done Done Sent false
34141 9745 Acked Read 10 Done Done Sent false
34142 9746 Acked Write 6 Done Done Sent false
34143 9747 Acked Read 1 Done Done Sent false
34144 9748 Acked Write 5 Done Done Sent false
34145 9749 Acked Write 6 Done Done Sent false
34146 9750 Acked Write 9 Done Done Sent false
34147 9751 Acked Read 6 Done Done Sent false
34148 9752 Acked Read 9 Done Done Sent false
34149 9753 Acked Read 2 Done Done Sent false
34150 9754 Acked Flush 0 Done Done Sent false
34151 STATES DS:0 DS:1 DS:2 TOTAL
34152 Sent 0 0 302 302
34153 Done 302 302 0 604
34154 Skipped 0 0 0 0
34155 Error 0 0 0 0
34156 Last Flush: 9754 9754 9452
34157 Downstairs last five completed: 9452 9451 9450 9449 9448
34158 Upstairs last five completed: 9754 9753 9752 9751 9750
34159 CLIENT: Up:0 ds:302 act:2
34160 ----------------------------------------------------------------
34161 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
34162 States: Active Active Offline
34163 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
34164 9453 Acked Write 7 Done Done Sent false
34165 9454 Acked Read 10 Done Done Sent false
34166 9455 Acked Write 4 Done Done Sent false
34167 9456 Acked Flush 0 Done Done Sent false
34168 9457 Acked Read 9 Done Done Sent false
34169 9458 Acked Write 7 Done Done Sent false
34170 9459 Acked Read 2 Done Done Sent false
34171 9460 Acked Write 1 Done Done Sent false
34172 9461 Acked Read 1 Done Done Sent false
34173 9462 Acked Read 6 Done Done Sent false
34174 9463 Acked Read 4 Done Done Sent false
34175 9464 Acked Read 1 Done Done Sent false
34176 9465 Acked Read 4 Done Done Sent false
34177 9466 Acked Read 9 Done Done Sent false
34178 9467 Acked Write 1 Done Done Sent false
34179 9468 Acked Flush 0 Done Done Sent false
34180 9469 Acked Read 7 Done Done Sent false
34181 9470 Acked Write 1 Done Done Sent false
34182 9471 Acked Write 6 Done Done Sent false
34183 9472 Acked Write 7 Done Done Sent false
34184 9473 Acked Write 6 Done Done Sent false
34185 9474 Acked Read 4 Done Done Sent false
34186 9475 Acked Write 7 Done Done Sent false
34187 9476 Acked Write 8 Done Done Sent false
34188 9477 Acked Write 1 Done Done Sent false
34189 9478 Acked Read 10 Done Done Sent false
34190 9479 Acked Write 1 Done Done Sent false
34191 9480 Acked Write 5 Done Done Sent false
34192 9481 Acked Write 4 Done Done Sent false
34193 9482 Acked Read 9 Done Done Sent false
34194 9483 Acked Read 8 Done Done Sent false
34195 9484 Acked Write 4 Done Done Sent false
34196 9485 Acked Write 9 Done Done Sent false
34197 9486 Acked Write 10 Done Done Sent false
34198 9487 Acked Write 1 Done Done Sent false
34199 9488 Acked Write 1 Done Done Sent false
34200 9489 Acked Write 2 Done Done Sent false
34201 9490 Acked Write 5 Done Done Sent false
34202 9491 Acked Write 3 Done Done Sent false
34203 9492 Acked Write 3 Done Done Sent false
34204 9493 Acked Flush 0 Done Done Sent false
34205 9494 Acked Read 7 Done Done Sent false
34206 9495 Acked Write 10 Done Done Sent false
34207 9496 Acked Read 5 Done Done Sent false
34208 9497 Acked Write 10 Done Done Sent false
34209 9498 Acked Read 6 Done Done Sent false
34210 9499 Acked Read 8 Done Done Sent false
34211 9500 Acked Read 5 Done Done Sent false
34212 9501 Acked Flush 0 Done Done Sent false
34213 9502 Acked Read 6 Done Done Sent false
34214 9503 Acked Read 6 Done Done Sent false
34215 9504 Acked Write 7 Done Done Sent false
34216 9505 Acked Write 4 Done Done Sent false
34217 9506 Acked Read 8 Done Done Sent false
34218 9507 Acked Read 1 Done Done Sent false
34219 9508 Acked Write 2 Done Done Sent false
34220 9509 Acked Write 2 Done Done Sent false
34221 9510 Acked Write 2 Done Done Sent false
34222 9511 Acked Read 2 Done Done Sent false
34223 9512 Acked Read 7 Done Done Sent false
34224 9513 Acked Write 4 Done Done Sent false
34225 9514 Acked Read 1 Done Done Sent false
34226 9515 Acked Read 7 Done Done Sent false
34227 9516 Acked Read 3 Done Done Sent false
34228 9517 Acked Read 1 Done Done Sent false
34229 9518 Acked Write 1 Done Done Sent false
34230 9519 Acked Write 5 Done Done Sent false
34231 9520 Acked Read 7 Done Done Sent false
34232 9521 Acked Read 3 Done Done Sent false
34233 9522 Acked Read 10 Done Done Sent false
34234 9523 Acked Flush 0 Done Done Sent false
34235 9524 Acked Flush 0 Done Done Sent false
34236 9525 Acked Write 10 Done Done Sent false
34237 9526 Acked Read 5 Done Done Sent false
34238 9527 Acked Read 1 Done Done Sent false
34239 9528 Acked Read 4 Done Done Sent false
34240 9529 Acked Flush 0 Done Done Sent false
34241 9530 Acked Write 1 Done Done Sent false
34242 9531 Acked Write 3 Done Done Sent false
34243 9532 Acked Write 10 Done Done Sent false
34244 9533 Acked Read 10 Done Done Sent false
34245 9534 Acked Read 1 Done Done Sent false
34246 9535 Acked Write 4 Done Done Sent false
34247 9536 Acked Read 1 Done Done Sent false
34248 9537 Acked Write 3 Done Done Sent false
34249 9538 Acked Write 9 Done Done Sent false
34250 9539 Acked Read 3 Done Done Sent false
34251 9540 Acked Write 10 Done Done Sent false
34252 9541 Acked Read 1 Done Done Sent false
34253 9542 Acked Read 2 Done Done Sent false
34254 9543 Acked Read 2 Done Done Sent false
34255 9544 Acked Read 7 Done Done Sent false
34256 9545 Acked Write 3 Done Done Sent false
34257 9546 Acked Read 6 Done Done Sent false
34258 9547 Acked Read 10 Done Done Sent false
34259 9548 Acked Write 10 Done Done Sent false
34260 9549 Acked Read 2 Done Done Sent false
34261 9550 Acked Read 4 Done Done Sent false
34262 9551 Acked Write 9 Done Done Sent false
34263 9552 Acked Read 9 Done Done Sent false
34264 9553 Acked Write 8 Done Done Sent false
34265 9554 Acked Write 8 Done Done Sent false
34266 9555 Acked Write 10 Done Done Sent false
34267 9556 Acked Flush 0 Done Done Sent false
34268 9557 Acked Write 1 Done Done Sent false
34269 9558 Acked Write 4 Done Done Sent false
34270 9559 Acked Read 6 Done Done Sent false
34271 9560 Acked Write 1 Done Done Sent false
34272 9561 Acked Read 6 Done Done Sent false
34273 9562 Acked Read 1 Done Done Sent false
34274 9563 Acked Read 9 Done Done Sent false
34275 9564 Acked Read 10 Done Done Sent false
34276 9565 Acked Read 7 Done Done Sent false
34277 9566 Acked Write 9 Done Done Sent false
34278 9567 Acked Read 7 Done Done Sent false
34279 9568 Acked Flush 0 Done Done Sent false
34280 9569 Acked Write 8 Done Done Sent false
34281 9570 Acked Read 7 Done Done Sent false
34282 9571 Acked Write 6 Done Done Sent false
34283 9572 Acked Read 6 Done Done Sent false
34284 9573 Acked Read 7 Done Done Sent false
34285 9574 Acked Read 4 Done Done Sent false
34286 9575 Acked Flush 0 Done Done Sent false
34287 9576 Acked Read 6 Done Done Sent false
34288 9577 Acked Write 1 Done Done Sent false
34289 9578 Acked Write 2 Done Done Sent false
34290 9579 Acked Read 5 Done Done Sent false
34291 9580 Acked Read 10 Done Done Sent false
34292 9581 Acked Read 1 Done Done Sent false
34293 9582 Acked Write 4 Done Done Sent false
34294 9583 Acked Write 8 Done Done Sent false
34295 9584 Acked Read 5 Done Done Sent false
34296 9585 Acked Write 10 Done Done Sent false
34297 9586 Acked Read 7 Done Done Sent false
34298 9587 Acked Read 4 Done Done Sent false
34299 9588 Acked Write 7 Done Done Sent false
34300 9589 Acked Write 5 Done Done Sent false
34301 9590 Acked Read 6 Done Done Sent false
34302 9591 Acked Read 6 Done Done Sent false
34303 9592 Acked Read 7 Done Done Sent false
34304 9593 Acked Write 9 Done Done Sent false
34305 9594 Acked Read 7 Done Done Sent false
34306 9595 Acked Write 8 Done Done Sent false
34307 9596 Acked Read 7 Done Done Sent false
34308 9597 Acked Write 3 Done Done Sent false
34309 9598 Acked Read 8 Done Done Sent false
34310 9599 Acked Write 1 Done Done Sent false
34311 9600 Acked Read 7 Done Done Sent false
34312 9601 Acked Write 1 Done Done Sent false
34313 9602 Acked Read 8 Done Done Sent false
34314 9603 Acked Read 4 Done Done Sent false
34315 9604 Acked Write 9 Done Done Sent false
34316 9605 Acked Read 10 Done Done Sent false
34317 9606 Acked Read 8 Done Done Sent false
34318 9607 Acked Read 2 Done Done Sent false
34319 9608 Acked Read 9 Done Done Sent false
34320 9609 Acked Write 5 Done Done Sent false
34321 9610 Acked Flush 0 Done Done Sent false
34322 9611 Acked Read 7 Done Done Sent false
34323 9612 Acked Read 4 Done Done Sent false
34324 9613 Acked Flush 0 Done Done Sent false
34325 9614 Acked Write 1 Done Done Sent false
34326 9615 Acked Read 5 Done Done Sent false
34327 9616 Acked Flush 0 Done Done Sent false
34328 9617 Acked Read 8 Done Done Sent false
34329 9618 Acked Read 4 Done Done Sent false
34330 9619 Acked Write 9 Done Done Sent false
34331 9620 Acked Write 3 Done Done Sent false
34332 9621 Acked Read 9 Done Done Sent false
34333 9622 Acked Read 5 Done Done Sent false
34334 9623 Acked Write 6 Done Done Sent false
34335 9624 Acked Write 7 Done Done Sent false
34336 9625 Acked Read 1 Done Done Sent false
34337 9626 Acked Read 8 Done Done Sent false
34338 9627 Acked Write 10 Done Done Sent false
34339 9628 Acked Write 7 Done Done Sent false
34340 9629 Acked Read 2 Done Done Sent false
34341 9630 Acked Write 4 Done Done Sent false
34342 9631 Acked Write 3 Done Done Sent false
34343 9632 Acked Write 6 Done Done Sent false
34344 9633 Acked Write 7 Done Done Sent false
34345 9634 Acked Write 2 Done Done Sent false
34346 9635 Acked Write 2 Done Done Sent false
34347 9636 Acked Read 6 Done Done Sent false
34348 9637 Acked Write 5 Done Done Sent false
34349 9638 Acked Read 2 Done Done Sent false
34350 9639 Acked Read 1 Done Done Sent false
34351 9640 Acked Write 6 Done Done Sent false
34352 9641 Acked Read 4 Done Done Sent false
34353 9642 Acked Flush 0 Done Done Sent false
34354 9643 Acked Read 9 Done Done Sent false
34355 9644 Acked Read 7 Done Done Sent false
34356 9645 Acked Write 3 Done Done Sent false
34357 9646 Acked Write 6 Done Done Sent false
34358 9647 Acked Write 9 Done Done Sent false
34359 9648 Acked Read 6 Done Done Sent false
34360 9649 Acked Flush 0 Done Done Sent false
34361 9650 Acked Read 10 Done Done Sent false
34362 9651 Acked Write 2 Done Done Sent false
34363 9652 Acked Write 4 Done Done Sent false
34364 9653 Acked Write 5 Done Done Sent false
34365 9654 Acked Read 9 Done Done Sent false
34366 9655 Acked Write 5 Done Done Sent false
34367 9656 Acked Write 7 Done Done Sent false
34368 9657 Acked Read 3 Done Done Sent false
34369 9658 Acked Write 6 Done Done Sent false
34370 9659 Acked Flush 0 Done Done Sent false
34371 9660 Acked Flush 0 Done Done Sent false
34372 9661 Acked Read 6 Done Done Sent false
34373 9662 Acked Read 4 Done Done Sent false
34374 9663 Acked Flush 0 Done Done Sent false
34375 9664 Acked Read 4 Done Done Sent false
34376 9665 Acked Read 6 Done Done Sent false
34377 9666 Acked Flush 0 Done Done Sent false
34378 9667 Acked Read 2 Done Done Sent false
34379 9668 Acked Write 1 Done Done Sent false
34380 9669 Acked Write 9 Done Done Sent false
34381 9670 Acked Write 8 Done Done Sent false
34382 9671 Acked Flush 0 Done Done Sent false
34383 9672 Acked Write 3 Done Done Sent false
34384 9673 Acked Read 4 Done Done Sent false
34385 9674 Acked Read 3 Done Done Sent false
34386 9675 Acked Write 9 Done Done Sent false
34387 9676 Acked Write 6 Done Done Sent false
34388 9677 Acked Flush 0 Done Done Sent false
34389 9678 Acked Write 7 Done Done Sent false
34390 9679 Acked Read 10 Done Done Sent false
34391 9680 Acked Write 3 Done Done Sent false
34392 9681 Acked Write 9 Done Done Sent false
34393 9682 Acked Write 7 Done Done Sent false
34394 9683 Acked Read 5 Done Done Sent false
34395 9684 Acked Write 7 Done Done Sent false
34396 9685 Acked Write 2 Done Done Sent false
34397 9686 Acked Flush 0 Done Done Sent false
34398 9687 Acked Flush 0 Done Done Sent false
34399 9688 Acked Write 8 Done Done Sent false
34400 9689 Acked Read 8 Done Done Sent false
34401 9690 Acked Write 9 Done Done Sent false
34402 9691 Acked Write 7 Done Done Sent false
34403 9692 Acked Read 3 Done Done Sent false
34404 9693 Acked Write 7 Done Done Sent false
34405 9694 Acked Read 3 Done Done Sent false
34406 9695 Acked Write 7 Done Done Sent false
34407 9696 Acked Read 10 Done Done Sent false
34408 9697 Acked Write 6 Done Done Sent false
34409 9698 Acked Read 9 Done Done Sent false
34410 9699 Acked Flush 0 Done Done Sent false
34411 9700 Acked Flush 0 Done Done Sent false
34412 9701 Acked Flush 0 Done Done Sent false
34413 9702 Acked Read 7 Done Done Sent false
34414 9703 Acked Read 10 Done Done Sent false
34415 9704 Acked Read 1 Done Done Sent false
34416 9705 Acked Write 8 Done Done Sent false
34417 9706 Acked Read 7 Done Done Sent false
34418 9707 Acked Read 2 Done Done Sent false
34419 9708 Acked Write 9 Done Done Sent false
34420 9709 Acked Write 9 Done Done Sent false
34421 9710 Acked Write 6 Done Done Sent false
34422 9711 Acked Read 5 Done Done Sent false
34423 9712 Acked Read 2 Done Done Sent false
34424 9713 Acked Flush 0 Done Done Sent false
34425 9714 Acked Write 1 Done Done Sent false
34426 9715 Acked Write 2 Done Done Sent false
34427 9716 Acked Flush 0 Done Done Sent false
34428 9717 Acked Write 1 Done Done Sent false
34429 9718 Acked Read 3 Done Done Sent false
34430 9719 Acked Read 1 Done Done Sent false
34431 9720 Acked Read 10 Done Done Sent false
34432 9721 Acked Write 6 Done Done Sent false
34433 9722 Acked Read 1 Done Done Sent false
34434 9723 Acked Write 1 Done Done Sent false
34435 9724 Acked Write 1 Done Done Sent false
34436 9725 Acked Read 10 Done Done Sent false
34437 9726 Acked Read 9 Done Done Sent false
34438 9727 Acked Read 3 Done Done Sent false
34439 9728 Acked Write 1 Done Done Sent false
34440 9729 Acked Write 6 Done Done Sent false
34441 9730 Acked Write 2 Done Done Sent false
34442 9731 Acked Read 9 Done Done Sent false
34443 9732 Acked Read 4 Done Done Sent false
34444 9733 Acked Read 8 Done Done Sent false
34445 9734 Acked Write 7 Done Done Sent false
34446 9735 Acked Flush 0 Done Done Sent false
34447 9736 Acked Read 1 Done Done Sent false
34448 9737 Acked Read 7 Done Done Sent false
34449 9738 Acked Read 7 Done Done Sent false
34450 9739 Acked Flush 0 Done Done Sent false
34451 9740 Acked Flush 0 Done Done Sent false
34452 9741 Acked Read 10 Done Done Sent false
34453 9742 Acked Read 10 Done Done Sent false
34454 9743 Acked Read 8 Done Done Sent false
34455 9744 Acked Read 3 Done Done Sent false
34456 9745 Acked Read 10 Done Done Sent false
34457 9746 Acked Write 6 Done Done Sent false
34458 9747 Acked Read 1 Done Done Sent false
34459 9748 Acked Write 5 Done Done Sent false
34460 9749 Acked Write 6 Done Done Sent false
34461 9750 Acked Write 9 Done Done Sent false
34462 9751 Acked Read 6 Done Done Sent false
34463 9752 Acked Read 9 Done Done Sent false
34464 9753 Acked Read 2 Done Done Sent false
34465 9754 Acked Flush 0 Done Done Sent false
34466 STATES DS:0 DS:1 DS:2 TOTAL
34467 Sent 0 0 302 302
34468 Done 302 302 0 604
34469 Skipped 0 0 0 0
34470 Error 0 0 0 0
34471 Last Flush: 9754 9754 9452
34472 Downstairs last five completed: 9452 9451 9450 9449 9448
34473 Upstairs last five completed: 9754 9753 9752 9751 9750
34474 CLIENT: Up:0 ds:302 act:2
34475 [8830] got start action
34476 [8830] I am going to restart
34477 Starting downstairs at port 8830
34478 Make output file at "/tmp/dsc/downstairs-8830.txt"
34479 [8830][2] reports Starting
34480 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1232)
34481 [8830][2] reports Running
344822024-11-26T22:11:47.268ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
344832024-11-26T22:11:47.268ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:55739) = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
344842024-11-26T22:11:47.268ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
344852024-11-26T22:11:47.269ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
344862024-11-26T22:11:47.269ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
344872024-11-26T22:11:47.269ZINFOcrucible: send last flush ID to this DS: 9452 = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
344882024-11-26T22:11:47.269ZINFOcrucible: Replied this last flush ID: 9452 = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
344892024-11-26T22:11:47.269ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
344902024-11-26T22:11:47.269ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
344912024-11-26T22:11:47.269ZINFOcrucible: [2] client re-new 302 jobs since flush 9452 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
344922024-11-26T22:11:47.270ZINFOcrucible: [2] Replayed 302 jobs since flush: 9452 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
34493 ----------------------------------------------------------------
34494 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
34495 Downstairs last five completed: 9754 9753 9752 9751 9750
34496 Upstairs last five completed: 9754 9753 9752 9751 9750
34497 CLIENT: Up:0 ds:0 act:3
34498 Replay: All jobs finished
34499 Main task has work to do, go find it
34500 got dsc Stop(2)
34501 stop 2
34502 Nov 26 22:11:47.507 INFO request completed, [8830] Got stop action so:false kr:false
34503 latency_us: 364, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 70bbaaa2-6fb9-48fd-a1f3-b3a9f3f7f27e, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
345042024-11-26T22:11:47.508ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
345052024-11-26T22:11:47.508ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
345062024-11-26T22:11:47.508ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
345072024-11-26T22:11:47.508ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
34508 Nov 26 22:11:47.508 INFO request completed, latency_us: 149, {"msg":"[2] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-26T22:11:47.508237015Z","hostname":"w-01JDN7ZQ17ABQG15KHKN3F2D26","pid":1203,"client":"2","":"downstairs","session_id":"86a36c17-3718-484c-87fd-2b8a1dc2ab18"}
34509 response_code: 200, uri: /state/cid/2, method: GET{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2024-11-26T22:11:47.508297487Z","hostname":"w-01JDN7ZQ17ABQG15KHKN3F2D26","pid":1203,"":"io task","client":"2","":"downstairs","session_id":"86a36c17-3718-484c-87fd-2b8a1dc2ab18"}
34510 , req_id: 54755d8f-363f-4f8e-945d-61792c31fae6, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
34511 [8830] Exited with: ExitStatus(unix_wait_status(9))
34512 [8830][2] reports Exit
34513 Nov 26 22:11:51.509 INFO request completed, latency_us: 197, response_code: 200, uri: /state/cid/2, method: GET, req_id: 1e7f7b05-9cc2-48b4-b1e5-7340cd6c1538, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
34514 001/300 Read block 1489 len 24576
34515 002/300 Write block 4381 len 16384 data: 4 4 5 5
34516 003/300 Write block 881 len 20480 data: 3 3 4 4 6
34517 004/300 Write block 1441 len 32768 data: 5 7 6 7 7 9 7 7
34518 005/300 Read block 1193 len 32768
34519 006/300 Read block 287 len 12288
34520 007/300 Write block 1730 len 32768 data: 7 6 6 6 7 8 8 7
34521 008/300 Write block 3626 len 4096 data: 6
34522 009/300 Read block 4752 len 12288
34523 010/300 Write block 1207 len 24576 data: 4 4 6 6 8 7
34524 011/300 Flush
34525 012/300 Write block 4824 len 12288 data: 6 7 5
34526 013/300 Write block 3251 len 40960 data: 3 3 5 5 5 6 6 9 10 9
34527 014/300 Read block 4525 len 40960
34528 015/300 Write block 1757 len 16384 data: 7 4 5 7
34529 016/300 Read block 3653 len 36864
34530 017/300 Write block 4194 len 20480 data: 6 6 7 10 10
34531 018/300 Read block 2090 len 20480
34532 019/300 Write block 528 len 8192 data: 6 6
34533 020/300 Read block 4456 len 24576
34534 021/300 Read block 2329 len 4096
34535 022/300 Read block 1897 len 36864
34536 023/300 Flush
34537 024/300 Write block 660 len 16384 data: 9 7 6 5
34538 025/300 Write block 320 len 4096 data: 6
34539 026/300 Read block 4792 len 12288
34540 027/300 Read block 2620 len 8192
34541 028/300 Read block 4553 len 24576
34542 029/300 Write block 520 len 28672 data: 5 6 6 6 7 7 5
34543 030/300 Read block 2036 len 28672
34544 031/300 Read block 2593 len 8192
34545 032/300 Write block 2392 len 16384 data: 7 6 6 8
34546 033/300 Write block 2058 len 4096 data: 11
34547 034/300 Read block 4642 len 12288
34548 035/300 Write block 3474 len 8192 data: 6 7
34549 036/300 Write block 1193 len 4096 data: 10
34550 037/300 Read block 2479 len 24576
34551 038/300 Read block 4186 len 32768
34552 039/300 Read block 1821 len 32768
34553 040/300 Write block 1019 len 20480 data: 6 7 8 7 8
34554 041/300 Read block 854 len 8192
34555 042/300 Read block 3981 len 28672
34556 043/300 Write block 4585 len 36864 data: 5 6 7 7 5 4 4 5 7
34557 044/300 Write block 787 len 32768 data: 6 7 8 8 7 7 6 6
34558 045/300 Write block 2664 len 24576 data: 6 8 7 8 7 6
34559 046/300 Write block 3418 len 36864 data: 5 6 5 6 5 5 6 7 6
34560 047/300 Read block 4377 len 40960
34561 048/300 Read block 3361 len 40960
34562 049/300 Read block 3079 len 40960
34563 050/300 Read block 4198 len 24576
34564 051/300 Read block 3787 len 28672
34565 052/300 Read block 3657 len 32768
34566 053/300 Write block 4608 len 20480 data: 6 5 6 5 5
34567 054/300 Read block 3837 len 12288
34568 055/300 Write block 4656 len 24576 data: 5 6 8 10 10 6
34569 056/300 Flush
34570 057/300 Read block 1606 len 4096
34571 058/300 Write block 4928 len 40960 data: 6 5 6 6 9 9 9 8 9 9
34572 059/300 Write block 1324 len 12288 data: 5 7 6
34573 060/300 Write block 3469 len 20480 data: 5 5 5 5 5
34574 061/300 Read block 1178 len 40960
34575 062/300 Read block 2459 len 12288
34576 063/300 Write block 4064 len 40960 data: 8 7 7 7 6 5 6 5 6 6
34577 064/300 Read block 43 len 24576
34578 065/300 Read block 3894 len 8192
34579 066/300 Read block 532 len 32768
34580 067/300 Read block 4877 len 12288
34581 068/300 Write block 3444 len 4096 data: 4
34582 069/300 Read block 2921 len 28672
34583 070/300 Write block 3752 len 8192 data: 7 6
34584 071/300 Write block 3304 len 36864 data: 7 6 6 6 5 7 6 6 7
34585 072/300 Write block 836 len 12288 data: 5 4 4
34586 073/300 Read block 410 len 20480
34587 074/300 Read block 1258 len 20480
34588 075/300 Read block 2020 len 20480
34589 076/300 Read block 1436 len 20480
34590 077/300 Write block 3009 len 20480 data: 6 10 7 7 7
34591 078/300 Write block 3038 len 36864 data: 3 3 4 3 3 3 3 5 4
34592 079/300 Flush
34593 080/300 Write block 3598 len 28672 data: 13 12 11 9 7 5 5
34594 081/300 Read block 589 len 32768
34595 082/300 Write block 2032 len 32768 data: 7 6 8 8 9 8 7 6
34596 083/300 Read block 1956 len 20480
34597 084/300 Write block 4563 len 8192 data: 6 7
34598 085/300 Write block 1916 len 4096 data: 5
34599 086/300 Write block 3230 len 32768 data: 14 10 10 8 6 4 2 3
34600 087/300 Read block 1753 len 12288
34601 088/300 Read block 2967 len 40960
34602 089/300 Write block 3006 len 36864 data: 6 6 6 7 11 8 8 8 6
34603 090/300 Write block 4199 len 36864 data: 10 9 9 8 8 6 5 6 6
34604 091/300 Read block 3944 len 32768
34605 092/300 Flush
34606 093/300 Flush
34607 094/300 Read block 4821 len 20480
34608 095/300 Read block 2097 len 16384
34609 096/300 Read block 2748 len 12288
34610 097/300 Read block 4887 len 16384
34611 098/300 Write block 1973 len 24576 data: 3 3 3 4 3 3
34612 099/300 Write block 1597 len 32768 data: 6 8 10 8 7 7 6 7
34613 100/300 Read block 2819 len 8192
34614 101/300 Flush
34615 102/300 Write block 2421 len 16384 data: 3 5 5 5
34616 103/300 Read block 51 len 4096
34617 104/300 Read block 4408 len 12288
34618 105/300 Read block 4159 len 16384
34619 106/300 Read block 3113 len 24576
34620 107/300 Read block 51 len 32768
34621 108/300 Write block 2687 len 12288 data: 6 6 7
34622 109/300 Write block 2451 len 36864 data: 8 8 6 7 9 5 4 4 5
34623 110/300 Read block 1702 len 40960
34624 111/300 Read block 1111 len 20480
34625 112/300 Write block 3954 len 20480 data: 7 5 5 3 5
34626 113/300 Read block 3297 len 28672
34627 114/300 Read block 4246 len 40960
34628 115/300 Read block 4307 len 36864
34629 116/300 Read block 1791 len 20480
34630 117/300 Write block 3451 len 40960 data: 7 7 7 7 8 6 6 4 4 5
34631 118/300 Write block 4753 len 20480 data: 7 7 6 4 5
34632 119/300 Read block 2862 len 12288
34633 120/300 Read block 3894 len 32768
34634 121/300 Read block 2812 len 4096
34635 122/300 Read block 1404 len 16384
34636 123/300 Flush
34637 124/300 Write block 1923 len 40960 data: 7 6 6 8 8 7 6 6 6 6
34638 125/300 Write block 831 len 32768 data: 5 5 5 7 5 6 5 5
34639 126/300 Read block 1964 len 32768
34640 127/300 Read block 1239 len 8192
34641 128/300 Write block 1300 len 28672 data: 6 6 7 8 8 9 8
34642 129/300 Write block 3791 len 40960 data: 8 7 7 6 5 5 4 4 3 4
34643 130/300 Write block 3549 len 16384 data: 4 4 6 7
34644 131/300 Write block 4439 len 12288 data: 8 7 8
34645 132/300 Read block 3029 len 20480
34646 133/300 Read block 1390 len 24576
34647 134/300 Write block 2406 len 8192 data: 6 6
34648 135/300 Read block 1430 len 40960
34649 136/300 Read block 3428 len 8192
34650 137/300 Read block 118 len 24576
34651 138/300 Write block 2527 len 24576 data: 2 4 4 6 5 4
34652 139/300 Write block 1420 len 4096 data: 4
34653 140/300 Read block 2976 len 20480
34654 141/300 Read block 2929 len 36864
34655 142/300 Read block 2032 len 4096
34656 143/300 Read block 3543 len 40960
34657 144/300 Read block 2560 len 20480
34658 145/300 Read block 914 len 32768
34659 146/300 Read block 4961 len 36864
34660 147/300 Read block 4461 len 20480
34661 148/300 Write block 2158 len 40960 data: 2 2 4 5 7 6 6 8 8 7
34662 149/300 Read block 3682 len 32768
34663 150/300 Read block 422 len 4096
34664 151/300 Read block 1979 len 4096
34665 152/300 Read block 3947 len 24576
34666 153/300 Read block 1718 len 24576
34667 154/300 Write block 739 len 32768 data: 5 6 7 8 7 6 6 5
34668 155/300 Write block 4226 len 24576 data: 4 4 3 3 5 5
34669 156/300 Write block 2595 len 28672 data: 9 9 9 10 12 12 10
34670 157/300 Read block 521 len 16384
34671 158/300 Read block 4377 len 12288
34672 159/300 Read block 2877 len 40960
34673 160/300 Flush
34674 161/300 Read block 1474 len 20480
34675 162/300 Write block 1390 len 28672 data: 3 4 5 5 5 7 7
34676 163/300 Write block 4057 len 32768 data: 6 6 7 8 9 9 8 9
34677 164/300 Read block 2962 len 28672
34678 165/300 Read block 3523 len 4096
34679 166/300 Write block 4637 len 28672 data: 9 9 10 9 9 9 7
34680 167/300 Read block 2241 len 40960
34681 168/300 Read block 1420 len 28672
34682 169/300 Write block 1204 len 24576 data: 8 7 4 5 5 7
34683 170/300 Read block 4004 len 24576
34684 171/300 Read block 1186 len 20480
34685 172/300 Flush
34686 173/300 Read block 1218 len 4096
34687 174/300 Flush
34688 175/300 Read block 41 len 12288
34689 176/300 Read block 3895 len 24576
34690 177/300 Write block 3554 len 12288 data: 6 5 5
34691 178/300 Read block 4816 len 8192
34692 179/300 Write block 2107 len 32768 data: 6 6 5 6 6 6 7 5
34693 180/300 Write block 880 len 12288 data: 4 4 4
34694 181/300 Read block 1596 len 32768
34695 182/300 Write block 2901 len 32768 data: 6 6 7 8 8 7 7 7
34696 183/300 Read block 1102 len 12288
34697 184/300 Write block 1001 len 28672 data: 4 7 7 8 8 9 8
34698 185/300 Write block 3961 len 28672 data: 5 4 3 3 4 4 3
34699 186/300 Write block 1575 len 24576 data: 7 7 6 5 5 8
34700 187/300 Read block 4473 len 28672
34701 188/300 Read block 4085 len 16384
34702 189/300 Flush
34703 190/300 Write block 1987 len 36864 data: 8 8 8 7 4 3 4 4 3
34704 191/300 Write block 4138 len 36864 data: 3 3 4 5 8 7 9 7 7
34705 192/300 Read block 2839 len 16384
34706 193/300 Write block 2097 len 28672 data: 6 5 5 6 8 9 9
34707 194/300 Write block 673 len 32768 data: 7 7 7 5 5 4 5 5
34708 195/300 Read block 1683 len 32768
34709 196/300 Flush
34710 197/300 Write block 2973 len 28672 data: 5 6 7 4 2 2 2
34711 198/300 Read block 4160 len 8192
34712 199/300 Read block 646 len 20480
34713 200/300 Read block 3147 len 24576
34714 201/300 Flush
34715 202/300 Write block 2435 len 40960 data: 6 7 6 7 6 4 5 7 9 8
34716 203/300 Read block 4568 len 24576
34717 204/300 Read block 1274 len 16384
34718 205/300 Write block 2963 len 40960 data: 7 6 6 6 5 7 7 7 6 6
34719 206/300 Read block 221 len 36864
34720 207/300 Read block 337 len 20480
34721 208/300 Read block 4005 len 36864
34722 209/300 Write block 13 len 12288 data: 5 5 4
34723 210/300 Read block 2160 len 32768
34724 211/300 Read block 3812 len 24576
34725 212/300 Read block 809 len 12288
34726 213/300 Write block 2442 len 36864 data: 8 10 9 8 7 6 7 8 7
34727 214/300 Flush
34728 215/300 Write block 4738 len 28672 data: 2 3 3 4 3 2 4
34729 216/300 Write block 4961 len 20480 data: 6 7 7 9 11
34730 217/300 Write block 706 len 20480 data: 3 5 6 7 7
34731 218/300 Read block 4020 len 20480
34732 219/300 Read block 448 len 4096
34733 220/300 Write block 325 len 12288 data: 5 4 3
34734 221/300 Read block 1389 len 20480
34735 222/300 Write block 4023 len 36864 data: 5 5 9 10 9 9 9 8 7
34736 223/300 Write block 2520 len 4096 data: 7
34737 224/300 Write block 2744 len 36864 data: 9 8 9 8 8 9 6 4 4
34738 225/300 Write block 2152 len 20480 data: 6 5 4 4 4
34739 226/300 Write block 348 len 20480 data: 7 9 8 9 9
34740 227/300 Read block 2129 len 32768
34741 228/300 Read block 2060 len 36864
34742 229/300 Read block 2715 len 8192
34743 230/300 Write block 2099 len 8192 data: 6 7
34744 231/300 Read block 3854 len 32768
34745 232/300 Write block 4355 len 16384 data: 4 6 8 5
34746 233/300 Flush
34747 234/300 Write block 3108 len 28672 data: 7 7 7 7 7 7 9
34748 235/300 Read block 801 len 4096
34749 236/300 Read block 372 len 36864
34750 237/300 Read block 2717 len 12288
34751 238/300 Read block 1763 len 28672
34752 239/300 Read block 3829 len 4096
34753 240/300 Read block 3103 len 4096
34754 241/300 Write block 4792 len 40960 data: 4 4 4 5 4 5 4 5 4 7
34755 242/300 Read block 1807 len 16384
34756 243/300 Flush
34757 244/300 Write block 905 len 36864 data: 4 5 4 5 7 9 9 9 8
34758 245/300 Write block 3973 len 4096 data: 3
34759 246/300 Read block 1639 len 24576
34760 247/300 Read block 802 len 16384
34761 248/300 Read block 3065 len 12288
34762 249/300 Read block 1478 len 16384
34763 250/300 Read block 3571 len 12288
34764 251/300 Read block 414 len 16384
34765 252/300 Write block 4067 len 12288 data: 8 7 6
34766 253/300 Read block 2944 len 12288
34767 254/300 Write block 2805 len 4096 data: 4
34768 255/300 Read block 4449 len 24576
34769 256/300 Write block 999 len 20480 data: 3 3 5 8 8
34770 257/300 Read block 1535 len 8192
34771 258/300 Read block 3249 len 40960
34772 259/300 Write block 1758 len 12288 data: 5 6 8
34773 260/300 Read block 649 len 40960
34774 261/300 Read block 4501 len 36864
34775 262/300 Write block 3949 len 4096 data: 6
34776 263/300 Write block 2484 len 12288 data: 10 11 12
34777 264/300 Read block 176 len 36864
34778 265/300 Read block 4866 len 28672
34779 266/300 Read block 91 len 4096
34780 267/300 Flush
34781 268/300 Write block 271 len 36864 data: 4 4 3 3 3 4 4 4 4
34782 269/300 Read block 610 len 28672
34783 270/300 Write block 2470 len 32768 data: 6 7 7 8 6 5 5 5
34784 271/300 Read block 3530 len 8192
34785 272/300 Write block 2316 len 12288 data: 6 6 7
34786 273/300 Read block 894 len 20480
34787 274/300 Write block 507 len 16384 data: 6 5 6 7
34788 275/300 Read block 3852 len 28672
34789 276/300 Read block 442 len 36864
34790 277/300 Write block 2060 len 12288 data: 10 9 8
34791 278/300 Read block 44 len 4096
34792 279/300 Flush
34793 280/300 Flush
34794 281/300 Read block 3869 len 28672
34795 282/300 Write block 481 len 32768 data: 6 7 5 5 6 6 5 6
34796 283/300 Read block 3723 len 28672
34797 284/300 Write block 1777 len 24576 data: 8 8 9 8 6 7
34798 285/300 Write block 482 len 12288 data: 8 6 6
34799 286/300 Read block 872 len 12288
34800 287/300 Read block 1376 len 16384
34801 288/300 Read block 3975 len 16384
34802 289/300 Write block 4231 len 16384 data: 6 7 7 8
34803 290/300 Read block 4150 len 8192
34804 291/300 Write block 2741 len 24576 data: 7 8 9 10 9 10
34805 292/300 Read block 2221 len 36864
34806 293/300 Write block 3314 len 28672 data: 8 9 9 9 9 9 8
34807 294/300 Read block 531 len 32768
34808 295/300 Read block 1156 len 16384
34809 296/300 Write block 252 len 28672 data: 4 5 5 5 6 4 3
34810 297/300 Read block 13 len 28672
34811 298/300 Read block 2000 len 28672
34812 299/300 Flush
34813 300/300 Flush
34814 301/300 Write block 4526 len 16384 data: 7 9 10 9
34815 Main task has work to do, go find it
34816 got dsc Start(2)
34817 start 2
34818 Nov 26 22:11:51.755 INFO request completed, latency_us: 224, response_code: 204, uri: /start/cid/2, method: POST, req_id: 476ed82c-b60a-4a0f-85f6-a89cf23b3c72, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
34819 Replay: started 2, returned:Ok(())
34820 ----------------------------------------------------------------
34821 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
34822 States: Active Active Offline
34823 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
34824 9755 Acked Read 6 Done Done Sent false
34825 9756 Acked Write 4 Done Done Sent false
34826 9757 Acked Write 5 Done Done Sent false
34827 9758 Acked Write 8 Done Done Sent false
34828 9759 Acked Read 8 Done Done Sent false
34829 9760 Acked Read 3 Done Done Sent false
34830 9761 Acked Write 8 Done Done Sent false
34831 9762 Acked Write 1 Done Done Sent false
34832 9763 Acked Read 3 Done Done Sent false
34833 9764 Acked Write 6 Done Done Sent false
34834 9765 Acked Flush 0 Done Done Sent false
34835 9766 Acked Write 3 Done Done Sent false
34836 9767 Acked Write 10 Done Done Sent false
34837 9768 Acked Read 10 Done Done Sent false
34838 9769 Acked Write 4 Done Done Sent false
34839 9770 Acked Read 9 Done Done Sent false
34840 9771 Acked Write 5 Done Done Sent false
34841 9772 Acked Read 5 Done Done Sent false
34842 9773 Acked Write 2 Done Done Sent false
34843 9774 Acked Read 6 Done Done Sent false
34844 9775 Acked Read 1 Done Done Sent false
34845 9776 Acked Read 9 Done Done Sent false
34846 9777 Acked Flush 0 Done Done Sent false
34847 9778 Acked Write 4 Done Done Sent false
34848 9779 Acked Write 1 Done Done Sent false
34849 9780 Acked Read 3 Done Done Sent false
34850 9781 Acked Read 2 Done Done Sent false
34851 9782 Acked Read 6 Done Done Sent false
34852 9783 Acked Write 7 Done Done Sent false
34853 9784 Acked Read 7 Done Done Sent false
34854 9785 Acked Read 2 Done Done Sent false
34855 9786 Acked Write 4 Done Done Sent false
34856 9787 Acked Write 1 Done Done Sent false
34857 9788 Acked Read 3 Done Done Sent false
34858 9789 Acked Write 2 Done Done Sent false
34859 9790 Acked Write 1 Done Done Sent false
34860 9791 Acked Read 6 Done Done Sent false
34861 9792 Acked Read 8 Done Done Sent false
34862 9793 Acked Read 8 Done Done Sent false
34863 9794 Acked Write 5 Done Done Sent false
34864 9795 Acked Read 2 Done Done Sent false
34865 9796 Acked Read 7 Done Done Sent false
34866 9797 Acked Write 9 Done Done Sent false
34867 9798 Acked Write 8 Done Done Sent false
34868 9799 Acked Write 6 Done Done Sent false
34869 9800 Acked Write 9 Done Done Sent false
34870 9801 Acked Read 10 Done Done Sent false
34871 9802 Acked Read 10 Done Done Sent false
34872 9803 Acked Read 10 Done Done Sent false
34873 9804 Acked Read 6 Done Done Sent false
34874 9805 Acked Read 7 Done Done Sent false
34875 9806 Acked Read 8 Done Done Sent false
34876 9807 Acked Write 5 Done Done Sent false
34877 9808 Acked Read 3 Done Done Sent false
34878 9809 Acked Write 6 Done Done Sent false
34879 9810 Acked Flush 0 Done Done Sent false
34880 9811 Acked Read 1 Done Done Sent false
34881 9812 Acked Write 10 Done Done Sent false
34882 9813 Acked Write 3 Done Done Sent false
34883 9814 Acked Write 5 Done Done Sent false
34884 9815 Acked Read 10 Done Done Sent false
34885 9816 Acked Read 3 Done Done Sent false
34886 9817 Acked Write 10 Done Done Sent false
34887 9818 Acked Read 6 Done Done Sent false
34888 9819 Acked Read 2 Done Done Sent false
34889 9820 Acked Read 8 Done Done Sent false
34890 9821 Acked Read 3 Done Done Sent false
34891 9822 Acked Write 1 Done Done Sent false
34892 9823 Acked Read 7 Done Done Sent false
34893 9824 Acked Write 2 Done Done Sent false
34894 9825 Acked Write 9 Done Done Sent false
34895 9826 Acked Write 3 Done Done Sent false
34896 9827 Acked Read 5 Done Done Sent false
34897 9828 Acked Read 5 Done Done Sent false
34898 9829 Acked Read 5 Done Done Sent false
34899 9830 Acked Read 5 Done Done Sent false
34900 9831 Acked Write 5 Done Done Sent false
34901 9832 Acked Write 9 Done Done Sent false
34902 9833 Acked Flush 0 Done Done Sent false
34903 9834 Acked Write 7 Done Done Sent false
34904 9835 Acked Read 8 Done Done Sent false
34905 9836 Acked Write 8 Done Done Sent false
34906 9837 Acked Read 5 Done Done Sent false
34907 9838 Acked Write 2 Done Done Sent false
34908 9839 Acked Write 1 Done Done Sent false
34909 9840 Acked Write 8 Done Done Sent false
34910 9841 Acked Read 3 Done Done Sent false
34911 9842 Acked Read 10 Done Done Sent false
34912 9843 Acked Write 9 Done Done Sent false
34913 9844 Acked Write 9 Done Done Sent false
34914 9845 Acked Read 8 Done Done Sent false
34915 9846 Acked Flush 0 Done Done Sent false
34916 9847 Acked Flush 0 Done Done Sent false
34917 9848 Acked Read 5 Done Done Sent false
34918 9849 Acked Read 4 Done Done Sent false
34919 9850 Acked Read 3 Done Done Sent false
34920 9851 Acked Read 4 Done Done Sent false
34921 9852 Acked Write 6 Done Done Sent false
34922 9853 Acked Write 8 Done Done Sent false
34923 9854 Acked Read 2 Done Done Sent false
34924 9855 Acked Flush 0 Done Done Sent false
34925 9856 Acked Write 4 Done Done Sent false
34926 9857 Acked Read 1 Done Done Sent false
34927 9858 Acked Read 3 Done Done Sent false
34928 9859 Acked Read 4 Done Done Sent false
34929 9860 Acked Read 6 Done Done Sent false
34930 9861 Acked Read 8 Done Done Sent false
34931 9862 Acked Write 3 Done Done Sent false
34932 9863 Acked Write 9 Done Done Sent false
34933 9864 Acked Read 10 Done Done Sent false
34934 9865 Acked Read 5 Done Done Sent false
34935 9866 Acked Write 5 Done Done Sent false
34936 9867 Acked Read 7 Done Done Sent false
34937 9868 Acked Read 10 Done Done Sent false
34938 9869 Acked Read 9 Done Done Sent false
34939 9870 Acked Read 5 Done Done Sent false
34940 9871 Acked Write 10 Done Done Sent false
34941 9872 Acked Write 5 Done Done Sent false
34942 9873 Acked Read 3 Done Done Sent false
34943 9874 Acked Read 8 Done Done Sent false
34944 9875 Acked Read 1 Done Done Sent false
34945 9876 Acked Read 4 Done Done Sent false
34946 9877 Acked Flush 0 Done Done Sent false
34947 9878 Acked Write 10 Done Done Sent false
34948 9879 Acked Write 8 Done Done Sent false
34949 9880 Acked Read 8 Done Done Sent false
34950 9881 Acked Read 2 Done Done Sent false
34951 9882 Acked Write 7 Done Done Sent false
34952 9883 Acked Write 10 Done Done Sent false
34953 9884 Acked Write 4 Done Done Sent false
34954 9885 Acked Write 3 Done Done Sent false
34955 9886 Acked Read 5 Done Done Sent false
34956 9887 Acked Read 6 Done Done Sent false
34957 9888 Acked Write 2 Done Done Sent false
34958 9889 Acked Read 10 Done Done Sent false
34959 9890 Acked Read 2 Done Done Sent false
34960 9891 Acked Read 6 Done Done Sent false
34961 9892 Acked Write 6 Done Done Sent false
34962 9893 Acked Write 1 Done Done Sent false
34963 9894 Acked Read 5 Done Done Sent false
34964 9895 Acked Read 9 Done Done Sent false
34965 9896 Acked Read 1 Done Done Sent false
34966 9897 Acked Read 10 Done Done Sent false
34967 9898 Acked Read 5 Done Done Sent false
34968 9899 Acked Read 8 Done Done Sent false
34969 9900 Acked Read 9 Done Done Sent false
34970 9901 Acked Read 5 Done Done Sent false
34971 9902 Acked Write 10 Done Done Sent false
34972 9903 Acked Read 8 Done Done Sent false
34973 9904 Acked Read 1 Done Done Sent false
34974 9905 Acked Read 1 Done Done Sent false
34975 9906 Acked Read 6 Done Done Sent false
34976 9907 Acked Read 6 Done Done Sent false
34977 9908 Acked Write 8 Done Done Sent false
34978 9909 Acked Write 6 Done Done Sent false
34979 9910 Acked Write 7 Done Done Sent false
34980 9911 Acked Read 4 Done Done Sent false
34981 9912 Acked Read 3 Done Done Sent false
34982 9913 Acked Read 10 Done Done Sent false
34983 9914 Acked Flush 0 Done Done Sent false
34984 9915 Acked Read 5 Done Done Sent false
34985 9916 Acked Write 7 Done Done Sent false
34986 9917 Acked Write 8 Done Done Sent false
34987 9918 Acked Read 7 Done Done Sent false
34988 9919 Acked Read 1 Done Done Sent false
34989 9920 Acked Write 7 Done Done Sent false
34990 9921 Acked Read 10 Done Done Sent false
34991 9922 Acked Read 7 Done Done Sent false
34992 9923 Acked Write 6 Done Done Sent false
34993 9924 Acked Read 6 Done Done Sent false
34994 9925 Acked Read 5 Done Done Sent false
34995 9926 Acked Flush 0 Done Done Sent false
34996 9927 Acked Read 1 Done Done Sent false
34997 9928 Acked Flush 0 Done Done Sent false
34998 9929 Acked Read 3 Done Done Sent false
34999 9930 Acked Read 6 Done Done Sent false
35000 9931 Acked Write 3 Done Done Sent false
35001 9932 Acked Read 2 Done Done Sent false
35002 9933 Acked Write 8 Done Done Sent false
35003 9934 Acked Write 3 Done Done Sent false
35004 9935 Acked Read 8 Done Done Sent false
35005 9936 Acked Write 8 Done Done Sent false
35006 9937 Acked Read 3 Done Done Sent false
35007 9938 Acked Write 7 Done Done Sent false
35008 9939 Acked Write 7 Done Done Sent false
35009 9940 Acked Write 6 Done Done Sent false
35010 9941 Acked Read 7 Done Done Sent false
35011 9942 Acked Read 4 Done Done Sent false
35012 9943 Acked Flush 0 Done Done Sent false
35013 9944 Acked Write 9 Done Done Sent false
35014 9945 Acked Write 9 Done Done Sent false
35015 9946 Acked Read 4 Done Done Sent false
35016 9947 Acked Write 7 Done Done Sent false
35017 9948 Acked Write 8 Done Done Sent false
35018 9949 Acked Read 8 Done Done Sent false
35019 9950 Acked Flush 0 Done Done Sent false
35020 9951 Acked Write 7 Done Done Sent false
35021 9952 Acked Read 2 Done Done Sent false
35022 9953 Acked Read 5 Done Done Sent false
35023 9954 Acked Read 6 Done Done Sent false
35024 9955 Acked Flush 0 Done Done Sent false
35025 9956 Acked Write 10 Done Done Sent false
35026 9957 Acked Read 6 Done Done Sent false
35027 9958 Acked Read 4 Done Done Sent false
35028 9959 Acked Write 10 Done Done Sent false
35029 9960 Acked Read 9 Done Done Sent false
35030 9961 Acked Read 5 Done Done Sent false
35031 9962 Acked Read 9 Done Done Sent false
35032 9963 Acked Write 3 Done Done Sent false
35033 9964 Acked Read 8 Done Done Sent false
35034 9965 Acked Read 6 Done Done Sent false
35035 9966 Acked Read 3 Done Done Sent false
35036 9967 Acked Write 9 Done Done Sent false
35037 9968 Acked Flush 0 Done Done Sent false
35038 9969 Acked Write 7 Done Done Sent false
35039 9970 Acked Write 5 Done Done Sent false
35040 9971 Acked Write 5 Done Done Sent false
35041 9972 Acked Read 5 Done Done Sent false
35042 9973 Acked Read 1 Done Done Sent false
35043 9974 Acked Write 3 Done Done Sent false
35044 9975 Acked Read 5 Done Done Sent false
35045 9976 Acked Write 9 Done Done Sent false
35046 9977 Acked Write 1 Done Done Sent false
35047 9978 Acked Write 9 Done Done Sent false
35048 9979 Acked Write 5 Done Done Sent false
35049 9980 Acked Write 5 Done Done Sent false
35050 9981 Acked Read 8 Done Done Sent false
35051 9982 Acked Read 9 Done Done Sent false
35052 9983 Acked Read 2 Done Done Sent false
35053 9984 Acked Write 2 Done Done Sent false
35054 9985 Acked Read 8 Done Done Sent false
35055 9986 Acked Write 4 Done Done Sent false
35056 9987 Acked Flush 0 Done Done Sent false
35057 9988 Acked Write 7 Done Done Sent false
35058 9989 Acked Read 1 Done Done Sent false
35059 9990 Acked Read 9 Done Done Sent false
35060 9991 Acked Read 3 Done Done Sent false
35061 9992 Acked Read 7 Done Done Sent false
35062 9993 Acked Read 1 Done Done Sent false
35063 9994 Acked Read 1 Done Done Sent false
35064 9995 Acked Write 10 Done Done Sent false
35065 9996 Acked Read 4 Done Done Sent false
35066 9997 Acked Flush 0 Done Done Sent false
35067 9998 Acked Write 9 Done Done Sent false
35068 9999 Acked Write 1 Done Done Sent false
35069 10000 Acked Read 6 Done Done Sent false
35070 10001 Acked Read 4 Done Done Sent false
35071 10002 Acked Read 3 Done Done Sent false
35072 10003 Acked Read 4 Done Done Sent false
35073 10004 Acked Read 3 Done Done Sent false
35074 10005 Acked Read 4 Done Done Sent false
35075 10006 Acked Write 3 Done Done Sent false
35076 10007 Acked Read 3 Done Done Sent false
35077 10008 Acked Write 1 Done Done Sent false
35078 10009 Acked Read 6 Done Done Sent false
35079 10010 Acked Write 5 Done Done Sent false
35080 10011 Acked Read 2 Done Done Sent false
35081 10012 Acked Read 10 Done Done Sent false
35082 10013 Acked Write 3 Done Done Sent false
35083 10014 Acked Read 10 Done Done Sent false
35084 10015 Acked Read 9 Done Done Sent false
35085 10016 Acked Write 1 Done Done Sent false
35086 10017 Acked Write 3 Done Done Sent false
35087 10018 Acked Read 9 Done Done Sent false
35088 10019 Acked Read 7 Done Done Sent false
35089 10020 Acked Read 1 Done Done Sent false
35090 10021 Acked Flush 0 Done Done Sent false
35091 10022 Acked Write 9 Done Done Sent false
35092 10023 Acked Read 7 Done Done Sent false
35093 10024 Acked Write 8 Done Done Sent false
35094 10025 Acked Read 2 Done Done Sent false
35095 10026 Acked Write 3 Done Done Sent false
35096 10027 Acked Read 5 Done Done Sent false
35097 10028 Acked Write 4 Done Done Sent false
35098 10029 Acked Read 7 Done Done Sent false
35099 10030 Acked Read 9 Done Done Sent false
35100 10031 Acked Write 3 Done Done Sent false
35101 10032 Acked Read 1 Done Done Sent false
35102 10033 Acked Flush 0 Done Done Sent false
35103 10034 Acked Flush 0 Done Done Sent false
35104 10035 Acked Read 7 Done Done Sent false
35105 10036 Acked Write 8 Done Done Sent false
35106 10037 Acked Read 7 Done Done Sent false
35107 10038 Acked Write 6 Done Done Sent false
35108 10039 Acked Write 3 Done Done Sent false
35109 10040 Acked Read 3 Done Done Sent false
35110 10041 Acked Read 4 Done Done Sent false
35111 10042 Acked Read 4 Done Done Sent false
35112 10043 Acked Write 4 Done Done Sent false
35113 10044 Acked Read 2 Done Done Sent false
35114 10045 Acked Write 6 Done Done Sent false
35115 10046 Acked Read 9 Done Done Sent false
35116 10047 Acked Write 7 Done Done Sent false
35117 10048 Acked Read 8 Done Done Sent false
35118 10049 Acked Read 4 Done Done Sent false
35119 10050 Acked Write 7 Done Done Sent false
35120 10051 Acked Read 7 Done Done Sent false
35121 10052 Acked Read 7 Done Done Sent false
35122 10053 Acked Flush 0 Done Done Sent false
35123 10054 Acked Flush 0 Done Done Sent false
35124 10055 Acked Write 4 Done Done Sent false
35125 STATES DS:0 DS:1 DS:2 TOTAL
35126 Sent 0 0 301 301
35127 Done 301 301 0 602
35128 Skipped 0 0 0 0
35129 Error 0 0 0 0
35130 Last Flush: 10054 10054 9754
35131 Downstairs last five completed: 9754 9753 9752 9751 9750
35132 Upstairs last five completed: 10055 10054 10053 10052 10051
35133 CLIENT: Up:0 ds:301 act:2
35134 [8830] got start action
35135 [8830] I am going to restart
35136 Starting downstairs at port 8830
35137 Make output file at "/tmp/dsc/downstairs-8830.txt"
35138 [8830][2] reports Starting
35139 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1233)
35140 [8830][2] reports Running
35141 ----------------------------------------------------------------
35142 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
35143 States: Active Active Offline
35144 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
35145 9755 Acked Read 6 Done Done Sent false
35146 9756 Acked Write 4 Done Done Sent false
35147 9757 Acked Write 5 Done Done Sent false
35148 9758 Acked Write 8 Done Done Sent false
35149 9759 Acked Read 8 Done Done Sent false
35150 9760 Acked Read 3 Done Done Sent false
35151 9761 Acked Write 8 Done Done Sent false
35152 9762 Acked Write 1 Done Done Sent false
35153 9763 Acked Read 3 Done Done Sent false
35154 9764 Acked Write 6 Done Done Sent false
35155 9765 Acked Flush 0 Done Done Sent false
35156 9766 Acked Write 3 Done Done Sent false
35157 9767 Acked Write 10 Done Done Sent false
35158 9768 Acked Read 10 Done Done Sent false
35159 9769 Acked Write 4 Done Done Sent false
35160 9770 Acked Read 9 Done Done Sent false
35161 9771 Acked Write 5 Done Done Sent false
35162 9772 Acked Read 5 Done Done Sent false
35163 9773 Acked Write 2 Done Done Sent false
35164 9774 Acked Read 6 Done Done Sent false
35165 9775 Acked Read 1 Done Done Sent false
35166 9776 Acked Read 9 Done Done Sent false
35167 9777 Acked Flush 0 Done Done Sent false
35168 9778 Acked Write 4 Done Done Sent false
35169 9779 Acked Write 1 Done Done Sent false
35170 9780 Acked Read 3 Done Done Sent false
35171 9781 Acked Read 2 Done Done Sent false
35172 9782 Acked Read 6 Done Done Sent false
35173 9783 Acked Write 7 Done Done Sent false
35174 9784 Acked Read 7 Done Done Sent false
35175 9785 Acked Read 2 Done Done Sent false
35176 9786 Acked Write 4 Done Done Sent false
35177 9787 Acked Write 1 Done Done Sent false
35178 9788 Acked Read 3 Done Done Sent false
35179 9789 Acked Write 2 Done Done Sent false
35180 9790 Acked Write 1 Done Done Sent false
35181 9791 Acked Read 6 Done Done Sent false
35182 9792 Acked Read 8 Done Done Sent false
35183 9793 Acked Read 8 Done Done Sent false
35184 9794 Acked Write 5 Done Done Sent false
35185 9795 Acked Read 2 Done Done Sent false
35186 9796 Acked Read 7 Done Done Sent false
35187 9797 Acked Write 9 Done Done Sent false
35188 9798 Acked Write 8 Done Done Sent false
35189 9799 Acked Write 6 Done Done Sent false
35190 9800 Acked Write 9 Done Done Sent false
35191 9801 Acked Read 10 Done Done Sent false
35192 9802 Acked Read 10 Done Done Sent false
35193 9803 Acked Read 10 Done Done Sent false
35194 9804 Acked Read 6 Done Done Sent false
35195 9805 Acked Read 7 Done Done Sent false
35196 9806 Acked Read 8 Done Done Sent false
35197 9807 Acked Write 5 Done Done Sent false
35198 9808 Acked Read 3 Done Done Sent false
35199 9809 Acked Write 6 Done Done Sent false
35200 9810 Acked Flush 0 Done Done Sent false
35201 9811 Acked Read 1 Done Done Sent false
35202 9812 Acked Write 10 Done Done Sent false
35203 9813 Acked Write 3 Done Done Sent false
35204 9814 Acked Write 5 Done Done Sent false
35205 9815 Acked Read 10 Done Done Sent false
35206 9816 Acked Read 3 Done Done Sent false
35207 9817 Acked Write 10 Done Done Sent false
35208 9818 Acked Read 6 Done Done Sent false
35209 9819 Acked Read 2 Done Done Sent false
35210 9820 Acked Read 8 Done Done Sent false
35211 9821 Acked Read 3 Done Done Sent false
35212 9822 Acked Write 1 Done Done Sent false
35213 9823 Acked Read 7 Done Done Sent false
35214 9824 Acked Write 2 Done Done Sent false
35215 9825 Acked Write 9 Done Done Sent false
35216 9826 Acked Write 3 Done Done Sent false
35217 9827 Acked Read 5 Done Done Sent false
35218 9828 Acked Read 5 Done Done Sent false
35219 9829 Acked Read 5 Done Done Sent false
35220 9830 Acked Read 5 Done Done Sent false
35221 9831 Acked Write 5 Done Done Sent false
35222 9832 Acked Write 9 Done Done Sent false
35223 9833 Acked Flush 0 Done Done Sent false
35224 9834 Acked Write 7 Done Done Sent false
35225 9835 Acked Read 8 Done Done Sent false
35226 9836 Acked Write 8 Done Done Sent false
35227 9837 Acked Read 5 Done Done Sent false
35228 9838 Acked Write 2 Done Done Sent false
35229 9839 Acked Write 1 Done Done Sent false
35230 9840 Acked Write 8 Done Done Sent false
35231 9841 Acked Read 3 Done Done Sent false
35232 9842 Acked Read 10 Done Done Sent false
35233 9843 Acked Write 9 Done Done Sent false
35234 9844 Acked Write 9 Done Done Sent false
35235 9845 Acked Read 8 Done Done Sent false
35236 9846 Acked Flush 0 Done Done Sent false
35237 9847 Acked Flush 0 Done Done Sent false
35238 9848 Acked Read 5 Done Done Sent false
35239 9849 Acked Read 4 Done Done Sent false
35240 9850 Acked Read 3 Done Done Sent false
35241 9851 Acked Read 4 Done Done Sent false
35242 9852 Acked Write 6 Done Done Sent false
35243 9853 Acked Write 8 Done Done Sent false
35244 9854 Acked Read 2 Done Done Sent false
35245 9855 Acked Flush 0 Done Done Sent false
35246 9856 Acked Write 4 Done Done Sent false
35247 9857 Acked Read 1 Done Done Sent false
35248 9858 Acked Read 3 Done Done Sent false
35249 9859 Acked Read 4 Done Done Sent false
35250 9860 Acked Read 6 Done Done Sent false
35251 9861 Acked Read 8 Done Done Sent false
35252 9862 Acked Write 3 Done Done Sent false
35253 9863 Acked Write 9 Done Done Sent false
35254 9864 Acked Read 10 Done Done Sent false
35255 9865 Acked Read 5 Done Done Sent false
35256 9866 Acked Write 5 Done Done Sent false
35257 9867 Acked Read 7 Done Done Sent false
35258 9868 Acked Read 10 Done Done Sent false
35259 9869 Acked Read 9 Done Done Sent false
35260 9870 Acked Read 5 Done Done Sent false
35261 9871 Acked Write 10 Done Done Sent false
35262 9872 Acked Write 5 Done Done Sent false
35263 9873 Acked Read 3 Done Done Sent false
35264 9874 Acked Read 8 Done Done Sent false
35265 9875 Acked Read 1 Done Done Sent false
35266 9876 Acked Read 4 Done Done Sent false
35267 9877 Acked Flush 0 Done Done Sent false
35268 9878 Acked Write 10 Done Done Sent false
35269 9879 Acked Write 8 Done Done Sent false
35270 9880 Acked Read 8 Done Done Sent false
35271 9881 Acked Read 2 Done Done Sent false
35272 9882 Acked Write 7 Done Done Sent false
35273 9883 Acked Write 10 Done Done Sent false
35274 9884 Acked Write 4 Done Done Sent false
35275 9885 Acked Write 3 Done Done Sent false
35276 9886 Acked Read 5 Done Done Sent false
35277 9887 Acked Read 6 Done Done Sent false
35278 9888 Acked Write 2 Done Done Sent false
35279 9889 Acked Read 10 Done Done Sent false
35280 9890 Acked Read 2 Done Done Sent false
35281 9891 Acked Read 6 Done Done Sent false
35282 9892 Acked Write 6 Done Done Sent false
35283 9893 Acked Write 1 Done Done Sent false
35284 9894 Acked Read 5 Done Done Sent false
35285 9895 Acked Read 9 Done Done Sent false
35286 9896 Acked Read 1 Done Done Sent false
35287 9897 Acked Read 10 Done Done Sent false
35288 9898 Acked Read 5 Done Done Sent false
35289 9899 Acked Read 8 Done Done Sent false
35290 9900 Acked Read 9 Done Done Sent false
35291 9901 Acked Read 5 Done Done Sent false
35292 9902 Acked Write 10 Done Done Sent false
35293 9903 Acked Read 8 Done Done Sent false
35294 9904 Acked Read 1 Done Done Sent false
35295 9905 Acked Read 1 Done Done Sent false
35296 9906 Acked Read 6 Done Done Sent false
35297 9907 Acked Read 6 Done Done Sent false
35298 9908 Acked Write 8 Done Done Sent false
35299 9909 Acked Write 6 Done Done Sent false
35300 9910 Acked Write 7 Done Done Sent false
35301 9911 Acked Read 4 Done Done Sent false
35302 9912 Acked Read 3 Done Done Sent false
35303 9913 Acked Read 10 Done Done Sent false
35304 9914 Acked Flush 0 Done Done Sent false
35305 9915 Acked Read 5 Done Done Sent false
35306 9916 Acked Write 7 Done Done Sent false
35307 9917 Acked Write 8 Done Done Sent false
35308 9918 Acked Read 7 Done Done Sent false
35309 9919 Acked Read 1 Done Done Sent false
35310 9920 Acked Write 7 Done Done Sent false
35311 9921 Acked Read 10 Done Done Sent false
35312 9922 Acked Read 7 Done Done Sent false
35313 9923 Acked Write 6 Done Done Sent false
35314 9924 Acked Read 6 Done Done Sent false
35315 9925 Acked Read 5 Done Done Sent false
35316 9926 Acked Flush 0 Done Done Sent false
35317 9927 Acked Read 1 Done Done Sent false
35318 9928 Acked Flush 0 Done Done Sent false
35319 9929 Acked Read 3 Done Done Sent false
35320 9930 Acked Read 6 Done Done Sent false
35321 9931 Acked Write 3 Done Done Sent false
35322 9932 Acked Read 2 Done Done Sent false
35323 9933 Acked Write 8 Done Done Sent false
35324 9934 Acked Write 3 Done Done Sent false
35325 9935 Acked Read 8 Done Done Sent false
35326 9936 Acked Write 8 Done Done Sent false
35327 9937 Acked Read 3 Done Done Sent false
35328 9938 Acked Write 7 Done Done Sent false
35329 9939 Acked Write 7 Done Done Sent false
35330 9940 Acked Write 6 Done Done Sent false
35331 9941 Acked Read 7 Done Done Sent false
35332 9942 Acked Read 4 Done Done Sent false
35333 9943 Acked Flush 0 Done Done Sent false
35334 9944 Acked Write 9 Done Done Sent false
35335 9945 Acked Write 9 Done Done Sent false
35336 9946 Acked Read 4 Done Done Sent false
35337 9947 Acked Write 7 Done Done Sent false
35338 9948 Acked Write 8 Done Done Sent false
35339 9949 Acked Read 8 Done Done Sent false
35340 9950 Acked Flush 0 Done Done Sent false
35341 9951 Acked Write 7 Done Done Sent false
35342 9952 Acked Read 2 Done Done Sent false
35343 9953 Acked Read 5 Done Done Sent false
35344 9954 Acked Read 6 Done Done Sent false
35345 9955 Acked Flush 0 Done Done Sent false
35346 9956 Acked Write 10 Done Done Sent false
35347 9957 Acked Read 6 Done Done Sent false
35348 9958 Acked Read 4 Done Done Sent false
35349 9959 Acked Write 10 Done Done Sent false
35350 9960 Acked Read 9 Done Done Sent false
35351 9961 Acked Read 5 Done Done Sent false
35352 9962 Acked Read 9 Done Done Sent false
35353 9963 Acked Write 3 Done Done Sent false
35354 9964 Acked Read 8 Done Done Sent false
35355 9965 Acked Read 6 Done Done Sent false
35356 9966 Acked Read 3 Done Done Sent false
35357 9967 Acked Write 9 Done Done Sent false
35358 9968 Acked Flush 0 Done Done Sent false
35359 9969 Acked Write 7 Done Done Sent false
35360 9970 Acked Write 5 Done Done Sent false
35361 9971 Acked Write 5 Done Done Sent false
35362 9972 Acked Read 5 Done Done Sent false
35363 9973 Acked Read 1 Done Done Sent false
35364 9974 Acked Write 3 Done Done Sent false
35365 9975 Acked Read 5 Done Done Sent false
35366 9976 Acked Write 9 Done Done Sent false
35367 9977 Acked Write 1 Done Done Sent false
35368 9978 Acked Write 9 Done Done Sent false
35369 9979 Acked Write 5 Done Done Sent false
35370 9980 Acked Write 5 Done Done Sent false
35371 9981 Acked Read 8 Done Done Sent false
35372 9982 Acked Read 9 Done Done Sent false
35373 9983 Acked Read 2 Done Done Sent false
35374 9984 Acked Write 2 Done Done Sent false
35375 9985 Acked Read 8 Done Done Sent false
35376 9986 Acked Write 4 Done Done Sent false
35377 9987 Acked Flush 0 Done Done Sent false
35378 9988 Acked Write 7 Done Done Sent false
35379 9989 Acked Read 1 Done Done Sent false
35380 9990 Acked Read 9 Done Done Sent false
35381 9991 Acked Read 3 Done Done Sent false
35382 9992 Acked Read 7 Done Done Sent false
35383 9993 Acked Read 1 Done Done Sent false
35384 9994 Acked Read 1 Done Done Sent false
35385 9995 Acked Write 10 Done Done Sent false
35386 9996 Acked Read 4 Done Done Sent false
35387 9997 Acked Flush 0 Done Done Sent false
35388 9998 Acked Write 9 Done Done Sent false
35389 9999 Acked Write 1 Done Done Sent false
35390 10000 Acked Read 6 Done Done Sent false
35391 10001 Acked Read 4 Done Done Sent false
35392 10002 Acked Read 3 Done Done Sent false
35393 10003 Acked Read 4 Done Done Sent false
35394 10004 Acked Read 3 Done Done Sent false
35395 10005 Acked Read 4 Done Done Sent false
35396 10006 Acked Write 3 Done Done Sent false
35397 10007 Acked Read 3 Done Done Sent false
35398 10008 Acked Write 1 Done Done Sent false
35399 10009 Acked Read 6 Done Done Sent false
35400 10010 Acked Write 5 Done Done Sent false
35401 10011 Acked Read 2 Done Done Sent false
35402 10012 Acked Read 10 Done Done Sent false
35403 10013 Acked Write 3 Done Done Sent false
35404 10014 Acked Read 10 Done Done Sent false
35405 10015 Acked Read 9 Done Done Sent false
35406 10016 Acked Write 1 Done Done Sent false
35407 10017 Acked Write 3 Done Done Sent false
35408 10018 Acked Read 9 Done Done Sent false
35409 10019 Acked Read 7 Done Done Sent false
35410 10020 Acked Read 1 Done Done Sent false
35411 10021 Acked Flush 0 Done Done Sent false
35412 10022 Acked Write 9 Done Done Sent false
35413 10023 Acked Read 7 Done Done Sent false
35414 10024 Acked Write 8 Done Done Sent false
35415 10025 Acked Read 2 Done Done Sent false
35416 10026 Acked Write 3 Done Done Sent false
35417 10027 Acked Read 5 Done Done Sent false
35418 10028 Acked Write 4 Done Done Sent false
35419 10029 Acked Read 7 Done Done Sent false
35420 10030 Acked Read 9 Done Done Sent false
35421 10031 Acked Write 3 Done Done Sent false
35422 10032 Acked Read 1 Done Done Sent false
35423 10033 Acked Flush 0 Done Done Sent false
35424 10034 Acked Flush 0 Done Done Sent false
35425 10035 Acked Read 7 Done Done Sent false
35426 10036 Acked Write 8 Done Done Sent false
35427 10037 Acked Read 7 Done Done Sent false
35428 10038 Acked Write 6 Done Done Sent false
35429 10039 Acked Write 3 Done Done Sent false
35430 10040 Acked Read 3 Done Done Sent false
35431 10041 Acked Read 4 Done Done Sent false
35432 10042 Acked Read 4 Done Done Sent false
35433 10043 Acked Write 4 Done Done Sent false
35434 10044 Acked Read 2 Done Done Sent false
35435 10045 Acked Write 6 Done Done Sent false
35436 10046 Acked Read 9 Done Done Sent false
35437 10047 Acked Write 7 Done Done Sent false
35438 10048 Acked Read 8 Done Done Sent false
35439 10049 Acked Read 4 Done Done Sent false
35440 10050 Acked Write 7 Done Done Sent false
35441 10051 Acked Read 7 Done Done Sent false
35442 10052 Acked Read 7 Done Done Sent false
35443 10053 Acked Flush 0 Done Done Sent false
35444 10054 Acked Flush 0 Done Done Sent false
35445 10055 Acked Write 4 Done Done Sent false
35446 10056 Acked Flush 0 Done Done Sent false
35447 STATES DS:0 DS:1 DS:2 TOTAL
35448 Sent 0 0 302 302
35449 Done 302 302 0 604
35450 Skipped 0 0 0 0
35451 Error 0 0 0 0
35452 Last Flush: 10056 10056 9754
35453 Downstairs last five completed: 9754 9753 9752 9751 9750
35454 Upstairs last five completed: 10056 10055 10054 10053 10052
35455 CLIENT: Up:0 ds:302 act:2
354562024-11-26T22:11:57.509ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
354572024-11-26T22:11:57.509ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:32895) = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
354582024-11-26T22:11:57.510ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
354592024-11-26T22:11:57.511ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
354602024-11-26T22:11:57.511ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
354612024-11-26T22:11:57.511ZINFOcrucible: send last flush ID to this DS: 9754 = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
354622024-11-26T22:11:57.511ZINFOcrucible: Replied this last flush ID: 9754 = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
354632024-11-26T22:11:57.511ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
354642024-11-26T22:11:57.511ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
354652024-11-26T22:11:57.511ZINFOcrucible: [2] client re-new 302 jobs since flush 9754 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
354662024-11-26T22:11:57.511ZINFOcrucible: [2] Replayed 302 jobs since flush: 9754 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
35467 ----------------------------------------------------------------
35468 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
35469 Downstairs last five completed: 10056 10055 10054 10053 10052
35470 Upstairs last five completed: 10056 10055 10054 10053 10052
35471 CLIENT: Up:0 ds:0 act:3
35472 Replay: All jobs finished
35473 Main task has work to do, go find it
35474 got dsc Stop(2)
35475 stop 2
35476 [8830] Got stop action so:false kr:false
35477 Nov 26 22:11:59.760 INFO request completed, latency_us: 339, response_code: 204, uri: /stop/cid/2, method: POST, req_id: feff6640-177d-42a4-afe3-e69c726a76cf, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
354782024-11-26T22:11:59.761ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
354792024-11-26T22:11:59.761ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
35480 Nov 26 22:11:59.761 INFO request completed, latency_us: 131, response_code: 200{"msg":"downstairs task for 2 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2024-11-26T22:11:59.761306525Z","hostname":"w-01JDN7ZQ17ABQG15KHKN3F2D26","pid":1203,"session_id":"86a36c17-3718-484c-87fd-2b8a1dc2ab18"}
35481 , uri: /state/cid/2, method: GET, req_id: 3b16fa8b-7366-4053-8e0d-fb4f6451482e, remote_addr: 127.0.{"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-26T22:11:59.761330602Z","hostname":"w-01JDN7ZQ17ABQG15KHKN3F2D26","pid":1203,"client":"2","":"downstairs","session_id":"86a36c17-3718-484c-87fd-2b8a1dc2ab18"}
35482 0.1:36829, local_addr: 127.0{"msg":"[2] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-26T22:11:59.761356748Z","hostname":"w-01JDN7ZQ17ABQG15KHKN3F2D26","pid":1203,"client":"2","":"downstairs","session_id":"86a36c17-3718-484c-87fd-2b8a1dc2ab18"}
35483 .{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2024-11-26T22:11:59.761391584Z","hostname":"w-01JDN7ZQ17ABQG15KHKN3F2D26","pid":1203,"":"io task","client":"2","":"downstairs","session_id":"86a36c17-3718-484c-87fd-2b8a1dc2ab18"}
35484 0.1:9998
35485 [8830] Exited with: ExitStatus(unix_wait_status(9))
35486 [8830][2] reports Exit
35487 Nov 26 22:12:03.762 INFO request completed, latency_us: 193, response_code: 200, uri: /state/cid/2, method: GET, req_id: 6daa529d-040d-46e0-986a-75f23ae4d27a, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
35488 001/300 Read block 4308 len 24576
35489 002/300 Write block 684 len 36864 data: 5 4 4 3 3 3 4 3 3
35490 003/300 Read block 4672 len 4096
35491 004/300 Read block 2365 len 12288
35492 005/300 Write block 3338 len 36864 data: 3 4 3 4 4 4 5 4 5
35493 006/300 Write block 432 len 24576 data: 6 7 8 6 6 5
35494 007/300 Write block 2881 len 16384 data: 4 4 6 6
35495 008/300 Write block 2108 len 4096 data: 7
35496 009/300 Write block 3369 len 36864 data: 3 3 4 4 5 6 7 7 6
35497 010/300 Write block 2728 len 12288 data: 7 7 8
35498 011/300 Read block 4946 len 28672
35499 012/300 Flush
35500 013/300 Read block 2867 len 12288
35501 014/300 Read block 3302 len 20480
35502 015/300 Flush
35503 016/300 Write block 894 len 12288 data: 7 5 5
35504 017/300 Read block 2181 len 40960
35505 018/300 Flush
35506 019/300 Flush
35507 020/300 Flush
35508 021/300 Write block 939 len 20480 data: 6 8 7 7 6
35509 022/300 Write block 4649 len 16384 data: 7 7 6 5
35510 023/300 Write block 3813 len 8192 data: 4 4
35511 024/300 Write block 116 len 12288 data: 4 5 7
35512 025/300 Write block 2346 len 24576 data: 4 3 4 5 5 7
35513 026/300 Read block 1747 len 32768
35514 027/300 Read block 4688 len 36864
35515 028/300 Read block 1421 len 28672
35516 029/300 Write block 603 len 4096 data: 6
35517 030/300 Read block 4573 len 40960
35518 031/300 Read block 3281 len 16384
35519 032/300 Read block 1281 len 32768
35520 033/300 Write block 2895 len 24576 data: 4 4 4 4 5 4
35521 034/300 Write block 265 len 20480 data: 5 3 4 4 5
35522 035/300 Read block 3126 len 20480
35523 036/300 Write block 3790 len 8192 data: 7 9
35524 037/300 Read block 2534 len 12288
35525 038/300 Read block 3153 len 32768
35526 039/300 Write block 1731 len 28672 data: 7 7 7 8 9 9 8
35527 040/300 Read block 1095 len 8192
35528 041/300 Flush
35529 042/300 Flush
35530 043/300 Write block 4344 len 24576 data: 12 11 9 10 10 8
35531 044/300 Read block 2648 len 32768
35532 045/300 Read block 508 len 12288
35533 046/300 Write block 2219 len 32768 data: 5 4 4 3 2 3 4 4
35534 047/300 Read block 4424 len 28672
35535 048/300 Read block 93 len 12288
35536 049/300 Flush
35537 050/300 Read block 2988 len 28672
35538 051/300 Flush
35539 052/300 Read block 2870 len 40960
35540 053/300 Read block 788 len 32768
35541 054/300 Read block 4346 len 16384
35542 055/300 Flush
35543 056/300 Read block 3767 len 8192
35544 057/300 Flush
35545 058/300 Read block 4393 len 8192
35546 059/300 Read block 527 len 28672
35547 060/300 Write block 3326 len 4096 data: 5
35548 061/300 Write block 2927 len 24576 data: 5 6 6 6 6 7
35549 062/300 Write block 3800 len 40960 data: 5 5 4 6 5 4 4 4 3 3
35550 063/300 Write block 558 len 16384 data: 6 5 5 6
35551 064/300 Read block 2032 len 36864
35552 065/300 Read block 4543 len 40960
35553 066/300 Write block 2467 len 4096 data: 7
35554 067/300 Flush
35555 068/300 Read block 1873 len 24576
35556 069/300 Read block 1658 len 40960
35557 070/300 Read block 413 len 4096
35558 071/300 Write block 4336 len 8192 data: 6 5
35559 072/300 Write block 1498 len 32768 data: 4 7 9 8 7 7 6 5
35560 073/300 Read block 430 len 28672
35561 074/300 Read block 845 len 32768
35562 075/300 Read block 3742 len 8192
35563 076/300 Read block 4480 len 12288
35564 077/300 Read block 1567 len 4096
35565 078/300 Read block 3984 len 32768
35566 079/300 Write block 502 len 28672 data: 7 7 7 8 7 7 6
35567 080/300 Write block 2607 len 20480 data: 11 9 7 8 9
35568 081/300 Read block 206 len 24576
35569 082/300 Read block 53 len 40960
35570 083/300 Write block 4950 len 4096 data: 6
35571 084/300 Read block 3156 len 4096
35572 085/300 Write block 2156 len 28672 data: 5 3 3 3 5 6 8
35573 086/300 Read block 1686 len 32768
35574 087/300 Read block 772 len 40960
35575 088/300 Read block 1035 len 16384
35576 089/300 Read block 1690 len 24576
35577 090/300 Read block 1463 len 8192
35578 091/300 Read block 2172 len 16384
35579 092/300 Read block 4622 len 28672
35580 093/300 Read block 544 len 32768
35581 094/300 Write block 7 len 20480 data: 10 11 10 8 7
35582 095/300 Write block 1730 len 40960 data: 8 8 8 8 9 10 10 9 6 3
35583 096/300 Read block 4222 len 40960
35584 097/300 Flush
35585 098/300 Read block 2376 len 20480
35586 099/300 Read block 1297 len 20480
35587 100/300 Read block 3900 len 16384
35588 101/300 Read block 4551 len 40960
35589 102/300 Write block 4417 len 40960 data: 6 4 4 4 6 6 5 5 5 5
35590 103/300 Read block 886 len 36864
35591 104/300 Read block 1152 len 4096
35592 105/300 Read block 4882 len 20480
35593 106/300 Write block 1637 len 4096 data: 2
35594 107/300 Flush
35595 108/300 Read block 1065 len 24576
35596 109/300 Write block 4000 len 20480 data: 3 4 6 8 10
35597 110/300 Write block 2811 len 24576 data: 8 8 9 7 6 7
35598 111/300 Write block 1311 len 20480 data: 2 2 2 2 2
35599 112/300 Read block 642 len 28672
35600 113/300 Read block 2141 len 16384
35601 114/300 Read block 1886 len 20480
35602 115/300 Flush
35603 116/300 Write block 3934 len 4096 data: 5
35604 117/300 Read block 4229 len 24576
35605 118/300 Read block 1151 len 8192
35606 119/300 Flush
35607 120/300 Read block 3431 len 28672
35608 121/300 Read block 3568 len 40960
35609 122/300 Read block 534 len 32768
35610 123/300 Write block 2565 len 16384 data: 5 5 6 4
35611 124/300 Read block 1379 len 8192
35612 125/300 Read block 4552 len 8192
35613 126/300 Read block 3601 len 32768
35614 127/300 Read block 301 len 40960
35615 128/300 Read block 1963 len 8192
35616 129/300 Write block 1894 len 24576 data: 4 6 7 6 4 3
35617 130/300 Read block 4983 len 4096
35618 131/300 Read block 570 len 12288
35619 132/300 Read block 2770 len 16384
35620 133/300 Read block 3444 len 40960
35621 134/300 Read block 3741 len 4096
35622 135/300 Read block 4992 len 24576
35623 136/300 Flush
35624 137/300 Write block 2887 len 36864 data: 11 11 9 8 6 6 5 5 5
35625 138/300 Read block 2954 len 20480
35626 139/300 Write block 651 len 40960 data: 4 5 5 6 9 9 9 6 7 10
35627 140/300 Write block 548 len 4096 data: 3
35628 141/300 Flush
35629 142/300 Write block 400 len 16384 data: 5 6 8 9
35630 143/300 Read block 1752 len 40960
35631 144/300 Read block 1925 len 24576
35632 145/300 Read block 4530 len 12288
35633 146/300 Read block 2573 len 32768
35634 147/300 Read block 1606 len 24576
35635 148/300 Read block 4377 len 28672
35636 149/300 Write block 4048 len 28672 data: 6 5 4 4 4 5 3
35637 150/300 Flush
35638 151/300 Read block 3322 len 24576
35639 152/300 Read block 559 len 40960
35640 153/300 Read block 2899 len 36864
35641 154/300 Read block 1952 len 16384
35642 155/300 Write block 4709 len 12288 data: 6 8 9
35643 156/300 Write block 577 len 20480 data: 8 7 7 8 7
35644 157/300 Read block 2105 len 4096
35645 158/300 Write block 4368 len 12288 data: 5 3 4
35646 159/300 Write block 2625 len 32768 data: 6 6 6 5 6 6 6 6
35647 160/300 Write block 331 len 24576 data: 6 5 6 5 6 5
35648 161/300 Read block 33 len 20480
35649 162/300 Read block 981 len 24576
35650 163/300 Read block 3927 len 8192
35651 164/300 Read block 1194 len 32768
35652 165/300 Read block 4082 len 12288
35653 166/300 Read block 4170 len 8192
35654 167/300 Write block 2311 len 12288 data: 5 7 7
35655 168/300 Write block 3734 len 40960 data: 4 3 2 2 2 4 6 6 6 5
35656 169/300 Read block 1910 len 12288
35657 170/300 Write block 4822 len 4096 data: 5
35658 171/300 Write block 856 len 12288 data: 5 7 7
35659 172/300 Write block 1578 len 4096 data: 6
35660 173/300 Write block 102 len 12288 data: 8 7 9
35661 174/300 Write block 3192 len 8192 data: 7 8
35662 175/300 Read block 467 len 40960
35663 176/300 Read block 413 len 36864
35664 177/300 Flush
35665 178/300 Read block 1163 len 36864
35666 179/300 Write block 3612 len 36864 data: 6 6 6 6 5 5 6 6 4
35667 180/300 Write block 4969 len 12288 data: 10 9 8
35668 181/300 Write block 318 len 36864 data: 6 6 7 6 6 6 5 6 5
35669 182/300 Read block 2559 len 24576
35670 183/300 Write block 2846 len 32768 data: 11 8 8 6 4 3 2 5
35671 184/300 Read block 2782 len 32768
35672 185/300 Write block 1518 len 8192 data: 4 4
35673 186/300 Flush
35674 187/300 Flush
35675 188/300 Read block 4854 len 12288
35676 189/300 Write block 1739 len 20480 data: 4 3 3 3 2
35677 190/300 Write block 460 len 28672 data: 3 5 6 5 4 4 6
35678 191/300 Read block 4084 len 40960
35679 192/300 Write block 1623 len 4096 data: 8
35680 193/300 Read block 3345 len 24576
35681 194/300 Read block 776 len 24576
35682 195/300 Read block 2929 len 32768
35683 196/300 Read block 72 len 36864
35684 197/300 Read block 836 len 40960
35685 198/300 Write block 3622 len 28672 data: 2 3 6 7 7 6 6
35686 199/300 Write block 3927 len 8192 data: 5 4
35687 200/300 Write block 650 len 32768 data: 3 5 6 6 7 10 10 10
35688 201/300 Write block 1215 len 4096 data: 7
35689 202/300 Read block 937 len 24576
35690 203/300 Read block 1854 len 32768
35691 204/300 Write block 3837 len 8192 data: 4 5
35692 205/300 Write block 2807 len 12288 data: 6 8 10
35693 206/300 Write block 1521 len 28672 data: 6 7 8 9 9 9 8
35694 207/300 Write block 1456 len 4096 data: 9
35695 208/300 Flush
35696 209/300 Write block 4946 len 28672 data: 8 8 9 7 7 5 5
35697 210/300 Flush
35698 211/300 Write block 2747 len 8192 data: 9 9
35699 212/300 Read block 4031 len 16384
35700 213/300 Write block 716 len 28672 data: 8 8 7 6 6 6 7
35701 214/300 Read block 4066 len 40960
35702 215/300 Read block 1935 len 12288
35703 216/300 Write block 2616 len 20480 data: 5 3 4 5 5
35704 217/300 Write block 1193 len 32768 data: 11 11 12 13 15 11 10 10
35705 218/300 Write block 3347 len 28672 data: 5 6 7 7 9 8 7
35706 219/300 Read block 3150 len 36864
35707 220/300 Read block 4396 len 4096
35708 221/300 Read block 3136 len 4096
35709 222/300 Write block 1661 len 4096 data: 7
35710 223/300 Read block 54 len 24576
35711 224/300 Write block 122 len 4096 data: 6
35712 225/300 Flush
35713 226/300 Read block 4987 len 36864
35714 227/300 Flush
35715 228/300 Write block 4879 len 36864 data: 7 7 5 3 3 4 5 5 5
35716 229/300 Write block 4103 len 8192 data: 6 5
35717 230/300 Read block 2522 len 8192
35718 231/300 Read block 2127 len 16384
35719 232/300 Flush
35720 233/300 Write block 3221 len 4096 data: 6
35721 234/300 Read block 1050 len 32768
35722 235/300 Read block 2451 len 40960
35723 236/300 Read block 1764 len 12288
35724 237/300 Write block 1767 len 32768 data: 5 4 6 7 9 7 7 7
35725 238/300 Write block 2631 len 40960 data: 7 7 5 5 5 5 6 6 6 5
35726 239/300 Read block 195 len 16384
35727 240/300 Write block 328 len 24576 data: 6 6 6 7 6 7
35728 241/300 Write block 328 len 16384 data: 7 7 7 8
35729 242/300 Read block 4844 len 16384
35730 243/300 Write block 2581 len 4096 data: 7
35731 244/300 Read block 588 len 32768
35732 245/300 Read block 3382 len 8192
35733 246/300 Write block 3739 len 12288 data: 5 7 7
35734 247/300 Write block 3766 len 12288 data: 7 6 6
35735 248/300 Read block 3214 len 12288
35736 249/300 Write block 2605 len 40960 data: 9 10 12 10 8 9 10 8 5 6
35737 250/300 Read block 506 len 40960
35738 251/300 Read block 1335 len 16384
35739 252/300 Write block 4916 len 36864 data: 5 4 4 4 5 6 7 8 9
35740 253/300 Read block 3894 len 40960
35741 254/300 Read block 3515 len 8192
35742 255/300 Read block 595 len 40960
35743 256/300 Read block 1556 len 12288
35744 257/300 Read block 3543 len 32768
35745 258/300 Write block 1215 len 32768 data: 8 6 4 4 4 4 3 3
35746 259/300 Write block 2505 len 24576 data: 8 6 5 5 4 4
35747 260/300 Read block 4972 len 4096
35748 261/300 Write block 2623 len 8192 data: 5 6
35749 262/300 Flush
35750 263/300 Write block 2859 len 32768 data: 6 7 7 5 7 6 5 5
35751 264/300 Write block 1707 len 36864 data: 3 3 3 4 6 6 5 5 7
35752 265/300 Write block 330 len 8192 data: 8 9
35753 266/300 Read block 1425 len 32768
35754 267/300 Write block 1559 len 40960 data: 6 6 6 7 6 6 6 5 4 4
35755 268/300 Flush
35756 269/300 Read block 3439 len 24576
35757 270/300 Read block 2474 len 32768
35758 271/300 Read block 1211 len 4096
35759 272/300 Write block 2842 len 20480 data: 9 10 11 9 12
35760 273/300 Read block 1955 len 12288
35761 274/300 Read block 2905 len 28672
35762 275/300 Write block 2825 len 16384 data: 6 6 6 6
35763 276/300 Read block 3121 len 8192
35764 277/300 Write block 3369 len 8192 data: 4 4
35765 278/300 Read block 3438 len 16384
35766 279/300 Write block 1940 len 24576 data: 11 10 9 9 9 6
35767 280/300 Read block 2258 len 4096
35768 281/300 Read block 3876 len 32768
35769 282/300 Write block 4769 len 40960 data: 6 7 7 7 8 7 6 6 5 5
35770 283/300 Read block 4987 len 32768
35771 284/300 Flush
35772 285/300 Read block 837 len 20480
35773 286/300 Write block 4655 len 32768 data: 6 6 7 9 11 11 7 5
35774 287/300 Flush
35775 288/300 Write block 660 len 36864 data: 11 8 7 6 4 3 5 7 7
35776 289/300 Read block 909 len 40960
35777 290/300 Write block 4336 len 28672 data: 7 6 5 5 5 6 7
35778 291/300 Flush
35779 292/300 Read block 233 len 28672
35780 293/300 Write block 319 len 4096 data: 7
35781 294/300 Read block 1080 len 16384
35782 295/300 Write block 434 len 28672 data: 9 7 7 6 5 4 5
35783 296/300 Write block 4983 len 36864 data: 8 9 7 7 6 8 11 12 11
35784 297/300 Read block 3502 len 24576
35785 298/300 Read block 4123 len 36864
35786 299/300 Read block 2797 len 28672
35787 300/300 Write block 1713 len 32768 data: 6 6 8 9 8 7 6 6
35788 301/300 Write block 1971 len 40960 data: 4 4 4 4 4 5 4 4 3 4
35789 Main task has work to do, go find it
35790 got dsc Start(2)
35791 start 2
35792 Nov 26 22:12:04.001 INFO request completed, latency_us: 286, response_code: 204, uri: /start/cid/2, method: POST, req_id: a459c725-ea3a-4b52-b31c-923fd4b24f1d, remote_addr: 127.0.0.1:36829, local_addr: 127.0.0.1:9998
35793 Replay: started 2, returned:Ok(())
35794 ----------------------------------------------------------------
35795 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
35796 States: Active Active Offline
35797 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
35798 10057 Acked Read 6 Done Done Sent false
35799 10058 Acked Write 9 Done Done Sent false
35800 10059 Acked Read 1 Done Done Sent false
35801 10060 Acked Read 3 Done Done Sent false
35802 10061 Acked Write 9 Done Done Sent false
35803 10062 Acked Write 6 Done Done Sent false
35804 10063 Acked Write 4 Done Done Sent false
35805 10064 Acked Write 1 Done Done Sent false
35806 10065 Acked Write 9 Done Done Sent false
35807 10066 Acked Write 3 Done Done Sent false
35808 10067 Acked Read 7 Done Done Sent false
35809 10068 Acked Flush 0 Done Done Sent false
35810 10069 Acked Read 3 Done Done Sent false
35811 10070 Acked Read 5 Done Done Sent false
35812 10071 Acked Flush 0 Done Done Sent false
35813 10072 Acked Write 3 Done Done Sent false
35814 10073 Acked Read 10 Done Done Sent false
35815 10074 Acked Flush 0 Done Done Sent false
35816 10075 Acked Flush 0 Done Done Sent false
35817 10076 Acked Flush 0 Done Done Sent false
35818 10077 Acked Write 5 Done Done Sent false
35819 10078 Acked Write 4 Done Done Sent false
35820 10079 Acked Write 2 Done Done Sent false
35821 10080 Acked Write 3 Done Done Sent false
35822 10081 Acked Write 6 Done Done Sent false
35823 10082 Acked Read 8 Done Done Sent false
35824 10083 Acked Read 9 Done Done Sent false
35825 10084 Acked Read 7 Done Done Sent false
35826 10085 Acked Write 1 Done Done Sent false
35827 10086 Acked Read 10 Done Done Sent false
35828 10087 Acked Read 4 Done Done Sent false
35829 10088 Acked Read 8 Done Done Sent false
35830 10089 Acked Write 6 Done Done Sent false
35831 10090 Acked Write 5 Done Done Sent false
35832 10091 Acked Read 5 Done Done Sent false
35833 10092 Acked Write 2 Done Done Sent false
35834 10093 Acked Read 3 Done Done Sent false
35835 10094 Acked Read 8 Done Done Sent false
35836 10095 Acked Write 7 Done Done Sent false
35837 10096 Acked Read 2 Done Done Sent false
35838 10097 Acked Flush 0 Done Done Sent false
35839 10098 Acked Flush 0 Done Done Sent false
35840 10099 Acked Write 6 Done Done Sent false
35841 10100 Acked Read 8 Done Done Sent false
35842 10101 Acked Read 3 Done Done Sent false
35843 10102 Acked Write 8 Done Done Sent false
35844 10103 Acked Read 7 Done Done Sent false
35845 10104 Acked Read 3 Done Done Sent false
35846 10105 Acked Flush 0 Done Done Sent false
35847 10106 Acked Read 7 Done Done Sent false
35848 10107 Acked Flush 0 Done Done Sent false
35849 10108 Acked Read 10 Done Done Sent false
35850 10109 Acked Read 8 Done Done Sent false
35851 10110 Acked Read 4 Done Done Sent false
35852 10111 Acked Flush 0 Done Done Sent false
35853 10112 Acked Read 2 Done Done Sent false
35854 10113 Acked Flush 0 Done Done Sent false
35855 10114 Acked Read 2 Done Done Sent false
35856 10115 Acked Read 7 Done Done Sent false
35857 10116 Acked Write 1 Done Done Sent false
35858 10117 Acked Write 6 Done Done Sent false
35859 10118 Acked Write 10 Done Done Sent false
35860 10119 Acked Write 4 Done Done Sent false
35861 10120 Acked Read 9 Done Done Sent false
35862 10121 Acked Read 10 Done Done Sent false
35863 10122 Acked Write 1 Done Done Sent false
35864 10123 Acked Flush 0 Done Done Sent false
35865 10124 Acked Read 6 Done Done Sent false
35866 10125 Acked Read 10 Done Done Sent false
35867 10126 Acked Read 1 Done Done Sent false
35868 10127 Acked Write 2 Done Done Sent false
35869 10128 Acked Write 8 Done Done Sent false
35870 10129 Acked Read 7 Done Done Sent false
35871 10130 Acked Read 8 Done Done Sent false
35872 10131 Acked Read 2 Done Done Sent false
35873 10132 Acked Read 3 Done Done Sent false
35874 10133 Acked Read 1 Done Done Sent false
35875 10134 Acked Read 8 Done Done Sent false
35876 10135 Acked Write 7 Done Done Sent false
35877 10136 Acked Write 5 Done Done Sent false
35878 10137 Acked Read 6 Done Done Sent false
35879 10138 Acked Read 10 Done Done Sent false
35880 10139 Acked Write 1 Done Done Sent false
35881 10140 Acked Read 1 Done Done Sent false
35882 10141 Acked Write 7 Done Done Sent false
35883 10142 Acked Read 8 Done Done Sent false
35884 10143 Acked Read 10 Done Done Sent false
35885 10144 Acked Read 4 Done Done Sent false
35886 10145 Acked Read 6 Done Done Sent false
35887 10146 Acked Read 2 Done Done Sent false
35888 10147 Acked Read 4 Done Done Sent false
35889 10148 Acked Read 7 Done Done Sent false
35890 10149 Acked Read 8 Done Done Sent false
35891 10150 Acked Write 5 Done Done Sent false
35892 10151 Acked Write 10 Done Done Sent false
35893 10152 Acked Read 10 Done Done Sent false
35894 10153 Acked Flush 0 Done Done Sent false
35895 10154 Acked Read 5 Done Done Sent false
35896 10155 Acked Read 5 Done Done Sent false
35897 10156 Acked Read 4 Done Done Sent false
35898 10157 Acked Read 10 Done Done Sent false
35899 10158 Acked Write 10 Done Done Sent false
35900 10159 Acked Read 9 Done Done Sent false
35901 10160 Acked Read 1 Done Done Sent false
35902 10161 Acked Read 5 Done Done Sent false
35903 10162 Acked Write 1 Done Done Sent false
35904 10163 Acked Flush 0 Done Done Sent false
35905 10164 Acked Read 6 Done Done Sent false
35906 10165 Acked Write 5 Done Done Sent false
35907 10166 Acked Write 6 Done Done Sent false
35908 10167 Acked Write 5 Done Done Sent false
35909 10168 Acked Read 7 Done Done Sent false
35910 10169 Acked Read 4 Done Done Sent false
35911 10170 Acked Read 5 Done Done Sent false
35912 10171 Acked Flush 0 Done Done Sent false
35913 10172 Acked Write 1 Done Done Sent false
35914 10173 Acked Read 6 Done Done Sent false
35915 10174 Acked Read 2 Done Done Sent false
35916 10175 Acked Flush 0 Done Done Sent false
35917 10176 Acked Read 7 Done Done Sent false
35918 10177 Acked Read 10 Done Done Sent false
35919 10178 Acked Read 8 Done Done Sent false
35920 10179 Acked Write 4 Done Done Sent false
35921 10180 Acked Read 2 Done Done Sent false
35922 10181 Acked Read 2 Done Done Sent false
35923 10182 Acked Read 8 Done Done Sent false
35924 10183 Acked Read 10 Done Done Sent false
35925 10184 Acked Read 2 Done Done Sent false
35926 10185 Acked Write 6 Done Done Sent false
35927 10186 Acked Read 1 Done Done Sent false
35928 10187 Acked Read 3 Done Done Sent false
35929 10188 Acked Read 4 Done Done Sent false
35930 10189 Acked Read 10 Done Done Sent false
35931 10190 Acked Read 1 Done Done Sent false
35932 10191 Acked Read 6 Done Done Sent false
35933 10192 Acked Flush 0 Done Done Sent false
35934 10193 Acked Write 9 Done Done Sent false
35935 10194 Acked Read 5 Done Done Sent false
35936 10195 Acked Write 10 Done Done Sent false
35937 10196 Acked Write 1 Done Done Sent false
35938 10197 Acked Flush 0 Done Done Sent false
35939 10198 Acked Write 4 Done Done Sent false
35940 10199 Acked Read 10 Done Done Sent false
35941 10200 Acked Read 6 Done Done Sent false
35942 10201 Acked Read 3 Done Done Sent false
35943 10202 Acked Read 8 Done Done Sent false
35944 10203 Acked Read 6 Done Done Sent false
35945 10204 Acked Read 7 Done Done Sent false
35946 10205 Acked Write 7 Done Done Sent false
35947 10206 Acked Flush 0 Done Done Sent false
35948 10207 Acked Read 6 Done Done Sent false
35949 10208 Acked Read 10 Done Done Sent false
35950 10209 Acked Read 9 Done Done Sent false
35951 10210 Acked Read 4 Done Done Sent false
35952 10211 Acked Write 3 Done Done Sent false
35953 10212 Acked Write 5 Done Done Sent false
35954 10213 Acked Read 1 Done Done Sent false
35955 10214 Acked Write 3 Done Done Sent false
35956 10215 Acked Write 8 Done Done Sent false
35957 10216 Acked Write 6 Done Done Sent false
35958 10217 Acked Read 5 Done Done Sent false
35959 10218 Acked Read 6 Done Done Sent false
35960 10219 Acked Read 2 Done Done Sent false
35961 10220 Acked Read 8 Done Done Sent false
35962 10221 Acked Read 3 Done Done Sent false
35963 10222 Acked Read 2 Done Done Sent false
35964 10223 Acked Write 3 Done Done Sent false
35965 10224 Acked Write 10 Done Done Sent false
35966 10225 Acked Read 3 Done Done Sent false
35967 10226 Acked Write 1 Done Done Sent false
35968 10227 Acked Write 3 Done Done Sent false
35969 10228 Acked Write 1 Done Done Sent false
35970 10229 Acked Write 3 Done Done Sent false
35971 10230 Acked Write 2 Done Done Sent false
35972 10231 Acked Read 10 Done Done Sent false
35973 10232 Acked Read 9 Done Done Sent false
35974 10233 Acked Flush 0 Done Done Sent false
35975 10234 Acked Read 9 Done Done Sent false
35976 10235 Acked Write 9 Done Done Sent false
35977 10236 Acked Write 3 Done Done Sent false
35978 10237 Acked Write 9 Done Done Sent false
35979 10238 Acked Read 6 Done Done Sent false
35980 10239 Acked Write 8 Done Done Sent false
35981 10240 Acked Read 8 Done Done Sent false
35982 10241 Acked Write 2 Done Done Sent false
35983 10242 Acked Flush 0 Done Done Sent false
35984 10243 Acked Flush 0 Done Done Sent false
35985 10244 Acked Read 3 Done Done Sent false
35986 10245 Acked Write 5 Done Done Sent false
35987 10246 Acked Write 7 Done Done Sent false
35988 10247 Acked Read 10 Done Done Sent false
35989 10248 Acked Write 1 Done Done Sent false
35990 10249 Acked Read 6 Done Done Sent false
35991 10250 Acked Read 6 Done Done Sent false
35992 10251 Acked Read 8 Done Done Sent false
35993 10252 Acked Read 9 Done Done Sent false
35994 10253 Acked Read 10 Done Done Sent false
35995 10254 Acked Write 7 Done Done Sent false
35996 10255 Acked Write 2 Done Done Sent false
35997 10256 Acked Write 8 Done Done Sent false
35998 10257 Acked Write 1 Done Done Sent false
35999 10258 Acked Read 6 Done Done Sent false
36000 10259 Acked Read 8 Done Done Sent false
36001 10260 Acked Write 2 Done Done Sent false
36002 10261 Acked Write 3 Done Done Sent false
36003 10262 Acked Write 7 Done Done Sent false
36004 10263 Acked Write 1 Done Done Sent false
36005 10264 Acked Flush 0 Done Done Sent false
36006 10265 Acked Write 7 Done Done Sent false
36007 10266 Acked Flush 0 Done Done Sent false
36008 10267 Acked Write 2 Done Done Sent false
36009 10268 Acked Read 4 Done Done Sent false
36010 10269 Acked Write 7 Done Done Sent false
36011 10270 Acked Read 10 Done Done Sent false
36012 10271 Acked Read 3 Done Done Sent false
36013 10272 Acked Write 5 Done Done Sent false
36014 10273 Acked Write 8 Done Done Sent false
36015 10274 Acked Write 7 Done Done Sent false
36016 10275 Acked Read 9 Done Done Sent false
36017 10276 Acked Read 1 Done Done Sent false
36018 10277 Acked Read 1 Done Done Sent false
36019 10278 Acked Write 1 Done Done Sent false
36020 10279 Acked Read 6 Done Done Sent false
36021 10280 Acked Write 1 Done Done Sent false
36022 10281 Acked Flush 0 Done Done Sent false
36023 10282 Acked Read 9 Done Done Sent false
36024 10283 Acked Flush 0 Done Done Sent false
36025 10284 Acked Write 9 Done Done Sent false
36026 10285 Acked Write 2 Done Done Sent false
36027 10286 Acked Read 2 Done Done Sent false
36028 10287 Acked Read 4 Done Done Sent false
36029 10288 Acked Flush 0 Done Done Sent false
36030 10289 Acked Write 1 Done Done Sent false
36031 10290 Acked Read 8 Done Done Sent false
36032 10291 Acked Read 10 Done Done Sent false
36033 10292 Acked Read 3 Done Done Sent false
36034 10293 Acked Write 8 Done Done Sent false
36035 10294 Acked Write 10 Done Done Sent false
36036 10295 Acked Read 4 Done Done Sent false
36037 10296 Acked Write 6 Done Done Sent false
36038 10297 Acked Write 4 Done Done Sent false
36039 10298 Acked Read 4 Done Done Sent false
36040 10299 Acked Write 1 Done Done Sent false
36041 10300 Acked Read 8 Done Done Sent false
36042 10301 Acked Read 2 Done Done Sent false
36043 10302 Acked Write 3 Done Done Sent false
36044 10303 Acked Write 3 Done Done Sent false
36045 10304 Acked Read 3 Done Done Sent false
36046 10305 Acked Write 10 Done Done Sent false
36047 10306 Acked Read 10 Done Done Sent false
36048 10307 Acked Read 4 Done Done Sent false
36049 10308 Acked Write 9 Done Done Sent false
36050 10309 Acked Read 10 Done Done Sent false
36051 10310 Acked Read 2 Done Done Sent false
36052 10311 Acked Read 10 Done Done Sent false
36053 10312 Acked Read 3 Done Done Sent false
36054 10313 Acked Read 8 Done Done Sent false
36055 10314 Acked Write 8 Done Done Sent false
36056 10315 Acked Write 6 Done Done Sent false
36057 10316 Acked Read 1 Done Done Sent false
36058 10317 Acked Write 2 Done Done Sent false
36059 10318 Acked Flush 0 Done Done Sent false
36060 10319 Acked Write 8 Done Done Sent false
36061 10320 Acked Write 9 Done Done Sent false
36062 10321 Acked Write 2 Done Done Sent false
36063 10322 Acked Read 8 Done Done Sent false
36064 10323 Acked Write 10 Done Done Sent false
36065 10324 Acked Flush 0 Done Done Sent false
36066 10325 Acked Read 6 Done Done Sent false
36067 10326 Acked Read 8 Done Done Sent false
36068 10327 Acked Read 1 Done Done Sent false
36069 10328 Acked Write 5 Done Done Sent false
36070 10329 Acked Read 3 Done Done Sent false
36071 10330 Acked Read 7 Done Done Sent false
36072 10331 Acked Write 4 Done Done Sent false
36073 10332 Acked Read 2 Done Done Sent false
36074 10333 Acked Write 2 Done Done Sent false
36075 10334 Acked Read 4 Done Done Sent false
36076 10335 Acked Write 6 Done Done Sent false
36077 10336 Acked Read 1 Done Done Sent false
36078 10337 Acked Read 8 Done Done Sent false
36079 10338 Acked Write 10 Done Done Sent false
36080 10339 Acked Read 8 Done Done Sent false
36081 10340 Acked Flush 0 Done Done Sent false
36082 10341 Acked Read 5 Done Done Sent false
36083 10342 Acked Write 8 Done Done Sent false
36084 10343 Acked Flush 0 Done Done Sent false
36085 10344 Acked Write 9 Done Done Sent false
36086 10345 Acked Read 10 Done Done Sent false
36087 10346 Acked Write 7 Done Done Sent false
36088 10347 Acked Flush 0 Done Done Sent false
36089 10348 Acked Read 7 Done Done Sent false
36090 10349 Acked Write 1 Done Done Sent false
36091 10350 Acked Read 4 Done Done Sent false
36092 10351 Acked Write 7 Done Done Sent false
36093 10352 Acked Write 9 Done Done Sent false
36094 10353 Acked Read 6 Done Done Sent false
36095 10354 Acked Read 9 Done Done Sent false
36096 10355 Acked Read 7 Done Done Sent false
36097 10356 Acked Write 8 Done Done Sent false
36098 10357 Acked Write 10 Sent Done Sent false
36099 STATES DS:0 DS:1 DS:2 TOTAL
36100 Sent 1 0 301 302
36101 Done 300 301 0 601
36102 Skipped 0 0 0 0
36103 Error 0 0 0 0
36104 Last Flush: 10347 10347 10056
36105 Downstairs last five completed: 10056 10055 10054 10053 10052
36106 Upstairs last five completed: 10357 10356 10355 10354 10353
36107 CLIENT: Up:0 ds:301 act:2
36108 ----------------------------------------------------------------
36109 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
36110 States: Active Active Offline
36111 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
36112 10057 Acked Read 6 Done Done Sent false
36113 10058 Acked Write 9 Done Done Sent false
36114 10059 Acked Read 1 Done Done Sent false
36115 10060 Acked Read 3 Done Done Sent false
36116 10061 Acked Write 9 Done Done Sent false
36117 10062 Acked Write 6 Done Done Sent false
36118 10063 Acked Write 4 Done Done Sent false
36119 10064 Acked Write 1 Done Done Sent false
36120 10065 Acked Write 9 Done Done Sent false
36121 10066 Acked Write 3 Done Done Sent false
36122 10067 Acked Read 7 Done Done Sent false
36123 10068 Acked Flush 0 Done Done Sent false
36124 10069 Acked Read 3 Done Done Sent false
36125 10070 Acked Read 5 Done Done Sent false
36126 10071 Acked Flush 0 Done Done Sent false
36127 10072 Acked Write 3 Done Done Sent false
36128 10073 Acked Read 10 Done Done Sent false
36129 10074 Acked Flush 0 Done Done Sent false
36130 10075 Acked Flush 0 Done Done Sent false
36131 10076 Acked Flush 0 Done Done Sent false
36132 10077 Acked Write 5 Done Done Sent false
36133 10078 Acked Write 4 Done Done Sent false
36134 10079 Acked Write 2 Done Done Sent false
36135 10080 Acked Write 3 Done Done Sent false
36136 10081 Acked Write 6 Done Done Sent false
36137 10082 Acked Read 8 Done Done Sent false
36138 10083 Acked Read 9 Done Done Sent false
36139 10084 Acked Read 7 Done Done Sent false
36140 10085 Acked Write 1 Done Done Sent false
36141 10086 Acked Read 10 Done Done Sent false
36142 10087 Acked Read 4 Done Done Sent false
36143 10088 Acked Read 8 Done Done Sent false
36144 10089 Acked Write 6 Done Done Sent false
36145 10090 Acked Write 5 Done Done Sent false
36146 10091 Acked Read 5 Done Done Sent false
36147 10092 Acked Write 2 Done Done Sent false
36148 10093 Acked Read 3 Done Done Sent false
36149 10094 Acked Read 8 Done Done Sent false
36150 10095 Acked Write 7 Done Done Sent false
36151 10096 Acked Read 2 Done Done Sent false
36152 10097 Acked Flush 0 Done Done Sent false
36153 10098 Acked Flush 0 Done Done Sent false
36154 10099 Acked Write 6 Done Done Sent false
36155 10100 Acked Read 8 Done Done Sent false
36156 10101 Acked Read 3 Done Done Sent false
36157 10102 Acked Write 8 Done Done Sent false
36158 10103 Acked Read 7 Done Done Sent false
36159 10104 Acked Read 3 Done Done Sent false
36160 10105 Acked Flush 0 Done Done Sent false
36161 10106 Acked Read 7 Done Done Sent false
36162 10107 Acked Flush 0 Done Done Sent false
36163 10108 Acked Read 10 Done Done Sent false
36164 10109 Acked Read 8 Done Done Sent false
36165 10110 Acked Read 4 Done Done Sent false
36166 10111 Acked Flush 0 Done Done Sent false
36167 10112 Acked Read 2 Done Done Sent false
36168 10113 Acked Flush 0 Done Done Sent false
36169 10114 Acked Read 2 Done Done Sent false
36170 10115 Acked Read 7 Done Done Sent false
36171 10116 Acked Write 1 Done Done Sent false
36172 10117 Acked Write 6 Done Done Sent false
36173 10118 Acked Write 10 Done Done Sent false
36174 10119 Acked Write 4 Done Done Sent false
36175 10120 Acked Read 9 Done Done Sent false
36176 10121 Acked Read 10 Done Done Sent false
36177 10122 Acked Write 1 Done Done Sent false
36178 10123 Acked Flush 0 Done Done Sent false
36179 10124 Acked Read 6 Done Done Sent false
36180 10125 Acked Read 10 Done Done Sent false
36181 10126 Acked Read 1 Done Done Sent false
36182 10127 Acked Write 2 Done Done Sent false
36183 10128 Acked Write 8 Done Done Sent false
36184 10129 Acked Read 7 Done Done Sent false
36185 10130 Acked Read 8 Done Done Sent false
36186 10131 Acked Read 2 Done Done Sent false
36187 10132 Acked Read 3 Done Done Sent false
36188 10133 Acked Read 1 Done Done Sent false
36189 10134 Acked Read 8 Done Done Sent false
36190 10135 Acked Write 7 Done Done Sent false
36191 10136 Acked Write 5 Done Done Sent false
36192 10137 Acked Read 6 Done Done Sent false
36193 10138 Acked Read 10 Done Done Sent false
36194 10139 Acked Write 1 Done Done Sent false
36195 10140 Acked Read 1 Done Done Sent false
36196 10141 Acked Write 7 Done Done Sent false
36197 10142 Acked Read 8 Done Done Sent false
36198 10143 Acked Read 10 Done Done Sent false
36199 10144 Acked Read 4 Done Done Sent false
36200 10145 Acked Read 6 Done Done Sent false
36201 10146 Acked Read 2 Done Done Sent false
36202 10147 Acked Read 4 Done Done Sent false
36203 10148 Acked Read 7 Done Done Sent false
36204 10149 Acked Read 8 Done Done Sent false
36205 10150 Acked Write 5 Done Done Sent false
36206 10151 Acked Write 10 Done Done Sent false
36207 10152 Acked Read 10 Done Done Sent false
36208 10153 Acked Flush 0 Done Done Sent false
36209 10154 Acked Read 5 Done Done Sent false
36210 10155 Acked Read 5 Done Done Sent false
36211 10156 Acked Read 4 Done Done Sent false
36212 10157 Acked Read 10 Done Done Sent false
36213 10158 Acked Write 10 Done Done Sent false
36214 10159 Acked Read 9 Done Done Sent false
36215 10160 Acked Read 1 Done Done Sent false
36216 10161 Acked Read 5 Done Done Sent false
36217 10162 Acked Write 1 Done Done Sent false
36218 10163 Acked Flush 0 Done Done Sent false
36219 10164 Acked Read 6 Done Done Sent false
36220 10165 Acked Write 5 Done Done Sent false
36221 10166 Acked Write 6 Done Done Sent false
36222 10167 Acked Write 5 Done Done Sent false
36223 10168 Acked Read 7 Done Done Sent false
36224 10169 Acked Read 4 Done Done Sent false
36225 10170 Acked Read 5 Done Done Sent false
36226 10171 Acked Flush 0 Done Done Sent false
36227 10172 Acked Write 1 Done Done Sent false
36228 10173 Acked Read 6 Done Done Sent false
36229 10174 Acked Read 2 Done Done Sent false
36230 10175 Acked Flush 0 Done Done Sent false
36231 10176 Acked Read 7 Done Done Sent false
36232 10177 Acked Read 10 Done Done Sent false
36233 10178 Acked Read 8 Done Done Sent false
36234 10179 Acked Write 4 Done Done Sent false
36235 10180 Acked Read 2 Done Done Sent false
36236 10181 Acked Read 2 Done Done Sent false
36237 10182 Acked Read 8 Done Done Sent false
36238 10183 Acked Read 10 Done Done Sent false
36239 10184 Acked Read 2 Done Done Sent false
36240 10185 Acked Write 6 Done Done Sent false
36241 10186 Acked Read 1 Done Done Sent false
36242 10187 Acked Read 3 Done Done Sent false
36243 10188 Acked Read 4 Done Done Sent false
36244 10189 Acked Read 10 Done Done Sent false
36245 10190 Acked Read 1 Done Done Sent false
36246 10191 Acked Read 6 Done Done Sent false
36247 10192 Acked Flush 0 Done Done Sent false
36248 10193 Acked Write 9 Done Done Sent false
36249 10194 Acked Read 5 Done Done Sent false
36250 10195 Acked Write 10 Done Done Sent false
36251 10196 Acked Write 1 Done Done Sent false
36252 10197 Acked Flush 0 Done Done Sent false
36253 10198 Acked Write 4 Done Done Sent false
36254 10199 Acked Read 10 Done Done Sent false
36255 10200 Acked Read 6 Done Done Sent false
36256 10201 Acked Read 3 Done Done Sent false
36257 10202 Acked Read 8 Done Done Sent false
36258 10203 Acked Read 6 Done Done Sent false
36259 10204 Acked Read 7 Done Done Sent false
36260 10205 Acked Write 7 Done Done Sent false
36261 10206 Acked Flush 0 Done Done Sent false
36262 10207 Acked Read 6 Done Done Sent false
36263 10208 Acked Read 10 Done Done Sent false
36264 10209 Acked Read 9 Done Done Sent false
36265 10210 Acked Read 4 Done Done Sent false
36266 10211 Acked Write 3 Done Done Sent false
36267 10212 Acked Write 5 Done Done Sent false
36268 10213 Acked Read 1 Done Done Sent false
36269 10214 Acked Write 3 Done Done Sent false
36270 10215 Acked Write 8 Done Done Sent false
36271 10216 Acked Write 6 Done Done Sent false
36272 10217 Acked Read 5 Done Done Sent false
36273 10218 Acked Read 6 Done Done Sent false
36274 10219 Acked Read 2 Done Done Sent false
36275 10220 Acked Read 8 Done Done Sent false
36276 10221 Acked Read 3 Done Done Sent false
36277 10222 Acked Read 2 Done Done Sent false
36278 10223 Acked Write 3 Done Done Sent false
36279 10224 Acked Write 10 Done Done Sent false
36280 10225 Acked Read 3 Done Done Sent false
36281 10226 Acked Write 1 Done Done Sent false
36282 10227 Acked Write 3 Done Done Sent false
36283 10228 Acked Write 1 Done Done Sent false
36284 10229 Acked Write 3 Done Done Sent false
36285 10230 Acked Write 2 Done Done Sent false
36286 10231 Acked Read 10 Done Done Sent false
36287 10232 Acked Read 9 Done Done Sent false
36288 10233 Acked Flush 0 Done Done Sent false
36289 10234 Acked Read 9 Done Done Sent false
36290 10235 Acked Write 9 Done Done Sent false
36291 10236 Acked Write 3 Done Done Sent false
36292 10237 Acked Write 9 Done Done Sent false
36293 10238 Acked Read 6 Done Done Sent false
36294 10239 Acked Write 8 Done Done Sent false
36295 10240 Acked Read 8 Done Done Sent false
36296 10241 Acked Write 2 Done Done Sent false
36297 10242 Acked Flush 0 Done Done Sent false
36298 10243 Acked Flush 0 Done Done Sent false
36299 10244 Acked Read 3 Done Done Sent false
36300 10245 Acked Write 5 Done Done Sent false
36301 10246 Acked Write 7 Done Done Sent false
36302 10247 Acked Read 10 Done Done Sent false
36303 10248 Acked Write 1 Done Done Sent false
36304 10249 Acked Read 6 Done Done Sent false
36305 10250 Acked Read 6 Done Done Sent false
36306 10251 Acked Read 8 Done Done Sent false
36307 10252 Acked Read 9 Done Done Sent false
36308 10253 Acked Read 10 Done Done Sent false
36309 10254 Acked Write 7 Done Done Sent false
36310 10255 Acked Write 2 Done Done Sent false
36311 10256 Acked Write 8 Done Done Sent false
36312 10257 Acked Write 1 Done Done Sent false
36313 10258 Acked Read 6 Done Done Sent false
36314 10259 Acked Read 8 Done Done Sent false
36315 10260 Acked Write 2 Done Done Sent false
36316 10261 Acked Write 3 Done Done Sent false
36317 10262 Acked Write 7 Done Done Sent false
36318 10263 Acked Write 1 Done Done Sent false
36319 10264 Acked Flush 0 Done Done Sent false
36320 10265 Acked Write 7 Done Done Sent false
36321 10266 Acked Flush 0 Done Done Sent false
36322 10267 Acked Write 2 Done Done Sent false
36323 10268 Acked Read 4 Done Done Sent false
36324 10269 Acked Write 7 Done Done Sent false
36325 10270 Acked Read 10 Done Done Sent false
36326 10271 Acked Read 3 Done Done Sent false
36327 10272 Acked Write 5 Done Done Sent false
36328 10273 Acked Write 8 Done Done Sent false
36329 10274 Acked Write 7 Done Done Sent false
36330 10275 Acked Read 9 Done Done Sent false
36331 10276 Acked Read 1 Done Done Sent false
36332 10277 Acked Read 1 Done Done Sent false
36333 10278 Acked Write 1 Done Done Sent false
36334 10279 Acked Read 6 Done Done Sent false
36335 10280 Acked Write 1 Done Done Sent false
36336 10281 Acked Flush 0 Done Done Sent false
36337 10282 Acked Read 9 Done Done Sent false
36338 10283 Acked Flush 0 Done Done Sent false
36339 10284 Acked Write 9 Done Done Sent false
36340 10285 Acked Write 2 Done Done Sent false
36341 10286 Acked Read 2 Done Done Sent false
36342 10287 Acked Read 4 Done Done Sent false
36343 10288 Acked Flush 0 Done Done Sent false
36344 10289 Acked Write 1 Done Done Sent false
36345 10290 Acked Read 8 Done Done Sent false
36346 10291 Acked Read 10 Done Done Sent false
36347 10292 Acked Read 3 Done Done Sent false
36348 10293 Acked Write 8 Done Done Sent false
36349 10294 Acked Write 10 Done Done Sent false
36350 10295 Acked Read 4 Done Done Sent false
36351 10296 Acked Write 6 Done Done Sent false
36352 10297 Acked Write 4 Done Done Sent false
36353 10298 Acked Read 4 Done Done Sent false
36354 10299 Acked Write 1 Done Done Sent false
36355 10300 Acked Read 8 Done Done Sent false
36356 10301 Acked Read 2 Done Done Sent false
36357 10302 Acked Write 3 Done Done Sent false
36358 10303 Acked Write 3 Done Done Sent false
36359 10304 Acked Read 3 Done Done Sent false
36360 10305 Acked Write 10 Done Done Sent false
36361 10306 Acked Read 10 Done Done Sent false
36362 10307 Acked Read 4 Done Done Sent false
36363 10308 Acked Write 9 Done Done Sent false
36364 10309 Acked Read 10 Done Done Sent false
36365 10310 Acked Read 2 Done Done Sent false
36366 10311 Acked Read 10 Done Done Sent false
36367 10312 Acked Read 3 Done Done Sent false
36368 10313 Acked Read 8 Done Done Sent false
36369 10314 Acked Write 8 Done Done Sent false
36370 10315 Acked Write 6 Done Done Sent false
36371 10316 Acked Read 1 Done Done Sent false
36372 10317 Acked Write 2 Done Done Sent false
36373 10318 Acked Flush 0 Done Done Sent false
36374 10319 Acked Write 8 Done Done Sent false
36375 10320 Acked Write 9 Done Done Sent false
36376 10321 Acked Write 2 Done Done Sent false
36377 10322 Acked Read 8 Done Done Sent false
36378 10323 Acked Write 10 Done Done Sent false
36379 10324 Acked Flush 0 Done Done Sent false
36380 10325 Acked Read 6 Done Done Sent false
36381 10326 Acked Read 8 Done Done Sent false
36382 10327 Acked Read 1 Done Done Sent false
36383 10328 Acked Write 5 Done Done Sent false
36384 10329 Acked Read 3 Done Done Sent false
36385 10330 Acked Read 7 Done Done Sent false
36386 10331 Acked Write 4 Done Done Sent false
36387 10332 Acked Read 2 Done Done Sent false
36388 10333 Acked Write 2 Done Done Sent false
36389 10334 Acked Read 4 Done Done Sent false
36390 10335 Acked Write 6 Done Done Sent false
36391 10336 Acked Read 1 Done Done Sent false
36392 10337 Acked Read 8 Done Done Sent false
36393 10338 Acked Write 10 Done Done Sent false
36394 10339 Acked Read 8 Done Done Sent false
36395 10340 Acked Flush 0 Done Done Sent false
36396 10341 Acked Read 5 Done Done Sent false
36397 10342 Acked Write 8 Done Done Sent false
36398 10343 Acked Flush 0 Done Done Sent false
36399 10344 Acked Write 9 Done Done Sent false
36400 10345 Acked Read 10 Done Done Sent false
36401 10346 Acked Write 7 Done Done Sent false
36402 10347 Acked Flush 0 Done Done Sent false
36403 10348 Acked Read 7 Done Done Sent false
36404 10349 Acked Write 1 Done Done Sent false
36405 10350 Acked Read 4 Done Done Sent false
36406 10351 Acked Write 7 Done Done Sent false
36407 10352 Acked Write 9 Done Done Sent false
36408 10353 Acked Read 6 Done Done Sent false
36409 10354 Acked Read 9 Done Done Sent false
36410 10355 Acked Read 7 Done Done Sent false
36411 10356 Acked Write 8 Done Done Sent false
36412 10357 Acked Write 10 Done Done Sent false
36413 10358 Acked Flush 0 Done Done Sent false
36414 STATES DS:0 DS:1 DS:2 TOTAL
36415 Sent 0 0 302 302
36416 Done 302 302 0 604
36417 Skipped 0 0 0 0
36418 Error 0 0 0 0
36419 Last Flush: 10358 10358 10056
36420 Downstairs last five completed: 10056 10055 10054 10053 10052
36421 Upstairs last five completed: 10358 10357 10356 10355 10354
36422 CLIENT: Up:0 ds:302 act:2
364232024-11-26T22:12:09.762ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
364242024-11-26T22:12:09.762ZWARNcrucible: ds_connection connect to 127.0.0.1:8830 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
364252024-11-26T22:12:09.762ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
364262024-11-26T22:12:09.762ZWARNcrucible: downstairs task for 2 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
364272024-11-26T22:12:09.762ZINFOcrucible: client task is exiting = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
36428 [8830] got start action
36429 [8830] I am going to restart
36430 Starting downstairs at port 8830
36431 Make output file at "/tmp/dsc/downstairs-8830.txt"
36432 [8830][2] reports Starting
36433 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1234)
36434 [8830][2] reports Running
36435 ----------------------------------------------------------------
36436 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
36437 States: Active Active Offline
36438 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
36439 10057 Acked Read 6 Done Done Sent false
36440 10058 Acked Write 9 Done Done Sent false
36441 10059 Acked Read 1 Done Done Sent false
36442 10060 Acked Read 3 Done Done Sent false
36443 10061 Acked Write 9 Done Done Sent false
36444 10062 Acked Write 6 Done Done Sent false
36445 10063 Acked Write 4 Done Done Sent false
36446 10064 Acked Write 1 Done Done Sent false
36447 10065 Acked Write 9 Done Done Sent false
36448 10066 Acked Write 3 Done Done Sent false
36449 10067 Acked Read 7 Done Done Sent false
36450 10068 Acked Flush 0 Done Done Sent false
36451 10069 Acked Read 3 Done Done Sent false
36452 10070 Acked Read 5 Done Done Sent false
36453 10071 Acked Flush 0 Done Done Sent false
36454 10072 Acked Write 3 Done Done Sent false
36455 10073 Acked Read 10 Done Done Sent false
36456 10074 Acked Flush 0 Done Done Sent false
36457 10075 Acked Flush 0 Done Done Sent false
36458 10076 Acked Flush 0 Done Done Sent false
36459 10077 Acked Write 5 Done Done Sent false
36460 10078 Acked Write 4 Done Done Sent false
36461 10079 Acked Write 2 Done Done Sent false
36462 10080 Acked Write 3 Done Done Sent false
36463 10081 Acked Write 6 Done Done Sent false
36464 10082 Acked Read 8 Done Done Sent false
36465 10083 Acked Read 9 Done Done Sent false
36466 10084 Acked Read 7 Done Done Sent false
36467 10085 Acked Write 1 Done Done Sent false
36468 10086 Acked Read 10 Done Done Sent false
36469 10087 Acked Read 4 Done Done Sent false
36470 10088 Acked Read 8 Done Done Sent false
36471 10089 Acked Write 6 Done Done Sent false
36472 10090 Acked Write 5 Done Done Sent false
36473 10091 Acked Read 5 Done Done Sent false
36474 10092 Acked Write 2 Done Done Sent false
36475 10093 Acked Read 3 Done Done Sent false
36476 10094 Acked Read 8 Done Done Sent false
36477 10095 Acked Write 7 Done Done Sent false
36478 10096 Acked Read 2 Done Done Sent false
36479 10097 Acked Flush 0 Done Done Sent false
36480 10098 Acked Flush 0 Done Done Sent false
36481 10099 Acked Write 6 Done Done Sent false
36482 10100 Acked Read 8 Done Done Sent false
36483 10101 Acked Read 3 Done Done Sent false
36484 10102 Acked Write 8 Done Done Sent false
36485 10103 Acked Read 7 Done Done Sent false
36486 10104 Acked Read 3 Done Done Sent false
36487 10105 Acked Flush 0 Done Done Sent false
36488 10106 Acked Read 7 Done Done Sent false
36489 10107 Acked Flush 0 Done Done Sent false
36490 10108 Acked Read 10 Done Done Sent false
36491 10109 Acked Read 8 Done Done Sent false
36492 10110 Acked Read 4 Done Done Sent false
36493 10111 Acked Flush 0 Done Done Sent false
36494 10112 Acked Read 2 Done Done Sent false
36495 10113 Acked Flush 0 Done Done Sent false
36496 10114 Acked Read 2 Done Done Sent false
36497 10115 Acked Read 7 Done Done Sent false
36498 10116 Acked Write 1 Done Done Sent false
36499 10117 Acked Write 6 Done Done Sent false
36500 10118 Acked Write 10 Done Done Sent false
36501 10119 Acked Write 4 Done Done Sent false
36502 10120 Acked Read 9 Done Done Sent false
36503 10121 Acked Read 10 Done Done Sent false
36504 10122 Acked Write 1 Done Done Sent false
36505 10123 Acked Flush 0 Done Done Sent false
36506 10124 Acked Read 6 Done Done Sent false
36507 10125 Acked Read 10 Done Done Sent false
36508 10126 Acked Read 1 Done Done Sent false
36509 10127 Acked Write 2 Done Done Sent false
36510 10128 Acked Write 8 Done Done Sent false
36511 10129 Acked Read 7 Done Done Sent false
36512 10130 Acked Read 8 Done Done Sent false
36513 10131 Acked Read 2 Done Done Sent false
36514 10132 Acked Read 3 Done Done Sent false
36515 10133 Acked Read 1 Done Done Sent false
36516 10134 Acked Read 8 Done Done Sent false
36517 10135 Acked Write 7 Done Done Sent false
36518 10136 Acked Write 5 Done Done Sent false
36519 10137 Acked Read 6 Done Done Sent false
36520 10138 Acked Read 10 Done Done Sent false
36521 10139 Acked Write 1 Done Done Sent false
36522 10140 Acked Read 1 Done Done Sent false
36523 10141 Acked Write 7 Done Done Sent false
36524 10142 Acked Read 8 Done Done Sent false
36525 10143 Acked Read 10 Done Done Sent false
36526 10144 Acked Read 4 Done Done Sent false
36527 10145 Acked Read 6 Done Done Sent false
36528 10146 Acked Read 2 Done Done Sent false
36529 10147 Acked Read 4 Done Done Sent false
36530 10148 Acked Read 7 Done Done Sent false
36531 10149 Acked Read 8 Done Done Sent false
36532 10150 Acked Write 5 Done Done Sent false
36533 10151 Acked Write 10 Done Done Sent false
36534 10152 Acked Read 10 Done Done Sent false
36535 10153 Acked Flush 0 Done Done Sent false
36536 10154 Acked Read 5 Done Done Sent false
36537 10155 Acked Read 5 Done Done Sent false
36538 10156 Acked Read 4 Done Done Sent false
36539 10157 Acked Read 10 Done Done Sent false
36540 10158 Acked Write 10 Done Done Sent false
36541 10159 Acked Read 9 Done Done Sent false
36542 10160 Acked Read 1 Done Done Sent false
36543 10161 Acked Read 5 Done Done Sent false
36544 10162 Acked Write 1 Done Done Sent false
36545 10163 Acked Flush 0 Done Done Sent false
36546 10164 Acked Read 6 Done Done Sent false
36547 10165 Acked Write 5 Done Done Sent false
36548 10166 Acked Write 6 Done Done Sent false
36549 10167 Acked Write 5 Done Done Sent false
36550 10168 Acked Read 7 Done Done Sent false
36551 10169 Acked Read 4 Done Done Sent false
36552 10170 Acked Read 5 Done Done Sent false
36553 10171 Acked Flush 0 Done Done Sent false
36554 10172 Acked Write 1 Done Done Sent false
36555 10173 Acked Read 6 Done Done Sent false
36556 10174 Acked Read 2 Done Done Sent false
36557 10175 Acked Flush 0 Done Done Sent false
36558 10176 Acked Read 7 Done Done Sent false
36559 10177 Acked Read 10 Done Done Sent false
36560 10178 Acked Read 8 Done Done Sent false
36561 10179 Acked Write 4 Done Done Sent false
36562 10180 Acked Read 2 Done Done Sent false
36563 10181 Acked Read 2 Done Done Sent false
36564 10182 Acked Read 8 Done Done Sent false
36565 10183 Acked Read 10 Done Done Sent false
36566 10184 Acked Read 2 Done Done Sent false
36567 10185 Acked Write 6 Done Done Sent false
36568 10186 Acked Read 1 Done Done Sent false
36569 10187 Acked Read 3 Done Done Sent false
36570 10188 Acked Read 4 Done Done Sent false
36571 10189 Acked Read 10 Done Done Sent false
36572 10190 Acked Read 1 Done Done Sent false
36573 10191 Acked Read 6 Done Done Sent false
36574 10192 Acked Flush 0 Done Done Sent false
36575 10193 Acked Write 9 Done Done Sent false
36576 10194 Acked Read 5 Done Done Sent false
36577 10195 Acked Write 10 Done Done Sent false
36578 10196 Acked Write 1 Done Done Sent false
36579 10197 Acked Flush 0 Done Done Sent false
36580 10198 Acked Write 4 Done Done Sent false
36581 10199 Acked Read 10 Done Done Sent false
36582 10200 Acked Read 6 Done Done Sent false
36583 10201 Acked Read 3 Done Done Sent false
36584 10202 Acked Read 8 Done Done Sent false
36585 10203 Acked Read 6 Done Done Sent false
36586 10204 Acked Read 7 Done Done Sent false
36587 10205 Acked Write 7 Done Done Sent false
36588 10206 Acked Flush 0 Done Done Sent false
36589 10207 Acked Read 6 Done Done Sent false
36590 10208 Acked Read 10 Done Done Sent false
36591 10209 Acked Read 9 Done Done Sent false
36592 10210 Acked Read 4 Done Done Sent false
36593 10211 Acked Write 3 Done Done Sent false
36594 10212 Acked Write 5 Done Done Sent false
36595 10213 Acked Read 1 Done Done Sent false
36596 10214 Acked Write 3 Done Done Sent false
36597 10215 Acked Write 8 Done Done Sent false
36598 10216 Acked Write 6 Done Done Sent false
36599 10217 Acked Read 5 Done Done Sent false
36600 10218 Acked Read 6 Done Done Sent false
36601 10219 Acked Read 2 Done Done Sent false
36602 10220 Acked Read 8 Done Done Sent false
36603 10221 Acked Read 3 Done Done Sent false
36604 10222 Acked Read 2 Done Done Sent false
36605 10223 Acked Write 3 Done Done Sent false
36606 10224 Acked Write 10 Done Done Sent false
36607 10225 Acked Read 3 Done Done Sent false
36608 10226 Acked Write 1 Done Done Sent false
36609 10227 Acked Write 3 Done Done Sent false
36610 10228 Acked Write 1 Done Done Sent false
36611 10229 Acked Write 3 Done Done Sent false
36612 10230 Acked Write 2 Done Done Sent false
36613 10231 Acked Read 10 Done Done Sent false
36614 10232 Acked Read 9 Done Done Sent false
36615 10233 Acked Flush 0 Done Done Sent false
36616 10234 Acked Read 9 Done Done Sent false
36617 10235 Acked Write 9 Done Done Sent false
36618 10236 Acked Write 3 Done Done Sent false
36619 10237 Acked Write 9 Done Done Sent false
36620 10238 Acked Read 6 Done Done Sent false
36621 10239 Acked Write 8 Done Done Sent false
36622 10240 Acked Read 8 Done Done Sent false
36623 10241 Acked Write 2 Done Done Sent false
36624 10242 Acked Flush 0 Done Done Sent false
36625 10243 Acked Flush 0 Done Done Sent false
36626 10244 Acked Read 3 Done Done Sent false
36627 10245 Acked Write 5 Done Done Sent false
36628 10246 Acked Write 7 Done Done Sent false
36629 10247 Acked Read 10 Done Done Sent false
36630 10248 Acked Write 1 Done Done Sent false
36631 10249 Acked Read 6 Done Done Sent false
36632 10250 Acked Read 6 Done Done Sent false
36633 10251 Acked Read 8 Done Done Sent false
36634 10252 Acked Read 9 Done Done Sent false
36635 10253 Acked Read 10 Done Done Sent false
36636 10254 Acked Write 7 Done Done Sent false
36637 10255 Acked Write 2 Done Done Sent false
36638 10256 Acked Write 8 Done Done Sent false
36639 10257 Acked Write 1 Done Done Sent false
36640 10258 Acked Read 6 Done Done Sent false
36641 10259 Acked Read 8 Done Done Sent false
36642 10260 Acked Write 2 Done Done Sent false
36643 10261 Acked Write 3 Done Done Sent false
36644 10262 Acked Write 7 Done Done Sent false
36645 10263 Acked Write 1 Done Done Sent false
36646 10264 Acked Flush 0 Done Done Sent false
36647 10265 Acked Write 7 Done Done Sent false
36648 10266 Acked Flush 0 Done Done Sent false
36649 10267 Acked Write 2 Done Done Sent false
36650 10268 Acked Read 4 Done Done Sent false
36651 10269 Acked Write 7 Done Done Sent false
36652 10270 Acked Read 10 Done Done Sent false
36653 10271 Acked Read 3 Done Done Sent false
36654 10272 Acked Write 5 Done Done Sent false
36655 10273 Acked Write 8 Done Done Sent false
36656 10274 Acked Write 7 Done Done Sent false
36657 10275 Acked Read 9 Done Done Sent false
36658 10276 Acked Read 1 Done Done Sent false
36659 10277 Acked Read 1 Done Done Sent false
36660 10278 Acked Write 1 Done Done Sent false
36661 10279 Acked Read 6 Done Done Sent false
36662 10280 Acked Write 1 Done Done Sent false
36663 10281 Acked Flush 0 Done Done Sent false
36664 10282 Acked Read 9 Done Done Sent false
36665 10283 Acked Flush 0 Done Done Sent false
36666 10284 Acked Write 9 Done Done Sent false
36667 10285 Acked Write 2 Done Done Sent false
36668 10286 Acked Read 2 Done Done Sent false
36669 10287 Acked Read 4 Done Done Sent false
36670 10288 Acked Flush 0 Done Done Sent false
36671 10289 Acked Write 1 Done Done Sent false
36672 10290 Acked Read 8 Done Done Sent false
36673 10291 Acked Read 10 Done Done Sent false
36674 10292 Acked Read 3 Done Done Sent false
36675 10293 Acked Write 8 Done Done Sent false
36676 10294 Acked Write 10 Done Done Sent false
36677 10295 Acked Read 4 Done Done Sent false
36678 10296 Acked Write 6 Done Done Sent false
36679 10297 Acked Write 4 Done Done Sent false
36680 10298 Acked Read 4 Done Done Sent false
36681 10299 Acked Write 1 Done Done Sent false
36682 10300 Acked Read 8 Done Done Sent false
36683 10301 Acked Read 2 Done Done Sent false
36684 10302 Acked Write 3 Done Done Sent false
36685 10303 Acked Write 3 Done Done Sent false
36686 10304 Acked Read 3 Done Done Sent false
36687 10305 Acked Write 10 Done Done Sent false
36688 10306 Acked Read 10 Done Done Sent false
36689 10307 Acked Read 4 Done Done Sent false
36690 10308 Acked Write 9 Done Done Sent false
36691 10309 Acked Read 10 Done Done Sent false
36692 10310 Acked Read 2 Done Done Sent false
36693 10311 Acked Read 10 Done Done Sent false
36694 10312 Acked Read 3 Done Done Sent false
36695 10313 Acked Read 8 Done Done Sent false
36696 10314 Acked Write 8 Done Done Sent false
36697 10315 Acked Write 6 Done Done Sent false
36698 10316 Acked Read 1 Done Done Sent false
36699 10317 Acked Write 2 Done Done Sent false
36700 10318 Acked Flush 0 Done Done Sent false
36701 10319 Acked Write 8 Done Done Sent false
36702 10320 Acked Write 9 Done Done Sent false
36703 10321 Acked Write 2 Done Done Sent false
36704 10322 Acked Read 8 Done Done Sent false
36705 10323 Acked Write 10 Done Done Sent false
36706 10324 Acked Flush 0 Done Done Sent false
36707 10325 Acked Read 6 Done Done Sent false
36708 10326 Acked Read 8 Done Done Sent false
36709 10327 Acked Read 1 Done Done Sent false
36710 10328 Acked Write 5 Done Done Sent false
36711 10329 Acked Read 3 Done Done Sent false
36712 10330 Acked Read 7 Done Done Sent false
36713 10331 Acked Write 4 Done Done Sent false
36714 10332 Acked Read 2 Done Done Sent false
36715 10333 Acked Write 2 Done Done Sent false
36716 10334 Acked Read 4 Done Done Sent false
36717 10335 Acked Write 6 Done Done Sent false
36718 10336 Acked Read 1 Done Done Sent false
36719 10337 Acked Read 8 Done Done Sent false
36720 10338 Acked Write 10 Done Done Sent false
36721 10339 Acked Read 8 Done Done Sent false
36722 10340 Acked Flush 0 Done Done Sent false
36723 10341 Acked Read 5 Done Done Sent false
36724 10342 Acked Write 8 Done Done Sent false
36725 10343 Acked Flush 0 Done Done Sent false
36726 10344 Acked Write 9 Done Done Sent false
36727 10345 Acked Read 10 Done Done Sent false
36728 10346 Acked Write 7 Done Done Sent false
36729 10347 Acked Flush 0 Done Done Sent false
36730 10348 Acked Read 7 Done Done Sent false
36731 10349 Acked Write 1 Done Done Sent false
36732 10350 Acked Read 4 Done Done Sent false
36733 10351 Acked Write 7 Done Done Sent false
36734 10352 Acked Write 9 Done Done Sent false
36735 10353 Acked Read 6 Done Done Sent false
36736 10354 Acked Read 9 Done Done Sent false
36737 10355 Acked Read 7 Done Done Sent false
36738 10356 Acked Write 8 Done Done Sent false
36739 10357 Acked Write 10 Done Done Sent false
36740 10358 Acked Flush 0 Done Done Sent false
36741 STATES DS:0 DS:1 DS:2 TOTAL
36742 Sent 0 0 302 302
36743 Done 302 302 0 604
36744 Skipped 0 0 0 0
36745 Error 0 0 0 0
36746 Last Flush: 10358 10358 10056
36747 Downstairs last five completed: 10056 10055 10054 10053 10052
36748 Upstairs last five completed: 10358 10357 10356 10355 10354
36749 CLIENT: Up:0 ds:302 act:2
36750 ----------------------------------------------------------------
36751 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
36752 States: Active Active Offline
36753 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
36754 10057 Acked Read 6 Done Done Sent false
36755 10058 Acked Write 9 Done Done Sent false
36756 10059 Acked Read 1 Done Done Sent false
36757 10060 Acked Read 3 Done Done Sent false
36758 10061 Acked Write 9 Done Done Sent false
36759 10062 Acked Write 6 Done Done Sent false
36760 10063 Acked Write 4 Done Done Sent false
36761 10064 Acked Write 1 Done Done Sent false
36762 10065 Acked Write 9 Done Done Sent false
36763 10066 Acked Write 3 Done Done Sent false
36764 10067 Acked Read 7 Done Done Sent false
36765 10068 Acked Flush 0 Done Done Sent false
36766 10069 Acked Read 3 Done Done Sent false
36767 10070 Acked Read 5 Done Done Sent false
36768 10071 Acked Flush 0 Done Done Sent false
36769 10072 Acked Write 3 Done Done Sent false
36770 10073 Acked Read 10 Done Done Sent false
36771 10074 Acked Flush 0 Done Done Sent false
36772 10075 Acked Flush 0 Done Done Sent false
36773 10076 Acked Flush 0 Done Done Sent false
36774 10077 Acked Write 5 Done Done Sent false
36775 10078 Acked Write 4 Done Done Sent false
36776 10079 Acked Write 2 Done Done Sent false
36777 10080 Acked Write 3 Done Done Sent false
36778 10081 Acked Write 6 Done Done Sent false
36779 10082 Acked Read 8 Done Done Sent false
36780 10083 Acked Read 9 Done Done Sent false
36781 10084 Acked Read 7 Done Done Sent false
36782 10085 Acked Write 1 Done Done Sent false
36783 10086 Acked Read 10 Done Done Sent false
36784 10087 Acked Read 4 Done Done Sent false
36785 10088 Acked Read 8 Done Done Sent false
36786 10089 Acked Write 6 Done Done Sent false
36787 10090 Acked Write 5 Done Done Sent false
36788 10091 Acked Read 5 Done Done Sent false
36789 10092 Acked Write 2 Done Done Sent false
36790 10093 Acked Read 3 Done Done Sent false
36791 10094 Acked Read 8 Done Done Sent false
36792 10095 Acked Write 7 Done Done Sent false
36793 10096 Acked Read 2 Done Done Sent false
36794 10097 Acked Flush 0 Done Done Sent false
36795 10098 Acked Flush 0 Done Done Sent false
36796 10099 Acked Write 6 Done Done Sent false
36797 10100 Acked Read 8 Done Done Sent false
36798 10101 Acked Read 3 Done Done Sent false
36799 10102 Acked Write 8 Done Done Sent false
36800 10103 Acked Read 7 Done Done Sent false
36801 10104 Acked Read 3 Done Done Sent false
36802 10105 Acked Flush 0 Done Done Sent false
36803 10106 Acked Read 7 Done Done Sent false
36804 10107 Acked Flush 0 Done Done Sent false
36805 10108 Acked Read 10 Done Done Sent false
36806 10109 Acked Read 8 Done Done Sent false
36807 10110 Acked Read 4 Done Done Sent false
36808 10111 Acked Flush 0 Done Done Sent false
36809 10112 Acked Read 2 Done Done Sent false
36810 10113 Acked Flush 0 Done Done Sent false
36811 10114 Acked Read 2 Done Done Sent false
36812 10115 Acked Read 7 Done Done Sent false
36813 10116 Acked Write 1 Done Done Sent false
36814 10117 Acked Write 6 Done Done Sent false
36815 10118 Acked Write 10 Done Done Sent false
36816 10119 Acked Write 4 Done Done Sent false
36817 10120 Acked Read 9 Done Done Sent false
36818 10121 Acked Read 10 Done Done Sent false
36819 10122 Acked Write 1 Done Done Sent false
36820 10123 Acked Flush 0 Done Done Sent false
36821 10124 Acked Read 6 Done Done Sent false
36822 10125 Acked Read 10 Done Done Sent false
36823 10126 Acked Read 1 Done Done Sent false
36824 10127 Acked Write 2 Done Done Sent false
36825 10128 Acked Write 8 Done Done Sent false
36826 10129 Acked Read 7 Done Done Sent false
36827 10130 Acked Read 8 Done Done Sent false
36828 10131 Acked Read 2 Done Done Sent false
36829 10132 Acked Read 3 Done Done Sent false
36830 10133 Acked Read 1 Done Done Sent false
36831 10134 Acked Read 8 Done Done Sent false
36832 10135 Acked Write 7 Done Done Sent false
36833 10136 Acked Write 5 Done Done Sent false
36834 10137 Acked Read 6 Done Done Sent false
36835 10138 Acked Read 10 Done Done Sent false
36836 10139 Acked Write 1 Done Done Sent false
36837 10140 Acked Read 1 Done Done Sent false
36838 10141 Acked Write 7 Done Done Sent false
36839 10142 Acked Read 8 Done Done Sent false
36840 10143 Acked Read 10 Done Done Sent false
36841 10144 Acked Read 4 Done Done Sent false
36842 10145 Acked Read 6 Done Done Sent false
36843 10146 Acked Read 2 Done Done Sent false
36844 10147 Acked Read 4 Done Done Sent false
36845 10148 Acked Read 7 Done Done Sent false
36846 10149 Acked Read 8 Done Done Sent false
36847 10150 Acked Write 5 Done Done Sent false
36848 10151 Acked Write 10 Done Done Sent false
36849 10152 Acked Read 10 Done Done Sent false
36850 10153 Acked Flush 0 Done Done Sent false
36851 10154 Acked Read 5 Done Done Sent false
36852 10155 Acked Read 5 Done Done Sent false
36853 10156 Acked Read 4 Done Done Sent false
36854 10157 Acked Read 10 Done Done Sent false
36855 10158 Acked Write 10 Done Done Sent false
36856 10159 Acked Read 9 Done Done Sent false
36857 10160 Acked Read 1 Done Done Sent false
36858 10161 Acked Read 5 Done Done Sent false
36859 10162 Acked Write 1 Done Done Sent false
36860 10163 Acked Flush 0 Done Done Sent false
36861 10164 Acked Read 6 Done Done Sent false
36862 10165 Acked Write 5 Done Done Sent false
36863 10166 Acked Write 6 Done Done Sent false
36864 10167 Acked Write 5 Done Done Sent false
36865 10168 Acked Read 7 Done Done Sent false
36866 10169 Acked Read 4 Done Done Sent false
36867 10170 Acked Read 5 Done Done Sent false
36868 10171 Acked Flush 0 Done Done Sent false
36869 10172 Acked Write 1 Done Done Sent false
36870 10173 Acked Read 6 Done Done Sent false
36871 10174 Acked Read 2 Done Done Sent false
36872 10175 Acked Flush 0 Done Done Sent false
36873 10176 Acked Read 7 Done Done Sent false
36874 10177 Acked Read 10 Done Done Sent false
36875 10178 Acked Read 8 Done Done Sent false
36876 10179 Acked Write 4 Done Done Sent false
36877 10180 Acked Read 2 Done Done Sent false
36878 10181 Acked Read 2 Done Done Sent false
36879 10182 Acked Read 8 Done Done Sent false
36880 10183 Acked Read 10 Done Done Sent false
36881 10184 Acked Read 2 Done Done Sent false
36882 10185 Acked Write 6 Done Done Sent false
36883 10186 Acked Read 1 Done Done Sent false
36884 10187 Acked Read 3 Done Done Sent false
36885 10188 Acked Read 4 Done Done Sent false
36886 10189 Acked Read 10 Done Done Sent false
36887 10190 Acked Read 1 Done Done Sent false
36888 10191 Acked Read 6 Done Done Sent false
36889 10192 Acked Flush 0 Done Done Sent false
36890 10193 Acked Write 9 Done Done Sent false
36891 10194 Acked Read 5 Done Done Sent false
36892 10195 Acked Write 10 Done Done Sent false
36893 10196 Acked Write 1 Done Done Sent false
36894 10197 Acked Flush 0 Done Done Sent false
36895 10198 Acked Write 4 Done Done Sent false
36896 10199 Acked Read 10 Done Done Sent false
36897 10200 Acked Read 6 Done Done Sent false
36898 10201 Acked Read 3 Done Done Sent false
36899 10202 Acked Read 8 Done Done Sent false
36900 10203 Acked Read 6 Done Done Sent false
36901 10204 Acked Read 7 Done Done Sent false
36902 10205 Acked Write 7 Done Done Sent false
36903 10206 Acked Flush 0 Done Done Sent false
36904 10207 Acked Read 6 Done Done Sent false
36905 10208 Acked Read 10 Done Done Sent false
36906 10209 Acked Read 9 Done Done Sent false
36907 10210 Acked Read 4 Done Done Sent false
36908 10211 Acked Write 3 Done Done Sent false
36909 10212 Acked Write 5 Done Done Sent false
36910 10213 Acked Read 1 Done Done Sent false
36911 10214 Acked Write 3 Done Done Sent false
36912 10215 Acked Write 8 Done Done Sent false
36913 10216 Acked Write 6 Done Done Sent false
36914 10217 Acked Read 5 Done Done Sent false
36915 10218 Acked Read 6 Done Done Sent false
36916 10219 Acked Read 2 Done Done Sent false
36917 10220 Acked Read 8 Done Done Sent false
36918 10221 Acked Read 3 Done Done Sent false
36919 10222 Acked Read 2 Done Done Sent false
36920 10223 Acked Write 3 Done Done Sent false
36921 10224 Acked Write 10 Done Done Sent false
36922 10225 Acked Read 3 Done Done Sent false
36923 10226 Acked Write 1 Done Done Sent false
36924 10227 Acked Write 3 Done Done Sent false
36925 10228 Acked Write 1 Done Done Sent false
36926 10229 Acked Write 3 Done Done Sent false
36927 10230 Acked Write 2 Done Done Sent false
36928 10231 Acked Read 10 Done Done Sent false
36929 10232 Acked Read 9 Done Done Sent false
36930 10233 Acked Flush 0 Done Done Sent false
36931 10234 Acked Read 9 Done Done Sent false
36932 10235 Acked Write 9 Done Done Sent false
36933 10236 Acked Write 3 Done Done Sent false
36934 10237 Acked Write 9 Done Done Sent false
36935 10238 Acked Read 6 Done Done Sent false
36936 10239 Acked Write 8 Done Done Sent false
36937 10240 Acked Read 8 Done Done Sent false
36938 10241 Acked Write 2 Done Done Sent false
36939 10242 Acked Flush 0 Done Done Sent false
36940 10243 Acked Flush 0 Done Done Sent false
36941 10244 Acked Read 3 Done Done Sent false
36942 10245 Acked Write 5 Done Done Sent false
36943 10246 Acked Write 7 Done Done Sent false
36944 10247 Acked Read 10 Done Done Sent false
36945 10248 Acked Write 1 Done Done Sent false
36946 10249 Acked Read 6 Done Done Sent false
36947 10250 Acked Read 6 Done Done Sent false
36948 10251 Acked Read 8 Done Done Sent false
36949 10252 Acked Read 9 Done Done Sent false
36950 10253 Acked Read 10 Done Done Sent false
36951 10254 Acked Write 7 Done Done Sent false
36952 10255 Acked Write 2 Done Done Sent false
36953 10256 Acked Write 8 Done Done Sent false
36954 10257 Acked Write 1 Done Done Sent false
36955 10258 Acked Read 6 Done Done Sent false
36956 10259 Acked Read 8 Done Done Sent false
36957 10260 Acked Write 2 Done Done Sent false
36958 10261 Acked Write 3 Done Done Sent false
36959 10262 Acked Write 7 Done Done Sent false
36960 10263 Acked Write 1 Done Done Sent false
36961 10264 Acked Flush 0 Done Done Sent false
36962 10265 Acked Write 7 Done Done Sent false
36963 10266 Acked Flush 0 Done Done Sent false
36964 10267 Acked Write 2 Done Done Sent false
36965 10268 Acked Read 4 Done Done Sent false
36966 10269 Acked Write 7 Done Done Sent false
36967 10270 Acked Read 10 Done Done Sent false
36968 10271 Acked Read 3 Done Done Sent false
36969 10272 Acked Write 5 Done Done Sent false
36970 10273 Acked Write 8 Done Done Sent false
36971 10274 Acked Write 7 Done Done Sent false
36972 10275 Acked Read 9 Done Done Sent false
36973 10276 Acked Read 1 Done Done Sent false
36974 10277 Acked Read 1 Done Done Sent false
36975 10278 Acked Write 1 Done Done Sent false
36976 10279 Acked Read 6 Done Done Sent false
36977 10280 Acked Write 1 Done Done Sent false
36978 10281 Acked Flush 0 Done Done Sent false
36979 10282 Acked Read 9 Done Done Sent false
36980 10283 Acked Flush 0 Done Done Sent false
36981 10284 Acked Write 9 Done Done Sent false
36982 10285 Acked Write 2 Done Done Sent false
36983 10286 Acked Read 2 Done Done Sent false
36984 10287 Acked Read 4 Done Done Sent false
36985 10288 Acked Flush 0 Done Done Sent false
36986 10289 Acked Write 1 Done Done Sent false
36987 10290 Acked Read 8 Done Done Sent false
36988 10291 Acked Read 10 Done Done Sent false
36989 10292 Acked Read 3 Done Done Sent false
36990 10293 Acked Write 8 Done Done Sent false
36991 10294 Acked Write 10 Done Done Sent false
36992 10295 Acked Read 4 Done Done Sent false
36993 10296 Acked Write 6 Done Done Sent false
36994 10297 Acked Write 4 Done Done Sent false
36995 10298 Acked Read 4 Done Done Sent false
36996 10299 Acked Write 1 Done Done Sent false
36997 10300 Acked Read 8 Done Done Sent false
36998 10301 Acked Read 2 Done Done Sent false
36999 10302 Acked Write 3 Done Done Sent false
37000 10303 Acked Write 3 Done Done Sent false
37001 10304 Acked Read 3 Done Done Sent false
37002 10305 Acked Write 10 Done Done Sent false
37003 10306 Acked Read 10 Done Done Sent false
37004 10307 Acked Read 4 Done Done Sent false
37005 10308 Acked Write 9 Done Done Sent false
37006 10309 Acked Read 10 Done Done Sent false
37007 10310 Acked Read 2 Done Done Sent false
37008 10311 Acked Read 10 Done Done Sent false
37009 10312 Acked Read 3 Done Done Sent false
37010 10313 Acked Read 8 Done Done Sent false
37011 10314 Acked Write 8 Done Done Sent false
37012 10315 Acked Write 6 Done Done Sent false
37013 10316 Acked Read 1 Done Done Sent false
37014 10317 Acked Write 2 Done Done Sent false
37015 10318 Acked Flush 0 Done Done Sent false
37016 10319 Acked Write 8 Done Done Sent false
37017 10320 Acked Write 9 Done Done Sent false
37018 10321 Acked Write 2 Done Done Sent false
37019 10322 Acked Read 8 Done Done Sent false
37020 10323 Acked Write 10 Done Done Sent false
37021 10324 Acked Flush 0 Done Done Sent false
37022 10325 Acked Read 6 Done Done Sent false
37023 10326 Acked Read 8 Done Done Sent false
37024 10327 Acked Read 1 Done Done Sent false
37025 10328 Acked Write 5 Done Done Sent false
37026 10329 Acked Read 3 Done Done Sent false
37027 10330 Acked Read 7 Done Done Sent false
37028 10331 Acked Write 4 Done Done Sent false
37029 10332 Acked Read 2 Done Done Sent false
37030 10333 Acked Write 2 Done Done Sent false
37031 10334 Acked Read 4 Done Done Sent false
37032 10335 Acked Write 6 Done Done Sent false
37033 10336 Acked Read 1 Done Done Sent false
37034 10337 Acked Read 8 Done Done Sent false
37035 10338 Acked Write 10 Done Done Sent false
37036 10339 Acked Read 8 Done Done Sent false
37037 10340 Acked Flush 0 Done Done Sent false
37038 10341 Acked Read 5 Done Done Sent false
37039 10342 Acked Write 8 Done Done Sent false
37040 10343 Acked Flush 0 Done Done Sent false
37041 10344 Acked Write 9 Done Done Sent false
37042 10345 Acked Read 10 Done Done Sent false
37043 10346 Acked Write 7 Done Done Sent false
37044 10347 Acked Flush 0 Done Done Sent false
37045 10348 Acked Read 7 Done Done Sent false
37046 10349 Acked Write 1 Done Done Sent false
37047 10350 Acked Read 4 Done Done Sent false
37048 10351 Acked Write 7 Done Done Sent false
37049 10352 Acked Write 9 Done Done Sent false
37050 10353 Acked Read 6 Done Done Sent false
37051 10354 Acked Read 9 Done Done Sent false
37052 10355 Acked Read 7 Done Done Sent false
37053 10356 Acked Write 8 Done Done Sent false
37054 10357 Acked Write 10 Done Done Sent false
37055 10358 Acked Flush 0 Done Done Sent false
37056 STATES DS:0 DS:1 DS:2 TOTAL
37057 Sent 0 0 302 302
37058 Done 302 302 0 604
37059 Skipped 0 0 0 0
37060 Error 0 0 0 0
37061 Last Flush: 10358 10358 10056
37062 Downstairs last five completed: 10056 10055 10054 10053 10052
37063 Upstairs last five completed: 10358 10357 10356 10355 10354
37064 CLIENT: Up:0 ds:302 act:2
370652024-11-26T22:12:19.764ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
370662024-11-26T22:12:19.764ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:39306) = io task client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
370672024-11-26T22:12:19.765ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
370682024-11-26T22:12:19.765ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
370692024-11-26T22:12:19.765ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
370702024-11-26T22:12:19.765ZINFOcrucible: send last flush ID to this DS: 10056 = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
370712024-11-26T22:12:19.765ZINFOcrucible: Replied this last flush ID: 10056 = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
370722024-11-26T22:12:19.765ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
370732024-11-26T22:12:19.765ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
370742024-11-26T22:12:19.765ZINFOcrucible: [2] client re-new 302 jobs since flush 10056 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
370752024-11-26T22:12:19.766ZINFOcrucible: [2] Replayed 302 jobs since flush: 10056 = downstairs session_id = 86a36c17-3718-484c-87fd-2b8a1dc2ab18
37076 ----------------------------------------------------------------
37077 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
37078 Downstairs last five completed: 10358 10357 10356 10355 10354
37079 Upstairs last five completed: 10358 10357 10356 10355 10354
37080 CLIENT: Up:0 ds:0 act:3
37081 Replay: All jobs finished
37082 Test replay has completed
37083 Wrote out file "/tmp/test_replay_verify.log"
37084 CLIENT: Tests done. All submitted work has been ACK'd
37085 ----------------------------------------------------------------
37086 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
37087 Downstairs last five completed: 10358 10357 10356 10355 10354
37088 Upstairs last five completed: 10358 10357 10356 10355 10354
37089 CLIENT: Up:0 ds:0 act:3
37090 CLIENT: All jobs finished, all DS active.
370912024-11-26T22:12:20.012ZWARNcrucible: Guest handle has been dropped
37092 Replays:30 time: 7:44
37093 Do final verify
37094 Nov 26 22:12:20.085 INFO accepted connection, remote_addr: 127.0.0.1:55973, local_addr: 127.0.0.1:9998
37095 Nov 26 22:12:20.086 INFO request completed, latency_us: 200, response_code: 200, uri: /regioninfo, method: GET, req_id: 32cf6428-8b52-4443-b983-62fbe47f57a0, remote_addr: 127.0.0.1:55973, local_addr: 127.0.0.1:9998
370962024-11-26T22:12:20.087ZINFOcrucible: Use this region info from dsc: RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 } task = crutest
37097 Nov 26 22:12:20.087 INFO request completed, latency_us: 91, response_code: 200, uri: /regioncount, method: GET, req_id: 1c4017c5-c300-43c8-84f7-e13c3c9b5d3b, remote_addr: 127.0.0.1:55973, local_addr: 127.0.0.1:9998
370982024-11-26T22:12:20.087ZINFOcrucible: dsc has 3 regions. This means 1 sub_volumes task = crutest
37099 Nov 26 22:12:20.088 INFO request completed, latency_us: 114, response_code: 200, uri: /port/cid/0, method: GET, req_id: c80d0436-876d-4f7a-974f-3c61064b8bbf, remote_addr: 127.0.0.1:55973, local_addr: 127.0.0.1:9998
37100 Nov 26 22:12:20.088 INFO request completed, latency_us: 94, response_code: 200, uri: /port/cid/1, method: GET, req_id: a22bf62b-6b05-4a53-8162-b1f13a0158ab, remote_addr: 127.0.0.1:55973, local_addr: 127.0.0.1:9998
37101 Nov 26 22:12:20.089 INFO request completed, latency_us: 96, response_code: 200, uri: /port/cid/2, method: GET, req_id: d0648618-8639-4562-960d-e7971c2c76b9, remote_addr: 127.0.0.1:55973, local_addr: 127.0.0.1:9998
371022024-11-26T22:12:20.089ZINFOcrucible: SV 0 has targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830] task = crutest
371032024-11-26T22:12:20.089ZINFOcrucible: Upstairs starts
371042024-11-26T22:12:20.089ZINFOcrucible: Crucible Version: BuildInfo { version: "0.0.1", git_sha: "6bdc5ae0b2033c80bbfa3a59474c7c52737638e4", git_commit_timestamp: "2024-11-26T21:48:03.000000000Z", git_branch: "mkeeter/defer-flush-and-barrier", rustc_semver: "1.82.0", rustc_channel: "stable", rustc_host_triple: "x86_64-unknown-illumos", rustc_commit_sha: "f6e511eec7342f59a25f7c0534f1dbea00d01b14", cargo_triple: "x86_64-unknown-illumos", debug: true, opt_level: 0, }
371052024-11-26T22:12:20.089ZINFOcrucible: Upstairs <-> Downstairs Message Version: 12
371062024-11-26T22:12:20.089ZINFOcrucible: Using region definition RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 00000000-0000-0000-0000-000000000000, encrypted: false, database_read_version: 1, database_write_version: 1 }
371072024-11-26T22:12:20.090ZINFOcrucible: Crucible ec94ae5e-cc8a-441a-adad-b6c2b84c2151 has session id: a28dfb8c-e395-40e0-9d67-efbf3a5ced03 session_id = a28dfb8c-e395-40e0-9d67-efbf3a5ced03
371082024-11-26T22:12:20.090ZINFOcrucible: Upstairs opts: Upstairs UUID: ec94ae5e-cc8a-441a-adad-b6c2b84c2151, Targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830], lossy: false, flush_timeout: None, key populated: false, cert_pem populated: false, key_pem populated: false, root_cert_pem populated: false, Control: None, read_only: false session_id = a28dfb8c-e395-40e0-9d67-efbf3a5ced03
371092024-11-26T22:12:20.090ZINFOcrucible: Crucible stats registered with UUID: ec94ae5e-cc8a-441a-adad-b6c2b84c2151 session_id = a28dfb8c-e395-40e0-9d67-efbf3a5ced03
371102024-11-26T22:12:20.123ZINFOcrucible: Downstairs in volume = 3 task = crutest
371112024-11-26T22:12:20.123ZINFOcrucible: The guest has requested activation with gen:3
371122024-11-26T22:12:20.123ZINFOcrucible: ec94ae5e-cc8a-441a-adad-b6c2b84c2151 active request set session_id = a28dfb8c-e395-40e0-9d67-efbf3a5ced03
371132024-11-26T22:12:20.123ZINFOcrucible: sending connect oneshot to client = downstairs client = 0 session_id = a28dfb8c-e395-40e0-9d67-efbf3a5ced03
371142024-11-26T22:12:20.123ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 0 session_id = a28dfb8c-e395-40e0-9d67-efbf3a5ced03
371152024-11-26T22:12:20.123ZINFOcrucible: sending connect oneshot to client = downstairs client = 1 session_id = a28dfb8c-e395-40e0-9d67-efbf3a5ced03
371162024-11-26T22:12:20.123ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 1 session_id = a28dfb8c-e395-40e0-9d67-efbf3a5ced03
371172024-11-26T22:12:20.123ZINFOcrucible: sending connect oneshot to client = downstairs client = 2 session_id = a28dfb8c-e395-40e0-9d67-efbf3a5ced03
371182024-11-26T22:12:20.123ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 2 session_id = a28dfb8c-e395-40e0-9d67-efbf3a5ced03
371192024-11-26T22:12:20.123ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = a28dfb8c-e395-40e0-9d67-efbf3a5ced03
371202024-11-26T22:12:20.124ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = a28dfb8c-e395-40e0-9d67-efbf3a5ced03
371212024-11-26T22:12:20.124ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = a28dfb8c-e395-40e0-9d67-efbf3a5ced03
371222024-11-26T22:12:20.124ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:54549) = io task client = 0 session_id = a28dfb8c-e395-40e0-9d67-efbf3a5ced03
371232024-11-26T22:12:20.124ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:65178) = io task client = 2 session_id = a28dfb8c-e395-40e0-9d67-efbf3a5ced03
371242024-11-26T22:12:20.124ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:39683) = io task client = 1 session_id = a28dfb8c-e395-40e0-9d67-efbf3a5ced03
371252024-11-26T22:12:20.124ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 0 session_id = a28dfb8c-e395-40e0-9d67-efbf3a5ced03
371262024-11-26T22:12:20.124ZINFOcrucible: [0] Transition from New to WaitActive = downstairs client = 0 session_id = a28dfb8c-e395-40e0-9d67-efbf3a5ced03
371272024-11-26T22:12:20.124ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 1 session_id = a28dfb8c-e395-40e0-9d67-efbf3a5ced03
371282024-11-26T22:12:20.124ZINFOcrucible: [1] Transition from New to WaitActive = downstairs client = 1 session_id = a28dfb8c-e395-40e0-9d67-efbf3a5ced03
371292024-11-26T22:12:20.124ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 2 session_id = a28dfb8c-e395-40e0-9d67-efbf3a5ced03
371302024-11-26T22:12:20.124ZINFOcrucible: [2] Transition from New to WaitActive = downstairs client = 2 session_id = a28dfb8c-e395-40e0-9d67-efbf3a5ced03
371312024-11-26T22:12:20.125ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = a28dfb8c-e395-40e0-9d67-efbf3a5ced03
371322024-11-26T22:12:20.125ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = a28dfb8c-e395-40e0-9d67-efbf3a5ced03
371332024-11-26T22:12:20.125ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = a28dfb8c-e395-40e0-9d67-efbf3a5ced03
371342024-11-26T22:12:20.126ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 1 session_id = a28dfb8c-e395-40e0-9d67-efbf3a5ced03
371352024-11-26T22:12:20.126ZINFOcrucible: [1] Transition from WaitActive to WaitQuorum = downstairs client = 1 session_id = a28dfb8c-e395-40e0-9d67-efbf3a5ced03
371362024-11-26T22:12:20.126ZINFOcrucible: Waiting for more clients to be ready session_id = a28dfb8c-e395-40e0-9d67-efbf3a5ced03
371372024-11-26T22:12:20.126ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 2 session_id = a28dfb8c-e395-40e0-9d67-efbf3a5ced03
371382024-11-26T22:12:20.126ZINFOcrucible: [2] Transition from WaitActive to WaitQuorum = downstairs client = 2 session_id = a28dfb8c-e395-40e0-9d67-efbf3a5ced03
371392024-11-26T22:12:20.126ZINFOcrucible: Waiting for more clients to be ready session_id = a28dfb8c-e395-40e0-9d67-efbf3a5ced03
371402024-11-26T22:12:20.126ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 0 session_id = a28dfb8c-e395-40e0-9d67-efbf3a5ced03
371412024-11-26T22:12:20.126ZINFOcrucible: [0] Transition from WaitActive to WaitQuorum = downstairs client = 0 session_id = a28dfb8c-e395-40e0-9d67-efbf3a5ced03
371422024-11-26T22:12:20.126ZINFOcrucible: [0]R flush_numbers[0..12]: [900, 912, 893, 920, 920, 907, 919, 912, 907, 893, 882, 912] = downstairs session_id = a28dfb8c-e395-40e0-9d67-efbf3a5ced03
371432024-11-26T22:12:20.126ZINFOcrucible: [0]R generation[0..12]: [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2] = downstairs session_id = a28dfb8c-e395-40e0-9d67-efbf3a5ced03
371442024-11-26T22:12:20.126ZINFOcrucible: [0]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = a28dfb8c-e395-40e0-9d67-efbf3a5ced03
371452024-11-26T22:12:20.126ZINFOcrucible: [1]R flush_numbers[0..12]: [900, 912, 893, 920, 920, 907, 919, 912, 907, 893, 882, 912] = downstairs session_id = a28dfb8c-e395-40e0-9d67-efbf3a5ced03
371462024-11-26T22:12:20.126ZINFOcrucible: [1]R generation[0..12]: [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2] = downstairs session_id = a28dfb8c-e395-40e0-9d67-efbf3a5ced03
371472024-11-26T22:12:20.126ZINFOcrucible: [1]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = a28dfb8c-e395-40e0-9d67-efbf3a5ced03
371482024-11-26T22:12:20.126ZINFOcrucible: [2]R flush_numbers[0..12]: [900, 912, 893, 920, 920, 907, 919, 912, 907, 893, 882, 912] = downstairs session_id = a28dfb8c-e395-40e0-9d67-efbf3a5ced03
371492024-11-26T22:12:20.126ZINFOcrucible: [2]R generation[0..12]: [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2] = downstairs session_id = a28dfb8c-e395-40e0-9d67-efbf3a5ced03
371502024-11-26T22:12:20.126ZINFOcrucible: [2]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = a28dfb8c-e395-40e0-9d67-efbf3a5ced03
371512024-11-26T22:12:20.126ZINFOcrucible: Max found gen is 3 = downstairs session_id = a28dfb8c-e395-40e0-9d67-efbf3a5ced03
371522024-11-26T22:12:20.126ZINFOcrucible: Generation requested: 3 >= found:3 = downstairs session_id = a28dfb8c-e395-40e0-9d67-efbf3a5ced03
371532024-11-26T22:12:20.126ZINFOcrucible: Next flush: 921 = downstairs session_id = a28dfb8c-e395-40e0-9d67-efbf3a5ced03
37154 Wait for a query_work_queue command to finish before sending IO
371552024-11-26T22:12:20.126ZINFOcrucible: All extents match = downstairs session_id = a28dfb8c-e395-40e0-9d67-efbf3a5ced03
371562024-11-26T22:12:20.126ZINFOcrucible: No downstairs reconciliation required session_id = a28dfb8c-e395-40e0-9d67-efbf3a5ced03
371572024-11-26T22:12:20.126ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 0 session_id = a28dfb8c-e395-40e0-9d67-efbf3a5ced03
371582024-11-26T22:12:20.126ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 1 session_id = a28dfb8c-e395-40e0-9d67-efbf3a5ced03
371592024-11-26T22:12:20.126ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 2 session_id = a28dfb8c-e395-40e0-9d67-efbf3a5ced03
371602024-11-26T22:12:20.126ZINFOcrucible: All required reconciliation work is completed session_id = a28dfb8c-e395-40e0-9d67-efbf3a5ced03
371612024-11-26T22:12:20.126ZINFOcrucible: Set Downstairs and Upstairs active after reconciliation session_id = a28dfb8c-e395-40e0-9d67-efbf3a5ced03
371622024-11-26T22:12:20.126ZINFOcrucible: ec94ae5e-cc8a-441a-adad-b6c2b84c2151 is now active with session: a28dfb8c-e395-40e0-9d67-efbf3a5ced03 session_id = a28dfb8c-e395-40e0-9d67-efbf3a5ced03
371632024-11-26T22:12:20.126ZINFOcrucible: Set Active after no reconciliation session_id = a28dfb8c-e395-40e0-9d67-efbf3a5ced03
371642024-11-26T22:12:20.126ZINFOcrucible: The guest has finished waiting for activation with:3
37165 Disk: sv:1 bs:4096 ts:20480000 tb:5000 max_io:256 or 1048576
37166 Loading write count information from file "/tmp/test_replay_verify.log"
37167 Read and Verify all blocks (0..5000 range:false)
37168 Wrote out file "/tmp/test_replay_verify.log"
37169 Verify test completed
37170 Wrote out file "/tmp/test_replay_verify.log"
37171 CLIENT: Tests done. All submitted work has been ACK'd
37172 ----------------------------------------------------------------
37173 Crucible gen:3 GIO:true work queues: Upstairs:0 downstairs:20
37174 States: Active Active Active
37175 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
37176 1000 Acked Read 256 Done Done Done false
37177 1001 Acked Read 256 Done Done Done false
37178 1002 Acked Read 256 Done Done Done false
37179 1003 Acked Read 256 Done Done Done false
37180 1004 Acked Read 256 Done Done Done false
37181 1005 Acked Read 256 Done Done Done false
37182 1006 Acked Read 256 Done Done Done false
37183 1007 Acked Read 256 Done Done Done false
37184 1008 Acked Read 256 Done Done Done false
37185 1009 Acked Read 256 Done Done Done false
37186 1010 Acked Read 256 Done Done Done false
37187 1011 Acked Read 256 Done Done Done false
37188 1012 Acked Read 256 Done Done Done false
37189 1013 Acked Read 256 Done Done Done false
37190 1014 Acked Read 256 Done Done Done false
37191 1015 Acked Read 256 Done Done Done false
37192 1016 Acked Read 256 Done Done Done false
37193 1017 Acked Read 256 Done Done Done false
37194 1018 Acked Read 256 Done Done Done false
37195 1019 Acked Read 136 Done Done Done false
37196 STATES DS:0 DS:1 DS:2 TOTAL
37197 Sent 0 0 0 0
37198 Done 20 20 20 60
37199 Skipped 0 0 0 0
37200 Error 0 0 0 0
37201 Last Flush: 0 0 0
37202 Downstairs last five completed:
37203 Upstairs last five completed: 1019 1018 1017 1016 1015
37204 CLIENT: Up:0 ds:20 act:3
37205 ----------------------------------------------------------------
37206 Crucible gen:3 GIO:true work queues: Upstairs:0 downstairs:0
37207 Downstairs last five completed: 1020 1019 1018 1017 1016
37208 Upstairs last five completed: 1020 1019 1018 1017 1016
37209 CLIENT: Up:0 ds:0 act:3
37210 CLIENT: All crucible jobs finished, exiting program
372112024-11-26T22:12:24.531ZWARNcrucible: Guest handle has been dropped
37212 Nov 26 22:12:24.598 INFO accepted connection, remote_addr: 127.0.0.1:45682, local_addr: 127.0.0.1:9998
37213 Main task has work to do, go find it
37214 got dsc Shutdown
37215 Shutdown
37216 Nov 26 22:12:24.599 INFO request completed, latency_us: 158, response_code: 204, uri: /shutdown, method: POST, req_id: 9c17e29b-6fb8-447d-9b51-cd540ec6225c, remote_addr: 127.0[8810] Disable keep_running
37217 .Shut it down
37218 0[8810] Got stop action so:false kr:false
37219 .1:45682, local_addr: 127.0.0.1:9998
37220 [8820] Disable keep_running
37221 [8820] Got stop action so:false kr:false
37222 [8830] Disable keep_running
37223 [8830] Got stop action so:false kr:false
37224 [8830] Exited with: ExitStatus(unix_wait_status(9))
37225 [8810] Exited with: ExitStatus(unix_wait_status(9))
37226 [8830][2] reports Exit
37227 [8810][0] reports Exit
37228 [8820] Exited with: ExitStatus(unix_wait_status(9))
37229 [8820][1] reports Exit
37230 November 26, 2024 at 10:12:29 PM UTC Test ends with 0