Max level shown:
1 starting November 15, 2024 at 01:57:21 AM UTC
2 Creating 3 downstairs regions
3 Creating region directory at: /var/tmp/dsc/region
4 Creating dsc directory at: /tmp/dsc
5 Downstairs region 0 created at /var/tmp/dsc/region/8810 in 0.26284888
6 Downstairs region 1 created at /var/tmp/dsc/region/8820 in 0.01969232
7 Downstairs region 2 created at /var/tmp/dsc/region/8830 in 0.0200529
8 Created 3 regions
9 Starting 3 downstairs
10 Using existing output directory "/tmp/dsc"
11 Update our region info with: Some(RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 })
12 start ds: 8810
13 start access at:127.0.0.1:9998
14 start ds: 8820
15 start ds: 8830
16 Starting downstairs at port 8810
17 Starting downstairs at port 8820
18 Make output file at "/tmp/dsc/downstairs-8810.txt"
19 Make output file at "/tmp/dsc/downstairs-8820.txt"
20 Starting downstairs at port 8830
21 Make output file at "/tmp/dsc/downstairs-8830.txt"
22 [8810][0] initial start wait reports Starting
23 [8820][1] initial start wait reports Starting
24 [8830][2] initial start wait reports Starting
25 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1195)
26 [8820][1] initial start wait reports Running
27 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1196)
28 [8830][2] initial start wait reports Running
29 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1197)
30 [8810][0] initial start wait reports Running
31 All downstairs are running
32 Nov 15 01:57:21.969 INFO listening, local_addr: 127.0.0.1:9998
33 Control access at:127.0.0.1:9998
34 Running initial fill
35 Nov 15 01:57:27.062 INFO accepted connection, remote_addr: 127.0.0.1:51852, local_addr: 127.0.0.1:9998
36 Nov 15 01:57:27.065 INFO request completed, latency_us: 466, response_code: 200, uri: /regioninfo, method: GET, req_id: f0677d52-6f34-48af-a0b3-db3c32acad6a, remote_addr: 127.0.0.1:51852, local_addr: 127.0.0.1:9998
37 Nov 15 01:57:27.068 INFO request completed, latency_us: 107, response_code: 200, uri: /regioncount, method: GET, req_id: 2d23c3cd-fe2b-4c73-b2c5-9a89df177a06, remote_addr: 127.0.0.1:51852, local_addr: 127.0.0.1:9998
382024-11-15T01:57:27.068ZINFOcrucible: Use this region info from dsc: RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 } task = crutest
392024-11-15T01:57:27.069ZINFOcrucible: dsc has 3 regions. This means 1 sub_volumes task = crutest
40 Nov 15 01:57:27.070 INFO request completed, latency_us: 157, response_code: 200, uri: /port/cid/0, method: GET, req_id: d8035949-4d77-4960-a005-9b1d045045ac, remote_addr: 127.0.0.1:51852, local_addr: 127.0.0.1:9998
41 Nov 15 01:57:27.070 INFO request completed, latency_us: 90, response_code: 200, uri: /port/cid/1, method: GET, req_id: c24a1f80-0278-4c05-82a2-a9c86d2b8169, remote_addr: 127.0.0.1:51852, local_addr: 127.0.0.1:9998
42 Nov 15 01:57:27.071 INFO request completed, latency_us: 105, response_code: 200, uri: /port/cid/2, method: GET, req_id: c2eebf3f-5a79-4bdc-a282-235b0f68a443, remote_addr: 127.0.0.1:51852, local_addr: 127.0.0.1:9998
432024-11-15T01:57:27.071ZINFOcrucible: SV 0 has targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830] task = crutest
442024-11-15T01:57:27.073ZINFOcrucible: Upstairs starts
452024-11-15T01:57:27.073ZINFOcrucible: Crucible Version: BuildInfo { version: "0.0.1", git_sha: "5698e843f5bfd164775ff7ff0d12625b59494d67", git_commit_timestamp: "2024-11-15T01:38:50.000000000Z", git_branch: "alan/global-state-from-dtrace", rustc_semver: "1.82.0", rustc_channel: "stable", rustc_host_triple: "x86_64-unknown-illumos", rustc_commit_sha: "f6e511eec7342f59a25f7c0534f1dbea00d01b14", cargo_triple: "x86_64-unknown-illumos", debug: true, opt_level: 0, }
462024-11-15T01:57:27.073ZINFOcrucible: Upstairs <-> Downstairs Message Version: 12
472024-11-15T01:57:27.073ZINFOcrucible: Using region definition RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 00000000-0000-0000-0000-000000000000, encrypted: false, database_read_version: 1, database_write_version: 1 }
482024-11-15T01:57:27.074ZINFOcrucible: Crucible fdf57624-93ac-4cfe-869c-9c87d7873006 has session id: b227cfca-72d4-403e-983b-c25f20d9cb78 session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
492024-11-15T01:57:27.074ZINFOcrucible: Upstairs opts: Upstairs UUID: fdf57624-93ac-4cfe-869c-9c87d7873006, Targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830], lossy: false, flush_timeout: None, key populated: false, cert_pem populated: false, key_pem populated: false, root_cert_pem populated: false, Control: None, read_only: false session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
502024-11-15T01:57:27.074ZINFOcrucible: Crucible stats registered with UUID: fdf57624-93ac-4cfe-869c-9c87d7873006 session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
512024-11-15T01:57:27.109ZINFOcrucible: Downstairs in volume = 3 task = crutest
522024-11-15T01:57:27.110ZINFOcrucible: The guest has requested activation with gen:1
532024-11-15T01:57:27.110ZINFOcrucible: fdf57624-93ac-4cfe-869c-9c87d7873006 active request set session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
542024-11-15T01:57:27.110ZINFOcrucible: sending connect oneshot to client = downstairs client = 0 session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
552024-11-15T01:57:27.110ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 0 session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
562024-11-15T01:57:27.110ZINFOcrucible: sending connect oneshot to client = downstairs client = 1 session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
572024-11-15T01:57:27.110ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 1 session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
582024-11-15T01:57:27.110ZINFOcrucible: sending connect oneshot to client = downstairs client = 2 session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
592024-11-15T01:57:27.110ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 2 session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
602024-11-15T01:57:27.110ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
612024-11-15T01:57:27.110ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
622024-11-15T01:57:27.110ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
632024-11-15T01:57:27.110ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:37653) = io task client = 2 session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
642024-11-15T01:57:27.110ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:54845) = io task client = 0 session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
652024-11-15T01:57:27.110ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:38383) = io task client = 1 session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
662024-11-15T01:57:27.112ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 0 session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
672024-11-15T01:57:27.112ZINFOcrucible: [0] Transition from New to WaitActive = downstairs client = 0 session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
682024-11-15T01:57:27.113ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 1 session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
692024-11-15T01:57:27.113ZINFOcrucible: [1] Transition from New to WaitActive = downstairs client = 1 session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
702024-11-15T01:57:27.113ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 2 session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
712024-11-15T01:57:27.113ZINFOcrucible: [2] Transition from New to WaitActive = downstairs client = 2 session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
722024-11-15T01:57:27.114ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
732024-11-15T01:57:27.114ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
742024-11-15T01:57:27.114ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
752024-11-15T01:57:27.114ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 1 session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
762024-11-15T01:57:27.115ZINFOcrucible: [1] Transition from WaitActive to WaitQuorum = downstairs client = 1 session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
772024-11-15T01:57:27.115ZINFOcrucible: Waiting for more clients to be ready session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
782024-11-15T01:57:27.115ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 2 session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
792024-11-15T01:57:27.115ZINFOcrucible: [2] Transition from WaitActive to WaitQuorum = downstairs client = 2 session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
802024-11-15T01:57:27.115ZINFOcrucible: Waiting for more clients to be ready session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
812024-11-15T01:57:27.115ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 0 session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
822024-11-15T01:57:27.115ZINFOcrucible: [0] Transition from WaitActive to WaitQuorum = downstairs client = 0 session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
832024-11-15T01:57:27.115ZINFOcrucible: [0]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
842024-11-15T01:57:27.115ZINFOcrucible: [0]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
852024-11-15T01:57:27.115ZINFOcrucible: [0]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
862024-11-15T01:57:27.115ZINFOcrucible: [1]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
872024-11-15T01:57:27.115ZINFOcrucible: [1]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
882024-11-15T01:57:27.115ZINFOcrucible: [1]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
892024-11-15T01:57:27.115ZINFOcrucible: [2]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
902024-11-15T01:57:27.115ZINFOcrucible: [2]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
912024-11-15T01:57:27.115ZINFOcrucible: [2]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
922024-11-15T01:57:27.115ZINFOcrucible: Max found gen is 1 = downstairs session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
932024-11-15T01:57:27.115ZINFOcrucible: Generation requested: 1 >= found:1 = downstairs session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
942024-11-15T01:57:27.115ZINFOcrucible: Next flush: 1 = downstairs session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
952024-11-15T01:57:27.116ZINFOcrucible: All extents match = downstairs session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
962024-11-15T01:57:27.116ZINFOcrucible: No downstairs reconciliation required session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
972024-11-15T01:57:27.116ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 0 session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
982024-11-15T01:57:27.116ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 1 session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
992024-11-15T01:57:27.116ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 2 session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
1002024-11-15T01:57:27.116ZINFOcrucible: All required reconciliation work is completed session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
1012024-11-15T01:57:27.116ZINFOcrucible: Set Downstairs and Upstairs active after reconciliation session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
1022024-11-15T01:57:27.116ZINFOcrucible: fdf57624-93ac-4cfe-869c-9c87d7873006 is now active with session: b227cfca-72d4-403e-983b-c25f20d9cb78 session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
1032024-11-15T01:57:27.116ZINFOcrucible: The guest has finished waiting for activation with:1
1042024-11-15T01:57:27.116ZINFOcrucible: Set Active after no reconciliation session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
105 Activate successful
106 Wait for a query_work_queue command to finish before sending IO
107 Region: sv:1 bs:4096 ts:20480000 tb:5000 max_io:256 or 1048576
108 Fill test
109 Read and Verify all blocks (0..5000 range:false)
110 Wrote out file "/tmp/test_replay_verify.log"
111 CLIENT: Tests done. All submitted work has been ACK'd
112 ----------------------------------------------------------------
113 Crucible gen:1 GIO:true work queues: Upstairs:0 downstairs:20
114 States: Active Active Active
115 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
116 1021 Acked Read 256 Done Done Done false
117 1022 Acked Read 256 Done Done Done false
118 1023 Acked Read 256 Done Done Done false
119 1024 Acked Read 256 Done Done Done false
120 1025 Acked Read 256 Done Done Done false
121 1026 Acked Read 256 Done Done Done false
122 1027 Acked Read 256 Done Done Done false
123 1028 Acked Read 256 Done Done Done false
124 1029 Acked Read 256 Done Done Done false
125 1030 Acked Read 256 Done Done Done false
126 1031 Acked Read 256 Done Done Done false
127 1032 Acked Read 256 Done Done Done false
128 1033 Acked Read 256 Done Done Done false
129 1034 Acked Read 256 Done Done Done false
130 1035 Acked Read 256 Done Done Done false
131 1036 Acked Read 256 Done Done Done false
132 1037 Acked Read 256 Done Done Done false
133 1038 Acked Read 136 Done Done Done false
134 1039 Acked Read 256 Done Done Done false
135 1040 Acked Read 256 Done Done Done false
136 STATES DS:0 DS:1 DS:2 TOTAL
137 Sent 0 0 0 0
138 Done 20 20 20 60
139 Skipped 0 0 0 0
140 Error 0 0 0 0
141 Last Flush: 1020 1020 1020
142 Downstairs last five completed: 1020 1019 1018 1017 1016
143 Upstairs last five completed: 1040 1039 1038 1037 1036
144 CLIENT: Up:0 ds:20 act:3
145 ----------------------------------------------------------------
146 Crucible gen:1 GIO:true work queues: Upstairs:0 downstairs:0
147 Downstairs last five completed: 1041 1040 1039 1038 1037
148 Upstairs last five completed: 1041 1040 1039 1038 1037
149 CLIENT: Up:0 ds:0 act:3
150 CLIENT: All crucible jobs finished, exiting program
1512024-11-15T01:57:31.698ZWARNcrucible: Guest handle has been dropped
1522024-11-15T01:57:31.698ZWARNcrucible: client_stop_rx closed unexpectedly: RecvError(()) = io task client = 0 session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
1532024-11-15T01:57:31.698ZWARNcrucible: client_stop_rx closed unexpectedly: RecvError(()) = io task client = 1 session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
1542024-11-15T01:57:31.698ZWARNcrucible: client task is sending Done(QueueClosed) = io task client = 0 session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
1552024-11-15T01:57:31.698ZWARNcrucible: client task could not reply to main task; shutting down? = io task client = 0 session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
1562024-11-15T01:57:31.698ZINFOcrucible: client task is exiting = io task client = 0 session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
1572024-11-15T01:57:31.698ZWARNcrucible: client task is sending Done(QueueClosed) = io task client = 1 session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
1582024-11-15T01:57:31.698ZWARNcrucible: client task could not reply to main task; shutting down? = io task client = 1 session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
1592024-11-15T01:57:31.698ZINFOcrucible: client task is exiting = io task client = 1 session_id = b227cfca-72d4-403e-983b-c25f20d9cb78
160 Replay loop starts now November 15, 2024 at 01:57:31 AM UTC
161 Nov 15 01:57:31.772 INFO accepted connection, remote_addr: 127.0.0.1:38967, local_addr: 127.0.0.1:9998
162 Nov 15 01:57:31.773 INFO request completed, latency_us: 175, response_code: 200, uri: /regioninfo, method: GET, req_id: 79a331d1-e9b2-475f-bd6e-163df1a24e3e, remote_addr: 127.0.0.1:38967, local_addr: 127.0.0.1:9998
1632024-11-15T01:57:31.773ZINFOcrucible: Use this region info from dsc: RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 } task = crutest
164 Nov 15 01:57:31.774 INFO request completed, latency_us: 92, response_code: 200, uri: /regioncount, method: GET, req_id: 2e172b24-8fb6-4004-ac7e-addfc0a9068f, remote_addr: 127.0.0.1:38967, local_addr: 127.0.0.1:9998
1652024-11-15T01:57:31.774ZINFOcrucible: dsc has 3 regions. This means 1 sub_volumes task = crutest
166 Nov 15 01:57:31.774 INFO request completed, latency_us: 116, response_code: 200, uri: /port/cid/0, method: GET, req_id: b3b81d11-47a7-4771-95b2-badf310bcfc0, remote_addr: 127.0.0.1:38967, local_addr: 127.0.0.1:9998
167 Nov 15 01:57:31.775 INFO request completed, latency_us: 93, response_code: 200, uri: /port/cid/1, method: GET, req_id: eafc65fb-bafd-435a-b1ac-34120357a85d, remote_addr: 127.0.0.1:38967, local_addr: 127.0.0.1:9998
168 Nov 15 01:57:31.775 INFO request completed, latency_us: 92, response_code: 200, uri: /port/cid/2, method: GET, req_id: 05b74187-3055-4e22-a8b4-34ef57bf4b6e, remote_addr: 127.0.0.1:38967, local_addr: 127.0.0.1:9998
1692024-11-15T01:57:31.776ZINFOcrucible: SV 0 has targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830] task = crutest
1702024-11-15T01:57:31.776ZINFOcrucible: Upstairs starts
1712024-11-15T01:57:31.776ZINFOcrucible: Crucible Version: BuildInfo { version: "0.0.1", git_sha: "5698e843f5bfd164775ff7ff0d12625b59494d67", git_commit_timestamp: "2024-11-15T01:38:50.000000000Z", git_branch: "alan/global-state-from-dtrace", rustc_semver: "1.82.0", rustc_channel: "stable", rustc_host_triple: "x86_64-unknown-illumos", rustc_commit_sha: "f6e511eec7342f59a25f7c0534f1dbea00d01b14", cargo_triple: "x86_64-unknown-illumos", debug: true, opt_level: 0, }
1722024-11-15T01:57:31.776ZINFOcrucible: Upstairs <-> Downstairs Message Version: 12
1732024-11-15T01:57:31.776ZINFOcrucible: Using region definition RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 00000000-0000-0000-0000-000000000000, encrypted: false, database_read_version: 1, database_write_version: 1 }
1742024-11-15T01:57:31.776ZINFOcrucible: Crucible ce9afbf0-8fb7-43ec-bd70-9dd45ce26334 has session id: 971dcb2e-6891-4ecf-b367-1e04e52065ee session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
1752024-11-15T01:57:31.776ZINFOcrucible: Upstairs opts: Upstairs UUID: ce9afbf0-8fb7-43ec-bd70-9dd45ce26334, Targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830], lossy: false, flush_timeout: None, key populated: false, cert_pem populated: false, key_pem populated: false, root_cert_pem populated: false, Control: None, read_only: false session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
1762024-11-15T01:57:31.776ZINFOcrucible: Crucible stats registered with UUID: ce9afbf0-8fb7-43ec-bd70-9dd45ce26334 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
1772024-11-15T01:57:31.809ZINFOcrucible: Downstairs in volume = 3 task = crutest
1782024-11-15T01:57:31.809ZINFOcrucible: The guest has requested activation with gen:2
1792024-11-15T01:57:31.809ZINFOcrucible: ce9afbf0-8fb7-43ec-bd70-9dd45ce26334 active request set session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
1802024-11-15T01:57:31.809ZINFOcrucible: sending connect oneshot to client = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
1812024-11-15T01:57:31.809ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
1822024-11-15T01:57:31.809ZINFOcrucible: sending connect oneshot to client = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
1832024-11-15T01:57:31.809ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
1842024-11-15T01:57:31.809ZINFOcrucible: sending connect oneshot to client = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
1852024-11-15T01:57:31.809ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
1862024-11-15T01:57:31.809ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
1872024-11-15T01:57:31.809ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
1882024-11-15T01:57:31.809ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
1892024-11-15T01:57:31.810ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:32904) = io task client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
1902024-11-15T01:57:31.810ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:38791) = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
1912024-11-15T01:57:31.810ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:54824) = io task client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
1922024-11-15T01:57:31.810ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
1932024-11-15T01:57:31.810ZINFOcrucible: [0] Transition from New to WaitActive = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
1942024-11-15T01:57:31.810ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
1952024-11-15T01:57:31.810ZINFOcrucible: [1] Transition from New to WaitActive = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
1962024-11-15T01:57:31.810ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
1972024-11-15T01:57:31.810ZINFOcrucible: [2] Transition from New to WaitActive = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
1982024-11-15T01:57:31.811ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
1992024-11-15T01:57:31.811ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
2002024-11-15T01:57:31.811ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
2012024-11-15T01:57:31.812ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
2022024-11-15T01:57:31.812ZINFOcrucible: [2] Transition from WaitActive to WaitQuorum = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
2032024-11-15T01:57:31.812ZINFOcrucible: Waiting for more clients to be ready session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
2042024-11-15T01:57:31.812ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
2052024-11-15T01:57:31.812ZINFOcrucible: [1] Transition from WaitActive to WaitQuorum = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
2062024-11-15T01:57:31.812ZINFOcrucible: Waiting for more clients to be ready session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
2072024-11-15T01:57:31.812ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
2082024-11-15T01:57:31.812ZINFOcrucible: [0] Transition from WaitActive to WaitQuorum = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
2092024-11-15T01:57:31.812ZINFOcrucible: [0]R flush_numbers[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
2102024-11-15T01:57:31.812ZINFOcrucible: [0]R generation[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
2112024-11-15T01:57:31.812ZINFOcrucible: [0]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
2122024-11-15T01:57:31.812ZINFOcrucible: [1]R flush_numbers[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
2132024-11-15T01:57:31.812ZINFOcrucible: [1]R generation[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
2142024-11-15T01:57:31.812ZINFOcrucible: [1]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
2152024-11-15T01:57:31.812ZINFOcrucible: [2]R flush_numbers[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
2162024-11-15T01:57:31.812ZINFOcrucible: [2]R generation[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
2172024-11-15T01:57:31.812ZINFOcrucible: [2]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
2182024-11-15T01:57:31.812ZINFOcrucible: Max found gen is 2 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
219 Activate successful
220 {"msg":"Wait for a query_work_queue command to finish before sending IO
221 Generation requested: 2 >= found:2","v":0,"name":"crucible","level":30,"time":"2024-11-15T01:57:31.812396842Z","hostname":"w-01JCPRHT5F2MPA44KVJHM3S2PB","pid":1205,"":"downstairs","":"downstairs","session_id":"971dcb2e-6891-4ecf-b367-1e04e52065ee"}
2222024-11-15T01:57:31.812ZINFOcrucible: Next flush: 2 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
2232024-11-15T01:57:31.812ZINFOcrucible: All extents match = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
2242024-11-15T01:57:31.812ZINFOcrucible: No downstairs reconciliation required session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
2252024-11-15T01:57:31.812ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
226 {"msg":"Transition from WaitQuorum to Active","v":0,"name":"crucible","level":30,"time":Region: sv:1 bs:4096 ts:20480000 tb:5000 max_io:256 or 1048576
227 "2024-11-15T01:57:31.812537724Z","hostname":"w-01JCPRHT5F2MPA44KVJHM3S2PB","pid":1205,"client":"1","":"downstairs","session_id":"971dcb2e-6891-4ecf-b367-1e04e52065ee"}
2282024-11-15T01:57:31.812ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
2292024-11-15T01:57:31.812ZINFOcrucible: All required reconciliation work is completed session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
2302024-11-15T01:57:31.812ZINFOcrucible: Set Downstairs and Upstairs active after reconciliation session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
2312024-11-15T01:57:31.812ZINFOcrucible: ce9afbf0-8fb7-43ec-bd70-9dd45ce26334 is now active with session: 971dcb2e-6891-4ecf-b367-1e04e52065ee session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
2322024-11-15T01:57:31.812ZINFOcrucible: Set Active after no reconciliation session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
2332024-11-15T01:57:31.812ZINFOcrucible: The guest has finished waiting for activation with:2
234 Loading write count information from file "/tmp/test_replay_verify.log"
235 Nov 15 01:57:31.845 INFO accepted connection, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
236 Main task has work to do, go find it
237 got dsc Stop(2)
238 stop 2
239 Nov 15 01:57:31.845 INFO request completed, latency_us: 197, response_code: 204, uri: /stop/cid/2, method:[8830] Got stop action so:false kr:false
240 POST, req_id: e547d559-5881-46ea-b821-b611b249ab32, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
2412024-11-15T01:57:31.846ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
242 Nov 15 01:57:31.846 INFO request completed, latency_us: 127, response_code: 200, uri: /state/cid/2, method: GET, req_id: 51ea8272-7ce6-473c-8228-4727c29bd882, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
2432024-11-15T01:57:31.846ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
2442024-11-15T01:57:31.846ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
2452024-11-15T01:57:31.846ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
2462024-11-15T01:57:31.846ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
2472024-11-15T01:57:31.846ZINFOcrucible: client task is exiting = io task client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
248 [8830] Exited with: ExitStatus(unix_wait_status(9))
249 [8830][2] reports Exit
250 Nov 15 01:57:35.849 INFO request completed, latency_us: 321, response_code: 200, uri: /state/cid/2, method: GET, req_id: 3667f054-4a2e-48b9-8bb1-a05c99124e13, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
251 001/300 Read block 3133 len 12288
252 002/300 Write block 1923 len 28672 data: 2 2 2 2 2 2 2
253 003/300 Write block 2223 len 20480 data: 2 2 2 2 2
254 004/300 Read block 4322 len 36864
255 005/300 Read block 3508 len 40960
256 006/300 Write block 2024 len 16384 data: 2 2 2 2
257 007/300 Read block 1920 len 28672
258 008/300 Read block 1418 len 40960
259 009/300 Write block 1098 len 28672 data: 2 2 2 2 2 2 2
260 010/300 Write block 841 len 24576 data: 2 2 2 2 2 2
261 011/300 Read block 1841 len 28672
262 012/300 Write block 2389 len 4096 data: 2
263 013/300 Read block 2349 len 4096
264 014/300 Read block 2635 len 16384
265 015/300 Write block 229 len 32768 data: 2 2 2 2 2 2 2 2
266 016/300 Read block 2209 len 40960
267 017/300 Read block 3955 len 16384
268 018/300 Write block 2576 len 12288 data: 2 2 2
269 019/300 Read block 4553 len 40960
270 020/300 Read block 936 len 20480
271 021/300 Read block 2865 len 40960
272 022/300 Read block 1089 len 28672
273 023/300 Read block 2189 len 24576
274 024/300 Read block 3872 len 36864
275 025/300 Read block 3175 len 40960
276 026/300 Write block 1681 len 24576 data: 2 2 2 2 2 2
277 027/300 Read block 2549 len 12288
278 028/300 Write block 1290 len 8192 data: 2 2
279 029/300 Write block 816 len 8192 data: 2 2
280 030/300 Write block 4063 len 36864 data: 2 2 2 2 2 2 2 2 2
281 031/300 Read block 2583 len 24576
282 032/300 Read block 500 len 24576
283 033/300 Read block 4037 len 24576
284 034/300 Write block 2555 len 4096 data: 2
285 035/300 Read block 1524 len 20480
286 036/300 Read block 437 len 28672
287 037/300 Write block 920 len 24576 data: 2 2 2 2 2 2
288 038/300 Read block 3105 len 36864
289 039/300 Flush
290 040/300 Read block 2158 len 28672
291 041/300 Read block 4336 len 28672
292 042/300 Read block 730 len 8192
293 043/300 Flush
294 044/300 Write block 3697 len 16384 data: 2 2 2 2
295 045/300 Read block 3975 len 8192
296 046/300 Write block 2246 len 16384 data: 2 2 2 2
297 047/300 Write block 4206 len 20480 data: 2 2 2 2 2
298 048/300 Read block 3683 len 36864
299 049/300 Write block 3051 len 28672 data: 2 2 2 2 2 2 2
300 050/300 Write block 2236 len 40960 data: 2 2 2 2 2 2 2 2 2 2
301 051/300 Write block 991 len 16384 data: 2 2 2 2
302 052/300 Write block 1932 len 12288 data: 2 2 2
303 053/300 Write block 3080 len 28672 data: 2 2 2 2 2 2 2
304 054/300 Read block 986 len 16384
305 055/300 Read block 3865 len 20480
306 056/300 Flush
307 057/300 Write block 2650 len 8192 data: 2 2
308 058/300 Write block 753 len 28672 data: 2 2 2 2 2 2 2
309 059/300 Read block 3518 len 16384
310 060/300 Write block 3077 len 40960 data: 2 2 2 3 3 3 3 3 3 3
311 061/300 Read block 531 len 36864
312 062/300 Flush
313 063/300 Read block 1619 len 28672
314 064/300 Write block 2284 len 16384 data: 2 2 2 2
315 065/300 Write block 667 len 28672 data: 2 2 2 2 2 2 2
316 066/300 Write block 598 len 36864 data: 2 2 2 2 2 2 2 2 2
317 067/300 Read block 1297 len 40960
318 068/300 Read block 3139 len 16384
319 069/300 Write block 1811 len 16384 data: 2 2 2 2
320 070/300 Write block 3875 len 8192 data: 2 2
321 071/300 Read block 2066 len 40960
322 072/300 Read block 4959 len 8192
323 073/300 Write block 4525 len 16384 data: 2 2 2 2
324 074/300 Read block 2178 len 28672
325 075/300 Read block 2239 len 8192
326 076/300 Read block 3738 len 32768
327 077/300 Flush
328 078/300 Read block 1145 len 28672
329 079/300 Read block 4034 len 16384
330 080/300 Write block 4738 len 24576 data: 2 2 2 2 2 2
331 081/300 Write block 3836 len 32768 data: 2 2 2 2 2 2 2 2
332 082/300 Read block 4256 len 24576
333 083/300 Write block 2778 len 40960 data: 2 2 2 2 2 2 2 2 2 2
334 084/300 Write block 1667 len 28672 data: 2 2 2 2 2 2 2
335 085/300 Write block 3647 len 20480 data: 2 2 2 2 2
336 086/300 Write block 270 len 40960 data: 2 2 2 2 2 2 2 2 2 2
337 087/300 Read block 4489 len 24576
338 088/300 Write block 3750 len 36864 data: 2 2 2 2 2 2 2 2 2
339 089/300 Read block 2662 len 16384
340 090/300 Read block 3130 len 32768
341 091/300 Write block 4622 len 12288 data: 2 2 2
342 092/300 Write block 1802 len 24576 data: 2 2 2 2 2 2
343 093/300 Flush
344 094/300 Flush
345 095/300 Read block 3060 len 8192
346 096/300 Read block 3548 len 8192
347 097/300 Write block 1687 len 28672 data: 2 2 2 2 2 2 2
348 098/300 Write block 64 len 24576 data: 2 2 2 2 2 2
349 099/300 Read block 3059 len 8192
350 100/300 Read block 1364 len 36864
351 101/300 Write block 501 len 40960 data: 2 2 2 2 2 2 2 2 2 2
352 102/300 Write block 333 len 4096 data: 2
353 103/300 Write block 1660 len 8192 data: 2 2
354 104/300 Write block 2886 len 20480 data: 2 2 2 2 2
355 105/300 Read block 908 len 28672
356 106/300 Read block 4681 len 40960
357 107/300 Write block 4142 len 8192 data: 2 2
358 108/300 Flush
359 109/300 Read block 2482 len 12288
360 110/300 Write block 3954 len 32768 data: 2 2 2 2 2 2 2 2
361 111/300 Write block 3751 len 4096 data: 3
362 112/300 Write block 2365 len 20480 data: 2 2 2 2 2
363 113/300 Write block 783 len 36864 data: 2 2 2 2 2 2 2 2 2
364 114/300 Read block 915 len 28672
365 115/300 Read block 4320 len 40960
366 116/300 Write block 522 len 40960 data: 2 2 2 2 2 2 2 2 2 2
367 117/300 Read block 3222 len 24576
368 118/300 Write block 3480 len 16384 data: 2 2 2 2
369 119/300 Read block 2281 len 4096
370 120/300 Read block 2978 len 28672
371 121/300 Read block 4823 len 8192
372 122/300 Write block 2212 len 40960 data: 2 2 2 2 2 2 2 2 2 2
373 123/300 Read block 168 len 16384
374 124/300 Write block 2941 len 24576 data: 2 2 2 2 2 2
375 125/300 Read block 3649 len 40960
376 126/300 Read block 864 len 4096
377 127/300 Write block 1973 len 12288 data: 2 2 2
378 128/300 Write block 3738 len 20480 data: 2 2 2 2 2
379 129/300 Read block 3496 len 8192
380 130/300 Write block 4046 len 8192 data: 2 2
381 131/300 Read block 15 len 24576
382 132/300 Read block 4513 len 16384
383 133/300 Write block 1017 len 20480 data: 2 2 2 2 2
384 134/300 Read block 1587 len 4096
385 135/300 Flush
386 136/300 Write block 1065 len 20480 data: 2 2 2 2 2
387 137/300 Read block 1793 len 8192
388 138/300 Read block 369 len 12288
389 139/300 Write block 1903 len 12288 data: 2 2 2
390 140/300 Write block 4419 len 32768 data: 2 2 2 2 2 2 2 2
391 141/300 Read block 3557 len 12288
392 142/300 Read block 3226 len 32768
393 143/300 Write block 1853 len 40960 data: 2 2 2 2 2 2 2 2 2 2
394 144/300 Write block 1483 len 40960 data: 2 2 2 2 2 2 2 2 2 2
395 145/300 Read block 2719 len 20480
396 146/300 Write block 2836 len 8192 data: 2 2
397 147/300 Write block 1539 len 12288 data: 2 2 2
398 148/300 Read block 329 len 16384
399 149/300 Write block 996 len 32768 data: 2 2 2 2 2 2 2 2
400 150/300 Write block 4643 len 4096 data: 2
401 151/300 Write block 4705 len 28672 data: 2 2 2 2 2 2 2
402 152/300 Write block 2609 len 40960 data: 2 2 2 2 2 2 2 2 2 2
403 153/300 Write block 3698 len 32768 data: 3 3 3 2 2 2 2 2
404 154/300 Read block 4439 len 20480
405 155/300 Write block 2148 len 32768 data: 2 2 2 2 2 2 2 2
406 156/300 Write block 405 len 40960 data: 2 2 2 2 2 2 2 2 2 2
407 157/300 Read block 1131 len 12288
408 158/300 Write block 2992 len 4096 data: 2
409 159/300 Flush
410 160/300 Read block 3596 len 32768
411 161/300 Read block 86 len 16384
412 162/300 Read block 4642 len 12288
413 163/300 Write block 3799 len 4096 data: 2
414 164/300 Flush
415 165/300 Write block 2149 len 16384 data: 3 3 3 3
416 166/300 Write block 2770 len 28672 data: 2 2 2 2 2 2 2
417 167/300 Read block 2757 len 16384
418 168/300 Read block 1007 len 20480
419 169/300 Flush
420 170/300 Write block 381 len 36864 data: 2 2 2 2 2 2 2 2 2
421 171/300 Flush
422 172/300 Read block 177 len 4096
423 173/300 Read block 3923 len 28672
424 174/300 Write block 2273 len 8192 data: 2 2
425 175/300 Read block 982 len 32768
426 176/300 Write block 3646 len 4096 data: 2
427 177/300 Write block 3735 len 24576 data: 2 2 2 3 3 3
428 178/300 Read block 4132 len 8192
429 179/300 Flush
430 180/300 Write block 1800 len 28672 data: 2 2 3 3 3 3 3
431 181/300 Flush
432 182/300 Write block 678 len 4096 data: 2
433 183/300 Write block 1869 len 20480 data: 2 2 2 2 2
434 184/300 Read block 1335 len 8192
435 185/300 Write block 497 len 36864 data: 2 2 2 2 3 3 3 3 3
436 186/300 Read block 836 len 40960
437 187/300 Write block 388 len 8192 data: 3 3
438 188/300 Read block 2442 len 32768
439 189/300 Write block 757 len 24576 data: 3 3 3 2 2 2
440 190/300 Flush
441 191/300 Write block 2587 len 24576 data: 2 2 2 2 2 2
442 192/300 Read block 4168 len 28672
443 193/300 Write block 4528 len 12288 data: 3 2 2
444 194/300 Read block 367 len 28672
445 195/300 Read block 4491 len 4096
446 196/300 Write block 4894 len 4096 data: 2
447 197/300 Write block 1121 len 36864 data: 2 2 2 2 2 2 2 2 2
448 198/300 Write block 2117 len 4096 data: 2
449 199/300 Read block 249 len 24576
450 200/300 Read block 3087 len 28672
451 201/300 Flush
452 202/300 Write block 3704 len 24576 data: 3 3 2 2 2 2
453 203/300 Read block 813 len 4096
454 204/300 Write block 4943 len 36864 data: 2 2 2 2 2 2 2 2 2
455 205/300 Write block 2129 len 40960 data: 2 2 2 2 2 2 2 2 2 2
456 206/300 Write block 3521 len 24576 data: 2 2 2 2 2 2
457 207/300 Read block 715 len 16384
458 208/300 Read block 2347 len 16384
459 209/300 Flush
460 210/300 Read block 1754 len 12288
461 211/300 Write block 3588 len 36864 data: 2 2 2 2 2 2 2 2 2
462 212/300 Read block 3625 len 36864
463 213/300 Read block 3678 len 24576
464 214/300 Read block 53 len 40960
465 215/300 Read block 1710 len 8192
466 216/300 Write block 1010 len 28672 data: 2 2 2 2 2 2 2
467 217/300 Write block 1910 len 12288 data: 2 2 2
468 218/300 Read block 4841 len 12288
469 219/300 Read block 3797 len 8192
470 220/300 Write block 3075 len 24576 data: 2 2 3 3 3 4
471 221/300 Write block 4777 len 8192 data: 2 2
472 222/300 Write block 4514 len 40960 data: 2 2 2 2 2 2 2 2 2 2
473 223/300 Read block 4540 len 24576
474 224/300 Read block 2361 len 36864
475 225/300 Read block 391 len 16384
476 226/300 Read block 1426 len 32768
477 227/300 Read block 3756 len 4096
478 228/300 Read block 86 len 4096
479 229/300 Read block 508 len 16384
480 230/300 Write block 1455 len 28672 data: 2 2 2 2 2 2 2
481 231/300 Write block 4769 len 32768 data: 2 2 2 2 2 2 2 2
482 232/300 Read block 4986 len 16384
483 233/300 Write block 1146 len 32768 data: 2 2 2 2 2 2 2 2
484 234/300 Write block 1766 len 16384 data: 2 2 2 2
485 235/300 Write block 3022 len 12288 data: 2 2 2
486 236/300 Read block 3626 len 40960
487 237/300 Read block 886 len 20480
488 238/300 Write block 1294 len 40960 data: 2 2 2 2 2 2 2 2 2 2
489 239/300 Read block 1417 len 16384
490 240/300 Read block 379 len 20480
491 241/300 Read block 4706 len 12288
492 242/300 Write block 1020 len 8192 data: 3 3
493 243/300 Read block 901 len 12288
494 244/300 Write block 2561 len 28672 data: 2 2 2 2 2 2 2
495 245/300 Write block 1559 len 28672 data: 2 2 2 2 2 2 2
496 246/300 Write block 4599 len 8192 data: 2 2
497 247/300 Read block 157 len 32768
498 248/300 Read block 3340 len 36864
499 249/300 Flush
500 250/300 Write block 4441 len 32768 data: 2 2 2 2 2 2 2 2
501 251/300 Read block 1242 len 40960
502 252/300 Write block 301 len 8192 data: 2 2
503 253/300 Write block 599 len 20480 data: 3 3 3 3 3
504 254/300 Flush
505 255/300 Read block 3691 len 28672
506 256/300 Read block 4515 len 32768
507 257/300 Write block 1938 len 24576 data: 2 2 2 2 2 2
508 258/300 Read block 678 len 12288
509 259/300 Write block 3400 len 40960 data: 2 2 2 2 2 2 2 2 2 2
510 260/300 Read block 197 len 28672
511 261/300 Write block 2074 len 32768 data: 2 2 2 2 2 2 2 2
512 262/300 Flush
513 263/300 Write block 4 len 4096 data: 2
514 264/300 Read block 2473 len 24576
515 265/300 Write block 3261 len 4096 data: 2
516 266/300 Read block 845 len 4096
517 267/300 Write block 476 len 40960 data: 2 2 2 2 2 2 2 2 2 2
518 268/300 Read block 2448 len 40960
519 269/300 Write block 4893 len 8192 data: 2 3
520 270/300 Read block 4905 len 12288
521 271/300 Write block 4955 len 8192 data: 2 2
522 272/300 Write block 4339 len 12288 data: 2 2 2
523 273/300 Read block 1548 len 24576
524 274/300 Write block 2767 len 12288 data: 2 2 2
525 275/300 Read block 3056 len 36864
526 276/300 Write block 4508 len 36864 data: 2 2 2 2 2 2 3 3 3
527 277/300 Write block 3819 len 40960 data: 2 2 2 2 2 2 2 2 2 2
528 278/300 Read block 3467 len 12288
529 279/300 Flush
530 280/300 Flush
531 281/300 Write block 2576 len 4096 data: 3
532 282/300 Write block 3555 len 28672 data: 2 2 2 2 2 2 2
533 283/300 Write block 1303 len 8192 data: 3 2
534 284/300 Read block 3527 len 4096
535 285/300 Write block 1926 len 4096 data: 3
536 286/300 Write block 2220 len 24576 data: 3 3 2 3 3 3
537 287/300 Write block 1982 len 40960 data: 2 2 2 2 2 2 2 2 2 2
538 288/300 Read block 866 len 36864
539 289/300 Read block 542 len 36864
540 290/300 Write block 2186 len 32768 data: 2 2 2 2 2 2 2 2
541 291/300 Read block 4521 len 28672
542 292/300 Read block 3305 len 32768
543 293/300 Read block 2942 len 36864
544 294/300 Read block 4073 len 16384
545 295/300 Read block 115 len 8192
546 296/300 Write block 1698 len 36864 data: 2 2 2 2 2 2 2 2 2
547 297/300 Read block 3175 len 40960
548 298/300 Read block 3003 len 24576
549 299/300 Flush
550 300/300 Read block 3541 len 4096
551 301/300 Read block 3946 len 20480
552 Main task has work to do, go find it
553 got dsc Start(2)
554 start 2
555 Nov 15 01:57:36.084 INFO request completed, latency_us: 243, response_code: 204, uri: /start/cid/2, method: POST, req_id: bb351916-2b60-4d1c-a847-5dfa40ba4d9b, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
556 Replay: started 2, returned:Ok(())
557 ----------------------------------------------------------------
558 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
559 States: Active Active Offline
560 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
561 1000 Acked Read 3 Done Done Sent false
562 1001 Acked Write 7 Done Done Sent false
563 1002 Acked Write 5 Done Done Sent false
564 1003 Acked Read 9 Done Done Sent false
565 1004 Acked Read 10 Done Done Sent false
566 1005 Acked Write 4 Done Done Sent false
567 1006 Acked Read 7 Done Done Sent false
568 1007 Acked Read 10 Done Done Sent false
569 1008 Acked Write 7 Done Done Sent false
570 1009 Acked Write 6 Done Done Sent false
571 1010 Acked Read 7 Done Done Sent false
572 1011 Acked Write 1 Done Done Sent false
573 1012 Acked Read 1 Done Done Sent false
574 1013 Acked Read 4 Done Done Sent false
575 1014 Acked Write 8 Done Done Sent false
576 1015 Acked Read 10 Done Done Sent false
577 1016 Acked Read 4 Done Done Sent false
578 1017 Acked Write 3 Done Done Sent false
579 1018 Acked Read 10 Done Done Sent false
580 1019 Acked Read 5 Done Done Sent false
581 1020 Acked Read 10 Done Done Sent false
582 1021 Acked Read 7 Done Done Sent false
583 1022 Acked Read 6 Done Done Sent false
584 1023 Acked Read 9 Done Done Sent false
585 1024 Acked Read 10 Done Done Sent false
586 1025 Acked Write 6 Done Done Sent false
587 1026 Acked Read 3 Done Done Sent false
588 1027 Acked Write 2 Done Done Sent false
589 1028 Acked Write 2 Done Done Sent false
590 1029 Acked Write 9 Done Done Sent false
591 1030 Acked Read 6 Done Done Sent false
592 1031 Acked Read 6 Done Done Sent false
593 1032 Acked Read 6 Done Done Sent false
594 1033 Acked Write 1 Done Done Sent false
595 1034 Acked Read 5 Done Done Sent false
596 1035 Acked Read 7 Done Done Sent false
597 1036 Acked Write 6 Done Done Sent false
598 1037 Acked Read 9 Done Done Sent false
599 1038 Acked Flush 0 Done Done Sent false
600 1039 Acked Read 7 Done Done Sent false
601 1040 Acked Read 7 Done Done Sent false
602 1041 Acked Read 2 Done Done Sent false
603 1042 Acked Flush 0 Done Done Sent false
604 1043 Acked Write 4 Done Done Sent false
605 1044 Acked Read 2 Done Done Sent false
606 1045 Acked Write 4 Done Done Sent false
607 1046 Acked Write 5 Done Done Sent false
608 1047 Acked Read 9 Done Done Sent false
609 1048 Acked Write 7 Done Done Sent false
610 1049 Acked Write 10 Done Done Sent false
611 1050 Acked Write 4 Done Done Sent false
612 1051 Acked Write 3 Done Done Sent false
613 1052 Acked Write 7 Done Done Sent false
614 1053 Acked Read 4 Done Done Sent false
615 1054 Acked Read 5 Done Done Sent false
616 1055 Acked Flush 0 Done Done Sent false
617 1056 Acked Write 2 Done Done Sent false
618 1057 Acked Write 7 Done Done Sent false
619 1058 Acked Read 4 Done Done Sent false
620 1059 Acked Write 10 Done Done Sent false
621 1060 Acked Read 9 Done Done Sent false
622 1061 Acked Flush 0 Done Done Sent false
623 1062 Acked Read 7 Done Done Sent false
624 1063 Acked Write 4 Done Done Sent false
625 1064 Acked Write 7 Done Done Sent false
626 1065 Acked Write 9 Done Done Sent false
627 1066 Acked Read 10 Done Done Sent false
628 1067 Acked Read 4 Done Done Sent false
629 1068 Acked Write 4 Done Done Sent false
630 1069 Acked Write 2 Done Done Sent false
631 1070 Acked Read 10 Done Done Sent false
632 1071 Acked Read 2 Done Done Sent false
633 1072 Acked Write 4 Done Done Sent false
634 1073 Acked Read 7 Done Done Sent false
635 1074 Acked Read 2 Done Done Sent false
636 1075 Acked Read 8 Done Done Sent false
637 1076 Acked Flush 0 Done Done Sent false
638 1077 Acked Read 7 Done Done Sent false
639 1078 Acked Read 4 Done Done Sent false
640 1079 Acked Write 6 Done Done Sent false
641 1080 Acked Write 8 Done Done Sent false
642 1081 Acked Read 6 Done Done Sent false
643 1082 Acked Write 10 Done Done Sent false
644 1083 Acked Write 7 Done Done Sent false
645 1084 Acked Write 5 Done Done Sent false
646 1085 Acked Write 10 Done Done Sent false
647 1086 Acked Read 6 Done Done Sent false
648 1087 Acked Write 9 Done Done Sent false
649 1088 Acked Read 4 Done Done Sent false
650 1089 Acked Read 8 Done Done Sent false
651 1090 Acked Write 3 Done Done Sent false
652 1091 Acked Write 6 Done Done Sent false
653 1092 Acked Flush 0 Done Done Sent false
654 1093 Acked Flush 0 Done Done Sent false
655 1094 Acked Read 2 Done Done Sent false
656 1095 Acked Read 2 Done Done Sent false
657 1096 Acked Write 7 Done Done Sent false
658 1097 Acked Write 6 Done Done Sent false
659 1098 Acked Read 2 Done Done Sent false
660 1099 Acked Read 9 Done Done Sent false
661 1100 Acked Write 10 Done Done Sent false
662 1101 Acked Write 1 Done Done Sent false
663 1102 Acked Write 2 Done Done Sent false
664 1103 Acked Write 5 Done Done Sent false
665 1104 Acked Read 7 Done Done Sent false
666 1105 Acked Read 10 Done Done Sent false
667 1106 Acked Write 2 Done Done Sent false
668 1107 Acked Flush 0 Done Done Sent false
669 1108 Acked Read 3 Done Done Sent false
670 1109 Acked Write 8 Done Done Sent false
671 1110 Acked Write 1 Done Done Sent false
672 1111 Acked Write 5 Done Done Sent false
673 1112 Acked Write 9 Done Done Sent false
674 1113 Acked Read 7 Done Done Sent false
675 1114 Acked Read 10 Done Done Sent false
676 1115 Acked Write 10 Done Done Sent false
677 1116 Acked Read 6 Done Done Sent false
678 1117 Acked Write 4 Done Done Sent false
679 1118 Acked Read 1 Done Done Sent false
680 1119 Acked Read 7 Done Done Sent false
681 1120 Acked Read 2 Done Done Sent false
682 1121 Acked Write 10 Done Done Sent false
683 1122 Acked Read 4 Done Done Sent false
684 1123 Acked Write 6 Done Done Sent false
685 1124 Acked Read 10 Done Done Sent false
686 1125 Acked Read 1 Done Done Sent false
687 1126 Acked Write 3 Done Done Sent false
688 1127 Acked Write 5 Done Done Sent false
689 1128 Acked Read 2 Done Done Sent false
690 1129 Acked Write 2 Done Done Sent false
691 1130 Acked Read 6 Done Done Sent false
692 1131 Acked Read 4 Done Done Sent false
693 1132 Acked Write 5 Done Done Sent false
694 1133 Acked Read 1 Done Done Sent false
695 1134 Acked Flush 0 Done Done Sent false
696 1135 Acked Write 5 Done Done Sent false
697 1136 Acked Read 2 Done Done Sent false
698 1137 Acked Read 3 Done Done Sent false
699 1138 Acked Write 3 Done Done Sent false
700 1139 Acked Write 8 Done Done Sent false
701 1140 Acked Read 3 Done Done Sent false
702 1141 Acked Read 8 Done Done Sent false
703 1142 Acked Write 10 Done Done Sent false
704 1143 Acked Write 10 Done Done Sent false
705 1144 Acked Read 5 Done Done Sent false
706 1145 Acked Write 2 Done Done Sent false
707 1146 Acked Write 3 Done Done Sent false
708 1147 Acked Read 4 Done Done Sent false
709 1148 Acked Write 8 Done Done Sent false
710 1149 Acked Write 1 Done Done Sent false
711 1150 Acked Write 7 Done Done Sent false
712 1151 Acked Write 10 Done Done Sent false
713 1152 Acked Write 8 Done Done Sent false
714 1153 Acked Read 5 Done Done Sent false
715 1154 Acked Write 8 Done Done Sent false
716 1155 Acked Write 10 Done Done Sent false
717 1156 Acked Read 3 Done Done Sent false
718 1157 Acked Write 1 Done Done Sent false
719 1158 Acked Flush 0 Done Done Sent false
720 1159 Acked Read 8 Done Done Sent false
721 1160 Acked Read 4 Done Done Sent false
722 1161 Acked Read 3 Done Done Sent false
723 1162 Acked Write 1 Done Done Sent false
724 1163 Acked Flush 0 Done Done Sent false
725 1164 Acked Write 4 Done Done Sent false
726 1165 Acked Write 7 Done Done Sent false
727 1166 Acked Read 4 Done Done Sent false
728 1167 Acked Read 5 Done Done Sent false
729 1168 Acked Flush 0 Done Done Sent false
730 1169 Acked Write 9 Done Done Sent false
731 1170 Acked Flush 0 Done Done Sent false
732 1171 Acked Read 1 Done Done Sent false
733 1172 Acked Read 7 Done Done Sent false
734 1173 Acked Write 2 Done Done Sent false
735 1174 Acked Read 8 Done Done Sent false
736 1175 Acked Write 1 Done Done Sent false
737 1176 Acked Write 6 Done Done Sent false
738 1177 Acked Read 2 Done Done Sent false
739 1178 Acked Flush 0 Done Done Sent false
740 1179 Acked Write 7 Done Done Sent false
741 1180 Acked Flush 0 Done Done Sent false
742 1181 Acked Write 1 Done Done Sent false
743 1182 Acked Write 5 Done Done Sent false
744 1183 Acked Read 2 Done Done Sent false
745 1184 Acked Write 9 Done Done Sent false
746 1185 Acked Read 10 Done Done Sent false
747 1186 Acked Write 2 Done Done Sent false
748 1187 Acked Read 8 Done Done Sent false
749 1188 Acked Write 6 Done Done Sent false
750 1189 Acked Flush 0 Done Done Sent false
751 1190 Acked Write 6 Done Done Sent false
752 1191 Acked Read 7 Done Done Sent false
753 1192 Acked Write 3 Done Done Sent false
754 1193 Acked Read 7 Done Done Sent false
755 1194 Acked Read 1 Done Done Sent false
756 1195 Acked Write 1 Done Done Sent false
757 1196 Acked Write 9 Done Done Sent false
758 1197 Acked Write 1 Done Done Sent false
759 1198 Acked Read 6 Done Done Sent false
760 1199 Acked Read 7 Done Done Sent false
761 1200 Acked Flush 0 Done Done Sent false
762 1201 Acked Write 6 Done Done Sent false
763 1202 Acked Read 1 Done Done Sent false
764 1203 Acked Write 9 Done Done Sent false
765 1204 Acked Write 10 Done Done Sent false
766 1205 Acked Write 6 Done Done Sent false
767 1206 Acked Read 4 Done Done Sent false
768 1207 Acked Read 4 Done Done Sent false
769 1208 Acked Flush 0 Done Done Sent false
770 1209 Acked Read 3 Done Done Sent false
771 1210 Acked Write 9 Done Done Sent false
772 1211 Acked Read 9 Done Done Sent false
773 1212 Acked Read 6 Done Done Sent false
774 1213 Acked Read 10 Done Done Sent false
775 1214 Acked Read 2 Done Done Sent false
776 1215 Acked Write 7 Done Done Sent false
777 1216 Acked Write 3 Done Done Sent false
778 1217 Acked Read 3 Done Done Sent false
779 1218 Acked Read 2 Done Done Sent false
780 1219 Acked Write 6 Done Done Sent false
781 1220 Acked Write 2 Done Done Sent false
782 1221 Acked Write 10 Done Done Sent false
783 1222 Acked Read 6 Done Done Sent false
784 1223 Acked Read 9 Done Done Sent false
785 1224 Acked Read 4 Done Done Sent false
786 1225 Acked Read 8 Done Done Sent false
787 1226 Acked Read 1 Done Done Sent false
788 1227 Acked Read 1 Done Done Sent false
789 1228 Acked Read 4 Done Done Sent false
790 1229 Acked Write 7 Done Done Sent false
791 1230 Acked Write 8 Done Done Sent false
792 1231 Acked Read 4 Done Done Sent false
793 1232 Acked Write 8 Done Done Sent false
794 1233 Acked Write 4 Done Done Sent false
795 1234 Acked Write 3 Done Done Sent false
796 1235 Acked Read 10 Done Done Sent false
797 1236 Acked Read 5 Done Done Sent false
798 1237 Acked Write 10 Done Done Sent false
799 1238 Acked Read 4 Done Done Sent false
800 1239 Acked Read 5 Done Done Sent false
801 1240 Acked Read 3 Done Done Sent false
802 1241 Acked Write 2 Done Done Sent false
803 1242 Acked Read 3 Done Done Sent false
804 1243 Acked Write 7 Done Done Sent false
805 1244 Acked Write 7 Done Done Sent false
806 1245 Acked Write 2 Done Done Sent false
807 1246 Acked Read 8 Done Done Sent false
808 1247 Acked Read 9 Done Done Sent false
809 1248 Acked Flush 0 Done Done Sent false
810 1249 Acked Write 8 Done Done Sent false
811 1250 Acked Read 10 Done Done Sent false
812 1251 Acked Write 2 Done Done Sent false
813 1252 Acked Write 5 Done Done Sent false
814 1253 Acked Flush 0 Done Done Sent false
815 1254 Acked Read 7 Done Done Sent false
816 1255 Acked Read 8 Done Done Sent false
817 1256 Acked Write 6 Done Done Sent false
818 1257 Acked Read 3 Done Done Sent false
819 1258 Acked Write 10 Done Done Sent false
820 1259 Acked Read 7 Done Done Sent false
821 1260 Acked Write 8 Done Done Sent false
822 1261 Acked Flush 0 Done Done Sent false
823 1262 Acked Write 1 Done Done Sent false
824 1263 Acked Read 6 Done Done Sent false
825 1264 Acked Write 1 Done Done Sent false
826 1265 Acked Read 1 Done Done Sent false
827 1266 Acked Write 10 Done Done Sent false
828 1267 Acked Read 10 Done Done Sent false
829 1268 Acked Write 2 Done Done Sent false
830 1269 Acked Read 3 Done Done Sent false
831 1270 Acked Write 2 Done Done Sent false
832 1271 Acked Write 3 Done Done Sent false
833 1272 Acked Read 6 Done Done Sent false
834 1273 Acked Write 3 Done Done Sent false
835 1274 Acked Read 9 Done Done Sent false
836 1275 Acked Write 9 Done Done Sent false
837 1276 Acked Write 10 Done Done Sent false
838 1277 Acked Read 3 Done Done Sent false
839 1278 Acked Flush 0 Done Done Sent false
840 1279 Acked Flush 0 Done Done Sent false
841 1280 Acked Write 1 Done Done Sent false
842 1281 Acked Write 7 Done Done Sent false
843 1282 Acked Write 2 Done Done Sent false
844 1283 Acked Read 1 Done Done Sent false
845 1284 Acked Write 1 Done Done Sent false
846 1285 Acked Write 6 Done Done Sent false
847 1286 Acked Write 10 Done Done Sent false
848 1287 Acked Read 9 Done Done Sent false
849 1288 Acked Read 9 Done Done Sent false
850 1289 Acked Write 8 Done Done Sent false
851 1290 Acked Read 7 Done Done Sent false
852 1291 Acked Read 8 Done Done Sent false
853 1292 Acked Read 9 Done Done Sent false
854 1293 Acked Read 4 Done Done Sent false
855 1294 Acked Read 2 Done Done Sent false
856 1295 Acked Write 9 Done Done Sent false
857 1296 Acked Read 10 Done Done Sent false
858 1297 Acked Read 6 Done Done Sent false
859 1298 Acked Flush 0 Done Done Sent false
860 1299 Acked Read 1 Done Done Sent false
861 1300 Acked Read 5 Done Done Sent false
862 STATES DS:0 DS:1 DS:2 TOTAL
863 Sent 0 0 301 301
864 Done 301 301 0 602
865 Skipped 0 0 0 0
866 Error 0 0 0 0
867 Last Flush: 1298 1298 0
868 Downstairs last five completed:
869 Upstairs last five completed: 1300 1299 1298 1297 1296
870 CLIENT: Up:0 ds:301 act:2
871 [8830] got start action
872 [8830] I am going to restart
873 Starting downstairs at port 8830
874 Make output file at "/tmp/dsc/downstairs-8830.txt"
875 [8830][2] reports Starting
876 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1206)
877 [8830][2] reports Running
878 ----------------------------------------------------------------
879 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
880 States: Active Active Offline
881 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
882 1000 Acked Read 3 Done Done Sent false
883 1001 Acked Write 7 Done Done Sent false
884 1002 Acked Write 5 Done Done Sent false
885 1003 Acked Read 9 Done Done Sent false
886 1004 Acked Read 10 Done Done Sent false
887 1005 Acked Write 4 Done Done Sent false
888 1006 Acked Read 7 Done Done Sent false
889 1007 Acked Read 10 Done Done Sent false
890 1008 Acked Write 7 Done Done Sent false
891 1009 Acked Write 6 Done Done Sent false
892 1010 Acked Read 7 Done Done Sent false
893 1011 Acked Write 1 Done Done Sent false
894 1012 Acked Read 1 Done Done Sent false
895 1013 Acked Read 4 Done Done Sent false
896 1014 Acked Write 8 Done Done Sent false
897 1015 Acked Read 10 Done Done Sent false
898 1016 Acked Read 4 Done Done Sent false
899 1017 Acked Write 3 Done Done Sent false
900 1018 Acked Read 10 Done Done Sent false
901 1019 Acked Read 5 Done Done Sent false
902 1020 Acked Read 10 Done Done Sent false
903 1021 Acked Read 7 Done Done Sent false
904 1022 Acked Read 6 Done Done Sent false
905 1023 Acked Read 9 Done Done Sent false
906 1024 Acked Read 10 Done Done Sent false
907 1025 Acked Write 6 Done Done Sent false
908 1026 Acked Read 3 Done Done Sent false
909 1027 Acked Write 2 Done Done Sent false
910 1028 Acked Write 2 Done Done Sent false
911 1029 Acked Write 9 Done Done Sent false
912 1030 Acked Read 6 Done Done Sent false
913 1031 Acked Read 6 Done Done Sent false
914 1032 Acked Read 6 Done Done Sent false
915 1033 Acked Write 1 Done Done Sent false
916 1034 Acked Read 5 Done Done Sent false
917 1035 Acked Read 7 Done Done Sent false
918 1036 Acked Write 6 Done Done Sent false
919 1037 Acked Read 9 Done Done Sent false
920 1038 Acked Flush 0 Done Done Sent false
921 1039 Acked Read 7 Done Done Sent false
922 1040 Acked Read 7 Done Done Sent false
923 1041 Acked Read 2 Done Done Sent false
924 1042 Acked Flush 0 Done Done Sent false
925 1043 Acked Write 4 Done Done Sent false
926 1044 Acked Read 2 Done Done Sent false
927 1045 Acked Write 4 Done Done Sent false
928 1046 Acked Write 5 Done Done Sent false
929 1047 Acked Read 9 Done Done Sent false
930 1048 Acked Write 7 Done Done Sent false
931 1049 Acked Write 10 Done Done Sent false
932 1050 Acked Write 4 Done Done Sent false
933 1051 Acked Write 3 Done Done Sent false
934 1052 Acked Write 7 Done Done Sent false
935 1053 Acked Read 4 Done Done Sent false
936 1054 Acked Read 5 Done Done Sent false
937 1055 Acked Flush 0 Done Done Sent false
938 1056 Acked Write 2 Done Done Sent false
939 1057 Acked Write 7 Done Done Sent false
940 1058 Acked Read 4 Done Done Sent false
941 1059 Acked Write 10 Done Done Sent false
942 1060 Acked Read 9 Done Done Sent false
943 1061 Acked Flush 0 Done Done Sent false
944 1062 Acked Read 7 Done Done Sent false
945 1063 Acked Write 4 Done Done Sent false
946 1064 Acked Write 7 Done Done Sent false
947 1065 Acked Write 9 Done Done Sent false
948 1066 Acked Read 10 Done Done Sent false
949 1067 Acked Read 4 Done Done Sent false
950 1068 Acked Write 4 Done Done Sent false
951 1069 Acked Write 2 Done Done Sent false
952 1070 Acked Read 10 Done Done Sent false
953 1071 Acked Read 2 Done Done Sent false
954 1072 Acked Write 4 Done Done Sent false
955 1073 Acked Read 7 Done Done Sent false
956 1074 Acked Read 2 Done Done Sent false
957 1075 Acked Read 8 Done Done Sent false
958 1076 Acked Flush 0 Done Done Sent false
959 1077 Acked Read 7 Done Done Sent false
960 1078 Acked Read 4 Done Done Sent false
961 1079 Acked Write 6 Done Done Sent false
962 1080 Acked Write 8 Done Done Sent false
963 1081 Acked Read 6 Done Done Sent false
964 1082 Acked Write 10 Done Done Sent false
965 1083 Acked Write 7 Done Done Sent false
966 1084 Acked Write 5 Done Done Sent false
967 1085 Acked Write 10 Done Done Sent false
968 1086 Acked Read 6 Done Done Sent false
969 1087 Acked Write 9 Done Done Sent false
970 1088 Acked Read 4 Done Done Sent false
971 1089 Acked Read 8 Done Done Sent false
972 1090 Acked Write 3 Done Done Sent false
973 1091 Acked Write 6 Done Done Sent false
974 1092 Acked Flush 0 Done Done Sent false
975 1093 Acked Flush 0 Done Done Sent false
976 1094 Acked Read 2 Done Done Sent false
977 1095 Acked Read 2 Done Done Sent false
978 1096 Acked Write 7 Done Done Sent false
979 1097 Acked Write 6 Done Done Sent false
980 1098 Acked Read 2 Done Done Sent false
981 1099 Acked Read 9 Done Done Sent false
982 1100 Acked Write 10 Done Done Sent false
983 1101 Acked Write 1 Done Done Sent false
984 1102 Acked Write 2 Done Done Sent false
985 1103 Acked Write 5 Done Done Sent false
986 1104 Acked Read 7 Done Done Sent false
987 1105 Acked Read 10 Done Done Sent false
988 1106 Acked Write 2 Done Done Sent false
989 1107 Acked Flush 0 Done Done Sent false
990 1108 Acked Read 3 Done Done Sent false
991 1109 Acked Write 8 Done Done Sent false
992 1110 Acked Write 1 Done Done Sent false
993 1111 Acked Write 5 Done Done Sent false
994 1112 Acked Write 9 Done Done Sent false
995 1113 Acked Read 7 Done Done Sent false
996 1114 Acked Read 10 Done Done Sent false
997 1115 Acked Write 10 Done Done Sent false
998 1116 Acked Read 6 Done Done Sent false
999 1117 Acked Write 4 Done Done Sent false
1000 1118 Acked Read 1 Done Done Sent false
1001 1119 Acked Read 7 Done Done Sent false
1002 1120 Acked Read 2 Done Done Sent false
1003 1121 Acked Write 10 Done Done Sent false
1004 1122 Acked Read 4 Done Done Sent false
1005 1123 Acked Write 6 Done Done Sent false
1006 1124 Acked Read 10 Done Done Sent false
1007 1125 Acked Read 1 Done Done Sent false
1008 1126 Acked Write 3 Done Done Sent false
1009 1127 Acked Write 5 Done Done Sent false
1010 1128 Acked Read 2 Done Done Sent false
1011 1129 Acked Write 2 Done Done Sent false
1012 1130 Acked Read 6 Done Done Sent false
1013 1131 Acked Read 4 Done Done Sent false
1014 1132 Acked Write 5 Done Done Sent false
1015 1133 Acked Read 1 Done Done Sent false
1016 1134 Acked Flush 0 Done Done Sent false
1017 1135 Acked Write 5 Done Done Sent false
1018 1136 Acked Read 2 Done Done Sent false
1019 1137 Acked Read 3 Done Done Sent false
1020 1138 Acked Write 3 Done Done Sent false
1021 1139 Acked Write 8 Done Done Sent false
1022 1140 Acked Read 3 Done Done Sent false
1023 1141 Acked Read 8 Done Done Sent false
1024 1142 Acked Write 10 Done Done Sent false
1025 1143 Acked Write 10 Done Done Sent false
1026 1144 Acked Read 5 Done Done Sent false
1027 1145 Acked Write 2 Done Done Sent false
1028 1146 Acked Write 3 Done Done Sent false
1029 1147 Acked Read 4 Done Done Sent false
1030 1148 Acked Write 8 Done Done Sent false
1031 1149 Acked Write 1 Done Done Sent false
1032 1150 Acked Write 7 Done Done Sent false
1033 1151 Acked Write 10 Done Done Sent false
1034 1152 Acked Write 8 Done Done Sent false
1035 1153 Acked Read 5 Done Done Sent false
1036 1154 Acked Write 8 Done Done Sent false
1037 1155 Acked Write 10 Done Done Sent false
1038 1156 Acked Read 3 Done Done Sent false
1039 1157 Acked Write 1 Done Done Sent false
1040 1158 Acked Flush 0 Done Done Sent false
1041 1159 Acked Read 8 Done Done Sent false
1042 1160 Acked Read 4 Done Done Sent false
1043 1161 Acked Read 3 Done Done Sent false
1044 1162 Acked Write 1 Done Done Sent false
1045 1163 Acked Flush 0 Done Done Sent false
1046 1164 Acked Write 4 Done Done Sent false
1047 1165 Acked Write 7 Done Done Sent false
1048 1166 Acked Read 4 Done Done Sent false
1049 1167 Acked Read 5 Done Done Sent false
1050 1168 Acked Flush 0 Done Done Sent false
1051 1169 Acked Write 9 Done Done Sent false
1052 1170 Acked Flush 0 Done Done Sent false
1053 1171 Acked Read 1 Done Done Sent false
1054 1172 Acked Read 7 Done Done Sent false
1055 1173 Acked Write 2 Done Done Sent false
1056 1174 Acked Read 8 Done Done Sent false
1057 1175 Acked Write 1 Done Done Sent false
1058 1176 Acked Write 6 Done Done Sent false
1059 1177 Acked Read 2 Done Done Sent false
1060 1178 Acked Flush 0 Done Done Sent false
1061 1179 Acked Write 7 Done Done Sent false
1062 1180 Acked Flush 0 Done Done Sent false
1063 1181 Acked Write 1 Done Done Sent false
1064 1182 Acked Write 5 Done Done Sent false
1065 1183 Acked Read 2 Done Done Sent false
1066 1184 Acked Write 9 Done Done Sent false
1067 1185 Acked Read 10 Done Done Sent false
1068 1186 Acked Write 2 Done Done Sent false
1069 1187 Acked Read 8 Done Done Sent false
1070 1188 Acked Write 6 Done Done Sent false
1071 1189 Acked Flush 0 Done Done Sent false
1072 1190 Acked Write 6 Done Done Sent false
1073 1191 Acked Read 7 Done Done Sent false
1074 1192 Acked Write 3 Done Done Sent false
1075 1193 Acked Read 7 Done Done Sent false
1076 1194 Acked Read 1 Done Done Sent false
1077 1195 Acked Write 1 Done Done Sent false
1078 1196 Acked Write 9 Done Done Sent false
1079 1197 Acked Write 1 Done Done Sent false
1080 1198 Acked Read 6 Done Done Sent false
1081 1199 Acked Read 7 Done Done Sent false
1082 1200 Acked Flush 0 Done Done Sent false
1083 1201 Acked Write 6 Done Done Sent false
1084 1202 Acked Read 1 Done Done Sent false
1085 1203 Acked Write 9 Done Done Sent false
1086 1204 Acked Write 10 Done Done Sent false
1087 1205 Acked Write 6 Done Done Sent false
1088 1206 Acked Read 4 Done Done Sent false
1089 1207 Acked Read 4 Done Done Sent false
1090 1208 Acked Flush 0 Done Done Sent false
1091 1209 Acked Read 3 Done Done Sent false
1092 1210 Acked Write 9 Done Done Sent false
1093 1211 Acked Read 9 Done Done Sent false
1094 1212 Acked Read 6 Done Done Sent false
1095 1213 Acked Read 10 Done Done Sent false
1096 1214 Acked Read 2 Done Done Sent false
1097 1215 Acked Write 7 Done Done Sent false
1098 1216 Acked Write 3 Done Done Sent false
1099 1217 Acked Read 3 Done Done Sent false
1100 1218 Acked Read 2 Done Done Sent false
1101 1219 Acked Write 6 Done Done Sent false
1102 1220 Acked Write 2 Done Done Sent false
1103 1221 Acked Write 10 Done Done Sent false
1104 1222 Acked Read 6 Done Done Sent false
1105 1223 Acked Read 9 Done Done Sent false
1106 1224 Acked Read 4 Done Done Sent false
1107 1225 Acked Read 8 Done Done Sent false
1108 1226 Acked Read 1 Done Done Sent false
1109 1227 Acked Read 1 Done Done Sent false
1110 1228 Acked Read 4 Done Done Sent false
1111 1229 Acked Write 7 Done Done Sent false
1112 1230 Acked Write 8 Done Done Sent false
1113 1231 Acked Read 4 Done Done Sent false
1114 1232 Acked Write 8 Done Done Sent false
1115 1233 Acked Write 4 Done Done Sent false
1116 1234 Acked Write 3 Done Done Sent false
1117 1235 Acked Read 10 Done Done Sent false
1118 1236 Acked Read 5 Done Done Sent false
1119 1237 Acked Write 10 Done Done Sent false
1120 1238 Acked Read 4 Done Done Sent false
1121 1239 Acked Read 5 Done Done Sent false
1122 1240 Acked Read 3 Done Done Sent false
1123 1241 Acked Write 2 Done Done Sent false
1124 1242 Acked Read 3 Done Done Sent false
1125 1243 Acked Write 7 Done Done Sent false
1126 1244 Acked Write 7 Done Done Sent false
1127 1245 Acked Write 2 Done Done Sent false
1128 1246 Acked Read 8 Done Done Sent false
1129 1247 Acked Read 9 Done Done Sent false
1130 1248 Acked Flush 0 Done Done Sent false
1131 1249 Acked Write 8 Done Done Sent false
1132 1250 Acked Read 10 Done Done Sent false
1133 1251 Acked Write 2 Done Done Sent false
1134 1252 Acked Write 5 Done Done Sent false
1135 1253 Acked Flush 0 Done Done Sent false
1136 1254 Acked Read 7 Done Done Sent false
1137 1255 Acked Read 8 Done Done Sent false
1138 1256 Acked Write 6 Done Done Sent false
1139 1257 Acked Read 3 Done Done Sent false
1140 1258 Acked Write 10 Done Done Sent false
1141 1259 Acked Read 7 Done Done Sent false
1142 1260 Acked Write 8 Done Done Sent false
1143 1261 Acked Flush 0 Done Done Sent false
1144 1262 Acked Write 1 Done Done Sent false
1145 1263 Acked Read 6 Done Done Sent false
1146 1264 Acked Write 1 Done Done Sent false
1147 1265 Acked Read 1 Done Done Sent false
1148 1266 Acked Write 10 Done Done Sent false
1149 1267 Acked Read 10 Done Done Sent false
1150 1268 Acked Write 2 Done Done Sent false
1151 1269 Acked Read 3 Done Done Sent false
1152 1270 Acked Write 2 Done Done Sent false
1153 1271 Acked Write 3 Done Done Sent false
1154 1272 Acked Read 6 Done Done Sent false
1155 1273 Acked Write 3 Done Done Sent false
1156 1274 Acked Read 9 Done Done Sent false
1157 1275 Acked Write 9 Done Done Sent false
1158 1276 Acked Write 10 Done Done Sent false
1159 1277 Acked Read 3 Done Done Sent false
1160 1278 Acked Flush 0 Done Done Sent false
1161 1279 Acked Flush 0 Done Done Sent false
1162 1280 Acked Write 1 Done Done Sent false
1163 1281 Acked Write 7 Done Done Sent false
1164 1282 Acked Write 2 Done Done Sent false
1165 1283 Acked Read 1 Done Done Sent false
1166 1284 Acked Write 1 Done Done Sent false
1167 1285 Acked Write 6 Done Done Sent false
1168 1286 Acked Write 10 Done Done Sent false
1169 1287 Acked Read 9 Done Done Sent false
1170 1288 Acked Read 9 Done Done Sent false
1171 1289 Acked Write 8 Done Done Sent false
1172 1290 Acked Read 7 Done Done Sent false
1173 1291 Acked Read 8 Done Done Sent false
1174 1292 Acked Read 9 Done Done Sent false
1175 1293 Acked Read 4 Done Done Sent false
1176 1294 Acked Read 2 Done Done Sent false
1177 1295 Acked Write 9 Done Done Sent false
1178 1296 Acked Read 10 Done Done Sent false
1179 1297 Acked Read 6 Done Done Sent false
1180 1298 Acked Flush 0 Done Done Sent false
1181 1299 Acked Read 1 Done Done Sent false
1182 1300 Acked Read 5 Done Done Sent false
1183 1301 Acked Flush 0 Done Done Sent false
1184 STATES DS:0 DS:1 DS:2 TOTAL
1185 Sent 0 0 302 302
1186 Done 302 302 0 604
1187 Skipped 0 0 0 0
1188 Error 0 0 0 0
1189 Last Flush: 1301 1301 0
1190 Downstairs last five completed:
1191 Upstairs last five completed: 1301 1300 1299 1298 1297
1192 CLIENT: Up:0 ds:302 act:2
11932024-11-15T01:57:41.848ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
11942024-11-15T01:57:41.848ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:54501) = io task client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
11952024-11-15T01:57:41.849ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
11962024-11-15T01:57:41.850ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
11972024-11-15T01:57:41.850ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
11982024-11-15T01:57:41.850ZINFOcrucible: send last flush ID to this DS: 0 = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
11992024-11-15T01:57:41.850ZINFOcrucible: Replied this last flush ID: 0 = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
12002024-11-15T01:57:41.850ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
12012024-11-15T01:57:41.850ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
12022024-11-15T01:57:41.850ZINFOcrucible: [2] client re-new 302 jobs since flush 0 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
12032024-11-15T01:57:41.851ZINFOcrucible: [2] Replayed 302 jobs since flush: 0 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
1204 ----------------------------------------------------------------
1205 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
1206 Downstairs last five completed: 1301 1300 1299 1298 1297
1207 Upstairs last five completed: 1301 1300 1299 1298 1297
1208 CLIENT: Up:0 ds:0 act:3
1209 Replay: All jobs finished
1210 Main task has work to do, go find it
1211 got dsc Stop(0)
1212 stop 0
1213 Nov 15 01:57:44.090 INFO request completed, [8810] Got stop action so:false kr:false
1214 latency_us: 366, response_code: 204, uri: /stop/cid/0, method: POST, req_id: e16a1c11-7853-4464-87f7-6228daec2b6e, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
12152024-11-15T01:57:44.091ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
12162024-11-15T01:57:44.091ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
12172024-11-15T01:57:44.091ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
12182024-11-15T01:57:44.091ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
12192024-11-15T01:57:44.091ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
1220 Nov 15 01:57:44.091 INFO request completed, latency_us: 150, response_code: 200, uri: /state/cid/0, method{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2024-11-15T01:57:44.091373526Z","hostname":"w-01JCPRHT5F2MPA44KVJHM3S2PB","pid":1205,"":"io task","client":"0","":"downstairs","session_id":"971dcb2e-6891-4ecf-b367-1e04e52065ee"}
1221 : GET, req_id: ab928313-5e19-45f3-88b0-bc2c75783c17, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
1222 [8810] Exited with: ExitStatus(unix_wait_status(9))
1223 [8810][0] reports Exit
1224 Nov 15 01:57:48.094 INFO request completed, latency_us: 236, response_code: 200, uri: /state/cid/0, method: GET, req_id: bf5a35e9-0f73-459a-ae31-a6271a44a86d, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
1225 001/300 Read block 4657 len 12288
1226 002/300 Write block 1196 len 20480 data: 2 2 2 2 2
1227 003/300 Read block 4261 len 16384
1228 004/300 Read block 2745 len 8192
1229 005/300 Write block 359 len 20480 data: 2 2 2 2 2
1230 006/300 Flush
1231 007/300 Read block 3111 len 16384
1232 008/300 Write block 1787 len 4096 data: 2
1233 009/300 Read block 3240 len 8192
1234 010/300 Read block 713 len 12288
1235 011/300 Flush
1236 012/300 Read block 4427 len 40960
1237 013/300 Write block 4091 len 28672 data: 2 2 2 2 2 2 2
1238 014/300 Write block 1211 len 4096 data: 2
1239 015/300 Flush
1240 016/300 Write block 3824 len 12288 data: 3 3 3
1241 017/300 Read block 2382 len 40960
1242 018/300 Read block 19 len 20480
1243 019/300 Write block 4366 len 8192 data: 2 2
1244 020/300 Write block 1491 len 28672 data: 3 3 2 2 2 2 2
1245 021/300 Read block 2067 len 20480
1246 022/300 Read block 2586 len 8192
1247 023/300 Flush
1248 024/300 Read block 3891 len 20480
1249 025/300 Read block 1556 len 36864
1250 026/300 Write block 2658 len 12288 data: 2 2 2
1251 027/300 Read block 2418 len 16384
1252 028/300 Read block 510 len 4096
1253 029/300 Read block 3359 len 24576
1254 030/300 Read block 3654 len 8192
1255 031/300 Read block 4749 len 8192
1256 032/300 Read block 957 len 36864
1257 033/300 Read block 3411 len 16384
1258 034/300 Flush
1259 035/300 Write block 2279 len 4096 data: 2
1260 036/300 Write block 4035 len 16384 data: 2 2 2 2
1261 037/300 Flush
1262 038/300 Read block 3726 len 32768
1263 039/300 Read block 1708 len 4096
1264 040/300 Flush
1265 041/300 Write block 4385 len 16384 data: 2 2 2 2
1266 042/300 Flush
1267 043/300 Write block 3878 len 4096 data: 2
1268 044/300 Flush
1269 045/300 Read block 2219 len 16384
1270 046/300 Write block 4146 len 20480 data: 2 2 2 2 2
1271 047/300 Read block 3910 len 12288
1272 048/300 Write block 1600 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1273 049/300 Read block 1830 len 32768
1274 050/300 Write block 2143 len 40960 data: 2 2 2 2 2 3 4 4 4 4
1275 051/300 Read block 1243 len 12288
1276 052/300 Read block 2943 len 32768
1277 053/300 Write block 3887 len 36864 data: 2 2 2 2 2 2 2 2 2
1278 054/300 Write block 4327 len 24576 data: 2 2 2 2 2 2
1279 055/300 Write block 4612 len 4096 data: 2
1280 056/300 Read block 1694 len 32768
1281 057/300 Read block 28 len 40960
1282 058/300 Read block 1146 len 32768
1283 059/300 Read block 4911 len 12288
1284 060/300 Read block 2216 len 24576
1285 061/300 Read block 1824 len 28672
1286 062/300 Write block 3578 len 12288 data: 2 2 2
1287 063/300 Read block 2785 len 40960
1288 064/300 Write block 4516 len 12288 data: 4 3 3
1289 065/300 Read block 4948 len 4096
1290 066/300 Read block 3114 len 28672
1291 067/300 Read block 1645 len 24576
1292 068/300 Write block 391 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1293 069/300 Read block 1461 len 32768
1294 070/300 Read block 3732 len 32768
1295 071/300 Read block 321 len 20480
1296 072/300 Write block 781 len 32768 data: 2 2 3 3 3 3 3 3
1297 073/300 Write block 1986 len 12288 data: 3 3 3
1298 074/300 Read block 1961 len 8192
1299 075/300 Read block 756 len 16384
1300 076/300 Read block 942 len 28672
1301 077/300 Read block 2031 len 24576
1302 078/300 Write block 3352 len 16384 data: 2 2 2 2
1303 079/300 Write block 3862 len 36864 data: 2 2 2 2 2 2 2 2 2
1304 080/300 Read block 303 len 8192
1305 081/300 Write block 894 len 12288 data: 2 2 2
1306 082/300 Write block 1513 len 36864 data: 2 2 2 2 2 2 2 2 2
1307 083/300 Write block 588 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1308 084/300 Write block 753 len 24576 data: 3 3 3 3 4 4
1309 085/300 Read block 4371 len 4096
1310 086/300 Write block 335 len 12288 data: 2 2 2
1311 087/300 Read block 562 len 20480
1312 088/300 Write block 839 len 24576 data: 2 2 3 3 3 3
1313 089/300 Write block 1275 len 16384 data: 2 2 2 2
1314 090/300 Write block 4660 len 12288 data: 2 2 2
1315 091/300 Read block 3128 len 4096
1316 092/300 Read block 2765 len 4096
1317 093/300 Write block 2971 len 20480 data: 2 2 2 2 2
1318 094/300 Read block 3836 len 12288
1319 095/300 Read block 2665 len 36864
1320 096/300 Read block 2898 len 16384
1321 097/300 Write block 3444 len 12288 data: 2 2 2
1322 098/300 Read block 620 len 28672
1323 099/300 Write block 4081 len 4096 data: 2
1324 100/300 Read block 300 len 28672
1325 101/300 Write block 2657 len 32768 data: 2 3 3 3 2 2 2 2
1326 102/300 Read block 4313 len 4096
1327 103/300 Read block 3022 len 16384
1328 104/300 Read block 3931 len 28672
1329 105/300 Read block 3731 len 28672
1330 106/300 Read block 3937 len 28672
1331 107/300 Read block 2036 len 24576
1332 108/300 Write block 1558 len 24576 data: 2 3 3 3 3 3
1333 109/300 Write block 3188 len 8192 data: 2 2
1334 110/300 Write block 963 len 24576 data: 2 2 2 2 2 2
1335 111/300 Read block 1100 len 40960
1336 112/300 Write block 937 len 8192 data: 2 2
1337 113/300 Read block 2566 len 24576
1338 114/300 Read block 1528 len 16384
1339 115/300 Write block 2522 len 16384 data: 2 2 2 2
1340 116/300 Write block 771 len 16384 data: 2 2 2 2
1341 117/300 Write block 4205 len 8192 data: 2 3
1342 118/300 Read block 3029 len 16384
1343 119/300 Read block 2949 len 40960
1344 120/300 Read block 972 len 20480
1345 121/300 Write block 878 len 28672 data: 2 2 2 2 2 2 2
1346 122/300 Read block 3474 len 36864
1347 123/300 Write block 3343 len 40960 data: 2 2 2 2 2 2 2 2 2 3
1348 124/300 Read block 3197 len 28672
1349 125/300 Read block 4264 len 28672
1350 126/300 Read block 1445 len 16384
1351 127/300 Read block 2190 len 40960
1352 128/300 Write block 2216 len 12288 data: 3 3 3
1353 129/300 Read block 2162 len 28672
1354 130/300 Write block 4253 len 32768 data: 2 2 2 2 2 2 2 2
1355 131/300 Read block 3772 len 40960
1356 132/300 Write block 872 len 8192 data: 2 2
1357 133/300 Write block 3452 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1358 134/300 Write block 2888 len 28672 data: 3 3 3 2 2 2 2
1359 135/300 Read block 3045 len 36864
1360 136/300 Flush
1361 137/300 Read block 2116 len 28672
1362 138/300 Write block 2292 len 16384 data: 2 2 2 2
1363 139/300 Write block 4202 len 28672 data: 2 2 2 3 4 3 3
1364 140/300 Write block 544 len 20480 data: 2 2 2 2 2
1365 141/300 Read block 3577 len 40960
1366 142/300 Flush
1367 143/300 Write block 2946 len 28672 data: 3 2 2 2 2 2 2
1368 144/300 Read block 3229 len 32768
1369 145/300 Write block 1845 len 24576 data: 2 2 2 2 2 2
1370 146/300 Read block 2320 len 24576
1371 147/300 Read block 3393 len 4096
1372 148/300 Flush
1373 149/300 Read block 1574 len 40960
1374 150/300 Read block 2642 len 24576
1375 151/300 Read block 3363 len 36864
1376 152/300 Flush
1377 153/300 Read block 2092 len 20480
1378 154/300 Write block 3408 len 8192 data: 3 3
1379 155/300 Write block 3669 len 24576 data: 2 2 2 2 2 2
1380 156/300 Read block 826 len 24576
1381 157/300 Read block 4066 len 12288
1382 158/300 Write block 4435 len 24576 data: 2 2 2 2 2 2
1383 159/300 Write block 236 len 24576 data: 3 2 2 2 2 2
1384 160/300 Read block 4952 len 32768
1385 161/300 Read block 552 len 4096
1386 162/300 Read block 3466 len 36864
1387 163/300 Write block 4347 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1388 164/300 Write block 3576 len 24576 data: 2 2 3 3 3 2
1389 165/300 Read block 2095 len 32768
1390 166/300 Write block 1881 len 36864 data: 2 2 2 2 2 2 2 2 2
1391 167/300 Read block 3580 len 4096
1392 168/300 Write block 833 len 16384 data: 2 2 2 2
1393 169/300 Read block 2227 len 32768
1394 170/300 Write block 4540 len 28672 data: 2 2 2 2 2 2 2
1395 171/300 Write block 1001 len 36864 data: 3 3 3 2 2 2 2 2 2
1396 172/300 Read block 3465 len 12288
1397 173/300 Read block 4068 len 32768
1398 174/300 Write block 1194 len 24576 data: 2 2 3 3 3 3
1399 175/300 Read block 1209 len 8192
1400 176/300 Read block 4817 len 32768
1401 177/300 Read block 1973 len 16384
1402 178/300 Write block 3076 len 28672 data: 3 4 4 4 5 4 4
1403 179/300 Write block 1308 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1404 180/300 Read block 2103 len 12288
1405 181/300 Write block 3794 len 4096 data: 2
1406 182/300 Write block 2867 len 36864 data: 2 2 2 2 2 2 2 2 2
1407 183/300 Flush
1408 184/300 Write block 4751 len 16384 data: 2 2 2 2
1409 185/300 Read block 4354 len 4096
1410 186/300 Read block 722 len 36864
1411 187/300 Write block 1506 len 40960 data: 2 2 2 2 2 2 2 3 3 3
1412 188/300 Read block 2686 len 40960
1413 189/300 Write block 4480 len 32768 data: 2 2 2 2 2 2 2 2
1414 190/300 Write block 2856 len 20480 data: 2 2 2 2 2
1415 191/300 Read block 3362 len 28672
1416 192/300 Write block 1837 len 36864 data: 2 2 2 2 2 2 2 2 3
1417 193/300 Flush
1418 194/300 Flush
1419 195/300 Read block 4299 len 28672
1420 196/300 Read block 2110 len 24576
1421 197/300 Read block 4876 len 36864
1422 198/300 Read block 4699 len 20480
1423 199/300 Read block 3786 len 16384
1424 200/300 Write block 40 len 32768 data: 2 2 2 2 2 2 2 2
1425 201/300 Write block 4282 len 36864 data: 2 2 2 2 2 2 2 2 2
1426 202/300 Write block 1777 len 28672 data: 2 2 2 2 2 2 2
1427 203/300 Write block 3531 len 12288 data: 2 2 2
1428 204/300 Write block 3149 len 20480 data: 2 2 2 2 2
1429 205/300 Read block 2369 len 4096
1430 206/300 Write block 2252 len 16384 data: 2 2 2 2
1431 207/300 Read block 998 len 8192
1432 208/300 Read block 2002 len 12288
1433 209/300 Read block 3865 len 36864
1434 210/300 Read block 1321 len 32768
1435 211/300 Flush
1436 212/300 Read block 3657 len 36864
1437 213/300 Write block 2127 len 20480 data: 2 2 3 3 3
1438 214/300 Write block 4301 len 32768 data: 2 2 2 2 2 2 2 2
1439 215/300 Read block 3642 len 20480
1440 216/300 Read block 2779 len 28672
1441 217/300 Read block 4428 len 36864
1442 218/300 Write block 3273 len 4096 data: 2
1443 219/300 Flush
1444 220/300 Write block 2340 len 32768 data: 2 2 2 2 2 2 2 2
1445 221/300 Read block 4413 len 16384
1446 222/300 Write block 2725 len 36864 data: 2 2 2 2 2 2 2 2 2
1447 223/300 Write block 2628 len 24576 data: 2 2 2 2 2 2
1448 224/300 Read block 104 len 40960
1449 225/300 Flush
1450 226/300 Read block 1363 len 16384
1451 227/300 Read block 1974 len 4096
1452 228/300 Read block 4252 len 8192
1453 229/300 Read block 3280 len 16384
1454 230/300 Write block 2954 len 16384 data: 2 2 2 2
1455 231/300 Read block 2643 len 28672
1456 232/300 Flush
1457 233/300 Write block 3754 len 40960 data: 3 3 3 3 3 2 2 2 2 2
1458 234/300 Write block 3874 len 28672 data: 2 3 3 2 3 2 2
1459 235/300 Read block 2574 len 28672
1460 236/300 Read block 735 len 40960
1461 237/300 Write block 2933 len 32768 data: 2 2 2 2 2 2 2 2
1462 238/300 Read block 1415 len 4096
1463 239/300 Read block 3549 len 20480
1464 240/300 Write block 3602 len 20480 data: 2 2 2 2 2
1465 241/300 Read block 884 len 32768
1466 242/300 Write block 587 len 28672 data: 2 3 3 3 3 3 3
1467 243/300 Read block 1465 len 32768
1468 244/300 Read block 1895 len 16384
1469 245/300 Flush
1470 246/300 Flush
1471 247/300 Write block 391 len 4096 data: 3
1472 248/300 Write block 3218 len 32768 data: 2 2 2 2 2 2 2 2
1473 249/300 Read block 1826 len 28672
1474 250/300 Read block 3022 len 36864
1475 251/300 Read block 2418 len 8192
1476 252/300 Write block 166 len 36864 data: 2 2 2 2 2 2 2 2 2
1477 253/300 Read block 2858 len 4096
1478 254/300 Read block 4659 len 24576
1479 255/300 Read block 3714 len 16384
1480 256/300 Read block 4854 len 40960
1481 257/300 Read block 4867 len 4096
1482 258/300 Read block 2641 len 20480
1483 259/300 Read block 4429 len 36864
1484 260/300 Write block 2360 len 12288 data: 2 2 2
1485 261/300 Write block 94 len 4096 data: 2
1486 262/300 Write block 2951 len 36864 data: 3 3 2 3 3 3 3 2 2
1487 263/300 Read block 146 len 32768
1488 264/300 Write block 2763 len 16384 data: 2 2 2 2
1489 265/300 Write block 3978 len 4096 data: 2
1490 266/300 Write block 909 len 4096 data: 2
1491 267/300 Read block 906 len 40960
1492 268/300 Read block 4278 len 40960
1493 269/300 Read block 4489 len 20480
1494 270/300 Flush
1495 271/300 Write block 1550 len 32768 data: 2 2 2 2 2 2 2 2
1496 272/300 Write block 4470 len 36864 data: 2 2 2 2 2 2 2 2 2
1497 273/300 Read block 4609 len 24576
1498 274/300 Write block 4724 len 16384 data: 2 2 2 2
1499 275/300 Flush
1500 276/300 Flush
1501 277/300 Read block 3785 len 4096
1502 278/300 Read block 1292 len 36864
1503 279/300 Write block 6 len 28672 data: 2 2 2 2 2 2 2
1504 280/300 Write block 4264 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1505 281/300 Write block 66 len 40960 data: 3 3 3 3 2 2 2 2 2 2
1506 282/300 Write block 793 len 12288 data: 2 2 2
1507 283/300 Read block 4534 len 36864
1508 284/300 Read block 4597 len 32768
1509 285/300 Write block 2515 len 8192 data: 2 2
1510 286/300 Read block 331 len 4096
1511 287/300 Read block 3482 len 12288
1512 288/300 Write block 961 len 32768 data: 2 2 3 3 3 3 3 3
1513 289/300 Read block 4157 len 4096
1514 290/300 Read block 338 len 28672
1515 291/300 Flush
1516 292/300 Write block 1449 len 12288 data: 2 2 2
1517 293/300 Read block 4979 len 24576
1518 294/300 Write block 1455 len 4096 data: 3
1519 295/300 Read block 3520 len 40960
1520 296/300 Write block 3256 len 28672 data: 2 2 2 2 2 3 2
1521 297/300 Read block 4245 len 8192
1522 298/300 Write block 3517 len 36864 data: 2 2 2 2 3 3 3 3 3
1523 299/300 Write block 3497 len 20480 data: 2 2 2 2 2
1524 300/300 Read block 3678 len 16384
1525 301/300 Flush
1526 Main task has work to do, go find it
1527 got dsc Start(0)
1528 start 0
1529 Nov 15 01:57:48.334 INFO request completed, latency_us: 226, response_code: 204, uri: /start/cid/0, method: POST, req_id: 755bd330-a9cb-419a-aad3-927a59a0382a, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
1530 Replay: started 0, returned:Ok(())
1531 ----------------------------------------------------------------
1532 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
1533 States: Offline Active Active
1534 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
1535 1302 Acked Read 3 Sent Done Done false
1536 1303 Acked Write 5 Sent Done Done false
1537 1304 Acked Read 4 Sent Done Done false
1538 1305 Acked Read 2 Sent Done Done false
1539 1306 Acked Write 5 Sent Done Done false
1540 1307 Acked Flush 0 Sent Done Done false
1541 1308 Acked Read 4 Sent Done Done false
1542 1309 Acked Write 1 Sent Done Done false
1543 1310 Acked Read 2 Sent Done Done false
1544 1311 Acked Read 3 Sent Done Done false
1545 1312 Acked Flush 0 Sent Done Done false
1546 1313 Acked Read 10 Sent Done Done false
1547 1314 Acked Write 7 Sent Done Done false
1548 1315 Acked Write 1 Sent Done Done false
1549 1316 Acked Flush 0 Sent Done Done false
1550 1317 Acked Write 3 Sent Done Done false
1551 1318 Acked Read 10 Sent Done Done false
1552 1319 Acked Read 5 Sent Done Done false
1553 1320 Acked Write 2 Sent Done Done false
1554 1321 Acked Write 7 Sent Done Done false
1555 1322 Acked Read 5 Sent Done Done false
1556 1323 Acked Read 2 Sent Done Done false
1557 1324 Acked Flush 0 Sent Done Done false
1558 1325 Acked Read 5 Sent Done Done false
1559 1326 Acked Read 9 Sent Done Done false
1560 1327 Acked Write 3 Sent Done Done false
1561 1328 Acked Read 4 Sent Done Done false
1562 1329 Acked Read 1 Sent Done Done false
1563 1330 Acked Read 6 Sent Done Done false
1564 1331 Acked Read 2 Sent Done Done false
1565 1332 Acked Read 2 Sent Done Done false
1566 1333 Acked Read 9 Sent Done Done false
1567 1334 Acked Read 4 Sent Done Done false
1568 1335 Acked Flush 0 Sent Done Done false
1569 1336 Acked Write 1 Sent Done Done false
1570 1337 Acked Write 4 Sent Done Done false
1571 1338 Acked Flush 0 Sent Done Done false
1572 1339 Acked Read 8 Sent Done Done false
1573 1340 Acked Read 1 Sent Done Done false
1574 1341 Acked Flush 0 Sent Done Done false
1575 1342 Acked Write 4 Sent Done Done false
1576 1343 Acked Flush 0 Sent Done Done false
1577 1344 Acked Write 1 Sent Done Done false
1578 1345 Acked Flush 0 Sent Done Done false
1579 1346 Acked Read 4 Sent Done Done false
1580 1347 Acked Write 5 Sent Done Done false
1581 1348 Acked Read 3 Sent Done Done false
1582 1349 Acked Write 10 Sent Done Done false
1583 1350 Acked Read 8 Sent Done Done false
1584 1351 Acked Write 10 Sent Done Done false
1585 1352 Acked Read 3 Sent Done Done false
1586 1353 Acked Read 8 Sent Done Done false
1587 1354 Acked Write 9 Sent Done Done false
1588 1355 Acked Write 6 Sent Done Done false
1589 1356 Acked Write 1 Sent Done Done false
1590 1357 Acked Read 8 Sent Done Done false
1591 1358 Acked Read 10 Sent Done Done false
1592 1359 Acked Read 8 Sent Done Done false
1593 1360 Acked Read 3 Sent Done Done false
1594 1361 Acked Read 6 Sent Done Done false
1595 1362 Acked Read 7 Sent Done Done false
1596 1363 Acked Write 3 Sent Done Done false
1597 1364 Acked Read 10 Sent Done Done false
1598 1365 Acked Write 3 Sent Done Done false
1599 1366 Acked Read 1 Sent Done Done false
1600 1367 Acked Read 7 Sent Done Done false
1601 1368 Acked Read 6 Sent Done Done false
1602 1369 Acked Write 10 Sent Done Done false
1603 1370 Acked Read 8 Sent Done Done false
1604 1371 Acked Read 8 Sent Done Done false
1605 1372 Acked Read 5 Sent Done Done false
1606 1373 Acked Write 8 Sent Done Done false
1607 1374 Acked Write 3 Sent Done Done false
1608 1375 Acked Read 2 Sent Done Done false
1609 1376 Acked Read 4 Sent Done Done false
1610 1377 Acked Read 7 Sent Done Done false
1611 1378 Acked Read 6 Sent Done Done false
1612 1379 Acked Write 4 Sent Done Done false
1613 1380 Acked Write 9 Sent Done Done false
1614 1381 Acked Read 2 Sent Done Done false
1615 1382 Acked Write 3 Sent Done Done false
1616 1383 Acked Write 9 Sent Done Done false
1617 1384 Acked Write 10 Sent Done Done false
1618 1385 Acked Write 6 Sent Done Done false
1619 1386 Acked Read 1 Sent Done Done false
1620 1387 Acked Write 3 Sent Done Done false
1621 1388 Acked Read 5 Sent Done Done false
1622 1389 Acked Write 6 Sent Done Done false
1623 1390 Acked Write 4 Sent Done Done false
1624 1391 Acked Write 3 Sent Done Done false
1625 1392 Acked Read 1 Sent Done Done false
1626 1393 Acked Read 1 Sent Done Done false
1627 1394 Acked Write 5 Sent Done Done false
1628 1395 Acked Read 3 Sent Done Done false
1629 1396 Acked Read 9 Sent Done Done false
1630 1397 Acked Read 4 Sent Done Done false
1631 1398 Acked Write 3 Sent Done Done false
1632 1399 Acked Read 7 Sent Done Done false
1633 1400 Acked Write 1 Sent Done Done false
1634 1401 Acked Read 7 Sent Done Done false
1635 1402 Acked Write 8 Sent Done Done false
1636 1403 Acked Read 1 Sent Done Done false
1637 1404 Acked Read 4 Sent Done Done false
1638 1405 Acked Read 7 Sent Done Done false
1639 1406 Acked Read 7 Sent Done Done false
1640 1407 Acked Read 7 Sent Done Done false
1641 1408 Acked Read 6 Sent Done Done false
1642 1409 Acked Write 6 Sent Done Done false
1643 1410 Acked Write 2 Sent Done Done false
1644 1411 Acked Write 6 Sent Done Done false
1645 1412 Acked Read 10 Sent Done Done false
1646 1413 Acked Write 2 Sent Done Done false
1647 1414 Acked Read 6 Sent Done Done false
1648 1415 Acked Read 4 Sent Done Done false
1649 1416 Acked Write 4 Sent Done Done false
1650 1417 Acked Write 4 Sent Done Done false
1651 1418 Acked Write 2 Sent Done Done false
1652 1419 Acked Read 4 Sent Done Done false
1653 1420 Acked Read 10 Sent Done Done false
1654 1421 Acked Read 5 Sent Done Done false
1655 1422 Acked Write 7 Sent Done Done false
1656 1423 Acked Read 9 Sent Done Done false
1657 1424 Acked Write 10 Sent Done Done false
1658 1425 Acked Read 7 Sent Done Done false
1659 1426 Acked Read 7 Sent Done Done false
1660 1427 Acked Read 4 Sent Done Done false
1661 1428 Acked Read 10 Sent Done Done false
1662 1429 Acked Write 3 Sent Done Done false
1663 1430 Acked Read 7 Sent Done Done false
1664 1431 Acked Write 8 Sent Done Done false
1665 1432 Acked Read 10 Sent Done Done false
1666 1433 Acked Write 2 Sent Done Done false
1667 1434 Acked Write 10 Sent Done Done false
1668 1435 Acked Write 7 Sent Done Done false
1669 1436 Acked Read 9 Sent Done Done false
1670 1437 Acked Flush 0 Sent Done Done false
1671 1438 Acked Read 7 Sent Done Done false
1672 1439 Acked Write 4 Sent Done Done false
1673 1440 Acked Write 7 Sent Done Done false
1674 1441 Acked Write 5 Sent Done Done false
1675 1442 Acked Read 10 Sent Done Done false
1676 1443 Acked Flush 0 Sent Done Done false
1677 1444 Acked Write 7 Sent Done Done false
1678 1445 Acked Read 8 Sent Done Done false
1679 1446 Acked Write 6 Sent Done Done false
1680 1447 Acked Read 6 Sent Done Done false
1681 1448 Acked Read 1 Sent Done Done false
1682 1449 Acked Flush 0 Sent Done Done false
1683 1450 Acked Read 10 Sent Done Done false
1684 1451 Acked Read 6 Sent Done Done false
1685 1452 Acked Read 9 Sent Done Done false
1686 1453 Acked Flush 0 Sent Done Done false
1687 1454 Acked Read 5 Sent Done Done false
1688 1455 Acked Write 2 Sent Done Done false
1689 1456 Acked Write 6 Sent Done Done false
1690 1457 Acked Read 6 Sent Done Done false
1691 1458 Acked Read 3 Sent Done Done false
1692 1459 Acked Write 6 Sent Done Done false
1693 1460 Acked Write 6 Sent Done Done false
1694 1461 Acked Read 8 Sent Done Done false
1695 1462 Acked Read 1 Sent Done Done false
1696 1463 Acked Read 9 Sent Done Done false
1697 1464 Acked Write 10 Sent Done Done false
1698 1465 Acked Write 6 Sent Done Done false
1699 1466 Acked Read 8 Sent Done Done false
1700 1467 Acked Write 9 Sent Done Done false
1701 1468 Acked Read 1 Sent Done Done false
1702 1469 Acked Write 4 Sent Done Done false
1703 1470 Acked Read 8 Sent Done Done false
1704 1471 Acked Write 7 Sent Done Done false
1705 1472 Acked Write 9 Sent Done Done false
1706 1473 Acked Read 3 Sent Done Done false
1707 1474 Acked Read 8 Sent Done Done false
1708 1475 Acked Write 6 Sent Done Done false
1709 1476 Acked Read 2 Sent Done Done false
1710 1477 Acked Read 8 Sent Done Done false
1711 1478 Acked Read 4 Sent Done Done false
1712 1479 Acked Write 7 Sent Done Done false
1713 1480 Acked Write 10 Sent Done Done false
1714 1481 Acked Read 3 Sent Done Done false
1715 1482 Acked Write 1 Sent Done Done false
1716 1483 Acked Write 9 Sent Done Done false
1717 1484 Acked Flush 0 Sent Done Done false
1718 1485 Acked Write 4 Sent Done Done false
1719 1486 Acked Read 1 Sent Done Done false
1720 1487 Acked Read 9 Sent Done Done false
1721 1488 Acked Write 10 Sent Done Done false
1722 1489 Acked Read 10 Sent Done Done false
1723 1490 Acked Write 8 Sent Done Done false
1724 1491 Acked Write 5 Sent Done Done false
1725 1492 Acked Read 7 Sent Done Done false
1726 1493 Acked Write 9 Sent Done Done false
1727 1494 Acked Flush 0 Sent Done Done false
1728 1495 Acked Flush 0 Sent Done Done false
1729 1496 Acked Read 7 Sent Done Done false
1730 1497 Acked Read 6 Sent Done Done false
1731 1498 Acked Read 9 Sent Done Done false
1732 1499 Acked Read 5 Sent Done Done false
1733 1500 Acked Read 4 Sent Done Done false
1734 1501 Acked Write 8 Sent Done Done false
1735 1502 Acked Write 9 Sent Done Done false
1736 1503 Acked Write 7 Sent Done Done false
1737 1504 Acked Write 3 Sent Done Done false
1738 1505 Acked Write 5 Sent Done Done false
1739 1506 Acked Read 1 Sent Done Done false
1740 1507 Acked Write 4 Sent Done Done false
1741 1508 Acked Read 2 Sent Done Done false
1742 1509 Acked Read 3 Sent Done Done false
1743 1510 Acked Read 9 Sent Done Done false
1744 1511 Acked Read 8 Sent Done Done false
1745 1512 Acked Flush 0 Sent Done Done false
1746 1513 Acked Read 9 Sent Done Done false
1747 1514 Acked Write 5 Sent Done Done false
1748 1515 Acked Write 8 Sent Done Done false
1749 1516 Acked Read 5 Sent Done Done false
1750 1517 Acked Read 7 Sent Done Done false
1751 1518 Acked Read 9 Sent Done Done false
1752 1519 Acked Write 1 Sent Done Done false
1753 1520 Acked Flush 0 Sent Done Done false
1754 1521 Acked Write 8 Sent Done Done false
1755 1522 Acked Read 4 Sent Done Done false
1756 1523 Acked Write 9 Sent Done Done false
1757 1524 Acked Write 6 Sent Done Done false
1758 1525 Acked Read 10 Sent Done Done false
1759 1526 Acked Flush 0 Sent Done Done false
1760 1527 Acked Read 4 Sent Done Done false
1761 1528 Acked Read 1 Sent Done Done false
1762 1529 Acked Read 2 Sent Done Done false
1763 1530 Acked Read 4 Sent Done Done false
1764 1531 Acked Write 4 Sent Done Done false
1765 1532 Acked Read 7 Sent Done Done false
1766 1533 Acked Flush 0 Sent Done Done false
1767 1534 Acked Write 10 Sent Done Done false
1768 1535 Acked Write 7 Sent Done Done false
1769 1536 Acked Read 7 Sent Done Done false
1770 1537 Acked Read 10 Sent Done Done false
1771 1538 Acked Write 8 Sent Done Done false
1772 1539 Acked Read 1 Sent Done Done false
1773 1540 Acked Read 5 Sent Done Done false
1774 1541 Acked Write 5 Sent Done Done false
1775 1542 Acked Read 8 Sent Done Done false
1776 1543 Acked Write 7 Sent Done Done false
1777 1544 Acked Read 8 Sent Done Done false
1778 1545 Acked Read 4 Sent Done Done false
1779 1546 Acked Flush 0 Sent Done Done false
1780 1547 Acked Flush 0 Sent Done Done false
1781 1548 Acked Write 1 Sent Done Done false
1782 1549 Acked Write 8 Sent Done Done false
1783 1550 Acked Read 7 Sent Done Done false
1784 1551 Acked Read 9 Sent Done Done false
1785 1552 Acked Read 2 Sent Done Done false
1786 1553 Acked Write 9 Sent Done Done false
1787 1554 Acked Read 1 Sent Done Done false
1788 1555 Acked Read 6 Sent Done Done false
1789 1556 Acked Read 4 Sent Done Done false
1790 1557 Acked Read 10 Sent Done Done false
1791 1558 Acked Read 1 Sent Done Done false
1792 1559 Acked Read 5 Sent Done Done false
1793 1560 Acked Read 9 Sent Done Done false
1794 1561 Acked Write 3 Sent Done Done false
1795 1562 Acked Write 1 Sent Done Done false
1796 1563 Acked Write 9 Sent Done Done false
1797 1564 Acked Read 8 Sent Done Done false
1798 1565 Acked Write 4 Sent Done Done false
1799 1566 Acked Write 1 Sent Done Done false
1800 1567 Acked Write 1 Sent Done Done false
1801 1568 Acked Read 10 Sent Done Done false
1802 1569 Acked Read 10 Sent Done Done false
1803 1570 Acked Read 5 Sent Done Done false
1804 1571 Acked Flush 0 Sent Done Done false
1805 1572 Acked Write 8 Sent Done Done false
1806 1573 Acked Write 9 Sent Done Done false
1807 1574 Acked Read 6 Sent Done Done false
1808 1575 Acked Write 4 Sent Done Done false
1809 1576 Acked Flush 0 Sent Done Done false
1810 1577 Acked Flush 0 Sent Done Done false
1811 1578 Acked Read 1 Sent Done Done false
1812 1579 Acked Read 9 Sent Done Done false
1813 1580 Acked Write 7 Sent Done Done false
1814 1581 Acked Write 10 Sent Done Done false
1815 1582 Acked Write 10 Sent Done Done false
1816 1583 Acked Write 3 Sent Done Done false
1817 1584 Acked Read 9 Sent Done Done false
1818 1585 Acked Read 8 Sent Done Done false
1819 1586 Acked Write 2 Sent Done Done false
1820 1587 Acked Read 1 Sent Done Done false
1821 1588 Acked Read 3 Sent Done Done false
1822 1589 Acked Write 8 Sent Done Done false
1823 1590 Acked Read 1 Sent Done Done false
1824 1591 Acked Read 7 Sent Done Done false
1825 1592 Acked Flush 0 Sent Done Done false
1826 1593 Acked Write 3 Sent Done Done false
1827 1594 Acked Read 6 Sent Done Done false
1828 1595 Acked Write 1 Sent Done Done false
1829 1596 Acked Read 10 Sent Done Done false
1830 1597 Acked Write 7 Sent Done Done false
1831 1598 Acked Read 2 Sent Done Done false
1832 1599 Acked Write 9 Sent Done Done false
1833 1600 Acked Write 5 Sent Done Done false
1834 1601 Acked Read 4 Sent Done Done false
1835 1602 Acked Flush 0 Sent Done Done false
1836 STATES DS:0 DS:1 DS:2 TOTAL
1837 Sent 301 0 0 301
1838 Done 0 301 301 602
1839 Skipped 0 0 0 0
1840 Error 0 0 0 0
1841 Last Flush: 1301 1602 1602
1842 Downstairs last five completed: 1301 1300 1299 1298 1297
1843 Upstairs last five completed: 1602 1601 1600 1599 1598
1844 CLIENT: Up:0 ds:301 act:2
1845 [8810] got start action
1846 [8810] I am going to restart
1847 Starting downstairs at port 8810
1848 Make output file at "/tmp/dsc/downstairs-8810.txt"
1849 [8810][0] reports Starting
1850 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1207)
1851 [8810][0] reports Running
1852 ----------------------------------------------------------------
1853 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
1854 States: Offline Active Active
1855 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
1856 1302 Acked Read 3 Sent Done Done false
1857 1303 Acked Write 5 Sent Done Done false
1858 1304 Acked Read 4 Sent Done Done false
1859 1305 Acked Read 2 Sent Done Done false
1860 1306 Acked Write 5 Sent Done Done false
1861 1307 Acked Flush 0 Sent Done Done false
1862 1308 Acked Read 4 Sent Done Done false
1863 1309 Acked Write 1 Sent Done Done false
1864 1310 Acked Read 2 Sent Done Done false
1865 1311 Acked Read 3 Sent Done Done false
1866 1312 Acked Flush 0 Sent Done Done false
1867 1313 Acked Read 10 Sent Done Done false
1868 1314 Acked Write 7 Sent Done Done false
1869 1315 Acked Write 1 Sent Done Done false
1870 1316 Acked Flush 0 Sent Done Done false
1871 1317 Acked Write 3 Sent Done Done false
1872 1318 Acked Read 10 Sent Done Done false
1873 1319 Acked Read 5 Sent Done Done false
1874 1320 Acked Write 2 Sent Done Done false
1875 1321 Acked Write 7 Sent Done Done false
1876 1322 Acked Read 5 Sent Done Done false
1877 1323 Acked Read 2 Sent Done Done false
1878 1324 Acked Flush 0 Sent Done Done false
1879 1325 Acked Read 5 Sent Done Done false
1880 1326 Acked Read 9 Sent Done Done false
1881 1327 Acked Write 3 Sent Done Done false
1882 1328 Acked Read 4 Sent Done Done false
1883 1329 Acked Read 1 Sent Done Done false
1884 1330 Acked Read 6 Sent Done Done false
1885 1331 Acked Read 2 Sent Done Done false
1886 1332 Acked Read 2 Sent Done Done false
1887 1333 Acked Read 9 Sent Done Done false
1888 1334 Acked Read 4 Sent Done Done false
1889 1335 Acked Flush 0 Sent Done Done false
1890 1336 Acked Write 1 Sent Done Done false
1891 1337 Acked Write 4 Sent Done Done false
1892 1338 Acked Flush 0 Sent Done Done false
1893 1339 Acked Read 8 Sent Done Done false
1894 1340 Acked Read 1 Sent Done Done false
1895 1341 Acked Flush 0 Sent Done Done false
1896 1342 Acked Write 4 Sent Done Done false
1897 1343 Acked Flush 0 Sent Done Done false
1898 1344 Acked Write 1 Sent Done Done false
1899 1345 Acked Flush 0 Sent Done Done false
1900 1346 Acked Read 4 Sent Done Done false
1901 1347 Acked Write 5 Sent Done Done false
1902 1348 Acked Read 3 Sent Done Done false
1903 1349 Acked Write 10 Sent Done Done false
1904 1350 Acked Read 8 Sent Done Done false
1905 1351 Acked Write 10 Sent Done Done false
1906 1352 Acked Read 3 Sent Done Done false
1907 1353 Acked Read 8 Sent Done Done false
1908 1354 Acked Write 9 Sent Done Done false
1909 1355 Acked Write 6 Sent Done Done false
1910 1356 Acked Write 1 Sent Done Done false
1911 1357 Acked Read 8 Sent Done Done false
1912 1358 Acked Read 10 Sent Done Done false
1913 1359 Acked Read 8 Sent Done Done false
1914 1360 Acked Read 3 Sent Done Done false
1915 1361 Acked Read 6 Sent Done Done false
1916 1362 Acked Read 7 Sent Done Done false
1917 1363 Acked Write 3 Sent Done Done false
1918 1364 Acked Read 10 Sent Done Done false
1919 1365 Acked Write 3 Sent Done Done false
1920 1366 Acked Read 1 Sent Done Done false
1921 1367 Acked Read 7 Sent Done Done false
1922 1368 Acked Read 6 Sent Done Done false
1923 1369 Acked Write 10 Sent Done Done false
1924 1370 Acked Read 8 Sent Done Done false
1925 1371 Acked Read 8 Sent Done Done false
1926 1372 Acked Read 5 Sent Done Done false
1927 1373 Acked Write 8 Sent Done Done false
1928 1374 Acked Write 3 Sent Done Done false
1929 1375 Acked Read 2 Sent Done Done false
1930 1376 Acked Read 4 Sent Done Done false
1931 1377 Acked Read 7 Sent Done Done false
1932 1378 Acked Read 6 Sent Done Done false
1933 1379 Acked Write 4 Sent Done Done false
1934 1380 Acked Write 9 Sent Done Done false
1935 1381 Acked Read 2 Sent Done Done false
1936 1382 Acked Write 3 Sent Done Done false
1937 1383 Acked Write 9 Sent Done Done false
1938 1384 Acked Write 10 Sent Done Done false
1939 1385 Acked Write 6 Sent Done Done false
1940 1386 Acked Read 1 Sent Done Done false
1941 1387 Acked Write 3 Sent Done Done false
1942 1388 Acked Read 5 Sent Done Done false
1943 1389 Acked Write 6 Sent Done Done false
1944 1390 Acked Write 4 Sent Done Done false
1945 1391 Acked Write 3 Sent Done Done false
1946 1392 Acked Read 1 Sent Done Done false
1947 1393 Acked Read 1 Sent Done Done false
1948 1394 Acked Write 5 Sent Done Done false
1949 1395 Acked Read 3 Sent Done Done false
1950 1396 Acked Read 9 Sent Done Done false
1951 1397 Acked Read 4 Sent Done Done false
1952 1398 Acked Write 3 Sent Done Done false
1953 1399 Acked Read 7 Sent Done Done false
1954 1400 Acked Write 1 Sent Done Done false
1955 1401 Acked Read 7 Sent Done Done false
1956 1402 Acked Write 8 Sent Done Done false
1957 1403 Acked Read 1 Sent Done Done false
1958 1404 Acked Read 4 Sent Done Done false
1959 1405 Acked Read 7 Sent Done Done false
1960 1406 Acked Read 7 Sent Done Done false
1961 1407 Acked Read 7 Sent Done Done false
1962 1408 Acked Read 6 Sent Done Done false
1963 1409 Acked Write 6 Sent Done Done false
1964 1410 Acked Write 2 Sent Done Done false
1965 1411 Acked Write 6 Sent Done Done false
1966 1412 Acked Read 10 Sent Done Done false
1967 1413 Acked Write 2 Sent Done Done false
1968 1414 Acked Read 6 Sent Done Done false
1969 1415 Acked Read 4 Sent Done Done false
1970 1416 Acked Write 4 Sent Done Done false
1971 1417 Acked Write 4 Sent Done Done false
1972 1418 Acked Write 2 Sent Done Done false
1973 1419 Acked Read 4 Sent Done Done false
1974 1420 Acked Read 10 Sent Done Done false
1975 1421 Acked Read 5 Sent Done Done false
1976 1422 Acked Write 7 Sent Done Done false
1977 1423 Acked Read 9 Sent Done Done false
1978 1424 Acked Write 10 Sent Done Done false
1979 1425 Acked Read 7 Sent Done Done false
1980 1426 Acked Read 7 Sent Done Done false
1981 1427 Acked Read 4 Sent Done Done false
1982 1428 Acked Read 10 Sent Done Done false
1983 1429 Acked Write 3 Sent Done Done false
1984 1430 Acked Read 7 Sent Done Done false
1985 1431 Acked Write 8 Sent Done Done false
1986 1432 Acked Read 10 Sent Done Done false
1987 1433 Acked Write 2 Sent Done Done false
1988 1434 Acked Write 10 Sent Done Done false
1989 1435 Acked Write 7 Sent Done Done false
1990 1436 Acked Read 9 Sent Done Done false
1991 1437 Acked Flush 0 Sent Done Done false
1992 1438 Acked Read 7 Sent Done Done false
1993 1439 Acked Write 4 Sent Done Done false
1994 1440 Acked Write 7 Sent Done Done false
1995 1441 Acked Write 5 Sent Done Done false
1996 1442 Acked Read 10 Sent Done Done false
1997 1443 Acked Flush 0 Sent Done Done false
1998 1444 Acked Write 7 Sent Done Done false
1999 1445 Acked Read 8 Sent Done Done false
2000 1446 Acked Write 6 Sent Done Done false
2001 1447 Acked Read 6 Sent Done Done false
2002 1448 Acked Read 1 Sent Done Done false
2003 1449 Acked Flush 0 Sent Done Done false
2004 1450 Acked Read 10 Sent Done Done false
2005 1451 Acked Read 6 Sent Done Done false
2006 1452 Acked Read 9 Sent Done Done false
2007 1453 Acked Flush 0 Sent Done Done false
2008 1454 Acked Read 5 Sent Done Done false
2009 1455 Acked Write 2 Sent Done Done false
2010 1456 Acked Write 6 Sent Done Done false
2011 1457 Acked Read 6 Sent Done Done false
2012 1458 Acked Read 3 Sent Done Done false
2013 1459 Acked Write 6 Sent Done Done false
2014 1460 Acked Write 6 Sent Done Done false
2015 1461 Acked Read 8 Sent Done Done false
2016 1462 Acked Read 1 Sent Done Done false
2017 1463 Acked Read 9 Sent Done Done false
2018 1464 Acked Write 10 Sent Done Done false
2019 1465 Acked Write 6 Sent Done Done false
2020 1466 Acked Read 8 Sent Done Done false
2021 1467 Acked Write 9 Sent Done Done false
2022 1468 Acked Read 1 Sent Done Done false
2023 1469 Acked Write 4 Sent Done Done false
2024 1470 Acked Read 8 Sent Done Done false
2025 1471 Acked Write 7 Sent Done Done false
2026 1472 Acked Write 9 Sent Done Done false
2027 1473 Acked Read 3 Sent Done Done false
2028 1474 Acked Read 8 Sent Done Done false
2029 1475 Acked Write 6 Sent Done Done false
2030 1476 Acked Read 2 Sent Done Done false
2031 1477 Acked Read 8 Sent Done Done false
2032 1478 Acked Read 4 Sent Done Done false
2033 1479 Acked Write 7 Sent Done Done false
2034 1480 Acked Write 10 Sent Done Done false
2035 1481 Acked Read 3 Sent Done Done false
2036 1482 Acked Write 1 Sent Done Done false
2037 1483 Acked Write 9 Sent Done Done false
2038 1484 Acked Flush 0 Sent Done Done false
2039 1485 Acked Write 4 Sent Done Done false
2040 1486 Acked Read 1 Sent Done Done false
2041 1487 Acked Read 9 Sent Done Done false
2042 1488 Acked Write 10 Sent Done Done false
2043 1489 Acked Read 10 Sent Done Done false
2044 1490 Acked Write 8 Sent Done Done false
2045 1491 Acked Write 5 Sent Done Done false
2046 1492 Acked Read 7 Sent Done Done false
2047 1493 Acked Write 9 Sent Done Done false
2048 1494 Acked Flush 0 Sent Done Done false
2049 1495 Acked Flush 0 Sent Done Done false
2050 1496 Acked Read 7 Sent Done Done false
2051 1497 Acked Read 6 Sent Done Done false
2052 1498 Acked Read 9 Sent Done Done false
2053 1499 Acked Read 5 Sent Done Done false
2054 1500 Acked Read 4 Sent Done Done false
2055 1501 Acked Write 8 Sent Done Done false
2056 1502 Acked Write 9 Sent Done Done false
2057 1503 Acked Write 7 Sent Done Done false
2058 1504 Acked Write 3 Sent Done Done false
2059 1505 Acked Write 5 Sent Done Done false
2060 1506 Acked Read 1 Sent Done Done false
2061 1507 Acked Write 4 Sent Done Done false
2062 1508 Acked Read 2 Sent Done Done false
2063 1509 Acked Read 3 Sent Done Done false
2064 1510 Acked Read 9 Sent Done Done false
2065 1511 Acked Read 8 Sent Done Done false
2066 1512 Acked Flush 0 Sent Done Done false
2067 1513 Acked Read 9 Sent Done Done false
2068 1514 Acked Write 5 Sent Done Done false
2069 1515 Acked Write 8 Sent Done Done false
2070 1516 Acked Read 5 Sent Done Done false
2071 1517 Acked Read 7 Sent Done Done false
2072 1518 Acked Read 9 Sent Done Done false
2073 1519 Acked Write 1 Sent Done Done false
2074 1520 Acked Flush 0 Sent Done Done false
2075 1521 Acked Write 8 Sent Done Done false
2076 1522 Acked Read 4 Sent Done Done false
2077 1523 Acked Write 9 Sent Done Done false
2078 1524 Acked Write 6 Sent Done Done false
2079 1525 Acked Read 10 Sent Done Done false
2080 1526 Acked Flush 0 Sent Done Done false
2081 1527 Acked Read 4 Sent Done Done false
2082 1528 Acked Read 1 Sent Done Done false
2083 1529 Acked Read 2 Sent Done Done false
2084 1530 Acked Read 4 Sent Done Done false
2085 1531 Acked Write 4 Sent Done Done false
2086 1532 Acked Read 7 Sent Done Done false
2087 1533 Acked Flush 0 Sent Done Done false
2088 1534 Acked Write 10 Sent Done Done false
2089 1535 Acked Write 7 Sent Done Done false
2090 1536 Acked Read 7 Sent Done Done false
2091 1537 Acked Read 10 Sent Done Done false
2092 1538 Acked Write 8 Sent Done Done false
2093 1539 Acked Read 1 Sent Done Done false
2094 1540 Acked Read 5 Sent Done Done false
2095 1541 Acked Write 5 Sent Done Done false
2096 1542 Acked Read 8 Sent Done Done false
2097 1543 Acked Write 7 Sent Done Done false
2098 1544 Acked Read 8 Sent Done Done false
2099 1545 Acked Read 4 Sent Done Done false
2100 1546 Acked Flush 0 Sent Done Done false
2101 1547 Acked Flush 0 Sent Done Done false
2102 1548 Acked Write 1 Sent Done Done false
2103 1549 Acked Write 8 Sent Done Done false
2104 1550 Acked Read 7 Sent Done Done false
2105 1551 Acked Read 9 Sent Done Done false
2106 1552 Acked Read 2 Sent Done Done false
2107 1553 Acked Write 9 Sent Done Done false
2108 1554 Acked Read 1 Sent Done Done false
2109 1555 Acked Read 6 Sent Done Done false
2110 1556 Acked Read 4 Sent Done Done false
2111 1557 Acked Read 10 Sent Done Done false
2112 1558 Acked Read 1 Sent Done Done false
2113 1559 Acked Read 5 Sent Done Done false
2114 1560 Acked Read 9 Sent Done Done false
2115 1561 Acked Write 3 Sent Done Done false
2116 1562 Acked Write 1 Sent Done Done false
2117 1563 Acked Write 9 Sent Done Done false
2118 1564 Acked Read 8 Sent Done Done false
2119 1565 Acked Write 4 Sent Done Done false
2120 1566 Acked Write 1 Sent Done Done false
2121 1567 Acked Write 1 Sent Done Done false
2122 1568 Acked Read 10 Sent Done Done false
2123 1569 Acked Read 10 Sent Done Done false
2124 1570 Acked Read 5 Sent Done Done false
2125 1571 Acked Flush 0 Sent Done Done false
2126 1572 Acked Write 8 Sent Done Done false
2127 1573 Acked Write 9 Sent Done Done false
2128 1574 Acked Read 6 Sent Done Done false
2129 1575 Acked Write 4 Sent Done Done false
2130 1576 Acked Flush 0 Sent Done Done false
2131 1577 Acked Flush 0 Sent Done Done false
2132 1578 Acked Read 1 Sent Done Done false
2133 1579 Acked Read 9 Sent Done Done false
2134 1580 Acked Write 7 Sent Done Done false
2135 1581 Acked Write 10 Sent Done Done false
2136 1582 Acked Write 10 Sent Done Done false
2137 1583 Acked Write 3 Sent Done Done false
2138 1584 Acked Read 9 Sent Done Done false
2139 1585 Acked Read 8 Sent Done Done false
2140 1586 Acked Write 2 Sent Done Done false
2141 1587 Acked Read 1 Sent Done Done false
2142 1588 Acked Read 3 Sent Done Done false
2143 1589 Acked Write 8 Sent Done Done false
2144 1590 Acked Read 1 Sent Done Done false
2145 1591 Acked Read 7 Sent Done Done false
2146 1592 Acked Flush 0 Sent Done Done false
2147 1593 Acked Write 3 Sent Done Done false
2148 1594 Acked Read 6 Sent Done Done false
2149 1595 Acked Write 1 Sent Done Done false
2150 1596 Acked Read 10 Sent Done Done false
2151 1597 Acked Write 7 Sent Done Done false
2152 1598 Acked Read 2 Sent Done Done false
2153 1599 Acked Write 9 Sent Done Done false
2154 1600 Acked Write 5 Sent Done Done false
2155 1601 Acked Read 4 Sent Done Done false
2156 1602 Acked Flush 0 Sent Done Done false
2157 STATES DS:0 DS:1 DS:2 TOTAL
2158 Sent 301 0 0 301
2159 Done 0 301 301 602
2160 Skipped 0 0 0 0
2161 Error 0 0 0 0
2162 Last Flush: 1301 1602 1602
2163 Downstairs last five completed: 1301 1300 1299 1298 1297
2164 Upstairs last five completed: 1602 1601 1600 1599 1598
2165 CLIENT: Up:0 ds:301 act:2
21662024-11-15T01:57:54.092ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
21672024-11-15T01:57:54.092ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:41812) = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
21682024-11-15T01:57:54.093ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
21692024-11-15T01:57:54.094ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
21702024-11-15T01:57:54.094ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
21712024-11-15T01:57:54.094ZINFOcrucible: send last flush ID to this DS: 1301 = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
21722024-11-15T01:57:54.094ZINFOcrucible: Replied this last flush ID: 1301 = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
21732024-11-15T01:57:54.094ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
21742024-11-15T01:57:54.094ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
21752024-11-15T01:57:54.094ZINFOcrucible: [0] client re-new 301 jobs since flush 1301 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
21762024-11-15T01:57:54.095ZINFOcrucible: [0] Replayed 301 jobs since flush: 1301 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
2177 ----------------------------------------------------------------
2178 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
2179 Downstairs last five completed: 1602 1601 1600 1599 1598
2180 Upstairs last five completed: 1602 1601 1600 1599 1598
2181 CLIENT: Up:0 ds:0 act:3
2182 Replay: All jobs finished
2183 Main task has work to do, go find it
2184 got dsc Stop(1)
2185 stop 1
2186 Nov 15 01:57:56.339 INFO request completed, latency_us: 347, response_code:[8820] Got stop action so:false kr:false
2187 204, uri: /stop/cid/1, method: POST, req_id: 5744e884-1ca0-4932-bc90-a528d30541a8, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
21882024-11-15T01:57:56.339ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
21892024-11-15T01:57:56.339ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
21902024-11-15T01:57:56.339ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
21912024-11-15T01:57:56.339ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
21922024-11-15T01:57:56.340ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
2193 Nov 15 01:57:56.340 INFO request completed, latency_us: 180, response_code: 200, uri: /state/cid/1, method: GET, req_id: 824c33b5-1716-47a3-8f20-668d7c94ac55, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
21942024-11-15T01:57:56.340ZINFOcrucible: client task is exiting = io task client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
2195 [8820] Exited with: ExitStatus(unix_wait_status(9))
2196 [8820][1] reports Exit
2197 Nov 15 01:58:00.343 INFO request completed, latency_us: 236, response_code: 200, uri: /state/cid/1, method: GET, req_id: 89fdd457-e683-4e48-91ac-89f35b51e570, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
2198 001/300 Read block 2104 len 12288
2199 002/300 Read block 2854 len 28672
2200 003/300 Read block 1615 len 40960
2201 004/300 Write block 4129 len 32768 data: 2 2 2 2 2 2 2 2
2202 005/300 Read block 2811 len 32768
2203 006/300 Flush
2204 007/300 Read block 2793 len 24576
2205 008/300 Write block 2773 len 8192 data: 3 3
2206 009/300 Write block 2950 len 20480 data: 3 4 4 3 4
2207 010/300 Write block 4513 len 20480 data: 3 4 4 5 4
2208 011/300 Write block 2739 len 8192 data: 2 2
2209 012/300 Write block 4323 len 4096 data: 2
2210 013/300 Write block 3039 len 32768 data: 2 2 2 2 2 2 2 2
2211 014/300 Read block 1380 len 28672
2212 015/300 Read block 3851 len 36864
2213 016/300 Write block 2981 len 32768 data: 2 2 2 2 2 2 2 2
2214 017/300 Write block 2285 len 8192 data: 3 3
2215 018/300 Write block 2269 len 40960 data: 2 2 2 2 3 3 2 2 2 2
2216 019/300 Read block 824 len 4096
2217 020/300 Flush
2218 021/300 Read block 1922 len 16384
2219 022/300 Write block 3134 len 20480 data: 2 2 2 2 2
2220 023/300 Flush
2221 024/300 Write block 4839 len 36864 data: 2 2 2 2 2 2 2 2 2
2222 025/300 Write block 3434 len 12288 data: 2 2 2
2223 026/300 Read block 2499 len 24576
2224 027/300 Flush
2225 028/300 Write block 4838 len 4096 data: 2
2226 029/300 Read block 1612 len 28672
2227 030/300 Read block 2 len 40960
2228 031/300 Read block 568 len 12288
2229 032/300 Read block 2175 len 36864
2230 033/300 Flush
2231 034/300 Write block 493 len 28672 data: 2 2 2 2 3 3 3
2232 035/300 Write block 4778 len 28672 data: 3 2 2 2 2 2 2
2233 036/300 Write block 1083 len 4096 data: 2
2234 037/300 Flush
2235 038/300 Read block 165 len 32768
2236 039/300 Read block 1562 len 24576
2237 040/300 Write block 1782 len 36864 data: 3 3 2 2 2 3 2 2 2
2238 041/300 Write block 1280 len 16384 data: 2 2 2 2
2239 042/300 Read block 2666 len 24576
2240 043/300 Read block 4916 len 28672
2241 044/300 Read block 2388 len 8192
2242 045/300 Write block 3631 len 8192 data: 2 2
2243 046/300 Flush
2244 047/300 Read block 4287 len 36864
2245 048/300 Write block 3073 len 20480 data: 2 2 3 4 5
2246 049/300 Flush
2247 050/300 Read block 1093 len 24576
2248 051/300 Write block 3901 len 24576 data: 2 2 2 2 2 2
2249 052/300 Write block 3935 len 24576 data: 2 2 2 2 2 2
2250 053/300 Read block 395 len 8192
2251 054/300 Write block 2003 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2252 055/300 Write block 284 len 20480 data: 2 2 2 2 2
2253 056/300 Write block 2490 len 4096 data: 2
2254 057/300 Write block 1119 len 28672 data: 2 2 3 3 3 3 3
2255 058/300 Read block 1050 len 24576
2256 059/300 Write block 4451 len 4096 data: 2
2257 060/300 Read block 2933 len 32768
2258 061/300 Read block 972 len 16384
2259 062/300 Read block 2870 len 28672
2260 063/300 Read block 2780 len 28672
2261 064/300 Flush
2262 065/300 Read block 4855 len 12288
2263 066/300 Write block 4140 len 4096 data: 2
2264 067/300 Read block 2373 len 20480
2265 068/300 Write block 2283 len 28672 data: 2 3 4 4 3 2 2
2266 069/300 Read block 3939 len 40960
2267 070/300 Read block 2444 len 8192
2268 071/300 Read block 724 len 24576
2269 072/300 Read block 4281 len 24576
2270 073/300 Write block 3259 len 20480 data: 3 3 4 3 2
2271 074/300 Read block 4734 len 36864
2272 075/300 Read block 4908 len 28672
2273 076/300 Write block 2307 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2274 077/300 Write block 1462 len 4096 data: 2
2275 078/300 Read block 1237 len 32768
2276 079/300 Read block 3339 len 32768
2277 080/300 Flush
2278 081/300 Write block 3919 len 36864 data: 2 2 2 2 2 2 2 2 2
2279 082/300 Read block 330 len 4096
2280 083/300 Write block 2460 len 24576 data: 2 2 2 2 2 2
2281 084/300 Write block 4902 len 12288 data: 2 2 2
2282 085/300 Read block 2377 len 4096
2283 086/300 Read block 608 len 12288
2284 087/300 Write block 225 len 32768 data: 2 2 2 2 3 3 3 3
2285 088/300 Write block 1464 len 32768 data: 2 2 2 2 2 2 2 2
2286 089/300 Read block 3585 len 4096
2287 090/300 Write block 582 len 28672 data: 2 2 2 2 2 3 4
2288 091/300 Read block 2627 len 28672
2289 092/300 Read block 1325 len 8192
2290 093/300 Write block 3897 len 32768 data: 2 2 2 2 3 3 3 3
2291 094/300 Write block 530 len 24576 data: 3 3 2 2 2 2
2292 095/300 Read block 2653 len 32768
2293 096/300 Write block 2633 len 12288 data: 3 2 2
2294 097/300 Write block 2647 len 32768 data: 2 2 2 3 3 2 2 2
2295 098/300 Read block 1128 len 36864
2296 099/300 Write block 4730 len 8192 data: 2 2
2297 100/300 Read block 475 len 4096
2298 101/300 Write block 1320 len 12288 data: 2 2 2
2299 102/300 Write block 2544 len 20480 data: 2 2 2 2 2
2300 103/300 Write block 2569 len 32768 data: 2 2 2 2 2 2 2 4
2301 104/300 Read block 241 len 28672
2302 105/300 Write block 3342 len 8192 data: 2 3
2303 106/300 Read block 4702 len 32768
2304 107/300 Read block 4918 len 8192
2305 108/300 Read block 2963 len 20480
2306 109/300 Read block 1924 len 4096
2307 110/300 Write block 3687 len 4096 data: 2
2308 111/300 Read block 3416 len 12288
2309 112/300 Flush
2310 113/300 Flush
2311 114/300 Write block 4394 len 8192 data: 2 2
2312 115/300 Read block 4267 len 24576
2313 116/300 Read block 1306 len 8192
2314 117/300 Write block 4682 len 24576 data: 2 2 2 2 2 2
2315 118/300 Write block 701 len 20480 data: 2 2 2 2 2
2316 119/300 Flush
2317 120/300 Write block 1662 len 20480 data: 2 2 2 2 2
2318 121/300 Flush
2319 122/300 Write block 4278 len 4096 data: 2
2320 123/300 Read block 4415 len 8192
2321 124/300 Write block 4781 len 16384 data: 3 3 3 3
2322 125/300 Read block 4066 len 32768
2323 126/300 Write block 54 len 20480 data: 2 2 2 2 2
2324 127/300 Read block 2017 len 16384
2325 128/300 Flush
2326 129/300 Read block 616 len 20480
2327 130/300 Write block 1420 len 28672 data: 2 2 2 2 2 2 2
2328 131/300 Read block 2998 len 32768
2329 132/300 Read block 1242 len 24576
2330 133/300 Read block 999 len 24576
2331 134/300 Read block 2862 len 32768
2332 135/300 Write block 667 len 8192 data: 3 3
2333 136/300 Read block 1411 len 8192
2334 137/300 Read block 2329 len 20480
2335 138/300 Write block 3701 len 12288 data: 3 3 3
2336 139/300 Read block 312 len 28672
2337 140/300 Write block 1365 len 28672 data: 2 2 2 2 2 2 2
2338 141/300 Write block 3581 len 8192 data: 3 2
2339 142/300 Write block 1958 len 8192 data: 2 2
2340 143/300 Write block 772 len 16384 data: 3 3 3 2
2341 144/300 Read block 3195 len 40960
2342 145/300 Write block 173 len 40960 data: 3 3 2 2 2 2 2 2 2 2
2343 146/300 Read block 4882 len 24576
2344 147/300 Write block 1227 len 12288 data: 2 2 2
2345 148/300 Write block 577 len 12288 data: 2 2 2
2346 149/300 Write block 2325 len 4096 data: 2
2347 150/300 Read block 1122 len 24576
2348 151/300 Read block 3014 len 12288
2349 152/300 Write block 3739 len 36864 data: 4 4 3 3 2 2 2 2 2
2350 153/300 Read block 1934 len 28672
2351 154/300 Read block 3256 len 32768
2352 155/300 Flush
2353 156/300 Read block 4988 len 4096
2354 157/300 Read block 1073 len 40960
2355 158/300 Read block 3017 len 28672
2356 159/300 Read block 975 len 16384
2357 160/300 Read block 2682 len 12288
2358 161/300 Read block 507 len 4096
2359 162/300 Write block 1741 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2360 163/300 Write block 3079 len 12288 data: 5 6 5
2361 164/300 Write block 2125 len 24576 data: 2 2 3 3 4 4
2362 165/300 Read block 1449 len 4096
2363 166/300 Flush
2364 167/300 Read block 1245 len 32768
2365 168/300 Write block 4108 len 28672 data: 2 2 2 2 2 2 2
2366 169/300 Write block 2078 len 20480 data: 3 3 3 3 2
2367 170/300 Write block 4604 len 12288 data: 2 2 2
2368 171/300 Flush
2369 172/300 Read block 297 len 36864
2370 173/300 Write block 490 len 12288 data: 2 2 2
2371 174/300 Write block 2130 len 16384 data: 5 4 3 3
2372 175/300 Flush
2373 176/300 Read block 4781 len 28672
2374 177/300 Read block 1409 len 20480
2375 178/300 Flush
2376 179/300 Read block 3071 len 12288
2377 180/300 Read block 2854 len 36864
2378 181/300 Flush
2379 182/300 Write block 4903 len 8192 data: 3 3
2380 183/300 Read block 1323 len 36864
2381 184/300 Write block 992 len 36864 data: 3 3 3 2 3 3 3 3 3
2382 185/300 Read block 2550 len 28672
2383 186/300 Read block 2450 len 40960
2384 187/300 Write block 4190 len 36864 data: 2 2 2 2 2 2 2 2 2
2385 188/300 Write block 2491 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2386 189/300 Write block 214 len 20480 data: 2 2 2 2 2
2387 190/300 Write block 4900 len 28672 data: 2 2 3 4 4 2 2
2388 191/300 Read block 4173 len 12288
2389 192/300 Read block 3957 len 24576
2390 193/300 Read block 3415 len 28672
2391 194/300 Write block 2527 len 12288 data: 2 2 2
2392 195/300 Flush
2393 196/300 Write block 2757 len 24576 data: 2 2 2 2 2 2
2394 197/300 Read block 4649 len 20480
2395 198/300 Write block 2359 len 20480 data: 2 3 3 3 2
2396 199/300 Read block 4223 len 36864
2397 200/300 Write block 1410 len 20480 data: 2 2 2 2 2
2398 201/300 Write block 3264 len 8192 data: 2 2
2399 202/300 Read block 2247 len 36864
2400 203/300 Flush
2401 204/300 Write block 4020 len 36864 data: 2 2 2 2 2 2 2 2 2
2402 205/300 Write block 758 len 8192 data: 5 4
2403 206/300 Write block 3531 len 36864 data: 3 3 3 2 2 2 2 2 2
2404 207/300 Write block 1015 len 40960 data: 3 3 3 3 3 4 4 2 2 2
2405 208/300 Flush
2406 209/300 Read block 1129 len 28672
2407 210/300 Write block 2229 len 20480 data: 2 2 2 2 2
2408 211/300 Flush
2409 212/300 Write block 1440 len 28672 data: 2 2 2 2 2 2 2
2410 213/300 Read block 3676 len 28672
2411 214/300 Write block 1405 len 16384 data: 2 2 2 2
2412 215/300 Write block 3548 len 24576 data: 2 2 2 2 2 2
2413 216/300 Flush
2414 217/300 Read block 2026 len 16384
2415 218/300 Write block 1295 len 28672 data: 3 3 3 3 3 3 3
2416 219/300 Read block 1883 len 32768
2417 220/300 Write block 4407 len 16384 data: 2 2 2 2
2418 221/300 Flush
2419 222/300 Flush
2420 223/300 Read block 284 len 12288
2421 224/300 Write block 1368 len 40960 data: 3 3 3 3 2 2 2 2 2 2
2422 225/300 Write block 4841 len 12288 data: 3 3 3
2423 226/300 Flush
2424 227/300 Write block 1743 len 20480 data: 3 3 3 3 3
2425 228/300 Read block 4023 len 40960
2426 229/300 Read block 2331 len 16384
2427 230/300 Read block 3143 len 40960
2428 231/300 Write block 4039 len 32768 data: 2 2 2 2 2 2 2 3
2429 232/300 Read block 839 len 16384
2430 233/300 Read block 488 len 28672
2431 234/300 Read block 2080 len 28672
2432 235/300 Read block 3061 len 4096
2433 236/300 Read block 557 len 32768
2434 237/300 Flush
2435 238/300 Write block 663 len 4096 data: 2
2436 239/300 Read block 4133 len 4096
2437 240/300 Flush
2438 241/300 Write block 918 len 12288 data: 2 2 3
2439 242/300 Flush
2440 243/300 Read block 1916 len 16384
2441 244/300 Write block 3628 len 20480 data: 2 2 2 3 3
2442 245/300 Write block 4018 len 40960 data: 2 2 3 3 3 3 3 3 3 3
2443 246/300 Read block 1542 len 40960
2444 247/300 Read block 2793 len 28672
2445 248/300 Read block 3793 len 16384
2446 249/300 Read block 4528 len 40960
2447 250/300 Write block 813 len 12288 data: 2 2 2
2448 251/300 Read block 3217 len 36864
2449 252/300 Read block 4793 len 20480
2450 253/300 Write block 1444 len 16384 data: 3 3 3 2
2451 254/300 Read block 4664 len 16384
2452 255/300 Read block 2381 len 12288
2453 256/300 Flush
2454 257/300 Read block 3258 len 28672
2455 258/300 Read block 4589 len 32768
2456 259/300 Read block 3155 len 24576
2457 260/300 Read block 1484 len 8192
2458 261/300 Read block 4674 len 36864
2459 262/300 Write block 2566 len 28672 data: 3 3 2 3 3 3 3
2460 263/300 Read block 3655 len 28672
2461 264/300 Read block 752 len 8192
2462 265/300 Write block 2696 len 32768 data: 2 2 2 2 2 2 2 2
2463 266/300 Read block 3463 len 12288
2464 267/300 Write block 3177 len 4096 data: 2
2465 268/300 Write block 3743 len 36864 data: 3 3 3 3 3 2 2 3 4
2466 269/300 Read block 1690 len 40960
2467 270/300 Flush
2468 271/300 Flush
2469 272/300 Read block 723 len 36864
2470 273/300 Write block 764 len 20480 data: 2 2 2 2 2
2471 274/300 Read block 3373 len 20480
2472 275/300 Read block 2887 len 8192
2473 276/300 Write block 4646 len 28672 data: 2 2 2 2 2 2 2
2474 277/300 Flush
2475 278/300 Write block 3408 len 36864 data: 4 4 2 2 2 2 2 2 2
2476 279/300 Read block 2708 len 36864
2477 280/300 Write block 4525 len 24576 data: 3 3 3 4 3 3
2478 281/300 Read block 2730 len 4096
2479 282/300 Read block 3426 len 20480
2480 283/300 Flush
2481 284/300 Write block 4493 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2482 285/300 Write block 4776 len 28672 data: 3 3 4 3 3 4 4
2483 286/300 Read block 2856 len 24576
2484 287/300 Write block 3798 len 36864 data: 2 3 2 2 2 2 2 2 2
2485 288/300 Write block 38 len 28672 data: 2 2 3 3 3 3 3
2486 289/300 Write block 1658 len 16384 data: 2 2 3 3
2487 290/300 Read block 3329 len 16384
2488 291/300 Read block 2851 len 20480
2489 292/300 Write block 4299 len 32768 data: 2 2 3 3 3 3 3 3
2490 293/300 Write block 3009 len 32768 data: 2 2 2 2 2 2 2 2
2491 294/300 Write block 4991 len 16384 data: 2 2 2 2
2492 295/300 Read block 4332 len 8192
2493 296/300 Write block 1678 len 36864 data: 2 2 2 3 3 3 3 3 3
2494 297/300 Read block 1730 len 8192
2495 298/300 Read block 4063 len 36864
2496 299/300 Write block 256 len 24576 data: 2 2 2 2 2 2
2497 300/300 Read block 156 len 32768
2498 301/300 Flush
2499 Main task has work to do, go find it
2500 got dsc Start(1)
2501 start 1
2502 Nov 15 01:58:00.573 INFO request completed, latency_us: 208, response_code: 204, uri: /start/cid/1, method: POST, req_id: 07a66168-ddbc-4bfc-80f4-fb95f74362a5, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
2503 Replay: started 1, returned:Ok(())
2504 ----------------------------------------------------------------
2505 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
2506 States: Active Offline Active
2507 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
2508 1603 Acked Read 3 Done Sent Done false
2509 1604 Acked Read 7 Done Sent Done false
2510 1605 Acked Read 10 Done Sent Done false
2511 1606 Acked Write 8 Done Sent Done false
2512 1607 Acked Read 8 Done Sent Done false
2513 1608 Acked Flush 0 Done Sent Done false
2514 1609 Acked Read 6 Done Sent Done false
2515 1610 Acked Write 2 Done Sent Done false
2516 1611 Acked Write 5 Done Sent Done false
2517 1612 Acked Write 5 Done Sent Done false
2518 1613 Acked Write 2 Done Sent Done false
2519 1614 Acked Write 1 Done Sent Done false
2520 1615 Acked Write 8 Done Sent Done false
2521 1616 Acked Read 7 Done Sent Done false
2522 1617 Acked Read 9 Done Sent Done false
2523 1618 Acked Write 8 Done Sent Done false
2524 1619 Acked Write 2 Done Sent Done false
2525 1620 Acked Write 10 Done Sent Done false
2526 1621 Acked Read 1 Done Sent Done false
2527 1622 Acked Flush 0 Done Sent Done false
2528 1623 Acked Read 4 Done Sent Done false
2529 1624 Acked Write 5 Done Sent Done false
2530 1625 Acked Flush 0 Done Sent Done false
2531 1626 Acked Write 9 Done Sent Done false
2532 1627 Acked Write 3 Done Sent Done false
2533 1628 Acked Read 6 Done Sent Done false
2534 1629 Acked Flush 0 Done Sent Done false
2535 1630 Acked Write 1 Done Sent Done false
2536 1631 Acked Read 7 Done Sent Done false
2537 1632 Acked Read 10 Done Sent Done false
2538 1633 Acked Read 3 Done Sent Done false
2539 1634 Acked Read 9 Done Sent Done false
2540 1635 Acked Flush 0 Done Sent Done false
2541 1636 Acked Write 7 Done Sent Done false
2542 1637 Acked Write 7 Done Sent Done false
2543 1638 Acked Write 1 Done Sent Done false
2544 1639 Acked Flush 0 Done Sent Done false
2545 1640 Acked Read 8 Done Sent Done false
2546 1641 Acked Read 6 Done Sent Done false
2547 1642 Acked Write 9 Done Sent Done false
2548 1643 Acked Write 4 Done Sent Done false
2549 1644 Acked Read 6 Done Sent Done false
2550 1645 Acked Read 7 Done Sent Done false
2551 1646 Acked Read 2 Done Sent Done false
2552 1647 Acked Write 2 Done Sent Done false
2553 1648 Acked Flush 0 Done Sent Done false
2554 1649 Acked Read 9 Done Sent Done false
2555 1650 Acked Write 5 Done Sent Done false
2556 1651 Acked Flush 0 Done Sent Done false
2557 1652 Acked Read 6 Done Sent Done false
2558 1653 Acked Write 6 Done Sent Done false
2559 1654 Acked Write 6 Done Sent Done false
2560 1655 Acked Read 2 Done Sent Done false
2561 1656 Acked Write 10 Done Sent Done false
2562 1657 Acked Write 5 Done Sent Done false
2563 1658 Acked Write 1 Done Sent Done false
2564 1659 Acked Write 7 Done Sent Done false
2565 1660 Acked Read 6 Done Sent Done false
2566 1661 Acked Write 1 Done Sent Done false
2567 1662 Acked Read 8 Done Sent Done false
2568 1663 Acked Read 4 Done Sent Done false
2569 1664 Acked Read 7 Done Sent Done false
2570 1665 Acked Read 7 Done Sent Done false
2571 1666 Acked Flush 0 Done Sent Done false
2572 1667 Acked Read 3 Done Sent Done false
2573 1668 Acked Write 1 Done Sent Done false
2574 1669 Acked Read 5 Done Sent Done false
2575 1670 Acked Write 7 Done Sent Done false
2576 1671 Acked Read 10 Done Sent Done false
2577 1672 Acked Read 2 Done Sent Done false
2578 1673 Acked Read 6 Done Sent Done false
2579 1674 Acked Read 6 Done Sent Done false
2580 1675 Acked Write 5 Done Sent Done false
2581 1676 Acked Read 9 Done Sent Done false
2582 1677 Acked Read 7 Done Sent Done false
2583 1678 Acked Write 10 Done Sent Done false
2584 1679 Acked Write 1 Done Sent Done false
2585 1680 Acked Read 8 Done Sent Done false
2586 1681 Acked Read 8 Done Sent Done false
2587 1682 Acked Flush 0 Done Sent Done false
2588 1683 Acked Write 9 Done Sent Done false
2589 1684 Acked Read 1 Done Sent Done false
2590 1685 Acked Write 6 Done Sent Done false
2591 1686 Acked Write 3 Done Sent Done false
2592 1687 Acked Read 1 Done Sent Done false
2593 1688 Acked Read 3 Done Sent Done false
2594 1689 Acked Write 8 Done Sent Done false
2595 1690 Acked Write 8 Done Sent Done false
2596 1691 Acked Read 1 Done Sent Done false
2597 1692 Acked Write 7 Done Sent Done false
2598 1693 Acked Read 7 Done Sent Done false
2599 1694 Acked Read 2 Done Sent Done false
2600 1695 Acked Write 8 Done Sent Done false
2601 1696 Acked Write 6 Done Sent Done false
2602 1697 Acked Read 8 Done Sent Done false
2603 1698 Acked Write 3 Done Sent Done false
2604 1699 Acked Write 8 Done Sent Done false
2605 1700 Acked Read 9 Done Sent Done false
2606 1701 Acked Write 2 Done Sent Done false
2607 1702 Acked Read 1 Done Sent Done false
2608 1703 Acked Write 3 Done Sent Done false
2609 1704 Acked Write 5 Done Sent Done false
2610 1705 Acked Write 8 Done Sent Done false
2611 1706 Acked Read 7 Done Sent Done false
2612 1707 Acked Write 2 Done Sent Done false
2613 1708 Acked Read 8 Done Sent Done false
2614 1709 Acked Read 2 Done Sent Done false
2615 1710 Acked Read 5 Done Sent Done false
2616 1711 Acked Read 1 Done Sent Done false
2617 1712 Acked Write 1 Done Sent Done false
2618 1713 Acked Read 3 Done Sent Done false
2619 1714 Acked Flush 0 Done Sent Done false
2620 1715 Acked Flush 0 Done Sent Done false
2621 1716 Acked Write 2 Done Sent Done false
2622 1717 Acked Read 6 Done Sent Done false
2623 1718 Acked Read 2 Done Sent Done false
2624 1719 Acked Write 6 Done Sent Done false
2625 1720 Acked Write 5 Done Sent Done false
2626 1721 Acked Flush 0 Done Sent Done false
2627 1722 Acked Write 5 Done Sent Done false
2628 1723 Acked Flush 0 Done Sent Done false
2629 1724 Acked Write 1 Done Sent Done false
2630 1725 Acked Read 2 Done Sent Done false
2631 1726 Acked Write 4 Done Sent Done false
2632 1727 Acked Read 8 Done Sent Done false
2633 1728 Acked Write 5 Done Sent Done false
2634 1729 Acked Read 4 Done Sent Done false
2635 1730 Acked Flush 0 Done Sent Done false
2636 1731 Acked Read 5 Done Sent Done false
2637 1732 Acked Write 7 Done Sent Done false
2638 1733 Acked Read 8 Done Sent Done false
2639 1734 Acked Read 6 Done Sent Done false
2640 1735 Acked Read 6 Done Sent Done false
2641 1736 Acked Read 8 Done Sent Done false
2642 1737 Acked Write 2 Done Sent Done false
2643 1738 Acked Read 2 Done Sent Done false
2644 1739 Acked Read 5 Done Sent Done false
2645 1740 Acked Write 3 Done Sent Done false
2646 1741 Acked Read 7 Done Sent Done false
2647 1742 Acked Write 7 Done Sent Done false
2648 1743 Acked Write 2 Done Sent Done false
2649 1744 Acked Write 2 Done Sent Done false
2650 1745 Acked Write 4 Done Sent Done false
2651 1746 Acked Read 10 Done Sent Done false
2652 1747 Acked Write 10 Done Sent Done false
2653 1748 Acked Read 6 Done Sent Done false
2654 1749 Acked Write 3 Done Sent Done false
2655 1750 Acked Write 3 Done Sent Done false
2656 1751 Acked Write 1 Done Sent Done false
2657 1752 Acked Read 6 Done Sent Done false
2658 1753 Acked Read 3 Done Sent Done false
2659 1754 Acked Write 9 Done Sent Done false
2660 1755 Acked Read 7 Done Sent Done false
2661 1756 Acked Read 8 Done Sent Done false
2662 1757 Acked Flush 0 Done Sent Done false
2663 1758 Acked Read 1 Done Sent Done false
2664 1759 Acked Read 10 Done Sent Done false
2665 1760 Acked Read 7 Done Sent Done false
2666 1761 Acked Read 4 Done Sent Done false
2667 1762 Acked Read 3 Done Sent Done false
2668 1763 Acked Read 1 Done Sent Done false
2669 1764 Acked Write 10 Done Sent Done false
2670 1765 Acked Write 3 Done Sent Done false
2671 1766 Acked Write 6 Done Sent Done false
2672 1767 Acked Read 1 Done Sent Done false
2673 1768 Acked Flush 0 Done Sent Done false
2674 1769 Acked Read 8 Done Sent Done false
2675 1770 Acked Write 7 Done Sent Done false
2676 1771 Acked Write 5 Done Sent Done false
2677 1772 Acked Write 3 Done Sent Done false
2678 1773 Acked Flush 0 Done Sent Done false
2679 1774 Acked Read 9 Done Sent Done false
2680 1775 Acked Write 3 Done Sent Done false
2681 1776 Acked Write 4 Done Sent Done false
2682 1777 Acked Flush 0 Done Sent Done false
2683 1778 Acked Read 7 Done Sent Done false
2684 1779 Acked Read 5 Done Sent Done false
2685 1780 Acked Flush 0 Done Sent Done false
2686 1781 Acked Read 3 Done Sent Done false
2687 1782 Acked Read 9 Done Sent Done false
2688 1783 Acked Flush 0 Done Sent Done false
2689 1784 Acked Write 2 Done Sent Done false
2690 1785 Acked Read 9 Done Sent Done false
2691 1786 Acked Write 9 Done Sent Done false
2692 1787 Acked Read 7 Done Sent Done false
2693 1788 Acked Read 10 Done Sent Done false
2694 1789 Acked Write 9 Done Sent Done false
2695 1790 Acked Write 10 Done Sent Done false
2696 1791 Acked Write 5 Done Sent Done false
2697 1792 Acked Write 7 Done Sent Done false
2698 1793 Acked Read 3 Done Sent Done false
2699 1794 Acked Read 6 Done Sent Done false
2700 1795 Acked Read 7 Done Sent Done false
2701 1796 Acked Write 3 Done Sent Done false
2702 1797 Acked Flush 0 Done Sent Done false
2703 1798 Acked Write 6 Done Sent Done false
2704 1799 Acked Read 5 Done Sent Done false
2705 1800 Acked Write 5 Done Sent Done false
2706 1801 Acked Read 9 Done Sent Done false
2707 1802 Acked Write 5 Done Sent Done false
2708 1803 Acked Write 2 Done Sent Done false
2709 1804 Acked Read 9 Done Sent Done false
2710 1805 Acked Flush 0 Done Sent Done false
2711 1806 Acked Write 9 Done Sent Done false
2712 1807 Acked Write 2 Done Sent Done false
2713 1808 Acked Write 9 Done Sent Done false
2714 1809 Acked Write 10 Done Sent Done false
2715 1810 Acked Flush 0 Done Sent Done false
2716 1811 Acked Read 7 Done Sent Done false
2717 1812 Acked Write 5 Done Sent Done false
2718 1813 Acked Flush 0 Done Sent Done false
2719 1814 Acked Write 7 Done Sent Done false
2720 1815 Acked Read 7 Done Sent Done false
2721 1816 Acked Write 4 Done Sent Done false
2722 1817 Acked Write 6 Done Sent Done false
2723 1818 Acked Flush 0 Done Sent Done false
2724 1819 Acked Read 4 Done Sent Done false
2725 1820 Acked Write 7 Done Sent Done false
2726 1821 Acked Read 8 Done Sent Done false
2727 1822 Acked Write 4 Done Sent Done false
2728 1823 Acked Flush 0 Done Sent Done false
2729 1824 Acked Flush 0 Done Sent Done false
2730 1825 Acked Read 3 Done Sent Done false
2731 1826 Acked Write 10 Done Sent Done false
2732 1827 Acked Write 3 Done Sent Done false
2733 1828 Acked Flush 0 Done Sent Done false
2734 1829 Acked Write 5 Done Sent Done false
2735 1830 Acked Read 10 Done Sent Done false
2736 1831 Acked Read 4 Done Sent Done false
2737 1832 Acked Read 10 Done Sent Done false
2738 1833 Acked Write 8 Done Sent Done false
2739 1834 Acked Read 4 Done Sent Done false
2740 1835 Acked Read 7 Done Sent Done false
2741 1836 Acked Read 7 Done Sent Done false
2742 1837 Acked Read 1 Done Sent Done false
2743 1838 Acked Read 8 Done Sent Done false
2744 1839 Acked Flush 0 Done Sent Done false
2745 1840 Acked Write 1 Done Sent Done false
2746 1841 Acked Read 1 Done Sent Done false
2747 1842 Acked Flush 0 Done Sent Done false
2748 1843 Acked Write 3 Done Sent Done false
2749 1844 Acked Flush 0 Done Sent Done false
2750 1845 Acked Read 4 Done Sent Done false
2751 1846 Acked Write 5 Done Sent Done false
2752 1847 Acked Write 10 Done Sent Done false
2753 1848 Acked Read 10 Done Sent Done false
2754 1849 Acked Read 7 Done Sent Done false
2755 1850 Acked Read 4 Done Sent Done false
2756 1851 Acked Read 10 Done Sent Done false
2757 1852 Acked Write 3 Done Sent Done false
2758 1853 Acked Read 9 Done Sent Done false
2759 1854 Acked Read 5 Done Sent Done false
2760 1855 Acked Write 4 Done Sent Done false
2761 1856 Acked Read 4 Done Sent Done false
2762 1857 Acked Read 3 Done Sent Done false
2763 1858 Acked Flush 0 Done Sent Done false
2764 1859 Acked Read 7 Done Sent Done false
2765 1860 Acked Read 8 Done Sent Done false
2766 1861 Acked Read 6 Done Sent Done false
2767 1862 Acked Read 2 Done Sent Done false
2768 1863 Acked Read 9 Done Sent Done false
2769 1864 Acked Write 7 Done Sent Done false
2770 1865 Acked Read 7 Done Sent Done false
2771 1866 Acked Read 2 Done Sent Done false
2772 1867 Acked Write 8 Done Sent Done false
2773 1868 Acked Read 3 Done Sent Done false
2774 1869 Acked Write 1 Done Sent Done false
2775 1870 Acked Write 9 Done Sent Done false
2776 1871 Acked Read 10 Done Sent Done false
2777 1872 Acked Flush 0 Done Sent Done false
2778 1873 Acked Flush 0 Done Sent Done false
2779 1874 Acked Read 9 Done Sent Done false
2780 1875 Acked Write 5 Done Sent Done false
2781 1876 Acked Read 5 Done Sent Done false
2782 1877 Acked Read 2 Done Sent Done false
2783 1878 Acked Write 7 Done Sent Done false
2784 1879 Acked Flush 0 Done Sent Done false
2785 1880 Acked Write 9 Done Sent Done false
2786 1881 Acked Read 9 Done Sent Done false
2787 1882 Acked Write 6 Done Sent Done false
2788 1883 Acked Read 1 Done Sent Done false
2789 1884 Acked Read 5 Done Sent Done false
2790 1885 Acked Flush 0 Done Sent Done false
2791 1886 Acked Write 10 Done Sent Done false
2792 1887 Acked Write 7 Done Sent Done false
2793 1888 Acked Read 6 Done Sent Done false
2794 1889 Acked Write 9 Done Sent Done false
2795 1890 Acked Write 7 Done Sent Done false
2796 1891 Acked Write 4 Done Sent Done false
2797 1892 Acked Read 4 Done Sent Done false
2798 1893 Acked Read 5 Done Sent Done false
2799 1894 Acked Write 8 Done Sent Done false
2800 1895 Acked Write 8 Done Sent Done false
2801 1896 Acked Write 4 Done Sent Done false
2802 1897 Acked Read 2 Done Sent Done false
2803 1898 Acked Write 9 Done Sent Done false
2804 1899 Acked Read 2 Done Sent Done false
2805 1900 Acked Read 9 Done Sent Done false
2806 1901 Acked Write 6 Done Sent Done false
2807 1902 Acked Read 8 Done Sent Done false
2808 1903 Acked Flush 0 Done Sent Done false
2809 STATES DS:0 DS:1 DS:2 TOTAL
2810 Sent 0 301 0 301
2811 Done 301 0 301 602
2812 Skipped 0 0 0 0
2813 Error 0 0 0 0
2814 Last Flush: 1903 1602 1903
2815 Downstairs last five completed: 1602 1601 1600 1599 1598
2816 Upstairs last five completed: 1903 1902 1901 1900 1899
2817 CLIENT: Up:0 ds:301 act:2
2818 [8820] got start action
2819 [8820] I am going to restart
2820 Starting downstairs at port 8820
2821 Make output file at "/tmp/dsc/downstairs-8820.txt"
2822 [8820][1] reports Starting
2823 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1208)
2824 [8820][1] reports Running
2825 ----------------------------------------------------------------
2826 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
2827 States: Active Offline Active
2828 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
2829 1603 Acked Read 3 Done Sent Done false
2830 1604 Acked Read 7 Done Sent Done false
2831 1605 Acked Read 10 Done Sent Done false
2832 1606 Acked Write 8 Done Sent Done false
2833 1607 Acked Read 8 Done Sent Done false
2834 1608 Acked Flush 0 Done Sent Done false
2835 1609 Acked Read 6 Done Sent Done false
2836 1610 Acked Write 2 Done Sent Done false
2837 1611 Acked Write 5 Done Sent Done false
2838 1612 Acked Write 5 Done Sent Done false
2839 1613 Acked Write 2 Done Sent Done false
2840 1614 Acked Write 1 Done Sent Done false
2841 1615 Acked Write 8 Done Sent Done false
2842 1616 Acked Read 7 Done Sent Done false
2843 1617 Acked Read 9 Done Sent Done false
2844 1618 Acked Write 8 Done Sent Done false
2845 1619 Acked Write 2 Done Sent Done false
2846 1620 Acked Write 10 Done Sent Done false
2847 1621 Acked Read 1 Done Sent Done false
2848 1622 Acked Flush 0 Done Sent Done false
2849 1623 Acked Read 4 Done Sent Done false
2850 1624 Acked Write 5 Done Sent Done false
2851 1625 Acked Flush 0 Done Sent Done false
2852 1626 Acked Write 9 Done Sent Done false
2853 1627 Acked Write 3 Done Sent Done false
2854 1628 Acked Read 6 Done Sent Done false
2855 1629 Acked Flush 0 Done Sent Done false
2856 1630 Acked Write 1 Done Sent Done false
2857 1631 Acked Read 7 Done Sent Done false
2858 1632 Acked Read 10 Done Sent Done false
2859 1633 Acked Read 3 Done Sent Done false
2860 1634 Acked Read 9 Done Sent Done false
2861 1635 Acked Flush 0 Done Sent Done false
2862 1636 Acked Write 7 Done Sent Done false
2863 1637 Acked Write 7 Done Sent Done false
2864 1638 Acked Write 1 Done Sent Done false
2865 1639 Acked Flush 0 Done Sent Done false
2866 1640 Acked Read 8 Done Sent Done false
2867 1641 Acked Read 6 Done Sent Done false
2868 1642 Acked Write 9 Done Sent Done false
2869 1643 Acked Write 4 Done Sent Done false
2870 1644 Acked Read 6 Done Sent Done false
2871 1645 Acked Read 7 Done Sent Done false
2872 1646 Acked Read 2 Done Sent Done false
2873 1647 Acked Write 2 Done Sent Done false
2874 1648 Acked Flush 0 Done Sent Done false
2875 1649 Acked Read 9 Done Sent Done false
2876 1650 Acked Write 5 Done Sent Done false
2877 1651 Acked Flush 0 Done Sent Done false
2878 1652 Acked Read 6 Done Sent Done false
2879 1653 Acked Write 6 Done Sent Done false
2880 1654 Acked Write 6 Done Sent Done false
2881 1655 Acked Read 2 Done Sent Done false
2882 1656 Acked Write 10 Done Sent Done false
2883 1657 Acked Write 5 Done Sent Done false
2884 1658 Acked Write 1 Done Sent Done false
2885 1659 Acked Write 7 Done Sent Done false
2886 1660 Acked Read 6 Done Sent Done false
2887 1661 Acked Write 1 Done Sent Done false
2888 1662 Acked Read 8 Done Sent Done false
2889 1663 Acked Read 4 Done Sent Done false
2890 1664 Acked Read 7 Done Sent Done false
2891 1665 Acked Read 7 Done Sent Done false
2892 1666 Acked Flush 0 Done Sent Done false
2893 1667 Acked Read 3 Done Sent Done false
2894 1668 Acked Write 1 Done Sent Done false
2895 1669 Acked Read 5 Done Sent Done false
2896 1670 Acked Write 7 Done Sent Done false
2897 1671 Acked Read 10 Done Sent Done false
2898 1672 Acked Read 2 Done Sent Done false
2899 1673 Acked Read 6 Done Sent Done false
2900 1674 Acked Read 6 Done Sent Done false
2901 1675 Acked Write 5 Done Sent Done false
2902 1676 Acked Read 9 Done Sent Done false
2903 1677 Acked Read 7 Done Sent Done false
2904 1678 Acked Write 10 Done Sent Done false
2905 1679 Acked Write 1 Done Sent Done false
2906 1680 Acked Read 8 Done Sent Done false
2907 1681 Acked Read 8 Done Sent Done false
2908 1682 Acked Flush 0 Done Sent Done false
2909 1683 Acked Write 9 Done Sent Done false
2910 1684 Acked Read 1 Done Sent Done false
2911 1685 Acked Write 6 Done Sent Done false
2912 1686 Acked Write 3 Done Sent Done false
2913 1687 Acked Read 1 Done Sent Done false
2914 1688 Acked Read 3 Done Sent Done false
2915 1689 Acked Write 8 Done Sent Done false
2916 1690 Acked Write 8 Done Sent Done false
2917 1691 Acked Read 1 Done Sent Done false
2918 1692 Acked Write 7 Done Sent Done false
2919 1693 Acked Read 7 Done Sent Done false
2920 1694 Acked Read 2 Done Sent Done false
2921 1695 Acked Write 8 Done Sent Done false
2922 1696 Acked Write 6 Done Sent Done false
2923 1697 Acked Read 8 Done Sent Done false
2924 1698 Acked Write 3 Done Sent Done false
2925 1699 Acked Write 8 Done Sent Done false
2926 1700 Acked Read 9 Done Sent Done false
2927 1701 Acked Write 2 Done Sent Done false
2928 1702 Acked Read 1 Done Sent Done false
2929 1703 Acked Write 3 Done Sent Done false
2930 1704 Acked Write 5 Done Sent Done false
2931 1705 Acked Write 8 Done Sent Done false
2932 1706 Acked Read 7 Done Sent Done false
2933 1707 Acked Write 2 Done Sent Done false
2934 1708 Acked Read 8 Done Sent Done false
2935 1709 Acked Read 2 Done Sent Done false
2936 1710 Acked Read 5 Done Sent Done false
2937 1711 Acked Read 1 Done Sent Done false
2938 1712 Acked Write 1 Done Sent Done false
2939 1713 Acked Read 3 Done Sent Done false
2940 1714 Acked Flush 0 Done Sent Done false
2941 1715 Acked Flush 0 Done Sent Done false
2942 1716 Acked Write 2 Done Sent Done false
2943 1717 Acked Read 6 Done Sent Done false
2944 1718 Acked Read 2 Done Sent Done false
2945 1719 Acked Write 6 Done Sent Done false
2946 1720 Acked Write 5 Done Sent Done false
2947 1721 Acked Flush 0 Done Sent Done false
2948 1722 Acked Write 5 Done Sent Done false
2949 1723 Acked Flush 0 Done Sent Done false
2950 1724 Acked Write 1 Done Sent Done false
2951 1725 Acked Read 2 Done Sent Done false
2952 1726 Acked Write 4 Done Sent Done false
2953 1727 Acked Read 8 Done Sent Done false
2954 1728 Acked Write 5 Done Sent Done false
2955 1729 Acked Read 4 Done Sent Done false
2956 1730 Acked Flush 0 Done Sent Done false
2957 1731 Acked Read 5 Done Sent Done false
2958 1732 Acked Write 7 Done Sent Done false
2959 1733 Acked Read 8 Done Sent Done false
2960 1734 Acked Read 6 Done Sent Done false
2961 1735 Acked Read 6 Done Sent Done false
2962 1736 Acked Read 8 Done Sent Done false
2963 1737 Acked Write 2 Done Sent Done false
2964 1738 Acked Read 2 Done Sent Done false
2965 1739 Acked Read 5 Done Sent Done false
2966 1740 Acked Write 3 Done Sent Done false
2967 1741 Acked Read 7 Done Sent Done false
2968 1742 Acked Write 7 Done Sent Done false
2969 1743 Acked Write 2 Done Sent Done false
2970 1744 Acked Write 2 Done Sent Done false
2971 1745 Acked Write 4 Done Sent Done false
2972 1746 Acked Read 10 Done Sent Done false
2973 1747 Acked Write 10 Done Sent Done false
2974 1748 Acked Read 6 Done Sent Done false
2975 1749 Acked Write 3 Done Sent Done false
2976 1750 Acked Write 3 Done Sent Done false
2977 1751 Acked Write 1 Done Sent Done false
2978 1752 Acked Read 6 Done Sent Done false
2979 1753 Acked Read 3 Done Sent Done false
2980 1754 Acked Write 9 Done Sent Done false
2981 1755 Acked Read 7 Done Sent Done false
2982 1756 Acked Read 8 Done Sent Done false
2983 1757 Acked Flush 0 Done Sent Done false
2984 1758 Acked Read 1 Done Sent Done false
2985 1759 Acked Read 10 Done Sent Done false
2986 1760 Acked Read 7 Done Sent Done false
2987 1761 Acked Read 4 Done Sent Done false
2988 1762 Acked Read 3 Done Sent Done false
2989 1763 Acked Read 1 Done Sent Done false
2990 1764 Acked Write 10 Done Sent Done false
2991 1765 Acked Write 3 Done Sent Done false
2992 1766 Acked Write 6 Done Sent Done false
2993 1767 Acked Read 1 Done Sent Done false
2994 1768 Acked Flush 0 Done Sent Done false
2995 1769 Acked Read 8 Done Sent Done false
2996 1770 Acked Write 7 Done Sent Done false
2997 1771 Acked Write 5 Done Sent Done false
2998 1772 Acked Write 3 Done Sent Done false
2999 1773 Acked Flush 0 Done Sent Done false
3000 1774 Acked Read 9 Done Sent Done false
3001 1775 Acked Write 3 Done Sent Done false
3002 1776 Acked Write 4 Done Sent Done false
3003 1777 Acked Flush 0 Done Sent Done false
3004 1778 Acked Read 7 Done Sent Done false
3005 1779 Acked Read 5 Done Sent Done false
3006 1780 Acked Flush 0 Done Sent Done false
3007 1781 Acked Read 3 Done Sent Done false
3008 1782 Acked Read 9 Done Sent Done false
3009 1783 Acked Flush 0 Done Sent Done false
3010 1784 Acked Write 2 Done Sent Done false
3011 1785 Acked Read 9 Done Sent Done false
3012 1786 Acked Write 9 Done Sent Done false
3013 1787 Acked Read 7 Done Sent Done false
3014 1788 Acked Read 10 Done Sent Done false
3015 1789 Acked Write 9 Done Sent Done false
3016 1790 Acked Write 10 Done Sent Done false
3017 1791 Acked Write 5 Done Sent Done false
3018 1792 Acked Write 7 Done Sent Done false
3019 1793 Acked Read 3 Done Sent Done false
3020 1794 Acked Read 6 Done Sent Done false
3021 1795 Acked Read 7 Done Sent Done false
3022 1796 Acked Write 3 Done Sent Done false
3023 1797 Acked Flush 0 Done Sent Done false
3024 1798 Acked Write 6 Done Sent Done false
3025 1799 Acked Read 5 Done Sent Done false
3026 1800 Acked Write 5 Done Sent Done false
3027 1801 Acked Read 9 Done Sent Done false
3028 1802 Acked Write 5 Done Sent Done false
3029 1803 Acked Write 2 Done Sent Done false
3030 1804 Acked Read 9 Done Sent Done false
3031 1805 Acked Flush 0 Done Sent Done false
3032 1806 Acked Write 9 Done Sent Done false
3033 1807 Acked Write 2 Done Sent Done false
3034 1808 Acked Write 9 Done Sent Done false
3035 1809 Acked Write 10 Done Sent Done false
3036 1810 Acked Flush 0 Done Sent Done false
3037 1811 Acked Read 7 Done Sent Done false
3038 1812 Acked Write 5 Done Sent Done false
3039 1813 Acked Flush 0 Done Sent Done false
3040 1814 Acked Write 7 Done Sent Done false
3041 1815 Acked Read 7 Done Sent Done false
3042 1816 Acked Write 4 Done Sent Done false
3043 1817 Acked Write 6 Done Sent Done false
3044 1818 Acked Flush 0 Done Sent Done false
3045 1819 Acked Read 4 Done Sent Done false
3046 1820 Acked Write 7 Done Sent Done false
3047 1821 Acked Read 8 Done Sent Done false
3048 1822 Acked Write 4 Done Sent Done false
3049 1823 Acked Flush 0 Done Sent Done false
3050 1824 Acked Flush 0 Done Sent Done false
3051 1825 Acked Read 3 Done Sent Done false
3052 1826 Acked Write 10 Done Sent Done false
3053 1827 Acked Write 3 Done Sent Done false
3054 1828 Acked Flush 0 Done Sent Done false
3055 1829 Acked Write 5 Done Sent Done false
3056 1830 Acked Read 10 Done Sent Done false
3057 1831 Acked Read 4 Done Sent Done false
3058 1832 Acked Read 10 Done Sent Done false
3059 1833 Acked Write 8 Done Sent Done false
3060 1834 Acked Read 4 Done Sent Done false
3061 1835 Acked Read 7 Done Sent Done false
3062 1836 Acked Read 7 Done Sent Done false
3063 1837 Acked Read 1 Done Sent Done false
3064 1838 Acked Read 8 Done Sent Done false
3065 1839 Acked Flush 0 Done Sent Done false
3066 1840 Acked Write 1 Done Sent Done false
3067 1841 Acked Read 1 Done Sent Done false
3068 1842 Acked Flush 0 Done Sent Done false
3069 1843 Acked Write 3 Done Sent Done false
3070 1844 Acked Flush 0 Done Sent Done false
3071 1845 Acked Read 4 Done Sent Done false
3072 1846 Acked Write 5 Done Sent Done false
3073 1847 Acked Write 10 Done Sent Done false
3074 1848 Acked Read 10 Done Sent Done false
3075 1849 Acked Read 7 Done Sent Done false
3076 1850 Acked Read 4 Done Sent Done false
3077 1851 Acked Read 10 Done Sent Done false
3078 1852 Acked Write 3 Done Sent Done false
3079 1853 Acked Read 9 Done Sent Done false
3080 1854 Acked Read 5 Done Sent Done false
3081 1855 Acked Write 4 Done Sent Done false
3082 1856 Acked Read 4 Done Sent Done false
3083 1857 Acked Read 3 Done Sent Done false
3084 1858 Acked Flush 0 Done Sent Done false
3085 1859 Acked Read 7 Done Sent Done false
3086 1860 Acked Read 8 Done Sent Done false
3087 1861 Acked Read 6 Done Sent Done false
3088 1862 Acked Read 2 Done Sent Done false
3089 1863 Acked Read 9 Done Sent Done false
3090 1864 Acked Write 7 Done Sent Done false
3091 1865 Acked Read 7 Done Sent Done false
3092 1866 Acked Read 2 Done Sent Done false
3093 1867 Acked Write 8 Done Sent Done false
3094 1868 Acked Read 3 Done Sent Done false
3095 1869 Acked Write 1 Done Sent Done false
3096 1870 Acked Write 9 Done Sent Done false
3097 1871 Acked Read 10 Done Sent Done false
3098 1872 Acked Flush 0 Done Sent Done false
3099 1873 Acked Flush 0 Done Sent Done false
3100 1874 Acked Read 9 Done Sent Done false
3101 1875 Acked Write 5 Done Sent Done false
3102 1876 Acked Read 5 Done Sent Done false
3103 1877 Acked Read 2 Done Sent Done false
3104 1878 Acked Write 7 Done Sent Done false
3105 1879 Acked Flush 0 Done Sent Done false
3106 1880 Acked Write 9 Done Sent Done false
3107 1881 Acked Read 9 Done Sent Done false
3108 1882 Acked Write 6 Done Sent Done false
3109 1883 Acked Read 1 Done Sent Done false
3110 1884 Acked Read 5 Done Sent Done false
3111 1885 Acked Flush 0 Done Sent Done false
3112 1886 Acked Write 10 Done Sent Done false
3113 1887 Acked Write 7 Done Sent Done false
3114 1888 Acked Read 6 Done Sent Done false
3115 1889 Acked Write 9 Done Sent Done false
3116 1890 Acked Write 7 Done Sent Done false
3117 1891 Acked Write 4 Done Sent Done false
3118 1892 Acked Read 4 Done Sent Done false
3119 1893 Acked Read 5 Done Sent Done false
3120 1894 Acked Write 8 Done Sent Done false
3121 1895 Acked Write 8 Done Sent Done false
3122 1896 Acked Write 4 Done Sent Done false
3123 1897 Acked Read 2 Done Sent Done false
3124 1898 Acked Write 9 Done Sent Done false
3125 1899 Acked Read 2 Done Sent Done false
3126 1900 Acked Read 9 Done Sent Done false
3127 1901 Acked Write 6 Done Sent Done false
3128 1902 Acked Read 8 Done Sent Done false
3129 1903 Acked Flush 0 Done Sent Done false
3130 STATES DS:0 DS:1 DS:2 TOTAL
3131 Sent 0 301 0 301
3132 Done 301 0 301 602
3133 Skipped 0 0 0 0
3134 Error 0 0 0 0
3135 Last Flush: 1903 1602 1903
3136 Downstairs last five completed: 1602 1601 1600 1599 1598
3137 Upstairs last five completed: 1903 1902 1901 1900 1899
3138 CLIENT: Up:0 ds:301 act:2
31392024-11-15T01:58:06.340ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
31402024-11-15T01:58:06.340ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:45003) = io task client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
31412024-11-15T01:58:06.341ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
31422024-11-15T01:58:06.342ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
31432024-11-15T01:58:06.342ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
31442024-11-15T01:58:06.342ZINFOcrucible: send last flush ID to this DS: 1602 = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
31452024-11-15T01:58:06.342ZINFOcrucible: Replied this last flush ID: 1602 = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
31462024-11-15T01:58:06.342ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
31472024-11-15T01:58:06.342ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
31482024-11-15T01:58:06.342ZINFOcrucible: [1] client re-new 301 jobs since flush 1602 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
31492024-11-15T01:58:06.343ZINFOcrucible: [1] Replayed 301 jobs since flush: 1602 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
3150 ----------------------------------------------------------------
3151 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
3152 Downstairs last five completed: 1903 1902 1901 1900 1899
3153 Upstairs last five completed: 1903 1902 1901 1900 1899
3154 CLIENT: Up:0 ds:0 act:3
3155 Replay: All jobs finished
3156 Main task has work to do, go find it
3157 got dsc Stop(2)
3158 stop 2
3159 Nov 15 01:58:08.579 INFO request completed, latency_us: 356, response_code: 204[8830] Got stop action so:false kr:false
3160 , uri: /stop/cid/2, method: POST, req_id: 48672cbd-b598-444e-9f17-cfe2d1966466, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
31612024-11-15T01:58:08.580ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
3162 Nov 15 01:58:08.580 INFO request completed, latency_us: 151, response_code: 200, uri: /state/cid/2, method: GET, req_id: 148aafc3-a73d-4810-9c6f-2852b1d59e8f, remote_addr: {"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2024-11-15T01:58:08.580375007Z","hostname":"w-01JCPRHT5F2MPA44KVJHM3S2PB","pid":1205,"":"io task","client":"2","":"downstairs","session_id":"971dcb2e-6891-4ecf-b367-1e04e52065ee"}
3163 127.0.0.1:40008, {"msg":"downstairs task for 2 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2024-11-15T01:58:08.580445923Z","hostname":"w-01JCPRHT5F2MPA44KVJHM3S2PB","pid":1205,"session_id":"971dcb2e-6891-4ecf-b367-1e04e52065ee"}
3164 local_addr{"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-15T01:58:08.58048922Z","hostname":"w-01JCPRHT5F2MPA44KVJHM3S2PB","pid":1205,"client":"2","":"downstairs","session_id":"971dcb2e-6891-4ecf-b367-1e04e52065ee"}
3165 : 127.0.0.1:9998
31662024-11-15T01:58:08.580ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
31672024-11-15T01:58:08.580ZINFOcrucible: client task is exiting = io task client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
3168 [8830] Exited with: ExitStatus(unix_wait_status(9))
3169 [8830][2] reports Exit
3170 Nov 15 01:58:12.583 INFO request completed, latency_us: 324, response_code: 200, uri: /state/cid/2, method: GET, req_id: 0bc5036d-5b62-4922-9fa3-1f2f4ce3f022, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
3171 001/300 Flush
3172 002/300 Read block 3197 len 16384
3173 003/300 Write block 4213 len 4096 data: 2
3174 004/300 Read block 496 len 12288
3175 005/300 Read block 4825 len 32768
3176 006/300 Write block 1029 len 36864 data: 2 2 2 2 2 2 2 2 2
3177 007/300 Write block 1404 len 40960 data: 2 3 3 3 3 2 3 3 3 3
3178 008/300 Read block 3599 len 4096
3179 009/300 Read block 4542 len 28672
3180 010/300 Read block 1491 len 28672
3181 011/300 Read block 3067 len 16384
3182 012/300 Flush
3183 013/300 Write block 3166 len 36864 data: 2 2 2 2 2 2 2 2 2
3184 014/300 Read block 2100 len 24576
3185 015/300 Write block 321 len 8192 data: 2 2
3186 016/300 Read block 4780 len 4096
3187 017/300 Write block 1763 len 8192 data: 2 2
3188 018/300 Read block 3621 len 28672
3189 019/300 Write block 677 len 28672 data: 2 3 2 2 2 2 2
3190 020/300 Read block 158 len 16384
3191 021/300 Write block 2818 len 8192 data: 2 2
3192 022/300 Write block 1636 len 36864 data: 2 2 2 2 2 2 2 2 2
3193 023/300 Flush
3194 024/300 Write block 2253 len 20480 data: 3 3 3 2 2
3195 025/300 Read block 4402 len 16384
3196 026/300 Write block 925 len 24576 data: 3 2 2 2 2 2
3197 027/300 Read block 1354 len 12288
3198 028/300 Read block 520 len 20480
3199 029/300 Read block 4826 len 24576
3200 030/300 Write block 680 len 24576 data: 3 3 3 3 2 2
3201 031/300 Flush
3202 032/300 Flush
3203 033/300 Read block 421 len 28672
3204 034/300 Flush
3205 035/300 Write block 4149 len 12288 data: 3 3 2
3206 036/300 Read block 4624 len 32768
3207 037/300 Read block 4504 len 16384
3208 038/300 Read block 3495 len 32768
3209 039/300 Write block 4874 len 8192 data: 2 2
3210 040/300 Write block 735 len 28672 data: 2 2 2 2 2 2 2
3211 041/300 Read block 3420 len 28672
3212 042/300 Read block 644 len 12288
3213 043/300 Write block 1562 len 40960 data: 4 4 3 3 2 2 2 2 2 2
3214 044/300 Flush
3215 045/300 Flush
3216 046/300 Read block 3139 len 12288
3217 047/300 Read block 1965 len 16384
3218 048/300 Read block 1831 len 32768
3219 049/300 Write block 2942 len 24576 data: 3 3 3 3 4 3
3220 050/300 Write block 477 len 16384 data: 3 3 3 3
3221 051/300 Read block 4187 len 4096
3222 052/300 Read block 4668 len 24576
3223 053/300 Read block 4391 len 20480
3224 054/300 Flush
3225 055/300 Read block 397 len 12288
3226 056/300 Read block 4393 len 24576
3227 057/300 Read block 3607 len 40960
3228 058/300 Read block 1325 len 24576
3229 059/300 Read block 2338 len 28672
3230 060/300 Write block 4920 len 40960 data: 2 2 2 2 2 2 2 2 2 2
3231 061/300 Read block 3795 len 4096
3232 062/300 Flush
3233 063/300 Read block 3576 len 20480
3234 064/300 Write block 1815 len 4096 data: 2
3235 065/300 Read block 561 len 24576
3236 066/300 Write block 3189 len 24576 data: 3 2 2 2 2 2
3237 067/300 Flush
3238 068/300 Read block 4267 len 40960
3239 069/300 Write block 1196 len 32768 data: 4 4 4 4 3 2 2 2
3240 070/300 Write block 1245 len 32768 data: 2 2 2 2 2 2 2 2
3241 071/300 Flush
3242 072/300 Read block 505 len 8192
3243 073/300 Write block 2793 len 4096 data: 2
3244 074/300 Read block 2942 len 24576
3245 075/300 Flush
3246 076/300 Read block 118 len 24576
3247 077/300 Read block 3514 len 40960
3248 078/300 Read block 1507 len 40960
3249 079/300 Read block 2141 len 36864
3250 080/300 Write block 4418 len 40960 data: 2 3 3 3 3 3 3 3 3 2
3251 081/300 Write block 4506 len 8192 data: 2 2
3252 082/300 Read block 902 len 32768
3253 083/300 Read block 571 len 12288
3254 084/300 Read block 2771 len 40960
3255 085/300 Read block 1491 len 20480
3256 086/300 Read block 2859 len 8192
3257 087/300 Write block 4923 len 20480 data: 3 3 3 3 3
3258 088/300 Read block 4065 len 16384
3259 089/300 Flush
3260 090/300 Read block 1303 len 32768
3261 091/300 Read block 3297 len 32768
3262 092/300 Write block 164 len 36864 data: 2 2 3 3 3 3 3 3 3
3263 093/300 Read block 3733 len 40960
3264 094/300 Flush
3265 095/300 Read block 1171 len 28672
3266 096/300 Read block 348 len 16384
3267 097/300 Read block 2039 len 12288
3268 098/300 Write block 679 len 24576 data: 3 4 4 4 4 3
3269 099/300 Write block 2977 len 40960 data: 2 2 2 2 3 3 3 3 3 3
3270 100/300 Flush
3271 101/300 Write block 4073 len 36864 data: 2 2 2 2 2 2 2 2 3
3272 102/300 Read block 1742 len 40960
3273 103/300 Read block 729 len 16384
3274 104/300 Write block 88 len 8192 data: 2 2
3275 105/300 Read block 3436 len 8192
3276 106/300 Write block 1558 len 40960 data: 3 4 4 4 5 5 4 4 3 3
3277 107/300 Write block 4727 len 16384 data: 3 2 2 3
3278 108/300 Read block 4773 len 28672
3279 109/300 Read block 599 len 4096
3280 110/300 Read block 1284 len 12288
3281 111/300 Read block 2668 len 24576
3282 112/300 Write block 2453 len 40960 data: 2 2 2 2 2 2 2 3 3 3
3283 113/300 Read block 4961 len 24576
3284 114/300 Read block 4969 len 16384
3285 115/300 Read block 1179 len 40960
3286 116/300 Write block 823 len 28672 data: 2 2 2 2 2 2 2
3287 117/300 Write block 3302 len 20480 data: 2 2 2 2 2
3288 118/300 Write block 3529 len 8192 data: 2 2
3289 119/300 Write block 3127 len 40960 data: 2 2 2 2 2 2 2 3 3 3
3290 120/300 Read block 4944 len 28672
3291 121/300 Read block 732 len 32768
3292 122/300 Write block 3117 len 40960 data: 2 2 2 2 2 2 2 2 2 2
3293 123/300 Flush
3294 124/300 Write block 4037 len 20480 data: 3 3 3 3 3
3295 125/300 Read block 2948 len 12288
3296 126/300 Flush
3297 127/300 Read block 3854 len 36864
3298 128/300 Read block 3691 len 40960
3299 129/300 Read block 4601 len 32768
3300 130/300 Write block 4529 len 20480 data: 4 4 2 2 2
3301 131/300 Read block 1983 len 16384
3302 132/300 Flush
3303 133/300 Read block 4221 len 8192
3304 134/300 Write block 337 len 12288 data: 3 2 2
3305 135/300 Read block 3428 len 36864
3306 136/300 Read block 401 len 16384
3307 137/300 Read block 2211 len 28672
3308 138/300 Write block 1667 len 16384 data: 3 3 3 3
3309 139/300 Write block 1376 len 24576 data: 3 3 2 2 2 2
3310 140/300 Write block 2947 len 12288 data: 4 3 3
3311 141/300 Read block 326 len 36864
3312 142/300 Read block 4462 len 40960
3313 143/300 Read block 3821 len 28672
3314 144/300 Flush
3315 145/300 Write block 3589 len 12288 data: 3 3 3
3316 146/300 Flush
3317 147/300 Write block 1067 len 36864 data: 3 3 3 2 2 2 2 2 2
3318 148/300 Read block 1944 len 24576
3319 149/300 Read block 3178 len 8192
3320 150/300 Read block 654 len 28672
3321 151/300 Read block 2764 len 8192
3322 152/300 Read block 2417 len 32768
3323 153/300 Read block 2995 len 40960
3324 154/300 Read block 3664 len 40960
3325 155/300 Read block 583 len 32768
3326 156/300 Read block 1409 len 8192
3327 157/300 Write block 2821 len 36864 data: 2 2 2 2 2 2 2 2 2
3328 158/300 Read block 4337 len 28672
3329 159/300 Read block 3787 len 12288
3330 160/300 Flush
3331 161/300 Read block 2214 len 24576
3332 162/300 Read block 3411 len 24576
3333 163/300 Flush
3334 164/300 Flush
3335 165/300 Read block 1798 len 36864
3336 166/300 Write block 2350 len 4096 data: 2
3337 167/300 Read block 715 len 8192
3338 168/300 Write block 3675 len 36864 data: 2 2 2 2 2 2 2 2 2
3339 169/300 Read block 325 len 12288
3340 170/300 Write block 94 len 20480 data: 3 2 2 2 2
3341 171/300 Write block 2958 len 8192 data: 3 3
3342 172/300 Read block 4040 len 12288
3343 173/300 Read block 1944 len 32768
3344 174/300 Flush
3345 175/300 Read block 2103 len 16384
3346 176/300 Read block 1742 len 36864
3347 177/300 Read block 1643 len 40960
3348 178/300 Write block 2155 len 32768 data: 3 2 2 2 2 2 2 2
3349 179/300 Read block 1771 len 12288
3350 180/300 Flush
3351 181/300 Write block 2177 len 36864 data: 2 2 2 2 2 2 2 2 2
3352 182/300 Write block 4632 len 16384 data: 2 2 2 2
3353 183/300 Write block 4265 len 32768 data: 3 3 3 3 3 3 3 3
3354 184/300 Write block 967 len 4096 data: 4
3355 185/300 Write block 4886 len 20480 data: 2 2 2 2 2
3356 186/300 Read block 542 len 20480
3357 187/300 Read block 4700 len 8192
3358 188/300 Read block 3386 len 4096
3359 189/300 Flush
3360 190/300 Read block 4617 len 28672
3361 191/300 Write block 1381 len 16384 data: 3 2 2 2
3362 192/300 Read block 2524 len 16384
3363 193/300 Read block 970 len 4096
3364 194/300 Read block 326 len 40960
3365 195/300 Read block 3604 len 24576
3366 196/300 Read block 975 len 36864
3367 197/300 Write block 4244 len 8192 data: 2 2
3368 198/300 Read block 4913 len 40960
3369 199/300 Write block 3719 len 28672 data: 2 2 2 2 2 2 2
3370 200/300 Write block 416 len 32768 data: 2 2 2 2 2 2 2 2
3371 201/300 Read block 2436 len 16384
3372 202/300 Write block 1451 len 12288 data: 3 2 2
3373 203/300 Read block 4594 len 4096
3374 204/300 Write block 2527 len 40960 data: 3 3 3 2 2 2 2 2 2 2
3375 205/300 Read block 2305 len 28672
3376 206/300 Flush
3377 207/300 Read block 3484 len 16384
3378 208/300 Read block 4109 len 12288
3379 209/300 Write block 604 len 4096 data: 3
3380 210/300 Read block 3480 len 16384
3381 211/300 Read block 4597 len 36864
3382 212/300 Read block 723 len 4096
3383 213/300 Write block 3737 len 4096 data: 3
3384 214/300 Read block 4568 len 4096
3385 215/300 Write block 1732 len 28672 data: 2 2 2 2 2 2 2
3386 216/300 Write block 3488 len 16384 data: 2 2 2 2
3387 217/300 Read block 4221 len 20480
3388 218/300 Write block 3864 len 20480 data: 3 3 3 3 3
3389 219/300 Read block 408 len 8192
3390 220/300 Read block 3380 len 28672
3391 221/300 Write block 1539 len 20480 data: 3 3 3 2 2
3392 222/300 Read block 3299 len 40960
3393 223/300 Write block 3379 len 40960 data: 2 2 2 2 2 2 2 2 2 2
3394 224/300 Read block 2242 len 36864
3395 225/300 Write block 2561 len 12288 data: 3 3 3
3396 226/300 Read block 2177 len 20480
3397 227/300 Write block 1871 len 24576 data: 3 3 3 2 2 2
3398 228/300 Read block 987 len 40960
3399 229/300 Flush
3400 230/300 Read block 3185 len 12288
3401 231/300 Write block 289 len 28672 data: 2 2 2 2 2 2 2
3402 232/300 Read block 511 len 16384
3403 233/300 Write block 2869 len 20480 data: 3 3 3 3 3
3404 234/300 Read block 3620 len 16384
3405 235/300 Read block 220 len 32768
3406 236/300 Flush
3407 237/300 Read block 4847 len 20480
3408 238/300 Write block 550 len 8192 data: 2 2
3409 239/300 Write block 2374 len 4096 data: 2
3410 240/300 Write block 1267 len 28672 data: 2 2 2 2 2 2 2
3411 241/300 Write block 4708 len 36864 data: 3 3 3 3 2 2 2 2 2
3412 242/300 Read block 4573 len 12288
3413 243/300 Write block 4263 len 16384 data: 2 3 4 4
3414 244/300 Write block 2514 len 12288 data: 2 3 3
3415 245/300 Flush
3416 246/300 Write block 581 len 16384 data: 2 3 3 3
3417 247/300 Write block 2059 len 24576 data: 2 2 2 2 2 2
3418 248/300 Read block 3364 len 16384
3419 249/300 Read block 1188 len 28672
3420 250/300 Read block 2529 len 8192
3421 251/300 Write block 1510 len 32768 data: 3 3 3 4 4 4 3 3
3422 252/300 Write block 420 len 16384 data: 3 3 3 3
3423 253/300 Read block 4313 len 24576
3424 254/300 Write block 74 len 36864 data: 3 3 2 2 2 2 2 2 2
3425 255/300 Read block 4254 len 28672
3426 256/300 Write block 1590 len 20480 data: 2 2 2 2 2
3427 257/300 Read block 1564 len 8192
3428 258/300 Flush
3429 259/300 Read block 1234 len 4096
3430 260/300 Write block 2531 len 36864 data: 3 3 3 3 3 3 2 2 2
3431 261/300 Write block 2473 len 20480 data: 2 2 2 2 2
3432 262/300 Write block 2596 len 28672 data: 2 2 2 2 2 2 2
3433 263/300 Read block 1613 len 32768
3434 264/300 Write block 3361 len 4096 data: 2
3435 265/300 Write block 2603 len 24576 data: 2 2 2 2 2 2
3436 266/300 Write block 292 len 24576 data: 3 3 3 3 2 2
3437 267/300 Read block 295 len 24576
3438 268/300 Write block 3079 len 12288 data: 6 7 6
3439 269/300 Read block 3641 len 36864
3440 270/300 Write block 2998 len 16384 data: 2 2 2 2
3441 271/300 Write block 2158 len 24576 data: 3 3 3 3 3 2
3442 272/300 Read block 4701 len 28672
3443 273/300 Write block 4843 len 16384 data: 4 3 3 3
3444 274/300 Read block 4037 len 8192
3445 275/300 Write block 1690 len 40960 data: 3 3 3 3 2 2 2 2 3 3
3446 276/300 Write block 1167 len 4096 data: 2
3447 277/300 Write block 699 len 28672 data: 2 2 3 3 3 3 3
3448 278/300 Read block 40 len 4096
3449 279/300 Read block 3460 len 12288
3450 280/300 Write block 699 len 16384 data: 3 3 4 4
3451 281/300 Read block 2042 len 24576
3452 282/300 Write block 4923 len 20480 data: 4 4 4 4 4
3453 283/300 Read block 2852 len 12288
3454 284/300 Write block 2455 len 8192 data: 3 3
3455 285/300 Read block 2468 len 12288
3456 286/300 Write block 2545 len 8192 data: 3 3
3457 287/300 Write block 3034 len 40960 data: 2 2 2 2 2 3 3 3 3 3
3458 288/300 Write block 2899 len 8192 data: 2 2
3459 289/300 Read block 3234 len 4096
3460 290/300 Read block 2816 len 28672
3461 291/300 Write block 3601 len 20480 data: 2 3 3 3 3
3462 292/300 Write block 3923 len 8192 data: 3 3
3463 293/300 Flush
3464 294/300 Read block 3253 len 32768
3465 295/300 Flush
3466 296/300 Write block 2971 len 32768 data: 3 3 3 3 3 2 3 3
3467 297/300 Write block 460 len 36864 data: 2 2 2 2 2 2 2 2 2
3468 298/300 Write block 3506 len 16384 data: 2 2 2 2
3469 299/300 Write block 1306 len 28672 data: 2 2 3 3 3 3 3
3470 300/300 Write block 1168 len 40960 data: 2 2 2 2 2 2 2 2 2 2
3471 301/300 Read block 2781 len 4096
3472 Main task has work to do, go find it
3473 got dsc Start(2)
3474 start 2
3475 Nov 15 01:58:12.820 INFO request completed, latency_us: 208, response_code: 204, uri: /start/cid/2, method: POST, req_id: 864abb45-0f45-459c-853d-803db90a0595, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
3476 Replay: started 2, returned:Ok(())
3477 ----------------------------------------------------------------
3478 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
3479 States: Active Active Offline
3480 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
3481 1904 Acked Flush 0 Done Done Sent false
3482 1905 Acked Read 4 Done Done Sent false
3483 1906 Acked Write 1 Done Done Sent false
3484 1907 Acked Read 3 Done Done Sent false
3485 1908 Acked Read 8 Done Done Sent false
3486 1909 Acked Write 9 Done Done Sent false
3487 1910 Acked Write 10 Done Done Sent false
3488 1911 Acked Read 1 Done Done Sent false
3489 1912 Acked Read 7 Done Done Sent false
3490 1913 Acked Read 7 Done Done Sent false
3491 1914 Acked Read 4 Done Done Sent false
3492 1915 Acked Flush 0 Done Done Sent false
3493 1916 Acked Write 9 Done Done Sent false
3494 1917 Acked Read 6 Done Done Sent false
3495 1918 Acked Write 2 Done Done Sent false
3496 1919 Acked Read 1 Done Done Sent false
3497 1920 Acked Write 2 Done Done Sent false
3498 1921 Acked Read 7 Done Done Sent false
3499 1922 Acked Write 7 Done Done Sent false
3500 1923 Acked Read 4 Done Done Sent false
3501 1924 Acked Write 2 Done Done Sent false
3502 1925 Acked Write 9 Done Done Sent false
3503 1926 Acked Flush 0 Done Done Sent false
3504 1927 Acked Write 5 Done Done Sent false
3505 1928 Acked Read 4 Done Done Sent false
3506 1929 Acked Write 6 Done Done Sent false
3507 1930 Acked Read 3 Done Done Sent false
3508 1931 Acked Read 5 Done Done Sent false
3509 1932 Acked Read 6 Done Done Sent false
3510 1933 Acked Write 6 Done Done Sent false
3511 1934 Acked Flush 0 Done Done Sent false
3512 1935 Acked Flush 0 Done Done Sent false
3513 1936 Acked Read 7 Done Done Sent false
3514 1937 Acked Flush 0 Done Done Sent false
3515 1938 Acked Write 3 Done Done Sent false
3516 1939 Acked Read 8 Done Done Sent false
3517 1940 Acked Read 4 Done Done Sent false
3518 1941 Acked Read 8 Done Done Sent false
3519 1942 Acked Write 2 Done Done Sent false
3520 1943 Acked Write 7 Done Done Sent false
3521 1944 Acked Read 7 Done Done Sent false
3522 1945 Acked Read 3 Done Done Sent false
3523 1946 Acked Write 10 Done Done Sent false
3524 1947 Acked Flush 0 Done Done Sent false
3525 1948 Acked Flush 0 Done Done Sent false
3526 1949 Acked Read 3 Done Done Sent false
3527 1950 Acked Read 4 Done Done Sent false
3528 1951 Acked Read 8 Done Done Sent false
3529 1952 Acked Write 6 Done Done Sent false
3530 1953 Acked Write 4 Done Done Sent false
3531 1954 Acked Read 1 Done Done Sent false
3532 1955 Acked Read 6 Done Done Sent false
3533 1956 Acked Read 5 Done Done Sent false
3534 1957 Acked Flush 0 Done Done Sent false
3535 1958 Acked Read 3 Done Done Sent false
3536 1959 Acked Read 6 Done Done Sent false
3537 1960 Acked Read 10 Done Done Sent false
3538 1961 Acked Read 6 Done Done Sent false
3539 1962 Acked Read 7 Done Done Sent false
3540 1963 Acked Write 10 Done Done Sent false
3541 1964 Acked Read 1 Done Done Sent false
3542 1965 Acked Flush 0 Done Done Sent false
3543 1966 Acked Read 5 Done Done Sent false
3544 1967 Acked Write 1 Done Done Sent false
3545 1968 Acked Read 6 Done Done Sent false
3546 1969 Acked Write 6 Done Done Sent false
3547 1970 Acked Flush 0 Done Done Sent false
3548 1971 Acked Read 10 Done Done Sent false
3549 1972 Acked Write 8 Done Done Sent false
3550 1973 Acked Write 8 Done Done Sent false
3551 1974 Acked Flush 0 Done Done Sent false
3552 1975 Acked Read 2 Done Done Sent false
3553 1976 Acked Write 1 Done Done Sent false
3554 1977 Acked Read 6 Done Done Sent false
3555 1978 Acked Flush 0 Done Done Sent false
3556 1979 Acked Read 6 Done Done Sent false
3557 1980 Acked Read 10 Done Done Sent false
3558 1981 Acked Read 10 Done Done Sent false
3559 1982 Acked Read 9 Done Done Sent false
3560 1983 Acked Write 10 Done Done Sent false
3561 1984 Acked Write 2 Done Done Sent false
3562 1985 Acked Read 8 Done Done Sent false
3563 1986 Acked Read 3 Done Done Sent false
3564 1987 Acked Read 10 Done Done Sent false
3565 1988 Acked Read 5 Done Done Sent false
3566 1989 Acked Read 2 Done Done Sent false
3567 1990 Acked Write 5 Done Done Sent false
3568 1991 Acked Read 4 Done Done Sent false
3569 1992 Acked Flush 0 Done Done Sent false
3570 1993 Acked Read 8 Done Done Sent false
3571 1994 Acked Read 8 Done Done Sent false
3572 1995 Acked Write 9 Done Done Sent false
3573 1996 Acked Read 10 Done Done Sent false
3574 1997 Acked Flush 0 Done Done Sent false
3575 1998 Acked Read 7 Done Done Sent false
3576 1999 Acked Read 4 Done Done Sent false
3577 2000 Acked Read 3 Done Done Sent false
3578 2001 Acked Write 6 Done Done Sent false
3579 2002 Acked Write 10 Done Done Sent false
3580 2003 Acked Flush 0 Done Done Sent false
3581 2004 Acked Write 9 Done Done Sent false
3582 2005 Acked Read 10 Done Done Sent false
3583 2006 Acked Read 4 Done Done Sent false
3584 2007 Acked Write 2 Done Done Sent false
3585 2008 Acked Read 2 Done Done Sent false
3586 2009 Acked Write 10 Done Done Sent false
3587 2010 Acked Write 4 Done Done Sent false
3588 2011 Acked Read 7 Done Done Sent false
3589 2012 Acked Read 1 Done Done Sent false
3590 2013 Acked Read 3 Done Done Sent false
3591 2014 Acked Read 6 Done Done Sent false
3592 2015 Acked Write 10 Done Done Sent false
3593 2016 Acked Read 6 Done Done Sent false
3594 2017 Acked Read 4 Done Done Sent false
3595 2018 Acked Read 10 Done Done Sent false
3596 2019 Acked Write 7 Done Done Sent false
3597 2020 Acked Write 5 Done Done Sent false
3598 2021 Acked Write 2 Done Done Sent false
3599 2022 Acked Write 10 Done Done Sent false
3600 2023 Acked Read 7 Done Done Sent false
3601 2024 Acked Read 8 Done Done Sent false
3602 2025 Acked Write 10 Done Done Sent false
3603 2026 Acked Flush 0 Done Done Sent false
3604 2027 Acked Write 5 Done Done Sent false
3605 2028 Acked Read 3 Done Done Sent false
3606 2029 Acked Flush 0 Done Done Sent false
3607 2030 Acked Read 9 Done Done Sent false
3608 2031 Acked Read 10 Done Done Sent false
3609 2032 Acked Read 8 Done Done Sent false
3610 2033 Acked Write 5 Done Done Sent false
3611 2034 Acked Read 4 Done Done Sent false
3612 2035 Acked Flush 0 Done Done Sent false
3613 2036 Acked Read 2 Done Done Sent false
3614 2037 Acked Write 3 Done Done Sent false
3615 2038 Acked Read 9 Done Done Sent false
3616 2039 Acked Read 4 Done Done Sent false
3617 2040 Acked Read 7 Done Done Sent false
3618 2041 Acked Write 4 Done Done Sent false
3619 2042 Acked Write 6 Done Done Sent false
3620 2043 Acked Write 3 Done Done Sent false
3621 2044 Acked Read 9 Done Done Sent false
3622 2045 Acked Read 10 Done Done Sent false
3623 2046 Acked Read 7 Done Done Sent false
3624 2047 Acked Flush 0 Done Done Sent false
3625 2048 Acked Write 3 Done Done Sent false
3626 2049 Acked Flush 0 Done Done Sent false
3627 2050 Acked Write 9 Done Done Sent false
3628 2051 Acked Read 6 Done Done Sent false
3629 2052 Acked Read 2 Done Done Sent false
3630 2053 Acked Read 7 Done Done Sent false
3631 2054 Acked Read 2 Done Done Sent false
3632 2055 Acked Read 8 Done Done Sent false
3633 2056 Acked Read 10 Done Done Sent false
3634 2057 Acked Read 10 Done Done Sent false
3635 2058 Acked Read 8 Done Done Sent false
3636 2059 Acked Read 2 Done Done Sent false
3637 2060 Acked Write 9 Done Done Sent false
3638 2061 Acked Read 7 Done Done Sent false
3639 2062 Acked Read 3 Done Done Sent false
3640 2063 Acked Flush 0 Done Done Sent false
3641 2064 Acked Read 6 Done Done Sent false
3642 2065 Acked Read 6 Done Done Sent false
3643 2066 Acked Flush 0 Done Done Sent false
3644 2067 Acked Flush 0 Done Done Sent false
3645 2068 Acked Read 9 Done Done Sent false
3646 2069 Acked Write 1 Done Done Sent false
3647 2070 Acked Read 2 Done Done Sent false
3648 2071 Acked Write 9 Done Done Sent false
3649 2072 Acked Read 3 Done Done Sent false
3650 2073 Acked Write 5 Done Done Sent false
3651 2074 Acked Write 2 Done Done Sent false
3652 2075 Acked Read 3 Done Done Sent false
3653 2076 Acked Read 8 Done Done Sent false
3654 2077 Acked Flush 0 Done Done Sent false
3655 2078 Acked Read 4 Done Done Sent false
3656 2079 Acked Read 9 Done Done Sent false
3657 2080 Acked Read 10 Done Done Sent false
3658 2081 Acked Write 8 Done Done Sent false
3659 2082 Acked Read 3 Done Done Sent false
3660 2083 Acked Flush 0 Done Done Sent false
3661 2084 Acked Write 9 Done Done Sent false
3662 2085 Acked Write 4 Done Done Sent false
3663 2086 Acked Write 8 Done Done Sent false
3664 2087 Acked Write 1 Done Done Sent false
3665 2088 Acked Write 5 Done Done Sent false
3666 2089 Acked Read 5 Done Done Sent false
3667 2090 Acked Read 2 Done Done Sent false
3668 2091 Acked Read 1 Done Done Sent false
3669 2092 Acked Flush 0 Done Done Sent false
3670 2093 Acked Read 7 Done Done Sent false
3671 2094 Acked Write 4 Done Done Sent false
3672 2095 Acked Read 4 Done Done Sent false
3673 2096 Acked Read 1 Done Done Sent false
3674 2097 Acked Read 10 Done Done Sent false
3675 2098 Acked Read 6 Done Done Sent false
3676 2099 Acked Read 9 Done Done Sent false
3677 2100 Acked Write 2 Done Done Sent false
3678 2101 Acked Read 10 Done Done Sent false
3679 2102 Acked Write 7 Done Done Sent false
3680 2103 Acked Write 8 Done Done Sent false
3681 2104 Acked Read 4 Done Done Sent false
3682 2105 Acked Write 3 Done Done Sent false
3683 2106 Acked Read 1 Done Done Sent false
3684 2107 Acked Write 10 Done Done Sent false
3685 2108 Acked Read 7 Done Done Sent false
3686 2109 Acked Flush 0 Done Done Sent false
3687 2110 Acked Read 4 Done Done Sent false
3688 2111 Acked Read 3 Done Done Sent false
3689 2112 Acked Write 1 Done Done Sent false
3690 2113 Acked Read 4 Done Done Sent false
3691 2114 Acked Read 9 Done Done Sent false
3692 2115 Acked Read 1 Done Done Sent false
3693 2116 Acked Write 1 Done Done Sent false
3694 2117 Acked Read 1 Done Done Sent false
3695 2118 Acked Write 7 Done Done Sent false
3696 2119 Acked Write 4 Done Done Sent false
3697 2120 Acked Read 5 Done Done Sent false
3698 2121 Acked Write 5 Done Done Sent false
3699 2122 Acked Read 2 Done Done Sent false
3700 2123 Acked Read 7 Done Done Sent false
3701 2124 Acked Write 5 Done Done Sent false
3702 2125 Acked Read 10 Done Done Sent false
3703 2126 Acked Write 10 Done Done Sent false
3704 2127 Acked Read 9 Done Done Sent false
3705 2128 Acked Write 3 Done Done Sent false
3706 2129 Acked Read 5 Done Done Sent false
3707 2130 Acked Write 6 Done Done Sent false
3708 2131 Acked Read 10 Done Done Sent false
3709 2132 Acked Flush 0 Done Done Sent false
3710 2133 Acked Read 3 Done Done Sent false
3711 2134 Acked Write 7 Done Done Sent false
3712 2135 Acked Read 4 Done Done Sent false
3713 2136 Acked Write 5 Done Done Sent false
3714 2137 Acked Read 4 Done Done Sent false
3715 2138 Acked Read 8 Done Done Sent false
3716 2139 Acked Flush 0 Done Done Sent false
3717 2140 Acked Read 5 Done Done Sent false
3718 2141 Acked Write 2 Done Done Sent false
3719 2142 Acked Write 1 Done Done Sent false
3720 2143 Acked Write 7 Done Done Sent false
3721 2144 Acked Write 9 Done Done Sent false
3722 2145 Acked Read 3 Done Done Sent false
3723 2146 Acked Write 4 Done Done Sent false
3724 2147 Acked Write 3 Done Done Sent false
3725 2148 Acked Flush 0 Done Done Sent false
3726 2149 Acked Write 4 Done Done Sent false
3727 2150 Acked Write 6 Done Done Sent false
3728 2151 Acked Read 4 Done Done Sent false
3729 2152 Acked Read 7 Done Done Sent false
3730 2153 Acked Read 2 Done Done Sent false
3731 2154 Acked Write 8 Done Done Sent false
3732 2155 Acked Write 4 Done Done Sent false
3733 2156 Acked Read 6 Done Done Sent false
3734 2157 Acked Write 9 Done Done Sent false
3735 2158 Acked Read 7 Done Done Sent false
3736 2159 Acked Write 5 Done Done Sent false
3737 2160 Acked Read 2 Done Done Sent false
3738 2161 Acked Flush 0 Done Done Sent false
3739 2162 Acked Read 1 Done Done Sent false
3740 2163 Acked Write 9 Done Done Sent false
3741 2164 Acked Write 5 Done Done Sent false
3742 2165 Acked Write 7 Done Done Sent false
3743 2166 Acked Read 8 Done Done Sent false
3744 2167 Acked Write 1 Done Done Sent false
3745 2168 Acked Write 6 Done Done Sent false
3746 2169 Acked Write 6 Done Done Sent false
3747 2170 Acked Read 6 Done Done Sent false
3748 2171 Acked Write 3 Done Done Sent false
3749 2172 Acked Read 9 Done Done Sent false
3750 2173 Acked Write 4 Done Done Sent false
3751 2174 Acked Write 6 Done Done Sent false
3752 2175 Acked Read 7 Done Done Sent false
3753 2176 Acked Write 4 Done Done Sent false
3754 2177 Acked Read 2 Done Done Sent false
3755 2178 Acked Write 10 Done Done Sent false
3756 2179 Acked Write 1 Done Done Sent false
3757 2180 Acked Write 7 Done Done Sent false
3758 2181 Acked Read 1 Done Done Sent false
3759 2182 Acked Read 3 Done Done Sent false
3760 2183 Acked Write 4 Done Done Sent false
3761 2184 Acked Read 6 Done Done Sent false
3762 2185 Acked Write 5 Done Done Sent false
3763 2186 Acked Read 3 Done Done Sent false
3764 2187 Acked Write 2 Done Done Sent false
3765 2188 Acked Read 3 Done Done Sent false
3766 2189 Acked Write 2 Done Done Sent false
3767 2190 Acked Write 10 Done Done Sent false
3768 2191 Acked Write 2 Done Done Sent false
3769 2192 Acked Read 1 Done Done Sent false
3770 2193 Acked Read 7 Done Done Sent false
3771 2194 Acked Write 5 Done Done Sent false
3772 2195 Acked Write 2 Done Done Sent false
3773 2196 Acked Flush 0 Done Done Sent false
3774 2197 Acked Read 8 Done Done Sent false
3775 2198 Acked Flush 0 Done Done Sent false
3776 2199 Acked Write 8 Done Done Sent false
3777 2200 Acked Write 9 Done Done Sent false
3778 2201 Acked Write 4 Done Done Sent false
3779 2202 Acked Write 7 Done Done Sent false
3780 2203 Acked Write 10 Done Done Sent false
3781 2204 Acked Read 1 Done Done Sent false
3782 STATES DS:0 DS:1 DS:2 TOTAL
3783 Sent 0 0 301 301
3784 Done 301 301 0 602
3785 Skipped 0 0 0 0
3786 Error 0 0 0 0
3787 Last Flush: 2198 2198 1903
3788 Downstairs last five completed: 1903 1902 1901 1900 1899
3789 Upstairs last five completed: 2204 2203 2202 2201 2200
3790 CLIENT: Up:0 ds:301 act:2
3791 [8830] got start action
3792 [8830] I am going to restart
3793 Starting downstairs at port 8830
3794 Make output file at "/tmp/dsc/downstairs-8830.txt"
3795 [8830][2] reports Starting
3796 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1209)
3797 [8830][2] reports Running
3798 ----------------------------------------------------------------
3799 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
3800 States: Active Active Offline
3801 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
3802 1904 Acked Flush 0 Done Done Sent false
3803 1905 Acked Read 4 Done Done Sent false
3804 1906 Acked Write 1 Done Done Sent false
3805 1907 Acked Read 3 Done Done Sent false
3806 1908 Acked Read 8 Done Done Sent false
3807 1909 Acked Write 9 Done Done Sent false
3808 1910 Acked Write 10 Done Done Sent false
3809 1911 Acked Read 1 Done Done Sent false
3810 1912 Acked Read 7 Done Done Sent false
3811 1913 Acked Read 7 Done Done Sent false
3812 1914 Acked Read 4 Done Done Sent false
3813 1915 Acked Flush 0 Done Done Sent false
3814 1916 Acked Write 9 Done Done Sent false
3815 1917 Acked Read 6 Done Done Sent false
3816 1918 Acked Write 2 Done Done Sent false
3817 1919 Acked Read 1 Done Done Sent false
3818 1920 Acked Write 2 Done Done Sent false
3819 1921 Acked Read 7 Done Done Sent false
3820 1922 Acked Write 7 Done Done Sent false
3821 1923 Acked Read 4 Done Done Sent false
3822 1924 Acked Write 2 Done Done Sent false
3823 1925 Acked Write 9 Done Done Sent false
3824 1926 Acked Flush 0 Done Done Sent false
3825 1927 Acked Write 5 Done Done Sent false
3826 1928 Acked Read 4 Done Done Sent false
3827 1929 Acked Write 6 Done Done Sent false
3828 1930 Acked Read 3 Done Done Sent false
3829 1931 Acked Read 5 Done Done Sent false
3830 1932 Acked Read 6 Done Done Sent false
3831 1933 Acked Write 6 Done Done Sent false
3832 1934 Acked Flush 0 Done Done Sent false
3833 1935 Acked Flush 0 Done Done Sent false
3834 1936 Acked Read 7 Done Done Sent false
3835 1937 Acked Flush 0 Done Done Sent false
3836 1938 Acked Write 3 Done Done Sent false
3837 1939 Acked Read 8 Done Done Sent false
3838 1940 Acked Read 4 Done Done Sent false
3839 1941 Acked Read 8 Done Done Sent false
3840 1942 Acked Write 2 Done Done Sent false
3841 1943 Acked Write 7 Done Done Sent false
3842 1944 Acked Read 7 Done Done Sent false
3843 1945 Acked Read 3 Done Done Sent false
3844 1946 Acked Write 10 Done Done Sent false
3845 1947 Acked Flush 0 Done Done Sent false
3846 1948 Acked Flush 0 Done Done Sent false
3847 1949 Acked Read 3 Done Done Sent false
3848 1950 Acked Read 4 Done Done Sent false
3849 1951 Acked Read 8 Done Done Sent false
3850 1952 Acked Write 6 Done Done Sent false
3851 1953 Acked Write 4 Done Done Sent false
3852 1954 Acked Read 1 Done Done Sent false
3853 1955 Acked Read 6 Done Done Sent false
3854 1956 Acked Read 5 Done Done Sent false
3855 1957 Acked Flush 0 Done Done Sent false
3856 1958 Acked Read 3 Done Done Sent false
3857 1959 Acked Read 6 Done Done Sent false
3858 1960 Acked Read 10 Done Done Sent false
3859 1961 Acked Read 6 Done Done Sent false
3860 1962 Acked Read 7 Done Done Sent false
3861 1963 Acked Write 10 Done Done Sent false
3862 1964 Acked Read 1 Done Done Sent false
3863 1965 Acked Flush 0 Done Done Sent false
3864 1966 Acked Read 5 Done Done Sent false
3865 1967 Acked Write 1 Done Done Sent false
3866 1968 Acked Read 6 Done Done Sent false
3867 1969 Acked Write 6 Done Done Sent false
3868 1970 Acked Flush 0 Done Done Sent false
3869 1971 Acked Read 10 Done Done Sent false
3870 1972 Acked Write 8 Done Done Sent false
3871 1973 Acked Write 8 Done Done Sent false
3872 1974 Acked Flush 0 Done Done Sent false
3873 1975 Acked Read 2 Done Done Sent false
3874 1976 Acked Write 1 Done Done Sent false
3875 1977 Acked Read 6 Done Done Sent false
3876 1978 Acked Flush 0 Done Done Sent false
3877 1979 Acked Read 6 Done Done Sent false
3878 1980 Acked Read 10 Done Done Sent false
3879 1981 Acked Read 10 Done Done Sent false
3880 1982 Acked Read 9 Done Done Sent false
3881 1983 Acked Write 10 Done Done Sent false
3882 1984 Acked Write 2 Done Done Sent false
3883 1985 Acked Read 8 Done Done Sent false
3884 1986 Acked Read 3 Done Done Sent false
3885 1987 Acked Read 10 Done Done Sent false
3886 1988 Acked Read 5 Done Done Sent false
3887 1989 Acked Read 2 Done Done Sent false
3888 1990 Acked Write 5 Done Done Sent false
3889 1991 Acked Read 4 Done Done Sent false
3890 1992 Acked Flush 0 Done Done Sent false
3891 1993 Acked Read 8 Done Done Sent false
3892 1994 Acked Read 8 Done Done Sent false
3893 1995 Acked Write 9 Done Done Sent false
3894 1996 Acked Read 10 Done Done Sent false
3895 1997 Acked Flush 0 Done Done Sent false
3896 1998 Acked Read 7 Done Done Sent false
3897 1999 Acked Read 4 Done Done Sent false
3898 2000 Acked Read 3 Done Done Sent false
3899 2001 Acked Write 6 Done Done Sent false
3900 2002 Acked Write 10 Done Done Sent false
3901 2003 Acked Flush 0 Done Done Sent false
3902 2004 Acked Write 9 Done Done Sent false
3903 2005 Acked Read 10 Done Done Sent false
3904 2006 Acked Read 4 Done Done Sent false
3905 2007 Acked Write 2 Done Done Sent false
3906 2008 Acked Read 2 Done Done Sent false
3907 2009 Acked Write 10 Done Done Sent false
3908 2010 Acked Write 4 Done Done Sent false
3909 2011 Acked Read 7 Done Done Sent false
3910 2012 Acked Read 1 Done Done Sent false
3911 2013 Acked Read 3 Done Done Sent false
3912 2014 Acked Read 6 Done Done Sent false
3913 2015 Acked Write 10 Done Done Sent false
3914 2016 Acked Read 6 Done Done Sent false
3915 2017 Acked Read 4 Done Done Sent false
3916 2018 Acked Read 10 Done Done Sent false
3917 2019 Acked Write 7 Done Done Sent false
3918 2020 Acked Write 5 Done Done Sent false
3919 2021 Acked Write 2 Done Done Sent false
3920 2022 Acked Write 10 Done Done Sent false
3921 2023 Acked Read 7 Done Done Sent false
3922 2024 Acked Read 8 Done Done Sent false
3923 2025 Acked Write 10 Done Done Sent false
3924 2026 Acked Flush 0 Done Done Sent false
3925 2027 Acked Write 5 Done Done Sent false
3926 2028 Acked Read 3 Done Done Sent false
3927 2029 Acked Flush 0 Done Done Sent false
3928 2030 Acked Read 9 Done Done Sent false
3929 2031 Acked Read 10 Done Done Sent false
3930 2032 Acked Read 8 Done Done Sent false
3931 2033 Acked Write 5 Done Done Sent false
3932 2034 Acked Read 4 Done Done Sent false
3933 2035 Acked Flush 0 Done Done Sent false
3934 2036 Acked Read 2 Done Done Sent false
3935 2037 Acked Write 3 Done Done Sent false
3936 2038 Acked Read 9 Done Done Sent false
3937 2039 Acked Read 4 Done Done Sent false
3938 2040 Acked Read 7 Done Done Sent false
3939 2041 Acked Write 4 Done Done Sent false
3940 2042 Acked Write 6 Done Done Sent false
3941 2043 Acked Write 3 Done Done Sent false
3942 2044 Acked Read 9 Done Done Sent false
3943 2045 Acked Read 10 Done Done Sent false
3944 2046 Acked Read 7 Done Done Sent false
3945 2047 Acked Flush 0 Done Done Sent false
3946 2048 Acked Write 3 Done Done Sent false
3947 2049 Acked Flush 0 Done Done Sent false
3948 2050 Acked Write 9 Done Done Sent false
3949 2051 Acked Read 6 Done Done Sent false
3950 2052 Acked Read 2 Done Done Sent false
3951 2053 Acked Read 7 Done Done Sent false
3952 2054 Acked Read 2 Done Done Sent false
3953 2055 Acked Read 8 Done Done Sent false
3954 2056 Acked Read 10 Done Done Sent false
3955 2057 Acked Read 10 Done Done Sent false
3956 2058 Acked Read 8 Done Done Sent false
3957 2059 Acked Read 2 Done Done Sent false
3958 2060 Acked Write 9 Done Done Sent false
3959 2061 Acked Read 7 Done Done Sent false
3960 2062 Acked Read 3 Done Done Sent false
3961 2063 Acked Flush 0 Done Done Sent false
3962 2064 Acked Read 6 Done Done Sent false
3963 2065 Acked Read 6 Done Done Sent false
3964 2066 Acked Flush 0 Done Done Sent false
3965 2067 Acked Flush 0 Done Done Sent false
3966 2068 Acked Read 9 Done Done Sent false
3967 2069 Acked Write 1 Done Done Sent false
3968 2070 Acked Read 2 Done Done Sent false
3969 2071 Acked Write 9 Done Done Sent false
3970 2072 Acked Read 3 Done Done Sent false
3971 2073 Acked Write 5 Done Done Sent false
3972 2074 Acked Write 2 Done Done Sent false
3973 2075 Acked Read 3 Done Done Sent false
3974 2076 Acked Read 8 Done Done Sent false
3975 2077 Acked Flush 0 Done Done Sent false
3976 2078 Acked Read 4 Done Done Sent false
3977 2079 Acked Read 9 Done Done Sent false
3978 2080 Acked Read 10 Done Done Sent false
3979 2081 Acked Write 8 Done Done Sent false
3980 2082 Acked Read 3 Done Done Sent false
3981 2083 Acked Flush 0 Done Done Sent false
3982 2084 Acked Write 9 Done Done Sent false
3983 2085 Acked Write 4 Done Done Sent false
3984 2086 Acked Write 8 Done Done Sent false
3985 2087 Acked Write 1 Done Done Sent false
3986 2088 Acked Write 5 Done Done Sent false
3987 2089 Acked Read 5 Done Done Sent false
3988 2090 Acked Read 2 Done Done Sent false
3989 2091 Acked Read 1 Done Done Sent false
3990 2092 Acked Flush 0 Done Done Sent false
3991 2093 Acked Read 7 Done Done Sent false
3992 2094 Acked Write 4 Done Done Sent false
3993 2095 Acked Read 4 Done Done Sent false
3994 2096 Acked Read 1 Done Done Sent false
3995 2097 Acked Read 10 Done Done Sent false
3996 2098 Acked Read 6 Done Done Sent false
3997 2099 Acked Read 9 Done Done Sent false
3998 2100 Acked Write 2 Done Done Sent false
3999 2101 Acked Read 10 Done Done Sent false
4000 2102 Acked Write 7 Done Done Sent false
4001 2103 Acked Write 8 Done Done Sent false
4002 2104 Acked Read 4 Done Done Sent false
4003 2105 Acked Write 3 Done Done Sent false
4004 2106 Acked Read 1 Done Done Sent false
4005 2107 Acked Write 10 Done Done Sent false
4006 2108 Acked Read 7 Done Done Sent false
4007 2109 Acked Flush 0 Done Done Sent false
4008 2110 Acked Read 4 Done Done Sent false
4009 2111 Acked Read 3 Done Done Sent false
4010 2112 Acked Write 1 Done Done Sent false
4011 2113 Acked Read 4 Done Done Sent false
4012 2114 Acked Read 9 Done Done Sent false
4013 2115 Acked Read 1 Done Done Sent false
4014 2116 Acked Write 1 Done Done Sent false
4015 2117 Acked Read 1 Done Done Sent false
4016 2118 Acked Write 7 Done Done Sent false
4017 2119 Acked Write 4 Done Done Sent false
4018 2120 Acked Read 5 Done Done Sent false
4019 2121 Acked Write 5 Done Done Sent false
4020 2122 Acked Read 2 Done Done Sent false
4021 2123 Acked Read 7 Done Done Sent false
4022 2124 Acked Write 5 Done Done Sent false
4023 2125 Acked Read 10 Done Done Sent false
4024 2126 Acked Write 10 Done Done Sent false
4025 2127 Acked Read 9 Done Done Sent false
4026 2128 Acked Write 3 Done Done Sent false
4027 2129 Acked Read 5 Done Done Sent false
4028 2130 Acked Write 6 Done Done Sent false
4029 2131 Acked Read 10 Done Done Sent false
4030 2132 Acked Flush 0 Done Done Sent false
4031 2133 Acked Read 3 Done Done Sent false
4032 2134 Acked Write 7 Done Done Sent false
4033 2135 Acked Read 4 Done Done Sent false
4034 2136 Acked Write 5 Done Done Sent false
4035 2137 Acked Read 4 Done Done Sent false
4036 2138 Acked Read 8 Done Done Sent false
4037 2139 Acked Flush 0 Done Done Sent false
4038 2140 Acked Read 5 Done Done Sent false
4039 2141 Acked Write 2 Done Done Sent false
4040 2142 Acked Write 1 Done Done Sent false
4041 2143 Acked Write 7 Done Done Sent false
4042 2144 Acked Write 9 Done Done Sent false
4043 2145 Acked Read 3 Done Done Sent false
4044 2146 Acked Write 4 Done Done Sent false
4045 2147 Acked Write 3 Done Done Sent false
4046 2148 Acked Flush 0 Done Done Sent false
4047 2149 Acked Write 4 Done Done Sent false
4048 2150 Acked Write 6 Done Done Sent false
4049 2151 Acked Read 4 Done Done Sent false
4050 2152 Acked Read 7 Done Done Sent false
4051 2153 Acked Read 2 Done Done Sent false
4052 2154 Acked Write 8 Done Done Sent false
4053 2155 Acked Write 4 Done Done Sent false
4054 2156 Acked Read 6 Done Done Sent false
4055 2157 Acked Write 9 Done Done Sent false
4056 2158 Acked Read 7 Done Done Sent false
4057 2159 Acked Write 5 Done Done Sent false
4058 2160 Acked Read 2 Done Done Sent false
4059 2161 Acked Flush 0 Done Done Sent false
4060 2162 Acked Read 1 Done Done Sent false
4061 2163 Acked Write 9 Done Done Sent false
4062 2164 Acked Write 5 Done Done Sent false
4063 2165 Acked Write 7 Done Done Sent false
4064 2166 Acked Read 8 Done Done Sent false
4065 2167 Acked Write 1 Done Done Sent false
4066 2168 Acked Write 6 Done Done Sent false
4067 2169 Acked Write 6 Done Done Sent false
4068 2170 Acked Read 6 Done Done Sent false
4069 2171 Acked Write 3 Done Done Sent false
4070 2172 Acked Read 9 Done Done Sent false
4071 2173 Acked Write 4 Done Done Sent false
4072 2174 Acked Write 6 Done Done Sent false
4073 2175 Acked Read 7 Done Done Sent false
4074 2176 Acked Write 4 Done Done Sent false
4075 2177 Acked Read 2 Done Done Sent false
4076 2178 Acked Write 10 Done Done Sent false
4077 2179 Acked Write 1 Done Done Sent false
4078 2180 Acked Write 7 Done Done Sent false
4079 2181 Acked Read 1 Done Done Sent false
4080 2182 Acked Read 3 Done Done Sent false
4081 2183 Acked Write 4 Done Done Sent false
4082 2184 Acked Read 6 Done Done Sent false
4083 2185 Acked Write 5 Done Done Sent false
4084 2186 Acked Read 3 Done Done Sent false
4085 2187 Acked Write 2 Done Done Sent false
4086 2188 Acked Read 3 Done Done Sent false
4087 2189 Acked Write 2 Done Done Sent false
4088 2190 Acked Write 10 Done Done Sent false
4089 2191 Acked Write 2 Done Done Sent false
4090 2192 Acked Read 1 Done Done Sent false
4091 2193 Acked Read 7 Done Done Sent false
4092 2194 Acked Write 5 Done Done Sent false
4093 2195 Acked Write 2 Done Done Sent false
4094 2196 Acked Flush 0 Done Done Sent false
4095 2197 Acked Read 8 Done Done Sent false
4096 2198 Acked Flush 0 Done Done Sent false
4097 2199 Acked Write 8 Done Done Sent false
4098 2200 Acked Write 9 Done Done Sent false
4099 2201 Acked Write 4 Done Done Sent false
4100 2202 Acked Write 7 Done Done Sent false
4101 2203 Acked Write 10 Done Done Sent false
4102 2204 Acked Read 1 Done Done Sent false
4103 2205 Acked Flush 0 Done Done Sent false
4104 STATES DS:0 DS:1 DS:2 TOTAL
4105 Sent 0 0 302 302
4106 Done 302 302 0 604
4107 Skipped 0 0 0 0
4108 Error 0 0 0 0
4109 Last Flush: 2205 2205 1903
4110 Downstairs last five completed: 1903 1902 1901 1900 1899
4111 Upstairs last five completed: 2205 2204 2203 2202 2201
4112 CLIENT: Up:0 ds:302 act:2
41132024-11-15T01:58:18.581ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
41142024-11-15T01:58:18.581ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:53594) = io task client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
41152024-11-15T01:58:18.582ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
41162024-11-15T01:58:18.583ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
41172024-11-15T01:58:18.583ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
41182024-11-15T01:58:18.583ZINFOcrucible: send last flush ID to this DS: 1903 = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
41192024-11-15T01:58:18.583ZINFOcrucible: Replied this last flush ID: 1903 = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
41202024-11-15T01:58:18.583ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
41212024-11-15T01:58:18.583ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
41222024-11-15T01:58:18.583ZINFOcrucible: [2] client re-new 302 jobs since flush 1903 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
41232024-11-15T01:58:18.584ZINFOcrucible: [2] Replayed 302 jobs since flush: 1903 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
4124 ----------------------------------------------------------------
4125 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
4126 Downstairs last five completed: 2205 2204 2203 2202 2201
4127 Upstairs last five completed: 2205 2204 2203 2202 2201
4128 CLIENT: Up:0 ds:0 act:3
4129 Replay: All jobs finished
4130 Main task has work to do, go find it
4131 got dsc Stop(0)
4132 stop 0
4133 Nov 15 01:58:20.826 INFO request completed, latency_us:[8810] Got stop action so:false kr:false
4134 369, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 77a64b0e-6327-4922-b6e3-00c26bff7a29, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
41352024-11-15T01:58:20.827ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
41362024-11-15T01:58:20.827ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
41372024-11-15T01:58:20.827ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
4138 Nov 15 01:58:20.827 INFO request completed, latency_us: 154, response_code: 200, uri: /state/cid/0, method: GET, req_id:{"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-15T01:58:20.82727122Z","hostname":"w-01JCPRHT5F2MPA44KVJHM3S2PB","pid":1205,"client":"0","":"downstairs","session_id":"971dcb2e-6891-4ecf-b367-1e04e52065ee"}
4139 f5a4c388-5fc6-430c-8191-9e563106c002, remote_addr: 127.{"msg":"[0] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-15T01:58:20.827293369Z","hostname":"w-01JCPRHT5F2MPA44KVJHM3S2PB","pid":1205,"client":"0","":"downstairs","session_id":"971dcb2e-6891-4ecf-b367-1e04e52065ee"}
4140 0.0.1:40008, local_addr: 127.0.0.1:9998
41412024-11-15T01:58:20.827ZINFOcrucible: client task is exiting = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
4142 [8810] Exited with: ExitStatus(unix_wait_status(9))
4143 [8810][0] reports Exit
4144 Nov 15 01:58:24.829 INFO request completed, latency_us: 307, response_code: 200, uri: /state/cid/0, method: GET, req_id: 2143a472-4a3a-42ab-b8bb-896c950ec273, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
4145 001/300 Write block 2446 len 24576 data: 2 2 2 2 2 2
4146 002/300 Write block 3709 len 8192 data: 3 2
4147 003/300 Read block 1850 len 36864
4148 004/300 Read block 1601 len 40960
4149 005/300 Read block 4351 len 12288
4150 006/300 Write block 4966 len 4096 data: 2
4151 007/300 Read block 4796 len 32768
4152 008/300 Flush
4153 009/300 Write block 591 len 20480 data: 4 4 4 3 3
4154 010/300 Read block 1388 len 24576
4155 011/300 Write block 2000 len 20480 data: 2 2 2 3 3
4156 012/300 Write block 473 len 4096 data: 2
4157 013/300 Read block 2186 len 4096
4158 014/300 Flush
4159 015/300 Read block 4773 len 24576
4160 016/300 Write block 755 len 8192 data: 4 4
4161 017/300 Write block 300 len 32768 data: 2 3 3 2 2 2 2 2
4162 018/300 Flush
4163 019/300 Read block 1998 len 40960
4164 020/300 Write block 4580 len 8192 data: 2 2
4165 021/300 Read block 3820 len 4096
4166 022/300 Flush
4167 023/300 Read block 190 len 28672
4168 024/300 Write block 1628 len 32768 data: 2 2 2 2 2 2 2 2
4169 025/300 Write block 375 len 16384 data: 2 2 2 2
4170 026/300 Read block 4991 len 28672
4171 027/300 Flush
4172 028/300 Write block 1885 len 40960 data: 3 3 3 3 3 2 2 2 2 2
4173 029/300 Read block 1500 len 40960
4174 030/300 Read block 1805 len 36864
4175 031/300 Write block 4213 len 20480 data: 3 2 2 2 2
4176 032/300 Write block 1013 len 24576 data: 3 3 4 4 4 4
4177 033/300 Write block 228 len 36864 data: 3 4 4 4 4 3 3 3 4
4178 034/300 Read block 4124 len 40960
4179 035/300 Read block 3236 len 40960
4180 036/300 Read block 3046 len 12288
4181 037/300 Read block 1411 len 36864
4182 038/300 Read block 3106 len 40960
4183 039/300 Read block 865 len 24576
4184 040/300 Read block 3644 len 40960
4185 041/300 Read block 3918 len 24576
4186 042/300 Flush
4187 043/300 Write block 3075 len 28672 data: 4 5 6 5 7 8 7
4188 044/300 Flush
4189 045/300 Read block 1911 len 24576
4190 046/300 Read block 2441 len 12288
4191 047/300 Write block 4339 len 8192 data: 3 3
4192 048/300 Read block 1685 len 8192
4193 049/300 Write block 902 len 32768 data: 2 2 2 2 2 2 2 3
4194 050/300 Read block 2038 len 4096
4195 051/300 Write block 4584 len 36864 data: 2 2 2 2 2 2 2 2 2
4196 052/300 Write block 2447 len 36864 data: 3 3 3 3 3 2 3 3 4
4197 053/300 Write block 4873 len 20480 data: 2 3 3 2 2
4198 054/300 Flush
4199 055/300 Read block 1274 len 20480
4200 056/300 Write block 3398 len 36864 data: 2 2 3 3 3 3 3 3 3
4201 057/300 Read block 4657 len 28672
4202 058/300 Read block 1330 len 4096
4203 059/300 Read block 512 len 4096
4204 060/300 Write block 3535 len 16384 data: 3 3 3 3
4205 061/300 Read block 2664 len 32768
4206 062/300 Flush
4207 063/300 Write block 201 len 32768 data: 2 2 2 2 2 2 2 2
4208 064/300 Read block 4536 len 12288
4209 065/300 Flush
4210 066/300 Read block 1808 len 12288
4211 067/300 Read block 4026 len 32768
4212 068/300 Read block 2562 len 4096
4213 069/300 Read block 878 len 12288
4214 070/300 Write block 4709 len 28672 data: 4 4 4 3 3 3 3
4215 071/300 Write block 1251 len 40960 data: 3 3 2 2 2 2 2 2 2 2
4216 072/300 Read block 4927 len 36864
4217 073/300 Read block 2322 len 4096
4218 074/300 Read block 590 len 36864
4219 075/300 Flush
4220 076/300 Write block 4289 len 20480 data: 3 3 2 2 2
4221 077/300 Read block 2503 len 8192
4222 078/300 Read block 2530 len 4096
4223 079/300 Read block 1161 len 24576
4224 080/300 Read block 8 len 24576
4225 081/300 Flush
4226 082/300 Write block 1739 len 36864 data: 2 2 3 3 4 4 4 4 4
4227 083/300 Write block 4106 len 36864 data: 2 2 3 3 3 3 3 3 3
4228 084/300 Read block 3783 len 28672
4229 085/300 Read block 3019 len 20480
4230 086/300 Read block 2264 len 40960
4231 087/300 Write block 224 len 8192 data: 2 3
4232 088/300 Read block 321 len 4096
4233 089/300 Write block 4778 len 12288 data: 5 4 4
4234 090/300 Read block 430 len 28672
4235 091/300 Read block 3223 len 32768
4236 092/300 Read block 3127 len 16384
4237 093/300 Read block 4036 len 32768
4238 094/300 Read block 3614 len 16384
4239 095/300 Read block 4953 len 12288
4240 096/300 Read block 3439 len 32768
4241 097/300 Write block 1187 len 4096 data: 2
4242 098/300 Write block 1384 len 24576 data: 3 2 2 2 2 2
4243 099/300 Write block 4690 len 28672 data: 2 2 2 2 2 2 2
4244 100/300 Read block 734 len 28672
4245 101/300 Write block 1928 len 32768 data: 3 3 2 2 3 3 3 2
4246 102/300 Read block 2987 len 40960
4247 103/300 Read block 3026 len 28672
4248 104/300 Read block 2429 len 12288
4249 105/300 Read block 4373 len 28672
4250 106/300 Read block 189 len 32768
4251 107/300 Write block 3327 len 40960 data: 2 2 2 2 2 2 2 2 2 2
4252 108/300 Read block 4695 len 16384
4253 109/300 Read block 3556 len 32768
4254 110/300 Read block 4949 len 4096
4255 111/300 Write block 1658 len 32768 data: 3 3 4 4 3 3 3 3
4256 112/300 Read block 1972 len 32768
4257 113/300 Read block 1365 len 16384
4258 114/300 Write block 176 len 8192 data: 3 3
4259 115/300 Write block 149 len 20480 data: 2 2 2 2 2
4260 116/300 Write block 1712 len 8192 data: 2 2
4261 117/300 Write block 4531 len 12288 data: 3 3 3
4262 118/300 Read block 1473 len 32768
4263 119/300 Read block 2566 len 16384
4264 120/300 Read block 53 len 32768
4265 121/300 Write block 3898 len 36864 data: 3 3 3 4 4 4 4 3 3
4266 122/300 Write block 2612 len 36864 data: 3 3 3 3 3 3 3 2 2
4267 123/300 Read block 100 len 24576
4268 124/300 Read block 2435 len 32768
4269 125/300 Write block 1282 len 20480 data: 3 3 2 2 2
4270 126/300 Read block 3757 len 12288
4271 127/300 Write block 3468 len 24576 data: 2 2 2 2 2 2
4272 128/300 Read block 2937 len 40960
4273 129/300 Read block 3725 len 4096
4274 130/300 Read block 2410 len 12288
4275 131/300 Flush
4276 132/300 Write block 4698 len 28672 data: 2 2 2 2 2 2 2
4277 133/300 Write block 1594 len 24576 data: 3 2 2 2 2 2
4278 134/300 Read block 1692 len 12288
4279 135/300 Flush
4280 136/300 Write block 513 len 8192 data: 2 2
4281 137/300 Flush
4282 138/300 Write block 2943 len 24576 data: 4 4 4 5 5 4
4283 139/300 Read block 1993 len 12288
4284 140/300 Read block 4855 len 4096
4285 141/300 Write block 2931 len 24576 data: 2 2 3 3 3 3
4286 142/300 Write block 3797 len 12288 data: 2 3 4
4287 143/300 Write block 672 len 32768 data: 3 3 2 2 2 3 4 4
4288 144/300 Write block 1075 len 4096 data: 3
4289 145/300 Read block 3579 len 36864
4290 146/300 Read block 362 len 28672
4291 147/300 Flush
4292 148/300 Read block 2350 len 4096
4293 149/300 Read block 3407 len 28672
4294 150/300 Read block 4970 len 8192
4295 151/300 Write block 2426 len 36864 data: 2 2 2 2 2 2 2 2 2
4296 152/300 Read block 4507 len 8192
4297 153/300 Write block 1571 len 16384 data: 3 2 2 2
4298 154/300 Read block 3220 len 8192
4299 155/300 Read block 2838 len 28672
4300 156/300 Read block 4841 len 28672
4301 157/300 Read block 873 len 20480
4302 158/300 Read block 4986 len 36864
4303 159/300 Read block 2841 len 36864
4304 160/300 Read block 2337 len 12288
4305 161/300 Read block 2710 len 32768
4306 162/300 Read block 1049 len 32768
4307 163/300 Read block 1976 len 8192
4308 164/300 Read block 3305 len 4096
4309 165/300 Read block 2035 len 12288
4310 166/300 Write block 2846 len 12288 data: 2 2 2
4311 167/300 Read block 3599 len 32768
4312 168/300 Write block 2062 len 8192 data: 3 3
4313 169/300 Write block 4298 len 8192 data: 2 3
4314 170/300 Read block 4381 len 8192
4315 171/300 Flush
4316 172/300 Read block 1838 len 4096
4317 173/300 Read block 161 len 20480
4318 174/300 Read block 3510 len 20480
4319 175/300 Read block 4942 len 36864
4320 176/300 Read block 3673 len 20480
4321 177/300 Read block 4607 len 28672
4322 178/300 Read block 2566 len 40960
4323 179/300 Write block 3673 len 32768 data: 3 3 3 3 3 3 3 3
4324 180/300 Write block 331 len 20480 data: 2 2 3 2 3
4325 181/300 Write block 2011 len 20480 data: 3 3 2 2 2
4326 182/300 Write block 3869 len 4096 data: 3
4327 183/300 Read block 1576 len 4096
4328 184/300 Read block 3721 len 16384
4329 185/300 Write block 4140 len 16384 data: 3 2 3 3
4330 186/300 Write block 3759 len 16384 data: 3 3 3 3
4331 187/300 Write block 3614 len 36864 data: 2 2 2 2 2 2 2 2 2
4332 188/300 Write block 4427 len 28672 data: 3 2 2 2 2 2 2
4333 189/300 Read block 3488 len 24576
4334 190/300 Read block 4340 len 36864
4335 191/300 Read block 303 len 12288
4336 192/300 Read block 1244 len 16384
4337 193/300 Read block 4082 len 8192
4338 194/300 Read block 4501 len 36864
4339 195/300 Write block 2318 len 20480 data: 2 2 2 2 2
4340 196/300 Write block 4316 len 8192 data: 2 2
4341 197/300 Write block 2561 len 28672 data: 4 4 4 3 3 4 4
4342 198/300 Flush
4343 199/300 Flush
4344 200/300 Read block 3489 len 36864
4345 201/300 Read block 877 len 32768
4346 202/300 Read block 4691 len 20480
4347 203/300 Read block 451 len 36864
4348 204/300 Read block 4153 len 24576
4349 205/300 Read block 4851 len 24576
4350 206/300 Flush
4351 207/300 Read block 1060 len 28672
4352 208/300 Read block 660 len 28672
4353 209/300 Write block 3372 len 24576 data: 2 2 2 2 2 2
4354 210/300 Read block 3264 len 32768
4355 211/300 Read block 1908 len 32768
4356 212/300 Read block 2536 len 40960
4357 213/300 Write block 2059 len 28672 data: 3 3 3 4 4 3 2
4358 214/300 Read block 319 len 28672
4359 215/300 Read block 1729 len 4096
4360 216/300 Read block 694 len 32768
4361 217/300 Read block 2832 len 12288
4362 218/300 Write block 2457 len 20480 data: 3 3 3 4 4
4363 219/300 Read block 511 len 28672
4364 220/300 Write block 1898 len 32768 data: 2 2 2 2 2 3 3 3
4365 221/300 Read block 1442 len 36864
4366 222/300 Read block 4648 len 20480
4367 223/300 Read block 4304 len 4096
4368 224/300 Read block 4891 len 40960
4369 225/300 Read block 346 len 40960
4370 226/300 Flush
4371 227/300 Read block 4642 len 8192
4372 228/300 Write block 4589 len 12288 data: 3 3 3
4373 229/300 Read block 1952 len 24576
4374 230/300 Write block 2227 len 28672 data: 3 2 3 3 3 3 3
4375 231/300 Flush
4376 232/300 Read block 4936 len 24576
4377 233/300 Flush
4378 234/300 Write block 703 len 40960 data: 4 4 4 2 2 2 2 2 2 2
4379 235/300 Write block 4290 len 4096 data: 4
4380 236/300 Write block 4893 len 12288 data: 3 4 2
4381 237/300 Flush
4382 238/300 Read block 990 len 16384
4383 239/300 Read block 4602 len 8192
4384 240/300 Write block 2930 len 20480 data: 2 3 3 4 4
4385 241/300 Read block 2540 len 24576
4386 242/300 Write block 3602 len 16384 data: 4 4 4 4
4387 243/300 Read block 2588 len 4096
4388 244/300 Read block 1713 len 24576
4389 245/300 Read block 1245 len 12288
4390 246/300 Read block 756 len 8192
4391 247/300 Read block 3746 len 20480
4392 248/300 Read block 4601 len 40960
4393 249/300 Write block 3407 len 12288 data: 3 5 5
4394 250/300 Flush
4395 251/300 Read block 2404 len 28672
4396 252/300 Write block 623 len 12288 data: 2 2 2
4397 253/300 Write block 1052 len 40960 data: 2 2 2 2 2 2 2 2 2 2
4398 254/300 Write block 3307 len 32768 data: 2 2 2 2 2 2 2 2
4399 255/300 Read block 872 len 32768
4400 256/300 Read block 895 len 40960
4401 257/300 Write block 4355 len 28672 data: 3 3 2 2 2 2 2
4402 258/300 Read block 4373 len 16384
4403 259/300 Write block 4220 len 40960 data: 2 2 2 2 2 2 2 2 2 2
4404 260/300 Read block 3593 len 40960
4405 261/300 Read block 1603 len 40960
4406 262/300 Flush
4407 263/300 Flush
4408 264/300 Read block 4789 len 28672
4409 265/300 Read block 4313 len 24576
4410 266/300 Write block 938 len 32768 data: 3 2 2 2 2 2 2 2
4411 267/300 Read block 1055 len 40960
4412 268/300 Flush
4413 269/300 Write block 4410 len 4096 data: 3
4414 270/300 Flush
4415 271/300 Read block 4712 len 8192
4416 272/300 Write block 3559 len 20480 data: 3 3 3 2 2
4417 273/300 Read block 768 len 16384
4418 274/300 Write block 2325 len 4096 data: 3
4419 275/300 Write block 4825 len 32768 data: 2 2 2 2 2 2 2 2
4420 276/300 Flush
4421 277/300 Read block 2665 len 4096
4422 278/300 Write block 4370 len 28672 data: 2 2 2 2 2 2 2
4423 279/300 Write block 3987 len 16384 data: 2 2 2 2
4424 280/300 Write block 1934 len 36864 data: 4 3 2 2 3 3 3 3 3
4425 281/300 Read block 2195 len 36864
4426 282/300 Read block 2496 len 16384
4427 283/300 Read block 370 len 16384
4428 284/300 Write block 156 len 40960 data: 2 2 2 2 2 2 2 2 3 3
4429 285/300 Read block 4692 len 40960
4430 286/300 Write block 1822 len 40960 data: 2 2 2 2 2 2 2 2 2 2
4431 287/300 Write block 3418 len 16384 data: 2 2 2 2
4432 288/300 Flush
4433 289/300 Read block 2094 len 4096
4434 290/300 Read block 2141 len 28672
4435 291/300 Read block 4603 len 12288
4436 292/300 Write block 4308 len 40960 data: 3 2 2 2 2 2 2 2 3 3
4437 293/300 Write block 2331 len 12288 data: 2 2 2
4438 294/300 Read block 4701 len 4096
4439 295/300 Write block 568 len 4096 data: 2
4440 296/300 Read block 3872 len 36864
4441 297/300 Write block 429 len 24576 data: 2 2 2 2 2 2
4442 298/300 Read block 3982 len 28672
4443 299/300 Write block 2463 len 16384 data: 3 3 3 2
4444 300/300 Write block 2770 len 8192 data: 3 3
4445 301/300 Read block 2372 len 20480
4446 Main task has work to do, go find it
4447 got dsc Start(0)
4448 start 0
4449 Nov 15 01:58:25.077 INFO request completed, latency_us: 213, response_code: 204, uri: /start/cid/0, method: POST, req_id: 01dedb24-d1f8-4460-8ace-075106081397, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
4450 Replay: started 0, returned:Ok(())
4451 ----------------------------------------------------------------
4452 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
4453 States: Offline Active Active
4454 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
4455 2206 Acked Write 6 Sent Done Done false
4456 2207 Acked Write 2 Sent Done Done false
4457 2208 Acked Read 9 Sent Done Done false
4458 2209 Acked Read 10 Sent Done Done false
4459 2210 Acked Read 3 Sent Done Done false
4460 2211 Acked Write 1 Sent Done Done false
4461 2212 Acked Read 8 Sent Done Done false
4462 2213 Acked Flush 0 Sent Done Done false
4463 2214 Acked Write 5 Sent Done Done false
4464 2215 Acked Read 6 Sent Done Done false
4465 2216 Acked Write 5 Sent Done Done false
4466 2217 Acked Write 1 Sent Done Done false
4467 2218 Acked Read 1 Sent Done Done false
4468 2219 Acked Flush 0 Sent Done Done false
4469 2220 Acked Read 6 Sent Done Done false
4470 2221 Acked Write 2 Sent Done Done false
4471 2222 Acked Write 8 Sent Done Done false
4472 2223 Acked Flush 0 Sent Done Done false
4473 2224 Acked Read 10 Sent Done Done false
4474 2225 Acked Write 2 Sent Done Done false
4475 2226 Acked Read 1 Sent Done Done false
4476 2227 Acked Flush 0 Sent Done Done false
4477 2228 Acked Read 7 Sent Done Done false
4478 2229 Acked Write 8 Sent Done Done false
4479 2230 Acked Write 4 Sent Done Done false
4480 2231 Acked Read 7 Sent Done Done false
4481 2232 Acked Flush 0 Sent Done Done false
4482 2233 Acked Write 10 Sent Done Done false
4483 2234 Acked Read 10 Sent Done Done false
4484 2235 Acked Read 9 Sent Done Done false
4485 2236 Acked Write 5 Sent Done Done false
4486 2237 Acked Write 6 Sent Done Done false
4487 2238 Acked Write 9 Sent Done Done false
4488 2239 Acked Read 10 Sent Done Done false
4489 2240 Acked Read 10 Sent Done Done false
4490 2241 Acked Read 3 Sent Done Done false
4491 2242 Acked Read 9 Sent Done Done false
4492 2243 Acked Read 10 Sent Done Done false
4493 2244 Acked Read 6 Sent Done Done false
4494 2245 Acked Read 10 Sent Done Done false
4495 2246 Acked Read 6 Sent Done Done false
4496 2247 Acked Flush 0 Sent Done Done false
4497 2248 Acked Write 7 Sent Done Done false
4498 2249 Acked Flush 0 Sent Done Done false
4499 2250 Acked Read 6 Sent Done Done false
4500 2251 Acked Read 3 Sent Done Done false
4501 2252 Acked Write 2 Sent Done Done false
4502 2253 Acked Read 2 Sent Done Done false
4503 2254 Acked Write 8 Sent Done Done false
4504 2255 Acked Read 1 Sent Done Done false
4505 2256 Acked Write 9 Sent Done Done false
4506 2257 Acked Write 9 Sent Done Done false
4507 2258 Acked Write 5 Sent Done Done false
4508 2259 Acked Flush 0 Sent Done Done false
4509 2260 Acked Read 5 Sent Done Done false
4510 2261 Acked Write 9 Sent Done Done false
4511 2262 Acked Read 7 Sent Done Done false
4512 2263 Acked Read 1 Sent Done Done false
4513 2264 Acked Read 1 Sent Done Done false
4514 2265 Acked Write 4 Sent Done Done false
4515 2266 Acked Read 8 Sent Done Done false
4516 2267 Acked Flush 0 Sent Done Done false
4517 2268 Acked Write 8 Sent Done Done false
4518 2269 Acked Read 3 Sent Done Done false
4519 2270 Acked Flush 0 Sent Done Done false
4520 2271 Acked Read 3 Sent Done Done false
4521 2272 Acked Read 8 Sent Done Done false
4522 2273 Acked Read 1 Sent Done Done false
4523 2274 Acked Read 3 Sent Done Done false
4524 2275 Acked Write 7 Sent Done Done false
4525 2276 Acked Write 10 Sent Done Done false
4526 2277 Acked Read 9 Sent Done Done false
4527 2278 Acked Read 1 Sent Done Done false
4528 2279 Acked Read 9 Sent Done Done false
4529 2280 Acked Flush 0 Sent Done Done false
4530 2281 Acked Write 5 Sent Done Done false
4531 2282 Acked Read 2 Sent Done Done false
4532 2283 Acked Read 1 Sent Done Done false
4533 2284 Acked Read 6 Sent Done Done false
4534 2285 Acked Read 6 Sent Done Done false
4535 2286 Acked Flush 0 Sent Done Done false
4536 2287 Acked Write 9 Sent Done Done false
4537 2288 Acked Write 9 Sent Done Done false
4538 2289 Acked Read 7 Sent Done Done false
4539 2290 Acked Read 5 Sent Done Done false
4540 2291 Acked Read 10 Sent Done Done false
4541 2292 Acked Write 2 Sent Done Done false
4542 2293 Acked Read 1 Sent Done Done false
4543 2294 Acked Write 3 Sent Done Done false
4544 2295 Acked Read 7 Sent Done Done false
4545 2296 Acked Read 8 Sent Done Done false
4546 2297 Acked Read 4 Sent Done Done false
4547 2298 Acked Read 8 Sent Done Done false
4548 2299 Acked Read 4 Sent Done Done false
4549 2300 Acked Read 3 Sent Done Done false
4550 2301 Acked Read 8 Sent Done Done false
4551 2302 Acked Write 1 Sent Done Done false
4552 2303 Acked Write 6 Sent Done Done false
4553 2304 Acked Write 7 Sent Done Done false
4554 2305 Acked Read 7 Sent Done Done false
4555 2306 Acked Write 8 Sent Done Done false
4556 2307 Acked Read 10 Sent Done Done false
4557 2308 Acked Read 7 Sent Done Done false
4558 2309 Acked Read 3 Sent Done Done false
4559 2310 Acked Read 7 Sent Done Done false
4560 2311 Acked Read 8 Sent Done Done false
4561 2312 Acked Write 10 Sent Done Done false
4562 2313 Acked Read 4 Sent Done Done false
4563 2314 Acked Read 8 Sent Done Done false
4564 2315 Acked Read 1 Sent Done Done false
4565 2316 Acked Write 8 Sent Done Done false
4566 2317 Acked Read 8 Sent Done Done false
4567 2318 Acked Read 4 Sent Done Done false
4568 2319 Acked Write 2 Sent Done Done false
4569 2320 Acked Write 5 Sent Done Done false
4570 2321 Acked Write 2 Sent Done Done false
4571 2322 Acked Write 3 Sent Done Done false
4572 2323 Acked Read 8 Sent Done Done false
4573 2324 Acked Read 4 Sent Done Done false
4574 2325 Acked Read 8 Sent Done Done false
4575 2326 Acked Write 9 Sent Done Done false
4576 2327 Acked Write 9 Sent Done Done false
4577 2328 Acked Read 6 Sent Done Done false
4578 2329 Acked Read 8 Sent Done Done false
4579 2330 Acked Write 5 Sent Done Done false
4580 2331 Acked Read 3 Sent Done Done false
4581 2332 Acked Write 6 Sent Done Done false
4582 2333 Acked Read 10 Sent Done Done false
4583 2334 Acked Read 1 Sent Done Done false
4584 2335 Acked Read 3 Sent Done Done false
4585 2336 Acked Flush 0 Sent Done Done false
4586 2337 Acked Write 7 Sent Done Done false
4587 2338 Acked Write 6 Sent Done Done false
4588 2339 Acked Read 3 Sent Done Done false
4589 2340 Acked Flush 0 Sent Done Done false
4590 2341 Acked Write 2 Sent Done Done false
4591 2342 Acked Flush 0 Sent Done Done false
4592 2343 Acked Write 6 Sent Done Done false
4593 2344 Acked Read 3 Sent Done Done false
4594 2345 Acked Read 1 Sent Done Done false
4595 2346 Acked Write 6 Sent Done Done false
4596 2347 Acked Write 3 Sent Done Done false
4597 2348 Acked Write 8 Sent Done Done false
4598 2349 Acked Write 1 Sent Done Done false
4599 2350 Acked Read 9 Sent Done Done false
4600 2351 Acked Read 7 Sent Done Done false
4601 2352 Acked Flush 0 Sent Done Done false
4602 2353 Acked Read 1 Sent Done Done false
4603 2354 Acked Read 7 Sent Done Done false
4604 2355 Acked Read 2 Sent Done Done false
4605 2356 Acked Write 9 Sent Done Done false
4606 2357 Acked Read 2 Sent Done Done false
4607 2358 Acked Write 4 Sent Done Done false
4608 2359 Acked Read 2 Sent Done Done false
4609 2360 Acked Read 7 Sent Done Done false
4610 2361 Acked Read 7 Sent Done Done false
4611 2362 Acked Read 5 Sent Done Done false
4612 2363 Acked Read 9 Sent Done Done false
4613 2364 Acked Read 9 Sent Done Done false
4614 2365 Acked Read 3 Sent Done Done false
4615 2366 Acked Read 8 Sent Done Done false
4616 2367 Acked Read 8 Sent Done Done false
4617 2368 Acked Read 2 Sent Done Done false
4618 2369 Acked Read 1 Sent Done Done false
4619 2370 Acked Read 3 Sent Done Done false
4620 2371 Acked Write 3 Sent Done Done false
4621 2372 Acked Read 8 Sent Done Done false
4622 2373 Acked Write 2 Sent Done Done false
4623 2374 Acked Write 2 Sent Done Done false
4624 2375 Acked Read 2 Sent Done Done false
4625 2376 Acked Flush 0 Sent Done Done false
4626 2377 Acked Read 1 Sent Done Done false
4627 2378 Acked Read 5 Sent Done Done false
4628 2379 Acked Read 5 Sent Done Done false
4629 2380 Acked Read 9 Sent Done Done false
4630 2381 Acked Read 5 Sent Done Done false
4631 2382 Acked Read 7 Sent Done Done false
4632 2383 Acked Read 10 Sent Done Done false
4633 2384 Acked Write 8 Sent Done Done false
4634 2385 Acked Write 5 Sent Done Done false
4635 2386 Acked Write 5 Sent Done Done false
4636 2387 Acked Write 1 Sent Done Done false
4637 2388 Acked Read 1 Sent Done Done false
4638 2389 Acked Read 4 Sent Done Done false
4639 2390 Acked Write 4 Sent Done Done false
4640 2391 Acked Write 4 Sent Done Done false
4641 2392 Acked Write 9 Sent Done Done false
4642 2393 Acked Write 7 Sent Done Done false
4643 2394 Acked Read 6 Sent Done Done false
4644 2395 Acked Read 9 Sent Done Done false
4645 2396 Acked Read 3 Sent Done Done false
4646 2397 Acked Read 4 Sent Done Done false
4647 2398 Acked Read 2 Sent Done Done false
4648 2399 Acked Read 9 Sent Done Done false
4649 2400 Acked Write 5 Sent Done Done false
4650 2401 Acked Write 2 Sent Done Done false
4651 2402 Acked Write 7 Sent Done Done false
4652 2403 Acked Flush 0 Sent Done Done false
4653 2404 Acked Flush 0 Sent Done Done false
4654 2405 Acked Read 9 Sent Done Done false
4655 2406 Acked Read 8 Sent Done Done false
4656 2407 Acked Read 5 Sent Done Done false
4657 2408 Acked Read 9 Sent Done Done false
4658 2409 Acked Read 6 Sent Done Done false
4659 2410 Acked Read 6 Sent Done Done false
4660 2411 Acked Flush 0 Sent Done Done false
4661 2412 Acked Read 7 Sent Done Done false
4662 2413 Acked Read 7 Sent Done Done false
4663 2414 Acked Write 6 Sent Done Done false
4664 2415 Acked Read 8 Sent Done Done false
4665 2416 Acked Read 8 Sent Done Done false
4666 2417 Acked Read 10 Sent Done Done false
4667 2418 Acked Write 7 Sent Done Done false
4668 2419 Acked Read 7 Sent Done Done false
4669 2420 Acked Read 1 Sent Done Done false
4670 2421 Acked Read 8 Sent Done Done false
4671 2422 Acked Read 3 Sent Done Done false
4672 2423 Acked Write 5 Sent Done Done false
4673 2424 Acked Read 7 Sent Done Done false
4674 2425 Acked Write 8 Sent Done Done false
4675 2426 Acked Read 9 Sent Done Done false
4676 2427 Acked Read 5 Sent Done Done false
4677 2428 Acked Read 1 Sent Done Done false
4678 2429 Acked Read 10 Sent Done Done false
4679 2430 Acked Read 10 Sent Done Done false
4680 2431 Acked Flush 0 Sent Done Done false
4681 2432 Acked Read 2 Sent Done Done false
4682 2433 Acked Write 3 Sent Done Done false
4683 2434 Acked Read 6 Sent Done Done false
4684 2435 Acked Write 7 Sent Done Done false
4685 2436 Acked Flush 0 Sent Done Done false
4686 2437 Acked Read 6 Sent Done Done false
4687 2438 Acked Flush 0 Sent Done Done false
4688 2439 Acked Write 10 Sent Done Done false
4689 2440 Acked Write 1 Sent Done Done false
4690 2441 Acked Write 3 Sent Done Done false
4691 2442 Acked Flush 0 Sent Done Done false
4692 2443 Acked Read 4 Sent Done Done false
4693 2444 Acked Read 2 Sent Done Done false
4694 2445 Acked Write 5 Sent Done Done false
4695 2446 Acked Read 6 Sent Done Done false
4696 2447 Acked Write 4 Sent Done Done false
4697 2448 Acked Read 1 Sent Done Done false
4698 2449 Acked Read 6 Sent Done Done false
4699 2450 Acked Read 3 Sent Done Done false
4700 2451 Acked Read 2 Sent Done Done false
4701 2452 Acked Read 5 Sent Done Done false
4702 2453 Acked Read 10 Sent Done Done false
4703 2454 Acked Write 3 Sent Done Done false
4704 2455 Acked Flush 0 Sent Done Done false
4705 2456 Acked Read 7 Sent Done Done false
4706 2457 Acked Write 3 Sent Done Done false
4707 2458 Acked Write 10 Sent Done Done false
4708 2459 Acked Write 8 Sent Done Done false
4709 2460 Acked Read 8 Sent Done Done false
4710 2461 Acked Read 10 Sent Done Done false
4711 2462 Acked Write 7 Sent Done Done false
4712 2463 Acked Read 4 Sent Done Done false
4713 2464 Acked Write 10 Sent Done Done false
4714 2465 Acked Read 10 Sent Done Done false
4715 2466 Acked Read 10 Sent Done Done false
4716 2467 Acked Flush 0 Sent Done Done false
4717 2468 Acked Flush 0 Sent Done Done false
4718 2469 Acked Read 7 Sent Done Done false
4719 2470 Acked Read 6 Sent Done Done false
4720 2471 Acked Write 8 Sent Done Done false
4721 2472 Acked Read 10 Sent Done Done false
4722 2473 Acked Flush 0 Sent Done Done false
4723 2474 Acked Write 1 Sent Done Done false
4724 2475 Acked Flush 0 Sent Done Done false
4725 2476 Acked Read 2 Sent Done Done false
4726 2477 Acked Write 5 Sent Done Done false
4727 2478 Acked Read 4 Sent Done Done false
4728 2479 Acked Write 1 Sent Done Done false
4729 2480 Acked Write 8 Sent Done Done false
4730 2481 Acked Flush 0 Sent Done Done false
4731 2482 Acked Read 1 Sent Done Done false
4732 2483 Acked Write 7 Sent Done Done false
4733 2484 Acked Write 4 Sent Done Done false
4734 2485 Acked Write 9 Sent Done Done false
4735 2486 Acked Read 9 Sent Done Done false
4736 2487 Acked Read 4 Sent Done Done false
4737 2488 Acked Read 4 Sent Done Done false
4738 2489 Acked Write 10 Sent Done Done false
4739 2490 Acked Read 10 Sent Done Done false
4740 2491 Acked Write 10 Sent Done Done false
4741 2492 Acked Write 4 Sent Done Done false
4742 2493 Acked Flush 0 Sent Done Done false
4743 2494 Acked Read 1 Sent Done Done false
4744 2495 Acked Read 7 Sent Done Done false
4745 2496 Acked Read 3 Sent Done Done false
4746 2497 Acked Write 10 Sent Done Done false
4747 2498 Acked Write 3 Sent Done Done false
4748 2499 Acked Read 1 Sent Done Done false
4749 2500 Acked Write 1 Sent Done Done false
4750 2501 Acked Read 9 Sent Done Done false
4751 2502 Acked Write 6 Sent Done Done false
4752 2503 Acked Read 7 Sent Done Done false
4753 2504 Acked Write 4 Sent Done Done false
4754 2505 Acked Write 2 Sent Done Done false
4755 2506 Acked Read 5 Sent Done Done false
4756 STATES DS:0 DS:1 DS:2 TOTAL
4757 Sent 301 0 0 301
4758 Done 0 301 301 602
4759 Skipped 0 0 0 0
4760 Error 0 0 0 0
4761 Last Flush: 2205 2493 2493
4762 Downstairs last five completed: 2205 2204 2203 2202 2201
4763 Upstairs last five completed: 2506 2505 2504 2503 2502
4764 CLIENT: Up:0 ds:301 act:2
4765 ----------------------------------------------------------------
4766 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
4767 States: Offline Active Active
4768 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
4769 2206 Acked Write 6 Sent Done Done false
4770 2207 Acked Write 2 Sent Done Done false
4771 2208 Acked Read 9 Sent Done Done false
4772 2209 Acked Read 10 Sent Done Done false
4773 2210 Acked Read 3 Sent Done Done false
4774 2211 Acked Write 1 Sent Done Done false
4775 2212 Acked Read 8 Sent Done Done false
4776 2213 Acked Flush 0 Sent Done Done false
4777 2214 Acked Write 5 Sent Done Done false
4778 2215 Acked Read 6 Sent Done Done false
4779 2216 Acked Write 5 Sent Done Done false
4780 2217 Acked Write 1 Sent Done Done false
4781 2218 Acked Read 1 Sent Done Done false
4782 2219 Acked Flush 0 Sent Done Done false
4783 2220 Acked Read 6 Sent Done Done false
4784 2221 Acked Write 2 Sent Done Done false
4785 2222 Acked Write 8 Sent Done Done false
4786 2223 Acked Flush 0 Sent Done Done false
4787 2224 Acked Read 10 Sent Done Done false
4788 2225 Acked Write 2 Sent Done Done false
4789 2226 Acked Read 1 Sent Done Done false
4790 2227 Acked Flush 0 Sent Done Done false
4791 2228 Acked Read 7 Sent Done Done false
4792 2229 Acked Write 8 Sent Done Done false
4793 2230 Acked Write 4 Sent Done Done false
4794 2231 Acked Read 7 Sent Done Done false
4795 2232 Acked Flush 0 Sent Done Done false
4796 2233 Acked Write 10 Sent Done Done false
4797 2234 Acked Read 10 Sent Done Done false
4798 2235 Acked Read 9 Sent Done Done false
4799 2236 Acked Write 5 Sent Done Done false
4800 2237 Acked Write 6 Sent Done Done false
4801 2238 Acked Write 9 Sent Done Done false
4802 2239 Acked Read 10 Sent Done Done false
4803 2240 Acked Read 10 Sent Done Done false
4804 2241 Acked Read 3 Sent Done Done false
4805 2242 Acked Read 9 Sent Done Done false
4806 2243 Acked Read 10 Sent Done Done false
4807 2244 Acked Read 6 Sent Done Done false
4808 2245 Acked Read 10 Sent Done Done false
4809 2246 Acked Read 6 Sent Done Done false
4810 2247 Acked Flush 0 Sent Done Done false
4811 2248 Acked Write 7 Sent Done Done false
4812 2249 Acked Flush 0 Sent Done Done false
4813 2250 Acked Read 6 Sent Done Done false
4814 2251 Acked Read 3 Sent Done Done false
4815 2252 Acked Write 2 Sent Done Done false
4816 2253 Acked Read 2 Sent Done Done false
4817 2254 Acked Write 8 Sent Done Done false
4818 2255 Acked Read 1 Sent Done Done false
4819 2256 Acked Write 9 Sent Done Done false
4820 2257 Acked Write 9 Sent Done Done false
4821 2258 Acked Write 5 Sent Done Done false
4822 2259 Acked Flush 0 Sent Done Done false
4823 2260 Acked Read 5 Sent Done Done false
4824 2261 Acked Write 9 Sent Done Done false
4825 2262 Acked Read 7 Sent Done Done false
4826 2263 Acked Read 1 Sent Done Done false
4827 2264 Acked Read 1 Sent Done Done false
4828 2265 Acked Write 4 Sent Done Done false
4829 2266 Acked Read 8 Sent Done Done false
4830 2267 Acked Flush 0 Sent Done Done false
4831 2268 Acked Write 8 Sent Done Done false
4832 2269 Acked Read 3 Sent Done Done false
4833 2270 Acked Flush 0 Sent Done Done false
4834 2271 Acked Read 3 Sent Done Done false
4835 2272 Acked Read 8 Sent Done Done false
4836 2273 Acked Read 1 Sent Done Done false
4837 2274 Acked Read 3 Sent Done Done false
4838 2275 Acked Write 7 Sent Done Done false
4839 2276 Acked Write 10 Sent Done Done false
4840 2277 Acked Read 9 Sent Done Done false
4841 2278 Acked Read 1 Sent Done Done false
4842 2279 Acked Read 9 Sent Done Done false
4843 2280 Acked Flush 0 Sent Done Done false
4844 2281 Acked Write 5 Sent Done Done false
4845 2282 Acked Read 2 Sent Done Done false
4846 2283 Acked Read 1 Sent Done Done false
4847 2284 Acked Read 6 Sent Done Done false
4848 2285 Acked Read 6 Sent Done Done false
4849 2286 Acked Flush 0 Sent Done Done false
4850 2287 Acked Write 9 Sent Done Done false
4851 2288 Acked Write 9 Sent Done Done false
4852 2289 Acked Read 7 Sent Done Done false
4853 2290 Acked Read 5 Sent Done Done false
4854 2291 Acked Read 10 Sent Done Done false
4855 2292 Acked Write 2 Sent Done Done false
4856 2293 Acked Read 1 Sent Done Done false
4857 2294 Acked Write 3 Sent Done Done false
4858 2295 Acked Read 7 Sent Done Done false
4859 2296 Acked Read 8 Sent Done Done false
4860 2297 Acked Read 4 Sent Done Done false
4861 2298 Acked Read 8 Sent Done Done false
4862 2299 Acked Read 4 Sent Done Done false
4863 2300 Acked Read 3 Sent Done Done false
4864 2301 Acked Read 8 Sent Done Done false
4865 2302 Acked Write 1 Sent Done Done false
4866 2303 Acked Write 6 Sent Done Done false
4867 2304 Acked Write 7 Sent Done Done false
4868 2305 Acked Read 7 Sent Done Done false
4869 2306 Acked Write 8 Sent Done Done false
4870 2307 Acked Read 10 Sent Done Done false
4871 2308 Acked Read 7 Sent Done Done false
4872 2309 Acked Read 3 Sent Done Done false
4873 2310 Acked Read 7 Sent Done Done false
4874 2311 Acked Read 8 Sent Done Done false
4875 2312 Acked Write 10 Sent Done Done false
4876 2313 Acked Read 4 Sent Done Done false
4877 2314 Acked Read 8 Sent Done Done false
4878 2315 Acked Read 1 Sent Done Done false
4879 2316 Acked Write 8 Sent Done Done false
4880 2317 Acked Read 8 Sent Done Done false
4881 2318 Acked Read 4 Sent Done Done false
4882 2319 Acked Write 2 Sent Done Done false
4883 2320 Acked Write 5 Sent Done Done false
4884 2321 Acked Write 2 Sent Done Done false
4885 2322 Acked Write 3 Sent Done Done false
4886 2323 Acked Read 8 Sent Done Done false
4887 2324 Acked Read 4 Sent Done Done false
4888 2325 Acked Read 8 Sent Done Done false
4889 2326 Acked Write 9 Sent Done Done false
4890 2327 Acked Write 9 Sent Done Done false
4891 2328 Acked Read 6 Sent Done Done false
4892 2329 Acked Read 8 Sent Done Done false
4893 2330 Acked Write 5 Sent Done Done false
4894 2331 Acked Read 3 Sent Done Done false
4895 2332 Acked Write 6 Sent Done Done false
4896 2333 Acked Read 10 Sent Done Done false
4897 2334 Acked Read 1 Sent Done Done false
4898 2335 Acked Read 3 Sent Done Done false
4899 2336 Acked Flush 0 Sent Done Done false
4900 2337 Acked Write 7 Sent Done Done false
4901 2338 Acked Write 6 Sent Done Done false
4902 2339 Acked Read 3 Sent Done Done false
4903 2340 Acked Flush 0 Sent Done Done false
4904 2341 Acked Write 2 Sent Done Done false
4905 2342 Acked Flush 0 Sent Done Done false
4906 2343 Acked Write 6 Sent Done Done false
4907 2344 Acked Read 3 Sent Done Done false
4908 2345 Acked Read 1 Sent Done Done false
4909 2346 Acked Write 6 Sent Done Done false
4910 2347 Acked Write 3 Sent Done Done false
4911 2348 Acked Write 8 Sent Done Done false
4912 2349 Acked Write 1 Sent Done Done false
4913 2350 Acked Read 9 Sent Done Done false
4914 2351 Acked Read 7 Sent Done Done false
4915 2352 Acked Flush 0 Sent Done Done false
4916 2353 Acked Read 1 Sent Done Done false
4917 2354 Acked Read 7 Sent Done Done false
4918 2355 Acked Read 2 Sent Done Done false
4919 2356 Acked Write 9 Sent Done Done false
4920 2357 Acked Read 2 Sent Done Done false
4921 2358 Acked Write 4 Sent Done Done false
4922 2359 Acked Read 2 Sent Done Done false
4923 2360 Acked Read 7 Sent Done Done false
4924 2361 Acked Read 7 Sent Done Done false
4925 2362 Acked Read 5 Sent Done Done false
4926 2363 Acked Read 9 Sent Done Done false
4927 2364 Acked Read 9 Sent Done Done false
4928 2365 Acked Read 3 Sent Done Done false
4929 2366 Acked Read 8 Sent Done Done false
4930 2367 Acked Read 8 Sent Done Done false
4931 2368 Acked Read 2 Sent Done Done false
4932 2369 Acked Read 1 Sent Done Done false
4933 2370 Acked Read 3 Sent Done Done false
4934 2371 Acked Write 3 Sent Done Done false
4935 2372 Acked Read 8 Sent Done Done false
4936 2373 Acked Write 2 Sent Done Done false
4937 2374 Acked Write 2 Sent Done Done false
4938 2375 Acked Read 2 Sent Done Done false
4939 2376 Acked Flush 0 Sent Done Done false
4940 2377 Acked Read 1 Sent Done Done false
4941 2378 Acked Read 5 Sent Done Done false
4942 2379 Acked Read 5 Sent Done Done false
4943 2380 Acked Read 9 Sent Done Done false
4944 2381 Acked Read 5 Sent Done Done false
4945 2382 Acked Read 7 Sent Done Done false
4946 2383 Acked Read 10 Sent Done Done false
4947 2384 Acked Write 8 Sent Done Done false
4948 2385 Acked Write 5 Sent Done Done false
4949 2386 Acked Write 5 Sent Done Done false
4950 2387 Acked Write 1 Sent Done Done false
4951 2388 Acked Read 1 Sent Done Done false
4952 2389 Acked Read 4 Sent Done Done false
4953 2390 Acked Write 4 Sent Done Done false
4954 2391 Acked Write 4 Sent Done Done false
4955 2392 Acked Write 9 Sent Done Done false
4956 2393 Acked Write 7 Sent Done Done false
4957 2394 Acked Read 6 Sent Done Done false
4958 2395 Acked Read 9 Sent Done Done false
4959 2396 Acked Read 3 Sent Done Done false
4960 2397 Acked Read 4 Sent Done Done false
4961 2398 Acked Read 2 Sent Done Done false
4962 2399 Acked Read 9 Sent Done Done false
4963 2400 Acked Write 5 Sent Done Done false
4964 2401 Acked Write 2 Sent Done Done false
4965 2402 Acked Write 7 Sent Done Done false
4966 2403 Acked Flush 0 Sent Done Done false
4967 2404 Acked Flush 0 Sent Done Done false
4968 2405 Acked Read 9 Sent Done Done false
4969 2406 Acked Read 8 Sent Done Done false
4970 2407 Acked Read 5 Sent Done Done false
4971 2408 Acked Read 9 Sent Done Done false
4972 2409 Acked Read 6 Sent Done Done false
4973 2410 Acked Read 6 Sent Done Done false
4974 2411 Acked Flush 0 Sent Done Done false
4975 2412 Acked Read 7 Sent Done Done false
4976 2413 Acked Read 7 Sent Done Done false
4977 2414 Acked Write 6 Sent Done Done false
4978 2415 Acked Read 8 Sent Done Done false
4979 2416 Acked Read 8 Sent Done Done false
4980 2417 Acked Read 10 Sent Done Done false
4981 2418 Acked Write 7 Sent Done Done false
4982 2419 Acked Read 7 Sent Done Done false
4983 2420 Acked Read 1 Sent Done Done false
4984 2421 Acked Read 8 Sent Done Done false
4985 2422 Acked Read 3 Sent Done Done false
4986 2423 Acked Write 5 Sent Done Done false
4987 2424 Acked Read 7 Sent Done Done false
4988 2425 Acked Write 8 Sent Done Done false
4989 2426 Acked Read 9 Sent Done Done false
4990 2427 Acked Read 5 Sent Done Done false
4991 2428 Acked Read 1 Sent Done Done false
4992 2429 Acked Read 10 Sent Done Done false
4993 2430 Acked Read 10 Sent Done Done false
4994 2431 Acked Flush 0 Sent Done Done false
4995 2432 Acked Read 2 Sent Done Done false
4996 2433 Acked Write 3 Sent Done Done false
4997 2434 Acked Read 6 Sent Done Done false
4998 2435 Acked Write 7 Sent Done Done false
4999 2436 Acked Flush 0 Sent Done Done false
5000 2437 Acked Read 6 Sent Done Done false
5001 2438 Acked Flush 0 Sent Done Done false
5002 2439 Acked Write 10 Sent Done Done false
5003 2440 Acked Write 1 Sent Done Done false
5004 2441 Acked Write 3 Sent Done Done false
5005 2442 Acked Flush 0 Sent Done Done false
5006 2443 Acked Read 4 Sent Done Done false
5007 2444 Acked Read 2 Sent Done Done false
5008 2445 Acked Write 5 Sent Done Done false
5009 2446 Acked Read 6 Sent Done Done false
5010 2447 Acked Write 4 Sent Done Done false
5011 2448 Acked Read 1 Sent Done Done false
5012 2449 Acked Read 6 Sent Done Done false
5013 2450 Acked Read 3 Sent Done Done false
5014 2451 Acked Read 2 Sent Done Done false
5015 2452 Acked Read 5 Sent Done Done false
5016 2453 Acked Read 10 Sent Done Done false
5017 2454 Acked Write 3 Sent Done Done false
5018 2455 Acked Flush 0 Sent Done Done false
5019 2456 Acked Read 7 Sent Done Done false
5020 2457 Acked Write 3 Sent Done Done false
5021 2458 Acked Write 10 Sent Done Done false
5022 2459 Acked Write 8 Sent Done Done false
5023 2460 Acked Read 8 Sent Done Done false
5024 2461 Acked Read 10 Sent Done Done false
5025 2462 Acked Write 7 Sent Done Done false
5026 2463 Acked Read 4 Sent Done Done false
5027 2464 Acked Write 10 Sent Done Done false
5028 2465 Acked Read 10 Sent Done Done false
5029 2466 Acked Read 10 Sent Done Done false
5030 2467 Acked Flush 0 Sent Done Done false
5031 2468 Acked Flush 0 Sent Done Done false
5032 2469 Acked Read 7 Sent Done Done false
5033 2470 Acked Read 6 Sent Done Done false
5034 2471 Acked Write 8 Sent Done Done false
5035 2472 Acked Read 10 Sent Done Done false
5036 2473 Acked Flush 0 Sent Done Done false
5037 2474 Acked Write 1 Sent Done Done false
5038 2475 Acked Flush 0 Sent Done Done false
5039 2476 Acked Read 2 Sent Done Done false
5040 2477 Acked Write 5 Sent Done Done false
5041 2478 Acked Read 4 Sent Done Done false
5042 2479 Acked Write 1 Sent Done Done false
5043 2480 Acked Write 8 Sent Done Done false
5044 2481 Acked Flush 0 Sent Done Done false
5045 2482 Acked Read 1 Sent Done Done false
5046 2483 Acked Write 7 Sent Done Done false
5047 2484 Acked Write 4 Sent Done Done false
5048 2485 Acked Write 9 Sent Done Done false
5049 2486 Acked Read 9 Sent Done Done false
5050 2487 Acked Read 4 Sent Done Done false
5051 2488 Acked Read 4 Sent Done Done false
5052 2489 Acked Write 10 Sent Done Done false
5053 2490 Acked Read 10 Sent Done Done false
5054 2491 Acked Write 10 Sent Done Done false
5055 2492 Acked Write 4 Sent Done Done false
5056 2493 Acked Flush 0 Sent Done Done false
5057 2494 Acked Read 1 Sent Done Done false
5058 2495 Acked Read 7 Sent Done Done false
5059 2496 Acked Read 3 Sent Done Done false
5060 2497 Acked Write 10 Sent Done Done false
5061 2498 Acked Write 3 Sent Done Done false
5062 2499 Acked Read 1 Sent Done Done false
5063 2500 Acked Write 1 Sent Done Done false
5064 2501 Acked Read 9 Sent Done Done false
5065 2502 Acked Write 6 Sent Done Done false
5066 2503 Acked Read 7 Sent Done Done false
5067 2504 Acked Write 4 Sent Done Done false
5068 2505 Acked Write 2 Sent Done Done false
5069 2506 Acked Read 5 Sent Done Done false
5070 2507 Acked Flush 0 Sent Done Done false
5071 STATES DS:0 DS:1 DS:2 TOTAL
5072 Sent 302 0 0 302
5073 Done 0 302 302 604
5074 Skipped 0 0 0 0
5075 Error 0 0 0 0
5076 Last Flush: 2205 2507 2507
5077 Downstairs last five completed: 2205 2204 2203 2202 2201
5078 Upstairs last five completed: 2507 2506 2505 2504 2503
5079 CLIENT: Up:0 ds:302 act:2
50802024-11-15T01:58:30.829ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
50812024-11-15T01:58:30.829ZWARNcrucible: ds_connection connect to 127.0.0.1:8810 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
50822024-11-15T01:58:30.829ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
50832024-11-15T01:58:30.829ZWARNcrucible: downstairs task for 0 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
50842024-11-15T01:58:30.829ZINFOcrucible: client task is exiting = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
5085 [8810] got start action
5086 [8810] I am going to restart
5087 Starting downstairs at port 8810
5088 Make output file at "/tmp/dsc/downstairs-8810.txt"
5089 [8810][0] reports Starting
5090 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1210)
5091 [8810][0] reports Running
5092 ----------------------------------------------------------------
5093 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
5094 States: Offline Active Active
5095 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
5096 2206 Acked Write 6 Sent Done Done false
5097 2207 Acked Write 2 Sent Done Done false
5098 2208 Acked Read 9 Sent Done Done false
5099 2209 Acked Read 10 Sent Done Done false
5100 2210 Acked Read 3 Sent Done Done false
5101 2211 Acked Write 1 Sent Done Done false
5102 2212 Acked Read 8 Sent Done Done false
5103 2213 Acked Flush 0 Sent Done Done false
5104 2214 Acked Write 5 Sent Done Done false
5105 2215 Acked Read 6 Sent Done Done false
5106 2216 Acked Write 5 Sent Done Done false
5107 2217 Acked Write 1 Sent Done Done false
5108 2218 Acked Read 1 Sent Done Done false
5109 2219 Acked Flush 0 Sent Done Done false
5110 2220 Acked Read 6 Sent Done Done false
5111 2221 Acked Write 2 Sent Done Done false
5112 2222 Acked Write 8 Sent Done Done false
5113 2223 Acked Flush 0 Sent Done Done false
5114 2224 Acked Read 10 Sent Done Done false
5115 2225 Acked Write 2 Sent Done Done false
5116 2226 Acked Read 1 Sent Done Done false
5117 2227 Acked Flush 0 Sent Done Done false
5118 2228 Acked Read 7 Sent Done Done false
5119 2229 Acked Write 8 Sent Done Done false
5120 2230 Acked Write 4 Sent Done Done false
5121 2231 Acked Read 7 Sent Done Done false
5122 2232 Acked Flush 0 Sent Done Done false
5123 2233 Acked Write 10 Sent Done Done false
5124 2234 Acked Read 10 Sent Done Done false
5125 2235 Acked Read 9 Sent Done Done false
5126 2236 Acked Write 5 Sent Done Done false
5127 2237 Acked Write 6 Sent Done Done false
5128 2238 Acked Write 9 Sent Done Done false
5129 2239 Acked Read 10 Sent Done Done false
5130 2240 Acked Read 10 Sent Done Done false
5131 2241 Acked Read 3 Sent Done Done false
5132 2242 Acked Read 9 Sent Done Done false
5133 2243 Acked Read 10 Sent Done Done false
5134 2244 Acked Read 6 Sent Done Done false
5135 2245 Acked Read 10 Sent Done Done false
5136 2246 Acked Read 6 Sent Done Done false
5137 2247 Acked Flush 0 Sent Done Done false
5138 2248 Acked Write 7 Sent Done Done false
5139 2249 Acked Flush 0 Sent Done Done false
5140 2250 Acked Read 6 Sent Done Done false
5141 2251 Acked Read 3 Sent Done Done false
5142 2252 Acked Write 2 Sent Done Done false
5143 2253 Acked Read 2 Sent Done Done false
5144 2254 Acked Write 8 Sent Done Done false
5145 2255 Acked Read 1 Sent Done Done false
5146 2256 Acked Write 9 Sent Done Done false
5147 2257 Acked Write 9 Sent Done Done false
5148 2258 Acked Write 5 Sent Done Done false
5149 2259 Acked Flush 0 Sent Done Done false
5150 2260 Acked Read 5 Sent Done Done false
5151 2261 Acked Write 9 Sent Done Done false
5152 2262 Acked Read 7 Sent Done Done false
5153 2263 Acked Read 1 Sent Done Done false
5154 2264 Acked Read 1 Sent Done Done false
5155 2265 Acked Write 4 Sent Done Done false
5156 2266 Acked Read 8 Sent Done Done false
5157 2267 Acked Flush 0 Sent Done Done false
5158 2268 Acked Write 8 Sent Done Done false
5159 2269 Acked Read 3 Sent Done Done false
5160 2270 Acked Flush 0 Sent Done Done false
5161 2271 Acked Read 3 Sent Done Done false
5162 2272 Acked Read 8 Sent Done Done false
5163 2273 Acked Read 1 Sent Done Done false
5164 2274 Acked Read 3 Sent Done Done false
5165 2275 Acked Write 7 Sent Done Done false
5166 2276 Acked Write 10 Sent Done Done false
5167 2277 Acked Read 9 Sent Done Done false
5168 2278 Acked Read 1 Sent Done Done false
5169 2279 Acked Read 9 Sent Done Done false
5170 2280 Acked Flush 0 Sent Done Done false
5171 2281 Acked Write 5 Sent Done Done false
5172 2282 Acked Read 2 Sent Done Done false
5173 2283 Acked Read 1 Sent Done Done false
5174 2284 Acked Read 6 Sent Done Done false
5175 2285 Acked Read 6 Sent Done Done false
5176 2286 Acked Flush 0 Sent Done Done false
5177 2287 Acked Write 9 Sent Done Done false
5178 2288 Acked Write 9 Sent Done Done false
5179 2289 Acked Read 7 Sent Done Done false
5180 2290 Acked Read 5 Sent Done Done false
5181 2291 Acked Read 10 Sent Done Done false
5182 2292 Acked Write 2 Sent Done Done false
5183 2293 Acked Read 1 Sent Done Done false
5184 2294 Acked Write 3 Sent Done Done false
5185 2295 Acked Read 7 Sent Done Done false
5186 2296 Acked Read 8 Sent Done Done false
5187 2297 Acked Read 4 Sent Done Done false
5188 2298 Acked Read 8 Sent Done Done false
5189 2299 Acked Read 4 Sent Done Done false
5190 2300 Acked Read 3 Sent Done Done false
5191 2301 Acked Read 8 Sent Done Done false
5192 2302 Acked Write 1 Sent Done Done false
5193 2303 Acked Write 6 Sent Done Done false
5194 2304 Acked Write 7 Sent Done Done false
5195 2305 Acked Read 7 Sent Done Done false
5196 2306 Acked Write 8 Sent Done Done false
5197 2307 Acked Read 10 Sent Done Done false
5198 2308 Acked Read 7 Sent Done Done false
5199 2309 Acked Read 3 Sent Done Done false
5200 2310 Acked Read 7 Sent Done Done false
5201 2311 Acked Read 8 Sent Done Done false
5202 2312 Acked Write 10 Sent Done Done false
5203 2313 Acked Read 4 Sent Done Done false
5204 2314 Acked Read 8 Sent Done Done false
5205 2315 Acked Read 1 Sent Done Done false
5206 2316 Acked Write 8 Sent Done Done false
5207 2317 Acked Read 8 Sent Done Done false
5208 2318 Acked Read 4 Sent Done Done false
5209 2319 Acked Write 2 Sent Done Done false
5210 2320 Acked Write 5 Sent Done Done false
5211 2321 Acked Write 2 Sent Done Done false
5212 2322 Acked Write 3 Sent Done Done false
5213 2323 Acked Read 8 Sent Done Done false
5214 2324 Acked Read 4 Sent Done Done false
5215 2325 Acked Read 8 Sent Done Done false
5216 2326 Acked Write 9 Sent Done Done false
5217 2327 Acked Write 9 Sent Done Done false
5218 2328 Acked Read 6 Sent Done Done false
5219 2329 Acked Read 8 Sent Done Done false
5220 2330 Acked Write 5 Sent Done Done false
5221 2331 Acked Read 3 Sent Done Done false
5222 2332 Acked Write 6 Sent Done Done false
5223 2333 Acked Read 10 Sent Done Done false
5224 2334 Acked Read 1 Sent Done Done false
5225 2335 Acked Read 3 Sent Done Done false
5226 2336 Acked Flush 0 Sent Done Done false
5227 2337 Acked Write 7 Sent Done Done false
5228 2338 Acked Write 6 Sent Done Done false
5229 2339 Acked Read 3 Sent Done Done false
5230 2340 Acked Flush 0 Sent Done Done false
5231 2341 Acked Write 2 Sent Done Done false
5232 2342 Acked Flush 0 Sent Done Done false
5233 2343 Acked Write 6 Sent Done Done false
5234 2344 Acked Read 3 Sent Done Done false
5235 2345 Acked Read 1 Sent Done Done false
5236 2346 Acked Write 6 Sent Done Done false
5237 2347 Acked Write 3 Sent Done Done false
5238 2348 Acked Write 8 Sent Done Done false
5239 2349 Acked Write 1 Sent Done Done false
5240 2350 Acked Read 9 Sent Done Done false
5241 2351 Acked Read 7 Sent Done Done false
5242 2352 Acked Flush 0 Sent Done Done false
5243 2353 Acked Read 1 Sent Done Done false
5244 2354 Acked Read 7 Sent Done Done false
5245 2355 Acked Read 2 Sent Done Done false
5246 2356 Acked Write 9 Sent Done Done false
5247 2357 Acked Read 2 Sent Done Done false
5248 2358 Acked Write 4 Sent Done Done false
5249 2359 Acked Read 2 Sent Done Done false
5250 2360 Acked Read 7 Sent Done Done false
5251 2361 Acked Read 7 Sent Done Done false
5252 2362 Acked Read 5 Sent Done Done false
5253 2363 Acked Read 9 Sent Done Done false
5254 2364 Acked Read 9 Sent Done Done false
5255 2365 Acked Read 3 Sent Done Done false
5256 2366 Acked Read 8 Sent Done Done false
5257 2367 Acked Read 8 Sent Done Done false
5258 2368 Acked Read 2 Sent Done Done false
5259 2369 Acked Read 1 Sent Done Done false
5260 2370 Acked Read 3 Sent Done Done false
5261 2371 Acked Write 3 Sent Done Done false
5262 2372 Acked Read 8 Sent Done Done false
5263 2373 Acked Write 2 Sent Done Done false
5264 2374 Acked Write 2 Sent Done Done false
5265 2375 Acked Read 2 Sent Done Done false
5266 2376 Acked Flush 0 Sent Done Done false
5267 2377 Acked Read 1 Sent Done Done false
5268 2378 Acked Read 5 Sent Done Done false
5269 2379 Acked Read 5 Sent Done Done false
5270 2380 Acked Read 9 Sent Done Done false
5271 2381 Acked Read 5 Sent Done Done false
5272 2382 Acked Read 7 Sent Done Done false
5273 2383 Acked Read 10 Sent Done Done false
5274 2384 Acked Write 8 Sent Done Done false
5275 2385 Acked Write 5 Sent Done Done false
5276 2386 Acked Write 5 Sent Done Done false
5277 2387 Acked Write 1 Sent Done Done false
5278 2388 Acked Read 1 Sent Done Done false
5279 2389 Acked Read 4 Sent Done Done false
5280 2390 Acked Write 4 Sent Done Done false
5281 2391 Acked Write 4 Sent Done Done false
5282 2392 Acked Write 9 Sent Done Done false
5283 2393 Acked Write 7 Sent Done Done false
5284 2394 Acked Read 6 Sent Done Done false
5285 2395 Acked Read 9 Sent Done Done false
5286 2396 Acked Read 3 Sent Done Done false
5287 2397 Acked Read 4 Sent Done Done false
5288 2398 Acked Read 2 Sent Done Done false
5289 2399 Acked Read 9 Sent Done Done false
5290 2400 Acked Write 5 Sent Done Done false
5291 2401 Acked Write 2 Sent Done Done false
5292 2402 Acked Write 7 Sent Done Done false
5293 2403 Acked Flush 0 Sent Done Done false
5294 2404 Acked Flush 0 Sent Done Done false
5295 2405 Acked Read 9 Sent Done Done false
5296 2406 Acked Read 8 Sent Done Done false
5297 2407 Acked Read 5 Sent Done Done false
5298 2408 Acked Read 9 Sent Done Done false
5299 2409 Acked Read 6 Sent Done Done false
5300 2410 Acked Read 6 Sent Done Done false
5301 2411 Acked Flush 0 Sent Done Done false
5302 2412 Acked Read 7 Sent Done Done false
5303 2413 Acked Read 7 Sent Done Done false
5304 2414 Acked Write 6 Sent Done Done false
5305 2415 Acked Read 8 Sent Done Done false
5306 2416 Acked Read 8 Sent Done Done false
5307 2417 Acked Read 10 Sent Done Done false
5308 2418 Acked Write 7 Sent Done Done false
5309 2419 Acked Read 7 Sent Done Done false
5310 2420 Acked Read 1 Sent Done Done false
5311 2421 Acked Read 8 Sent Done Done false
5312 2422 Acked Read 3 Sent Done Done false
5313 2423 Acked Write 5 Sent Done Done false
5314 2424 Acked Read 7 Sent Done Done false
5315 2425 Acked Write 8 Sent Done Done false
5316 2426 Acked Read 9 Sent Done Done false
5317 2427 Acked Read 5 Sent Done Done false
5318 2428 Acked Read 1 Sent Done Done false
5319 2429 Acked Read 10 Sent Done Done false
5320 2430 Acked Read 10 Sent Done Done false
5321 2431 Acked Flush 0 Sent Done Done false
5322 2432 Acked Read 2 Sent Done Done false
5323 2433 Acked Write 3 Sent Done Done false
5324 2434 Acked Read 6 Sent Done Done false
5325 2435 Acked Write 7 Sent Done Done false
5326 2436 Acked Flush 0 Sent Done Done false
5327 2437 Acked Read 6 Sent Done Done false
5328 2438 Acked Flush 0 Sent Done Done false
5329 2439 Acked Write 10 Sent Done Done false
5330 2440 Acked Write 1 Sent Done Done false
5331 2441 Acked Write 3 Sent Done Done false
5332 2442 Acked Flush 0 Sent Done Done false
5333 2443 Acked Read 4 Sent Done Done false
5334 2444 Acked Read 2 Sent Done Done false
5335 2445 Acked Write 5 Sent Done Done false
5336 2446 Acked Read 6 Sent Done Done false
5337 2447 Acked Write 4 Sent Done Done false
5338 2448 Acked Read 1 Sent Done Done false
5339 2449 Acked Read 6 Sent Done Done false
5340 2450 Acked Read 3 Sent Done Done false
5341 2451 Acked Read 2 Sent Done Done false
5342 2452 Acked Read 5 Sent Done Done false
5343 2453 Acked Read 10 Sent Done Done false
5344 2454 Acked Write 3 Sent Done Done false
5345 2455 Acked Flush 0 Sent Done Done false
5346 2456 Acked Read 7 Sent Done Done false
5347 2457 Acked Write 3 Sent Done Done false
5348 2458 Acked Write 10 Sent Done Done false
5349 2459 Acked Write 8 Sent Done Done false
5350 2460 Acked Read 8 Sent Done Done false
5351 2461 Acked Read 10 Sent Done Done false
5352 2462 Acked Write 7 Sent Done Done false
5353 2463 Acked Read 4 Sent Done Done false
5354 2464 Acked Write 10 Sent Done Done false
5355 2465 Acked Read 10 Sent Done Done false
5356 2466 Acked Read 10 Sent Done Done false
5357 2467 Acked Flush 0 Sent Done Done false
5358 2468 Acked Flush 0 Sent Done Done false
5359 2469 Acked Read 7 Sent Done Done false
5360 2470 Acked Read 6 Sent Done Done false
5361 2471 Acked Write 8 Sent Done Done false
5362 2472 Acked Read 10 Sent Done Done false
5363 2473 Acked Flush 0 Sent Done Done false
5364 2474 Acked Write 1 Sent Done Done false
5365 2475 Acked Flush 0 Sent Done Done false
5366 2476 Acked Read 2 Sent Done Done false
5367 2477 Acked Write 5 Sent Done Done false
5368 2478 Acked Read 4 Sent Done Done false
5369 2479 Acked Write 1 Sent Done Done false
5370 2480 Acked Write 8 Sent Done Done false
5371 2481 Acked Flush 0 Sent Done Done false
5372 2482 Acked Read 1 Sent Done Done false
5373 2483 Acked Write 7 Sent Done Done false
5374 2484 Acked Write 4 Sent Done Done false
5375 2485 Acked Write 9 Sent Done Done false
5376 2486 Acked Read 9 Sent Done Done false
5377 2487 Acked Read 4 Sent Done Done false
5378 2488 Acked Read 4 Sent Done Done false
5379 2489 Acked Write 10 Sent Done Done false
5380 2490 Acked Read 10 Sent Done Done false
5381 2491 Acked Write 10 Sent Done Done false
5382 2492 Acked Write 4 Sent Done Done false
5383 2493 Acked Flush 0 Sent Done Done false
5384 2494 Acked Read 1 Sent Done Done false
5385 2495 Acked Read 7 Sent Done Done false
5386 2496 Acked Read 3 Sent Done Done false
5387 2497 Acked Write 10 Sent Done Done false
5388 2498 Acked Write 3 Sent Done Done false
5389 2499 Acked Read 1 Sent Done Done false
5390 2500 Acked Write 1 Sent Done Done false
5391 2501 Acked Read 9 Sent Done Done false
5392 2502 Acked Write 6 Sent Done Done false
5393 2503 Acked Read 7 Sent Done Done false
5394 2504 Acked Write 4 Sent Done Done false
5395 2505 Acked Write 2 Sent Done Done false
5396 2506 Acked Read 5 Sent Done Done false
5397 2507 Acked Flush 0 Sent Done Done false
5398 STATES DS:0 DS:1 DS:2 TOTAL
5399 Sent 302 0 0 302
5400 Done 0 302 302 604
5401 Skipped 0 0 0 0
5402 Error 0 0 0 0
5403 Last Flush: 2205 2507 2507
5404 Downstairs last five completed: 2205 2204 2203 2202 2201
5405 Upstairs last five completed: 2507 2506 2505 2504 2503
5406 CLIENT: Up:0 ds:302 act:2
5407 ----------------------------------------------------------------
5408 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
5409 States: Offline Active Active
5410 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
5411 2206 Acked Write 6 Sent Done Done false
5412 2207 Acked Write 2 Sent Done Done false
5413 2208 Acked Read 9 Sent Done Done false
5414 2209 Acked Read 10 Sent Done Done false
5415 2210 Acked Read 3 Sent Done Done false
5416 2211 Acked Write 1 Sent Done Done false
5417 2212 Acked Read 8 Sent Done Done false
5418 2213 Acked Flush 0 Sent Done Done false
5419 2214 Acked Write 5 Sent Done Done false
5420 2215 Acked Read 6 Sent Done Done false
5421 2216 Acked Write 5 Sent Done Done false
5422 2217 Acked Write 1 Sent Done Done false
5423 2218 Acked Read 1 Sent Done Done false
5424 2219 Acked Flush 0 Sent Done Done false
5425 2220 Acked Read 6 Sent Done Done false
5426 2221 Acked Write 2 Sent Done Done false
5427 2222 Acked Write 8 Sent Done Done false
5428 2223 Acked Flush 0 Sent Done Done false
5429 2224 Acked Read 10 Sent Done Done false
5430 2225 Acked Write 2 Sent Done Done false
5431 2226 Acked Read 1 Sent Done Done false
5432 2227 Acked Flush 0 Sent Done Done false
5433 2228 Acked Read 7 Sent Done Done false
5434 2229 Acked Write 8 Sent Done Done false
5435 2230 Acked Write 4 Sent Done Done false
5436 2231 Acked Read 7 Sent Done Done false
5437 2232 Acked Flush 0 Sent Done Done false
5438 2233 Acked Write 10 Sent Done Done false
5439 2234 Acked Read 10 Sent Done Done false
5440 2235 Acked Read 9 Sent Done Done false
5441 2236 Acked Write 5 Sent Done Done false
5442 2237 Acked Write 6 Sent Done Done false
5443 2238 Acked Write 9 Sent Done Done false
5444 2239 Acked Read 10 Sent Done Done false
5445 2240 Acked Read 10 Sent Done Done false
5446 2241 Acked Read 3 Sent Done Done false
5447 2242 Acked Read 9 Sent Done Done false
5448 2243 Acked Read 10 Sent Done Done false
5449 2244 Acked Read 6 Sent Done Done false
5450 2245 Acked Read 10 Sent Done Done false
5451 2246 Acked Read 6 Sent Done Done false
5452 2247 Acked Flush 0 Sent Done Done false
5453 2248 Acked Write 7 Sent Done Done false
5454 2249 Acked Flush 0 Sent Done Done false
5455 2250 Acked Read 6 Sent Done Done false
5456 2251 Acked Read 3 Sent Done Done false
5457 2252 Acked Write 2 Sent Done Done false
5458 2253 Acked Read 2 Sent Done Done false
5459 2254 Acked Write 8 Sent Done Done false
5460 2255 Acked Read 1 Sent Done Done false
5461 2256 Acked Write 9 Sent Done Done false
5462 2257 Acked Write 9 Sent Done Done false
5463 2258 Acked Write 5 Sent Done Done false
5464 2259 Acked Flush 0 Sent Done Done false
5465 2260 Acked Read 5 Sent Done Done false
5466 2261 Acked Write 9 Sent Done Done false
5467 2262 Acked Read 7 Sent Done Done false
5468 2263 Acked Read 1 Sent Done Done false
5469 2264 Acked Read 1 Sent Done Done false
5470 2265 Acked Write 4 Sent Done Done false
5471 2266 Acked Read 8 Sent Done Done false
5472 2267 Acked Flush 0 Sent Done Done false
5473 2268 Acked Write 8 Sent Done Done false
5474 2269 Acked Read 3 Sent Done Done false
5475 2270 Acked Flush 0 Sent Done Done false
5476 2271 Acked Read 3 Sent Done Done false
5477 2272 Acked Read 8 Sent Done Done false
5478 2273 Acked Read 1 Sent Done Done false
5479 2274 Acked Read 3 Sent Done Done false
5480 2275 Acked Write 7 Sent Done Done false
5481 2276 Acked Write 10 Sent Done Done false
5482 2277 Acked Read 9 Sent Done Done false
5483 2278 Acked Read 1 Sent Done Done false
5484 2279 Acked Read 9 Sent Done Done false
5485 2280 Acked Flush 0 Sent Done Done false
5486 2281 Acked Write 5 Sent Done Done false
5487 2282 Acked Read 2 Sent Done Done false
5488 2283 Acked Read 1 Sent Done Done false
5489 2284 Acked Read 6 Sent Done Done false
5490 2285 Acked Read 6 Sent Done Done false
5491 2286 Acked Flush 0 Sent Done Done false
5492 2287 Acked Write 9 Sent Done Done false
5493 2288 Acked Write 9 Sent Done Done false
5494 2289 Acked Read 7 Sent Done Done false
5495 2290 Acked Read 5 Sent Done Done false
5496 2291 Acked Read 10 Sent Done Done false
5497 2292 Acked Write 2 Sent Done Done false
5498 2293 Acked Read 1 Sent Done Done false
5499 2294 Acked Write 3 Sent Done Done false
5500 2295 Acked Read 7 Sent Done Done false
5501 2296 Acked Read 8 Sent Done Done false
5502 2297 Acked Read 4 Sent Done Done false
5503 2298 Acked Read 8 Sent Done Done false
5504 2299 Acked Read 4 Sent Done Done false
5505 2300 Acked Read 3 Sent Done Done false
5506 2301 Acked Read 8 Sent Done Done false
5507 2302 Acked Write 1 Sent Done Done false
5508 2303 Acked Write 6 Sent Done Done false
5509 2304 Acked Write 7 Sent Done Done false
5510 2305 Acked Read 7 Sent Done Done false
5511 2306 Acked Write 8 Sent Done Done false
5512 2307 Acked Read 10 Sent Done Done false
5513 2308 Acked Read 7 Sent Done Done false
5514 2309 Acked Read 3 Sent Done Done false
5515 2310 Acked Read 7 Sent Done Done false
5516 2311 Acked Read 8 Sent Done Done false
5517 2312 Acked Write 10 Sent Done Done false
5518 2313 Acked Read 4 Sent Done Done false
5519 2314 Acked Read 8 Sent Done Done false
5520 2315 Acked Read 1 Sent Done Done false
5521 2316 Acked Write 8 Sent Done Done false
5522 2317 Acked Read 8 Sent Done Done false
5523 2318 Acked Read 4 Sent Done Done false
5524 2319 Acked Write 2 Sent Done Done false
5525 2320 Acked Write 5 Sent Done Done false
5526 2321 Acked Write 2 Sent Done Done false
5527 2322 Acked Write 3 Sent Done Done false
5528 2323 Acked Read 8 Sent Done Done false
5529 2324 Acked Read 4 Sent Done Done false
5530 2325 Acked Read 8 Sent Done Done false
5531 2326 Acked Write 9 Sent Done Done false
5532 2327 Acked Write 9 Sent Done Done false
5533 2328 Acked Read 6 Sent Done Done false
5534 2329 Acked Read 8 Sent Done Done false
5535 2330 Acked Write 5 Sent Done Done false
5536 2331 Acked Read 3 Sent Done Done false
5537 2332 Acked Write 6 Sent Done Done false
5538 2333 Acked Read 10 Sent Done Done false
5539 2334 Acked Read 1 Sent Done Done false
5540 2335 Acked Read 3 Sent Done Done false
5541 2336 Acked Flush 0 Sent Done Done false
5542 2337 Acked Write 7 Sent Done Done false
5543 2338 Acked Write 6 Sent Done Done false
5544 2339 Acked Read 3 Sent Done Done false
5545 2340 Acked Flush 0 Sent Done Done false
5546 2341 Acked Write 2 Sent Done Done false
5547 2342 Acked Flush 0 Sent Done Done false
5548 2343 Acked Write 6 Sent Done Done false
5549 2344 Acked Read 3 Sent Done Done false
5550 2345 Acked Read 1 Sent Done Done false
5551 2346 Acked Write 6 Sent Done Done false
5552 2347 Acked Write 3 Sent Done Done false
5553 2348 Acked Write 8 Sent Done Done false
5554 2349 Acked Write 1 Sent Done Done false
5555 2350 Acked Read 9 Sent Done Done false
5556 2351 Acked Read 7 Sent Done Done false
5557 2352 Acked Flush 0 Sent Done Done false
5558 2353 Acked Read 1 Sent Done Done false
5559 2354 Acked Read 7 Sent Done Done false
5560 2355 Acked Read 2 Sent Done Done false
5561 2356 Acked Write 9 Sent Done Done false
5562 2357 Acked Read 2 Sent Done Done false
5563 2358 Acked Write 4 Sent Done Done false
5564 2359 Acked Read 2 Sent Done Done false
5565 2360 Acked Read 7 Sent Done Done false
5566 2361 Acked Read 7 Sent Done Done false
5567 2362 Acked Read 5 Sent Done Done false
5568 2363 Acked Read 9 Sent Done Done false
5569 2364 Acked Read 9 Sent Done Done false
5570 2365 Acked Read 3 Sent Done Done false
5571 2366 Acked Read 8 Sent Done Done false
5572 2367 Acked Read 8 Sent Done Done false
5573 2368 Acked Read 2 Sent Done Done false
5574 2369 Acked Read 1 Sent Done Done false
5575 2370 Acked Read 3 Sent Done Done false
5576 2371 Acked Write 3 Sent Done Done false
5577 2372 Acked Read 8 Sent Done Done false
5578 2373 Acked Write 2 Sent Done Done false
5579 2374 Acked Write 2 Sent Done Done false
5580 2375 Acked Read 2 Sent Done Done false
5581 2376 Acked Flush 0 Sent Done Done false
5582 2377 Acked Read 1 Sent Done Done false
5583 2378 Acked Read 5 Sent Done Done false
5584 2379 Acked Read 5 Sent Done Done false
5585 2380 Acked Read 9 Sent Done Done false
5586 2381 Acked Read 5 Sent Done Done false
5587 2382 Acked Read 7 Sent Done Done false
5588 2383 Acked Read 10 Sent Done Done false
5589 2384 Acked Write 8 Sent Done Done false
5590 2385 Acked Write 5 Sent Done Done false
5591 2386 Acked Write 5 Sent Done Done false
5592 2387 Acked Write 1 Sent Done Done false
5593 2388 Acked Read 1 Sent Done Done false
5594 2389 Acked Read 4 Sent Done Done false
5595 2390 Acked Write 4 Sent Done Done false
5596 2391 Acked Write 4 Sent Done Done false
5597 2392 Acked Write 9 Sent Done Done false
5598 2393 Acked Write 7 Sent Done Done false
5599 2394 Acked Read 6 Sent Done Done false
5600 2395 Acked Read 9 Sent Done Done false
5601 2396 Acked Read 3 Sent Done Done false
5602 2397 Acked Read 4 Sent Done Done false
5603 2398 Acked Read 2 Sent Done Done false
5604 2399 Acked Read 9 Sent Done Done false
5605 2400 Acked Write 5 Sent Done Done false
5606 2401 Acked Write 2 Sent Done Done false
5607 2402 Acked Write 7 Sent Done Done false
5608 2403 Acked Flush 0 Sent Done Done false
5609 2404 Acked Flush 0 Sent Done Done false
5610 2405 Acked Read 9 Sent Done Done false
5611 2406 Acked Read 8 Sent Done Done false
5612 2407 Acked Read 5 Sent Done Done false
5613 2408 Acked Read 9 Sent Done Done false
5614 2409 Acked Read 6 Sent Done Done false
5615 2410 Acked Read 6 Sent Done Done false
5616 2411 Acked Flush 0 Sent Done Done false
5617 2412 Acked Read 7 Sent Done Done false
5618 2413 Acked Read 7 Sent Done Done false
5619 2414 Acked Write 6 Sent Done Done false
5620 2415 Acked Read 8 Sent Done Done false
5621 2416 Acked Read 8 Sent Done Done false
5622 2417 Acked Read 10 Sent Done Done false
5623 2418 Acked Write 7 Sent Done Done false
5624 2419 Acked Read 7 Sent Done Done false
5625 2420 Acked Read 1 Sent Done Done false
5626 2421 Acked Read 8 Sent Done Done false
5627 2422 Acked Read 3 Sent Done Done false
5628 2423 Acked Write 5 Sent Done Done false
5629 2424 Acked Read 7 Sent Done Done false
5630 2425 Acked Write 8 Sent Done Done false
5631 2426 Acked Read 9 Sent Done Done false
5632 2427 Acked Read 5 Sent Done Done false
5633 2428 Acked Read 1 Sent Done Done false
5634 2429 Acked Read 10 Sent Done Done false
5635 2430 Acked Read 10 Sent Done Done false
5636 2431 Acked Flush 0 Sent Done Done false
5637 2432 Acked Read 2 Sent Done Done false
5638 2433 Acked Write 3 Sent Done Done false
5639 2434 Acked Read 6 Sent Done Done false
5640 2435 Acked Write 7 Sent Done Done false
5641 2436 Acked Flush 0 Sent Done Done false
5642 2437 Acked Read 6 Sent Done Done false
5643 2438 Acked Flush 0 Sent Done Done false
5644 2439 Acked Write 10 Sent Done Done false
5645 2440 Acked Write 1 Sent Done Done false
5646 2441 Acked Write 3 Sent Done Done false
5647 2442 Acked Flush 0 Sent Done Done false
5648 2443 Acked Read 4 Sent Done Done false
5649 2444 Acked Read 2 Sent Done Done false
5650 2445 Acked Write 5 Sent Done Done false
5651 2446 Acked Read 6 Sent Done Done false
5652 2447 Acked Write 4 Sent Done Done false
5653 2448 Acked Read 1 Sent Done Done false
5654 2449 Acked Read 6 Sent Done Done false
5655 2450 Acked Read 3 Sent Done Done false
5656 2451 Acked Read 2 Sent Done Done false
5657 2452 Acked Read 5 Sent Done Done false
5658 2453 Acked Read 10 Sent Done Done false
5659 2454 Acked Write 3 Sent Done Done false
5660 2455 Acked Flush 0 Sent Done Done false
5661 2456 Acked Read 7 Sent Done Done false
5662 2457 Acked Write 3 Sent Done Done false
5663 2458 Acked Write 10 Sent Done Done false
5664 2459 Acked Write 8 Sent Done Done false
5665 2460 Acked Read 8 Sent Done Done false
5666 2461 Acked Read 10 Sent Done Done false
5667 2462 Acked Write 7 Sent Done Done false
5668 2463 Acked Read 4 Sent Done Done false
5669 2464 Acked Write 10 Sent Done Done false
5670 2465 Acked Read 10 Sent Done Done false
5671 2466 Acked Read 10 Sent Done Done false
5672 2467 Acked Flush 0 Sent Done Done false
5673 2468 Acked Flush 0 Sent Done Done false
5674 2469 Acked Read 7 Sent Done Done false
5675 2470 Acked Read 6 Sent Done Done false
5676 2471 Acked Write 8 Sent Done Done false
5677 2472 Acked Read 10 Sent Done Done false
5678 2473 Acked Flush 0 Sent Done Done false
5679 2474 Acked Write 1 Sent Done Done false
5680 2475 Acked Flush 0 Sent Done Done false
5681 2476 Acked Read 2 Sent Done Done false
5682 2477 Acked Write 5 Sent Done Done false
5683 2478 Acked Read 4 Sent Done Done false
5684 2479 Acked Write 1 Sent Done Done false
5685 2480 Acked Write 8 Sent Done Done false
5686 2481 Acked Flush 0 Sent Done Done false
5687 2482 Acked Read 1 Sent Done Done false
5688 2483 Acked Write 7 Sent Done Done false
5689 2484 Acked Write 4 Sent Done Done false
5690 2485 Acked Write 9 Sent Done Done false
5691 2486 Acked Read 9 Sent Done Done false
5692 2487 Acked Read 4 Sent Done Done false
5693 2488 Acked Read 4 Sent Done Done false
5694 2489 Acked Write 10 Sent Done Done false
5695 2490 Acked Read 10 Sent Done Done false
5696 2491 Acked Write 10 Sent Done Done false
5697 2492 Acked Write 4 Sent Done Done false
5698 2493 Acked Flush 0 Sent Done Done false
5699 2494 Acked Read 1 Sent Done Done false
5700 2495 Acked Read 7 Sent Done Done false
5701 2496 Acked Read 3 Sent Done Done false
5702 2497 Acked Write 10 Sent Done Done false
5703 2498 Acked Write 3 Sent Done Done false
5704 2499 Acked Read 1 Sent Done Done false
5705 2500 Acked Write 1 Sent Done Done false
5706 2501 Acked Read 9 Sent Done Done false
5707 2502 Acked Write 6 Sent Done Done false
5708 2503 Acked Read 7 Sent Done Done false
5709 2504 Acked Write 4 Sent Done Done false
5710 2505 Acked Write 2 Sent Done Done false
5711 2506 Acked Read 5 Sent Done Done false
5712 2507 Acked Flush 0 Sent Done Done false
5713 STATES DS:0 DS:1 DS:2 TOTAL
5714 Sent 302 0 0 302
5715 Done 0 302 302 604
5716 Skipped 0 0 0 0
5717 Error 0 0 0 0
5718 Last Flush: 2205 2507 2507
5719 Downstairs last five completed: 2205 2204 2203 2202 2201
5720 Upstairs last five completed: 2507 2506 2505 2504 2503
5721 CLIENT: Up:0 ds:302 act:2
57222024-11-15T01:58:40.831ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
57232024-11-15T01:58:40.832ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:36136) = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
57242024-11-15T01:58:40.833ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
57252024-11-15T01:58:40.833ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
57262024-11-15T01:58:40.833ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
57272024-11-15T01:58:40.833ZINFOcrucible: send last flush ID to this DS: 2205 = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
57282024-11-15T01:58:40.833ZINFOcrucible: Replied this last flush ID: 2205 = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
57292024-11-15T01:58:40.833ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
57302024-11-15T01:58:40.833ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
57312024-11-15T01:58:40.833ZINFOcrucible: [0] client re-new 302 jobs since flush 2205 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
57322024-11-15T01:58:40.834ZINFOcrucible: [0] Replayed 302 jobs since flush: 2205 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
5733 ----------------------------------------------------------------
5734 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
5735 Downstairs last five completed: 2507 2506 2505 2504 2503
5736 Upstairs last five completed: 2507 2506 2505 2504 2503
5737 CLIENT: Up:0 ds:0 act:3
5738 Replay: All jobs finished
5739 Main task has work to do, go find it
5740 got dsc Stop(0)
5741 stop 0
5742 Nov 15 01:58:41.088 INFO request completed, latency_us: 345, response_code: [8810] Got stop action so:false kr:false
5743 204, uri: /stop/cid/0, method: POST, req_id: ec92fe84-3ccd-4cf0-a503-87cb058f92c1, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
57442024-11-15T01:58:41.089ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
57452024-11-15T01:58:41.089ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
57462024-11-15T01:58:41.089ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
5747 Nov 15 01:58:41.089 INFO request completed, latency_us: 128, response_code: 200, uri: /state/cid/0, method: GET, req_id: 25091858-043c-4355-b25b-33d2759385c1, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
57482024-11-15T01:58:41.089ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
57492024-11-15T01:58:41.089ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
57502024-11-15T01:58:41.089ZINFOcrucible: client task is exiting = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
5751 [8810] Exited with: ExitStatus(unix_wait_status(9))
5752 [8810][0] reports Exit
5753 Nov 15 01:58:45.092 INFO request completed, latency_us: 201, response_code: 200, uri: /state/cid/0, method: GET, req_id: 69b75b7d-9e71-4c54-81a9-fd40ae99126a, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
5754 001/300 Read block 2521 len 36864
5755 002/300 Flush
5756 003/300 Write block 130 len 8192 data: 2 2
5757 004/300 Flush
5758 005/300 Write block 1317 len 4096 data: 3
5759 006/300 Write block 3775 len 32768 data: 2 2 2 2 2 2 2 2
5760 007/300 Read block 393 len 24576
5761 008/300 Read block 3917 len 32768
5762 009/300 Read block 2738 len 12288
5763 010/300 Read block 4103 len 24576
5764 011/300 Write block 4917 len 8192 data: 2 2
5765 012/300 Read block 2136 len 12288
5766 013/300 Write block 2991 len 4096 data: 2
5767 014/300 Write block 840 len 24576 data: 3 4 4 4 4 3
5768 015/300 Write block 4585 len 12288 data: 3 3 3
5769 016/300 Read block 2520 len 20480
5770 017/300 Write block 1274 len 40960 data: 2 3 3 3 3 2 3 3 4 4
5771 018/300 Flush
5772 019/300 Read block 3997 len 24576
5773 020/300 Write block 3562 len 12288 data: 3 3 2
5774 021/300 Read block 2903 len 36864
5775 022/300 Write block 3483 len 32768 data: 3 2 2 2 2 3 3 3
5776 023/300 Read block 3222 len 20480
5777 024/300 Write block 185 len 4096 data: 2
5778 025/300 Read block 42 len 32768
5779 026/300 Read block 4194 len 8192
5780 027/300 Read block 111 len 8192
5781 028/300 Read block 182 len 28672
5782 029/300 Write block 1187 len 8192 data: 3 2
5783 030/300 Write block 1890 len 12288 data: 3 3 3
5784 031/300 Write block 1955 len 12288 data: 2 2 2
5785 032/300 Read block 1177 len 16384
5786 033/300 Read block 2199 len 40960
5787 034/300 Write block 3729 len 12288 data: 2 2 2
5788 035/300 Read block 3167 len 36864
5789 036/300 Write block 1664 len 32768 data: 4 4 3 4 4 4 4 3
5790 037/300 Read block 1573 len 32768
5791 038/300 Read block 280 len 20480
5792 039/300 Write block 888 len 20480 data: 2 2 2 2 2
5793 040/300 Write block 2498 len 16384 data: 3 3 3 2
5794 041/300 Read block 4967 len 32768
5795 042/300 Write block 4916 len 4096 data: 2
5796 043/300 Read block 3504 len 12288
5797 044/300 Read block 3203 len 40960
5798 045/300 Read block 4765 len 16384
5799 046/300 Write block 2591 len 36864 data: 3 3 2 2 2 3 3 3 3
5800 047/300 Write block 2234 len 4096 data: 2
5801 048/300 Write block 3600 len 12288 data: 2 3 5
5802 049/300 Flush
5803 050/300 Read block 903 len 8192
5804 051/300 Write block 1314 len 8192 data: 3 3
5805 052/300 Read block 1598 len 32768
5806 053/300 Read block 3511 len 20480
5807 054/300 Write block 777 len 12288 data: 2 2 2
5808 055/300 Read block 590 len 24576
5809 056/300 Read block 704 len 40960
5810 057/300 Flush
5811 058/300 Read block 1167 len 8192
5812 059/300 Write block 1782 len 8192 data: 4 4
5813 060/300 Read block 1214 len 4096
5814 061/300 Write block 303 len 40960 data: 3 3 3 3 3 2 2 2 2 2
5815 062/300 Read block 567 len 8192
5816 063/300 Write block 931 len 12288 data: 2 2 2
5817 064/300 Read block 3388 len 32768
5818 065/300 Read block 825 len 16384
5819 066/300 Write block 107 len 8192 data: 2 2
5820 067/300 Write block 229 len 20480 data: 5 5 5 5 4
5821 068/300 Write block 3649 len 28672 data: 3 3 3 2 2 2 2
5822 069/300 Read block 241 len 8192
5823 070/300 Flush
5824 071/300 Read block 2380 len 40960
5825 072/300 Read block 1072 len 20480
5826 073/300 Read block 3925 len 8192
5827 074/300 Read block 3855 len 16384
5828 075/300 Read block 1154 len 16384
5829 076/300 Read block 2887 len 32768
5830 077/300 Read block 3417 len 16384
5831 078/300 Write block 1087 len 4096 data: 2
5832 079/300 Write block 4890 len 16384 data: 3 2 2 4
5833 080/300 Write block 530 len 20480 data: 4 4 3 3 3
5834 081/300 Write block 783 len 32768 data: 4 4 4 4 4 4 3 3
5835 082/300 Read block 1316 len 12288
5836 083/300 Read block 2630 len 28672
5837 084/300 Write block 1546 len 32768 data: 2 2 2 2 3 3 3 3
5838 085/300 Write block 3836 len 24576 data: 3 3 3 3 3 3
5839 086/300 Read block 1087 len 36864
5840 087/300 Write block 4324 len 40960 data: 2 2 2 3 3 3 3 3 3 2
5841 088/300 Write block 3952 len 32768 data: 2 2 3 3 3 3 3 3
5842 089/300 Flush
5843 090/300 Read block 1355 len 8192
5844 091/300 Write block 2631 len 4096 data: 3
5845 092/300 Flush
5846 093/300 Flush
5847 094/300 Flush
5848 095/300 Read block 1409 len 24576
5849 096/300 Write block 4202 len 32768 data: 3 3 3 4 5 4 4 3
5850 097/300 Read block 3258 len 28672
5851 098/300 Flush
5852 099/300 Flush
5853 100/300 Read block 3646 len 8192
5854 101/300 Write block 1535 len 40960 data: 2 2 2 2 4 4 4 3 3 2
5855 102/300 Write block 3936 len 16384 data: 3 3 3 3
5856 103/300 Write block 825 len 20480 data: 3 3 3 3 3
5857 104/300 Write block 1814 len 12288 data: 3 3 2
5858 105/300 Read block 2266 len 36864
5859 106/300 Read block 3295 len 20480
5860 107/300 Read block 4565 len 40960
5861 108/300 Flush
5862 109/300 Read block 4119 len 8192
5863 110/300 Write block 1171 len 32768 data: 3 3 3 3 3 3 3 2
5864 111/300 Read block 4526 len 28672
5865 112/300 Read block 1797 len 8192
5866 113/300 Write block 445 len 40960 data: 2 2 2 2 2 2 2 2 2 2
5867 114/300 Read block 4989 len 4096
5868 115/300 Flush
5869 116/300 Write block 4513 len 8192 data: 4 5
5870 117/300 Write block 83 len 32768 data: 2 2 2 2 2 3 3 2
5871 118/300 Read block 2013 len 8192
5872 119/300 Write block 2737 len 40960 data: 2 2 3 3 2 2 2 2 2 2
5873 120/300 Read block 737 len 12288
5874 121/300 Write block 1095 len 12288 data: 2 2 2
5875 122/300 Write block 2319 len 20480 data: 3 3 3 3 2
5876 123/300 Read block 279 len 24576
5877 124/300 Read block 3413 len 36864
5878 125/300 Write block 327 len 4096 data: 2
5879 126/300 Write block 511 len 12288 data: 2 2 3
5880 127/300 Read block 2871 len 40960
5881 128/300 Write block 2451 len 28672 data: 4 3 4 4 5 4 4
5882 129/300 Read block 1952 len 4096
5883 130/300 Read block 1601 len 20480
5884 131/300 Write block 4589 len 36864 data: 4 4 4 3 2 2 2 2 2
5885 132/300 Read block 316 len 4096
5886 133/300 Write block 4658 len 32768 data: 2 2 3 3 3 2 2 2
5887 134/300 Write block 534 len 40960 data: 4 3 2 2 2 2 2 2 2 2
5888 135/300 Write block 3907 len 12288 data: 2 2 2
5889 136/300 Write block 4229 len 36864 data: 3 2 2 2 2 2 2 2 2
5890 137/300 Read block 4557 len 16384
5891 138/300 Read block 1612 len 32768
5892 139/300 Read block 243 len 36864
5893 140/300 Read block 3591 len 12288
5894 141/300 Write block 2896 len 12288 data: 2 2 2
5895 142/300 Read block 2390 len 16384
5896 143/300 Write block 4195 len 24576 data: 3 3 3 3 2 2
5897 144/300 Read block 1384 len 4096
5898 145/300 Write block 3688 len 36864 data: 2 2 2 2 2 2 2 2 2
5899 146/300 Write block 4173 len 24576 data: 2 2 2 2 2 2
5900 147/300 Read block 3517 len 8192
5901 148/300 Read block 485 len 4096
5902 149/300 Write block 3568 len 32768 data: 2 2 2 2 2 2 2 2
5903 150/300 Read block 4229 len 4096
5904 151/300 Read block 543 len 8192
5905 152/300 Write block 4435 len 8192 data: 3 3
5906 153/300 Write block 65 len 32768 data: 3 4 4 4 4 3 3 3
5907 154/300 Read block 4272 len 28672
5908 155/300 Read block 1355 len 32768
5909 156/300 Write block 1928 len 28672 data: 4 4 3 3 4 4 5
5910 157/300 Write block 3521 len 20480 data: 4 4 4 4 4
5911 158/300 Read block 765 len 4096
5912 159/300 Write block 413 len 28672 data: 3 3 2 3 3 3 3
5913 160/300 Read block 2661 len 8192
5914 161/300 Read block 110 len 28672
5915 162/300 Write block 829 len 12288 data: 4 2 2
5916 163/300 Write block 896 len 36864 data: 3 2 2 2 2 2 3 3 3
5917 164/300 Flush
5918 165/300 Write block 1727 len 40960 data: 2 2 2 2 2 3 3 3 3 3
5919 166/300 Write block 1143 len 32768 data: 2 2 2 3 3 3 3 3
5920 167/300 Flush
5921 168/300 Write block 2288 len 4096 data: 3
5922 169/300 Write block 659 len 20480 data: 2 2 2 2 3
5923 170/300 Write block 29 len 40960 data: 2 2 2 2 2 2 2 2 2 3
5924 171/300 Write block 3682 len 40960 data: 3 3 2 2 2 3 3 3 3 3
5925 172/300 Read block 416 len 20480
5926 173/300 Write block 1654 len 12288 data: 2 2 2
5927 174/300 Flush
5928 175/300 Flush
5929 176/300 Write block 1881 len 4096 data: 3
5930 177/300 Flush
5931 178/300 Read block 2892 len 36864
5932 179/300 Read block 2611 len 4096
5933 180/300 Read block 2120 len 32768
5934 181/300 Write block 3646 len 40960 data: 3 3 3 4 4 4 3 3 3 3
5935 182/300 Write block 4039 len 36864 data: 4 4 4 3 3 3 3 4 3
5936 183/300 Write block 248 len 24576 data: 2 2 2 2 2 2
5937 184/300 Read block 2586 len 12288
5938 185/300 Read block 1237 len 4096
5939 186/300 Read block 7 len 12288
5940 187/300 Read block 3250 len 36864
5941 188/300 Write block 841 len 32768 data: 5 5 5 5 4 3 2 2
5942 189/300 Read block 4092 len 28672
5943 190/300 Read block 2186 len 24576
5944 191/300 Read block 4677 len 16384
5945 192/300 Write block 4791 len 36864 data: 2 2 2 2 2 2 2 2 2
5946 193/300 Write block 703 len 40960 data: 5 5 5 3 3 3 3 3 3 3
5947 194/300 Write block 1841 len 36864 data: 3 3 3 3 4 3 3 3 3
5948 195/300 Write block 4490 len 40960 data: 2 2 2 3 3 3 3 3 3 3
5949 196/300 Read block 1960 len 24576
5950 197/300 Read block 2463 len 36864
5951 198/300 Read block 4399 len 8192
5952 199/300 Flush
5953 200/300 Read block 601 len 32768
5954 201/300 Read block 3706 len 24576
5955 202/300 Read block 1309 len 36864
5956 203/300 Read block 4430 len 16384
5957 204/300 Read block 2410 len 4096
5958 205/300 Read block 412 len 8192
5959 206/300 Write block 1766 len 16384 data: 3 3 3 3
5960 207/300 Write block 4101 len 36864 data: 2 2 2 2 2 3 3 4 4
5961 208/300 Read block 3664 len 36864
5962 209/300 Write block 172 len 8192 data: 4 4
5963 210/300 Read block 2756 len 12288
5964 211/300 Write block 1096 len 24576 data: 3 3 3 3 3 3
5965 212/300 Read block 2993 len 36864
5966 213/300 Read block 1253 len 40960
5967 214/300 Write block 2753 len 36864 data: 2 2 2 2 3 3 3 3 3
5968 215/300 Write block 281 len 40960 data: 2 2 2 3 3 3 3 3 3 3
5969 216/300 Read block 3702 len 32768
5970 217/300 Write block 4276 len 28672 data: 2 2 3 2 2 2 3
5971 218/300 Read block 625 len 20480
5972 219/300 Read block 1687 len 12288
5973 220/300 Read block 383 len 24576
5974 221/300 Write block 4528 len 12288 data: 5 5 5
5975 222/300 Read block 3869 len 16384
5976 223/300 Read block 4685 len 8192
5977 224/300 Write block 4669 len 12288 data: 2 2 2
5978 225/300 Read block 4420 len 24576
5979 226/300 Read block 598 len 4096
5980 227/300 Write block 3090 len 20480 data: 2 2 2 2 2
5981 228/300 Read block 1928 len 24576
5982 229/300 Write block 1789 len 32768 data: 3 3 2 2 2 2 2 2
5983 230/300 Write block 3591 len 40960 data: 4 3 3 3 3 3 2 2 2 3
5984 231/300 Read block 308 len 16384
5985 232/300 Flush
5986 233/300 Flush
5987 234/300 Write block 1914 len 40960 data: 2 2 2 2 2 2 2 2 2 3
5988 235/300 Write block 1776 len 4096 data: 2
5989 236/300 Read block 95 len 28672
5990 237/300 Write block 4473 len 4096 data: 3
5991 238/300 Write block 4209 len 4096 data: 4
5992 239/300 Write block 1314 len 8192 data: 4 4
5993 240/300 Flush
5994 241/300 Write block 2642 len 8192 data: 2 2
5995 242/300 Read block 3063 len 8192
5996 243/300 Read block 504 len 40960
5997 244/300 Write block 1663 len 16384 data: 4 5 5 4
5998 245/300 Write block 1378 len 20480 data: 3 3 3 4 3
5999 246/300 Read block 2816 len 8192
6000 247/300 Write block 2673 len 40960 data: 2 2 2 2 2 2 2 2 2 2
6001 248/300 Read block 57 len 28672
6002 249/300 Write block 2530 len 12288 data: 3 4 4
6003 250/300 Write block 1903 len 24576 data: 4 4 4 2 2 2
6004 251/300 Read block 3094 len 32768
6005 252/300 Write block 3218 len 20480 data: 3 3 3 3 3
6006 253/300 Write block 2790 len 12288 data: 2 2 2
6007 254/300 Write block 4308 len 8192 data: 4 3
6008 255/300 Read block 1335 len 40960
6009 256/300 Write block 137 len 20480 data: 2 2 2 2 2
6010 257/300 Read block 1789 len 8192
6011 258/300 Read block 3168 len 36864
6012 259/300 Read block 3337 len 12288
6013 260/300 Read block 3943 len 28672
6014 261/300 Read block 4102 len 32768
6015 262/300 Write block 4921 len 16384 data: 3 3 5 5
6016 263/300 Read block 360 len 24576
6017 264/300 Flush
6018 265/300 Write block 1661 len 24576 data: 5 4 5 6 6 5
6019 266/300 Read block 3211 len 36864
6020 267/300 Read block 3436 len 36864
6021 268/300 Flush
6022 269/300 Flush
6023 270/300 Read block 4335 len 20480
6024 271/300 Write block 752 len 20480 data: 2 4 4 5 5
6025 272/300 Read block 811 len 40960
6026 273/300 Write block 4684 len 4096 data: 3
6027 274/300 Read block 1025 len 4096
6028 275/300 Read block 3499 len 24576
6029 276/300 Read block 3698 len 4096
6030 277/300 Flush
6031 278/300 Write block 851 len 32768 data: 2 2 2 2 2 2 2 2
6032 279/300 Read block 3400 len 28672
6033 280/300 Write block 905 len 28672 data: 3 3 3 3 4 2 2
6034 281/300 Read block 4566 len 12288
6035 282/300 Write block 2336 len 24576 data: 2 2 2 2 3 3
6036 283/300 Write block 2231 len 4096 data: 4
6037 284/300 Read block 2193 len 28672
6038 285/300 Read block 3353 len 20480
6039 286/300 Write block 836 len 16384 data: 3 2 2 3
6040 287/300 Write block 1314 len 4096 data: 5
6041 288/300 Write block 3150 len 12288 data: 3 3 3
6042 289/300 Read block 4217 len 32768
6043 290/300 Read block 2130 len 8192
6044 291/300 Read block 2468 len 20480
6045 292/300 Read block 1733 len 28672
6046 293/300 Write block 3185 len 32768 data: 2 2 2 3 4 3 3 3
6047 294/300 Read block 1088 len 36864
6048 295/300 Read block 145 len 20480
6049 296/300 Flush
6050 297/300 Write block 4564 len 28672 data: 2 2 2 2 2 2 2
6051 298/300 Read block 1645 len 36864
6052 299/300 Write block 3881 len 32768 data: 2 2 2 2 2 2 3 3
6053 300/300 Read block 111 len 24576
6054 301/300 Write block 1011 len 36864 data: 3 3 4 4 5 5 5 5 4
6055 Main task has work to do, go find it
6056 got dsc Start(0)
6057 start 0
6058 Nov 15 01:58:45.324 INFO request completed, latency_us: 217, response_code: 204, uri: /start/cid/0, method: POST, req_id: 962fdf87-cadf-47ce-bb1f-a0bd46014256, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
6059 Replay: started 0, returned:Ok(())
6060 ----------------------------------------------------------------
6061 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
6062 States: Offline Active Active
6063 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
6064 2508 Acked Read 9 Sent Done Done false
6065 2509 Acked Flush 0 Sent Done Done false
6066 2510 Acked Write 2 Sent Done Done false
6067 2511 Acked Flush 0 Sent Done Done false
6068 2512 Acked Write 1 Sent Done Done false
6069 2513 Acked Write 8 Sent Done Done false
6070 2514 Acked Read 6 Sent Done Done false
6071 2515 Acked Read 8 Sent Done Done false
6072 2516 Acked Read 3 Sent Done Done false
6073 2517 Acked Read 6 Sent Done Done false
6074 2518 Acked Write 2 Sent Done Done false
6075 2519 Acked Read 3 Sent Done Done false
6076 2520 Acked Write 1 Sent Done Done false
6077 2521 Acked Write 6 Sent Done Done false
6078 2522 Acked Write 3 Sent Done Done false
6079 2523 Acked Read 5 Sent Done Done false
6080 2524 Acked Write 10 Sent Done Done false
6081 2525 Acked Flush 0 Sent Done Done false
6082 2526 Acked Read 6 Sent Done Done false
6083 2527 Acked Write 3 Sent Done Done false
6084 2528 Acked Read 9 Sent Done Done false
6085 2529 Acked Write 8 Sent Done Done false
6086 2530 Acked Read 5 Sent Done Done false
6087 2531 Acked Write 1 Sent Done Done false
6088 2532 Acked Read 8 Sent Done Done false
6089 2533 Acked Read 2 Sent Done Done false
6090 2534 Acked Read 2 Sent Done Done false
6091 2535 Acked Read 7 Sent Done Done false
6092 2536 Acked Write 2 Sent Done Done false
6093 2537 Acked Write 3 Sent Done Done false
6094 2538 Acked Write 3 Sent Done Done false
6095 2539 Acked Read 4 Sent Done Done false
6096 2540 Acked Read 10 Sent Done Done false
6097 2541 Acked Write 3 Sent Done Done false
6098 2542 Acked Read 9 Sent Done Done false
6099 2543 Acked Write 8 Sent Done Done false
6100 2544 Acked Read 8 Sent Done Done false
6101 2545 Acked Read 5 Sent Done Done false
6102 2546 Acked Write 5 Sent Done Done false
6103 2547 Acked Write 4 Sent Done Done false
6104 2548 Acked Read 8 Sent Done Done false
6105 2549 Acked Write 1 Sent Done Done false
6106 2550 Acked Read 3 Sent Done Done false
6107 2551 Acked Read 10 Sent Done Done false
6108 2552 Acked Read 4 Sent Done Done false
6109 2553 Acked Write 9 Sent Done Done false
6110 2554 Acked Write 1 Sent Done Done false
6111 2555 Acked Write 3 Sent Done Done false
6112 2556 Acked Flush 0 Sent Done Done false
6113 2557 Acked Read 2 Sent Done Done false
6114 2558 Acked Write 2 Sent Done Done false
6115 2559 Acked Read 8 Sent Done Done false
6116 2560 Acked Read 5 Sent Done Done false
6117 2561 Acked Write 3 Sent Done Done false
6118 2562 Acked Read 6 Sent Done Done false
6119 2563 Acked Read 10 Sent Done Done false
6120 2564 Acked Flush 0 Sent Done Done false
6121 2565 Acked Read 2 Sent Done Done false
6122 2566 Acked Write 2 Sent Done Done false
6123 2567 Acked Read 1 Sent Done Done false
6124 2568 Acked Write 10 Sent Done Done false
6125 2569 Acked Read 2 Sent Done Done false
6126 2570 Acked Write 3 Sent Done Done false
6127 2571 Acked Read 8 Sent Done Done false
6128 2572 Acked Read 4 Sent Done Done false
6129 2573 Acked Write 2 Sent Done Done false
6130 2574 Acked Write 5 Sent Done Done false
6131 2575 Acked Write 7 Sent Done Done false
6132 2576 Acked Read 2 Sent Done Done false
6133 2577 Acked Flush 0 Sent Done Done false
6134 2578 Acked Read 10 Sent Done Done false
6135 2579 Acked Read 5 Sent Done Done false
6136 2580 Acked Read 2 Sent Done Done false
6137 2581 Acked Read 4 Sent Done Done false
6138 2582 Acked Read 4 Sent Done Done false
6139 2583 Acked Read 8 Sent Done Done false
6140 2584 Acked Read 4 Sent Done Done false
6141 2585 Acked Write 1 Sent Done Done false
6142 2586 Acked Write 4 Sent Done Done false
6143 2587 Acked Write 5 Sent Done Done false
6144 2588 Acked Write 8 Sent Done Done false
6145 2589 Acked Read 3 Sent Done Done false
6146 2590 Acked Read 7 Sent Done Done false
6147 2591 Acked Write 8 Sent Done Done false
6148 2592 Acked Write 6 Sent Done Done false
6149 2593 Acked Read 9 Sent Done Done false
6150 2594 Acked Write 10 Sent Done Done false
6151 2595 Acked Write 8 Sent Done Done false
6152 2596 Acked Flush 0 Sent Done Done false
6153 2597 Acked Read 2 Sent Done Done false
6154 2598 Acked Write 1 Sent Done Done false
6155 2599 Acked Flush 0 Sent Done Done false
6156 2600 Acked Flush 0 Sent Done Done false
6157 2601 Acked Flush 0 Sent Done Done false
6158 2602 Acked Read 6 Sent Done Done false
6159 2603 Acked Write 8 Sent Done Done false
6160 2604 Acked Read 7 Sent Done Done false
6161 2605 Acked Flush 0 Sent Done Done false
6162 2606 Acked Flush 0 Sent Done Done false
6163 2607 Acked Read 2 Sent Done Done false
6164 2608 Acked Write 10 Sent Done Done false
6165 2609 Acked Write 4 Sent Done Done false
6166 2610 Acked Write 5 Sent Done Done false
6167 2611 Acked Write 3 Sent Done Done false
6168 2612 Acked Read 9 Sent Done Done false
6169 2613 Acked Read 5 Sent Done Done false
6170 2614 Acked Read 10 Sent Done Done false
6171 2615 Acked Flush 0 Sent Done Done false
6172 2616 Acked Read 2 Sent Done Done false
6173 2617 Acked Write 8 Sent Done Done false
6174 2618 Acked Read 7 Sent Done Done false
6175 2619 Acked Read 2 Sent Done Done false
6176 2620 Acked Write 10 Sent Done Done false
6177 2621 Acked Read 1 Sent Done Done false
6178 2622 Acked Flush 0 Sent Done Done false
6179 2623 Acked Write 2 Sent Done Done false
6180 2624 Acked Write 8 Sent Done Done false
6181 2625 Acked Read 2 Sent Done Done false
6182 2626 Acked Write 10 Sent Done Done false
6183 2627 Acked Read 3 Sent Done Done false
6184 2628 Acked Write 3 Sent Done Done false
6185 2629 Acked Write 5 Sent Done Done false
6186 2630 Acked Read 6 Sent Done Done false
6187 2631 Acked Read 9 Sent Done Done false
6188 2632 Acked Write 1 Sent Done Done false
6189 2633 Acked Write 3 Sent Done Done false
6190 2634 Acked Read 10 Sent Done Done false
6191 2635 Acked Write 7 Sent Done Done false
6192 2636 Acked Read 1 Sent Done Done false
6193 2637 Acked Read 5 Sent Done Done false
6194 2638 Acked Write 9 Sent Done Done false
6195 2639 Acked Read 1 Sent Done Done false
6196 2640 Acked Write 8 Sent Done Done false
6197 2641 Acked Write 10 Sent Done Done false
6198 2642 Acked Write 3 Sent Done Done false
6199 2643 Acked Write 9 Sent Done Done false
6200 2644 Acked Read 4 Sent Done Done false
6201 2645 Acked Read 8 Sent Done Done false
6202 2646 Acked Read 9 Sent Done Done false
6203 2647 Acked Read 3 Sent Done Done false
6204 2648 Acked Write 3 Sent Done Done false
6205 2649 Acked Read 4 Sent Done Done false
6206 2650 Acked Write 6 Sent Done Done false
6207 2651 Acked Read 1 Sent Done Done false
6208 2652 Acked Write 9 Sent Done Done false
6209 2653 Acked Write 6 Sent Done Done false
6210 2654 Acked Read 2 Sent Done Done false
6211 2655 Acked Read 1 Sent Done Done false
6212 2656 Acked Write 8 Sent Done Done false
6213 2657 Acked Read 1 Sent Done Done false
6214 2658 Acked Read 2 Sent Done Done false
6215 2659 Acked Write 2 Sent Done Done false
6216 2660 Acked Write 8 Sent Done Done false
6217 2661 Acked Read 7 Sent Done Done false
6218 2662 Acked Read 8 Sent Done Done false
6219 2663 Acked Write 7 Sent Done Done false
6220 2664 Acked Write 5 Sent Done Done false
6221 2665 Acked Read 1 Sent Done Done false
6222 2666 Acked Write 7 Sent Done Done false
6223 2667 Acked Read 2 Sent Done Done false
6224 2668 Acked Read 7 Sent Done Done false
6225 2669 Acked Write 3 Sent Done Done false
6226 2670 Acked Write 9 Sent Done Done false
6227 2671 Acked Flush 0 Sent Done Done false
6228 2672 Acked Write 10 Sent Done Done false
6229 2673 Acked Write 8 Sent Done Done false
6230 2674 Acked Flush 0 Sent Done Done false
6231 2675 Acked Write 1 Sent Done Done false
6232 2676 Acked Write 5 Sent Done Done false
6233 2677 Acked Write 10 Sent Done Done false
6234 2678 Acked Write 10 Sent Done Done false
6235 2679 Acked Read 5 Sent Done Done false
6236 2680 Acked Write 3 Sent Done Done false
6237 2681 Acked Flush 0 Sent Done Done false
6238 2682 Acked Flush 0 Sent Done Done false
6239 2683 Acked Write 1 Sent Done Done false
6240 2684 Acked Flush 0 Sent Done Done false
6241 2685 Acked Read 9 Sent Done Done false
6242 2686 Acked Read 1 Sent Done Done false
6243 2687 Acked Read 8 Sent Done Done false
6244 2688 Acked Write 10 Sent Done Done false
6245 2689 Acked Write 9 Sent Done Done false
6246 2690 Acked Write 6 Sent Done Done false
6247 2691 Acked Read 3 Sent Done Done false
6248 2692 Acked Read 1 Sent Done Done false
6249 2693 Acked Read 3 Sent Done Done false
6250 2694 Acked Read 9 Sent Done Done false
6251 2695 Acked Write 8 Sent Done Done false
6252 2696 Acked Read 7 Sent Done Done false
6253 2697 Acked Read 6 Sent Done Done false
6254 2698 Acked Read 4 Sent Done Done false
6255 2699 Acked Write 9 Sent Done Done false
6256 2700 Acked Write 10 Sent Done Done false
6257 2701 Acked Write 9 Sent Done Done false
6258 2702 Acked Write 10 Sent Done Done false
6259 2703 Acked Read 6 Sent Done Done false
6260 2704 Acked Read 9 Sent Done Done false
6261 2705 Acked Read 2 Sent Done Done false
6262 2706 Acked Flush 0 Sent Done Done false
6263 2707 Acked Read 8 Sent Done Done false
6264 2708 Acked Read 6 Sent Done Done false
6265 2709 Acked Read 9 Sent Done Done false
6266 2710 Acked Read 4 Sent Done Done false
6267 2711 Acked Read 1 Sent Done Done false
6268 2712 Acked Read 2 Sent Done Done false
6269 2713 Acked Write 4 Sent Done Done false
6270 2714 Acked Write 9 Sent Done Done false
6271 2715 Acked Read 9 Sent Done Done false
6272 2716 Acked Write 2 Sent Done Done false
6273 2717 Acked Read 3 Sent Done Done false
6274 2718 Acked Write 6 Sent Done Done false
6275 2719 Acked Read 9 Sent Done Done false
6276 2720 Acked Read 10 Sent Done Done false
6277 2721 Acked Write 9 Sent Done Done false
6278 2722 Acked Write 10 Sent Done Done false
6279 2723 Acked Read 8 Sent Done Done false
6280 2724 Acked Write 7 Sent Done Done false
6281 2725 Acked Read 5 Sent Done Done false
6282 2726 Acked Read 3 Sent Done Done false
6283 2727 Acked Read 6 Sent Done Done false
6284 2728 Acked Write 3 Sent Done Done false
6285 2729 Acked Read 4 Sent Done Done false
6286 2730 Acked Read 2 Sent Done Done false
6287 2731 Acked Write 3 Sent Done Done false
6288 2732 Acked Read 6 Sent Done Done false
6289 2733 Acked Read 1 Sent Done Done false
6290 2734 Acked Write 5 Sent Done Done false
6291 2735 Acked Read 6 Sent Done Done false
6292 2736 Acked Write 8 Sent Done Done false
6293 2737 Acked Write 10 Sent Done Done false
6294 2738 Acked Read 4 Sent Done Done false
6295 2739 Acked Flush 0 Sent Done Done false
6296 2740 Acked Flush 0 Sent Done Done false
6297 2741 Acked Write 10 Sent Done Done false
6298 2742 Acked Write 1 Sent Done Done false
6299 2743 Acked Read 7 Sent Done Done false
6300 2744 Acked Write 1 Sent Done Done false
6301 2745 Acked Write 1 Sent Done Done false
6302 2746 Acked Write 2 Sent Done Done false
6303 2747 Acked Flush 0 Sent Done Done false
6304 2748 Acked Write 2 Sent Done Done false
6305 2749 Acked Read 2 Sent Done Done false
6306 2750 Acked Read 10 Sent Done Done false
6307 2751 Acked Write 4 Sent Done Done false
6308 2752 Acked Write 5 Sent Done Done false
6309 2753 Acked Read 2 Sent Done Done false
6310 2754 Acked Write 10 Sent Done Done false
6311 2755 Acked Read 7 Sent Done Done false
6312 2756 Acked Write 3 Sent Done Done false
6313 2757 Acked Write 6 Sent Done Done false
6314 2758 Acked Read 8 Sent Done Done false
6315 2759 Acked Write 5 Sent Done Done false
6316 2760 Acked Write 3 Sent Done Done false
6317 2761 Acked Write 2 Sent Done Done false
6318 2762 Acked Read 10 Sent Done Done false
6319 2763 Acked Write 5 Sent Done Done false
6320 2764 Acked Read 2 Sent Done Done false
6321 2765 Acked Read 9 Sent Done Done false
6322 2766 Acked Read 3 Sent Done Done false
6323 2767 Acked Read 7 Sent Done Done false
6324 2768 Acked Read 8 Sent Done Done false
6325 2769 Acked Write 4 Sent Done Done false
6326 2770 Acked Read 6 Sent Done Done false
6327 2771 Acked Flush 0 Sent Done Done false
6328 2772 Acked Write 6 Sent Done Done false
6329 2773 Acked Read 9 Sent Done Done false
6330 2774 Acked Read 9 Sent Done Done false
6331 2775 Acked Flush 0 Sent Done Done false
6332 2776 Acked Flush 0 Sent Done Done false
6333 2777 Acked Read 5 Sent Done Done false
6334 2778 Acked Write 5 Sent Done Done false
6335 2779 Acked Read 10 Sent Done Done false
6336 2780 Acked Write 1 Sent Done Done false
6337 2781 Acked Read 1 Sent Done Done false
6338 2782 Acked Read 6 Sent Done Done false
6339 2783 Acked Read 1 Sent Done Done false
6340 2784 Acked Flush 0 Sent Done Done false
6341 2785 Acked Write 8 Sent Done Done false
6342 2786 Acked Read 7 Sent Done Done false
6343 2787 Acked Write 7 Sent Done Done false
6344 2788 Acked Read 3 Sent Done Done false
6345 2789 Acked Write 6 Sent Done Done false
6346 2790 Acked Write 1 Sent Done Done false
6347 2791 Acked Read 7 Sent Done Done false
6348 2792 Acked Read 5 Sent Done Done false
6349 2793 Acked Write 4 Sent Done Done false
6350 2794 Acked Write 1 Sent Done Done false
6351 2795 Acked Write 3 Sent Done Done false
6352 2796 Acked Read 8 Sent Done Done false
6353 2797 Acked Read 2 Sent Done Done false
6354 2798 Acked Read 5 Sent Done Done false
6355 2799 Acked Read 7 Sent Done Done false
6356 2800 Acked Write 8 Sent Done Done false
6357 2801 Acked Read 9 Sent Done Done false
6358 2802 Acked Read 5 Sent Done Done false
6359 2803 Acked Flush 0 Sent Done Done false
6360 2804 Acked Write 7 Sent Done Done false
6361 2805 Acked Read 9 Sent Done Done false
6362 2806 Acked Write 8 Sent Done Done false
6363 2807 Acked Read 6 Sent Done Done false
6364 2808 Acked Write 9 Sent Done Done false
6365 STATES DS:0 DS:1 DS:2 TOTAL
6366 Sent 301 0 0 301
6367 Done 0 301 301 602
6368 Skipped 0 0 0 0
6369 Error 0 0 0 0
6370 Last Flush: 2507 2803 2803
6371 Downstairs last five completed: 2507 2506 2505 2504 2503
6372 Upstairs last five completed: 2808 2807 2806 2805 2804
6373 CLIENT: Up:0 ds:301 act:2
6374 ----------------------------------------------------------------
6375 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
6376 States: Offline Active Active
6377 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
6378 2508 Acked Read 9 Sent Done Done false
6379 2509 Acked Flush 0 Sent Done Done false
6380 2510 Acked Write 2 Sent Done Done false
6381 2511 Acked Flush 0 Sent Done Done false
6382 2512 Acked Write 1 Sent Done Done false
6383 2513 Acked Write 8 Sent Done Done false
6384 2514 Acked Read 6 Sent Done Done false
6385 2515 Acked Read 8 Sent Done Done false
6386 2516 Acked Read 3 Sent Done Done false
6387 2517 Acked Read 6 Sent Done Done false
6388 2518 Acked Write 2 Sent Done Done false
6389 2519 Acked Read 3 Sent Done Done false
6390 2520 Acked Write 1 Sent Done Done false
6391 2521 Acked Write 6 Sent Done Done false
6392 2522 Acked Write 3 Sent Done Done false
6393 2523 Acked Read 5 Sent Done Done false
6394 2524 Acked Write 10 Sent Done Done false
6395 2525 Acked Flush 0 Sent Done Done false
6396 2526 Acked Read 6 Sent Done Done false
6397 2527 Acked Write 3 Sent Done Done false
6398 2528 Acked Read 9 Sent Done Done false
6399 2529 Acked Write 8 Sent Done Done false
6400 2530 Acked Read 5 Sent Done Done false
6401 2531 Acked Write 1 Sent Done Done false
6402 2532 Acked Read 8 Sent Done Done false
6403 2533 Acked Read 2 Sent Done Done false
6404 2534 Acked Read 2 Sent Done Done false
6405 2535 Acked Read 7 Sent Done Done false
6406 2536 Acked Write 2 Sent Done Done false
6407 2537 Acked Write 3 Sent Done Done false
6408 2538 Acked Write 3 Sent Done Done false
6409 2539 Acked Read 4 Sent Done Done false
6410 2540 Acked Read 10 Sent Done Done false
6411 2541 Acked Write 3 Sent Done Done false
6412 2542 Acked Read 9 Sent Done Done false
6413 2543 Acked Write 8 Sent Done Done false
6414 2544 Acked Read 8 Sent Done Done false
6415 2545 Acked Read 5 Sent Done Done false
6416 2546 Acked Write 5 Sent Done Done false
6417 2547 Acked Write 4 Sent Done Done false
6418 2548 Acked Read 8 Sent Done Done false
6419 2549 Acked Write 1 Sent Done Done false
6420 2550 Acked Read 3 Sent Done Done false
6421 2551 Acked Read 10 Sent Done Done false
6422 2552 Acked Read 4 Sent Done Done false
6423 2553 Acked Write 9 Sent Done Done false
6424 2554 Acked Write 1 Sent Done Done false
6425 2555 Acked Write 3 Sent Done Done false
6426 2556 Acked Flush 0 Sent Done Done false
6427 2557 Acked Read 2 Sent Done Done false
6428 2558 Acked Write 2 Sent Done Done false
6429 2559 Acked Read 8 Sent Done Done false
6430 2560 Acked Read 5 Sent Done Done false
6431 2561 Acked Write 3 Sent Done Done false
6432 2562 Acked Read 6 Sent Done Done false
6433 2563 Acked Read 10 Sent Done Done false
6434 2564 Acked Flush 0 Sent Done Done false
6435 2565 Acked Read 2 Sent Done Done false
6436 2566 Acked Write 2 Sent Done Done false
6437 2567 Acked Read 1 Sent Done Done false
6438 2568 Acked Write 10 Sent Done Done false
6439 2569 Acked Read 2 Sent Done Done false
6440 2570 Acked Write 3 Sent Done Done false
6441 2571 Acked Read 8 Sent Done Done false
6442 2572 Acked Read 4 Sent Done Done false
6443 2573 Acked Write 2 Sent Done Done false
6444 2574 Acked Write 5 Sent Done Done false
6445 2575 Acked Write 7 Sent Done Done false
6446 2576 Acked Read 2 Sent Done Done false
6447 2577 Acked Flush 0 Sent Done Done false
6448 2578 Acked Read 10 Sent Done Done false
6449 2579 Acked Read 5 Sent Done Done false
6450 2580 Acked Read 2 Sent Done Done false
6451 2581 Acked Read 4 Sent Done Done false
6452 2582 Acked Read 4 Sent Done Done false
6453 2583 Acked Read 8 Sent Done Done false
6454 2584 Acked Read 4 Sent Done Done false
6455 2585 Acked Write 1 Sent Done Done false
6456 2586 Acked Write 4 Sent Done Done false
6457 2587 Acked Write 5 Sent Done Done false
6458 2588 Acked Write 8 Sent Done Done false
6459 2589 Acked Read 3 Sent Done Done false
6460 2590 Acked Read 7 Sent Done Done false
6461 2591 Acked Write 8 Sent Done Done false
6462 2592 Acked Write 6 Sent Done Done false
6463 2593 Acked Read 9 Sent Done Done false
6464 2594 Acked Write 10 Sent Done Done false
6465 2595 Acked Write 8 Sent Done Done false
6466 2596 Acked Flush 0 Sent Done Done false
6467 2597 Acked Read 2 Sent Done Done false
6468 2598 Acked Write 1 Sent Done Done false
6469 2599 Acked Flush 0 Sent Done Done false
6470 2600 Acked Flush 0 Sent Done Done false
6471 2601 Acked Flush 0 Sent Done Done false
6472 2602 Acked Read 6 Sent Done Done false
6473 2603 Acked Write 8 Sent Done Done false
6474 2604 Acked Read 7 Sent Done Done false
6475 2605 Acked Flush 0 Sent Done Done false
6476 2606 Acked Flush 0 Sent Done Done false
6477 2607 Acked Read 2 Sent Done Done false
6478 2608 Acked Write 10 Sent Done Done false
6479 2609 Acked Write 4 Sent Done Done false
6480 2610 Acked Write 5 Sent Done Done false
6481 2611 Acked Write 3 Sent Done Done false
6482 2612 Acked Read 9 Sent Done Done false
6483 2613 Acked Read 5 Sent Done Done false
6484 2614 Acked Read 10 Sent Done Done false
6485 2615 Acked Flush 0 Sent Done Done false
6486 2616 Acked Read 2 Sent Done Done false
6487 2617 Acked Write 8 Sent Done Done false
6488 2618 Acked Read 7 Sent Done Done false
6489 2619 Acked Read 2 Sent Done Done false
6490 2620 Acked Write 10 Sent Done Done false
6491 2621 Acked Read 1 Sent Done Done false
6492 2622 Acked Flush 0 Sent Done Done false
6493 2623 Acked Write 2 Sent Done Done false
6494 2624 Acked Write 8 Sent Done Done false
6495 2625 Acked Read 2 Sent Done Done false
6496 2626 Acked Write 10 Sent Done Done false
6497 2627 Acked Read 3 Sent Done Done false
6498 2628 Acked Write 3 Sent Done Done false
6499 2629 Acked Write 5 Sent Done Done false
6500 2630 Acked Read 6 Sent Done Done false
6501 2631 Acked Read 9 Sent Done Done false
6502 2632 Acked Write 1 Sent Done Done false
6503 2633 Acked Write 3 Sent Done Done false
6504 2634 Acked Read 10 Sent Done Done false
6505 2635 Acked Write 7 Sent Done Done false
6506 2636 Acked Read 1 Sent Done Done false
6507 2637 Acked Read 5 Sent Done Done false
6508 2638 Acked Write 9 Sent Done Done false
6509 2639 Acked Read 1 Sent Done Done false
6510 2640 Acked Write 8 Sent Done Done false
6511 2641 Acked Write 10 Sent Done Done false
6512 2642 Acked Write 3 Sent Done Done false
6513 2643 Acked Write 9 Sent Done Done false
6514 2644 Acked Read 4 Sent Done Done false
6515 2645 Acked Read 8 Sent Done Done false
6516 2646 Acked Read 9 Sent Done Done false
6517 2647 Acked Read 3 Sent Done Done false
6518 2648 Acked Write 3 Sent Done Done false
6519 2649 Acked Read 4 Sent Done Done false
6520 2650 Acked Write 6 Sent Done Done false
6521 2651 Acked Read 1 Sent Done Done false
6522 2652 Acked Write 9 Sent Done Done false
6523 2653 Acked Write 6 Sent Done Done false
6524 2654 Acked Read 2 Sent Done Done false
6525 2655 Acked Read 1 Sent Done Done false
6526 2656 Acked Write 8 Sent Done Done false
6527 2657 Acked Read 1 Sent Done Done false
6528 2658 Acked Read 2 Sent Done Done false
6529 2659 Acked Write 2 Sent Done Done false
6530 2660 Acked Write 8 Sent Done Done false
6531 2661 Acked Read 7 Sent Done Done false
6532 2662 Acked Read 8 Sent Done Done false
6533 2663 Acked Write 7 Sent Done Done false
6534 2664 Acked Write 5 Sent Done Done false
6535 2665 Acked Read 1 Sent Done Done false
6536 2666 Acked Write 7 Sent Done Done false
6537 2667 Acked Read 2 Sent Done Done false
6538 2668 Acked Read 7 Sent Done Done false
6539 2669 Acked Write 3 Sent Done Done false
6540 2670 Acked Write 9 Sent Done Done false
6541 2671 Acked Flush 0 Sent Done Done false
6542 2672 Acked Write 10 Sent Done Done false
6543 2673 Acked Write 8 Sent Done Done false
6544 2674 Acked Flush 0 Sent Done Done false
6545 2675 Acked Write 1 Sent Done Done false
6546 2676 Acked Write 5 Sent Done Done false
6547 2677 Acked Write 10 Sent Done Done false
6548 2678 Acked Write 10 Sent Done Done false
6549 2679 Acked Read 5 Sent Done Done false
6550 2680 Acked Write 3 Sent Done Done false
6551 2681 Acked Flush 0 Sent Done Done false
6552 2682 Acked Flush 0 Sent Done Done false
6553 2683 Acked Write 1 Sent Done Done false
6554 2684 Acked Flush 0 Sent Done Done false
6555 2685 Acked Read 9 Sent Done Done false
6556 2686 Acked Read 1 Sent Done Done false
6557 2687 Acked Read 8 Sent Done Done false
6558 2688 Acked Write 10 Sent Done Done false
6559 2689 Acked Write 9 Sent Done Done false
6560 2690 Acked Write 6 Sent Done Done false
6561 2691 Acked Read 3 Sent Done Done false
6562 2692 Acked Read 1 Sent Done Done false
6563 2693 Acked Read 3 Sent Done Done false
6564 2694 Acked Read 9 Sent Done Done false
6565 2695 Acked Write 8 Sent Done Done false
6566 2696 Acked Read 7 Sent Done Done false
6567 2697 Acked Read 6 Sent Done Done false
6568 2698 Acked Read 4 Sent Done Done false
6569 2699 Acked Write 9 Sent Done Done false
6570 2700 Acked Write 10 Sent Done Done false
6571 2701 Acked Write 9 Sent Done Done false
6572 2702 Acked Write 10 Sent Done Done false
6573 2703 Acked Read 6 Sent Done Done false
6574 2704 Acked Read 9 Sent Done Done false
6575 2705 Acked Read 2 Sent Done Done false
6576 2706 Acked Flush 0 Sent Done Done false
6577 2707 Acked Read 8 Sent Done Done false
6578 2708 Acked Read 6 Sent Done Done false
6579 2709 Acked Read 9 Sent Done Done false
6580 2710 Acked Read 4 Sent Done Done false
6581 2711 Acked Read 1 Sent Done Done false
6582 2712 Acked Read 2 Sent Done Done false
6583 2713 Acked Write 4 Sent Done Done false
6584 2714 Acked Write 9 Sent Done Done false
6585 2715 Acked Read 9 Sent Done Done false
6586 2716 Acked Write 2 Sent Done Done false
6587 2717 Acked Read 3 Sent Done Done false
6588 2718 Acked Write 6 Sent Done Done false
6589 2719 Acked Read 9 Sent Done Done false
6590 2720 Acked Read 10 Sent Done Done false
6591 2721 Acked Write 9 Sent Done Done false
6592 2722 Acked Write 10 Sent Done Done false
6593 2723 Acked Read 8 Sent Done Done false
6594 2724 Acked Write 7 Sent Done Done false
6595 2725 Acked Read 5 Sent Done Done false
6596 2726 Acked Read 3 Sent Done Done false
6597 2727 Acked Read 6 Sent Done Done false
6598 2728 Acked Write 3 Sent Done Done false
6599 2729 Acked Read 4 Sent Done Done false
6600 2730 Acked Read 2 Sent Done Done false
6601 2731 Acked Write 3 Sent Done Done false
6602 2732 Acked Read 6 Sent Done Done false
6603 2733 Acked Read 1 Sent Done Done false
6604 2734 Acked Write 5 Sent Done Done false
6605 2735 Acked Read 6 Sent Done Done false
6606 2736 Acked Write 8 Sent Done Done false
6607 2737 Acked Write 10 Sent Done Done false
6608 2738 Acked Read 4 Sent Done Done false
6609 2739 Acked Flush 0 Sent Done Done false
6610 2740 Acked Flush 0 Sent Done Done false
6611 2741 Acked Write 10 Sent Done Done false
6612 2742 Acked Write 1 Sent Done Done false
6613 2743 Acked Read 7 Sent Done Done false
6614 2744 Acked Write 1 Sent Done Done false
6615 2745 Acked Write 1 Sent Done Done false
6616 2746 Acked Write 2 Sent Done Done false
6617 2747 Acked Flush 0 Sent Done Done false
6618 2748 Acked Write 2 Sent Done Done false
6619 2749 Acked Read 2 Sent Done Done false
6620 2750 Acked Read 10 Sent Done Done false
6621 2751 Acked Write 4 Sent Done Done false
6622 2752 Acked Write 5 Sent Done Done false
6623 2753 Acked Read 2 Sent Done Done false
6624 2754 Acked Write 10 Sent Done Done false
6625 2755 Acked Read 7 Sent Done Done false
6626 2756 Acked Write 3 Sent Done Done false
6627 2757 Acked Write 6 Sent Done Done false
6628 2758 Acked Read 8 Sent Done Done false
6629 2759 Acked Write 5 Sent Done Done false
6630 2760 Acked Write 3 Sent Done Done false
6631 2761 Acked Write 2 Sent Done Done false
6632 2762 Acked Read 10 Sent Done Done false
6633 2763 Acked Write 5 Sent Done Done false
6634 2764 Acked Read 2 Sent Done Done false
6635 2765 Acked Read 9 Sent Done Done false
6636 2766 Acked Read 3 Sent Done Done false
6637 2767 Acked Read 7 Sent Done Done false
6638 2768 Acked Read 8 Sent Done Done false
6639 2769 Acked Write 4 Sent Done Done false
6640 2770 Acked Read 6 Sent Done Done false
6641 2771 Acked Flush 0 Sent Done Done false
6642 2772 Acked Write 6 Sent Done Done false
6643 2773 Acked Read 9 Sent Done Done false
6644 2774 Acked Read 9 Sent Done Done false
6645 2775 Acked Flush 0 Sent Done Done false
6646 2776 Acked Flush 0 Sent Done Done false
6647 2777 Acked Read 5 Sent Done Done false
6648 2778 Acked Write 5 Sent Done Done false
6649 2779 Acked Read 10 Sent Done Done false
6650 2780 Acked Write 1 Sent Done Done false
6651 2781 Acked Read 1 Sent Done Done false
6652 2782 Acked Read 6 Sent Done Done false
6653 2783 Acked Read 1 Sent Done Done false
6654 2784 Acked Flush 0 Sent Done Done false
6655 2785 Acked Write 8 Sent Done Done false
6656 2786 Acked Read 7 Sent Done Done false
6657 2787 Acked Write 7 Sent Done Done false
6658 2788 Acked Read 3 Sent Done Done false
6659 2789 Acked Write 6 Sent Done Done false
6660 2790 Acked Write 1 Sent Done Done false
6661 2791 Acked Read 7 Sent Done Done false
6662 2792 Acked Read 5 Sent Done Done false
6663 2793 Acked Write 4 Sent Done Done false
6664 2794 Acked Write 1 Sent Done Done false
6665 2795 Acked Write 3 Sent Done Done false
6666 2796 Acked Read 8 Sent Done Done false
6667 2797 Acked Read 2 Sent Done Done false
6668 2798 Acked Read 5 Sent Done Done false
6669 2799 Acked Read 7 Sent Done Done false
6670 2800 Acked Write 8 Sent Done Done false
6671 2801 Acked Read 9 Sent Done Done false
6672 2802 Acked Read 5 Sent Done Done false
6673 2803 Acked Flush 0 Sent Done Done false
6674 2804 Acked Write 7 Sent Done Done false
6675 2805 Acked Read 9 Sent Done Done false
6676 2806 Acked Write 8 Sent Done Done false
6677 2807 Acked Read 6 Sent Done Done false
6678 2808 Acked Write 9 Sent Done Done false
6679 2809 Acked Flush 0 Sent Done Done false
6680 STATES DS:0 DS:1 DS:2 TOTAL
6681 Sent 302 0 0 302
6682 Done 0 302 302 604
6683 Skipped 0 0 0 0
6684 Error 0 0 0 0
6685 Last Flush: 2507 2809 2809
6686 Downstairs last five completed: 2507 2506 2505 2504 2503
6687 Upstairs last five completed: 2809 2808 2807 2806 2805
6688 CLIENT: Up:0 ds:302 act:2
66892024-11-15T01:58:51.090ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
66902024-11-15T01:58:51.090ZWARNcrucible: ds_connection connect to 127.0.0.1:8810 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
66912024-11-15T01:58:51.090ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
66922024-11-15T01:58:51.090ZWARNcrucible: downstairs task for 0 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
66932024-11-15T01:58:51.090ZINFOcrucible: client task is exiting = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
6694 [8810] got start action
6695 [8810] I am going to restart
6696 Starting downstairs at port 8810
6697 Make output file at "/tmp/dsc/downstairs-8810.txt"
6698 [8810][0] reports Starting
6699 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1211)
6700 [8810][0] reports Running
6701 ----------------------------------------------------------------
6702 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
6703 States: Offline Active Active
6704 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
6705 2508 Acked Read 9 Sent Done Done false
6706 2509 Acked Flush 0 Sent Done Done false
6707 2510 Acked Write 2 Sent Done Done false
6708 2511 Acked Flush 0 Sent Done Done false
6709 2512 Acked Write 1 Sent Done Done false
6710 2513 Acked Write 8 Sent Done Done false
6711 2514 Acked Read 6 Sent Done Done false
6712 2515 Acked Read 8 Sent Done Done false
6713 2516 Acked Read 3 Sent Done Done false
6714 2517 Acked Read 6 Sent Done Done false
6715 2518 Acked Write 2 Sent Done Done false
6716 2519 Acked Read 3 Sent Done Done false
6717 2520 Acked Write 1 Sent Done Done false
6718 2521 Acked Write 6 Sent Done Done false
6719 2522 Acked Write 3 Sent Done Done false
6720 2523 Acked Read 5 Sent Done Done false
6721 2524 Acked Write 10 Sent Done Done false
6722 2525 Acked Flush 0 Sent Done Done false
6723 2526 Acked Read 6 Sent Done Done false
6724 2527 Acked Write 3 Sent Done Done false
6725 2528 Acked Read 9 Sent Done Done false
6726 2529 Acked Write 8 Sent Done Done false
6727 2530 Acked Read 5 Sent Done Done false
6728 2531 Acked Write 1 Sent Done Done false
6729 2532 Acked Read 8 Sent Done Done false
6730 2533 Acked Read 2 Sent Done Done false
6731 2534 Acked Read 2 Sent Done Done false
6732 2535 Acked Read 7 Sent Done Done false
6733 2536 Acked Write 2 Sent Done Done false
6734 2537 Acked Write 3 Sent Done Done false
6735 2538 Acked Write 3 Sent Done Done false
6736 2539 Acked Read 4 Sent Done Done false
6737 2540 Acked Read 10 Sent Done Done false
6738 2541 Acked Write 3 Sent Done Done false
6739 2542 Acked Read 9 Sent Done Done false
6740 2543 Acked Write 8 Sent Done Done false
6741 2544 Acked Read 8 Sent Done Done false
6742 2545 Acked Read 5 Sent Done Done false
6743 2546 Acked Write 5 Sent Done Done false
6744 2547 Acked Write 4 Sent Done Done false
6745 2548 Acked Read 8 Sent Done Done false
6746 2549 Acked Write 1 Sent Done Done false
6747 2550 Acked Read 3 Sent Done Done false
6748 2551 Acked Read 10 Sent Done Done false
6749 2552 Acked Read 4 Sent Done Done false
6750 2553 Acked Write 9 Sent Done Done false
6751 2554 Acked Write 1 Sent Done Done false
6752 2555 Acked Write 3 Sent Done Done false
6753 2556 Acked Flush 0 Sent Done Done false
6754 2557 Acked Read 2 Sent Done Done false
6755 2558 Acked Write 2 Sent Done Done false
6756 2559 Acked Read 8 Sent Done Done false
6757 2560 Acked Read 5 Sent Done Done false
6758 2561 Acked Write 3 Sent Done Done false
6759 2562 Acked Read 6 Sent Done Done false
6760 2563 Acked Read 10 Sent Done Done false
6761 2564 Acked Flush 0 Sent Done Done false
6762 2565 Acked Read 2 Sent Done Done false
6763 2566 Acked Write 2 Sent Done Done false
6764 2567 Acked Read 1 Sent Done Done false
6765 2568 Acked Write 10 Sent Done Done false
6766 2569 Acked Read 2 Sent Done Done false
6767 2570 Acked Write 3 Sent Done Done false
6768 2571 Acked Read 8 Sent Done Done false
6769 2572 Acked Read 4 Sent Done Done false
6770 2573 Acked Write 2 Sent Done Done false
6771 2574 Acked Write 5 Sent Done Done false
6772 2575 Acked Write 7 Sent Done Done false
6773 2576 Acked Read 2 Sent Done Done false
6774 2577 Acked Flush 0 Sent Done Done false
6775 2578 Acked Read 10 Sent Done Done false
6776 2579 Acked Read 5 Sent Done Done false
6777 2580 Acked Read 2 Sent Done Done false
6778 2581 Acked Read 4 Sent Done Done false
6779 2582 Acked Read 4 Sent Done Done false
6780 2583 Acked Read 8 Sent Done Done false
6781 2584 Acked Read 4 Sent Done Done false
6782 2585 Acked Write 1 Sent Done Done false
6783 2586 Acked Write 4 Sent Done Done false
6784 2587 Acked Write 5 Sent Done Done false
6785 2588 Acked Write 8 Sent Done Done false
6786 2589 Acked Read 3 Sent Done Done false
6787 2590 Acked Read 7 Sent Done Done false
6788 2591 Acked Write 8 Sent Done Done false
6789 2592 Acked Write 6 Sent Done Done false
6790 2593 Acked Read 9 Sent Done Done false
6791 2594 Acked Write 10 Sent Done Done false
6792 2595 Acked Write 8 Sent Done Done false
6793 2596 Acked Flush 0 Sent Done Done false
6794 2597 Acked Read 2 Sent Done Done false
6795 2598 Acked Write 1 Sent Done Done false
6796 2599 Acked Flush 0 Sent Done Done false
6797 2600 Acked Flush 0 Sent Done Done false
6798 2601 Acked Flush 0 Sent Done Done false
6799 2602 Acked Read 6 Sent Done Done false
6800 2603 Acked Write 8 Sent Done Done false
6801 2604 Acked Read 7 Sent Done Done false
6802 2605 Acked Flush 0 Sent Done Done false
6803 2606 Acked Flush 0 Sent Done Done false
6804 2607 Acked Read 2 Sent Done Done false
6805 2608 Acked Write 10 Sent Done Done false
6806 2609 Acked Write 4 Sent Done Done false
6807 2610 Acked Write 5 Sent Done Done false
6808 2611 Acked Write 3 Sent Done Done false
6809 2612 Acked Read 9 Sent Done Done false
6810 2613 Acked Read 5 Sent Done Done false
6811 2614 Acked Read 10 Sent Done Done false
6812 2615 Acked Flush 0 Sent Done Done false
6813 2616 Acked Read 2 Sent Done Done false
6814 2617 Acked Write 8 Sent Done Done false
6815 2618 Acked Read 7 Sent Done Done false
6816 2619 Acked Read 2 Sent Done Done false
6817 2620 Acked Write 10 Sent Done Done false
6818 2621 Acked Read 1 Sent Done Done false
6819 2622 Acked Flush 0 Sent Done Done false
6820 2623 Acked Write 2 Sent Done Done false
6821 2624 Acked Write 8 Sent Done Done false
6822 2625 Acked Read 2 Sent Done Done false
6823 2626 Acked Write 10 Sent Done Done false
6824 2627 Acked Read 3 Sent Done Done false
6825 2628 Acked Write 3 Sent Done Done false
6826 2629 Acked Write 5 Sent Done Done false
6827 2630 Acked Read 6 Sent Done Done false
6828 2631 Acked Read 9 Sent Done Done false
6829 2632 Acked Write 1 Sent Done Done false
6830 2633 Acked Write 3 Sent Done Done false
6831 2634 Acked Read 10 Sent Done Done false
6832 2635 Acked Write 7 Sent Done Done false
6833 2636 Acked Read 1 Sent Done Done false
6834 2637 Acked Read 5 Sent Done Done false
6835 2638 Acked Write 9 Sent Done Done false
6836 2639 Acked Read 1 Sent Done Done false
6837 2640 Acked Write 8 Sent Done Done false
6838 2641 Acked Write 10 Sent Done Done false
6839 2642 Acked Write 3 Sent Done Done false
6840 2643 Acked Write 9 Sent Done Done false
6841 2644 Acked Read 4 Sent Done Done false
6842 2645 Acked Read 8 Sent Done Done false
6843 2646 Acked Read 9 Sent Done Done false
6844 2647 Acked Read 3 Sent Done Done false
6845 2648 Acked Write 3 Sent Done Done false
6846 2649 Acked Read 4 Sent Done Done false
6847 2650 Acked Write 6 Sent Done Done false
6848 2651 Acked Read 1 Sent Done Done false
6849 2652 Acked Write 9 Sent Done Done false
6850 2653 Acked Write 6 Sent Done Done false
6851 2654 Acked Read 2 Sent Done Done false
6852 2655 Acked Read 1 Sent Done Done false
6853 2656 Acked Write 8 Sent Done Done false
6854 2657 Acked Read 1 Sent Done Done false
6855 2658 Acked Read 2 Sent Done Done false
6856 2659 Acked Write 2 Sent Done Done false
6857 2660 Acked Write 8 Sent Done Done false
6858 2661 Acked Read 7 Sent Done Done false
6859 2662 Acked Read 8 Sent Done Done false
6860 2663 Acked Write 7 Sent Done Done false
6861 2664 Acked Write 5 Sent Done Done false
6862 2665 Acked Read 1 Sent Done Done false
6863 2666 Acked Write 7 Sent Done Done false
6864 2667 Acked Read 2 Sent Done Done false
6865 2668 Acked Read 7 Sent Done Done false
6866 2669 Acked Write 3 Sent Done Done false
6867 2670 Acked Write 9 Sent Done Done false
6868 2671 Acked Flush 0 Sent Done Done false
6869 2672 Acked Write 10 Sent Done Done false
6870 2673 Acked Write 8 Sent Done Done false
6871 2674 Acked Flush 0 Sent Done Done false
6872 2675 Acked Write 1 Sent Done Done false
6873 2676 Acked Write 5 Sent Done Done false
6874 2677 Acked Write 10 Sent Done Done false
6875 2678 Acked Write 10 Sent Done Done false
6876 2679 Acked Read 5 Sent Done Done false
6877 2680 Acked Write 3 Sent Done Done false
6878 2681 Acked Flush 0 Sent Done Done false
6879 2682 Acked Flush 0 Sent Done Done false
6880 2683 Acked Write 1 Sent Done Done false
6881 2684 Acked Flush 0 Sent Done Done false
6882 2685 Acked Read 9 Sent Done Done false
6883 2686 Acked Read 1 Sent Done Done false
6884 2687 Acked Read 8 Sent Done Done false
6885 2688 Acked Write 10 Sent Done Done false
6886 2689 Acked Write 9 Sent Done Done false
6887 2690 Acked Write 6 Sent Done Done false
6888 2691 Acked Read 3 Sent Done Done false
6889 2692 Acked Read 1 Sent Done Done false
6890 2693 Acked Read 3 Sent Done Done false
6891 2694 Acked Read 9 Sent Done Done false
6892 2695 Acked Write 8 Sent Done Done false
6893 2696 Acked Read 7 Sent Done Done false
6894 2697 Acked Read 6 Sent Done Done false
6895 2698 Acked Read 4 Sent Done Done false
6896 2699 Acked Write 9 Sent Done Done false
6897 2700 Acked Write 10 Sent Done Done false
6898 2701 Acked Write 9 Sent Done Done false
6899 2702 Acked Write 10 Sent Done Done false
6900 2703 Acked Read 6 Sent Done Done false
6901 2704 Acked Read 9 Sent Done Done false
6902 2705 Acked Read 2 Sent Done Done false
6903 2706 Acked Flush 0 Sent Done Done false
6904 2707 Acked Read 8 Sent Done Done false
6905 2708 Acked Read 6 Sent Done Done false
6906 2709 Acked Read 9 Sent Done Done false
6907 2710 Acked Read 4 Sent Done Done false
6908 2711 Acked Read 1 Sent Done Done false
6909 2712 Acked Read 2 Sent Done Done false
6910 2713 Acked Write 4 Sent Done Done false
6911 2714 Acked Write 9 Sent Done Done false
6912 2715 Acked Read 9 Sent Done Done false
6913 2716 Acked Write 2 Sent Done Done false
6914 2717 Acked Read 3 Sent Done Done false
6915 2718 Acked Write 6 Sent Done Done false
6916 2719 Acked Read 9 Sent Done Done false
6917 2720 Acked Read 10 Sent Done Done false
6918 2721 Acked Write 9 Sent Done Done false
6919 2722 Acked Write 10 Sent Done Done false
6920 2723 Acked Read 8 Sent Done Done false
6921 2724 Acked Write 7 Sent Done Done false
6922 2725 Acked Read 5 Sent Done Done false
6923 2726 Acked Read 3 Sent Done Done false
6924 2727 Acked Read 6 Sent Done Done false
6925 2728 Acked Write 3 Sent Done Done false
6926 2729 Acked Read 4 Sent Done Done false
6927 2730 Acked Read 2 Sent Done Done false
6928 2731 Acked Write 3 Sent Done Done false
6929 2732 Acked Read 6 Sent Done Done false
6930 2733 Acked Read 1 Sent Done Done false
6931 2734 Acked Write 5 Sent Done Done false
6932 2735 Acked Read 6 Sent Done Done false
6933 2736 Acked Write 8 Sent Done Done false
6934 2737 Acked Write 10 Sent Done Done false
6935 2738 Acked Read 4 Sent Done Done false
6936 2739 Acked Flush 0 Sent Done Done false
6937 2740 Acked Flush 0 Sent Done Done false
6938 2741 Acked Write 10 Sent Done Done false
6939 2742 Acked Write 1 Sent Done Done false
6940 2743 Acked Read 7 Sent Done Done false
6941 2744 Acked Write 1 Sent Done Done false
6942 2745 Acked Write 1 Sent Done Done false
6943 2746 Acked Write 2 Sent Done Done false
6944 2747 Acked Flush 0 Sent Done Done false
6945 2748 Acked Write 2 Sent Done Done false
6946 2749 Acked Read 2 Sent Done Done false
6947 2750 Acked Read 10 Sent Done Done false
6948 2751 Acked Write 4 Sent Done Done false
6949 2752 Acked Write 5 Sent Done Done false
6950 2753 Acked Read 2 Sent Done Done false
6951 2754 Acked Write 10 Sent Done Done false
6952 2755 Acked Read 7 Sent Done Done false
6953 2756 Acked Write 3 Sent Done Done false
6954 2757 Acked Write 6 Sent Done Done false
6955 2758 Acked Read 8 Sent Done Done false
6956 2759 Acked Write 5 Sent Done Done false
6957 2760 Acked Write 3 Sent Done Done false
6958 2761 Acked Write 2 Sent Done Done false
6959 2762 Acked Read 10 Sent Done Done false
6960 2763 Acked Write 5 Sent Done Done false
6961 2764 Acked Read 2 Sent Done Done false
6962 2765 Acked Read 9 Sent Done Done false
6963 2766 Acked Read 3 Sent Done Done false
6964 2767 Acked Read 7 Sent Done Done false
6965 2768 Acked Read 8 Sent Done Done false
6966 2769 Acked Write 4 Sent Done Done false
6967 2770 Acked Read 6 Sent Done Done false
6968 2771 Acked Flush 0 Sent Done Done false
6969 2772 Acked Write 6 Sent Done Done false
6970 2773 Acked Read 9 Sent Done Done false
6971 2774 Acked Read 9 Sent Done Done false
6972 2775 Acked Flush 0 Sent Done Done false
6973 2776 Acked Flush 0 Sent Done Done false
6974 2777 Acked Read 5 Sent Done Done false
6975 2778 Acked Write 5 Sent Done Done false
6976 2779 Acked Read 10 Sent Done Done false
6977 2780 Acked Write 1 Sent Done Done false
6978 2781 Acked Read 1 Sent Done Done false
6979 2782 Acked Read 6 Sent Done Done false
6980 2783 Acked Read 1 Sent Done Done false
6981 2784 Acked Flush 0 Sent Done Done false
6982 2785 Acked Write 8 Sent Done Done false
6983 2786 Acked Read 7 Sent Done Done false
6984 2787 Acked Write 7 Sent Done Done false
6985 2788 Acked Read 3 Sent Done Done false
6986 2789 Acked Write 6 Sent Done Done false
6987 2790 Acked Write 1 Sent Done Done false
6988 2791 Acked Read 7 Sent Done Done false
6989 2792 Acked Read 5 Sent Done Done false
6990 2793 Acked Write 4 Sent Done Done false
6991 2794 Acked Write 1 Sent Done Done false
6992 2795 Acked Write 3 Sent Done Done false
6993 2796 Acked Read 8 Sent Done Done false
6994 2797 Acked Read 2 Sent Done Done false
6995 2798 Acked Read 5 Sent Done Done false
6996 2799 Acked Read 7 Sent Done Done false
6997 2800 Acked Write 8 Sent Done Done false
6998 2801 Acked Read 9 Sent Done Done false
6999 2802 Acked Read 5 Sent Done Done false
7000 2803 Acked Flush 0 Sent Done Done false
7001 2804 Acked Write 7 Sent Done Done false
7002 2805 Acked Read 9 Sent Done Done false
7003 2806 Acked Write 8 Sent Done Done false
7004 2807 Acked Read 6 Sent Done Done false
7005 2808 Acked Write 9 Sent Done Done false
7006 2809 Acked Flush 0 Sent Done Done false
7007 STATES DS:0 DS:1 DS:2 TOTAL
7008 Sent 302 0 0 302
7009 Done 0 302 302 604
7010 Skipped 0 0 0 0
7011 Error 0 0 0 0
7012 Last Flush: 2507 2809 2809
7013 Downstairs last five completed: 2507 2506 2505 2504 2503
7014 Upstairs last five completed: 2809 2808 2807 2806 2805
7015 CLIENT: Up:0 ds:302 act:2
7016 ----------------------------------------------------------------
7017 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
7018 States: Offline Active Active
7019 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
7020 2508 Acked Read 9 Sent Done Done false
7021 2509 Acked Flush 0 Sent Done Done false
7022 2510 Acked Write 2 Sent Done Done false
7023 2511 Acked Flush 0 Sent Done Done false
7024 2512 Acked Write 1 Sent Done Done false
7025 2513 Acked Write 8 Sent Done Done false
7026 2514 Acked Read 6 Sent Done Done false
7027 2515 Acked Read 8 Sent Done Done false
7028 2516 Acked Read 3 Sent Done Done false
7029 2517 Acked Read 6 Sent Done Done false
7030 2518 Acked Write 2 Sent Done Done false
7031 2519 Acked Read 3 Sent Done Done false
7032 2520 Acked Write 1 Sent Done Done false
7033 2521 Acked Write 6 Sent Done Done false
7034 2522 Acked Write 3 Sent Done Done false
7035 2523 Acked Read 5 Sent Done Done false
7036 2524 Acked Write 10 Sent Done Done false
7037 2525 Acked Flush 0 Sent Done Done false
7038 2526 Acked Read 6 Sent Done Done false
7039 2527 Acked Write 3 Sent Done Done false
7040 2528 Acked Read 9 Sent Done Done false
7041 2529 Acked Write 8 Sent Done Done false
7042 2530 Acked Read 5 Sent Done Done false
7043 2531 Acked Write 1 Sent Done Done false
7044 2532 Acked Read 8 Sent Done Done false
7045 2533 Acked Read 2 Sent Done Done false
7046 2534 Acked Read 2 Sent Done Done false
7047 2535 Acked Read 7 Sent Done Done false
7048 2536 Acked Write 2 Sent Done Done false
7049 2537 Acked Write 3 Sent Done Done false
7050 2538 Acked Write 3 Sent Done Done false
7051 2539 Acked Read 4 Sent Done Done false
7052 2540 Acked Read 10 Sent Done Done false
7053 2541 Acked Write 3 Sent Done Done false
7054 2542 Acked Read 9 Sent Done Done false
7055 2543 Acked Write 8 Sent Done Done false
7056 2544 Acked Read 8 Sent Done Done false
7057 2545 Acked Read 5 Sent Done Done false
7058 2546 Acked Write 5 Sent Done Done false
7059 2547 Acked Write 4 Sent Done Done false
7060 2548 Acked Read 8 Sent Done Done false
7061 2549 Acked Write 1 Sent Done Done false
7062 2550 Acked Read 3 Sent Done Done false
7063 2551 Acked Read 10 Sent Done Done false
7064 2552 Acked Read 4 Sent Done Done false
7065 2553 Acked Write 9 Sent Done Done false
7066 2554 Acked Write 1 Sent Done Done false
7067 2555 Acked Write 3 Sent Done Done false
7068 2556 Acked Flush 0 Sent Done Done false
7069 2557 Acked Read 2 Sent Done Done false
7070 2558 Acked Write 2 Sent Done Done false
7071 2559 Acked Read 8 Sent Done Done false
7072 2560 Acked Read 5 Sent Done Done false
7073 2561 Acked Write 3 Sent Done Done false
7074 2562 Acked Read 6 Sent Done Done false
7075 2563 Acked Read 10 Sent Done Done false
7076 2564 Acked Flush 0 Sent Done Done false
7077 2565 Acked Read 2 Sent Done Done false
7078 2566 Acked Write 2 Sent Done Done false
7079 2567 Acked Read 1 Sent Done Done false
7080 2568 Acked Write 10 Sent Done Done false
7081 2569 Acked Read 2 Sent Done Done false
7082 2570 Acked Write 3 Sent Done Done false
7083 2571 Acked Read 8 Sent Done Done false
7084 2572 Acked Read 4 Sent Done Done false
7085 2573 Acked Write 2 Sent Done Done false
7086 2574 Acked Write 5 Sent Done Done false
7087 2575 Acked Write 7 Sent Done Done false
7088 2576 Acked Read 2 Sent Done Done false
7089 2577 Acked Flush 0 Sent Done Done false
7090 2578 Acked Read 10 Sent Done Done false
7091 2579 Acked Read 5 Sent Done Done false
7092 2580 Acked Read 2 Sent Done Done false
7093 2581 Acked Read 4 Sent Done Done false
7094 2582 Acked Read 4 Sent Done Done false
7095 2583 Acked Read 8 Sent Done Done false
7096 2584 Acked Read 4 Sent Done Done false
7097 2585 Acked Write 1 Sent Done Done false
7098 2586 Acked Write 4 Sent Done Done false
7099 2587 Acked Write 5 Sent Done Done false
7100 2588 Acked Write 8 Sent Done Done false
7101 2589 Acked Read 3 Sent Done Done false
7102 2590 Acked Read 7 Sent Done Done false
7103 2591 Acked Write 8 Sent Done Done false
7104 2592 Acked Write 6 Sent Done Done false
7105 2593 Acked Read 9 Sent Done Done false
7106 2594 Acked Write 10 Sent Done Done false
7107 2595 Acked Write 8 Sent Done Done false
7108 2596 Acked Flush 0 Sent Done Done false
7109 2597 Acked Read 2 Sent Done Done false
7110 2598 Acked Write 1 Sent Done Done false
7111 2599 Acked Flush 0 Sent Done Done false
7112 2600 Acked Flush 0 Sent Done Done false
7113 2601 Acked Flush 0 Sent Done Done false
7114 2602 Acked Read 6 Sent Done Done false
7115 2603 Acked Write 8 Sent Done Done false
7116 2604 Acked Read 7 Sent Done Done false
7117 2605 Acked Flush 0 Sent Done Done false
7118 2606 Acked Flush 0 Sent Done Done false
7119 2607 Acked Read 2 Sent Done Done false
7120 2608 Acked Write 10 Sent Done Done false
7121 2609 Acked Write 4 Sent Done Done false
7122 2610 Acked Write 5 Sent Done Done false
7123 2611 Acked Write 3 Sent Done Done false
7124 2612 Acked Read 9 Sent Done Done false
7125 2613 Acked Read 5 Sent Done Done false
7126 2614 Acked Read 10 Sent Done Done false
7127 2615 Acked Flush 0 Sent Done Done false
7128 2616 Acked Read 2 Sent Done Done false
7129 2617 Acked Write 8 Sent Done Done false
7130 2618 Acked Read 7 Sent Done Done false
7131 2619 Acked Read 2 Sent Done Done false
7132 2620 Acked Write 10 Sent Done Done false
7133 2621 Acked Read 1 Sent Done Done false
7134 2622 Acked Flush 0 Sent Done Done false
7135 2623 Acked Write 2 Sent Done Done false
7136 2624 Acked Write 8 Sent Done Done false
7137 2625 Acked Read 2 Sent Done Done false
7138 2626 Acked Write 10 Sent Done Done false
7139 2627 Acked Read 3 Sent Done Done false
7140 2628 Acked Write 3 Sent Done Done false
7141 2629 Acked Write 5 Sent Done Done false
7142 2630 Acked Read 6 Sent Done Done false
7143 2631 Acked Read 9 Sent Done Done false
7144 2632 Acked Write 1 Sent Done Done false
7145 2633 Acked Write 3 Sent Done Done false
7146 2634 Acked Read 10 Sent Done Done false
7147 2635 Acked Write 7 Sent Done Done false
7148 2636 Acked Read 1 Sent Done Done false
7149 2637 Acked Read 5 Sent Done Done false
7150 2638 Acked Write 9 Sent Done Done false
7151 2639 Acked Read 1 Sent Done Done false
7152 2640 Acked Write 8 Sent Done Done false
7153 2641 Acked Write 10 Sent Done Done false
7154 2642 Acked Write 3 Sent Done Done false
7155 2643 Acked Write 9 Sent Done Done false
7156 2644 Acked Read 4 Sent Done Done false
7157 2645 Acked Read 8 Sent Done Done false
7158 2646 Acked Read 9 Sent Done Done false
7159 2647 Acked Read 3 Sent Done Done false
7160 2648 Acked Write 3 Sent Done Done false
7161 2649 Acked Read 4 Sent Done Done false
7162 2650 Acked Write 6 Sent Done Done false
7163 2651 Acked Read 1 Sent Done Done false
7164 2652 Acked Write 9 Sent Done Done false
7165 2653 Acked Write 6 Sent Done Done false
7166 2654 Acked Read 2 Sent Done Done false
7167 2655 Acked Read 1 Sent Done Done false
7168 2656 Acked Write 8 Sent Done Done false
7169 2657 Acked Read 1 Sent Done Done false
7170 2658 Acked Read 2 Sent Done Done false
7171 2659 Acked Write 2 Sent Done Done false
7172 2660 Acked Write 8 Sent Done Done false
7173 2661 Acked Read 7 Sent Done Done false
7174 2662 Acked Read 8 Sent Done Done false
7175 2663 Acked Write 7 Sent Done Done false
7176 2664 Acked Write 5 Sent Done Done false
7177 2665 Acked Read 1 Sent Done Done false
7178 2666 Acked Write 7 Sent Done Done false
7179 2667 Acked Read 2 Sent Done Done false
7180 2668 Acked Read 7 Sent Done Done false
7181 2669 Acked Write 3 Sent Done Done false
7182 2670 Acked Write 9 Sent Done Done false
7183 2671 Acked Flush 0 Sent Done Done false
7184 2672 Acked Write 10 Sent Done Done false
7185 2673 Acked Write 8 Sent Done Done false
7186 2674 Acked Flush 0 Sent Done Done false
7187 2675 Acked Write 1 Sent Done Done false
7188 2676 Acked Write 5 Sent Done Done false
7189 2677 Acked Write 10 Sent Done Done false
7190 2678 Acked Write 10 Sent Done Done false
7191 2679 Acked Read 5 Sent Done Done false
7192 2680 Acked Write 3 Sent Done Done false
7193 2681 Acked Flush 0 Sent Done Done false
7194 2682 Acked Flush 0 Sent Done Done false
7195 2683 Acked Write 1 Sent Done Done false
7196 2684 Acked Flush 0 Sent Done Done false
7197 2685 Acked Read 9 Sent Done Done false
7198 2686 Acked Read 1 Sent Done Done false
7199 2687 Acked Read 8 Sent Done Done false
7200 2688 Acked Write 10 Sent Done Done false
7201 2689 Acked Write 9 Sent Done Done false
7202 2690 Acked Write 6 Sent Done Done false
7203 2691 Acked Read 3 Sent Done Done false
7204 2692 Acked Read 1 Sent Done Done false
7205 2693 Acked Read 3 Sent Done Done false
7206 2694 Acked Read 9 Sent Done Done false
7207 2695 Acked Write 8 Sent Done Done false
7208 2696 Acked Read 7 Sent Done Done false
7209 2697 Acked Read 6 Sent Done Done false
7210 2698 Acked Read 4 Sent Done Done false
7211 2699 Acked Write 9 Sent Done Done false
7212 2700 Acked Write 10 Sent Done Done false
7213 2701 Acked Write 9 Sent Done Done false
7214 2702 Acked Write 10 Sent Done Done false
7215 2703 Acked Read 6 Sent Done Done false
7216 2704 Acked Read 9 Sent Done Done false
7217 2705 Acked Read 2 Sent Done Done false
7218 2706 Acked Flush 0 Sent Done Done false
7219 2707 Acked Read 8 Sent Done Done false
7220 2708 Acked Read 6 Sent Done Done false
7221 2709 Acked Read 9 Sent Done Done false
7222 2710 Acked Read 4 Sent Done Done false
7223 2711 Acked Read 1 Sent Done Done false
7224 2712 Acked Read 2 Sent Done Done false
7225 2713 Acked Write 4 Sent Done Done false
7226 2714 Acked Write 9 Sent Done Done false
7227 2715 Acked Read 9 Sent Done Done false
7228 2716 Acked Write 2 Sent Done Done false
7229 2717 Acked Read 3 Sent Done Done false
7230 2718 Acked Write 6 Sent Done Done false
7231 2719 Acked Read 9 Sent Done Done false
7232 2720 Acked Read 10 Sent Done Done false
7233 2721 Acked Write 9 Sent Done Done false
7234 2722 Acked Write 10 Sent Done Done false
7235 2723 Acked Read 8 Sent Done Done false
7236 2724 Acked Write 7 Sent Done Done false
7237 2725 Acked Read 5 Sent Done Done false
7238 2726 Acked Read 3 Sent Done Done false
7239 2727 Acked Read 6 Sent Done Done false
7240 2728 Acked Write 3 Sent Done Done false
7241 2729 Acked Read 4 Sent Done Done false
7242 2730 Acked Read 2 Sent Done Done false
7243 2731 Acked Write 3 Sent Done Done false
7244 2732 Acked Read 6 Sent Done Done false
7245 2733 Acked Read 1 Sent Done Done false
7246 2734 Acked Write 5 Sent Done Done false
7247 2735 Acked Read 6 Sent Done Done false
7248 2736 Acked Write 8 Sent Done Done false
7249 2737 Acked Write 10 Sent Done Done false
7250 2738 Acked Read 4 Sent Done Done false
7251 2739 Acked Flush 0 Sent Done Done false
7252 2740 Acked Flush 0 Sent Done Done false
7253 2741 Acked Write 10 Sent Done Done false
7254 2742 Acked Write 1 Sent Done Done false
7255 2743 Acked Read 7 Sent Done Done false
7256 2744 Acked Write 1 Sent Done Done false
7257 2745 Acked Write 1 Sent Done Done false
7258 2746 Acked Write 2 Sent Done Done false
7259 2747 Acked Flush 0 Sent Done Done false
7260 2748 Acked Write 2 Sent Done Done false
7261 2749 Acked Read 2 Sent Done Done false
7262 2750 Acked Read 10 Sent Done Done false
7263 2751 Acked Write 4 Sent Done Done false
7264 2752 Acked Write 5 Sent Done Done false
7265 2753 Acked Read 2 Sent Done Done false
7266 2754 Acked Write 10 Sent Done Done false
7267 2755 Acked Read 7 Sent Done Done false
7268 2756 Acked Write 3 Sent Done Done false
7269 2757 Acked Write 6 Sent Done Done false
7270 2758 Acked Read 8 Sent Done Done false
7271 2759 Acked Write 5 Sent Done Done false
7272 2760 Acked Write 3 Sent Done Done false
7273 2761 Acked Write 2 Sent Done Done false
7274 2762 Acked Read 10 Sent Done Done false
7275 2763 Acked Write 5 Sent Done Done false
7276 2764 Acked Read 2 Sent Done Done false
7277 2765 Acked Read 9 Sent Done Done false
7278 2766 Acked Read 3 Sent Done Done false
7279 2767 Acked Read 7 Sent Done Done false
7280 2768 Acked Read 8 Sent Done Done false
7281 2769 Acked Write 4 Sent Done Done false
7282 2770 Acked Read 6 Sent Done Done false
7283 2771 Acked Flush 0 Sent Done Done false
7284 2772 Acked Write 6 Sent Done Done false
7285 2773 Acked Read 9 Sent Done Done false
7286 2774 Acked Read 9 Sent Done Done false
7287 2775 Acked Flush 0 Sent Done Done false
7288 2776 Acked Flush 0 Sent Done Done false
7289 2777 Acked Read 5 Sent Done Done false
7290 2778 Acked Write 5 Sent Done Done false
7291 2779 Acked Read 10 Sent Done Done false
7292 2780 Acked Write 1 Sent Done Done false
7293 2781 Acked Read 1 Sent Done Done false
7294 2782 Acked Read 6 Sent Done Done false
7295 2783 Acked Read 1 Sent Done Done false
7296 2784 Acked Flush 0 Sent Done Done false
7297 2785 Acked Write 8 Sent Done Done false
7298 2786 Acked Read 7 Sent Done Done false
7299 2787 Acked Write 7 Sent Done Done false
7300 2788 Acked Read 3 Sent Done Done false
7301 2789 Acked Write 6 Sent Done Done false
7302 2790 Acked Write 1 Sent Done Done false
7303 2791 Acked Read 7 Sent Done Done false
7304 2792 Acked Read 5 Sent Done Done false
7305 2793 Acked Write 4 Sent Done Done false
7306 2794 Acked Write 1 Sent Done Done false
7307 2795 Acked Write 3 Sent Done Done false
7308 2796 Acked Read 8 Sent Done Done false
7309 2797 Acked Read 2 Sent Done Done false
7310 2798 Acked Read 5 Sent Done Done false
7311 2799 Acked Read 7 Sent Done Done false
7312 2800 Acked Write 8 Sent Done Done false
7313 2801 Acked Read 9 Sent Done Done false
7314 2802 Acked Read 5 Sent Done Done false
7315 2803 Acked Flush 0 Sent Done Done false
7316 2804 Acked Write 7 Sent Done Done false
7317 2805 Acked Read 9 Sent Done Done false
7318 2806 Acked Write 8 Sent Done Done false
7319 2807 Acked Read 6 Sent Done Done false
7320 2808 Acked Write 9 Sent Done Done false
7321 2809 Acked Flush 0 Sent Done Done false
7322 STATES DS:0 DS:1 DS:2 TOTAL
7323 Sent 302 0 0 302
7324 Done 0 302 302 604
7325 Skipped 0 0 0 0
7326 Error 0 0 0 0
7327 Last Flush: 2507 2809 2809
7328 Downstairs last five completed: 2507 2506 2505 2504 2503
7329 Upstairs last five completed: 2809 2808 2807 2806 2805
7330 CLIENT: Up:0 ds:302 act:2
73312024-11-15T01:59:01.092ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
73322024-11-15T01:59:01.092ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:64344) = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
73332024-11-15T01:59:01.093ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
73342024-11-15T01:59:01.094ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
73352024-11-15T01:59:01.094ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
73362024-11-15T01:59:01.094ZINFOcrucible: send last flush ID to this DS: 2507 = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
73372024-11-15T01:59:01.094ZINFOcrucible: Replied this last flush ID: 2507 = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
73382024-11-15T01:59:01.094ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
73392024-11-15T01:59:01.094ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
73402024-11-15T01:59:01.094ZINFOcrucible: [0] client re-new 302 jobs since flush 2507 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
73412024-11-15T01:59:01.094ZINFOcrucible: [0] Replayed 302 jobs since flush: 2507 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
7342 ----------------------------------------------------------------
7343 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
7344 Downstairs last five completed: 2809 2808 2807 2806 2805
7345 Upstairs last five completed: 2809 2808 2807 2806 2805
7346 CLIENT: Up:0 ds:0 act:3
7347 Replay: All jobs finished
7348 Main task has work to do, go find it
7349 got dsc Stop(0)
7350 stop 0
7351 Nov 15 01:59:01.333 INFO request completed, [8810] Got stop action so:false kr:false
7352 latency_us: 362, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 836a1cd6-cc78-419b-b920-ae76202de1be, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
73532024-11-15T01:59:01.333ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
73542024-11-15T01:59:01.333ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
73552024-11-15T01:59:01.333ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
73562024-11-15T01:59:01.333ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
73572024-11-15T01:59:01.333ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
7358 Nov 15 01:59:01.334 INFO request completed, latency_us: 205, response_code: 200, uri: /state/cid/0{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2024-11-15T01:59:01.334061409Z","hostname":"w-01JCPRHT5F2MPA44KVJHM3S2PB","pid":1205,"":"io task","client":"0","":"downstairs","session_id":"971dcb2e-6891-4ecf-b367-1e04e52065ee"}
7359 , method: GET, req_id: f1da6756-1d26-4fa5-920e-b7d5ef0705c4, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
7360 [8810] Exited with: ExitStatus(unix_wait_status(9))
7361 [8810][0] reports Exit
7362 Nov 15 01:59:05.336 INFO request completed, latency_us: 213, response_code: 200, uri: /state/cid/0, method: GET, req_id: 59288ae0-6d78-43f1-97b5-60fe67a72f1e, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
7363 001/300 Read block 161 len 20480
7364 002/300 Read block 4497 len 8192
7365 003/300 Read block 3758 len 36864
7366 004/300 Read block 3010 len 24576
7367 005/300 Read block 3053 len 12288
7368 006/300 Read block 257 len 40960
7369 007/300 Read block 4553 len 8192
7370 008/300 Read block 4914 len 40960
7371 009/300 Read block 3535 len 36864
7372 010/300 Write block 4010 len 36864 data: 2 2 2 2 2 2 2 2 3
7373 011/300 Flush
7374 012/300 Write block 1491 len 4096 data: 4
7375 013/300 Write block 2863 len 20480 data: 2 2 2 2 3
7376 014/300 Flush
7377 015/300 Write block 4875 len 12288 data: 4 3 3
7378 016/300 Read block 63 len 16384
7379 017/300 Read block 28 len 12288
7380 018/300 Read block 685 len 4096
7381 019/300 Write block 1543 len 36864 data: 4 3 2 3 3 3 3 4 4
7382 020/300 Read block 4917 len 4096
7383 021/300 Read block 3476 len 40960
7384 022/300 Read block 4199 len 32768
7385 023/300 Write block 1174 len 20480 data: 4 4 4 4 3
7386 024/300 Write block 191 len 16384 data: 2 2 2 2
7387 025/300 Write block 290 len 4096 data: 4
7388 026/300 Write block 2336 len 20480 data: 3 3 3 3 4
7389 027/300 Read block 766 len 36864
7390 028/300 Write block 4889 len 16384 data: 3 4 3 3
7391 029/300 Read block 1711 len 24576
7392 030/300 Write block 2931 len 16384 data: 4 4 5 5
7393 031/300 Write block 2308 len 8192 data: 3 3
7394 032/300 Write block 1717 len 36864 data: 2 2 2 2 2 2 2 2 2
7395 033/300 Write block 1124 len 36864 data: 4 4 3 3 3 3 2 2 2
7396 034/300 Read block 3117 len 28672
7397 035/300 Read block 2306 len 8192
7398 036/300 Read block 1168 len 4096
7399 037/300 Flush
7400 038/300 Read block 2780 len 16384
7401 039/300 Read block 1317 len 24576
7402 040/300 Read block 3946 len 40960
7403 041/300 Read block 413 len 40960
7404 042/300 Write block 4175 len 20480 data: 3 3 3 3 2
7405 043/300 Read block 2443 len 28672
7406 044/300 Write block 3243 len 36864 data: 2 2 2 2 2 2 2 2 2
7407 045/300 Read block 4102 len 36864
7408 046/300 Read block 3999 len 40960
7409 047/300 Read block 4315 len 4096
7410 048/300 Read block 435 len 12288
7411 049/300 Read block 3756 len 36864
7412 050/300 Write block 1701 len 32768 data: 3 3 3 3 3 3 2 2
7413 051/300 Read block 3735 len 8192
7414 052/300 Read block 2706 len 36864
7415 053/300 Write block 4122 len 20480 data: 2 2 2 2 2
7416 054/300 Read block 4365 len 16384
7417 055/300 Read block 1325 len 28672
7418 056/300 Write block 410 len 4096 data: 3
7419 057/300 Read block 1778 len 36864
7420 058/300 Write block 3860 len 12288 data: 2 2 3
7421 059/300 Read block 782 len 12288
7422 060/300 Flush
7423 061/300 Read block 1026 len 4096
7424 062/300 Read block 1728 len 36864
7425 063/300 Read block 4553 len 36864
7426 064/300 Write block 1547 len 16384 data: 4 4 4 5
7427 065/300 Write block 2052 len 40960 data: 2 2 2 2 2 2 2 4 4 4
7428 066/300 Read block 581 len 36864
7429 067/300 Flush
7430 068/300 Flush
7431 069/300 Read block 1060 len 24576
7432 070/300 Write block 4246 len 4096 data: 2
7433 071/300 Read block 4555 len 24576
7434 072/300 Write block 3280 len 20480 data: 2 2 2 2 2
7435 073/300 Read block 4316 len 16384
7436 074/300 Read block 3060 len 8192
7437 075/300 Write block 2096 len 36864 data: 2 2 2 2 2 2 2 2 2
7438 076/300 Write block 195 len 28672 data: 2 2 2 2 2 2 3
7439 077/300 Flush
7440 078/300 Write block 2980 len 28672 data: 3 4 4 4 4 4 4
7441 079/300 Flush
7442 080/300 Flush
7443 081/300 Read block 4233 len 32768
7444 082/300 Write block 1899 len 16384 data: 3 3 3 3
7445 083/300 Write block 589 len 28672 data: 4 4 5 5 5 4 4
7446 084/300 Read block 1238 len 8192
7447 085/300 Write block 3254 len 4096 data: 2
7448 086/300 Read block 1852 len 4096
7449 087/300 Read block 3924 len 20480
7450 088/300 Read block 1450 len 24576
7451 089/300 Read block 414 len 40960
7452 090/300 Write block 441 len 20480 data: 2 2 2 2 3
7453 091/300 Read block 4286 len 12288
7454 092/300 Read block 866 len 32768
7455 093/300 Write block 3251 len 32768 data: 3 2 2 3 2 3 3 3
7456 094/300 Write block 1114 len 36864 data: 2 2 2 2 2 3 3 4 4
7457 095/300 Write block 427 len 28672 data: 2 2 3 3 3 3 3
7458 096/300 Write block 3030 len 20480 data: 2 2 2 2 3
7459 097/300 Write block 1290 len 20480 data: 3 3 2 2 3
7460 098/300 Write block 4488 len 32768 data: 2 2 3 3 3 4 4 4
7461 099/300 Read block 1198 len 12288
7462 100/300 Read block 992 len 4096
7463 101/300 Read block 2615 len 28672
7464 102/300 Read block 4218 len 36864
7465 103/300 Read block 280 len 8192
7466 104/300 Read block 618 len 28672
7467 105/300 Write block 767 len 8192 data: 3 3
7468 106/300 Read block 996 len 8192
7469 107/300 Write block 3920 len 36864 data: 3 3 3 4 4 3 3 3 2
7470 108/300 Flush
7471 109/300 Read block 12 len 24576
7472 110/300 Write block 392 len 16384 data: 3 3 3 3
7473 111/300 Read block 3182 len 40960
7474 112/300 Read block 739 len 36864
7475 113/300 Read block 4726 len 40960
7476 114/300 Flush
7477 115/300 Write block 4646 len 20480 data: 3 3 3 3 3
7478 116/300 Write block 2427 len 28672 data: 3 3 3 3 3 3 3
7479 117/300 Write block 3434 len 4096 data: 3
7480 118/300 Read block 72 len 24576
7481 119/300 Read block 3792 len 8192
7482 120/300 Write block 2271 len 12288 data: 3 3 4
7483 121/300 Write block 2963 len 4096 data: 2
7484 122/300 Write block 2748 len 4096 data: 2
7485 123/300 Read block 1352 len 32768
7486 124/300 Write block 1425 len 8192 data: 3 3
7487 125/300 Read block 734 len 28672
7488 126/300 Write block 121 len 36864 data: 2 2 2 2 2 2 2 2 2
7489 127/300 Read block 3925 len 32768
7490 128/300 Write block 2150 len 32768 data: 5 5 5 3 3 4 3 3
7491 129/300 Flush
7492 130/300 Read block 1462 len 28672
7493 131/300 Write block 4413 len 36864 data: 2 2 2 2 2 3 4 4 4
7494 132/300 Flush
7495 133/300 Write block 3744 len 36864 data: 4 4 4 4 3 3 4 5 3
7496 134/300 Write block 4194 len 16384 data: 3 4 4 4
7497 135/300 Write block 1539 len 40960 data: 5 5 5 4 5 4 3 4 5 5
7498 136/300 Read block 3035 len 36864
7499 137/300 Read block 1978 len 40960
7500 138/300 Write block 2595 len 36864 data: 3 4 4 4 4 3 3 3 3
7501 139/300 Read block 1714 len 24576
7502 140/300 Read block 418 len 8192
7503 141/300 Read block 1032 len 24576
7504 142/300 Write block 477 len 36864 data: 4 4 4 4 3 3 3 3 3
7505 143/300 Write block 667 len 20480 data: 4 4 3 3 3
7506 144/300 Read block 2541 len 4096
7507 145/300 Write block 630 len 28672 data: 2 2 2 2 2 2 2
7508 146/300 Read block 623 len 28672
7509 147/300 Write block 3868 len 32768 data: 4 4 3 2 2 2 3 4
7510 148/300 Read block 2070 len 28672
7511 149/300 Flush
7512 150/300 Read block 2133 len 4096
7513 151/300 Flush
7514 152/300 Write block 4129 len 12288 data: 3 3 3
7515 153/300 Flush
7516 154/300 Read block 4244 len 36864
7517 155/300 Read block 3318 len 20480
7518 156/300 Read block 3295 len 20480
7519 157/300 Read block 269 len 16384
7520 158/300 Write block 2201 len 12288 data: 2 2 2
7521 159/300 Write block 48 len 20480 data: 2 2 2 2 2
7522 160/300 Read block 2223 len 12288
7523 161/300 Read block 2404 len 28672
7524 162/300 Write block 1258 len 20480 data: 3 3 3 2 2
7525 163/300 Read block 89 len 8192
7526 164/300 Write block 1861 len 40960 data: 3 3 2 2 2 2 2 2 3 3
7527 165/300 Write block 1303 len 12288 data: 4 3 2
7528 166/300 Write block 788 len 32768 data: 5 4 4 3 2 3 3 3
7529 167/300 Write block 3412 len 24576 data: 3 3 3 3 3 2
7530 168/300 Read block 4825 len 4096
7531 169/300 Read block 2759 len 8192
7532 170/300 Read block 3929 len 32768
7533 171/300 Read block 2870 len 36864
7534 172/300 Write block 2170 len 32768 data: 2 2 2 2 2 2 2 3
7535 173/300 Write block 2076 len 36864 data: 3 3 4 4 4 4 3 2 2
7536 174/300 Read block 1907 len 36864
7537 175/300 Write block 1434 len 32768 data: 2 2 2 2 2 2 3 3
7538 176/300 Write block 727 len 36864 data: 2 2 2 2 2 2 2 2 3
7539 177/300 Write block 2192 len 12288 data: 3 3 2
7540 178/300 Read block 2838 len 40960
7541 179/300 Read block 2891 len 32768
7542 180/300 Write block 2525 len 8192 data: 3 2
7543 181/300 Flush
7544 182/300 Write block 2445 len 24576 data: 2 3 4 4 4 4
7545 183/300 Read block 4146 len 36864
7546 184/300 Write block 4184 len 28672 data: 2 2 2 2 2 2 3
7547 185/300 Read block 765 len 20480
7548 186/300 Read block 2285 len 20480
7549 187/300 Write block 3529 len 12288 data: 3 3 4
7550 188/300 Write block 4829 len 36864 data: 3 3 3 3 2 2 2 2 2
7551 189/300 Read block 2103 len 28672
7552 190/300 Write block 4767 len 32768 data: 2 2 3 3 3 3 3 3
7553 191/300 Write block 2400 len 28672 data: 2 2 2 2 2 2 2
7554 192/300 Write block 4237 len 12288 data: 3 2 2
7555 193/300 Read block 642 len 4096
7556 194/300 Read block 1296 len 24576
7557 195/300 Write block 1950 len 20480 data: 2 2 2 2 2
7558 196/300 Read block 3933 len 16384
7559 197/300 Read block 2152 len 16384
7560 198/300 Write block 3863 len 40960 data: 3 4 4 4 4 5 5 4 3 3
7561 199/300 Read block 2619 len 4096
7562 200/300 Write block 2822 len 12288 data: 3 3 3
7563 201/300 Read block 4412 len 4096
7564 202/300 Write block 3505 len 16384 data: 2 3 3 3
7565 203/300 Read block 1198 len 8192
7566 204/300 Read block 3724 len 4096
7567 205/300 Write block 1140 len 36864 data: 2 2 2 3 3 3 4 4 4
7568 206/300 Write block 570 len 36864 data: 2 2 2 2 2 2 2 3 3
7569 207/300 Read block 2357 len 12288
7570 208/300 Write block 3440 len 28672 data: 2 2 2 2 3 3 3
7571 209/300 Read block 514 len 4096
7572 210/300 Write block 353 len 32768 data: 2 2 2 2 2 2 3 3
7573 211/300 Write block 71 len 12288 data: 4 4 3
7574 212/300 Read block 650 len 28672
7575 213/300 Read block 4733 len 12288
7576 214/300 Write block 2209 len 4096 data: 2
7577 215/300 Write block 4331 len 40960 data: 4 4 3 2 2 2 2 2 4 4
7578 216/300 Flush
7579 217/300 Write block 4037 len 28672 data: 4 4 5 5 5 4 4
7580 218/300 Read block 2789 len 4096
7581 219/300 Write block 19 len 8192 data: 2 2
7582 220/300 Read block 4701 len 20480
7583 221/300 Read block 3131 len 28672
7584 222/300 Read block 1394 len 12288
7585 223/300 Flush
7586 224/300 Write block 4151 len 40960 data: 3 2 2 2 2 2 2 2 2 2
7587 225/300 Flush
7588 226/300 Read block 2316 len 24576
7589 227/300 Read block 135 len 4096
7590 228/300 Write block 1288 len 16384 data: 2 2 4 4
7591 229/300 Read block 649 len 20480
7592 230/300 Flush
7593 231/300 Read block 4361 len 4096
7594 232/300 Write block 1698 len 20480 data: 4 4 3 4 4
7595 233/300 Write block 228 len 20480 data: 4 6 6 6 6
7596 234/300 Write block 4124 len 36864 data: 3 3 3 2 2 4 4 4 3
7597 235/300 Write block 2324 len 4096 data: 2
7598 236/300 Read block 3630 len 24576
7599 237/300 Read block 1501 len 28672
7600 238/300 Write block 1192 len 32768 data: 2 2 3 3 5 5 5 5
7601 239/300 Write block 637 len 24576 data: 2 2 2 2 2 2
7602 240/300 Read block 4821 len 36864
7603 241/300 Read block 3154 len 20480
7604 242/300 Write block 79 len 16384 data: 3 3 3 3
7605 243/300 Write block 1594 len 8192 data: 4 3
7606 244/300 Read block 67 len 8192
7607 245/300 Read block 795 len 16384
7608 246/300 Read block 1919 len 4096
7609 247/300 Write block 3885 len 4096 data: 3
7610 248/300 Write block 2682 len 32768 data: 3 2 2 2 2 2 2 2
7611 249/300 Read block 1198 len 40960
7612 250/300 Read block 4118 len 32768
7613 251/300 Write block 1021 len 28672 data: 5 3 3 3 2 2 2
7614 252/300 Write block 710 len 36864 data: 4 4 4 2 2 2 2 2 2
7615 253/300 Read block 1116 len 32768
7616 254/300 Read block 4224 len 4096
7617 255/300 Write block 2693 len 32768 data: 2 2 2 3 3 3 3 3
7618 256/300 Read block 3797 len 12288
7619 257/300 Read block 1524 len 12288
7620 258/300 Read block 4057 len 28672
7621 259/300 Write block 4833 len 12288 data: 3 3 3
7622 260/300 Read block 4973 len 32768
7623 261/300 Read block 1656 len 12288
7624 262/300 Read block 3222 len 20480
7625 263/300 Write block 4338 len 8192 data: 3 5
7626 264/300 Write block 3667 len 4096 data: 2
7627 265/300 Read block 2770 len 32768
7628 266/300 Read block 1234 len 4096
7629 267/300 Write block 2697 len 4096 data: 4
7630 268/300 Write block 4706 len 4096 data: 3
7631 269/300 Read block 2964 len 40960
7632 270/300 Write block 479 len 16384 data: 5 5 4 4
7633 271/300 Flush
7634 272/300 Write block 2486 len 28672 data: 2 2 2 2 3 3 3
7635 273/300 Read block 4818 len 36864
7636 274/300 Flush
7637 275/300 Write block 3758 len 36864 data: 4 4 4 4 4 3 2 2 2
7638 276/300 Read block 1926 len 36864
7639 277/300 Read block 834 len 28672
7640 278/300 Write block 4890 len 36864 data: 5 4 4 5 5 3 2 2 2
7641 279/300 Read block 432 len 4096
7642 280/300 Write block 2383 len 32768 data: 2 2 2 2 2 2 3 2
7643 281/300 Read block 3733 len 28672
7644 282/300 Read block 1924 len 28672
7645 283/300 Read block 4673 len 36864
7646 284/300 Read block 1617 len 28672
7647 285/300 Read block 3007 len 24576
7648 286/300 Flush
7649 287/300 Read block 3184 len 40960
7650 288/300 Read block 499 len 20480
7651 289/300 Read block 926 len 4096
7652 290/300 Read block 543 len 32768
7653 291/300 Write block 2573 len 32768 data: 3 3 3 5 3 3 2 2
7654 292/300 Write block 851 len 36864 data: 3 3 3 3 3 3 3 3 2
7655 293/300 Write block 651 len 28672 data: 2 2 2 2 2 2 2
7656 294/300 Write block 4918 len 36864 data: 3 2 3 4 4 6 6 5 5
7657 295/300 Read block 2439 len 12288
7658 296/300 Flush
7659 297/300 Read block 3 len 20480
7660 298/300 Read block 2206 len 20480
7661 299/300 Flush
7662 300/300 Read block 707 len 4096
7663 301/300 Write block 4528 len 36864 data: 6 6 6 4 4 4 2 2 2
7664 Main task has work to do, go find it
7665 got dsc Start(0)
7666 start 0
7667 Nov 15 01:59:05.575 INFO request completed, latency_us: 208, response_code: 204, uri: /start/cid/0, method: POST, req_id: 38533f31-8560-4362-87a9-77ddf6e81e9f, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
7668 Replay: started 0, returned:Ok(())
7669 ----------------------------------------------------------------
7670 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
7671 States: Offline Active Active
7672 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
7673 2810 Acked Read 5 Sent Done Done false
7674 2811 Acked Read 2 Sent Done Done false
7675 2812 Acked Read 9 Sent Done Done false
7676 2813 Acked Read 6 Sent Done Done false
7677 2814 Acked Read 3 Sent Done Done false
7678 2815 Acked Read 10 Sent Done Done false
7679 2816 Acked Read 2 Sent Done Done false
7680 2817 Acked Read 10 Sent Done Done false
7681 2818 Acked Read 9 Sent Done Done false
7682 2819 Acked Write 9 Sent Done Done false
7683 2820 Acked Flush 0 Sent Done Done false
7684 2821 Acked Write 1 Sent Done Done false
7685 2822 Acked Write 5 Sent Done Done false
7686 2823 Acked Flush 0 Sent Done Done false
7687 2824 Acked Write 3 Sent Done Done false
7688 2825 Acked Read 4 Sent Done Done false
7689 2826 Acked Read 3 Sent Done Done false
7690 2827 Acked Read 1 Sent Done Done false
7691 2828 Acked Write 9 Sent Done Done false
7692 2829 Acked Read 1 Sent Done Done false
7693 2830 Acked Read 10 Sent Done Done false
7694 2831 Acked Read 8 Sent Done Done false
7695 2832 Acked Write 5 Sent Done Done false
7696 2833 Acked Write 4 Sent Done Done false
7697 2834 Acked Write 1 Sent Done Done false
7698 2835 Acked Write 5 Sent Done Done false
7699 2836 Acked Read 9 Sent Done Done false
7700 2837 Acked Write 4 Sent Done Done false
7701 2838 Acked Read 6 Sent Done Done false
7702 2839 Acked Write 4 Sent Done Done false
7703 2840 Acked Write 2 Sent Done Done false
7704 2841 Acked Write 9 Sent Done Done false
7705 2842 Acked Write 9 Sent Done Done false
7706 2843 Acked Read 7 Sent Done Done false
7707 2844 Acked Read 2 Sent Done Done false
7708 2845 Acked Read 1 Sent Done Done false
7709 2846 Acked Flush 0 Sent Done Done false
7710 2847 Acked Read 4 Sent Done Done false
7711 2848 Acked Read 6 Sent Done Done false
7712 2849 Acked Read 10 Sent Done Done false
7713 2850 Acked Read 10 Sent Done Done false
7714 2851 Acked Write 5 Sent Done Done false
7715 2852 Acked Read 7 Sent Done Done false
7716 2853 Acked Write 9 Sent Done Done false
7717 2854 Acked Read 9 Sent Done Done false
7718 2855 Acked Read 10 Sent Done Done false
7719 2856 Acked Read 1 Sent Done Done false
7720 2857 Acked Read 3 Sent Done Done false
7721 2858 Acked Read 9 Sent Done Done false
7722 2859 Acked Write 8 Sent Done Done false
7723 2860 Acked Read 2 Sent Done Done false
7724 2861 Acked Read 9 Sent Done Done false
7725 2862 Acked Write 5 Sent Done Done false
7726 2863 Acked Read 4 Sent Done Done false
7727 2864 Acked Read 7 Sent Done Done false
7728 2865 Acked Write 1 Sent Done Done false
7729 2866 Acked Read 9 Sent Done Done false
7730 2867 Acked Write 3 Sent Done Done false
7731 2868 Acked Read 3 Sent Done Done false
7732 2869 Acked Flush 0 Sent Done Done false
7733 2870 Acked Read 1 Sent Done Done false
7734 2871 Acked Read 9 Sent Done Done false
7735 2872 Acked Read 9 Sent Done Done false
7736 2873 Acked Write 4 Sent Done Done false
7737 2874 Acked Write 10 Sent Done Done false
7738 2875 Acked Read 9 Sent Done Done false
7739 2876 Acked Flush 0 Sent Done Done false
7740 2877 Acked Flush 0 Sent Done Done false
7741 2878 Acked Read 6 Sent Done Done false
7742 2879 Acked Write 1 Sent Done Done false
7743 2880 Acked Read 6 Sent Done Done false
7744 2881 Acked Write 5 Sent Done Done false
7745 2882 Acked Read 4 Sent Done Done false
7746 2883 Acked Read 2 Sent Done Done false
7747 2884 Acked Write 9 Sent Done Done false
7748 2885 Acked Write 7 Sent Done Done false
7749 2886 Acked Flush 0 Sent Done Done false
7750 2887 Acked Write 7 Sent Done Done false
7751 2888 Acked Flush 0 Sent Done Done false
7752 2889 Acked Flush 0 Sent Done Done false
7753 2890 Acked Read 8 Sent Done Done false
7754 2891 Acked Write 4 Sent Done Done false
7755 2892 Acked Write 7 Sent Done Done false
7756 2893 Acked Read 2 Sent Done Done false
7757 2894 Acked Write 1 Sent Done Done false
7758 2895 Acked Read 1 Sent Done Done false
7759 2896 Acked Read 5 Sent Done Done false
7760 2897 Acked Read 6 Sent Done Done false
7761 2898 Acked Read 10 Sent Done Done false
7762 2899 Acked Write 5 Sent Done Done false
7763 2900 Acked Read 3 Sent Done Done false
7764 2901 Acked Read 8 Sent Done Done false
7765 2902 Acked Write 8 Sent Done Done false
7766 2903 Acked Write 9 Sent Done Done false
7767 2904 Acked Write 7 Sent Done Done false
7768 2905 Acked Write 5 Sent Done Done false
7769 2906 Acked Write 5 Sent Done Done false
7770 2907 Acked Write 8 Sent Done Done false
7771 2908 Acked Read 3 Sent Done Done false
7772 2909 Acked Read 1 Sent Done Done false
7773 2910 Acked Read 7 Sent Done Done false
7774 2911 Acked Read 9 Sent Done Done false
7775 2912 Acked Read 2 Sent Done Done false
7776 2913 Acked Read 7 Sent Done Done false
7777 2914 Acked Write 2 Sent Done Done false
7778 2915 Acked Read 2 Sent Done Done false
7779 2916 Acked Write 9 Sent Done Done false
7780 2917 Acked Flush 0 Sent Done Done false
7781 2918 Acked Read 6 Sent Done Done false
7782 2919 Acked Write 4 Sent Done Done false
7783 2920 Acked Read 10 Sent Done Done false
7784 2921 Acked Read 9 Sent Done Done false
7785 2922 Acked Read 10 Sent Done Done false
7786 2923 Acked Flush 0 Sent Done Done false
7787 2924 Acked Write 5 Sent Done Done false
7788 2925 Acked Write 7 Sent Done Done false
7789 2926 Acked Write 1 Sent Done Done false
7790 2927 Acked Read 6 Sent Done Done false
7791 2928 Acked Read 2 Sent Done Done false
7792 2929 Acked Write 3 Sent Done Done false
7793 2930 Acked Write 1 Sent Done Done false
7794 2931 Acked Write 1 Sent Done Done false
7795 2932 Acked Read 8 Sent Done Done false
7796 2933 Acked Write 2 Sent Done Done false
7797 2934 Acked Read 7 Sent Done Done false
7798 2935 Acked Write 9 Sent Done Done false
7799 2936 Acked Read 8 Sent Done Done false
7800 2937 Acked Write 8 Sent Done Done false
7801 2938 Acked Flush 0 Sent Done Done false
7802 2939 Acked Read 7 Sent Done Done false
7803 2940 Acked Write 9 Sent Done Done false
7804 2941 Acked Flush 0 Sent Done Done false
7805 2942 Acked Write 9 Sent Done Done false
7806 2943 Acked Write 4 Sent Done Done false
7807 2944 Acked Write 10 Sent Done Done false
7808 2945 Acked Read 9 Sent Done Done false
7809 2946 Acked Read 10 Sent Done Done false
7810 2947 Acked Write 9 Sent Done Done false
7811 2948 Acked Read 6 Sent Done Done false
7812 2949 Acked Read 2 Sent Done Done false
7813 2950 Acked Read 6 Sent Done Done false
7814 2951 Acked Write 9 Sent Done Done false
7815 2952 Acked Write 5 Sent Done Done false
7816 2953 Acked Read 1 Sent Done Done false
7817 2954 Acked Write 7 Sent Done Done false
7818 2955 Acked Read 7 Sent Done Done false
7819 2956 Acked Write 8 Sent Done Done false
7820 2957 Acked Read 7 Sent Done Done false
7821 2958 Acked Flush 0 Sent Done Done false
7822 2959 Acked Read 1 Sent Done Done false
7823 2960 Acked Flush 0 Sent Done Done false
7824 2961 Acked Write 3 Sent Done Done false
7825 2962 Acked Flush 0 Sent Done Done false
7826 2963 Acked Read 9 Sent Done Done false
7827 2964 Acked Read 5 Sent Done Done false
7828 2965 Acked Read 5 Sent Done Done false
7829 2966 Acked Read 4 Sent Done Done false
7830 2967 Acked Write 3 Sent Done Done false
7831 2968 Acked Write 5 Sent Done Done false
7832 2969 Acked Read 3 Sent Done Done false
7833 2970 Acked Read 7 Sent Done Done false
7834 2971 Acked Write 5 Sent Done Done false
7835 2972 Acked Read 2 Sent Done Done false
7836 2973 Acked Write 10 Sent Done Done false
7837 2974 Acked Write 3 Sent Done Done false
7838 2975 Acked Write 8 Sent Done Done false
7839 2976 Acked Write 6 Sent Done Done false
7840 2977 Acked Read 1 Sent Done Done false
7841 2978 Acked Read 2 Sent Done Done false
7842 2979 Acked Read 8 Sent Done Done false
7843 2980 Acked Read 9 Sent Done Done false
7844 2981 Acked Write 8 Sent Done Done false
7845 2982 Acked Write 9 Sent Done Done false
7846 2983 Acked Read 9 Sent Done Done false
7847 2984 Acked Write 8 Sent Done Done false
7848 2985 Acked Write 9 Sent Done Done false
7849 2986 Acked Write 3 Sent Done Done false
7850 2987 Acked Read 10 Sent Done Done false
7851 2988 Acked Read 8 Sent Done Done false
7852 2989 Acked Write 2 Sent Done Done false
7853 2990 Acked Flush 0 Sent Done Done false
7854 2991 Acked Write 6 Sent Done Done false
7855 2992 Acked Read 9 Sent Done Done false
7856 2993 Acked Write 7 Sent Done Done false
7857 2994 Acked Read 5 Sent Done Done false
7858 2995 Acked Read 5 Sent Done Done false
7859 2996 Acked Write 3 Sent Done Done false
7860 2997 Acked Write 9 Sent Done Done false
7861 2998 Acked Read 7 Sent Done Done false
7862 2999 Acked Write 8 Sent Done Done false
7863 3000 Acked Write 7 Sent Done Done false
7864 3001 Acked Write 3 Sent Done Done false
7865 3002 Acked Read 1 Sent Done Done false
7866 3003 Acked Read 6 Sent Done Done false
7867 3004 Acked Write 5 Sent Done Done false
7868 3005 Acked Read 4 Sent Done Done false
7869 3006 Acked Read 4 Sent Done Done false
7870 3007 Acked Write 10 Sent Done Done false
7871 3008 Acked Read 1 Sent Done Done false
7872 3009 Acked Write 3 Sent Done Done false
7873 3010 Acked Read 1 Sent Done Done false
7874 3011 Acked Write 4 Sent Done Done false
7875 3012 Acked Read 2 Sent Done Done false
7876 3013 Acked Read 1 Sent Done Done false
7877 3014 Acked Write 9 Sent Done Done false
7878 3015 Acked Write 9 Sent Done Done false
7879 3016 Acked Read 3 Sent Done Done false
7880 3017 Acked Write 7 Sent Done Done false
7881 3018 Acked Read 1 Sent Done Done false
7882 3019 Acked Write 8 Sent Done Done false
7883 3020 Acked Write 3 Sent Done Done false
7884 3021 Acked Read 7 Sent Done Done false
7885 3022 Acked Read 3 Sent Done Done false
7886 3023 Acked Write 1 Sent Done Done false
7887 3024 Acked Write 10 Sent Done Done false
7888 3025 Acked Flush 0 Sent Done Done false
7889 3026 Acked Write 7 Sent Done Done false
7890 3027 Acked Read 1 Sent Done Done false
7891 3028 Acked Write 2 Sent Done Done false
7892 3029 Acked Read 5 Sent Done Done false
7893 3030 Acked Read 7 Sent Done Done false
7894 3031 Acked Read 3 Sent Done Done false
7895 3032 Acked Flush 0 Sent Done Done false
7896 3033 Acked Write 10 Sent Done Done false
7897 3034 Acked Flush 0 Sent Done Done false
7898 3035 Acked Read 6 Sent Done Done false
7899 3036 Acked Read 1 Sent Done Done false
7900 3037 Acked Write 4 Sent Done Done false
7901 3038 Acked Read 5 Sent Done Done false
7902 3039 Acked Flush 0 Sent Done Done false
7903 3040 Acked Read 1 Sent Done Done false
7904 3041 Acked Write 5 Sent Done Done false
7905 3042 Acked Write 5 Sent Done Done false
7906 3043 Acked Write 9 Sent Done Done false
7907 3044 Acked Write 1 Sent Done Done false
7908 3045 Acked Read 6 Sent Done Done false
7909 3046 Acked Read 7 Sent Done Done false
7910 3047 Acked Write 8 Sent Done Done false
7911 3048 Acked Write 6 Sent Done Done false
7912 3049 Acked Read 9 Sent Done Done false
7913 3050 Acked Read 5 Sent Done Done false
7914 3051 Acked Write 4 Sent Done Done false
7915 3052 Acked Write 2 Sent Done Done false
7916 3053 Acked Read 2 Sent Done Done false
7917 3054 Acked Read 4 Sent Done Done false
7918 3055 Acked Read 1 Sent Done Done false
7919 3056 Acked Write 1 Sent Done Done false
7920 3057 Acked Write 8 Sent Done Done false
7921 3058 Acked Read 10 Sent Done Done false
7922 3059 Acked Read 8 Sent Done Done false
7923 3060 Acked Write 7 Sent Done Done false
7924 3061 Acked Write 9 Sent Done Done false
7925 3062 Acked Read 8 Sent Done Done false
7926 3063 Acked Read 1 Sent Done Done false
7927 3064 Acked Write 8 Sent Done Done false
7928 3065 Acked Read 3 Sent Done Done false
7929 3066 Acked Read 3 Sent Done Done false
7930 3067 Acked Read 7 Sent Done Done false
7931 3068 Acked Write 3 Sent Done Done false
7932 3069 Acked Read 8 Sent Done Done false
7933 3070 Acked Read 3 Sent Done Done false
7934 3071 Acked Read 5 Sent Done Done false
7935 3072 Acked Write 2 Sent Done Done false
7936 3073 Acked Write 1 Sent Done Done false
7937 3074 Acked Read 8 Sent Done Done false
7938 3075 Acked Read 1 Sent Done Done false
7939 3076 Acked Write 1 Sent Done Done false
7940 3077 Acked Write 1 Sent Done Done false
7941 3078 Acked Read 10 Sent Done Done false
7942 3079 Acked Write 4 Sent Done Done false
7943 3080 Acked Flush 0 Sent Done Done false
7944 3081 Acked Write 7 Sent Done Done false
7945 3082 Acked Read 9 Sent Done Done false
7946 3083 Acked Flush 0 Sent Done Done false
7947 3084 Acked Write 9 Sent Done Done false
7948 3085 Acked Read 9 Sent Done Done false
7949 3086 Acked Read 7 Sent Done Done false
7950 3087 Acked Write 9 Sent Done Done false
7951 3088 Acked Read 1 Sent Done Done false
7952 3089 Acked Write 8 Sent Done Done false
7953 3090 Acked Read 7 Sent Done Done false
7954 3091 Acked Read 7 Sent Done Done false
7955 3092 Acked Read 9 Sent Done Done false
7956 3093 Acked Read 7 Sent Done Done false
7957 3094 Acked Read 6 Sent Done Done false
7958 3095 Acked Flush 0 Sent Done Done false
7959 3096 Acked Read 10 Sent Done Done false
7960 3097 Acked Read 5 Sent Done Done false
7961 3098 Acked Read 1 Sent Done Done false
7962 3099 Acked Read 8 Sent Done Done false
7963 3100 Acked Write 8 Sent Done Done false
7964 3101 Acked Write 9 Sent Done Done false
7965 3102 Acked Write 7 Sent Done Done false
7966 3103 Acked Write 9 Sent Done Done false
7967 3104 Acked Read 3 Sent Done Done false
7968 3105 Acked Flush 0 Sent Done Done false
7969 3106 Acked Read 5 Sent Done Done false
7970 3107 Acked Read 5 Sent Done Done false
7971 3108 Acked Flush 0 Sent Done Done false
7972 3109 Acked Read 1 Sent Done Done false
7973 3110 Acked Write 9 Sent Done Done false
7974 STATES DS:0 DS:1 DS:2 TOTAL
7975 Sent 301 0 0 301
7976 Done 0 301 301 602
7977 Skipped 0 0 0 0
7978 Error 0 0 0 0
7979 Last Flush: 2809 3108 3108
7980 Downstairs last five completed: 2809 2808 2807 2806 2805
7981 Upstairs last five completed: 3110 3109 3108 3107 3106
7982 CLIENT: Up:0 ds:301 act:2
7983 ----------------------------------------------------------------
7984 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
7985 States: Offline Active Active
7986 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
7987 2810 Acked Read 5 Sent Done Done false
7988 2811 Acked Read 2 Sent Done Done false
7989 2812 Acked Read 9 Sent Done Done false
7990 2813 Acked Read 6 Sent Done Done false
7991 2814 Acked Read 3 Sent Done Done false
7992 2815 Acked Read 10 Sent Done Done false
7993 2816 Acked Read 2 Sent Done Done false
7994 2817 Acked Read 10 Sent Done Done false
7995 2818 Acked Read 9 Sent Done Done false
7996 2819 Acked Write 9 Sent Done Done false
7997 2820 Acked Flush 0 Sent Done Done false
7998 2821 Acked Write 1 Sent Done Done false
7999 2822 Acked Write 5 Sent Done Done false
8000 2823 Acked Flush 0 Sent Done Done false
8001 2824 Acked Write 3 Sent Done Done false
8002 2825 Acked Read 4 Sent Done Done false
8003 2826 Acked Read 3 Sent Done Done false
8004 2827 Acked Read 1 Sent Done Done false
8005 2828 Acked Write 9 Sent Done Done false
8006 2829 Acked Read 1 Sent Done Done false
8007 2830 Acked Read 10 Sent Done Done false
8008 2831 Acked Read 8 Sent Done Done false
8009 2832 Acked Write 5 Sent Done Done false
8010 2833 Acked Write 4 Sent Done Done false
8011 2834 Acked Write 1 Sent Done Done false
8012 2835 Acked Write 5 Sent Done Done false
8013 2836 Acked Read 9 Sent Done Done false
8014 2837 Acked Write 4 Sent Done Done false
8015 2838 Acked Read 6 Sent Done Done false
8016 2839 Acked Write 4 Sent Done Done false
8017 2840 Acked Write 2 Sent Done Done false
8018 2841 Acked Write 9 Sent Done Done false
8019 2842 Acked Write 9 Sent Done Done false
8020 2843 Acked Read 7 Sent Done Done false
8021 2844 Acked Read 2 Sent Done Done false
8022 2845 Acked Read 1 Sent Done Done false
8023 2846 Acked Flush 0 Sent Done Done false
8024 2847 Acked Read 4 Sent Done Done false
8025 2848 Acked Read 6 Sent Done Done false
8026 2849 Acked Read 10 Sent Done Done false
8027 2850 Acked Read 10 Sent Done Done false
8028 2851 Acked Write 5 Sent Done Done false
8029 2852 Acked Read 7 Sent Done Done false
8030 2853 Acked Write 9 Sent Done Done false
8031 2854 Acked Read 9 Sent Done Done false
8032 2855 Acked Read 10 Sent Done Done false
8033 2856 Acked Read 1 Sent Done Done false
8034 2857 Acked Read 3 Sent Done Done false
8035 2858 Acked Read 9 Sent Done Done false
8036 2859 Acked Write 8 Sent Done Done false
8037 2860 Acked Read 2 Sent Done Done false
8038 2861 Acked Read 9 Sent Done Done false
8039 2862 Acked Write 5 Sent Done Done false
8040 2863 Acked Read 4 Sent Done Done false
8041 2864 Acked Read 7 Sent Done Done false
8042 2865 Acked Write 1 Sent Done Done false
8043 2866 Acked Read 9 Sent Done Done false
8044 2867 Acked Write 3 Sent Done Done false
8045 2868 Acked Read 3 Sent Done Done false
8046 2869 Acked Flush 0 Sent Done Done false
8047 2870 Acked Read 1 Sent Done Done false
8048 2871 Acked Read 9 Sent Done Done false
8049 2872 Acked Read 9 Sent Done Done false
8050 2873 Acked Write 4 Sent Done Done false
8051 2874 Acked Write 10 Sent Done Done false
8052 2875 Acked Read 9 Sent Done Done false
8053 2876 Acked Flush 0 Sent Done Done false
8054 2877 Acked Flush 0 Sent Done Done false
8055 2878 Acked Read 6 Sent Done Done false
8056 2879 Acked Write 1 Sent Done Done false
8057 2880 Acked Read 6 Sent Done Done false
8058 2881 Acked Write 5 Sent Done Done false
8059 2882 Acked Read 4 Sent Done Done false
8060 2883 Acked Read 2 Sent Done Done false
8061 2884 Acked Write 9 Sent Done Done false
8062 2885 Acked Write 7 Sent Done Done false
8063 2886 Acked Flush 0 Sent Done Done false
8064 2887 Acked Write 7 Sent Done Done false
8065 2888 Acked Flush 0 Sent Done Done false
8066 2889 Acked Flush 0 Sent Done Done false
8067 2890 Acked Read 8 Sent Done Done false
8068 2891 Acked Write 4 Sent Done Done false
8069 2892 Acked Write 7 Sent Done Done false
8070 2893 Acked Read 2 Sent Done Done false
8071 2894 Acked Write 1 Sent Done Done false
8072 2895 Acked Read 1 Sent Done Done false
8073 2896 Acked Read 5 Sent Done Done false
8074 2897 Acked Read 6 Sent Done Done false
8075 2898 Acked Read 10 Sent Done Done false
8076 2899 Acked Write 5 Sent Done Done false
8077 2900 Acked Read 3 Sent Done Done false
8078 2901 Acked Read 8 Sent Done Done false
8079 2902 Acked Write 8 Sent Done Done false
8080 2903 Acked Write 9 Sent Done Done false
8081 2904 Acked Write 7 Sent Done Done false
8082 2905 Acked Write 5 Sent Done Done false
8083 2906 Acked Write 5 Sent Done Done false
8084 2907 Acked Write 8 Sent Done Done false
8085 2908 Acked Read 3 Sent Done Done false
8086 2909 Acked Read 1 Sent Done Done false
8087 2910 Acked Read 7 Sent Done Done false
8088 2911 Acked Read 9 Sent Done Done false
8089 2912 Acked Read 2 Sent Done Done false
8090 2913 Acked Read 7 Sent Done Done false
8091 2914 Acked Write 2 Sent Done Done false
8092 2915 Acked Read 2 Sent Done Done false
8093 2916 Acked Write 9 Sent Done Done false
8094 2917 Acked Flush 0 Sent Done Done false
8095 2918 Acked Read 6 Sent Done Done false
8096 2919 Acked Write 4 Sent Done Done false
8097 2920 Acked Read 10 Sent Done Done false
8098 2921 Acked Read 9 Sent Done Done false
8099 2922 Acked Read 10 Sent Done Done false
8100 2923 Acked Flush 0 Sent Done Done false
8101 2924 Acked Write 5 Sent Done Done false
8102 2925 Acked Write 7 Sent Done Done false
8103 2926 Acked Write 1 Sent Done Done false
8104 2927 Acked Read 6 Sent Done Done false
8105 2928 Acked Read 2 Sent Done Done false
8106 2929 Acked Write 3 Sent Done Done false
8107 2930 Acked Write 1 Sent Done Done false
8108 2931 Acked Write 1 Sent Done Done false
8109 2932 Acked Read 8 Sent Done Done false
8110 2933 Acked Write 2 Sent Done Done false
8111 2934 Acked Read 7 Sent Done Done false
8112 2935 Acked Write 9 Sent Done Done false
8113 2936 Acked Read 8 Sent Done Done false
8114 2937 Acked Write 8 Sent Done Done false
8115 2938 Acked Flush 0 Sent Done Done false
8116 2939 Acked Read 7 Sent Done Done false
8117 2940 Acked Write 9 Sent Done Done false
8118 2941 Acked Flush 0 Sent Done Done false
8119 2942 Acked Write 9 Sent Done Done false
8120 2943 Acked Write 4 Sent Done Done false
8121 2944 Acked Write 10 Sent Done Done false
8122 2945 Acked Read 9 Sent Done Done false
8123 2946 Acked Read 10 Sent Done Done false
8124 2947 Acked Write 9 Sent Done Done false
8125 2948 Acked Read 6 Sent Done Done false
8126 2949 Acked Read 2 Sent Done Done false
8127 2950 Acked Read 6 Sent Done Done false
8128 2951 Acked Write 9 Sent Done Done false
8129 2952 Acked Write 5 Sent Done Done false
8130 2953 Acked Read 1 Sent Done Done false
8131 2954 Acked Write 7 Sent Done Done false
8132 2955 Acked Read 7 Sent Done Done false
8133 2956 Acked Write 8 Sent Done Done false
8134 2957 Acked Read 7 Sent Done Done false
8135 2958 Acked Flush 0 Sent Done Done false
8136 2959 Acked Read 1 Sent Done Done false
8137 2960 Acked Flush 0 Sent Done Done false
8138 2961 Acked Write 3 Sent Done Done false
8139 2962 Acked Flush 0 Sent Done Done false
8140 2963 Acked Read 9 Sent Done Done false
8141 2964 Acked Read 5 Sent Done Done false
8142 2965 Acked Read 5 Sent Done Done false
8143 2966 Acked Read 4 Sent Done Done false
8144 2967 Acked Write 3 Sent Done Done false
8145 2968 Acked Write 5 Sent Done Done false
8146 2969 Acked Read 3 Sent Done Done false
8147 2970 Acked Read 7 Sent Done Done false
8148 2971 Acked Write 5 Sent Done Done false
8149 2972 Acked Read 2 Sent Done Done false
8150 2973 Acked Write 10 Sent Done Done false
8151 2974 Acked Write 3 Sent Done Done false
8152 2975 Acked Write 8 Sent Done Done false
8153 2976 Acked Write 6 Sent Done Done false
8154 2977 Acked Read 1 Sent Done Done false
8155 2978 Acked Read 2 Sent Done Done false
8156 2979 Acked Read 8 Sent Done Done false
8157 2980 Acked Read 9 Sent Done Done false
8158 2981 Acked Write 8 Sent Done Done false
8159 2982 Acked Write 9 Sent Done Done false
8160 2983 Acked Read 9 Sent Done Done false
8161 2984 Acked Write 8 Sent Done Done false
8162 2985 Acked Write 9 Sent Done Done false
8163 2986 Acked Write 3 Sent Done Done false
8164 2987 Acked Read 10 Sent Done Done false
8165 2988 Acked Read 8 Sent Done Done false
8166 2989 Acked Write 2 Sent Done Done false
8167 2990 Acked Flush 0 Sent Done Done false
8168 2991 Acked Write 6 Sent Done Done false
8169 2992 Acked Read 9 Sent Done Done false
8170 2993 Acked Write 7 Sent Done Done false
8171 2994 Acked Read 5 Sent Done Done false
8172 2995 Acked Read 5 Sent Done Done false
8173 2996 Acked Write 3 Sent Done Done false
8174 2997 Acked Write 9 Sent Done Done false
8175 2998 Acked Read 7 Sent Done Done false
8176 2999 Acked Write 8 Sent Done Done false
8177 3000 Acked Write 7 Sent Done Done false
8178 3001 Acked Write 3 Sent Done Done false
8179 3002 Acked Read 1 Sent Done Done false
8180 3003 Acked Read 6 Sent Done Done false
8181 3004 Acked Write 5 Sent Done Done false
8182 3005 Acked Read 4 Sent Done Done false
8183 3006 Acked Read 4 Sent Done Done false
8184 3007 Acked Write 10 Sent Done Done false
8185 3008 Acked Read 1 Sent Done Done false
8186 3009 Acked Write 3 Sent Done Done false
8187 3010 Acked Read 1 Sent Done Done false
8188 3011 Acked Write 4 Sent Done Done false
8189 3012 Acked Read 2 Sent Done Done false
8190 3013 Acked Read 1 Sent Done Done false
8191 3014 Acked Write 9 Sent Done Done false
8192 3015 Acked Write 9 Sent Done Done false
8193 3016 Acked Read 3 Sent Done Done false
8194 3017 Acked Write 7 Sent Done Done false
8195 3018 Acked Read 1 Sent Done Done false
8196 3019 Acked Write 8 Sent Done Done false
8197 3020 Acked Write 3 Sent Done Done false
8198 3021 Acked Read 7 Sent Done Done false
8199 3022 Acked Read 3 Sent Done Done false
8200 3023 Acked Write 1 Sent Done Done false
8201 3024 Acked Write 10 Sent Done Done false
8202 3025 Acked Flush 0 Sent Done Done false
8203 3026 Acked Write 7 Sent Done Done false
8204 3027 Acked Read 1 Sent Done Done false
8205 3028 Acked Write 2 Sent Done Done false
8206 3029 Acked Read 5 Sent Done Done false
8207 3030 Acked Read 7 Sent Done Done false
8208 3031 Acked Read 3 Sent Done Done false
8209 3032 Acked Flush 0 Sent Done Done false
8210 3033 Acked Write 10 Sent Done Done false
8211 3034 Acked Flush 0 Sent Done Done false
8212 3035 Acked Read 6 Sent Done Done false
8213 3036 Acked Read 1 Sent Done Done false
8214 3037 Acked Write 4 Sent Done Done false
8215 3038 Acked Read 5 Sent Done Done false
8216 3039 Acked Flush 0 Sent Done Done false
8217 3040 Acked Read 1 Sent Done Done false
8218 3041 Acked Write 5 Sent Done Done false
8219 3042 Acked Write 5 Sent Done Done false
8220 3043 Acked Write 9 Sent Done Done false
8221 3044 Acked Write 1 Sent Done Done false
8222 3045 Acked Read 6 Sent Done Done false
8223 3046 Acked Read 7 Sent Done Done false
8224 3047 Acked Write 8 Sent Done Done false
8225 3048 Acked Write 6 Sent Done Done false
8226 3049 Acked Read 9 Sent Done Done false
8227 3050 Acked Read 5 Sent Done Done false
8228 3051 Acked Write 4 Sent Done Done false
8229 3052 Acked Write 2 Sent Done Done false
8230 3053 Acked Read 2 Sent Done Done false
8231 3054 Acked Read 4 Sent Done Done false
8232 3055 Acked Read 1 Sent Done Done false
8233 3056 Acked Write 1 Sent Done Done false
8234 3057 Acked Write 8 Sent Done Done false
8235 3058 Acked Read 10 Sent Done Done false
8236 3059 Acked Read 8 Sent Done Done false
8237 3060 Acked Write 7 Sent Done Done false
8238 3061 Acked Write 9 Sent Done Done false
8239 3062 Acked Read 8 Sent Done Done false
8240 3063 Acked Read 1 Sent Done Done false
8241 3064 Acked Write 8 Sent Done Done false
8242 3065 Acked Read 3 Sent Done Done false
8243 3066 Acked Read 3 Sent Done Done false
8244 3067 Acked Read 7 Sent Done Done false
8245 3068 Acked Write 3 Sent Done Done false
8246 3069 Acked Read 8 Sent Done Done false
8247 3070 Acked Read 3 Sent Done Done false
8248 3071 Acked Read 5 Sent Done Done false
8249 3072 Acked Write 2 Sent Done Done false
8250 3073 Acked Write 1 Sent Done Done false
8251 3074 Acked Read 8 Sent Done Done false
8252 3075 Acked Read 1 Sent Done Done false
8253 3076 Acked Write 1 Sent Done Done false
8254 3077 Acked Write 1 Sent Done Done false
8255 3078 Acked Read 10 Sent Done Done false
8256 3079 Acked Write 4 Sent Done Done false
8257 3080 Acked Flush 0 Sent Done Done false
8258 3081 Acked Write 7 Sent Done Done false
8259 3082 Acked Read 9 Sent Done Done false
8260 3083 Acked Flush 0 Sent Done Done false
8261 3084 Acked Write 9 Sent Done Done false
8262 3085 Acked Read 9 Sent Done Done false
8263 3086 Acked Read 7 Sent Done Done false
8264 3087 Acked Write 9 Sent Done Done false
8265 3088 Acked Read 1 Sent Done Done false
8266 3089 Acked Write 8 Sent Done Done false
8267 3090 Acked Read 7 Sent Done Done false
8268 3091 Acked Read 7 Sent Done Done false
8269 3092 Acked Read 9 Sent Done Done false
8270 3093 Acked Read 7 Sent Done Done false
8271 3094 Acked Read 6 Sent Done Done false
8272 3095 Acked Flush 0 Sent Done Done false
8273 3096 Acked Read 10 Sent Done Done false
8274 3097 Acked Read 5 Sent Done Done false
8275 3098 Acked Read 1 Sent Done Done false
8276 3099 Acked Read 8 Sent Done Done false
8277 3100 Acked Write 8 Sent Done Done false
8278 3101 Acked Write 9 Sent Done Done false
8279 3102 Acked Write 7 Sent Done Done false
8280 3103 Acked Write 9 Sent Done Done false
8281 3104 Acked Read 3 Sent Done Done false
8282 3105 Acked Flush 0 Sent Done Done false
8283 3106 Acked Read 5 Sent Done Done false
8284 3107 Acked Read 5 Sent Done Done false
8285 3108 Acked Flush 0 Sent Done Done false
8286 3109 Acked Read 1 Sent Done Done false
8287 3110 Acked Write 9 Sent Done Done false
8288 3111 Acked Flush 0 Sent Done Done false
8289 STATES DS:0 DS:1 DS:2 TOTAL
8290 Sent 302 0 0 302
8291 Done 0 302 302 604
8292 Skipped 0 0 0 0
8293 Error 0 0 0 0
8294 Last Flush: 2809 3111 3111
8295 Downstairs last five completed: 2809 2808 2807 2806 2805
8296 Upstairs last five completed: 3111 3110 3109 3108 3107
8297 CLIENT: Up:0 ds:302 act:2
82982024-11-15T01:59:11.334ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
82992024-11-15T01:59:11.335ZWARNcrucible: ds_connection connect to 127.0.0.1:8810 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
83002024-11-15T01:59:11.335ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
83012024-11-15T01:59:11.335ZWARNcrucible: downstairs task for 0 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
83022024-11-15T01:59:11.335ZINFOcrucible: client task is exiting = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
8303 [8810] got start action
8304 [8810] I am going to restart
8305 Starting downstairs at port 8810
8306 Make output file at "/tmp/dsc/downstairs-8810.txt"
8307 [8810][0] reports Starting
8308 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1212)
8309 [8810][0] reports Running
8310 ----------------------------------------------------------------
8311 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
8312 States: Offline Active Active
8313 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
8314 2810 Acked Read 5 Sent Done Done false
8315 2811 Acked Read 2 Sent Done Done false
8316 2812 Acked Read 9 Sent Done Done false
8317 2813 Acked Read 6 Sent Done Done false
8318 2814 Acked Read 3 Sent Done Done false
8319 2815 Acked Read 10 Sent Done Done false
8320 2816 Acked Read 2 Sent Done Done false
8321 2817 Acked Read 10 Sent Done Done false
8322 2818 Acked Read 9 Sent Done Done false
8323 2819 Acked Write 9 Sent Done Done false
8324 2820 Acked Flush 0 Sent Done Done false
8325 2821 Acked Write 1 Sent Done Done false
8326 2822 Acked Write 5 Sent Done Done false
8327 2823 Acked Flush 0 Sent Done Done false
8328 2824 Acked Write 3 Sent Done Done false
8329 2825 Acked Read 4 Sent Done Done false
8330 2826 Acked Read 3 Sent Done Done false
8331 2827 Acked Read 1 Sent Done Done false
8332 2828 Acked Write 9 Sent Done Done false
8333 2829 Acked Read 1 Sent Done Done false
8334 2830 Acked Read 10 Sent Done Done false
8335 2831 Acked Read 8 Sent Done Done false
8336 2832 Acked Write 5 Sent Done Done false
8337 2833 Acked Write 4 Sent Done Done false
8338 2834 Acked Write 1 Sent Done Done false
8339 2835 Acked Write 5 Sent Done Done false
8340 2836 Acked Read 9 Sent Done Done false
8341 2837 Acked Write 4 Sent Done Done false
8342 2838 Acked Read 6 Sent Done Done false
8343 2839 Acked Write 4 Sent Done Done false
8344 2840 Acked Write 2 Sent Done Done false
8345 2841 Acked Write 9 Sent Done Done false
8346 2842 Acked Write 9 Sent Done Done false
8347 2843 Acked Read 7 Sent Done Done false
8348 2844 Acked Read 2 Sent Done Done false
8349 2845 Acked Read 1 Sent Done Done false
8350 2846 Acked Flush 0 Sent Done Done false
8351 2847 Acked Read 4 Sent Done Done false
8352 2848 Acked Read 6 Sent Done Done false
8353 2849 Acked Read 10 Sent Done Done false
8354 2850 Acked Read 10 Sent Done Done false
8355 2851 Acked Write 5 Sent Done Done false
8356 2852 Acked Read 7 Sent Done Done false
8357 2853 Acked Write 9 Sent Done Done false
8358 2854 Acked Read 9 Sent Done Done false
8359 2855 Acked Read 10 Sent Done Done false
8360 2856 Acked Read 1 Sent Done Done false
8361 2857 Acked Read 3 Sent Done Done false
8362 2858 Acked Read 9 Sent Done Done false
8363 2859 Acked Write 8 Sent Done Done false
8364 2860 Acked Read 2 Sent Done Done false
8365 2861 Acked Read 9 Sent Done Done false
8366 2862 Acked Write 5 Sent Done Done false
8367 2863 Acked Read 4 Sent Done Done false
8368 2864 Acked Read 7 Sent Done Done false
8369 2865 Acked Write 1 Sent Done Done false
8370 2866 Acked Read 9 Sent Done Done false
8371 2867 Acked Write 3 Sent Done Done false
8372 2868 Acked Read 3 Sent Done Done false
8373 2869 Acked Flush 0 Sent Done Done false
8374 2870 Acked Read 1 Sent Done Done false
8375 2871 Acked Read 9 Sent Done Done false
8376 2872 Acked Read 9 Sent Done Done false
8377 2873 Acked Write 4 Sent Done Done false
8378 2874 Acked Write 10 Sent Done Done false
8379 2875 Acked Read 9 Sent Done Done false
8380 2876 Acked Flush 0 Sent Done Done false
8381 2877 Acked Flush 0 Sent Done Done false
8382 2878 Acked Read 6 Sent Done Done false
8383 2879 Acked Write 1 Sent Done Done false
8384 2880 Acked Read 6 Sent Done Done false
8385 2881 Acked Write 5 Sent Done Done false
8386 2882 Acked Read 4 Sent Done Done false
8387 2883 Acked Read 2 Sent Done Done false
8388 2884 Acked Write 9 Sent Done Done false
8389 2885 Acked Write 7 Sent Done Done false
8390 2886 Acked Flush 0 Sent Done Done false
8391 2887 Acked Write 7 Sent Done Done false
8392 2888 Acked Flush 0 Sent Done Done false
8393 2889 Acked Flush 0 Sent Done Done false
8394 2890 Acked Read 8 Sent Done Done false
8395 2891 Acked Write 4 Sent Done Done false
8396 2892 Acked Write 7 Sent Done Done false
8397 2893 Acked Read 2 Sent Done Done false
8398 2894 Acked Write 1 Sent Done Done false
8399 2895 Acked Read 1 Sent Done Done false
8400 2896 Acked Read 5 Sent Done Done false
8401 2897 Acked Read 6 Sent Done Done false
8402 2898 Acked Read 10 Sent Done Done false
8403 2899 Acked Write 5 Sent Done Done false
8404 2900 Acked Read 3 Sent Done Done false
8405 2901 Acked Read 8 Sent Done Done false
8406 2902 Acked Write 8 Sent Done Done false
8407 2903 Acked Write 9 Sent Done Done false
8408 2904 Acked Write 7 Sent Done Done false
8409 2905 Acked Write 5 Sent Done Done false
8410 2906 Acked Write 5 Sent Done Done false
8411 2907 Acked Write 8 Sent Done Done false
8412 2908 Acked Read 3 Sent Done Done false
8413 2909 Acked Read 1 Sent Done Done false
8414 2910 Acked Read 7 Sent Done Done false
8415 2911 Acked Read 9 Sent Done Done false
8416 2912 Acked Read 2 Sent Done Done false
8417 2913 Acked Read 7 Sent Done Done false
8418 2914 Acked Write 2 Sent Done Done false
8419 2915 Acked Read 2 Sent Done Done false
8420 2916 Acked Write 9 Sent Done Done false
8421 2917 Acked Flush 0 Sent Done Done false
8422 2918 Acked Read 6 Sent Done Done false
8423 2919 Acked Write 4 Sent Done Done false
8424 2920 Acked Read 10 Sent Done Done false
8425 2921 Acked Read 9 Sent Done Done false
8426 2922 Acked Read 10 Sent Done Done false
8427 2923 Acked Flush 0 Sent Done Done false
8428 2924 Acked Write 5 Sent Done Done false
8429 2925 Acked Write 7 Sent Done Done false
8430 2926 Acked Write 1 Sent Done Done false
8431 2927 Acked Read 6 Sent Done Done false
8432 2928 Acked Read 2 Sent Done Done false
8433 2929 Acked Write 3 Sent Done Done false
8434 2930 Acked Write 1 Sent Done Done false
8435 2931 Acked Write 1 Sent Done Done false
8436 2932 Acked Read 8 Sent Done Done false
8437 2933 Acked Write 2 Sent Done Done false
8438 2934 Acked Read 7 Sent Done Done false
8439 2935 Acked Write 9 Sent Done Done false
8440 2936 Acked Read 8 Sent Done Done false
8441 2937 Acked Write 8 Sent Done Done false
8442 2938 Acked Flush 0 Sent Done Done false
8443 2939 Acked Read 7 Sent Done Done false
8444 2940 Acked Write 9 Sent Done Done false
8445 2941 Acked Flush 0 Sent Done Done false
8446 2942 Acked Write 9 Sent Done Done false
8447 2943 Acked Write 4 Sent Done Done false
8448 2944 Acked Write 10 Sent Done Done false
8449 2945 Acked Read 9 Sent Done Done false
8450 2946 Acked Read 10 Sent Done Done false
8451 2947 Acked Write 9 Sent Done Done false
8452 2948 Acked Read 6 Sent Done Done false
8453 2949 Acked Read 2 Sent Done Done false
8454 2950 Acked Read 6 Sent Done Done false
8455 2951 Acked Write 9 Sent Done Done false
8456 2952 Acked Write 5 Sent Done Done false
8457 2953 Acked Read 1 Sent Done Done false
8458 2954 Acked Write 7 Sent Done Done false
8459 2955 Acked Read 7 Sent Done Done false
8460 2956 Acked Write 8 Sent Done Done false
8461 2957 Acked Read 7 Sent Done Done false
8462 2958 Acked Flush 0 Sent Done Done false
8463 2959 Acked Read 1 Sent Done Done false
8464 2960 Acked Flush 0 Sent Done Done false
8465 2961 Acked Write 3 Sent Done Done false
8466 2962 Acked Flush 0 Sent Done Done false
8467 2963 Acked Read 9 Sent Done Done false
8468 2964 Acked Read 5 Sent Done Done false
8469 2965 Acked Read 5 Sent Done Done false
8470 2966 Acked Read 4 Sent Done Done false
8471 2967 Acked Write 3 Sent Done Done false
8472 2968 Acked Write 5 Sent Done Done false
8473 2969 Acked Read 3 Sent Done Done false
8474 2970 Acked Read 7 Sent Done Done false
8475 2971 Acked Write 5 Sent Done Done false
8476 2972 Acked Read 2 Sent Done Done false
8477 2973 Acked Write 10 Sent Done Done false
8478 2974 Acked Write 3 Sent Done Done false
8479 2975 Acked Write 8 Sent Done Done false
8480 2976 Acked Write 6 Sent Done Done false
8481 2977 Acked Read 1 Sent Done Done false
8482 2978 Acked Read 2 Sent Done Done false
8483 2979 Acked Read 8 Sent Done Done false
8484 2980 Acked Read 9 Sent Done Done false
8485 2981 Acked Write 8 Sent Done Done false
8486 2982 Acked Write 9 Sent Done Done false
8487 2983 Acked Read 9 Sent Done Done false
8488 2984 Acked Write 8 Sent Done Done false
8489 2985 Acked Write 9 Sent Done Done false
8490 2986 Acked Write 3 Sent Done Done false
8491 2987 Acked Read 10 Sent Done Done false
8492 2988 Acked Read 8 Sent Done Done false
8493 2989 Acked Write 2 Sent Done Done false
8494 2990 Acked Flush 0 Sent Done Done false
8495 2991 Acked Write 6 Sent Done Done false
8496 2992 Acked Read 9 Sent Done Done false
8497 2993 Acked Write 7 Sent Done Done false
8498 2994 Acked Read 5 Sent Done Done false
8499 2995 Acked Read 5 Sent Done Done false
8500 2996 Acked Write 3 Sent Done Done false
8501 2997 Acked Write 9 Sent Done Done false
8502 2998 Acked Read 7 Sent Done Done false
8503 2999 Acked Write 8 Sent Done Done false
8504 3000 Acked Write 7 Sent Done Done false
8505 3001 Acked Write 3 Sent Done Done false
8506 3002 Acked Read 1 Sent Done Done false
8507 3003 Acked Read 6 Sent Done Done false
8508 3004 Acked Write 5 Sent Done Done false
8509 3005 Acked Read 4 Sent Done Done false
8510 3006 Acked Read 4 Sent Done Done false
8511 3007 Acked Write 10 Sent Done Done false
8512 3008 Acked Read 1 Sent Done Done false
8513 3009 Acked Write 3 Sent Done Done false
8514 3010 Acked Read 1 Sent Done Done false
8515 3011 Acked Write 4 Sent Done Done false
8516 3012 Acked Read 2 Sent Done Done false
8517 3013 Acked Read 1 Sent Done Done false
8518 3014 Acked Write 9 Sent Done Done false
8519 3015 Acked Write 9 Sent Done Done false
8520 3016 Acked Read 3 Sent Done Done false
8521 3017 Acked Write 7 Sent Done Done false
8522 3018 Acked Read 1 Sent Done Done false
8523 3019 Acked Write 8 Sent Done Done false
8524 3020 Acked Write 3 Sent Done Done false
8525 3021 Acked Read 7 Sent Done Done false
8526 3022 Acked Read 3 Sent Done Done false
8527 3023 Acked Write 1 Sent Done Done false
8528 3024 Acked Write 10 Sent Done Done false
8529 3025 Acked Flush 0 Sent Done Done false
8530 3026 Acked Write 7 Sent Done Done false
8531 3027 Acked Read 1 Sent Done Done false
8532 3028 Acked Write 2 Sent Done Done false
8533 3029 Acked Read 5 Sent Done Done false
8534 3030 Acked Read 7 Sent Done Done false
8535 3031 Acked Read 3 Sent Done Done false
8536 3032 Acked Flush 0 Sent Done Done false
8537 3033 Acked Write 10 Sent Done Done false
8538 3034 Acked Flush 0 Sent Done Done false
8539 3035 Acked Read 6 Sent Done Done false
8540 3036 Acked Read 1 Sent Done Done false
8541 3037 Acked Write 4 Sent Done Done false
8542 3038 Acked Read 5 Sent Done Done false
8543 3039 Acked Flush 0 Sent Done Done false
8544 3040 Acked Read 1 Sent Done Done false
8545 3041 Acked Write 5 Sent Done Done false
8546 3042 Acked Write 5 Sent Done Done false
8547 3043 Acked Write 9 Sent Done Done false
8548 3044 Acked Write 1 Sent Done Done false
8549 3045 Acked Read 6 Sent Done Done false
8550 3046 Acked Read 7 Sent Done Done false
8551 3047 Acked Write 8 Sent Done Done false
8552 3048 Acked Write 6 Sent Done Done false
8553 3049 Acked Read 9 Sent Done Done false
8554 3050 Acked Read 5 Sent Done Done false
8555 3051 Acked Write 4 Sent Done Done false
8556 3052 Acked Write 2 Sent Done Done false
8557 3053 Acked Read 2 Sent Done Done false
8558 3054 Acked Read 4 Sent Done Done false
8559 3055 Acked Read 1 Sent Done Done false
8560 3056 Acked Write 1 Sent Done Done false
8561 3057 Acked Write 8 Sent Done Done false
8562 3058 Acked Read 10 Sent Done Done false
8563 3059 Acked Read 8 Sent Done Done false
8564 3060 Acked Write 7 Sent Done Done false
8565 3061 Acked Write 9 Sent Done Done false
8566 3062 Acked Read 8 Sent Done Done false
8567 3063 Acked Read 1 Sent Done Done false
8568 3064 Acked Write 8 Sent Done Done false
8569 3065 Acked Read 3 Sent Done Done false
8570 3066 Acked Read 3 Sent Done Done false
8571 3067 Acked Read 7 Sent Done Done false
8572 3068 Acked Write 3 Sent Done Done false
8573 3069 Acked Read 8 Sent Done Done false
8574 3070 Acked Read 3 Sent Done Done false
8575 3071 Acked Read 5 Sent Done Done false
8576 3072 Acked Write 2 Sent Done Done false
8577 3073 Acked Write 1 Sent Done Done false
8578 3074 Acked Read 8 Sent Done Done false
8579 3075 Acked Read 1 Sent Done Done false
8580 3076 Acked Write 1 Sent Done Done false
8581 3077 Acked Write 1 Sent Done Done false
8582 3078 Acked Read 10 Sent Done Done false
8583 3079 Acked Write 4 Sent Done Done false
8584 3080 Acked Flush 0 Sent Done Done false
8585 3081 Acked Write 7 Sent Done Done false
8586 3082 Acked Read 9 Sent Done Done false
8587 3083 Acked Flush 0 Sent Done Done false
8588 3084 Acked Write 9 Sent Done Done false
8589 3085 Acked Read 9 Sent Done Done false
8590 3086 Acked Read 7 Sent Done Done false
8591 3087 Acked Write 9 Sent Done Done false
8592 3088 Acked Read 1 Sent Done Done false
8593 3089 Acked Write 8 Sent Done Done false
8594 3090 Acked Read 7 Sent Done Done false
8595 3091 Acked Read 7 Sent Done Done false
8596 3092 Acked Read 9 Sent Done Done false
8597 3093 Acked Read 7 Sent Done Done false
8598 3094 Acked Read 6 Sent Done Done false
8599 3095 Acked Flush 0 Sent Done Done false
8600 3096 Acked Read 10 Sent Done Done false
8601 3097 Acked Read 5 Sent Done Done false
8602 3098 Acked Read 1 Sent Done Done false
8603 3099 Acked Read 8 Sent Done Done false
8604 3100 Acked Write 8 Sent Done Done false
8605 3101 Acked Write 9 Sent Done Done false
8606 3102 Acked Write 7 Sent Done Done false
8607 3103 Acked Write 9 Sent Done Done false
8608 3104 Acked Read 3 Sent Done Done false
8609 3105 Acked Flush 0 Sent Done Done false
8610 3106 Acked Read 5 Sent Done Done false
8611 3107 Acked Read 5 Sent Done Done false
8612 3108 Acked Flush 0 Sent Done Done false
8613 3109 Acked Read 1 Sent Done Done false
8614 3110 Acked Write 9 Sent Done Done false
8615 3111 Acked Flush 0 Sent Done Done false
8616 STATES DS:0 DS:1 DS:2 TOTAL
8617 Sent 302 0 0 302
8618 Done 0 302 302 604
8619 Skipped 0 0 0 0
8620 Error 0 0 0 0
8621 Last Flush: 2809 3111 3111
8622 Downstairs last five completed: 2809 2808 2807 2806 2805
8623 Upstairs last five completed: 3111 3110 3109 3108 3107
8624 CLIENT: Up:0 ds:302 act:2
8625 ----------------------------------------------------------------
8626 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
8627 States: Offline Active Active
8628 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
8629 2810 Acked Read 5 Sent Done Done false
8630 2811 Acked Read 2 Sent Done Done false
8631 2812 Acked Read 9 Sent Done Done false
8632 2813 Acked Read 6 Sent Done Done false
8633 2814 Acked Read 3 Sent Done Done false
8634 2815 Acked Read 10 Sent Done Done false
8635 2816 Acked Read 2 Sent Done Done false
8636 2817 Acked Read 10 Sent Done Done false
8637 2818 Acked Read 9 Sent Done Done false
8638 2819 Acked Write 9 Sent Done Done false
8639 2820 Acked Flush 0 Sent Done Done false
8640 2821 Acked Write 1 Sent Done Done false
8641 2822 Acked Write 5 Sent Done Done false
8642 2823 Acked Flush 0 Sent Done Done false
8643 2824 Acked Write 3 Sent Done Done false
8644 2825 Acked Read 4 Sent Done Done false
8645 2826 Acked Read 3 Sent Done Done false
8646 2827 Acked Read 1 Sent Done Done false
8647 2828 Acked Write 9 Sent Done Done false
8648 2829 Acked Read 1 Sent Done Done false
8649 2830 Acked Read 10 Sent Done Done false
8650 2831 Acked Read 8 Sent Done Done false
8651 2832 Acked Write 5 Sent Done Done false
8652 2833 Acked Write 4 Sent Done Done false
8653 2834 Acked Write 1 Sent Done Done false
8654 2835 Acked Write 5 Sent Done Done false
8655 2836 Acked Read 9 Sent Done Done false
8656 2837 Acked Write 4 Sent Done Done false
8657 2838 Acked Read 6 Sent Done Done false
8658 2839 Acked Write 4 Sent Done Done false
8659 2840 Acked Write 2 Sent Done Done false
8660 2841 Acked Write 9 Sent Done Done false
8661 2842 Acked Write 9 Sent Done Done false
8662 2843 Acked Read 7 Sent Done Done false
8663 2844 Acked Read 2 Sent Done Done false
8664 2845 Acked Read 1 Sent Done Done false
8665 2846 Acked Flush 0 Sent Done Done false
8666 2847 Acked Read 4 Sent Done Done false
8667 2848 Acked Read 6 Sent Done Done false
8668 2849 Acked Read 10 Sent Done Done false
8669 2850 Acked Read 10 Sent Done Done false
8670 2851 Acked Write 5 Sent Done Done false
8671 2852 Acked Read 7 Sent Done Done false
8672 2853 Acked Write 9 Sent Done Done false
8673 2854 Acked Read 9 Sent Done Done false
8674 2855 Acked Read 10 Sent Done Done false
8675 2856 Acked Read 1 Sent Done Done false
8676 2857 Acked Read 3 Sent Done Done false
8677 2858 Acked Read 9 Sent Done Done false
8678 2859 Acked Write 8 Sent Done Done false
8679 2860 Acked Read 2 Sent Done Done false
8680 2861 Acked Read 9 Sent Done Done false
8681 2862 Acked Write 5 Sent Done Done false
8682 2863 Acked Read 4 Sent Done Done false
8683 2864 Acked Read 7 Sent Done Done false
8684 2865 Acked Write 1 Sent Done Done false
8685 2866 Acked Read 9 Sent Done Done false
8686 2867 Acked Write 3 Sent Done Done false
8687 2868 Acked Read 3 Sent Done Done false
8688 2869 Acked Flush 0 Sent Done Done false
8689 2870 Acked Read 1 Sent Done Done false
8690 2871 Acked Read 9 Sent Done Done false
8691 2872 Acked Read 9 Sent Done Done false
8692 2873 Acked Write 4 Sent Done Done false
8693 2874 Acked Write 10 Sent Done Done false
8694 2875 Acked Read 9 Sent Done Done false
8695 2876 Acked Flush 0 Sent Done Done false
8696 2877 Acked Flush 0 Sent Done Done false
8697 2878 Acked Read 6 Sent Done Done false
8698 2879 Acked Write 1 Sent Done Done false
8699 2880 Acked Read 6 Sent Done Done false
8700 2881 Acked Write 5 Sent Done Done false
8701 2882 Acked Read 4 Sent Done Done false
8702 2883 Acked Read 2 Sent Done Done false
8703 2884 Acked Write 9 Sent Done Done false
8704 2885 Acked Write 7 Sent Done Done false
8705 2886 Acked Flush 0 Sent Done Done false
8706 2887 Acked Write 7 Sent Done Done false
8707 2888 Acked Flush 0 Sent Done Done false
8708 2889 Acked Flush 0 Sent Done Done false
8709 2890 Acked Read 8 Sent Done Done false
8710 2891 Acked Write 4 Sent Done Done false
8711 2892 Acked Write 7 Sent Done Done false
8712 2893 Acked Read 2 Sent Done Done false
8713 2894 Acked Write 1 Sent Done Done false
8714 2895 Acked Read 1 Sent Done Done false
8715 2896 Acked Read 5 Sent Done Done false
8716 2897 Acked Read 6 Sent Done Done false
8717 2898 Acked Read 10 Sent Done Done false
8718 2899 Acked Write 5 Sent Done Done false
8719 2900 Acked Read 3 Sent Done Done false
8720 2901 Acked Read 8 Sent Done Done false
8721 2902 Acked Write 8 Sent Done Done false
8722 2903 Acked Write 9 Sent Done Done false
8723 2904 Acked Write 7 Sent Done Done false
8724 2905 Acked Write 5 Sent Done Done false
8725 2906 Acked Write 5 Sent Done Done false
8726 2907 Acked Write 8 Sent Done Done false
8727 2908 Acked Read 3 Sent Done Done false
8728 2909 Acked Read 1 Sent Done Done false
8729 2910 Acked Read 7 Sent Done Done false
8730 2911 Acked Read 9 Sent Done Done false
8731 2912 Acked Read 2 Sent Done Done false
8732 2913 Acked Read 7 Sent Done Done false
8733 2914 Acked Write 2 Sent Done Done false
8734 2915 Acked Read 2 Sent Done Done false
8735 2916 Acked Write 9 Sent Done Done false
8736 2917 Acked Flush 0 Sent Done Done false
8737 2918 Acked Read 6 Sent Done Done false
8738 2919 Acked Write 4 Sent Done Done false
8739 2920 Acked Read 10 Sent Done Done false
8740 2921 Acked Read 9 Sent Done Done false
8741 2922 Acked Read 10 Sent Done Done false
8742 2923 Acked Flush 0 Sent Done Done false
8743 2924 Acked Write 5 Sent Done Done false
8744 2925 Acked Write 7 Sent Done Done false
8745 2926 Acked Write 1 Sent Done Done false
8746 2927 Acked Read 6 Sent Done Done false
8747 2928 Acked Read 2 Sent Done Done false
8748 2929 Acked Write 3 Sent Done Done false
8749 2930 Acked Write 1 Sent Done Done false
8750 2931 Acked Write 1 Sent Done Done false
8751 2932 Acked Read 8 Sent Done Done false
8752 2933 Acked Write 2 Sent Done Done false
8753 2934 Acked Read 7 Sent Done Done false
8754 2935 Acked Write 9 Sent Done Done false
8755 2936 Acked Read 8 Sent Done Done false
8756 2937 Acked Write 8 Sent Done Done false
8757 2938 Acked Flush 0 Sent Done Done false
8758 2939 Acked Read 7 Sent Done Done false
8759 2940 Acked Write 9 Sent Done Done false
8760 2941 Acked Flush 0 Sent Done Done false
8761 2942 Acked Write 9 Sent Done Done false
8762 2943 Acked Write 4 Sent Done Done false
8763 2944 Acked Write 10 Sent Done Done false
8764 2945 Acked Read 9 Sent Done Done false
8765 2946 Acked Read 10 Sent Done Done false
8766 2947 Acked Write 9 Sent Done Done false
8767 2948 Acked Read 6 Sent Done Done false
8768 2949 Acked Read 2 Sent Done Done false
8769 2950 Acked Read 6 Sent Done Done false
8770 2951 Acked Write 9 Sent Done Done false
8771 2952 Acked Write 5 Sent Done Done false
8772 2953 Acked Read 1 Sent Done Done false
8773 2954 Acked Write 7 Sent Done Done false
8774 2955 Acked Read 7 Sent Done Done false
8775 2956 Acked Write 8 Sent Done Done false
8776 2957 Acked Read 7 Sent Done Done false
8777 2958 Acked Flush 0 Sent Done Done false
8778 2959 Acked Read 1 Sent Done Done false
8779 2960 Acked Flush 0 Sent Done Done false
8780 2961 Acked Write 3 Sent Done Done false
8781 2962 Acked Flush 0 Sent Done Done false
8782 2963 Acked Read 9 Sent Done Done false
8783 2964 Acked Read 5 Sent Done Done false
8784 2965 Acked Read 5 Sent Done Done false
8785 2966 Acked Read 4 Sent Done Done false
8786 2967 Acked Write 3 Sent Done Done false
8787 2968 Acked Write 5 Sent Done Done false
8788 2969 Acked Read 3 Sent Done Done false
8789 2970 Acked Read 7 Sent Done Done false
8790 2971 Acked Write 5 Sent Done Done false
8791 2972 Acked Read 2 Sent Done Done false
8792 2973 Acked Write 10 Sent Done Done false
8793 2974 Acked Write 3 Sent Done Done false
8794 2975 Acked Write 8 Sent Done Done false
8795 2976 Acked Write 6 Sent Done Done false
8796 2977 Acked Read 1 Sent Done Done false
8797 2978 Acked Read 2 Sent Done Done false
8798 2979 Acked Read 8 Sent Done Done false
8799 2980 Acked Read 9 Sent Done Done false
8800 2981 Acked Write 8 Sent Done Done false
8801 2982 Acked Write 9 Sent Done Done false
8802 2983 Acked Read 9 Sent Done Done false
8803 2984 Acked Write 8 Sent Done Done false
8804 2985 Acked Write 9 Sent Done Done false
8805 2986 Acked Write 3 Sent Done Done false
8806 2987 Acked Read 10 Sent Done Done false
8807 2988 Acked Read 8 Sent Done Done false
8808 2989 Acked Write 2 Sent Done Done false
8809 2990 Acked Flush 0 Sent Done Done false
8810 2991 Acked Write 6 Sent Done Done false
8811 2992 Acked Read 9 Sent Done Done false
8812 2993 Acked Write 7 Sent Done Done false
8813 2994 Acked Read 5 Sent Done Done false
8814 2995 Acked Read 5 Sent Done Done false
8815 2996 Acked Write 3 Sent Done Done false
8816 2997 Acked Write 9 Sent Done Done false
8817 2998 Acked Read 7 Sent Done Done false
8818 2999 Acked Write 8 Sent Done Done false
8819 3000 Acked Write 7 Sent Done Done false
8820 3001 Acked Write 3 Sent Done Done false
8821 3002 Acked Read 1 Sent Done Done false
8822 3003 Acked Read 6 Sent Done Done false
8823 3004 Acked Write 5 Sent Done Done false
8824 3005 Acked Read 4 Sent Done Done false
8825 3006 Acked Read 4 Sent Done Done false
8826 3007 Acked Write 10 Sent Done Done false
8827 3008 Acked Read 1 Sent Done Done false
8828 3009 Acked Write 3 Sent Done Done false
8829 3010 Acked Read 1 Sent Done Done false
8830 3011 Acked Write 4 Sent Done Done false
8831 3012 Acked Read 2 Sent Done Done false
8832 3013 Acked Read 1 Sent Done Done false
8833 3014 Acked Write 9 Sent Done Done false
8834 3015 Acked Write 9 Sent Done Done false
8835 3016 Acked Read 3 Sent Done Done false
8836 3017 Acked Write 7 Sent Done Done false
8837 3018 Acked Read 1 Sent Done Done false
8838 3019 Acked Write 8 Sent Done Done false
8839 3020 Acked Write 3 Sent Done Done false
8840 3021 Acked Read 7 Sent Done Done false
8841 3022 Acked Read 3 Sent Done Done false
8842 3023 Acked Write 1 Sent Done Done false
8843 3024 Acked Write 10 Sent Done Done false
8844 3025 Acked Flush 0 Sent Done Done false
8845 3026 Acked Write 7 Sent Done Done false
8846 3027 Acked Read 1 Sent Done Done false
8847 3028 Acked Write 2 Sent Done Done false
8848 3029 Acked Read 5 Sent Done Done false
8849 3030 Acked Read 7 Sent Done Done false
8850 3031 Acked Read 3 Sent Done Done false
8851 3032 Acked Flush 0 Sent Done Done false
8852 3033 Acked Write 10 Sent Done Done false
8853 3034 Acked Flush 0 Sent Done Done false
8854 3035 Acked Read 6 Sent Done Done false
8855 3036 Acked Read 1 Sent Done Done false
8856 3037 Acked Write 4 Sent Done Done false
8857 3038 Acked Read 5 Sent Done Done false
8858 3039 Acked Flush 0 Sent Done Done false
8859 3040 Acked Read 1 Sent Done Done false
8860 3041 Acked Write 5 Sent Done Done false
8861 3042 Acked Write 5 Sent Done Done false
8862 3043 Acked Write 9 Sent Done Done false
8863 3044 Acked Write 1 Sent Done Done false
8864 3045 Acked Read 6 Sent Done Done false
8865 3046 Acked Read 7 Sent Done Done false
8866 3047 Acked Write 8 Sent Done Done false
8867 3048 Acked Write 6 Sent Done Done false
8868 3049 Acked Read 9 Sent Done Done false
8869 3050 Acked Read 5 Sent Done Done false
8870 3051 Acked Write 4 Sent Done Done false
8871 3052 Acked Write 2 Sent Done Done false
8872 3053 Acked Read 2 Sent Done Done false
8873 3054 Acked Read 4 Sent Done Done false
8874 3055 Acked Read 1 Sent Done Done false
8875 3056 Acked Write 1 Sent Done Done false
8876 3057 Acked Write 8 Sent Done Done false
8877 3058 Acked Read 10 Sent Done Done false
8878 3059 Acked Read 8 Sent Done Done false
8879 3060 Acked Write 7 Sent Done Done false
8880 3061 Acked Write 9 Sent Done Done false
8881 3062 Acked Read 8 Sent Done Done false
8882 3063 Acked Read 1 Sent Done Done false
8883 3064 Acked Write 8 Sent Done Done false
8884 3065 Acked Read 3 Sent Done Done false
8885 3066 Acked Read 3 Sent Done Done false
8886 3067 Acked Read 7 Sent Done Done false
8887 3068 Acked Write 3 Sent Done Done false
8888 3069 Acked Read 8 Sent Done Done false
8889 3070 Acked Read 3 Sent Done Done false
8890 3071 Acked Read 5 Sent Done Done false
8891 3072 Acked Write 2 Sent Done Done false
8892 3073 Acked Write 1 Sent Done Done false
8893 3074 Acked Read 8 Sent Done Done false
8894 3075 Acked Read 1 Sent Done Done false
8895 3076 Acked Write 1 Sent Done Done false
8896 3077 Acked Write 1 Sent Done Done false
8897 3078 Acked Read 10 Sent Done Done false
8898 3079 Acked Write 4 Sent Done Done false
8899 3080 Acked Flush 0 Sent Done Done false
8900 3081 Acked Write 7 Sent Done Done false
8901 3082 Acked Read 9 Sent Done Done false
8902 3083 Acked Flush 0 Sent Done Done false
8903 3084 Acked Write 9 Sent Done Done false
8904 3085 Acked Read 9 Sent Done Done false
8905 3086 Acked Read 7 Sent Done Done false
8906 3087 Acked Write 9 Sent Done Done false
8907 3088 Acked Read 1 Sent Done Done false
8908 3089 Acked Write 8 Sent Done Done false
8909 3090 Acked Read 7 Sent Done Done false
8910 3091 Acked Read 7 Sent Done Done false
8911 3092 Acked Read 9 Sent Done Done false
8912 3093 Acked Read 7 Sent Done Done false
8913 3094 Acked Read 6 Sent Done Done false
8914 3095 Acked Flush 0 Sent Done Done false
8915 3096 Acked Read 10 Sent Done Done false
8916 3097 Acked Read 5 Sent Done Done false
8917 3098 Acked Read 1 Sent Done Done false
8918 3099 Acked Read 8 Sent Done Done false
8919 3100 Acked Write 8 Sent Done Done false
8920 3101 Acked Write 9 Sent Done Done false
8921 3102 Acked Write 7 Sent Done Done false
8922 3103 Acked Write 9 Sent Done Done false
8923 3104 Acked Read 3 Sent Done Done false
8924 3105 Acked Flush 0 Sent Done Done false
8925 3106 Acked Read 5 Sent Done Done false
8926 3107 Acked Read 5 Sent Done Done false
8927 3108 Acked Flush 0 Sent Done Done false
8928 3109 Acked Read 1 Sent Done Done false
8929 3110 Acked Write 9 Sent Done Done false
8930 3111 Acked Flush 0 Sent Done Done false
8931 STATES DS:0 DS:1 DS:2 TOTAL
8932 Sent 302 0 0 302
8933 Done 0 302 302 604
8934 Skipped 0 0 0 0
8935 Error 0 0 0 0
8936 Last Flush: 2809 3111 3111
8937 Downstairs last five completed: 2809 2808 2807 2806 2805
8938 Upstairs last five completed: 3111 3110 3109 3108 3107
8939 CLIENT: Up:0 ds:302 act:2
89402024-11-15T01:59:21.335ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
89412024-11-15T01:59:21.335ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:61603) = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
89422024-11-15T01:59:21.336ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
89432024-11-15T01:59:21.337ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
89442024-11-15T01:59:21.337ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
89452024-11-15T01:59:21.337ZINFOcrucible: send last flush ID to this DS: 2809 = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
89462024-11-15T01:59:21.337ZINFOcrucible: Replied this last flush ID: 2809 = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
89472024-11-15T01:59:21.337ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
89482024-11-15T01:59:21.337ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
89492024-11-15T01:59:21.337ZINFOcrucible: [0] client re-new 302 jobs since flush 2809 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
89502024-11-15T01:59:21.337ZINFOcrucible: [0] Replayed 302 jobs since flush: 2809 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
8951 ----------------------------------------------------------------
8952 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
8953 Downstairs last five completed: 3111 3110 3109 3108 3107
8954 Upstairs last five completed: 3111 3110 3109 3108 3107
8955 CLIENT: Up:0 ds:0 act:3
8956 Replay: All jobs finished
8957 Main task has work to do, go find it
8958 got dsc Stop(1)
8959 stop 1
8960 Nov 15 01:59:21.585 INFO request completed, [8820] Got stop action so:false kr:false
8961 latency_us: 376, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 8fdf3ef1-08eb-42da-879f-c357c3805dd6, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
89622024-11-15T01:59:21.586ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
89632024-11-15T01:59:21.586ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
89642024-11-15T01:59:21.586ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
89652024-11-15T01:59:21.586ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
89662024-11-15T01:59:21.586ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
8967 Nov 15 01:59:21.586 INFO request completed, latency_us: 175, response_code: 200, uri: /state/cid/1, method: GET, req_id: 39e07acf-0f37-4a0d-a771-9cd63feb02c2, remote_addr: 127.0.0.1:40008, local_addr: 127.0.{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2024-11-15T01:59:21.586345864Z","hostname":"w-01JCPRHT5F2MPA44KVJHM3S2PB","pid":1205,"":"io task","client":"1","":"downstairs","session_id":"971dcb2e-6891-4ecf-b367-1e04e52065ee"}
8968 0.1:9998
8969 [8820] Exited with: ExitStatus(unix_wait_status(9))
8970 [8820][1] reports Exit
8971 Nov 15 01:59:25.588 INFO request completed, latency_us: 308, response_code: 200, uri: /state/cid/1, method: GET, req_id: 516863b5-1792-460c-9e75-ae154ca39e34, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
8972 001/300 Read block 4331 len 16384
8973 002/300 Flush
8974 003/300 Read block 2248 len 20480
8975 004/300 Write block 4913 len 40960 data: 2 2 2 3 3 4 3 4 5 5
8976 005/300 Read block 516 len 12288
8977 006/300 Read block 2010 len 12288
8978 007/300 Write block 3859 len 12288 data: 2 3 3
8979 008/300 Write block 3770 len 28672 data: 2 2 2 2 2 3 3
8980 009/300 Write block 9 len 12288 data: 3 3 3
8981 010/300 Read block 4261 len 28672
8982 011/300 Write block 705 len 4096 data: 6
8983 012/300 Read block 28 len 28672
8984 013/300 Read block 3791 len 40960
8985 014/300 Flush
8986 015/300 Read block 4394 len 36864
8987 016/300 Write block 6 len 40960 data: 3 3 3 4 4 4 3 2 2 2
8988 017/300 Read block 2850 len 8192
8989 018/300 Flush
8990 019/300 Write block 2542 len 36864 data: 2 2 3 4 4 3 3 2 2
8991 020/300 Read block 471 len 36864
8992 021/300 Write block 2937 len 20480 data: 3 3 3 3 3
8993 022/300 Read block 873 len 20480
8994 023/300 Write block 4225 len 20480 data: 3 3 3 3 4
8995 024/300 Read block 4189 len 4096
8996 025/300 Write block 203 len 12288 data: 3 3 3
8997 026/300 Write block 2457 len 12288 data: 5 4 4
8998 027/300 Read block 4333 len 20480
8999 028/300 Read block 165 len 28672
9000 029/300 Write block 1157 len 12288 data: 2 2 2
9001 030/300 Read block 2204 len 40960
9002 031/300 Read block 568 len 20480
9003 032/300 Read block 3851 len 16384
9004 033/300 Flush
9005 034/300 Read block 4537 len 16384
9006 035/300 Write block 2016 len 20480 data: 2 2 2 2 2
9007 036/300 Write block 4939 len 16384 data: 2 2 2 2
9008 037/300 Write block 2842 len 16384 data: 2 2 2 2
9009 038/300 Write block 3736 len 16384 data: 3 4 4 5
9010 039/300 Write block 1997 len 8192 data: 2 2
9011 040/300 Read block 2945 len 28672
9012 041/300 Write block 1897 len 20480 data: 2 3 4 4 4
9013 042/300 Read block 3199 len 36864
9014 043/300 Read block 2551 len 28672
9015 044/300 Read block 4426 len 32768
9016 045/300 Write block 3741 len 28672 data: 4 4 4 5 5 5 5
9017 046/300 Read block 3353 len 32768
9018 047/300 Write block 4834 len 36864 data: 4 4 3 3 3 3 3 4 4
9019 048/300 Write block 975 len 24576 data: 2 2 2 2 2 2
9020 049/300 Read block 3567 len 40960
9021 050/300 Read block 3551 len 32768
9022 051/300 Write block 4643 len 16384 data: 3 2 2 4
9023 052/300 Read block 1415 len 40960
9024 053/300 Write block 3582 len 16384 data: 3 2 2 2
9025 054/300 Read block 3014 len 4096
9026 055/300 Write block 1106 len 16384 data: 2 2 2 2
9027 056/300 Read block 422 len 32768
9028 057/300 Write block 2504 len 28672 data: 2 2 2 2 2 2 2
9029 058/300 Read block 325 len 16384
9030 059/300 Read block 3329 len 40960
9031 060/300 Flush
9032 061/300 Flush
9033 062/300 Read block 2915 len 16384
9034 063/300 Read block 438 len 40960
9035 064/300 Read block 433 len 24576
9036 065/300 Read block 499 len 28672
9037 066/300 Write block 2859 len 12288 data: 3 3 2
9038 067/300 Read block 4608 len 32768
9039 068/300 Write block 3136 len 4096 data: 4
9040 069/300 Read block 1312 len 4096
9041 070/300 Write block 3687 len 40960 data: 4 4 4 4 4 3 3 3 3 3
9042 071/300 Read block 3730 len 24576
9043 072/300 Write block 2322 len 28672 data: 4 3 3 4 2 2 2
9044 073/300 Write block 2237 len 8192 data: 3 3
9045 074/300 Write block 3718 len 40960 data: 2 3 3 3 3 3 3 3 2 2
9046 075/300 Read block 815 len 12288
9047 076/300 Read block 1616 len 20480
9048 077/300 Read block 1375 len 12288
9049 078/300 Write block 3282 len 16384 data: 3 3 3 2
9050 079/300 Read block 3826 len 40960
9051 080/300 Flush
9052 081/300 Write block 4214 len 28672 data: 3 3 3 3 2 2 3
9053 082/300 Write block 4804 len 36864 data: 2 2 2 2 2 2 2 2 2
9054 083/300 Write block 2030 len 40960 data: 2 2 2 2 2 2 2 2 2 2
9055 084/300 Read block 4643 len 32768
9056 085/300 Read block 4590 len 28672
9057 086/300 Read block 4027 len 40960
9058 087/300 Read block 3462 len 4096
9059 088/300 Read block 977 len 40960
9060 089/300 Write block 4179 len 28672 data: 3 2 2 2 2 3 3
9061 090/300 Write block 2360 len 36864 data: 4 4 4 3 2 3 3 3 3
9062 091/300 Flush
9063 092/300 Read block 1193 len 36864
9064 093/300 Write block 2278 len 36864 data: 3 3 2 2 2 3 4 5 5
9065 094/300 Write block 58 len 24576 data: 3 2 2 2 2 2
9066 095/300 Flush
9067 096/300 Read block 3659 len 12288
9068 097/300 Write block 781 len 28672 data: 3 3 5 5 5 5 5
9069 098/300 Read block 1382 len 28672
9070 099/300 Read block 1814 len 36864
9071 100/300 Write block 4209 len 12288 data: 5 3 2
9072 101/300 Read block 4849 len 4096
9073 102/300 Flush
9074 103/300 Read block 3911 len 12288
9075 104/300 Flush
9076 105/300 Read block 4786 len 40960
9077 106/300 Write block 1675 len 24576 data: 2 2 2 3 3 3
9078 107/300 Read block 4626 len 40960
9079 108/300 Read block 3202 len 4096
9080 109/300 Write block 4422 len 8192 data: 4 4
9081 110/300 Write block 1187 len 12288 data: 4 3 2
9082 111/300 Read block 4807 len 8192
9083 112/300 Read block 4232 len 28672
9084 113/300 Read block 2136 len 4096
9085 114/300 Read block 761 len 20480
9086 115/300 Read block 2604 len 4096
9087 116/300 Read block 996 len 4096
9088 117/300 Read block 421 len 20480
9089 118/300 Read block 4953 len 20480
9090 119/300 Read block 1839 len 12288
9091 120/300 Write block 3731 len 20480 data: 3 2 2 2 3
9092 121/300 Read block 4602 len 12288
9093 122/300 Flush
9094 123/300 Read block 837 len 12288
9095 124/300 Read block 68 len 4096
9096 125/300 Write block 2173 len 16384 data: 3 3 3 3
9097 126/300 Read block 2738 len 8192
9098 127/300 Write block 3 len 20480 data: 2 3 2 4 4
9099 128/300 Write block 2542 len 32768 data: 3 3 4 5 5 4 4 3
9100 129/300 Read block 1820 len 40960
9101 130/300 Write block 461 len 40960 data: 3 3 3 3 3 3 3 3 2 2
9102 131/300 Write block 1759 len 24576 data: 2 2 2 2 3 3
9103 132/300 Read block 290 len 12288
9104 133/300 Write block 527 len 4096 data: 3
9105 134/300 Read block 1636 len 20480
9106 135/300 Write block 3639 len 4096 data: 2
9107 136/300 Read block 1765 len 16384
9108 137/300 Write block 2815 len 40960 data: 2 2 2 3 3 2 3 4 4 4
9109 138/300 Read block 1555 len 12288
9110 139/300 Write block 1662 len 36864 data: 5 6 7 7 6 5 5 5 5
9111 140/300 Flush
9112 141/300 Write block 4258 len 36864 data: 3 3 3 2 2 3 4 5 5
9113 142/300 Write block 1241 len 40960 data: 2 2 2 2 3 3 3 3 3 3
9114 143/300 Read block 1978 len 12288
9115 144/300 Write block 4231 len 16384 data: 3 3 3 3
9116 145/300 Write block 1372 len 8192 data: 3 3
9117 146/300 Read block 4055 len 4096
9118 147/300 Write block 3849 len 24576 data: 2 2 2 2 2 2
9119 148/300 Read block 2682 len 32768
9120 149/300 Read block 1992 len 24576
9121 150/300 Write block 2372 len 16384 data: 2 2 3 2
9122 151/300 Write block 826 len 8192 data: 4 4
9123 152/300 Read block 938 len 20480
9124 153/300 Write block 3392 len 20480 data: 2 2 2 2 2
9125 154/300 Read block 3858 len 36864
9126 155/300 Read block 4187 len 20480
9127 156/300 Read block 2605 len 12288
9128 157/300 Write block 1530 len 16384 data: 2 2 2 2
9129 158/300 Read block 3654 len 4096
9130 159/300 Write block 4775 len 8192 data: 3 4
9131 160/300 Write block 1458 len 28672 data: 3 3 3 3 3 2 3
9132 161/300 Write block 4250 len 40960 data: 2 2 2 3 3 3 3 3 4 4
9133 162/300 Read block 529 len 20480
9134 163/300 Read block 4380 len 28672
9135 164/300 Read block 88 len 4096
9136 165/300 Read block 4418 len 24576
9137 166/300 Write block 2134 len 32768 data: 3 3 3 3 3 2 2 2
9138 167/300 Write block 3249 len 28672 data: 3 3 4 3 3 4 3
9139 168/300 Read block 1002 len 36864
9140 169/300 Read block 275 len 20480
9141 170/300 Write block 2470 len 40960 data: 2 2 2 3 3 3 3 3 2 2
9142 171/300 Read block 2952 len 12288
9143 172/300 Read block 1824 len 32768
9144 173/300 Read block 1586 len 4096
9145 174/300 Write block 985 len 28672 data: 2 2 2 2 2 2 3
9146 175/300 Write block 1288 len 4096 data: 3
9147 176/300 Write block 3932 len 20480 data: 2 2 2 3 4
9148 177/300 Read block 510 len 12288
9149 178/300 Read block 3873 len 4096
9150 179/300 Read block 3273 len 28672
9151 180/300 Write block 4577 len 12288 data: 2 2 2
9152 181/300 Read block 4025 len 20480
9153 182/300 Read block 3253 len 8192
9154 183/300 Read block 1234 len 36864
9155 184/300 Write block 2919 len 36864 data: 2 2 2 2 2 2 2 2 2
9156 185/300 Flush
9157 186/300 Read block 3771 len 32768
9158 187/300 Read block 1801 len 36864
9159 188/300 Write block 4242 len 28672 data: 2 2 3 3 3 2 2
9160 189/300 Read block 1115 len 16384
9161 190/300 Read block 4267 len 24576
9162 191/300 Write block 1208 len 36864 data: 2 2 2 3 2 2 2 2 2
9163 192/300 Read block 1774 len 28672
9164 193/300 Write block 87 len 32768 data: 3 4 4 3 2 2 2 4
9165 194/300 Read block 600 len 16384
9166 195/300 Read block 151 len 20480
9167 196/300 Write block 1362 len 16384 data: 2 2 2 3
9168 197/300 Flush
9169 198/300 Write block 3666 len 28672 data: 2 3 2 3 3 3 3
9170 199/300 Write block 20 len 36864 data: 3 2 2 2 2 2 2 2 2
9171 200/300 Read block 3197 len 28672
9172 201/300 Write block 3602 len 4096 data: 6
9173 202/300 Read block 3262 len 20480
9174 203/300 Read block 3936 len 24576
9175 204/300 Write block 3289 len 28672 data: 2 2 2 2 2 2 2
9176 205/300 Flush
9177 206/300 Write block 781 len 36864 data: 4 4 6 6 6 6 6 6 5
9178 207/300 Write block 1874 len 40960 data: 3 3 3 2 2 2 2 4 3 3
9179 208/300 Flush
9180 209/300 Read block 4420 len 20480
9181 210/300 Flush
9182 211/300 Write block 2463 len 12288 data: 4 4 4
9183 212/300 Write block 1832 len 4096 data: 2
9184 213/300 Write block 3922 len 32768 data: 4 5 5 4 4 4 3 2
9185 214/300 Write block 209 len 24576 data: 2 2 2 2 2 3
9186 215/300 Read block 4362 len 20480
9187 216/300 Write block 1524 len 40960 data: 2 2 2 2 2 2 3 3 3 3
9188 217/300 Read block 342 len 16384
9189 218/300 Read block 1545 len 20480
9190 219/300 Read block 2511 len 36864
9191 220/300 Write block 4088 len 4096 data: 2
9192 221/300 Read block 1012 len 4096
9193 222/300 Flush
9194 223/300 Read block 1623 len 32768
9195 224/300 Read block 1674 len 16384
9196 225/300 Read block 206 len 8192
9197 226/300 Write block 998 len 16384 data: 4 4 4 4
9198 227/300 Read block 880 len 12288
9199 228/300 Read block 1338 len 4096
9200 229/300 Read block 2440 len 40960
9201 230/300 Read block 4305 len 40960
9202 231/300 Read block 3011 len 32768
9203 232/300 Read block 874 len 16384
9204 233/300 Read block 4622 len 28672
9205 234/300 Write block 1558 len 32768 data: 4 5 5 5 6 6 5 5
9206 235/300 Write block 10 len 20480 data: 5 5 4 3 3
9207 236/300 Read block 1606 len 32768
9208 237/300 Read block 2573 len 36864
9209 238/300 Write block 3074 len 32768 data: 3 5 6 7 6 8 9 8
9210 239/300 Read block 4879 len 24576
9211 240/300 Read block 1511 len 8192
9212 241/300 Write block 3337 len 16384 data: 2 2 2 2
9213 242/300 Flush
9214 243/300 Write block 1663 len 28672 data: 7 8 8 7 6 6 6
9215 244/300 Read block 1037 len 36864
9216 245/300 Write block 76 len 24576 data: 3 3 3 4 4 4
9217 246/300 Flush
9218 247/300 Flush
9219 248/300 Write block 2258 len 8192 data: 2 2
9220 249/300 Write block 3187 len 40960 data: 3 4 5 4 4 4 3 3 2 2
9221 250/300 Read block 423 len 4096
9222 251/300 Write block 1315 len 12288 data: 5 3 4
9223 252/300 Flush
9224 253/300 Write block 2594 len 40960 data: 3 4 5 5 5 5 4 4 4 4
9225 254/300 Flush
9226 255/300 Write block 2951 len 28672 data: 5 5 4 5 4 4 4
9227 256/300 Read block 4075 len 24576
9228 257/300 Write block 4538 len 8192 data: 2 2
9229 258/300 Read block 928 len 40960
9230 259/300 Read block 2835 len 32768
9231 260/300 Flush
9232 261/300 Write block 2775 len 16384 data: 3 3 2 3
9233 262/300 Read block 1109 len 28672
9234 263/300 Read block 3052 len 20480
9235 264/300 Write block 4607 len 16384 data: 2 2 2 2
9236 265/300 Read block 2017 len 8192
9237 266/300 Write block 4794 len 32768 data: 3 3 3 3 3 3 2 2
9238 267/300 Read block 3743 len 4096
9239 268/300 Read block 2943 len 4096
9240 269/300 Read block 4426 len 40960
9241 270/300 Write block 1820 len 12288 data: 2 2 3
9242 271/300 Read block 4351 len 36864
9243 272/300 Read block 3477 len 24576
9244 273/300 Read block 2257 len 12288
9245 274/300 Write block 739 len 4096 data: 3
9246 275/300 Write block 3158 len 28672 data: 2 2 2 2 2 2 2
9247 276/300 Write block 1776 len 36864 data: 3 3 3 3 3 3 5 5 3
9248 277/300 Read block 2695 len 12288
9249 278/300 Read block 2479 len 16384
9250 279/300 Read block 3025 len 28672
9251 280/300 Write block 2284 len 20480 data: 5 6 6 4 4
9252 281/300 Write block 3361 len 20480 data: 3 2 2 2 2
9253 282/300 Read block 138 len 24576
9254 283/300 Read block 2024 len 16384
9255 284/300 Write block 4189 len 36864 data: 3 4 3 3 3 4 5 5 5
9256 285/300 Read block 357 len 40960
9257 286/300 Write block 3323 len 8192 data: 2 2
9258 287/300 Write block 3154 len 32768 data: 2 2 2 2 3 3 3 3
9259 288/300 Write block 314 len 36864 data: 2 2 2 2 2 2 2 3 3
9260 289/300 Flush
9261 290/300 Write block 54 len 4096 data: 3
9262 291/300 Read block 1254 len 36864
9263 292/300 Flush
9264 293/300 Read block 181 len 12288
9265 294/300 Read block 1802 len 36864
9266 295/300 Flush
9267 296/300 Write block 109 len 4096 data: 2
9268 297/300 Write block 4174 len 28672 data: 3 4 4 4 4 4 3
9269 298/300 Write block 2727 len 32768 data: 3 3 3 3 3 3 3 2
9270 299/300 Write block 4015 len 4096 data: 3
9271 300/300 Write block 928 len 16384 data: 3 3 3 3
9272 301/300 Read block 2125 len 20480
9273 Main task has work to do, go find it
9274 got dsc Start(1)
9275 start 1
9276 Nov 15 01:59:25.829 INFO request completed, latency_us: 210, response_code: 204, uri: /start/cid/1, method: POST, req_id: c6ff1180-4f44-4aba-b1f4-02b17dd9ce93, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
9277 Replay: started 1, returned:Ok(())
9278 ----------------------------------------------------------------
9279 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
9280 States: Active Offline Active
9281 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
9282 3112 Acked Read 4 Done Sent Done false
9283 3113 Acked Flush 0 Done Sent Done false
9284 3114 Acked Read 5 Done Sent Done false
9285 3115 Acked Write 10 Done Sent Done false
9286 3116 Acked Read 3 Done Sent Done false
9287 3117 Acked Read 3 Done Sent Done false
9288 3118 Acked Write 3 Done Sent Done false
9289 3119 Acked Write 7 Done Sent Done false
9290 3120 Acked Write 3 Done Sent Done false
9291 3121 Acked Read 7 Done Sent Done false
9292 3122 Acked Write 1 Done Sent Done false
9293 3123 Acked Read 7 Done Sent Done false
9294 3124 Acked Read 10 Done Sent Done false
9295 3125 Acked Flush 0 Done Sent Done false
9296 3126 Acked Read 9 Done Sent Done false
9297 3127 Acked Write 10 Done Sent Done false
9298 3128 Acked Read 2 Done Sent Done false
9299 3129 Acked Flush 0 Done Sent Done false
9300 3130 Acked Write 9 Done Sent Done false
9301 3131 Acked Read 9 Done Sent Done false
9302 3132 Acked Write 5 Done Sent Done false
9303 3133 Acked Read 5 Done Sent Done false
9304 3134 Acked Write 5 Done Sent Done false
9305 3135 Acked Read 1 Done Sent Done false
9306 3136 Acked Write 3 Done Sent Done false
9307 3137 Acked Write 3 Done Sent Done false
9308 3138 Acked Read 5 Done Sent Done false
9309 3139 Acked Read 7 Done Sent Done false
9310 3140 Acked Write 3 Done Sent Done false
9311 3141 Acked Read 10 Done Sent Done false
9312 3142 Acked Read 5 Done Sent Done false
9313 3143 Acked Read 4 Done Sent Done false
9314 3144 Acked Flush 0 Done Sent Done false
9315 3145 Acked Read 4 Done Sent Done false
9316 3146 Acked Write 5 Done Sent Done false
9317 3147 Acked Write 4 Done Sent Done false
9318 3148 Acked Write 4 Done Sent Done false
9319 3149 Acked Write 4 Done Sent Done false
9320 3150 Acked Write 2 Done Sent Done false
9321 3151 Acked Read 7 Done Sent Done false
9322 3152 Acked Write 5 Done Sent Done false
9323 3153 Acked Read 9 Done Sent Done false
9324 3154 Acked Read 7 Done Sent Done false
9325 3155 Acked Read 8 Done Sent Done false
9326 3156 Acked Write 7 Done Sent Done false
9327 3157 Acked Read 8 Done Sent Done false
9328 3158 Acked Write 9 Done Sent Done false
9329 3159 Acked Write 6 Done Sent Done false
9330 3160 Acked Read 10 Done Sent Done false
9331 3161 Acked Read 8 Done Sent Done false
9332 3162 Acked Write 4 Done Sent Done false
9333 3163 Acked Read 10 Done Sent Done false
9334 3164 Acked Write 4 Done Sent Done false
9335 3165 Acked Read 1 Done Sent Done false
9336 3166 Acked Write 4 Done Sent Done false
9337 3167 Acked Read 8 Done Sent Done false
9338 3168 Acked Write 7 Done Sent Done false
9339 3169 Acked Read 4 Done Sent Done false
9340 3170 Acked Read 10 Done Sent Done false
9341 3171 Acked Flush 0 Done Sent Done false
9342 3172 Acked Flush 0 Done Sent Done false
9343 3173 Acked Read 4 Done Sent Done false
9344 3174 Acked Read 10 Done Sent Done false
9345 3175 Acked Read 6 Done Sent Done false
9346 3176 Acked Read 7 Done Sent Done false
9347 3177 Acked Write 3 Done Sent Done false
9348 3178 Acked Read 8 Done Sent Done false
9349 3179 Acked Write 1 Done Sent Done false
9350 3180 Acked Read 1 Done Sent Done false
9351 3181 Acked Write 10 Done Sent Done false
9352 3182 Acked Read 6 Done Sent Done false
9353 3183 Acked Write 7 Done Sent Done false
9354 3184 Acked Write 2 Done Sent Done false
9355 3185 Acked Write 10 Done Sent Done false
9356 3186 Acked Read 3 Done Sent Done false
9357 3187 Acked Read 5 Done Sent Done false
9358 3188 Acked Read 3 Done Sent Done false
9359 3189 Acked Write 4 Done Sent Done false
9360 3190 Acked Read 10 Done Sent Done false
9361 3191 Acked Flush 0 Done Sent Done false
9362 3192 Acked Write 7 Done Sent Done false
9363 3193 Acked Write 9 Done Sent Done false
9364 3194 Acked Write 10 Done Sent Done false
9365 3195 Acked Read 8 Done Sent Done false
9366 3196 Acked Read 7 Done Sent Done false
9367 3197 Acked Read 10 Done Sent Done false
9368 3198 Acked Read 1 Done Sent Done false
9369 3199 Acked Read 10 Done Sent Done false
9370 3200 Acked Write 7 Done Sent Done false
9371 3201 Acked Write 9 Done Sent Done false
9372 3202 Acked Flush 0 Done Sent Done false
9373 3203 Acked Read 9 Done Sent Done false
9374 3204 Acked Write 9 Done Sent Done false
9375 3205 Acked Write 6 Done Sent Done false
9376 3206 Acked Flush 0 Done Sent Done false
9377 3207 Acked Read 3 Done Sent Done false
9378 3208 Acked Write 7 Done Sent Done false
9379 3209 Acked Read 7 Done Sent Done false
9380 3210 Acked Read 9 Done Sent Done false
9381 3211 Acked Write 3 Done Sent Done false
9382 3212 Acked Read 1 Done Sent Done false
9383 3213 Acked Flush 0 Done Sent Done false
9384 3214 Acked Read 3 Done Sent Done false
9385 3215 Acked Flush 0 Done Sent Done false
9386 3216 Acked Read 10 Done Sent Done false
9387 3217 Acked Write 6 Done Sent Done false
9388 3218 Acked Read 10 Done Sent Done false
9389 3219 Acked Read 1 Done Sent Done false
9390 3220 Acked Write 2 Done Sent Done false
9391 3221 Acked Write 3 Done Sent Done false
9392 3222 Acked Read 2 Done Sent Done false
9393 3223 Acked Read 7 Done Sent Done false
9394 3224 Acked Read 1 Done Sent Done false
9395 3225 Acked Read 5 Done Sent Done false
9396 3226 Acked Read 1 Done Sent Done false
9397 3227 Acked Read 1 Done Sent Done false
9398 3228 Acked Read 5 Done Sent Done false
9399 3229 Acked Read 5 Done Sent Done false
9400 3230 Acked Read 3 Done Sent Done false
9401 3231 Acked Write 5 Done Sent Done false
9402 3232 Acked Read 3 Done Sent Done false
9403 3233 Acked Flush 0 Done Sent Done false
9404 3234 Acked Read 3 Done Sent Done false
9405 3235 Acked Read 1 Done Sent Done false
9406 3236 Acked Write 4 Done Sent Done false
9407 3237 Acked Read 2 Done Sent Done false
9408 3238 Acked Write 5 Done Sent Done false
9409 3239 Acked Write 8 Done Sent Done false
9410 3240 Acked Read 10 Done Sent Done false
9411 3241 Acked Write 10 Done Sent Done false
9412 3242 Acked Write 6 Done Sent Done false
9413 3243 Acked Read 3 Done Sent Done false
9414 3244 Acked Write 1 Done Sent Done false
9415 3245 Acked Read 5 Done Sent Done false
9416 3246 Acked Write 1 Done Sent Done false
9417 3247 Acked Read 4 Done Sent Done false
9418 3248 Acked Write 10 Done Sent Done false
9419 3249 Acked Read 3 Done Sent Done false
9420 3250 Acked Write 9 Done Sent Done false
9421 3251 Acked Flush 0 Done Sent Done false
9422 3252 Acked Write 9 Done Sent Done false
9423 3253 Acked Write 10 Done Sent Done false
9424 3254 Acked Read 3 Done Sent Done false
9425 3255 Acked Write 4 Done Sent Done false
9426 3256 Acked Write 2 Done Sent Done false
9427 3257 Acked Read 1 Done Sent Done false
9428 3258 Acked Write 6 Done Sent Done false
9429 3259 Acked Read 8 Done Sent Done false
9430 3260 Acked Read 6 Done Sent Done false
9431 3261 Acked Write 4 Done Sent Done false
9432 3262 Acked Write 2 Done Sent Done false
9433 3263 Acked Read 5 Done Sent Done false
9434 3264 Acked Write 5 Done Sent Done false
9435 3265 Acked Read 9 Done Sent Done false
9436 3266 Acked Read 5 Done Sent Done false
9437 3267 Acked Read 3 Done Sent Done false
9438 3268 Acked Write 4 Done Sent Done false
9439 3269 Acked Read 1 Done Sent Done false
9440 3270 Acked Write 2 Done Sent Done false
9441 3271 Acked Write 7 Done Sent Done false
9442 3272 Acked Write 10 Done Sent Done false
9443 3273 Acked Read 5 Done Sent Done false
9444 3274 Acked Read 7 Done Sent Done false
9445 3275 Acked Read 1 Done Sent Done false
9446 3276 Acked Read 6 Done Sent Done false
9447 3277 Acked Write 8 Done Sent Done false
9448 3278 Acked Write 7 Done Sent Done false
9449 3279 Acked Read 9 Done Sent Done false
9450 3280 Acked Read 5 Done Sent Done false
9451 3281 Acked Write 10 Done Sent Done false
9452 3282 Acked Read 3 Done Sent Done false
9453 3283 Acked Read 8 Done Sent Done false
9454 3284 Acked Read 1 Done Sent Done false
9455 3285 Acked Write 7 Done Sent Done false
9456 3286 Acked Write 1 Done Sent Done false
9457 3287 Acked Write 5 Done Sent Done false
9458 3288 Acked Read 3 Done Sent Done false
9459 3289 Acked Read 1 Done Sent Done false
9460 3290 Acked Read 7 Done Sent Done false
9461 3291 Acked Write 3 Done Sent Done false
9462 3292 Acked Read 5 Done Sent Done false
9463 3293 Acked Read 2 Done Sent Done false
9464 3294 Acked Read 9 Done Sent Done false
9465 3295 Acked Write 9 Done Sent Done false
9466 3296 Acked Flush 0 Done Sent Done false
9467 3297 Acked Read 8 Done Sent Done false
9468 3298 Acked Read 9 Done Sent Done false
9469 3299 Acked Write 7 Done Sent Done false
9470 3300 Acked Read 4 Done Sent Done false
9471 3301 Acked Read 6 Done Sent Done false
9472 3302 Acked Write 9 Done Sent Done false
9473 3303 Acked Read 7 Done Sent Done false
9474 3304 Acked Write 8 Done Sent Done false
9475 3305 Acked Read 4 Done Sent Done false
9476 3306 Acked Read 5 Done Sent Done false
9477 3307 Acked Write 4 Done Sent Done false
9478 3308 Acked Flush 0 Done Sent Done false
9479 3309 Acked Write 7 Done Sent Done false
9480 3310 Acked Write 9 Done Sent Done false
9481 3311 Acked Read 7 Done Sent Done false
9482 3312 Acked Write 1 Done Sent Done false
9483 3313 Acked Read 5 Done Sent Done false
9484 3314 Acked Read 6 Done Sent Done false
9485 3315 Acked Write 7 Done Sent Done false
9486 3316 Acked Flush 0 Done Sent Done false
9487 3317 Acked Write 9 Done Sent Done false
9488 3318 Acked Write 10 Done Sent Done false
9489 3319 Acked Flush 0 Done Sent Done false
9490 3320 Acked Read 5 Done Sent Done false
9491 3321 Acked Flush 0 Done Sent Done false
9492 3322 Acked Write 3 Done Sent Done false
9493 3323 Acked Write 1 Done Sent Done false
9494 3324 Acked Write 8 Done Sent Done false
9495 3325 Acked Write 6 Done Sent Done false
9496 3326 Acked Read 5 Done Sent Done false
9497 3327 Acked Write 10 Done Sent Done false
9498 3328 Acked Read 4 Done Sent Done false
9499 3329 Acked Read 5 Done Sent Done false
9500 3330 Acked Read 9 Done Sent Done false
9501 3331 Acked Write 1 Done Sent Done false
9502 3332 Acked Read 1 Done Sent Done false
9503 3333 Acked Flush 0 Done Sent Done false
9504 3334 Acked Read 8 Done Sent Done false
9505 3335 Acked Read 4 Done Sent Done false
9506 3336 Acked Read 2 Done Sent Done false
9507 3337 Acked Write 4 Done Sent Done false
9508 3338 Acked Read 3 Done Sent Done false
9509 3339 Acked Read 1 Done Sent Done false
9510 3340 Acked Read 10 Done Sent Done false
9511 3341 Acked Read 10 Done Sent Done false
9512 3342 Acked Read 8 Done Sent Done false
9513 3343 Acked Read 4 Done Sent Done false
9514 3344 Acked Read 7 Done Sent Done false
9515 3345 Acked Write 8 Done Sent Done false
9516 3346 Acked Write 5 Done Sent Done false
9517 3347 Acked Read 8 Done Sent Done false
9518 3348 Acked Read 9 Done Sent Done false
9519 3349 Acked Write 8 Done Sent Done false
9520 3350 Acked Read 6 Done Sent Done false
9521 3351 Acked Read 2 Done Sent Done false
9522 3352 Acked Write 4 Done Sent Done false
9523 3353 Acked Flush 0 Done Sent Done false
9524 3354 Acked Write 7 Done Sent Done false
9525 3355 Acked Read 9 Done Sent Done false
9526 3356 Acked Write 6 Done Sent Done false
9527 3357 Acked Flush 0 Done Sent Done false
9528 3358 Acked Flush 0 Done Sent Done false
9529 3359 Acked Write 2 Done Sent Done false
9530 3360 Acked Write 10 Done Sent Done false
9531 3361 Acked Read 1 Done Sent Done false
9532 3362 Acked Write 3 Done Sent Done false
9533 3363 Acked Flush 0 Done Sent Done false
9534 3364 Acked Write 10 Done Sent Done false
9535 3365 Acked Flush 0 Done Sent Done false
9536 3366 Acked Write 7 Done Sent Done false
9537 3367 Acked Read 6 Done Sent Done false
9538 3368 Acked Write 2 Done Sent Done false
9539 3369 Acked Read 10 Done Sent Done false
9540 3370 Acked Read 8 Done Sent Done false
9541 3371 Acked Flush 0 Done Sent Done false
9542 3372 Acked Write 4 Done Sent Done false
9543 3373 Acked Read 7 Done Sent Done false
9544 3374 Acked Read 5 Done Sent Done false
9545 3375 Acked Write 4 Done Sent Done false
9546 3376 Acked Read 2 Done Sent Done false
9547 3377 Acked Write 8 Done Sent Done false
9548 3378 Acked Read 1 Done Sent Done false
9549 3379 Acked Read 1 Done Sent Done false
9550 3380 Acked Read 10 Done Sent Done false
9551 3381 Acked Write 3 Done Sent Done false
9552 3382 Acked Read 9 Done Sent Done false
9553 3383 Acked Read 6 Done Sent Done false
9554 3384 Acked Read 3 Done Sent Done false
9555 3385 Acked Write 1 Done Sent Done false
9556 3386 Acked Write 7 Done Sent Done false
9557 3387 Acked Write 9 Done Sent Done false
9558 3388 Acked Read 3 Done Sent Done false
9559 3389 Acked Read 4 Done Sent Done false
9560 3390 Acked Read 7 Done Sent Done false
9561 3391 Acked Write 5 Done Sent Done false
9562 3392 Acked Write 5 Done Sent Done false
9563 3393 Acked Read 6 Done Sent Done false
9564 3394 Acked Read 4 Done Sent Done false
9565 3395 Acked Write 9 Done Sent Done false
9566 3396 Acked Read 10 Done Sent Done false
9567 3397 Acked Write 2 Done Sent Done false
9568 3398 Acked Write 8 Done Sent Done false
9569 3399 Acked Write 9 Done Sent Done false
9570 3400 Acked Flush 0 Done Sent Done false
9571 3401 Acked Write 1 Done Sent Done false
9572 3402 Acked Read 9 Done Sent Done false
9573 3403 Acked Flush 0 Done Sent Done false
9574 3404 Acked Read 3 Done Sent Done false
9575 3405 Acked Read 9 Done Sent Done false
9576 3406 Acked Flush 0 Done Sent Done false
9577 3407 Acked Write 1 Done Sent Done false
9578 3408 Acked Write 7 Done Sent Done false
9579 3409 Acked Write 8 Done Sent Done false
9580 3410 Acked Write 1 Done Sent Done false
9581 3411 Acked Write 4 Done Sent Done false
9582 3412 Acked Read 5 Done Sent Done false
9583 STATES DS:0 DS:1 DS:2 TOTAL
9584 Sent 0 301 0 301
9585 Done 301 0 301 602
9586 Skipped 0 0 0 0
9587 Error 0 0 0 0
9588 Last Flush: 3406 3111 3406
9589 Downstairs last five completed: 3111 3110 3109 3108 3107
9590 Upstairs last five completed: 3412 3411 3410 3409 3408
9591 CLIENT: Up:0 ds:301 act:2
9592 ----------------------------------------------------------------
9593 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
9594 States: Active Offline Active
9595 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
9596 3112 Acked Read 4 Done Sent Done false
9597 3113 Acked Flush 0 Done Sent Done false
9598 3114 Acked Read 5 Done Sent Done false
9599 3115 Acked Write 10 Done Sent Done false
9600 3116 Acked Read 3 Done Sent Done false
9601 3117 Acked Read 3 Done Sent Done false
9602 3118 Acked Write 3 Done Sent Done false
9603 3119 Acked Write 7 Done Sent Done false
9604 3120 Acked Write 3 Done Sent Done false
9605 3121 Acked Read 7 Done Sent Done false
9606 3122 Acked Write 1 Done Sent Done false
9607 3123 Acked Read 7 Done Sent Done false
9608 3124 Acked Read 10 Done Sent Done false
9609 3125 Acked Flush 0 Done Sent Done false
9610 3126 Acked Read 9 Done Sent Done false
9611 3127 Acked Write 10 Done Sent Done false
9612 3128 Acked Read 2 Done Sent Done false
9613 3129 Acked Flush 0 Done Sent Done false
9614 3130 Acked Write 9 Done Sent Done false
9615 3131 Acked Read 9 Done Sent Done false
9616 3132 Acked Write 5 Done Sent Done false
9617 3133 Acked Read 5 Done Sent Done false
9618 3134 Acked Write 5 Done Sent Done false
9619 3135 Acked Read 1 Done Sent Done false
9620 3136 Acked Write 3 Done Sent Done false
9621 3137 Acked Write 3 Done Sent Done false
9622 3138 Acked Read 5 Done Sent Done false
9623 3139 Acked Read 7 Done Sent Done false
9624 3140 Acked Write 3 Done Sent Done false
9625 3141 Acked Read 10 Done Sent Done false
9626 3142 Acked Read 5 Done Sent Done false
9627 3143 Acked Read 4 Done Sent Done false
9628 3144 Acked Flush 0 Done Sent Done false
9629 3145 Acked Read 4 Done Sent Done false
9630 3146 Acked Write 5 Done Sent Done false
9631 3147 Acked Write 4 Done Sent Done false
9632 3148 Acked Write 4 Done Sent Done false
9633 3149 Acked Write 4 Done Sent Done false
9634 3150 Acked Write 2 Done Sent Done false
9635 3151 Acked Read 7 Done Sent Done false
9636 3152 Acked Write 5 Done Sent Done false
9637 3153 Acked Read 9 Done Sent Done false
9638 3154 Acked Read 7 Done Sent Done false
9639 3155 Acked Read 8 Done Sent Done false
9640 3156 Acked Write 7 Done Sent Done false
9641 3157 Acked Read 8 Done Sent Done false
9642 3158 Acked Write 9 Done Sent Done false
9643 3159 Acked Write 6 Done Sent Done false
9644 3160 Acked Read 10 Done Sent Done false
9645 3161 Acked Read 8 Done Sent Done false
9646 3162 Acked Write 4 Done Sent Done false
9647 3163 Acked Read 10 Done Sent Done false
9648 3164 Acked Write 4 Done Sent Done false
9649 3165 Acked Read 1 Done Sent Done false
9650 3166 Acked Write 4 Done Sent Done false
9651 3167 Acked Read 8 Done Sent Done false
9652 3168 Acked Write 7 Done Sent Done false
9653 3169 Acked Read 4 Done Sent Done false
9654 3170 Acked Read 10 Done Sent Done false
9655 3171 Acked Flush 0 Done Sent Done false
9656 3172 Acked Flush 0 Done Sent Done false
9657 3173 Acked Read 4 Done Sent Done false
9658 3174 Acked Read 10 Done Sent Done false
9659 3175 Acked Read 6 Done Sent Done false
9660 3176 Acked Read 7 Done Sent Done false
9661 3177 Acked Write 3 Done Sent Done false
9662 3178 Acked Read 8 Done Sent Done false
9663 3179 Acked Write 1 Done Sent Done false
9664 3180 Acked Read 1 Done Sent Done false
9665 3181 Acked Write 10 Done Sent Done false
9666 3182 Acked Read 6 Done Sent Done false
9667 3183 Acked Write 7 Done Sent Done false
9668 3184 Acked Write 2 Done Sent Done false
9669 3185 Acked Write 10 Done Sent Done false
9670 3186 Acked Read 3 Done Sent Done false
9671 3187 Acked Read 5 Done Sent Done false
9672 3188 Acked Read 3 Done Sent Done false
9673 3189 Acked Write 4 Done Sent Done false
9674 3190 Acked Read 10 Done Sent Done false
9675 3191 Acked Flush 0 Done Sent Done false
9676 3192 Acked Write 7 Done Sent Done false
9677 3193 Acked Write 9 Done Sent Done false
9678 3194 Acked Write 10 Done Sent Done false
9679 3195 Acked Read 8 Done Sent Done false
9680 3196 Acked Read 7 Done Sent Done false
9681 3197 Acked Read 10 Done Sent Done false
9682 3198 Acked Read 1 Done Sent Done false
9683 3199 Acked Read 10 Done Sent Done false
9684 3200 Acked Write 7 Done Sent Done false
9685 3201 Acked Write 9 Done Sent Done false
9686 3202 Acked Flush 0 Done Sent Done false
9687 3203 Acked Read 9 Done Sent Done false
9688 3204 Acked Write 9 Done Sent Done false
9689 3205 Acked Write 6 Done Sent Done false
9690 3206 Acked Flush 0 Done Sent Done false
9691 3207 Acked Read 3 Done Sent Done false
9692 3208 Acked Write 7 Done Sent Done false
9693 3209 Acked Read 7 Done Sent Done false
9694 3210 Acked Read 9 Done Sent Done false
9695 3211 Acked Write 3 Done Sent Done false
9696 3212 Acked Read 1 Done Sent Done false
9697 3213 Acked Flush 0 Done Sent Done false
9698 3214 Acked Read 3 Done Sent Done false
9699 3215 Acked Flush 0 Done Sent Done false
9700 3216 Acked Read 10 Done Sent Done false
9701 3217 Acked Write 6 Done Sent Done false
9702 3218 Acked Read 10 Done Sent Done false
9703 3219 Acked Read 1 Done Sent Done false
9704 3220 Acked Write 2 Done Sent Done false
9705 3221 Acked Write 3 Done Sent Done false
9706 3222 Acked Read 2 Done Sent Done false
9707 3223 Acked Read 7 Done Sent Done false
9708 3224 Acked Read 1 Done Sent Done false
9709 3225 Acked Read 5 Done Sent Done false
9710 3226 Acked Read 1 Done Sent Done false
9711 3227 Acked Read 1 Done Sent Done false
9712 3228 Acked Read 5 Done Sent Done false
9713 3229 Acked Read 5 Done Sent Done false
9714 3230 Acked Read 3 Done Sent Done false
9715 3231 Acked Write 5 Done Sent Done false
9716 3232 Acked Read 3 Done Sent Done false
9717 3233 Acked Flush 0 Done Sent Done false
9718 3234 Acked Read 3 Done Sent Done false
9719 3235 Acked Read 1 Done Sent Done false
9720 3236 Acked Write 4 Done Sent Done false
9721 3237 Acked Read 2 Done Sent Done false
9722 3238 Acked Write 5 Done Sent Done false
9723 3239 Acked Write 8 Done Sent Done false
9724 3240 Acked Read 10 Done Sent Done false
9725 3241 Acked Write 10 Done Sent Done false
9726 3242 Acked Write 6 Done Sent Done false
9727 3243 Acked Read 3 Done Sent Done false
9728 3244 Acked Write 1 Done Sent Done false
9729 3245 Acked Read 5 Done Sent Done false
9730 3246 Acked Write 1 Done Sent Done false
9731 3247 Acked Read 4 Done Sent Done false
9732 3248 Acked Write 10 Done Sent Done false
9733 3249 Acked Read 3 Done Sent Done false
9734 3250 Acked Write 9 Done Sent Done false
9735 3251 Acked Flush 0 Done Sent Done false
9736 3252 Acked Write 9 Done Sent Done false
9737 3253 Acked Write 10 Done Sent Done false
9738 3254 Acked Read 3 Done Sent Done false
9739 3255 Acked Write 4 Done Sent Done false
9740 3256 Acked Write 2 Done Sent Done false
9741 3257 Acked Read 1 Done Sent Done false
9742 3258 Acked Write 6 Done Sent Done false
9743 3259 Acked Read 8 Done Sent Done false
9744 3260 Acked Read 6 Done Sent Done false
9745 3261 Acked Write 4 Done Sent Done false
9746 3262 Acked Write 2 Done Sent Done false
9747 3263 Acked Read 5 Done Sent Done false
9748 3264 Acked Write 5 Done Sent Done false
9749 3265 Acked Read 9 Done Sent Done false
9750 3266 Acked Read 5 Done Sent Done false
9751 3267 Acked Read 3 Done Sent Done false
9752 3268 Acked Write 4 Done Sent Done false
9753 3269 Acked Read 1 Done Sent Done false
9754 3270 Acked Write 2 Done Sent Done false
9755 3271 Acked Write 7 Done Sent Done false
9756 3272 Acked Write 10 Done Sent Done false
9757 3273 Acked Read 5 Done Sent Done false
9758 3274 Acked Read 7 Done Sent Done false
9759 3275 Acked Read 1 Done Sent Done false
9760 3276 Acked Read 6 Done Sent Done false
9761 3277 Acked Write 8 Done Sent Done false
9762 3278 Acked Write 7 Done Sent Done false
9763 3279 Acked Read 9 Done Sent Done false
9764 3280 Acked Read 5 Done Sent Done false
9765 3281 Acked Write 10 Done Sent Done false
9766 3282 Acked Read 3 Done Sent Done false
9767 3283 Acked Read 8 Done Sent Done false
9768 3284 Acked Read 1 Done Sent Done false
9769 3285 Acked Write 7 Done Sent Done false
9770 3286 Acked Write 1 Done Sent Done false
9771 3287 Acked Write 5 Done Sent Done false
9772 3288 Acked Read 3 Done Sent Done false
9773 3289 Acked Read 1 Done Sent Done false
9774 3290 Acked Read 7 Done Sent Done false
9775 3291 Acked Write 3 Done Sent Done false
9776 3292 Acked Read 5 Done Sent Done false
9777 3293 Acked Read 2 Done Sent Done false
9778 3294 Acked Read 9 Done Sent Done false
9779 3295 Acked Write 9 Done Sent Done false
9780 3296 Acked Flush 0 Done Sent Done false
9781 3297 Acked Read 8 Done Sent Done false
9782 3298 Acked Read 9 Done Sent Done false
9783 3299 Acked Write 7 Done Sent Done false
9784 3300 Acked Read 4 Done Sent Done false
9785 3301 Acked Read 6 Done Sent Done false
9786 3302 Acked Write 9 Done Sent Done false
9787 3303 Acked Read 7 Done Sent Done false
9788 3304 Acked Write 8 Done Sent Done false
9789 3305 Acked Read 4 Done Sent Done false
9790 3306 Acked Read 5 Done Sent Done false
9791 3307 Acked Write 4 Done Sent Done false
9792 3308 Acked Flush 0 Done Sent Done false
9793 3309 Acked Write 7 Done Sent Done false
9794 3310 Acked Write 9 Done Sent Done false
9795 3311 Acked Read 7 Done Sent Done false
9796 3312 Acked Write 1 Done Sent Done false
9797 3313 Acked Read 5 Done Sent Done false
9798 3314 Acked Read 6 Done Sent Done false
9799 3315 Acked Write 7 Done Sent Done false
9800 3316 Acked Flush 0 Done Sent Done false
9801 3317 Acked Write 9 Done Sent Done false
9802 3318 Acked Write 10 Done Sent Done false
9803 3319 Acked Flush 0 Done Sent Done false
9804 3320 Acked Read 5 Done Sent Done false
9805 3321 Acked Flush 0 Done Sent Done false
9806 3322 Acked Write 3 Done Sent Done false
9807 3323 Acked Write 1 Done Sent Done false
9808 3324 Acked Write 8 Done Sent Done false
9809 3325 Acked Write 6 Done Sent Done false
9810 3326 Acked Read 5 Done Sent Done false
9811 3327 Acked Write 10 Done Sent Done false
9812 3328 Acked Read 4 Done Sent Done false
9813 3329 Acked Read 5 Done Sent Done false
9814 3330 Acked Read 9 Done Sent Done false
9815 3331 Acked Write 1 Done Sent Done false
9816 3332 Acked Read 1 Done Sent Done false
9817 3333 Acked Flush 0 Done Sent Done false
9818 3334 Acked Read 8 Done Sent Done false
9819 3335 Acked Read 4 Done Sent Done false
9820 3336 Acked Read 2 Done Sent Done false
9821 3337 Acked Write 4 Done Sent Done false
9822 3338 Acked Read 3 Done Sent Done false
9823 3339 Acked Read 1 Done Sent Done false
9824 3340 Acked Read 10 Done Sent Done false
9825 3341 Acked Read 10 Done Sent Done false
9826 3342 Acked Read 8 Done Sent Done false
9827 3343 Acked Read 4 Done Sent Done false
9828 3344 Acked Read 7 Done Sent Done false
9829 3345 Acked Write 8 Done Sent Done false
9830 3346 Acked Write 5 Done Sent Done false
9831 3347 Acked Read 8 Done Sent Done false
9832 3348 Acked Read 9 Done Sent Done false
9833 3349 Acked Write 8 Done Sent Done false
9834 3350 Acked Read 6 Done Sent Done false
9835 3351 Acked Read 2 Done Sent Done false
9836 3352 Acked Write 4 Done Sent Done false
9837 3353 Acked Flush 0 Done Sent Done false
9838 3354 Acked Write 7 Done Sent Done false
9839 3355 Acked Read 9 Done Sent Done false
9840 3356 Acked Write 6 Done Sent Done false
9841 3357 Acked Flush 0 Done Sent Done false
9842 3358 Acked Flush 0 Done Sent Done false
9843 3359 Acked Write 2 Done Sent Done false
9844 3360 Acked Write 10 Done Sent Done false
9845 3361 Acked Read 1 Done Sent Done false
9846 3362 Acked Write 3 Done Sent Done false
9847 3363 Acked Flush 0 Done Sent Done false
9848 3364 Acked Write 10 Done Sent Done false
9849 3365 Acked Flush 0 Done Sent Done false
9850 3366 Acked Write 7 Done Sent Done false
9851 3367 Acked Read 6 Done Sent Done false
9852 3368 Acked Write 2 Done Sent Done false
9853 3369 Acked Read 10 Done Sent Done false
9854 3370 Acked Read 8 Done Sent Done false
9855 3371 Acked Flush 0 Done Sent Done false
9856 3372 Acked Write 4 Done Sent Done false
9857 3373 Acked Read 7 Done Sent Done false
9858 3374 Acked Read 5 Done Sent Done false
9859 3375 Acked Write 4 Done Sent Done false
9860 3376 Acked Read 2 Done Sent Done false
9861 3377 Acked Write 8 Done Sent Done false
9862 3378 Acked Read 1 Done Sent Done false
9863 3379 Acked Read 1 Done Sent Done false
9864 3380 Acked Read 10 Done Sent Done false
9865 3381 Acked Write 3 Done Sent Done false
9866 3382 Acked Read 9 Done Sent Done false
9867 3383 Acked Read 6 Done Sent Done false
9868 3384 Acked Read 3 Done Sent Done false
9869 3385 Acked Write 1 Done Sent Done false
9870 3386 Acked Write 7 Done Sent Done false
9871 3387 Acked Write 9 Done Sent Done false
9872 3388 Acked Read 3 Done Sent Done false
9873 3389 Acked Read 4 Done Sent Done false
9874 3390 Acked Read 7 Done Sent Done false
9875 3391 Acked Write 5 Done Sent Done false
9876 3392 Acked Write 5 Done Sent Done false
9877 3393 Acked Read 6 Done Sent Done false
9878 3394 Acked Read 4 Done Sent Done false
9879 3395 Acked Write 9 Done Sent Done false
9880 3396 Acked Read 10 Done Sent Done false
9881 3397 Acked Write 2 Done Sent Done false
9882 3398 Acked Write 8 Done Sent Done false
9883 3399 Acked Write 9 Done Sent Done false
9884 3400 Acked Flush 0 Done Sent Done false
9885 3401 Acked Write 1 Done Sent Done false
9886 3402 Acked Read 9 Done Sent Done false
9887 3403 Acked Flush 0 Done Sent Done false
9888 3404 Acked Read 3 Done Sent Done false
9889 3405 Acked Read 9 Done Sent Done false
9890 3406 Acked Flush 0 Done Sent Done false
9891 3407 Acked Write 1 Done Sent Done false
9892 3408 Acked Write 7 Done Sent Done false
9893 3409 Acked Write 8 Done Sent Done false
9894 3410 Acked Write 1 Done Sent Done false
9895 3411 Acked Write 4 Done Sent Done false
9896 3412 Acked Read 5 Done Sent Done false
9897 3413 Acked Flush 0 Done Sent Done false
9898 STATES DS:0 DS:1 DS:2 TOTAL
9899 Sent 0 302 0 302
9900 Done 302 0 302 604
9901 Skipped 0 0 0 0
9902 Error 0 0 0 0
9903 Last Flush: 3413 3111 3413
9904 Downstairs last five completed: 3111 3110 3109 3108 3107
9905 Upstairs last five completed: 3413 3412 3411 3410 3409
9906 CLIENT: Up:0 ds:302 act:2
99072024-11-15T01:59:31.587ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
99082024-11-15T01:59:31.587ZWARNcrucible: ds_connection connect to 127.0.0.1:8820 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
99092024-11-15T01:59:31.587ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
99102024-11-15T01:59:31.587ZWARNcrucible: downstairs task for 1 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
99112024-11-15T01:59:31.587ZINFOcrucible: client task is exiting = io task client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
9912 [8820] got start action
9913 [8820] I am going to restart
9914 Starting downstairs at port 8820
9915 Make output file at "/tmp/dsc/downstairs-8820.txt"
9916 [8820][1] reports Starting
9917 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1213)
9918 [8820][1] reports Running
9919 ----------------------------------------------------------------
9920 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
9921 States: Active Offline Active
9922 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
9923 3112 Acked Read 4 Done Sent Done false
9924 3113 Acked Flush 0 Done Sent Done false
9925 3114 Acked Read 5 Done Sent Done false
9926 3115 Acked Write 10 Done Sent Done false
9927 3116 Acked Read 3 Done Sent Done false
9928 3117 Acked Read 3 Done Sent Done false
9929 3118 Acked Write 3 Done Sent Done false
9930 3119 Acked Write 7 Done Sent Done false
9931 3120 Acked Write 3 Done Sent Done false
9932 3121 Acked Read 7 Done Sent Done false
9933 3122 Acked Write 1 Done Sent Done false
9934 3123 Acked Read 7 Done Sent Done false
9935 3124 Acked Read 10 Done Sent Done false
9936 3125 Acked Flush 0 Done Sent Done false
9937 3126 Acked Read 9 Done Sent Done false
9938 3127 Acked Write 10 Done Sent Done false
9939 3128 Acked Read 2 Done Sent Done false
9940 3129 Acked Flush 0 Done Sent Done false
9941 3130 Acked Write 9 Done Sent Done false
9942 3131 Acked Read 9 Done Sent Done false
9943 3132 Acked Write 5 Done Sent Done false
9944 3133 Acked Read 5 Done Sent Done false
9945 3134 Acked Write 5 Done Sent Done false
9946 3135 Acked Read 1 Done Sent Done false
9947 3136 Acked Write 3 Done Sent Done false
9948 3137 Acked Write 3 Done Sent Done false
9949 3138 Acked Read 5 Done Sent Done false
9950 3139 Acked Read 7 Done Sent Done false
9951 3140 Acked Write 3 Done Sent Done false
9952 3141 Acked Read 10 Done Sent Done false
9953 3142 Acked Read 5 Done Sent Done false
9954 3143 Acked Read 4 Done Sent Done false
9955 3144 Acked Flush 0 Done Sent Done false
9956 3145 Acked Read 4 Done Sent Done false
9957 3146 Acked Write 5 Done Sent Done false
9958 3147 Acked Write 4 Done Sent Done false
9959 3148 Acked Write 4 Done Sent Done false
9960 3149 Acked Write 4 Done Sent Done false
9961 3150 Acked Write 2 Done Sent Done false
9962 3151 Acked Read 7 Done Sent Done false
9963 3152 Acked Write 5 Done Sent Done false
9964 3153 Acked Read 9 Done Sent Done false
9965 3154 Acked Read 7 Done Sent Done false
9966 3155 Acked Read 8 Done Sent Done false
9967 3156 Acked Write 7 Done Sent Done false
9968 3157 Acked Read 8 Done Sent Done false
9969 3158 Acked Write 9 Done Sent Done false
9970 3159 Acked Write 6 Done Sent Done false
9971 3160 Acked Read 10 Done Sent Done false
9972 3161 Acked Read 8 Done Sent Done false
9973 3162 Acked Write 4 Done Sent Done false
9974 3163 Acked Read 10 Done Sent Done false
9975 3164 Acked Write 4 Done Sent Done false
9976 3165 Acked Read 1 Done Sent Done false
9977 3166 Acked Write 4 Done Sent Done false
9978 3167 Acked Read 8 Done Sent Done false
9979 3168 Acked Write 7 Done Sent Done false
9980 3169 Acked Read 4 Done Sent Done false
9981 3170 Acked Read 10 Done Sent Done false
9982 3171 Acked Flush 0 Done Sent Done false
9983 3172 Acked Flush 0 Done Sent Done false
9984 3173 Acked Read 4 Done Sent Done false
9985 3174 Acked Read 10 Done Sent Done false
9986 3175 Acked Read 6 Done Sent Done false
9987 3176 Acked Read 7 Done Sent Done false
9988 3177 Acked Write 3 Done Sent Done false
9989 3178 Acked Read 8 Done Sent Done false
9990 3179 Acked Write 1 Done Sent Done false
9991 3180 Acked Read 1 Done Sent Done false
9992 3181 Acked Write 10 Done Sent Done false
9993 3182 Acked Read 6 Done Sent Done false
9994 3183 Acked Write 7 Done Sent Done false
9995 3184 Acked Write 2 Done Sent Done false
9996 3185 Acked Write 10 Done Sent Done false
9997 3186 Acked Read 3 Done Sent Done false
9998 3187 Acked Read 5 Done Sent Done false
9999 3188 Acked Read 3 Done Sent Done false
10000 3189 Acked Write 4 Done Sent Done false
10001 3190 Acked Read 10 Done Sent Done false
10002 3191 Acked Flush 0 Done Sent Done false
10003 3192 Acked Write 7 Done Sent Done false
10004 3193 Acked Write 9 Done Sent Done false
10005 3194 Acked Write 10 Done Sent Done false
10006 3195 Acked Read 8 Done Sent Done false
10007 3196 Acked Read 7 Done Sent Done false
10008 3197 Acked Read 10 Done Sent Done false
10009 3198 Acked Read 1 Done Sent Done false
10010 3199 Acked Read 10 Done Sent Done false
10011 3200 Acked Write 7 Done Sent Done false
10012 3201 Acked Write 9 Done Sent Done false
10013 3202 Acked Flush 0 Done Sent Done false
10014 3203 Acked Read 9 Done Sent Done false
10015 3204 Acked Write 9 Done Sent Done false
10016 3205 Acked Write 6 Done Sent Done false
10017 3206 Acked Flush 0 Done Sent Done false
10018 3207 Acked Read 3 Done Sent Done false
10019 3208 Acked Write 7 Done Sent Done false
10020 3209 Acked Read 7 Done Sent Done false
10021 3210 Acked Read 9 Done Sent Done false
10022 3211 Acked Write 3 Done Sent Done false
10023 3212 Acked Read 1 Done Sent Done false
10024 3213 Acked Flush 0 Done Sent Done false
10025 3214 Acked Read 3 Done Sent Done false
10026 3215 Acked Flush 0 Done Sent Done false
10027 3216 Acked Read 10 Done Sent Done false
10028 3217 Acked Write 6 Done Sent Done false
10029 3218 Acked Read 10 Done Sent Done false
10030 3219 Acked Read 1 Done Sent Done false
10031 3220 Acked Write 2 Done Sent Done false
10032 3221 Acked Write 3 Done Sent Done false
10033 3222 Acked Read 2 Done Sent Done false
10034 3223 Acked Read 7 Done Sent Done false
10035 3224 Acked Read 1 Done Sent Done false
10036 3225 Acked Read 5 Done Sent Done false
10037 3226 Acked Read 1 Done Sent Done false
10038 3227 Acked Read 1 Done Sent Done false
10039 3228 Acked Read 5 Done Sent Done false
10040 3229 Acked Read 5 Done Sent Done false
10041 3230 Acked Read 3 Done Sent Done false
10042 3231 Acked Write 5 Done Sent Done false
10043 3232 Acked Read 3 Done Sent Done false
10044 3233 Acked Flush 0 Done Sent Done false
10045 3234 Acked Read 3 Done Sent Done false
10046 3235 Acked Read 1 Done Sent Done false
10047 3236 Acked Write 4 Done Sent Done false
10048 3237 Acked Read 2 Done Sent Done false
10049 3238 Acked Write 5 Done Sent Done false
10050 3239 Acked Write 8 Done Sent Done false
10051 3240 Acked Read 10 Done Sent Done false
10052 3241 Acked Write 10 Done Sent Done false
10053 3242 Acked Write 6 Done Sent Done false
10054 3243 Acked Read 3 Done Sent Done false
10055 3244 Acked Write 1 Done Sent Done false
10056 3245 Acked Read 5 Done Sent Done false
10057 3246 Acked Write 1 Done Sent Done false
10058 3247 Acked Read 4 Done Sent Done false
10059 3248 Acked Write 10 Done Sent Done false
10060 3249 Acked Read 3 Done Sent Done false
10061 3250 Acked Write 9 Done Sent Done false
10062 3251 Acked Flush 0 Done Sent Done false
10063 3252 Acked Write 9 Done Sent Done false
10064 3253 Acked Write 10 Done Sent Done false
10065 3254 Acked Read 3 Done Sent Done false
10066 3255 Acked Write 4 Done Sent Done false
10067 3256 Acked Write 2 Done Sent Done false
10068 3257 Acked Read 1 Done Sent Done false
10069 3258 Acked Write 6 Done Sent Done false
10070 3259 Acked Read 8 Done Sent Done false
10071 3260 Acked Read 6 Done Sent Done false
10072 3261 Acked Write 4 Done Sent Done false
10073 3262 Acked Write 2 Done Sent Done false
10074 3263 Acked Read 5 Done Sent Done false
10075 3264 Acked Write 5 Done Sent Done false
10076 3265 Acked Read 9 Done Sent Done false
10077 3266 Acked Read 5 Done Sent Done false
10078 3267 Acked Read 3 Done Sent Done false
10079 3268 Acked Write 4 Done Sent Done false
10080 3269 Acked Read 1 Done Sent Done false
10081 3270 Acked Write 2 Done Sent Done false
10082 3271 Acked Write 7 Done Sent Done false
10083 3272 Acked Write 10 Done Sent Done false
10084 3273 Acked Read 5 Done Sent Done false
10085 3274 Acked Read 7 Done Sent Done false
10086 3275 Acked Read 1 Done Sent Done false
10087 3276 Acked Read 6 Done Sent Done false
10088 3277 Acked Write 8 Done Sent Done false
10089 3278 Acked Write 7 Done Sent Done false
10090 3279 Acked Read 9 Done Sent Done false
10091 3280 Acked Read 5 Done Sent Done false
10092 3281 Acked Write 10 Done Sent Done false
10093 3282 Acked Read 3 Done Sent Done false
10094 3283 Acked Read 8 Done Sent Done false
10095 3284 Acked Read 1 Done Sent Done false
10096 3285 Acked Write 7 Done Sent Done false
10097 3286 Acked Write 1 Done Sent Done false
10098 3287 Acked Write 5 Done Sent Done false
10099 3288 Acked Read 3 Done Sent Done false
10100 3289 Acked Read 1 Done Sent Done false
10101 3290 Acked Read 7 Done Sent Done false
10102 3291 Acked Write 3 Done Sent Done false
10103 3292 Acked Read 5 Done Sent Done false
10104 3293 Acked Read 2 Done Sent Done false
10105 3294 Acked Read 9 Done Sent Done false
10106 3295 Acked Write 9 Done Sent Done false
10107 3296 Acked Flush 0 Done Sent Done false
10108 3297 Acked Read 8 Done Sent Done false
10109 3298 Acked Read 9 Done Sent Done false
10110 3299 Acked Write 7 Done Sent Done false
10111 3300 Acked Read 4 Done Sent Done false
10112 3301 Acked Read 6 Done Sent Done false
10113 3302 Acked Write 9 Done Sent Done false
10114 3303 Acked Read 7 Done Sent Done false
10115 3304 Acked Write 8 Done Sent Done false
10116 3305 Acked Read 4 Done Sent Done false
10117 3306 Acked Read 5 Done Sent Done false
10118 3307 Acked Write 4 Done Sent Done false
10119 3308 Acked Flush 0 Done Sent Done false
10120 3309 Acked Write 7 Done Sent Done false
10121 3310 Acked Write 9 Done Sent Done false
10122 3311 Acked Read 7 Done Sent Done false
10123 3312 Acked Write 1 Done Sent Done false
10124 3313 Acked Read 5 Done Sent Done false
10125 3314 Acked Read 6 Done Sent Done false
10126 3315 Acked Write 7 Done Sent Done false
10127 3316 Acked Flush 0 Done Sent Done false
10128 3317 Acked Write 9 Done Sent Done false
10129 3318 Acked Write 10 Done Sent Done false
10130 3319 Acked Flush 0 Done Sent Done false
10131 3320 Acked Read 5 Done Sent Done false
10132 3321 Acked Flush 0 Done Sent Done false
10133 3322 Acked Write 3 Done Sent Done false
10134 3323 Acked Write 1 Done Sent Done false
10135 3324 Acked Write 8 Done Sent Done false
10136 3325 Acked Write 6 Done Sent Done false
10137 3326 Acked Read 5 Done Sent Done false
10138 3327 Acked Write 10 Done Sent Done false
10139 3328 Acked Read 4 Done Sent Done false
10140 3329 Acked Read 5 Done Sent Done false
10141 3330 Acked Read 9 Done Sent Done false
10142 3331 Acked Write 1 Done Sent Done false
10143 3332 Acked Read 1 Done Sent Done false
10144 3333 Acked Flush 0 Done Sent Done false
10145 3334 Acked Read 8 Done Sent Done false
10146 3335 Acked Read 4 Done Sent Done false
10147 3336 Acked Read 2 Done Sent Done false
10148 3337 Acked Write 4 Done Sent Done false
10149 3338 Acked Read 3 Done Sent Done false
10150 3339 Acked Read 1 Done Sent Done false
10151 3340 Acked Read 10 Done Sent Done false
10152 3341 Acked Read 10 Done Sent Done false
10153 3342 Acked Read 8 Done Sent Done false
10154 3343 Acked Read 4 Done Sent Done false
10155 3344 Acked Read 7 Done Sent Done false
10156 3345 Acked Write 8 Done Sent Done false
10157 3346 Acked Write 5 Done Sent Done false
10158 3347 Acked Read 8 Done Sent Done false
10159 3348 Acked Read 9 Done Sent Done false
10160 3349 Acked Write 8 Done Sent Done false
10161 3350 Acked Read 6 Done Sent Done false
10162 3351 Acked Read 2 Done Sent Done false
10163 3352 Acked Write 4 Done Sent Done false
10164 3353 Acked Flush 0 Done Sent Done false
10165 3354 Acked Write 7 Done Sent Done false
10166 3355 Acked Read 9 Done Sent Done false
10167 3356 Acked Write 6 Done Sent Done false
10168 3357 Acked Flush 0 Done Sent Done false
10169 3358 Acked Flush 0 Done Sent Done false
10170 3359 Acked Write 2 Done Sent Done false
10171 3360 Acked Write 10 Done Sent Done false
10172 3361 Acked Read 1 Done Sent Done false
10173 3362 Acked Write 3 Done Sent Done false
10174 3363 Acked Flush 0 Done Sent Done false
10175 3364 Acked Write 10 Done Sent Done false
10176 3365 Acked Flush 0 Done Sent Done false
10177 3366 Acked Write 7 Done Sent Done false
10178 3367 Acked Read 6 Done Sent Done false
10179 3368 Acked Write 2 Done Sent Done false
10180 3369 Acked Read 10 Done Sent Done false
10181 3370 Acked Read 8 Done Sent Done false
10182 3371 Acked Flush 0 Done Sent Done false
10183 3372 Acked Write 4 Done Sent Done false
10184 3373 Acked Read 7 Done Sent Done false
10185 3374 Acked Read 5 Done Sent Done false
10186 3375 Acked Write 4 Done Sent Done false
10187 3376 Acked Read 2 Done Sent Done false
10188 3377 Acked Write 8 Done Sent Done false
10189 3378 Acked Read 1 Done Sent Done false
10190 3379 Acked Read 1 Done Sent Done false
10191 3380 Acked Read 10 Done Sent Done false
10192 3381 Acked Write 3 Done Sent Done false
10193 3382 Acked Read 9 Done Sent Done false
10194 3383 Acked Read 6 Done Sent Done false
10195 3384 Acked Read 3 Done Sent Done false
10196 3385 Acked Write 1 Done Sent Done false
10197 3386 Acked Write 7 Done Sent Done false
10198 3387 Acked Write 9 Done Sent Done false
10199 3388 Acked Read 3 Done Sent Done false
10200 3389 Acked Read 4 Done Sent Done false
10201 3390 Acked Read 7 Done Sent Done false
10202 3391 Acked Write 5 Done Sent Done false
10203 3392 Acked Write 5 Done Sent Done false
10204 3393 Acked Read 6 Done Sent Done false
10205 3394 Acked Read 4 Done Sent Done false
10206 3395 Acked Write 9 Done Sent Done false
10207 3396 Acked Read 10 Done Sent Done false
10208 3397 Acked Write 2 Done Sent Done false
10209 3398 Acked Write 8 Done Sent Done false
10210 3399 Acked Write 9 Done Sent Done false
10211 3400 Acked Flush 0 Done Sent Done false
10212 3401 Acked Write 1 Done Sent Done false
10213 3402 Acked Read 9 Done Sent Done false
10214 3403 Acked Flush 0 Done Sent Done false
10215 3404 Acked Read 3 Done Sent Done false
10216 3405 Acked Read 9 Done Sent Done false
10217 3406 Acked Flush 0 Done Sent Done false
10218 3407 Acked Write 1 Done Sent Done false
10219 3408 Acked Write 7 Done Sent Done false
10220 3409 Acked Write 8 Done Sent Done false
10221 3410 Acked Write 1 Done Sent Done false
10222 3411 Acked Write 4 Done Sent Done false
10223 3412 Acked Read 5 Done Sent Done false
10224 3413 Acked Flush 0 Done Sent Done false
10225 STATES DS:0 DS:1 DS:2 TOTAL
10226 Sent 0 302 0 302
10227 Done 302 0 302 604
10228 Skipped 0 0 0 0
10229 Error 0 0 0 0
10230 Last Flush: 3413 3111 3413
10231 Downstairs last five completed: 3111 3110 3109 3108 3107
10232 Upstairs last five completed: 3413 3412 3411 3410 3409
10233 CLIENT: Up:0 ds:302 act:2
10234 ----------------------------------------------------------------
10235 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
10236 States: Active Offline Active
10237 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
10238 3112 Acked Read 4 Done Sent Done false
10239 3113 Acked Flush 0 Done Sent Done false
10240 3114 Acked Read 5 Done Sent Done false
10241 3115 Acked Write 10 Done Sent Done false
10242 3116 Acked Read 3 Done Sent Done false
10243 3117 Acked Read 3 Done Sent Done false
10244 3118 Acked Write 3 Done Sent Done false
10245 3119 Acked Write 7 Done Sent Done false
10246 3120 Acked Write 3 Done Sent Done false
10247 3121 Acked Read 7 Done Sent Done false
10248 3122 Acked Write 1 Done Sent Done false
10249 3123 Acked Read 7 Done Sent Done false
10250 3124 Acked Read 10 Done Sent Done false
10251 3125 Acked Flush 0 Done Sent Done false
10252 3126 Acked Read 9 Done Sent Done false
10253 3127 Acked Write 10 Done Sent Done false
10254 3128 Acked Read 2 Done Sent Done false
10255 3129 Acked Flush 0 Done Sent Done false
10256 3130 Acked Write 9 Done Sent Done false
10257 3131 Acked Read 9 Done Sent Done false
10258 3132 Acked Write 5 Done Sent Done false
10259 3133 Acked Read 5 Done Sent Done false
10260 3134 Acked Write 5 Done Sent Done false
10261 3135 Acked Read 1 Done Sent Done false
10262 3136 Acked Write 3 Done Sent Done false
10263 3137 Acked Write 3 Done Sent Done false
10264 3138 Acked Read 5 Done Sent Done false
10265 3139 Acked Read 7 Done Sent Done false
10266 3140 Acked Write 3 Done Sent Done false
10267 3141 Acked Read 10 Done Sent Done false
10268 3142 Acked Read 5 Done Sent Done false
10269 3143 Acked Read 4 Done Sent Done false
10270 3144 Acked Flush 0 Done Sent Done false
10271 3145 Acked Read 4 Done Sent Done false
10272 3146 Acked Write 5 Done Sent Done false
10273 3147 Acked Write 4 Done Sent Done false
10274 3148 Acked Write 4 Done Sent Done false
10275 3149 Acked Write 4 Done Sent Done false
10276 3150 Acked Write 2 Done Sent Done false
10277 3151 Acked Read 7 Done Sent Done false
10278 3152 Acked Write 5 Done Sent Done false
10279 3153 Acked Read 9 Done Sent Done false
10280 3154 Acked Read 7 Done Sent Done false
10281 3155 Acked Read 8 Done Sent Done false
10282 3156 Acked Write 7 Done Sent Done false
10283 3157 Acked Read 8 Done Sent Done false
10284 3158 Acked Write 9 Done Sent Done false
10285 3159 Acked Write 6 Done Sent Done false
10286 3160 Acked Read 10 Done Sent Done false
10287 3161 Acked Read 8 Done Sent Done false
10288 3162 Acked Write 4 Done Sent Done false
10289 3163 Acked Read 10 Done Sent Done false
10290 3164 Acked Write 4 Done Sent Done false
10291 3165 Acked Read 1 Done Sent Done false
10292 3166 Acked Write 4 Done Sent Done false
10293 3167 Acked Read 8 Done Sent Done false
10294 3168 Acked Write 7 Done Sent Done false
10295 3169 Acked Read 4 Done Sent Done false
10296 3170 Acked Read 10 Done Sent Done false
10297 3171 Acked Flush 0 Done Sent Done false
10298 3172 Acked Flush 0 Done Sent Done false
10299 3173 Acked Read 4 Done Sent Done false
10300 3174 Acked Read 10 Done Sent Done false
10301 3175 Acked Read 6 Done Sent Done false
10302 3176 Acked Read 7 Done Sent Done false
10303 3177 Acked Write 3 Done Sent Done false
10304 3178 Acked Read 8 Done Sent Done false
10305 3179 Acked Write 1 Done Sent Done false
10306 3180 Acked Read 1 Done Sent Done false
10307 3181 Acked Write 10 Done Sent Done false
10308 3182 Acked Read 6 Done Sent Done false
10309 3183 Acked Write 7 Done Sent Done false
10310 3184 Acked Write 2 Done Sent Done false
10311 3185 Acked Write 10 Done Sent Done false
10312 3186 Acked Read 3 Done Sent Done false
10313 3187 Acked Read 5 Done Sent Done false
10314 3188 Acked Read 3 Done Sent Done false
10315 3189 Acked Write 4 Done Sent Done false
10316 3190 Acked Read 10 Done Sent Done false
10317 3191 Acked Flush 0 Done Sent Done false
10318 3192 Acked Write 7 Done Sent Done false
10319 3193 Acked Write 9 Done Sent Done false
10320 3194 Acked Write 10 Done Sent Done false
10321 3195 Acked Read 8 Done Sent Done false
10322 3196 Acked Read 7 Done Sent Done false
10323 3197 Acked Read 10 Done Sent Done false
10324 3198 Acked Read 1 Done Sent Done false
10325 3199 Acked Read 10 Done Sent Done false
10326 3200 Acked Write 7 Done Sent Done false
10327 3201 Acked Write 9 Done Sent Done false
10328 3202 Acked Flush 0 Done Sent Done false
10329 3203 Acked Read 9 Done Sent Done false
10330 3204 Acked Write 9 Done Sent Done false
10331 3205 Acked Write 6 Done Sent Done false
10332 3206 Acked Flush 0 Done Sent Done false
10333 3207 Acked Read 3 Done Sent Done false
10334 3208 Acked Write 7 Done Sent Done false
10335 3209 Acked Read 7 Done Sent Done false
10336 3210 Acked Read 9 Done Sent Done false
10337 3211 Acked Write 3 Done Sent Done false
10338 3212 Acked Read 1 Done Sent Done false
10339 3213 Acked Flush 0 Done Sent Done false
10340 3214 Acked Read 3 Done Sent Done false
10341 3215 Acked Flush 0 Done Sent Done false
10342 3216 Acked Read 10 Done Sent Done false
10343 3217 Acked Write 6 Done Sent Done false
10344 3218 Acked Read 10 Done Sent Done false
10345 3219 Acked Read 1 Done Sent Done false
10346 3220 Acked Write 2 Done Sent Done false
10347 3221 Acked Write 3 Done Sent Done false
10348 3222 Acked Read 2 Done Sent Done false
10349 3223 Acked Read 7 Done Sent Done false
10350 3224 Acked Read 1 Done Sent Done false
10351 3225 Acked Read 5 Done Sent Done false
10352 3226 Acked Read 1 Done Sent Done false
10353 3227 Acked Read 1 Done Sent Done false
10354 3228 Acked Read 5 Done Sent Done false
10355 3229 Acked Read 5 Done Sent Done false
10356 3230 Acked Read 3 Done Sent Done false
10357 3231 Acked Write 5 Done Sent Done false
10358 3232 Acked Read 3 Done Sent Done false
10359 3233 Acked Flush 0 Done Sent Done false
10360 3234 Acked Read 3 Done Sent Done false
10361 3235 Acked Read 1 Done Sent Done false
10362 3236 Acked Write 4 Done Sent Done false
10363 3237 Acked Read 2 Done Sent Done false
10364 3238 Acked Write 5 Done Sent Done false
10365 3239 Acked Write 8 Done Sent Done false
10366 3240 Acked Read 10 Done Sent Done false
10367 3241 Acked Write 10 Done Sent Done false
10368 3242 Acked Write 6 Done Sent Done false
10369 3243 Acked Read 3 Done Sent Done false
10370 3244 Acked Write 1 Done Sent Done false
10371 3245 Acked Read 5 Done Sent Done false
10372 3246 Acked Write 1 Done Sent Done false
10373 3247 Acked Read 4 Done Sent Done false
10374 3248 Acked Write 10 Done Sent Done false
10375 3249 Acked Read 3 Done Sent Done false
10376 3250 Acked Write 9 Done Sent Done false
10377 3251 Acked Flush 0 Done Sent Done false
10378 3252 Acked Write 9 Done Sent Done false
10379 3253 Acked Write 10 Done Sent Done false
10380 3254 Acked Read 3 Done Sent Done false
10381 3255 Acked Write 4 Done Sent Done false
10382 3256 Acked Write 2 Done Sent Done false
10383 3257 Acked Read 1 Done Sent Done false
10384 3258 Acked Write 6 Done Sent Done false
10385 3259 Acked Read 8 Done Sent Done false
10386 3260 Acked Read 6 Done Sent Done false
10387 3261 Acked Write 4 Done Sent Done false
10388 3262 Acked Write 2 Done Sent Done false
10389 3263 Acked Read 5 Done Sent Done false
10390 3264 Acked Write 5 Done Sent Done false
10391 3265 Acked Read 9 Done Sent Done false
10392 3266 Acked Read 5 Done Sent Done false
10393 3267 Acked Read 3 Done Sent Done false
10394 3268 Acked Write 4 Done Sent Done false
10395 3269 Acked Read 1 Done Sent Done false
10396 3270 Acked Write 2 Done Sent Done false
10397 3271 Acked Write 7 Done Sent Done false
10398 3272 Acked Write 10 Done Sent Done false
10399 3273 Acked Read 5 Done Sent Done false
10400 3274 Acked Read 7 Done Sent Done false
10401 3275 Acked Read 1 Done Sent Done false
10402 3276 Acked Read 6 Done Sent Done false
10403 3277 Acked Write 8 Done Sent Done false
10404 3278 Acked Write 7 Done Sent Done false
10405 3279 Acked Read 9 Done Sent Done false
10406 3280 Acked Read 5 Done Sent Done false
10407 3281 Acked Write 10 Done Sent Done false
10408 3282 Acked Read 3 Done Sent Done false
10409 3283 Acked Read 8 Done Sent Done false
10410 3284 Acked Read 1 Done Sent Done false
10411 3285 Acked Write 7 Done Sent Done false
10412 3286 Acked Write 1 Done Sent Done false
10413 3287 Acked Write 5 Done Sent Done false
10414 3288 Acked Read 3 Done Sent Done false
10415 3289 Acked Read 1 Done Sent Done false
10416 3290 Acked Read 7 Done Sent Done false
10417 3291 Acked Write 3 Done Sent Done false
10418 3292 Acked Read 5 Done Sent Done false
10419 3293 Acked Read 2 Done Sent Done false
10420 3294 Acked Read 9 Done Sent Done false
10421 3295 Acked Write 9 Done Sent Done false
10422 3296 Acked Flush 0 Done Sent Done false
10423 3297 Acked Read 8 Done Sent Done false
10424 3298 Acked Read 9 Done Sent Done false
10425 3299 Acked Write 7 Done Sent Done false
10426 3300 Acked Read 4 Done Sent Done false
10427 3301 Acked Read 6 Done Sent Done false
10428 3302 Acked Write 9 Done Sent Done false
10429 3303 Acked Read 7 Done Sent Done false
10430 3304 Acked Write 8 Done Sent Done false
10431 3305 Acked Read 4 Done Sent Done false
10432 3306 Acked Read 5 Done Sent Done false
10433 3307 Acked Write 4 Done Sent Done false
10434 3308 Acked Flush 0 Done Sent Done false
10435 3309 Acked Write 7 Done Sent Done false
10436 3310 Acked Write 9 Done Sent Done false
10437 3311 Acked Read 7 Done Sent Done false
10438 3312 Acked Write 1 Done Sent Done false
10439 3313 Acked Read 5 Done Sent Done false
10440 3314 Acked Read 6 Done Sent Done false
10441 3315 Acked Write 7 Done Sent Done false
10442 3316 Acked Flush 0 Done Sent Done false
10443 3317 Acked Write 9 Done Sent Done false
10444 3318 Acked Write 10 Done Sent Done false
10445 3319 Acked Flush 0 Done Sent Done false
10446 3320 Acked Read 5 Done Sent Done false
10447 3321 Acked Flush 0 Done Sent Done false
10448 3322 Acked Write 3 Done Sent Done false
10449 3323 Acked Write 1 Done Sent Done false
10450 3324 Acked Write 8 Done Sent Done false
10451 3325 Acked Write 6 Done Sent Done false
10452 3326 Acked Read 5 Done Sent Done false
10453 3327 Acked Write 10 Done Sent Done false
10454 3328 Acked Read 4 Done Sent Done false
10455 3329 Acked Read 5 Done Sent Done false
10456 3330 Acked Read 9 Done Sent Done false
10457 3331 Acked Write 1 Done Sent Done false
10458 3332 Acked Read 1 Done Sent Done false
10459 3333 Acked Flush 0 Done Sent Done false
10460 3334 Acked Read 8 Done Sent Done false
10461 3335 Acked Read 4 Done Sent Done false
10462 3336 Acked Read 2 Done Sent Done false
10463 3337 Acked Write 4 Done Sent Done false
10464 3338 Acked Read 3 Done Sent Done false
10465 3339 Acked Read 1 Done Sent Done false
10466 3340 Acked Read 10 Done Sent Done false
10467 3341 Acked Read 10 Done Sent Done false
10468 3342 Acked Read 8 Done Sent Done false
10469 3343 Acked Read 4 Done Sent Done false
10470 3344 Acked Read 7 Done Sent Done false
10471 3345 Acked Write 8 Done Sent Done false
10472 3346 Acked Write 5 Done Sent Done false
10473 3347 Acked Read 8 Done Sent Done false
10474 3348 Acked Read 9 Done Sent Done false
10475 3349 Acked Write 8 Done Sent Done false
10476 3350 Acked Read 6 Done Sent Done false
10477 3351 Acked Read 2 Done Sent Done false
10478 3352 Acked Write 4 Done Sent Done false
10479 3353 Acked Flush 0 Done Sent Done false
10480 3354 Acked Write 7 Done Sent Done false
10481 3355 Acked Read 9 Done Sent Done false
10482 3356 Acked Write 6 Done Sent Done false
10483 3357 Acked Flush 0 Done Sent Done false
10484 3358 Acked Flush 0 Done Sent Done false
10485 3359 Acked Write 2 Done Sent Done false
10486 3360 Acked Write 10 Done Sent Done false
10487 3361 Acked Read 1 Done Sent Done false
10488 3362 Acked Write 3 Done Sent Done false
10489 3363 Acked Flush 0 Done Sent Done false
10490 3364 Acked Write 10 Done Sent Done false
10491 3365 Acked Flush 0 Done Sent Done false
10492 3366 Acked Write 7 Done Sent Done false
10493 3367 Acked Read 6 Done Sent Done false
10494 3368 Acked Write 2 Done Sent Done false
10495 3369 Acked Read 10 Done Sent Done false
10496 3370 Acked Read 8 Done Sent Done false
10497 3371 Acked Flush 0 Done Sent Done false
10498 3372 Acked Write 4 Done Sent Done false
10499 3373 Acked Read 7 Done Sent Done false
10500 3374 Acked Read 5 Done Sent Done false
10501 3375 Acked Write 4 Done Sent Done false
10502 3376 Acked Read 2 Done Sent Done false
10503 3377 Acked Write 8 Done Sent Done false
10504 3378 Acked Read 1 Done Sent Done false
10505 3379 Acked Read 1 Done Sent Done false
10506 3380 Acked Read 10 Done Sent Done false
10507 3381 Acked Write 3 Done Sent Done false
10508 3382 Acked Read 9 Done Sent Done false
10509 3383 Acked Read 6 Done Sent Done false
10510 3384 Acked Read 3 Done Sent Done false
10511 3385 Acked Write 1 Done Sent Done false
10512 3386 Acked Write 7 Done Sent Done false
10513 3387 Acked Write 9 Done Sent Done false
10514 3388 Acked Read 3 Done Sent Done false
10515 3389 Acked Read 4 Done Sent Done false
10516 3390 Acked Read 7 Done Sent Done false
10517 3391 Acked Write 5 Done Sent Done false
10518 3392 Acked Write 5 Done Sent Done false
10519 3393 Acked Read 6 Done Sent Done false
10520 3394 Acked Read 4 Done Sent Done false
10521 3395 Acked Write 9 Done Sent Done false
10522 3396 Acked Read 10 Done Sent Done false
10523 3397 Acked Write 2 Done Sent Done false
10524 3398 Acked Write 8 Done Sent Done false
10525 3399 Acked Write 9 Done Sent Done false
10526 3400 Acked Flush 0 Done Sent Done false
10527 3401 Acked Write 1 Done Sent Done false
10528 3402 Acked Read 9 Done Sent Done false
10529 3403 Acked Flush 0 Done Sent Done false
10530 3404 Acked Read 3 Done Sent Done false
10531 3405 Acked Read 9 Done Sent Done false
10532 3406 Acked Flush 0 Done Sent Done false
10533 3407 Acked Write 1 Done Sent Done false
10534 3408 Acked Write 7 Done Sent Done false
10535 3409 Acked Write 8 Done Sent Done false
10536 3410 Acked Write 1 Done Sent Done false
10537 3411 Acked Write 4 Done Sent Done false
10538 3412 Acked Read 5 Done Sent Done false
10539 3413 Acked Flush 0 Done Sent Done false
10540 STATES DS:0 DS:1 DS:2 TOTAL
10541 Sent 0 302 0 302
10542 Done 302 0 302 604
10543 Skipped 0 0 0 0
10544 Error 0 0 0 0
10545 Last Flush: 3413 3111 3413
10546 Downstairs last five completed: 3111 3110 3109 3108 3107
10547 Upstairs last five completed: 3413 3412 3411 3410 3409
10548 CLIENT: Up:0 ds:302 act:2
105492024-11-15T01:59:41.589ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
105502024-11-15T01:59:41.589ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:55151) = io task client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
105512024-11-15T01:59:41.590ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
105522024-11-15T01:59:41.591ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
105532024-11-15T01:59:41.591ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
105542024-11-15T01:59:41.591ZINFOcrucible: send last flush ID to this DS: 3111 = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
105552024-11-15T01:59:41.591ZINFOcrucible: Replied this last flush ID: 3111 = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
105562024-11-15T01:59:41.591ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
105572024-11-15T01:59:41.591ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
105582024-11-15T01:59:41.591ZINFOcrucible: [1] client re-new 302 jobs since flush 3111 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
105592024-11-15T01:59:41.592ZINFOcrucible: [1] Replayed 302 jobs since flush: 3111 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
10560 ----------------------------------------------------------------
10561 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
10562 Downstairs last five completed: 3413 3412 3411 3410 3409
10563 Upstairs last five completed: 3413 3412 3411 3410 3409
10564 CLIENT: Up:0 ds:0 act:3
10565 Replay: All jobs finished
10566 Main task has work to do, go find it
10567 got dsc Stop(2)
10568 stop 2
10569 Nov 15 01:59:41.838 INFO request completed[8830] Got stop action so:false kr:false
10570 , latency_us: 327, response_code: 204, uri: /stop/cid/2, method: POST, req_id: cb9bfa95-06df-4567-9bb5-d571a2a6deb4, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
105712024-11-15T01:59:41.839ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
105722024-11-15T01:59:41.839ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
105732024-11-15T01:59:41.839ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
105742024-11-15T01:59:41.839ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
105752024-11-15T01:59:41.839ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
105762024-11-15T01:59:41.839ZINFOcrucible: client task is exiting = io task client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
10577 Nov 15 01:59:41.839 INFO request completed, latency_us: 143, response_code: 200, uri: /state/cid/2, method: GET, req_id: db1d50ec-d7bd-468d-ab21-b9469498cf34, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
10578 [8830] Exited with: ExitStatus(unix_wait_status(9))
10579 [8830][2] reports Exit
10580 Nov 15 01:59:45.841 INFO request completed, latency_us: 339, response_code: 200, uri: /state/cid/2, method: GET, req_id: 8e1eacb0-a983-4ca9-9b54-cb777367eb8a, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
10581 001/300 Write block 1888 len 40960 data: 4 4 4 4 4 3 3 2 2 3
10582 002/300 Read block 4919 len 20480
10583 003/300 Read block 788 len 8192
10584 004/300 Read block 599 len 4096
10585 005/300 Read block 4311 len 12288
10586 006/300 Write block 2951 len 32768 data: 6 6 5 6 5 5 5 4
10587 007/300 Write block 3192 len 8192 data: 5 4
10588 008/300 Write block 4172 len 36864 data: 2 3 4 5 5 5 5 5 4
10589 009/300 Read block 840 len 4096
10590 010/300 Write block 3252 len 28672 data: 4 4 5 4 4 4 4
10591 011/300 Write block 2778 len 32768 data: 4 3 3 3 3 3 3 3
10592 012/300 Read block 562 len 20480
10593 013/300 Flush
10594 014/300 Read block 1213 len 16384
10595 015/300 Flush
10596 016/300 Read block 2196 len 40960
10597 017/300 Read block 2012 len 16384
10598 018/300 Read block 329 len 36864
10599 019/300 Write block 4312 len 40960 data: 3 3 3 3 4 4 2 2 2 2
10600 020/300 Flush
10601 021/300 Read block 3938 len 36864
10602 022/300 Flush
10603 023/300 Write block 3245 len 36864 data: 3 3 3 3 4 4 5 5 5
10604 024/300 Read block 4739 len 40960
10605 025/300 Write block 4492 len 20480 data: 4 5 5 5 4
10606 026/300 Read block 881 len 36864
10607 027/300 Read block 2133 len 32768
10608 028/300 Read block 4021 len 28672
10609 029/300 Read block 1440 len 16384
10610 030/300 Write block 2912 len 28672 data: 2 2 2 2 2 2 2
10611 031/300 Flush
10612 032/300 Read block 2198 len 20480
10613 033/300 Read block 2509 len 32768
10614 034/300 Read block 3041 len 32768
10615 035/300 Write block 2318 len 32768 data: 3 4 4 4 5 4 4 5
10616 036/300 Read block 4700 len 8192
10617 037/300 Write block 3032 len 36864 data: 3 3 4 3 3 3 3 4 4
10618 038/300 Write block 4651 len 20480 data: 3 3 2 2 2
10619 039/300 Write block 388 len 16384 data: 4 4 2 4
10620 040/300 Flush
10621 041/300 Read block 492 len 4096
10622 042/300 Read block 4456 len 24576
10623 043/300 Read block 402 len 4096
10624 044/300 Read block 293 len 20480
10625 045/300 Flush
10626 046/300 Read block 3973 len 8192
10627 047/300 Write block 1880 len 4096 data: 3
10628 048/300 Read block 1723 len 8192
10629 049/300 Read block 1737 len 16384
10630 050/300 Read block 1674 len 36864
10631 051/300 Write block 905 len 8192 data: 4 4
10632 052/300 Read block 2719 len 36864
10633 053/300 Read block 2482 len 24576
10634 054/300 Read block 2512 len 4096
10635 055/300 Flush
10636 056/300 Flush
10637 057/300 Write block 132 len 20480 data: 2 2 2 2 2
10638 058/300 Write block 3787 len 12288 data: 2 2 2
10639 059/300 Read block 2797 len 16384
10640 060/300 Write block 690 len 16384 data: 2 2 2 2
10641 061/300 Read block 2789 len 36864
10642 062/300 Read block 3871 len 16384
10643 063/300 Flush
10644 064/300 Read block 4413 len 8192
10645 065/300 Read block 983 len 8192
10646 066/300 Read block 447 len 8192
10647 067/300 Write block 1323 len 20480 data: 2 2 2 2 2
10648 068/300 Read block 3001 len 36864
10649 069/300 Read block 2252 len 16384
10650 070/300 Flush
10651 071/300 Write block 500 len 28672 data: 3 4 4 4 4 4 3
10652 072/300 Flush
10653 073/300 Read block 186 len 40960
10654 074/300 Write block 4155 len 24576 data: 3 3 3 3 3 3
10655 075/300 Read block 1305 len 40960
10656 076/300 Write block 4035 len 16384 data: 3 3 5 5
10657 077/300 Read block 2072 len 24576
10658 078/300 Read block 1039 len 24576
10659 079/300 Read block 241 len 36864
10660 080/300 Read block 2451 len 24576
10661 081/300 Write block 2001 len 24576 data: 3 3 4 4 3 3
10662 082/300 Read block 3207 len 4096
10663 083/300 Read block 1276 len 24576
10664 084/300 Read block 4363 len 40960
10665 085/300 Write block 653 len 36864 data: 3 3 3 3 3 2 3 3 3
10666 086/300 Write block 4569 len 4096 data: 3
10667 087/300 Write block 526 len 8192 data: 3 4
10668 088/300 Read block 1343 len 28672
10669 089/300 Read block 716 len 12288
10670 090/300 Read block 194 len 28672
10671 091/300 Write block 2387 len 12288 data: 3 3 4
10672 092/300 Read block 1472 len 20480
10673 093/300 Write block 2393 len 8192 data: 2 2
10674 094/300 Read block 1227 len 28672
10675 095/300 Read block 4539 len 24576
10676 096/300 Read block 2396 len 32768
10677 097/300 Read block 4502 len 20480
10678 098/300 Write block 3552 len 32768 data: 3 3 2 3 3 3 3 4
10679 099/300 Read block 4161 len 20480
10680 100/300 Flush
10681 101/300 Read block 4250 len 4096
10682 102/300 Read block 1823 len 32768
10683 103/300 Write block 780 len 4096 data: 2
10684 104/300 Read block 601 len 8192
10685 105/300 Read block 3954 len 20480
10686 106/300 Write block 4665 len 4096 data: 3
10687 107/300 Read block 3706 len 8192
10688 108/300 Read block 2949 len 20480
10689 109/300 Write block 717 len 8192 data: 3 3
10690 110/300 Write block 2367 len 8192 data: 4 4
10691 111/300 Read block 1709 len 24576
10692 112/300 Write block 4922 len 8192 data: 6 7
10693 113/300 Write block 4861 len 28672 data: 2 2 2 2 2 2 2
10694 114/300 Write block 3781 len 8192 data: 3 3
10695 115/300 Write block 2566 len 4096 data: 5
10696 116/300 Write block 258 len 24576 data: 3 3 3 3 2 2
10697 117/300 Read block 1310 len 20480
10698 118/300 Read block 4537 len 16384
10699 119/300 Read block 4387 len 20480
10700 120/300 Read block 3011 len 16384
10701 121/300 Write block 3335 len 4096 data: 3
10702 122/300 Write block 4560 len 40960 data: 2 2 2 2 3 3 3 3 3 4
10703 123/300 Write block 31 len 32768 data: 3 3 3 3 3 3 3 4
10704 124/300 Write block 2646 len 24576 data: 2 3 3 3 4 4
10705 125/300 Write block 4555 len 28672 data: 2 2 2 2 2 3 3
10706 126/300 Write block 2339 len 12288 data: 4 5 4
10707 127/300 Write block 4385 len 4096 data: 3
10708 128/300 Write block 557 len 32768 data: 2 2 2 2 2 2 2 2
10709 129/300 Write block 412 len 40960 data: 3 4 4 3 4 4 4 4 4 4
10710 130/300 Write block 327 len 32768 data: 3 2 2 2 3 3 4 3
10711 131/300 Read block 1764 len 20480
10712 132/300 Read block 4406 len 20480
10713 133/300 Write block 3415 len 20480 data: 4 4 3 3 3
10714 134/300 Read block 2523 len 36864
10715 135/300 Read block 3628 len 4096
10716 136/300 Read block 2097 len 12288
10717 137/300 Read block 4042 len 40960
10718 138/300 Read block 81 len 28672
10719 139/300 Write block 4704 len 12288 data: 3 3 4
10720 140/300 Read block 3331 len 16384
10721 141/300 Write block 3648 len 24576 data: 4 5 5 5 4 4
10722 142/300 Read block 1853 len 4096
10723 143/300 Read block 2470 len 32768
10724 144/300 Read block 4801 len 12288
10725 145/300 Read block 2943 len 4096
10726 146/300 Flush
10727 147/300 Read block 4735 len 28672
10728 148/300 Read block 1697 len 12288
10729 149/300 Read block 2211 len 36864
10730 150/300 Flush
10731 151/300 Read block 4627 len 36864
10732 152/300 Write block 796 len 32768 data: 2 2 2 2 2 2 2 2
10733 153/300 Read block 1165 len 28672
10734 154/300 Write block 2448 len 28672 data: 5 5 5 5 4 5 5
10735 155/300 Read block 3733 len 16384
10736 156/300 Read block 4942 len 40960
10737 157/300 Read block 1313 len 16384
10738 158/300 Read block 417 len 8192
10739 159/300 Read block 196 len 20480
10740 160/300 Read block 3057 len 24576
10741 161/300 Write block 995 len 8192 data: 3 4
10742 162/300 Read block 3496 len 32768
10743 163/300 Read block 556 len 36864
10744 164/300 Read block 4141 len 20480
10745 165/300 Flush
10746 166/300 Write block 116 len 8192 data: 2 2
10747 167/300 Write block 485 len 28672 data: 4 2 2 2 2 3 3
10748 168/300 Read block 4163 len 32768
10749 169/300 Read block 4463 len 36864
10750 170/300 Write block 3140 len 36864 data: 2 2 2 2 2 2 2 2 2
10751 171/300 Write block 3369 len 8192 data: 2 2
10752 172/300 Write block 2934 len 28672 data: 6 4 4 4 4 4 4
10753 173/300 Read block 355 len 24576
10754 174/300 Write block 3955 len 28672 data: 4 4 4 4 4 3 3
10755 175/300 Read block 461 len 12288
10756 176/300 Read block 2195 len 36864
10757 177/300 Flush
10758 178/300 Read block 750 len 32768
10759 179/300 Read block 6 len 8192
10760 180/300 Read block 1977 len 40960
10761 181/300 Read block 1687 len 12288
10762 182/300 Read block 4550 len 28672
10763 183/300 Read block 3214 len 36864
10764 184/300 Write block 4291 len 4096 data: 3
10765 185/300 Write block 1218 len 12288 data: 2 2 2
10766 186/300 Write block 4444 len 36864 data: 3 3 3 3 3 2 2 3 2
10767 187/300 Read block 608 len 28672
10768 188/300 Write block 4940 len 8192 data: 3 3
10769 189/300 Write block 4689 len 8192 data: 2 3
10770 190/300 Write block 163 len 12288 data: 3 4 4
10771 191/300 Read block 792 len 4096
10772 192/300 Read block 3811 len 36864
10773 193/300 Read block 1802 len 28672
10774 194/300 Flush
10775 195/300 Read block 4386 len 28672
10776 196/300 Write block 4551 len 36864 data: 2 2 2 2 3 3 3 3 3
10777 197/300 Read block 1308 len 4096
10778 198/300 Read block 1133 len 16384
10779 199/300 Read block 2888 len 8192
10780 200/300 Write block 3897 len 40960 data: 3 4 4 4 5 5 5 5 4 4
10781 201/300 Write block 102 len 16384 data: 2 2 2 2
10782 202/300 Read block 4138 len 20480
10783 203/300 Flush
10784 204/300 Read block 2555 len 4096
10785 205/300 Read block 3883 len 32768
10786 206/300 Read block 426 len 16384
10787 207/300 Write block 3809 len 12288 data: 2 2 2
10788 208/300 Read block 4532 len 40960
10789 209/300 Write block 1892 len 32768 data: 5 4 4 3 3 4 4 5
10790 210/300 Write block 2365 len 16384 data: 4 4 5 5
10791 211/300 Read block 227 len 4096
10792 212/300 Write block 2209 len 32768 data: 3 2 2 3 3 3 3 4
10793 213/300 Write block 3286 len 20480 data: 2 2 2 3 3
10794 214/300 Write block 4719 len 4096 data: 2
10795 215/300 Read block 1734 len 8192
10796 216/300 Write block 2261 len 28672 data: 2 2 2 2 2 2 2
10797 217/300 Write block 78 len 20480 data: 4 5 5 5 4
10798 218/300 Read block 1983 len 24576
10799 219/300 Write block 3219 len 8192 data: 4 4
10800 220/300 Read block 3297 len 8192
10801 221/300 Write block 4521 len 12288 data: 3 3 3
10802 222/300 Write block 4684 len 12288 data: 4 3 3
10803 223/300 Read block 3162 len 32768
10804 224/300 Flush
10805 225/300 Flush
10806 226/300 Write block 1800 len 4096 data: 3
10807 227/300 Write block 1890 len 32768 data: 5 5 6 5 5 4 4 5
10808 228/300 Read block 436 len 4096
10809 229/300 Read block 836 len 12288
10810 230/300 Read block 1609 len 16384
10811 231/300 Read block 3960 len 16384
10812 232/300 Read block 1410 len 16384
10813 233/300 Write block 2983 len 40960 data: 5 5 5 5 3 3 2 2 3 3
10814 234/300 Flush
10815 235/300 Read block 3491 len 24576
10816 236/300 Read block 710 len 28672
10817 237/300 Read block 2283 len 20480
10818 238/300 Read block 2427 len 24576
10819 239/300 Read block 1402 len 40960
10820 240/300 Read block 364 len 24576
10821 241/300 Write block 947 len 12288 data: 2 2 2
10822 242/300 Read block 4602 len 20480
10823 243/300 Write block 4004 len 36864 data: 2 2 2 2 2 2 3 3 3
10824 244/300 Write block 4252 len 12288 data: 3 4 4
10825 245/300 Read block 92 len 4096
10826 246/300 Read block 4035 len 40960
10827 247/300 Read block 1435 len 8192
10828 248/300 Read block 4239 len 16384
10829 249/300 Read block 4264 len 8192
10830 250/300 Write block 3880 len 28672 data: 3 3 3 3 3 4 3
10831 251/300 Read block 1494 len 24576
10832 252/300 Read block 1942 len 36864
10833 253/300 Write block 2281 len 20480 data: 3 3 4 6 7
10834 254/300 Read block 776 len 24576
10835 255/300 Read block 4483 len 28672
10836 256/300 Read block 4163 len 36864
10837 257/300 Write block 167 len 16384 data: 4 4 4 4
10838 258/300 Write block 3349 len 32768 data: 3 3 3 4 3 3 3 2
10839 259/300 Read block 4325 len 4096
10840 260/300 Flush
10841 261/300 Flush
10842 262/300 Read block 940 len 32768
10843 263/300 Read block 4960 len 4096
10844 264/300 Read block 3131 len 36864
10845 265/300 Write block 171 len 36864 data: 4 5 5 4 3 4 4 3 3
10846 266/300 Write block 4882 len 24576 data: 2 2 2 2 3 3
10847 267/300 Read block 3523 len 36864
10848 268/300 Read block 873 len 8192
10849 269/300 Read block 514 len 16384
10850 270/300 Write block 2453 len 40960 data: 6 6 6 5 6 5 5 5 5 4
10851 271/300 Write block 3183 len 36864 data: 2 2 3 3 4 5 6 5 5
10852 272/300 Read block 793 len 24576
10853 273/300 Read block 2908 len 12288
10854 274/300 Flush
10855 275/300 Read block 1255 len 40960
10856 276/300 Write block 4160 len 16384 data: 4 2 2 2
10857 277/300 Write block 4827 len 16384 data: 3 3 4 4
10858 278/300 Read block 8 len 32768
10859 279/300 Write block 2323 len 28672 data: 5 5 6 3 3 3 2
10860 280/300 Flush
10861 281/300 Read block 4634 len 12288
10862 282/300 Read block 2162 len 20480
10863 283/300 Read block 2622 len 4096
10864 284/300 Write block 2379 len 36864 data: 2 2 2 2 3 3 3 3 4
10865 285/300 Read block 2591 len 16384
10866 286/300 Read block 3838 len 20480
10867 287/300 Write block 2897 len 40960 data: 3 3 3 3 2 2 2 2 2 2
10868 288/300 Write block 1638 len 8192 data: 3 3
10869 289/300 Read block 3979 len 20480
10870 290/300 Read block 4172 len 28672
10871 291/300 Write block 3565 len 20480 data: 2 2 2 3 3
10872 292/300 Read block 4094 len 16384
10873 293/300 Read block 912 len 12288
10874 294/300 Flush
10875 295/300 Read block 2718 len 32768
10876 296/300 Read block 1372 len 4096
10877 297/300 Flush
10878 298/300 Write block 2441 len 20480 data: 2 2 2 2 3
10879 299/300 Read block 1947 len 40960
10880 300/300 Read block 816 len 36864
10881 301/300 Read block 3374 len 24576
10882 Main task has work to do, go find it
10883 got dsc Start(2)
10884 start 2
10885 Nov 15 01:59:46.085 INFO request completed, latency_us: 216, response_code: 204, uri: /start/cid/2, method: POST, req_id: f193cb6c-ef42-48b4-be1c-7afcf3be9577, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
10886 Replay: started 2, returned:Ok(())
10887 ----------------------------------------------------------------
10888 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
10889 States: Active Active Offline
10890 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
10891 3414 Acked Write 10 Done Done Sent false
10892 3415 Acked Read 5 Done Done Sent false
10893 3416 Acked Read 2 Done Done Sent false
10894 3417 Acked Read 1 Done Done Sent false
10895 3418 Acked Read 3 Done Done Sent false
10896 3419 Acked Write 8 Done Done Sent false
10897 3420 Acked Write 2 Done Done Sent false
10898 3421 Acked Write 9 Done Done Sent false
10899 3422 Acked Read 1 Done Done Sent false
10900 3423 Acked Write 7 Done Done Sent false
10901 3424 Acked Write 8 Done Done Sent false
10902 3425 Acked Read 5 Done Done Sent false
10903 3426 Acked Flush 0 Done Done Sent false
10904 3427 Acked Read 4 Done Done Sent false
10905 3428 Acked Flush 0 Done Done Sent false
10906 3429 Acked Read 10 Done Done Sent false
10907 3430 Acked Read 4 Done Done Sent false
10908 3431 Acked Read 9 Done Done Sent false
10909 3432 Acked Write 10 Done Done Sent false
10910 3433 Acked Flush 0 Done Done Sent false
10911 3434 Acked Read 9 Done Done Sent false
10912 3435 Acked Flush 0 Done Done Sent false
10913 3436 Acked Write 9 Done Done Sent false
10914 3437 Acked Read 10 Done Done Sent false
10915 3438 Acked Write 5 Done Done Sent false
10916 3439 Acked Read 9 Done Done Sent false
10917 3440 Acked Read 8 Done Done Sent false
10918 3441 Acked Read 7 Done Done Sent false
10919 3442 Acked Read 4 Done Done Sent false
10920 3443 Acked Write 7 Done Done Sent false
10921 3444 Acked Flush 0 Done Done Sent false
10922 3445 Acked Read 5 Done Done Sent false
10923 3446 Acked Read 8 Done Done Sent false
10924 3447 Acked Read 8 Done Done Sent false
10925 3448 Acked Write 8 Done Done Sent false
10926 3449 Acked Read 2 Done Done Sent false
10927 3450 Acked Write 9 Done Done Sent false
10928 3451 Acked Write 5 Done Done Sent false
10929 3452 Acked Write 4 Done Done Sent false
10930 3453 Acked Flush 0 Done Done Sent false
10931 3454 Acked Read 1 Done Done Sent false
10932 3455 Acked Read 6 Done Done Sent false
10933 3456 Acked Read 1 Done Done Sent false
10934 3457 Acked Read 5 Done Done Sent false
10935 3458 Acked Flush 0 Done Done Sent false
10936 3459 Acked Read 2 Done Done Sent false
10937 3460 Acked Write 1 Done Done Sent false
10938 3461 Acked Read 2 Done Done Sent false
10939 3462 Acked Read 4 Done Done Sent false
10940 3463 Acked Read 9 Done Done Sent false
10941 3464 Acked Write 2 Done Done Sent false
10942 3465 Acked Read 9 Done Done Sent false
10943 3466 Acked Read 6 Done Done Sent false
10944 3467 Acked Read 1 Done Done Sent false
10945 3468 Acked Flush 0 Done Done Sent false
10946 3469 Acked Flush 0 Done Done Sent false
10947 3470 Acked Write 5 Done Done Sent false
10948 3471 Acked Write 3 Done Done Sent false
10949 3472 Acked Read 4 Done Done Sent false
10950 3473 Acked Write 4 Done Done Sent false
10951 3474 Acked Read 9 Done Done Sent false
10952 3475 Acked Read 4 Done Done Sent false
10953 3476 Acked Flush 0 Done Done Sent false
10954 3477 Acked Read 2 Done Done Sent false
10955 3478 Acked Read 2 Done Done Sent false
10956 3479 Acked Read 2 Done Done Sent false
10957 3480 Acked Write 5 Done Done Sent false
10958 3481 Acked Read 9 Done Done Sent false
10959 3482 Acked Read 4 Done Done Sent false
10960 3483 Acked Flush 0 Done Done Sent false
10961 3484 Acked Write 7 Done Done Sent false
10962 3485 Acked Flush 0 Done Done Sent false
10963 3486 Acked Read 10 Done Done Sent false
10964 3487 Acked Write 6 Done Done Sent false
10965 3488 Acked Read 10 Done Done Sent false
10966 3489 Acked Write 4 Done Done Sent false
10967 3490 Acked Read 6 Done Done Sent false
10968 3491 Acked Read 6 Done Done Sent false
10969 3492 Acked Read 9 Done Done Sent false
10970 3493 Acked Read 6 Done Done Sent false
10971 3494 Acked Write 6 Done Done Sent false
10972 3495 Acked Read 1 Done Done Sent false
10973 3496 Acked Read 6 Done Done Sent false
10974 3497 Acked Read 10 Done Done Sent false
10975 3498 Acked Write 9 Done Done Sent false
10976 3499 Acked Write 1 Done Done Sent false
10977 3500 Acked Write 2 Done Done Sent false
10978 3501 Acked Read 7 Done Done Sent false
10979 3502 Acked Read 3 Done Done Sent false
10980 3503 Acked Read 7 Done Done Sent false
10981 3504 Acked Write 3 Done Done Sent false
10982 3505 Acked Read 5 Done Done Sent false
10983 3506 Acked Write 2 Done Done Sent false
10984 3507 Acked Read 7 Done Done Sent false
10985 3508 Acked Read 6 Done Done Sent false
10986 3509 Acked Read 8 Done Done Sent false
10987 3510 Acked Read 5 Done Done Sent false
10988 3511 Acked Write 8 Done Done Sent false
10989 3512 Acked Read 5 Done Done Sent false
10990 3513 Acked Flush 0 Done Done Sent false
10991 3514 Acked Read 1 Done Done Sent false
10992 3515 Acked Read 8 Done Done Sent false
10993 3516 Acked Write 1 Done Done Sent false
10994 3517 Acked Read 2 Done Done Sent false
10995 3518 Acked Read 5 Done Done Sent false
10996 3519 Acked Write 1 Done Done Sent false
10997 3520 Acked Read 2 Done Done Sent false
10998 3521 Acked Read 5 Done Done Sent false
10999 3522 Acked Write 2 Done Done Sent false
11000 3523 Acked Write 2 Done Done Sent false
11001 3524 Acked Read 6 Done Done Sent false
11002 3525 Acked Write 2 Done Done Sent false
11003 3526 Acked Write 7 Done Done Sent false
11004 3527 Acked Write 2 Done Done Sent false
11005 3528 Acked Write 1 Done Done Sent false
11006 3529 Acked Write 6 Done Done Sent false
11007 3530 Acked Read 5 Done Done Sent false
11008 3531 Acked Read 4 Done Done Sent false
11009 3532 Acked Read 5 Done Done Sent false
11010 3533 Acked Read 4 Done Done Sent false
11011 3534 Acked Write 1 Done Done Sent false
11012 3535 Acked Write 10 Done Done Sent false
11013 3536 Acked Write 8 Done Done Sent false
11014 3537 Acked Write 6 Done Done Sent false
11015 3538 Acked Write 7 Done Done Sent false
11016 3539 Acked Write 3 Done Done Sent false
11017 3540 Acked Write 1 Done Done Sent false
11018 3541 Acked Write 8 Done Done Sent false
11019 3542 Acked Write 10 Done Done Sent false
11020 3543 Acked Write 8 Done Done Sent false
11021 3544 Acked Read 5 Done Done Sent false
11022 3545 Acked Read 5 Done Done Sent false
11023 3546 Acked Write 5 Done Done Sent false
11024 3547 Acked Read 9 Done Done Sent false
11025 3548 Acked Read 1 Done Done Sent false
11026 3549 Acked Read 3 Done Done Sent false
11027 3550 Acked Read 10 Done Done Sent false
11028 3551 Acked Read 7 Done Done Sent false
11029 3552 Acked Write 3 Done Done Sent false
11030 3553 Acked Read 4 Done Done Sent false
11031 3554 Acked Write 6 Done Done Sent false
11032 3555 Acked Read 1 Done Done Sent false
11033 3556 Acked Read 8 Done Done Sent false
11034 3557 Acked Read 3 Done Done Sent false
11035 3558 Acked Read 1 Done Done Sent false
11036 3559 Acked Flush 0 Done Done Sent false
11037 3560 Acked Read 7 Done Done Sent false
11038 3561 Acked Read 3 Done Done Sent false
11039 3562 Acked Read 9 Done Done Sent false
11040 3563 Acked Flush 0 Done Done Sent false
11041 3564 Acked Read 9 Done Done Sent false
11042 3565 Acked Write 8 Done Done Sent false
11043 3566 Acked Read 7 Done Done Sent false
11044 3567 Acked Write 7 Done Done Sent false
11045 3568 Acked Read 4 Done Done Sent false
11046 3569 Acked Read 10 Done Done Sent false
11047 3570 Acked Read 4 Done Done Sent false
11048 3571 Acked Read 2 Done Done Sent false
11049 3572 Acked Read 5 Done Done Sent false
11050 3573 Acked Read 6 Done Done Sent false
11051 3574 Acked Write 2 Done Done Sent false
11052 3575 Acked Read 8 Done Done Sent false
11053 3576 Acked Read 9 Done Done Sent false
11054 3577 Acked Read 5 Done Done Sent false
11055 3578 Acked Flush 0 Done Done Sent false
11056 3579 Acked Write 2 Done Done Sent false
11057 3580 Acked Write 7 Done Done Sent false
11058 3581 Acked Read 8 Done Done Sent false
11059 3582 Acked Read 9 Done Done Sent false
11060 3583 Acked Write 9 Done Done Sent false
11061 3584 Acked Write 2 Done Done Sent false
11062 3585 Acked Write 7 Done Done Sent false
11063 3586 Acked Read 6 Done Done Sent false
11064 3587 Acked Write 7 Done Done Sent false
11065 3588 Acked Read 3 Done Done Sent false
11066 3589 Acked Read 9 Done Done Sent false
11067 3590 Acked Flush 0 Done Done Sent false
11068 3591 Acked Read 8 Done Done Sent false
11069 3592 Acked Read 2 Done Done Sent false
11070 3593 Acked Read 10 Done Done Sent false
11071 3594 Acked Read 3 Done Done Sent false
11072 3595 Acked Read 7 Done Done Sent false
11073 3596 Acked Read 9 Done Done Sent false
11074 3597 Acked Write 1 Done Done Sent false
11075 3598 Acked Write 3 Done Done Sent false
11076 3599 Acked Write 9 Done Done Sent false
11077 3600 Acked Read 7 Done Done Sent false
11078 3601 Acked Write 2 Done Done Sent false
11079 3602 Acked Write 2 Done Done Sent false
11080 3603 Acked Write 3 Done Done Sent false
11081 3604 Acked Read 1 Done Done Sent false
11082 3605 Acked Read 9 Done Done Sent false
11083 3606 Acked Read 7 Done Done Sent false
11084 3607 Acked Flush 0 Done Done Sent false
11085 3608 Acked Read 7 Done Done Sent false
11086 3609 Acked Write 9 Done Done Sent false
11087 3610 Acked Read 1 Done Done Sent false
11088 3611 Acked Read 4 Done Done Sent false
11089 3612 Acked Read 2 Done Done Sent false
11090 3613 Acked Write 10 Done Done Sent false
11091 3614 Acked Write 4 Done Done Sent false
11092 3615 Acked Read 5 Done Done Sent false
11093 3616 Acked Flush 0 Done Done Sent false
11094 3617 Acked Read 1 Done Done Sent false
11095 3618 Acked Read 8 Done Done Sent false
11096 3619 Acked Read 4 Done Done Sent false
11097 3620 Acked Write 3 Done Done Sent false
11098 3621 Acked Read 10 Done Done Sent false
11099 3622 Acked Write 8 Done Done Sent false
11100 3623 Acked Write 4 Done Done Sent false
11101 3624 Acked Read 1 Done Done Sent false
11102 3625 Acked Write 8 Done Done Sent false
11103 3626 Acked Write 5 Done Done Sent false
11104 3627 Acked Write 1 Done Done Sent false
11105 3628 Acked Read 2 Done Done Sent false
11106 3629 Acked Write 7 Done Done Sent false
11107 3630 Acked Write 5 Done Done Sent false
11108 3631 Acked Read 6 Done Done Sent false
11109 3632 Acked Write 2 Done Done Sent false
11110 3633 Acked Read 2 Done Done Sent false
11111 3634 Acked Write 3 Done Done Sent false
11112 3635 Acked Write 3 Done Done Sent false
11113 3636 Acked Read 8 Done Done Sent false
11114 3637 Acked Flush 0 Done Done Sent false
11115 3638 Acked Flush 0 Done Done Sent false
11116 3639 Acked Write 1 Done Done Sent false
11117 3640 Acked Write 8 Done Done Sent false
11118 3641 Acked Read 1 Done Done Sent false
11119 3642 Acked Read 3 Done Done Sent false
11120 3643 Acked Read 4 Done Done Sent false
11121 3644 Acked Read 4 Done Done Sent false
11122 3645 Acked Read 4 Done Done Sent false
11123 3646 Acked Write 10 Done Done Sent false
11124 3647 Acked Flush 0 Done Done Sent false
11125 3648 Acked Read 6 Done Done Sent false
11126 3649 Acked Read 7 Done Done Sent false
11127 3650 Acked Read 5 Done Done Sent false
11128 3651 Acked Read 6 Done Done Sent false
11129 3652 Acked Read 10 Done Done Sent false
11130 3653 Acked Read 6 Done Done Sent false
11131 3654 Acked Write 3 Done Done Sent false
11132 3655 Acked Read 5 Done Done Sent false
11133 3656 Acked Write 9 Done Done Sent false
11134 3657 Acked Write 3 Done Done Sent false
11135 3658 Acked Read 1 Done Done Sent false
11136 3659 Acked Read 10 Done Done Sent false
11137 3660 Acked Read 2 Done Done Sent false
11138 3661 Acked Read 4 Done Done Sent false
11139 3662 Acked Read 2 Done Done Sent false
11140 3663 Acked Write 7 Done Done Sent false
11141 3664 Acked Read 6 Done Done Sent false
11142 3665 Acked Read 9 Done Done Sent false
11143 3666 Acked Write 5 Done Done Sent false
11144 3667 Acked Read 6 Done Done Sent false
11145 3668 Acked Read 7 Done Done Sent false
11146 3669 Acked Read 9 Done Done Sent false
11147 3670 Acked Write 4 Done Done Sent false
11148 3671 Acked Write 8 Done Done Sent false
11149 3672 Acked Read 1 Done Done Sent false
11150 3673 Acked Flush 0 Done Done Sent false
11151 3674 Acked Flush 0 Done Done Sent false
11152 3675 Acked Read 8 Done Done Sent false
11153 3676 Acked Read 1 Done Done Sent false
11154 3677 Acked Read 9 Done Done Sent false
11155 3678 Acked Write 9 Done Done Sent false
11156 3679 Acked Write 6 Done Done Sent false
11157 3680 Acked Read 9 Done Done Sent false
11158 3681 Acked Read 2 Done Done Sent false
11159 3682 Acked Read 4 Done Done Sent false
11160 3683 Acked Write 10 Done Done Sent false
11161 3684 Acked Write 9 Done Done Sent false
11162 3685 Acked Read 6 Done Done Sent false
11163 3686 Acked Read 3 Done Done Sent false
11164 3687 Acked Flush 0 Done Done Sent false
11165 3688 Acked Read 10 Done Done Sent false
11166 3689 Acked Write 4 Done Done Sent false
11167 3690 Acked Write 4 Done Done Sent false
11168 3691 Acked Read 8 Done Done Sent false
11169 3692 Acked Write 7 Done Done Sent false
11170 3693 Acked Flush 0 Done Done Sent false
11171 3694 Acked Read 3 Done Done Sent false
11172 3695 Acked Read 5 Done Done Sent false
11173 3696 Acked Read 1 Done Done Sent false
11174 3697 Acked Write 9 Done Done Sent false
11175 3698 Acked Read 4 Done Done Sent false
11176 3699 Acked Read 5 Done Done Sent false
11177 3700 Acked Write 10 Done Done Sent false
11178 3701 Acked Write 2 Done Done Sent false
11179 3702 Acked Read 5 Done Done Sent false
11180 3703 Acked Read 7 Done Done Sent false
11181 3704 Acked Write 5 Done Done Sent false
11182 3705 Acked Read 4 Done Done Sent false
11183 3706 Acked Read 3 Done Done Sent false
11184 3707 Acked Flush 0 Done Done Sent false
11185 3708 Acked Read 8 Done Done Sent false
11186 3709 Acked Read 1 Done Done Sent false
11187 3710 Acked Flush 0 Done Done Sent false
11188 3711 Acked Write 5 Done Done Sent false
11189 3712 Acked Read 10 Done Done Sent false
11190 3713 Acked Read 9 Done Done Sent false
11191 3714 Acked Read 6 Done Done Sent false
11192 STATES DS:0 DS:1 DS:2 TOTAL
11193 Sent 0 0 301 301
11194 Done 301 301 0 602
11195 Skipped 0 0 0 0
11196 Error 0 0 0 0
11197 Last Flush: 3710 3710 3413
11198 Downstairs last five completed: 3413 3412 3411 3410 3409
11199 Upstairs last five completed: 3714 3713 3712 3711 3710
11200 CLIENT: Up:0 ds:301 act:2
11201 [8830] got start action
11202 [8830] I am going to restart
11203 Starting downstairs at port 8830
11204 Make output file at "/tmp/dsc/downstairs-8830.txt"
11205 [8830][2] reports Starting
11206 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1214)
11207 [8830][2] reports Running
11208 ----------------------------------------------------------------
11209 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
11210 States: Active Active Offline
11211 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
11212 3414 Acked Write 10 Done Done Sent false
11213 3415 Acked Read 5 Done Done Sent false
11214 3416 Acked Read 2 Done Done Sent false
11215 3417 Acked Read 1 Done Done Sent false
11216 3418 Acked Read 3 Done Done Sent false
11217 3419 Acked Write 8 Done Done Sent false
11218 3420 Acked Write 2 Done Done Sent false
11219 3421 Acked Write 9 Done Done Sent false
11220 3422 Acked Read 1 Done Done Sent false
11221 3423 Acked Write 7 Done Done Sent false
11222 3424 Acked Write 8 Done Done Sent false
11223 3425 Acked Read 5 Done Done Sent false
11224 3426 Acked Flush 0 Done Done Sent false
11225 3427 Acked Read 4 Done Done Sent false
11226 3428 Acked Flush 0 Done Done Sent false
11227 3429 Acked Read 10 Done Done Sent false
11228 3430 Acked Read 4 Done Done Sent false
11229 3431 Acked Read 9 Done Done Sent false
11230 3432 Acked Write 10 Done Done Sent false
11231 3433 Acked Flush 0 Done Done Sent false
11232 3434 Acked Read 9 Done Done Sent false
11233 3435 Acked Flush 0 Done Done Sent false
11234 3436 Acked Write 9 Done Done Sent false
11235 3437 Acked Read 10 Done Done Sent false
11236 3438 Acked Write 5 Done Done Sent false
11237 3439 Acked Read 9 Done Done Sent false
11238 3440 Acked Read 8 Done Done Sent false
11239 3441 Acked Read 7 Done Done Sent false
11240 3442 Acked Read 4 Done Done Sent false
11241 3443 Acked Write 7 Done Done Sent false
11242 3444 Acked Flush 0 Done Done Sent false
11243 3445 Acked Read 5 Done Done Sent false
11244 3446 Acked Read 8 Done Done Sent false
11245 3447 Acked Read 8 Done Done Sent false
11246 3448 Acked Write 8 Done Done Sent false
11247 3449 Acked Read 2 Done Done Sent false
11248 3450 Acked Write 9 Done Done Sent false
11249 3451 Acked Write 5 Done Done Sent false
11250 3452 Acked Write 4 Done Done Sent false
11251 3453 Acked Flush 0 Done Done Sent false
11252 3454 Acked Read 1 Done Done Sent false
11253 3455 Acked Read 6 Done Done Sent false
11254 3456 Acked Read 1 Done Done Sent false
11255 3457 Acked Read 5 Done Done Sent false
11256 3458 Acked Flush 0 Done Done Sent false
11257 3459 Acked Read 2 Done Done Sent false
11258 3460 Acked Write 1 Done Done Sent false
11259 3461 Acked Read 2 Done Done Sent false
11260 3462 Acked Read 4 Done Done Sent false
11261 3463 Acked Read 9 Done Done Sent false
11262 3464 Acked Write 2 Done Done Sent false
11263 3465 Acked Read 9 Done Done Sent false
11264 3466 Acked Read 6 Done Done Sent false
11265 3467 Acked Read 1 Done Done Sent false
11266 3468 Acked Flush 0 Done Done Sent false
11267 3469 Acked Flush 0 Done Done Sent false
11268 3470 Acked Write 5 Done Done Sent false
11269 3471 Acked Write 3 Done Done Sent false
11270 3472 Acked Read 4 Done Done Sent false
11271 3473 Acked Write 4 Done Done Sent false
11272 3474 Acked Read 9 Done Done Sent false
11273 3475 Acked Read 4 Done Done Sent false
11274 3476 Acked Flush 0 Done Done Sent false
11275 3477 Acked Read 2 Done Done Sent false
11276 3478 Acked Read 2 Done Done Sent false
11277 3479 Acked Read 2 Done Done Sent false
11278 3480 Acked Write 5 Done Done Sent false
11279 3481 Acked Read 9 Done Done Sent false
11280 3482 Acked Read 4 Done Done Sent false
11281 3483 Acked Flush 0 Done Done Sent false
11282 3484 Acked Write 7 Done Done Sent false
11283 3485 Acked Flush 0 Done Done Sent false
11284 3486 Acked Read 10 Done Done Sent false
11285 3487 Acked Write 6 Done Done Sent false
11286 3488 Acked Read 10 Done Done Sent false
11287 3489 Acked Write 4 Done Done Sent false
11288 3490 Acked Read 6 Done Done Sent false
11289 3491 Acked Read 6 Done Done Sent false
11290 3492 Acked Read 9 Done Done Sent false
11291 3493 Acked Read 6 Done Done Sent false
11292 3494 Acked Write 6 Done Done Sent false
11293 3495 Acked Read 1 Done Done Sent false
11294 3496 Acked Read 6 Done Done Sent false
11295 3497 Acked Read 10 Done Done Sent false
11296 3498 Acked Write 9 Done Done Sent false
11297 3499 Acked Write 1 Done Done Sent false
11298 3500 Acked Write 2 Done Done Sent false
11299 3501 Acked Read 7 Done Done Sent false
11300 3502 Acked Read 3 Done Done Sent false
11301 3503 Acked Read 7 Done Done Sent false
11302 3504 Acked Write 3 Done Done Sent false
11303 3505 Acked Read 5 Done Done Sent false
11304 3506 Acked Write 2 Done Done Sent false
11305 3507 Acked Read 7 Done Done Sent false
11306 3508 Acked Read 6 Done Done Sent false
11307 3509 Acked Read 8 Done Done Sent false
11308 3510 Acked Read 5 Done Done Sent false
11309 3511 Acked Write 8 Done Done Sent false
11310 3512 Acked Read 5 Done Done Sent false
11311 3513 Acked Flush 0 Done Done Sent false
11312 3514 Acked Read 1 Done Done Sent false
11313 3515 Acked Read 8 Done Done Sent false
11314 3516 Acked Write 1 Done Done Sent false
11315 3517 Acked Read 2 Done Done Sent false
11316 3518 Acked Read 5 Done Done Sent false
11317 3519 Acked Write 1 Done Done Sent false
11318 3520 Acked Read 2 Done Done Sent false
11319 3521 Acked Read 5 Done Done Sent false
11320 3522 Acked Write 2 Done Done Sent false
11321 3523 Acked Write 2 Done Done Sent false
11322 3524 Acked Read 6 Done Done Sent false
11323 3525 Acked Write 2 Done Done Sent false
11324 3526 Acked Write 7 Done Done Sent false
11325 3527 Acked Write 2 Done Done Sent false
11326 3528 Acked Write 1 Done Done Sent false
11327 3529 Acked Write 6 Done Done Sent false
11328 3530 Acked Read 5 Done Done Sent false
11329 3531 Acked Read 4 Done Done Sent false
11330 3532 Acked Read 5 Done Done Sent false
11331 3533 Acked Read 4 Done Done Sent false
11332 3534 Acked Write 1 Done Done Sent false
11333 3535 Acked Write 10 Done Done Sent false
11334 3536 Acked Write 8 Done Done Sent false
11335 3537 Acked Write 6 Done Done Sent false
11336 3538 Acked Write 7 Done Done Sent false
11337 3539 Acked Write 3 Done Done Sent false
11338 3540 Acked Write 1 Done Done Sent false
11339 3541 Acked Write 8 Done Done Sent false
11340 3542 Acked Write 10 Done Done Sent false
11341 3543 Acked Write 8 Done Done Sent false
11342 3544 Acked Read 5 Done Done Sent false
11343 3545 Acked Read 5 Done Done Sent false
11344 3546 Acked Write 5 Done Done Sent false
11345 3547 Acked Read 9 Done Done Sent false
11346 3548 Acked Read 1 Done Done Sent false
11347 3549 Acked Read 3 Done Done Sent false
11348 3550 Acked Read 10 Done Done Sent false
11349 3551 Acked Read 7 Done Done Sent false
11350 3552 Acked Write 3 Done Done Sent false
11351 3553 Acked Read 4 Done Done Sent false
11352 3554 Acked Write 6 Done Done Sent false
11353 3555 Acked Read 1 Done Done Sent false
11354 3556 Acked Read 8 Done Done Sent false
11355 3557 Acked Read 3 Done Done Sent false
11356 3558 Acked Read 1 Done Done Sent false
11357 3559 Acked Flush 0 Done Done Sent false
11358 3560 Acked Read 7 Done Done Sent false
11359 3561 Acked Read 3 Done Done Sent false
11360 3562 Acked Read 9 Done Done Sent false
11361 3563 Acked Flush 0 Done Done Sent false
11362 3564 Acked Read 9 Done Done Sent false
11363 3565 Acked Write 8 Done Done Sent false
11364 3566 Acked Read 7 Done Done Sent false
11365 3567 Acked Write 7 Done Done Sent false
11366 3568 Acked Read 4 Done Done Sent false
11367 3569 Acked Read 10 Done Done Sent false
11368 3570 Acked Read 4 Done Done Sent false
11369 3571 Acked Read 2 Done Done Sent false
11370 3572 Acked Read 5 Done Done Sent false
11371 3573 Acked Read 6 Done Done Sent false
11372 3574 Acked Write 2 Done Done Sent false
11373 3575 Acked Read 8 Done Done Sent false
11374 3576 Acked Read 9 Done Done Sent false
11375 3577 Acked Read 5 Done Done Sent false
11376 3578 Acked Flush 0 Done Done Sent false
11377 3579 Acked Write 2 Done Done Sent false
11378 3580 Acked Write 7 Done Done Sent false
11379 3581 Acked Read 8 Done Done Sent false
11380 3582 Acked Read 9 Done Done Sent false
11381 3583 Acked Write 9 Done Done Sent false
11382 3584 Acked Write 2 Done Done Sent false
11383 3585 Acked Write 7 Done Done Sent false
11384 3586 Acked Read 6 Done Done Sent false
11385 3587 Acked Write 7 Done Done Sent false
11386 3588 Acked Read 3 Done Done Sent false
11387 3589 Acked Read 9 Done Done Sent false
11388 3590 Acked Flush 0 Done Done Sent false
11389 3591 Acked Read 8 Done Done Sent false
11390 3592 Acked Read 2 Done Done Sent false
11391 3593 Acked Read 10 Done Done Sent false
11392 3594 Acked Read 3 Done Done Sent false
11393 3595 Acked Read 7 Done Done Sent false
11394 3596 Acked Read 9 Done Done Sent false
11395 3597 Acked Write 1 Done Done Sent false
11396 3598 Acked Write 3 Done Done Sent false
11397 3599 Acked Write 9 Done Done Sent false
11398 3600 Acked Read 7 Done Done Sent false
11399 3601 Acked Write 2 Done Done Sent false
11400 3602 Acked Write 2 Done Done Sent false
11401 3603 Acked Write 3 Done Done Sent false
11402 3604 Acked Read 1 Done Done Sent false
11403 3605 Acked Read 9 Done Done Sent false
11404 3606 Acked Read 7 Done Done Sent false
11405 3607 Acked Flush 0 Done Done Sent false
11406 3608 Acked Read 7 Done Done Sent false
11407 3609 Acked Write 9 Done Done Sent false
11408 3610 Acked Read 1 Done Done Sent false
11409 3611 Acked Read 4 Done Done Sent false
11410 3612 Acked Read 2 Done Done Sent false
11411 3613 Acked Write 10 Done Done Sent false
11412 3614 Acked Write 4 Done Done Sent false
11413 3615 Acked Read 5 Done Done Sent false
11414 3616 Acked Flush 0 Done Done Sent false
11415 3617 Acked Read 1 Done Done Sent false
11416 3618 Acked Read 8 Done Done Sent false
11417 3619 Acked Read 4 Done Done Sent false
11418 3620 Acked Write 3 Done Done Sent false
11419 3621 Acked Read 10 Done Done Sent false
11420 3622 Acked Write 8 Done Done Sent false
11421 3623 Acked Write 4 Done Done Sent false
11422 3624 Acked Read 1 Done Done Sent false
11423 3625 Acked Write 8 Done Done Sent false
11424 3626 Acked Write 5 Done Done Sent false
11425 3627 Acked Write 1 Done Done Sent false
11426 3628 Acked Read 2 Done Done Sent false
11427 3629 Acked Write 7 Done Done Sent false
11428 3630 Acked Write 5 Done Done Sent false
11429 3631 Acked Read 6 Done Done Sent false
11430 3632 Acked Write 2 Done Done Sent false
11431 3633 Acked Read 2 Done Done Sent false
11432 3634 Acked Write 3 Done Done Sent false
11433 3635 Acked Write 3 Done Done Sent false
11434 3636 Acked Read 8 Done Done Sent false
11435 3637 Acked Flush 0 Done Done Sent false
11436 3638 Acked Flush 0 Done Done Sent false
11437 3639 Acked Write 1 Done Done Sent false
11438 3640 Acked Write 8 Done Done Sent false
11439 3641 Acked Read 1 Done Done Sent false
11440 3642 Acked Read 3 Done Done Sent false
11441 3643 Acked Read 4 Done Done Sent false
11442 3644 Acked Read 4 Done Done Sent false
11443 3645 Acked Read 4 Done Done Sent false
11444 3646 Acked Write 10 Done Done Sent false
11445 3647 Acked Flush 0 Done Done Sent false
11446 3648 Acked Read 6 Done Done Sent false
11447 3649 Acked Read 7 Done Done Sent false
11448 3650 Acked Read 5 Done Done Sent false
11449 3651 Acked Read 6 Done Done Sent false
11450 3652 Acked Read 10 Done Done Sent false
11451 3653 Acked Read 6 Done Done Sent false
11452 3654 Acked Write 3 Done Done Sent false
11453 3655 Acked Read 5 Done Done Sent false
11454 3656 Acked Write 9 Done Done Sent false
11455 3657 Acked Write 3 Done Done Sent false
11456 3658 Acked Read 1 Done Done Sent false
11457 3659 Acked Read 10 Done Done Sent false
11458 3660 Acked Read 2 Done Done Sent false
11459 3661 Acked Read 4 Done Done Sent false
11460 3662 Acked Read 2 Done Done Sent false
11461 3663 Acked Write 7 Done Done Sent false
11462 3664 Acked Read 6 Done Done Sent false
11463 3665 Acked Read 9 Done Done Sent false
11464 3666 Acked Write 5 Done Done Sent false
11465 3667 Acked Read 6 Done Done Sent false
11466 3668 Acked Read 7 Done Done Sent false
11467 3669 Acked Read 9 Done Done Sent false
11468 3670 Acked Write 4 Done Done Sent false
11469 3671 Acked Write 8 Done Done Sent false
11470 3672 Acked Read 1 Done Done Sent false
11471 3673 Acked Flush 0 Done Done Sent false
11472 3674 Acked Flush 0 Done Done Sent false
11473 3675 Acked Read 8 Done Done Sent false
11474 3676 Acked Read 1 Done Done Sent false
11475 3677 Acked Read 9 Done Done Sent false
11476 3678 Acked Write 9 Done Done Sent false
11477 3679 Acked Write 6 Done Done Sent false
11478 3680 Acked Read 9 Done Done Sent false
11479 3681 Acked Read 2 Done Done Sent false
11480 3682 Acked Read 4 Done Done Sent false
11481 3683 Acked Write 10 Done Done Sent false
11482 3684 Acked Write 9 Done Done Sent false
11483 3685 Acked Read 6 Done Done Sent false
11484 3686 Acked Read 3 Done Done Sent false
11485 3687 Acked Flush 0 Done Done Sent false
11486 3688 Acked Read 10 Done Done Sent false
11487 3689 Acked Write 4 Done Done Sent false
11488 3690 Acked Write 4 Done Done Sent false
11489 3691 Acked Read 8 Done Done Sent false
11490 3692 Acked Write 7 Done Done Sent false
11491 3693 Acked Flush 0 Done Done Sent false
11492 3694 Acked Read 3 Done Done Sent false
11493 3695 Acked Read 5 Done Done Sent false
11494 3696 Acked Read 1 Done Done Sent false
11495 3697 Acked Write 9 Done Done Sent false
11496 3698 Acked Read 4 Done Done Sent false
11497 3699 Acked Read 5 Done Done Sent false
11498 3700 Acked Write 10 Done Done Sent false
11499 3701 Acked Write 2 Done Done Sent false
11500 3702 Acked Read 5 Done Done Sent false
11501 3703 Acked Read 7 Done Done Sent false
11502 3704 Acked Write 5 Done Done Sent false
11503 3705 Acked Read 4 Done Done Sent false
11504 3706 Acked Read 3 Done Done Sent false
11505 3707 Acked Flush 0 Done Done Sent false
11506 3708 Acked Read 8 Done Done Sent false
11507 3709 Acked Read 1 Done Done Sent false
11508 3710 Acked Flush 0 Done Done Sent false
11509 3711 Acked Write 5 Done Done Sent false
11510 3712 Acked Read 10 Done Done Sent false
11511 3713 Acked Read 9 Done Done Sent false
11512 3714 Acked Read 6 Done Done Sent false
11513 3715 Acked Flush 0 Done Done Sent false
11514 STATES DS:0 DS:1 DS:2 TOTAL
11515 Sent 0 0 302 302
11516 Done 302 302 0 604
11517 Skipped 0 0 0 0
11518 Error 0 0 0 0
11519 Last Flush: 3715 3715 3413
11520 Downstairs last five completed: 3413 3412 3411 3410 3409
11521 Upstairs last five completed: 3715 3714 3713 3712 3711
11522 CLIENT: Up:0 ds:302 act:2
115232024-11-15T01:59:51.841ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
115242024-11-15T01:59:51.841ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:47088) = io task client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
115252024-11-15T01:59:51.842ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
115262024-11-15T01:59:51.842ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
115272024-11-15T01:59:51.842ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
115282024-11-15T01:59:51.842ZINFOcrucible: send last flush ID to this DS: 3413 = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
115292024-11-15T01:59:51.842ZINFOcrucible: Replied this last flush ID: 3413 = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
115302024-11-15T01:59:51.842ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
115312024-11-15T01:59:51.842ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
115322024-11-15T01:59:51.842ZINFOcrucible: [2] client re-new 302 jobs since flush 3413 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
115332024-11-15T01:59:51.843ZINFOcrucible: [2] Replayed 302 jobs since flush: 3413 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
11534 ----------------------------------------------------------------
11535 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
11536 Downstairs last five completed: 3715 3714 3713 3712 3711
11537 Upstairs last five completed: 3715 3714 3713 3712 3711
11538 CLIENT: Up:0 ds:0 act:3
11539 Replay: All jobs finished
11540 Main task has work to do, go find it
11541 got dsc Stop(0)
11542 stop 0
11543 Nov 15 01:59:54.092 [8810] Got stop action so:false kr:false
11544 INFO request completed, latency_us: 332, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 57fa6ee5-0325-40d0-b3d5-e09061e304b2, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
115452024-11-15T01:59:54.092ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
115462024-11-15T01:59:54.093ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
115472024-11-15T01:59:54.093ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
115482024-11-15T01:59:54.093ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
11549 Nov 15 01:59:54.093 INFO request completed, latency_us: 159, response_code: 200, uri: /state/cid/0, method: GET, req_id: 3006773f-f52c-4fd4-8a5a-e758499536ab, remote_addr: 127.0.0.1:40008{"msg":"[0] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-15T01:59:54.093193262Z","hostname":"w-01JCPRHT5F2MPA44KVJHM3S2PB","pid":1205,"client":"0","":"downstairs","session_id":"971dcb2e-6891-4ecf-b367-1e04e52065ee"}
11550 , local_addr: 127.0.0.1:9998
115512024-11-15T01:59:54.093ZINFOcrucible: client task is exiting = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
11552 [8810] Exited with: ExitStatus(unix_wait_status(9))
11553 [8810][0] reports Exit
11554 Nov 15 01:59:58.095 INFO request completed, latency_us: 216, response_code: 200, uri: /state/cid/0, method: GET, req_id: b76097ee-9c83-40b8-b2f1-47b73fe22d86, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
11555 001/300 Write block 3674 len 20480 data: 4 4 4 4 4
11556 002/300 Write block 837 len 20480 data: 3 3 4 4 6
11557 003/300 Read block 3402 len 12288
11558 004/300 Read block 1539 len 20480
11559 005/300 Write block 3956 len 28672 data: 5 5 5 5 4 4 2
11560 006/300 Flush
11561 007/300 Write block 4206 len 8192 data: 6 5
11562 008/300 Write block 4675 len 28672 data: 2 2 2 2 2 2 2
11563 009/300 Write block 543 len 32768 data: 3 3 3 3 3 3 2 3
11564 010/300 Read block 4098 len 12288
11565 011/300 Write block 1932 len 40960 data: 5 5 6 4 3 3 4 4 4 4
11566 012/300 Write block 4164 len 32768 data: 2 2 2 2 2 2 2 2
11567 013/300 Write block 1957 len 20480 data: 3 3 3 2 2
11568 014/300 Write block 1844 len 16384 data: 4 5 4 4
11569 015/300 Read block 135 len 28672
11570 016/300 Write block 1681 len 4096 data: 4
11571 017/300 Read block 3858 len 24576
11572 018/300 Write block 2894 len 24576 data: 3 2 3 4 4 4
11573 019/300 Write block 457 len 28672 data: 2 2 2 3 4 4 4
11574 020/300 Write block 4195 len 16384 data: 6 6 6 4
11575 021/300 Flush
11576 022/300 Write block 83 len 40960 data: 3 3 3 3 4 5 5 4 3 3
11577 023/300 Read block 2911 len 24576
11578 024/300 Write block 3884 len 20480 data: 4 5 4 4 4
11579 025/300 Read block 77 len 40960
11580 026/300 Read block 4739 len 16384
11581 027/300 Flush
11582 028/300 Read block 1167 len 4096
11583 029/300 Flush
11584 030/300 Write block 3075 len 4096 data: 6
11585 031/300 Read block 2683 len 36864
11586 032/300 Write block 2847 len 12288 data: 3 3 2
11587 033/300 Read block 1524 len 28672
11588 034/300 Write block 1230 len 8192 data: 2 2
11589 035/300 Write block 4038 len 12288 data: 6 6 6
11590 036/300 Write block 2516 len 4096 data: 4
11591 037/300 Write block 2751 len 4096 data: 2
11592 038/300 Write block 2460 len 20480 data: 6 6 5 5 5
11593 039/300 Write block 3348 len 28672 data: 3 4 4 4 5 4 4
11594 040/300 Read block 468 len 8192
11595 041/300 Write block 2262 len 24576 data: 3 3 3 3 3 3
11596 042/300 Read block 2156 len 32768
11597 043/300 Read block 3551 len 36864
11598 044/300 Read block 2868 len 16384
11599 045/300 Write block 2508 len 16384 data: 3 3 3 2
11600 046/300 Flush
11601 047/300 Write block 4326 len 40960 data: 3 4 4 4 4 5 5 4 3 3
11602 048/300 Read block 2680 len 36864
11603 049/300 Flush
11604 050/300 Write block 2201 len 20480 data: 3 3 3 2 2
11605 051/300 Read block 1329 len 8192
11606 052/300 Read block 3578 len 24576
11607 053/300 Write block 471 len 28672 data: 2 2 3 2 2 3 5
11608 054/300 Write block 3903 len 28672 data: 6 6 5 5 3 3 3
11609 055/300 Read block 1875 len 8192
11610 056/300 Read block 2205 len 16384
11611 057/300 Read block 832 len 40960
11612 058/300 Write block 1271 len 20480 data: 3 3 3 3 4
11613 059/300 Read block 3915 len 36864
11614 060/300 Write block 3538 len 36864 data: 4 3 2 2 2 2 2 2 2
11615 061/300 Write block 3467 len 8192 data: 2 3
11616 062/300 Write block 4280 len 40960 data: 3 3 4 3 3 3 3 3 3 4
11617 063/300 Write block 4786 len 8192 data: 2 2
11618 064/300 Read block 574 len 28672
11619 065/300 Write block 1118 len 16384 data: 3 4 4 5
11620 066/300 Read block 4947 len 40960
11621 067/300 Read block 4949 len 40960
11622 068/300 Read block 2383 len 24576
11623 069/300 Write block 1776 len 24576 data: 4 4 4 4 4 4
11624 070/300 Write block 4626 len 4096 data: 2
11625 071/300 Write block 1979 len 32768 data: 2 2 2 3 3 3 3 4
11626 072/300 Read block 4209 len 8192
11627 073/300 Read block 928 len 36864
11628 074/300 Read block 3219 len 12288
11629 075/300 Read block 3544 len 8192
11630 076/300 Write block 3898 len 36864 data: 5 5 5 6 6 7 7 6 6
11631 077/300 Write block 3368 len 36864 data: 2 3 3 2 3 3 3 3 3
11632 078/300 Read block 2512 len 28672
11633 079/300 Read block 2774 len 8192
11634 080/300 Read block 556 len 12288
11635 081/300 Read block 3198 len 24576
11636 082/300 Write block 3000 len 36864 data: 3 3 2 2 2 2 2 2 2
11637 083/300 Flush
11638 084/300 Read block 3807 len 28672
11639 085/300 Write block 943 len 4096 data: 3
11640 086/300 Read block 2532 len 36864
11641 087/300 Read block 4185 len 28672
11642 088/300 Flush
11643 089/300 Read block 1511 len 16384
11644 090/300 Read block 1331 len 28672
11645 091/300 Read block 4271 len 28672
11646 092/300 Write block 4120 len 12288 data: 2 2 3
11647 093/300 Write block 3967 len 32768 data: 2 2 2 2 2 2 2 2
11648 094/300 Write block 3968 len 20480 data: 3 3 3 3 3
11649 095/300 Read block 3627 len 36864
11650 096/300 Write block 4865 len 12288 data: 3 3 3
11651 097/300 Write block 4789 len 28672 data: 2 2 3 3 3 4 4
11652 098/300 Read block 2105 len 36864
11653 099/300 Read block 3708 len 12288
11654 100/300 Flush
11655 101/300 Write block 1339 len 24576 data: 2 2 2 2 2 2
11656 102/300 Read block 2948 len 36864
11657 103/300 Read block 3140 len 12288
11658 104/300 Read block 3093 len 36864
11659 105/300 Read block 750 len 40960
11660 106/300 Write block 3022 len 12288 data: 3 3 3
11661 107/300 Write block 309 len 20480 data: 3 3 3 3 2
11662 108/300 Write block 1852 len 36864 data: 2 3 3 3 3 3 3 3 3
11663 109/300 Write block 215 len 28672 data: 3 3 3 3 2 2 2
11664 110/300 Read block 1383 len 8192
11665 111/300 Read block 215 len 8192
11666 112/300 Write block 3658 len 28672 data: 2 2 2 2 2 2 2
11667 113/300 Read block 1953 len 36864
11668 114/300 Flush
11669 115/300 Read block 2980 len 8192
11670 116/300 Write block 4796 len 36864 data: 4 4 4 4 3 3 2 2 3
11671 117/300 Write block 4683 len 24576 data: 3 5 4 4 3 2
11672 118/300 Read block 2336 len 32768
11673 119/300 Write block 2833 len 24576 data: 2 2 2 3 3 2
11674 120/300 Read block 1734 len 40960
11675 121/300 Read block 108 len 20480
11676 122/300 Write block 3316 len 28672 data: 2 2 2 2 2 2 2
11677 123/300 Write block 1338 len 40960 data: 2 3 3 3 3 3 3 2 2 2
11678 124/300 Read block 3409 len 28672
11679 125/300 Read block 4370 len 20480
11680 126/300 Read block 1957 len 4096
11681 127/300 Write block 4593 len 4096 data: 3
11682 128/300 Flush
11683 129/300 Write block 3180 len 16384 data: 2 2 2 3
11684 130/300 Flush
11685 131/300 Write block 3057 len 4096 data: 3
11686 132/300 Write block 2558 len 8192 data: 2 2
11687 133/300 Flush
11688 134/300 Read block 3751 len 20480
11689 135/300 Write block 3110 len 24576 data: 2 2 2 2 2 2
11690 136/300 Read block 1286 len 12288
11691 137/300 Read block 2775 len 8192
11692 138/300 Read block 4292 len 4096
11693 139/300 Write block 4312 len 24576 data: 4 4 4 4 5 5
11694 140/300 Write block 4784 len 32768 data: 4 2 3 3 2 3 3 4
11695 141/300 Read block 4480 len 32768
11696 142/300 Write block 1024 len 32768 data: 4 3 3 3 2 3 3 3
11697 143/300 Read block 2227 len 24576
11698 144/300 Read block 3695 len 28672
11699 145/300 Read block 1773 len 24576
11700 146/300 Write block 4901 len 24576 data: 3 4 5 5 3 3
11701 147/300 Read block 4470 len 16384
11702 148/300 Read block 1518 len 36864
11703 149/300 Write block 3245 len 16384 data: 4 4 4 4
11704 150/300 Read block 2987 len 36864
11705 151/300 Read block 2901 len 24576
11706 152/300 Write block 1193 len 32768 data: 3 4 4 6 6 6 6 4
11707 153/300 Read block 3971 len 28672
11708 154/300 Read block 4990 len 8192
11709 155/300 Write block 3949 len 16384 data: 2 2 2 3
11710 156/300 Read block 226 len 36864
11711 157/300 Read block 279 len 40960
11712 158/300 Write block 3850 len 20480 data: 3 3 3 3 3
11713 159/300 Read block 2480 len 4096
11714 160/300 Write block 1063 len 8192 data: 2 2
11715 161/300 Read block 2216 len 24576
11716 162/300 Read block 1362 len 4096
11717 163/300 Read block 2557 len 40960
11718 164/300 Read block 3218 len 12288
11719 165/300 Flush
11720 166/300 Write block 2817 len 40960 data: 3 4 4 3 4 5 5 5 3 3
11721 167/300 Write block 4612 len 40960 data: 3 2 2 2 2 2 2 2 2 2
11722 168/300 Write block 4512 len 20480 data: 3 5 6 5 6
11723 169/300 Flush
11724 170/300 Flush
11725 171/300 Flush
11726 172/300 Read block 4182 len 24576
11727 173/300 Read block 1964 len 28672
11728 174/300 Read block 4482 len 36864
11729 175/300 Read block 2388 len 4096
11730 176/300 Write block 3372 len 12288 data: 4 4 4
11731 177/300 Flush
11732 178/300 Flush
11733 179/300 Read block 2795 len 28672
11734 180/300 Write block 4228 len 16384 data: 4 5 3 4
11735 181/300 Write block 2482 len 12288 data: 2 2 2
11736 182/300 Read block 2885 len 24576
11737 183/300 Write block 4608 len 4096 data: 3
11738 184/300 Write block 2481 len 32768 data: 2 3 3 3 2 3 3 3
11739 185/300 Write block 3764 len 4096 data: 3
11740 186/300 Write block 4956 len 8192 data: 3 2
11741 187/300 Write block 4724 len 32768 data: 3 3 3 4 3 3 4 3
11742 188/300 Read block 2380 len 20480
11743 189/300 Flush
11744 190/300 Read block 3059 len 8192
11745 191/300 Flush
11746 192/300 Write block 4122 len 28672 data: 4 3 4 4 4 3 3
11747 193/300 Read block 4588 len 32768
11748 194/300 Read block 850 len 20480
11749 195/300 Read block 1190 len 36864
11750 196/300 Read block 669 len 16384
11751 197/300 Read block 2520 len 8192
11752 198/300 Read block 4031 len 24576
11753 199/300 Read block 2517 len 32768
11754 200/300 Read block 2524 len 20480
11755 201/300 Read block 2282 len 12288
11756 202/300 Read block 3874 len 28672
11757 203/300 Flush
11758 204/300 Read block 3764 len 20480
11759 205/300 Write block 2550 len 20480 data: 3 2 2 2 2
11760 206/300 Write block 79 len 40960 data: 6 6 6 5 4 4 4 4 5 6
11761 207/300 Read block 2573 len 12288
11762 208/300 Read block 4775 len 16384
11763 209/300 Read block 2003 len 28672
11764 210/300 Read block 4517 len 16384
11765 211/300 Read block 983 len 28672
11766 212/300 Write block 2232 len 28672 data: 4 4 3 2 3 4 4
11767 213/300 Read block 2304 len 16384
11768 214/300 Write block 638 len 12288 data: 3 3 3
11769 215/300 Flush
11770 216/300 Read block 1645 len 24576
11771 217/300 Write block 2557 len 8192 data: 2 3
11772 218/300 Read block 109 len 36864
11773 219/300 Flush
11774 220/300 Flush
11775 221/300 Read block 945 len 36864
11776 222/300 Read block 3678 len 28672
11777 223/300 Write block 2426 len 40960 data: 3 4 4 4 4 4 4 4 3 2
11778 224/300 Read block 3378 len 32768
11779 225/300 Read block 4111 len 40960
11780 226/300 Read block 1958 len 20480
11781 227/300 Flush
11782 228/300 Write block 1276 len 24576 data: 4 4 4 3 4 4
11783 229/300 Read block 1336 len 32768
11784 230/300 Read block 305 len 40960
11785 231/300 Read block 4940 len 20480
11786 232/300 Read block 4624 len 20480
11787 233/300 Flush
11788 234/300 Read block 911 len 24576
11789 235/300 Write block 3179 len 40960 data: 2 3 3 3 4 3 4 4 5 6
11790 236/300 Flush
11791 237/300 Read block 169 len 8192
11792 238/300 Write block 2062 len 4096 data: 5
11793 239/300 Read block 3844 len 8192
11794 240/300 Write block 4870 len 12288 data: 2 2 2
11795 241/300 Read block 3772 len 36864
11796 242/300 Read block 2983 len 12288
11797 243/300 Read block 3400 len 20480
11798 244/300 Read block 4362 len 4096
11799 245/300 Read block 3105 len 4096
11800 246/300 Flush
11801 247/300 Read block 1995 len 40960
11802 248/300 Read block 2236 len 8192
11803 249/300 Write block 2938 len 40960 data: 5 5 5 4 4 5 5 5 6 6
11804 250/300 Write block 419 len 16384 data: 5 5 5 4
11805 251/300 Flush
11806 252/300 Write block 4435 len 32768 data: 4 4 3 3 3 3 3 3
11807 253/300 Write block 2384 len 16384 data: 4 4 4 5
11808 254/300 Flush
11809 255/300 Write block 759 len 24576 data: 5 3 3 3 2 3
11810 256/300 Read block 142 len 20480
11811 257/300 Write block 2741 len 4096 data: 3
11812 258/300 Write block 1617 len 24576 data: 2 2 2 2 2 2
11813 259/300 Write block 135 len 8192 data: 3 3
11814 260/300 Write block 4756 len 8192 data: 2 2
11815 261/300 Read block 1233 len 32768
11816 262/300 Read block 2495 len 36864
11817 263/300 Flush
11818 264/300 Write block 4703 len 36864 data: 3 4 4 5 3 4 5 5 5
11819 265/300 Write block 2482 len 20480 data: 4 4 4 3 4
11820 266/300 Write block 3606 len 16384 data: 3 2 2 2
11821 267/300 Read block 4226 len 16384
11822 268/300 Read block 232 len 40960
11823 269/300 Write block 4401 len 16384 data: 2 2 2 2
11824 270/300 Read block 3461 len 36864
11825 271/300 Read block 3099 len 24576
11826 272/300 Write block 4026 len 40960 data: 4 4 3 2 2 2 2 2 2 4
11827 273/300 Write block 1831 len 16384 data: 3 3 2 2
11828 274/300 Read block 1547 len 36864
11829 275/300 Read block 93 len 12288
11830 276/300 Write block 3912 len 12288 data: 2 2 2
11831 277/300 Read block 3433 len 36864
11832 278/300 Read block 1531 len 28672
11833 279/300 Write block 700 len 40960 data: 4 5 5 6 6 7 4 4 4 4
11834 280/300 Read block 1601 len 4096
11835 281/300 Write block 191 len 40960 data: 3 3 3 3 3 3 3 3 3 3
11836 282/300 Read block 2546 len 8192
11837 283/300 Read block 1617 len 12288
11838 284/300 Read block 222 len 40960
11839 285/300 Read block 1313 len 28672
11840 286/300 Read block 3394 len 40960
11841 287/300 Read block 2275 len 12288
11842 288/300 Read block 307 len 12288
11843 289/300 Read block 4318 len 4096
11844 290/300 Write block 3046 len 24576 data: 3 2 2 2 2 3
11845 291/300 Read block 4319 len 4096
11846 292/300 Read block 352 len 40960
11847 293/300 Read block 3784 len 12288
11848 294/300 Read block 3898 len 8192
11849 295/300 Write block 2387 len 32768 data: 6 4 5 3 2 2 3 3
11850 296/300 Flush
11851 297/300 Read block 2219 len 20480
11852 298/300 Flush
11853 299/300 Write block 3376 len 36864 data: 4 3 2 3 3 3 3 3 3
11854 300/300 Read block 4706 len 20480
11855 301/300 Write block 1658 len 40960 data: 4 4 5 6 6 8 9 9 8 7
11856 Main task has work to do, go find it
11857 got dsc Start(0)
11858 start 0
11859 Nov 15 01:59:58.334 INFO request completed, latency_us: 293, response_code: 204, uri: /start/cid/0, method: POST, req_id: ad329d9f-38d9-4f5f-975a-f8384e9f1733, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
11860 Replay: started 0, returned:Ok(())
11861 ----------------------------------------------------------------
11862 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
11863 States: Offline Active Active
11864 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
11865 3716 Acked Write 5 Sent Done Done false
11866 3717 Acked Write 5 Sent Done Done false
11867 3718 Acked Read 3 Sent Done Done false
11868 3719 Acked Read 5 Sent Done Done false
11869 3720 Acked Write 7 Sent Done Done false
11870 3721 Acked Flush 0 Sent Done Done false
11871 3722 Acked Write 2 Sent Done Done false
11872 3723 Acked Write 7 Sent Done Done false
11873 3724 Acked Write 8 Sent Done Done false
11874 3725 Acked Read 3 Sent Done Done false
11875 3726 Acked Write 10 Sent Done Done false
11876 3727 Acked Write 8 Sent Done Done false
11877 3728 Acked Write 5 Sent Done Done false
11878 3729 Acked Write 4 Sent Done Done false
11879 3730 Acked Read 7 Sent Done Done false
11880 3731 Acked Write 1 Sent Done Done false
11881 3732 Acked Read 6 Sent Done Done false
11882 3733 Acked Write 6 Sent Done Done false
11883 3734 Acked Write 7 Sent Done Done false
11884 3735 Acked Write 4 Sent Done Done false
11885 3736 Acked Flush 0 Sent Done Done false
11886 3737 Acked Write 10 Sent Done Done false
11887 3738 Acked Read 6 Sent Done Done false
11888 3739 Acked Write 5 Sent Done Done false
11889 3740 Acked Read 10 Sent Done Done false
11890 3741 Acked Read 4 Sent Done Done false
11891 3742 Acked Flush 0 Sent Done Done false
11892 3743 Acked Read 1 Sent Done Done false
11893 3744 Acked Flush 0 Sent Done Done false
11894 3745 Acked Write 1 Sent Done Done false
11895 3746 Acked Read 9 Sent Done Done false
11896 3747 Acked Write 3 Sent Done Done false
11897 3748 Acked Read 7 Sent Done Done false
11898 3749 Acked Write 2 Sent Done Done false
11899 3750 Acked Write 3 Sent Done Done false
11900 3751 Acked Write 1 Sent Done Done false
11901 3752 Acked Write 1 Sent Done Done false
11902 3753 Acked Write 5 Sent Done Done false
11903 3754 Acked Write 7 Sent Done Done false
11904 3755 Acked Read 2 Sent Done Done false
11905 3756 Acked Write 6 Sent Done Done false
11906 3757 Acked Read 8 Sent Done Done false
11907 3758 Acked Read 9 Sent Done Done false
11908 3759 Acked Read 4 Sent Done Done false
11909 3760 Acked Write 4 Sent Done Done false
11910 3761 Acked Flush 0 Sent Done Done false
11911 3762 Acked Write 10 Sent Done Done false
11912 3763 Acked Read 9 Sent Done Done false
11913 3764 Acked Flush 0 Sent Done Done false
11914 3765 Acked Write 5 Sent Done Done false
11915 3766 Acked Read 2 Sent Done Done false
11916 3767 Acked Read 6 Sent Done Done false
11917 3768 Acked Write 7 Sent Done Done false
11918 3769 Acked Write 7 Sent Done Done false
11919 3770 Acked Read 2 Sent Done Done false
11920 3771 Acked Read 4 Sent Done Done false
11921 3772 Acked Read 10 Sent Done Done false
11922 3773 Acked Write 5 Sent Done Done false
11923 3774 Acked Read 9 Sent Done Done false
11924 3775 Acked Write 9 Sent Done Done false
11925 3776 Acked Write 2 Sent Done Done false
11926 3777 Acked Write 10 Sent Done Done false
11927 3778 Acked Write 2 Sent Done Done false
11928 3779 Acked Read 7 Sent Done Done false
11929 3780 Acked Write 4 Sent Done Done false
11930 3781 Acked Read 10 Sent Done Done false
11931 3782 Acked Read 10 Sent Done Done false
11932 3783 Acked Read 6 Sent Done Done false
11933 3784 Acked Write 6 Sent Done Done false
11934 3785 Acked Write 1 Sent Done Done false
11935 3786 Acked Write 8 Sent Done Done false
11936 3787 Acked Read 2 Sent Done Done false
11937 3788 Acked Read 9 Sent Done Done false
11938 3789 Acked Read 3 Sent Done Done false
11939 3790 Acked Read 2 Sent Done Done false
11940 3791 Acked Write 9 Sent Done Done false
11941 3792 Acked Write 9 Sent Done Done false
11942 3793 Acked Read 7 Sent Done Done false
11943 3794 Acked Read 2 Sent Done Done false
11944 3795 Acked Read 3 Sent Done Done false
11945 3796 Acked Read 6 Sent Done Done false
11946 3797 Acked Write 9 Sent Done Done false
11947 3798 Acked Flush 0 Sent Done Done false
11948 3799 Acked Read 7 Sent Done Done false
11949 3800 Acked Write 1 Sent Done Done false
11950 3801 Acked Read 9 Sent Done Done false
11951 3802 Acked Read 7 Sent Done Done false
11952 3803 Acked Flush 0 Sent Done Done false
11953 3804 Acked Read 4 Sent Done Done false
11954 3805 Acked Read 7 Sent Done Done false
11955 3806 Acked Read 7 Sent Done Done false
11956 3807 Acked Write 3 Sent Done Done false
11957 3808 Acked Write 8 Sent Done Done false
11958 3809 Acked Write 5 Sent Done Done false
11959 3810 Acked Read 9 Sent Done Done false
11960 3811 Acked Write 3 Sent Done Done false
11961 3812 Acked Write 7 Sent Done Done false
11962 3813 Acked Read 9 Sent Done Done false
11963 3814 Acked Read 3 Sent Done Done false
11964 3815 Acked Flush 0 Sent Done Done false
11965 3816 Acked Write 6 Sent Done Done false
11966 3817 Acked Read 9 Sent Done Done false
11967 3818 Acked Read 3 Sent Done Done false
11968 3819 Acked Read 9 Sent Done Done false
11969 3820 Acked Read 10 Sent Done Done false
11970 3821 Acked Write 3 Sent Done Done false
11971 3822 Acked Write 5 Sent Done Done false
11972 3823 Acked Write 9 Sent Done Done false
11973 3824 Acked Write 7 Sent Done Done false
11974 3825 Acked Read 2 Sent Done Done false
11975 3826 Acked Read 2 Sent Done Done false
11976 3827 Acked Write 7 Sent Done Done false
11977 3828 Acked Read 9 Sent Done Done false
11978 3829 Acked Flush 0 Sent Done Done false
11979 3830 Acked Read 2 Sent Done Done false
11980 3831 Acked Write 9 Sent Done Done false
11981 3832 Acked Write 6 Sent Done Done false
11982 3833 Acked Read 8 Sent Done Done false
11983 3834 Acked Write 6 Sent Done Done false
11984 3835 Acked Read 10 Sent Done Done false
11985 3836 Acked Read 5 Sent Done Done false
11986 3837 Acked Write 7 Sent Done Done false
11987 3838 Acked Write 10 Sent Done Done false
11988 3839 Acked Read 7 Sent Done Done false
11989 3840 Acked Read 5 Sent Done Done false
11990 3841 Acked Read 1 Sent Done Done false
11991 3842 Acked Write 1 Sent Done Done false
11992 3843 Acked Flush 0 Sent Done Done false
11993 3844 Acked Write 4 Sent Done Done false
11994 3845 Acked Flush 0 Sent Done Done false
11995 3846 Acked Write 1 Sent Done Done false
11996 3847 Acked Write 2 Sent Done Done false
11997 3848 Acked Flush 0 Sent Done Done false
11998 3849 Acked Read 5 Sent Done Done false
11999 3850 Acked Write 6 Sent Done Done false
12000 3851 Acked Read 3 Sent Done Done false
12001 3852 Acked Read 2 Sent Done Done false
12002 3853 Acked Read 1 Sent Done Done false
12003 3854 Acked Write 6 Sent Done Done false
12004 3855 Acked Write 8 Sent Done Done false
12005 3856 Acked Read 8 Sent Done Done false
12006 3857 Acked Write 8 Sent Done Done false
12007 3858 Acked Read 6 Sent Done Done false
12008 3859 Acked Read 7 Sent Done Done false
12009 3860 Acked Read 6 Sent Done Done false
12010 3861 Acked Write 6 Sent Done Done false
12011 3862 Acked Read 4 Sent Done Done false
12012 3863 Acked Read 9 Sent Done Done false
12013 3864 Acked Write 4 Sent Done Done false
12014 3865 Acked Read 9 Sent Done Done false
12015 3866 Acked Read 6 Sent Done Done false
12016 3867 Acked Write 8 Sent Done Done false
12017 3868 Acked Read 7 Sent Done Done false
12018 3869 Acked Read 2 Sent Done Done false
12019 3870 Acked Write 4 Sent Done Done false
12020 3871 Acked Read 9 Sent Done Done false
12021 3872 Acked Read 10 Sent Done Done false
12022 3873 Acked Write 5 Sent Done Done false
12023 3874 Acked Read 1 Sent Done Done false
12024 3875 Acked Write 2 Sent Done Done false
12025 3876 Acked Read 6 Sent Done Done false
12026 3877 Acked Read 1 Sent Done Done false
12027 3878 Acked Read 10 Sent Done Done false
12028 3879 Acked Read 3 Sent Done Done false
12029 3880 Acked Flush 0 Sent Done Done false
12030 3881 Acked Write 10 Sent Done Done false
12031 3882 Acked Write 10 Sent Done Done false
12032 3883 Acked Write 5 Sent Done Done false
12033 3884 Acked Flush 0 Sent Done Done false
12034 3885 Acked Flush 0 Sent Done Done false
12035 3886 Acked Flush 0 Sent Done Done false
12036 3887 Acked Read 6 Sent Done Done false
12037 3888 Acked Read 7 Sent Done Done false
12038 3889 Acked Read 9 Sent Done Done false
12039 3890 Acked Read 1 Sent Done Done false
12040 3891 Acked Write 3 Sent Done Done false
12041 3892 Acked Flush 0 Sent Done Done false
12042 3893 Acked Flush 0 Sent Done Done false
12043 3894 Acked Read 7 Sent Done Done false
12044 3895 Acked Write 4 Sent Done Done false
12045 3896 Acked Write 3 Sent Done Done false
12046 3897 Acked Read 6 Sent Done Done false
12047 3898 Acked Write 1 Sent Done Done false
12048 3899 Acked Write 8 Sent Done Done false
12049 3900 Acked Write 1 Sent Done Done false
12050 3901 Acked Write 2 Sent Done Done false
12051 3902 Acked Write 8 Sent Done Done false
12052 3903 Acked Read 5 Sent Done Done false
12053 3904 Acked Flush 0 Sent Done Done false
12054 3905 Acked Read 2 Sent Done Done false
12055 3906 Acked Flush 0 Sent Done Done false
12056 3907 Acked Write 7 Sent Done Done false
12057 3908 Acked Read 8 Sent Done Done false
12058 3909 Acked Read 5 Sent Done Done false
12059 3910 Acked Read 9 Sent Done Done false
12060 3911 Acked Read 4 Sent Done Done false
12061 3912 Acked Read 2 Sent Done Done false
12062 3913 Acked Read 6 Sent Done Done false
12063 3914 Acked Read 8 Sent Done Done false
12064 3915 Acked Read 5 Sent Done Done false
12065 3916 Acked Read 3 Sent Done Done false
12066 3917 Acked Read 7 Sent Done Done false
12067 3918 Acked Flush 0 Sent Done Done false
12068 3919 Acked Read 5 Sent Done Done false
12069 3920 Acked Write 5 Sent Done Done false
12070 3921 Acked Write 10 Sent Done Done false
12071 3922 Acked Read 3 Sent Done Done false
12072 3923 Acked Read 4 Sent Done Done false
12073 3924 Acked Read 7 Sent Done Done false
12074 3925 Acked Read 4 Sent Done Done false
12075 3926 Acked Read 7 Sent Done Done false
12076 3927 Acked Write 7 Sent Done Done false
12077 3928 Acked Read 4 Sent Done Done false
12078 3929 Acked Write 3 Sent Done Done false
12079 3930 Acked Flush 0 Sent Done Done false
12080 3931 Acked Read 6 Sent Done Done false
12081 3932 Acked Write 2 Sent Done Done false
12082 3933 Acked Read 9 Sent Done Done false
12083 3934 Acked Flush 0 Sent Done Done false
12084 3935 Acked Flush 0 Sent Done Done false
12085 3936 Acked Read 9 Sent Done Done false
12086 3937 Acked Read 7 Sent Done Done false
12087 3938 Acked Write 10 Sent Done Done false
12088 3939 Acked Read 8 Sent Done Done false
12089 3940 Acked Read 10 Sent Done Done false
12090 3941 Acked Read 5 Sent Done Done false
12091 3942 Acked Flush 0 Sent Done Done false
12092 3943 Acked Write 6 Sent Done Done false
12093 3944 Acked Read 8 Sent Done Done false
12094 3945 Acked Read 10 Sent Done Done false
12095 3946 Acked Read 5 Sent Done Done false
12096 3947 Acked Read 5 Sent Done Done false
12097 3948 Acked Flush 0 Sent Done Done false
12098 3949 Acked Read 6 Sent Done Done false
12099 3950 Acked Write 10 Sent Done Done false
12100 3951 Acked Flush 0 Sent Done Done false
12101 3952 Acked Read 2 Sent Done Done false
12102 3953 Acked Write 1 Sent Done Done false
12103 3954 Acked Read 2 Sent Done Done false
12104 3955 Acked Write 3 Sent Done Done false
12105 3956 Acked Read 9 Sent Done Done false
12106 3957 Acked Read 3 Sent Done Done false
12107 3958 Acked Read 5 Sent Done Done false
12108 3959 Acked Read 1 Sent Done Done false
12109 3960 Acked Read 1 Sent Done Done false
12110 3961 Acked Flush 0 Sent Done Done false
12111 3962 Acked Read 10 Sent Done Done false
12112 3963 Acked Read 2 Sent Done Done false
12113 3964 Acked Write 10 Sent Done Done false
12114 3965 Acked Write 4 Sent Done Done false
12115 3966 Acked Flush 0 Sent Done Done false
12116 3967 Acked Write 8 Sent Done Done false
12117 3968 Acked Write 4 Sent Done Done false
12118 3969 Acked Flush 0 Sent Done Done false
12119 3970 Acked Write 6 Sent Done Done false
12120 3971 Acked Read 5 Sent Done Done false
12121 3972 Acked Write 1 Sent Done Done false
12122 3973 Acked Write 6 Sent Done Done false
12123 3974 Acked Write 2 Sent Done Done false
12124 3975 Acked Write 2 Sent Done Done false
12125 3976 Acked Read 8 Sent Done Done false
12126 3977 Acked Read 9 Sent Done Done false
12127 3978 Acked Flush 0 Sent Done Done false
12128 3979 Acked Write 9 Sent Done Done false
12129 3980 Acked Write 5 Sent Done Done false
12130 3981 Acked Write 4 Sent Done Done false
12131 3982 Acked Read 4 Sent Done Done false
12132 3983 Acked Read 10 Sent Done Done false
12133 3984 Acked Write 4 Sent Done Done false
12134 3985 Acked Read 9 Sent Done Done false
12135 3986 Acked Read 6 Sent Done Done false
12136 3987 Acked Write 10 Sent Done Done false
12137 3988 Acked Write 4 Sent Done Done false
12138 3989 Acked Read 9 Sent Done Done false
12139 3990 Acked Read 3 Sent Done Done false
12140 3991 Acked Write 3 Sent Done Done false
12141 3992 Acked Read 9 Sent Done Done false
12142 3993 Acked Read 7 Sent Done Done false
12143 3994 Acked Write 10 Sent Done Done false
12144 3995 Acked Read 1 Sent Done Done false
12145 3996 Acked Write 10 Sent Done Done false
12146 3997 Acked Read 2 Sent Done Done false
12147 3998 Acked Read 3 Sent Done Done false
12148 3999 Acked Read 10 Sent Done Done false
12149 4000 Acked Read 7 Sent Done Done false
12150 4001 Acked Read 10 Sent Done Done false
12151 4002 Acked Read 3 Sent Done Done false
12152 4003 Acked Read 3 Sent Done Done false
12153 4004 Acked Read 1 Sent Done Done false
12154 4005 Acked Write 6 Sent Done Done false
12155 4006 Acked Read 1 Sent Done Done false
12156 4007 Acked Read 10 Sent Done Done false
12157 4008 Acked Read 3 Sent Done Done false
12158 4009 Acked Read 2 Sent Done Done false
12159 4010 Acked Write 8 Sent Done Done false
12160 4011 Acked Flush 0 Sent Done Done false
12161 4012 Acked Read 5 Sent Done Done false
12162 4013 Acked Flush 0 Sent Done Done false
12163 4014 Acked Write 9 Sent Done Done false
12164 4015 Acked Read 5 Sent Done Done false
12165 4016 Acked Write 10 Sent Done Done false
12166 STATES DS:0 DS:1 DS:2 TOTAL
12167 Sent 301 0 0 301
12168 Done 0 301 301 602
12169 Skipped 0 0 0 0
12170 Error 0 0 0 0
12171 Last Flush: 3715 4013 4013
12172 Downstairs last five completed: 3715 3714 3713 3712 3711
12173 Upstairs last five completed: 4016 4015 4014 4013 4012
12174 CLIENT: Up:0 ds:301 act:2
12175 ----------------------------------------------------------------
12176 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
12177 States: Offline Active Active
12178 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
12179 3716 Acked Write 5 Sent Done Done false
12180 3717 Acked Write 5 Sent Done Done false
12181 3718 Acked Read 3 Sent Done Done false
12182 3719 Acked Read 5 Sent Done Done false
12183 3720 Acked Write 7 Sent Done Done false
12184 3721 Acked Flush 0 Sent Done Done false
12185 3722 Acked Write 2 Sent Done Done false
12186 3723 Acked Write 7 Sent Done Done false
12187 3724 Acked Write 8 Sent Done Done false
12188 3725 Acked Read 3 Sent Done Done false
12189 3726 Acked Write 10 Sent Done Done false
12190 3727 Acked Write 8 Sent Done Done false
12191 3728 Acked Write 5 Sent Done Done false
12192 3729 Acked Write 4 Sent Done Done false
12193 3730 Acked Read 7 Sent Done Done false
12194 3731 Acked Write 1 Sent Done Done false
12195 3732 Acked Read 6 Sent Done Done false
12196 3733 Acked Write 6 Sent Done Done false
12197 3734 Acked Write 7 Sent Done Done false
12198 3735 Acked Write 4 Sent Done Done false
12199 3736 Acked Flush 0 Sent Done Done false
12200 3737 Acked Write 10 Sent Done Done false
12201 3738 Acked Read 6 Sent Done Done false
12202 3739 Acked Write 5 Sent Done Done false
12203 3740 Acked Read 10 Sent Done Done false
12204 3741 Acked Read 4 Sent Done Done false
12205 3742 Acked Flush 0 Sent Done Done false
12206 3743 Acked Read 1 Sent Done Done false
12207 3744 Acked Flush 0 Sent Done Done false
12208 3745 Acked Write 1 Sent Done Done false
12209 3746 Acked Read 9 Sent Done Done false
12210 3747 Acked Write 3 Sent Done Done false
12211 3748 Acked Read 7 Sent Done Done false
12212 3749 Acked Write 2 Sent Done Done false
12213 3750 Acked Write 3 Sent Done Done false
12214 3751 Acked Write 1 Sent Done Done false
12215 3752 Acked Write 1 Sent Done Done false
12216 3753 Acked Write 5 Sent Done Done false
12217 3754 Acked Write 7 Sent Done Done false
12218 3755 Acked Read 2 Sent Done Done false
12219 3756 Acked Write 6 Sent Done Done false
12220 3757 Acked Read 8 Sent Done Done false
12221 3758 Acked Read 9 Sent Done Done false
12222 3759 Acked Read 4 Sent Done Done false
12223 3760 Acked Write 4 Sent Done Done false
12224 3761 Acked Flush 0 Sent Done Done false
12225 3762 Acked Write 10 Sent Done Done false
12226 3763 Acked Read 9 Sent Done Done false
12227 3764 Acked Flush 0 Sent Done Done false
12228 3765 Acked Write 5 Sent Done Done false
12229 3766 Acked Read 2 Sent Done Done false
12230 3767 Acked Read 6 Sent Done Done false
12231 3768 Acked Write 7 Sent Done Done false
12232 3769 Acked Write 7 Sent Done Done false
12233 3770 Acked Read 2 Sent Done Done false
12234 3771 Acked Read 4 Sent Done Done false
12235 3772 Acked Read 10 Sent Done Done false
12236 3773 Acked Write 5 Sent Done Done false
12237 3774 Acked Read 9 Sent Done Done false
12238 3775 Acked Write 9 Sent Done Done false
12239 3776 Acked Write 2 Sent Done Done false
12240 3777 Acked Write 10 Sent Done Done false
12241 3778 Acked Write 2 Sent Done Done false
12242 3779 Acked Read 7 Sent Done Done false
12243 3780 Acked Write 4 Sent Done Done false
12244 3781 Acked Read 10 Sent Done Done false
12245 3782 Acked Read 10 Sent Done Done false
12246 3783 Acked Read 6 Sent Done Done false
12247 3784 Acked Write 6 Sent Done Done false
12248 3785 Acked Write 1 Sent Done Done false
12249 3786 Acked Write 8 Sent Done Done false
12250 3787 Acked Read 2 Sent Done Done false
12251 3788 Acked Read 9 Sent Done Done false
12252 3789 Acked Read 3 Sent Done Done false
12253 3790 Acked Read 2 Sent Done Done false
12254 3791 Acked Write 9 Sent Done Done false
12255 3792 Acked Write 9 Sent Done Done false
12256 3793 Acked Read 7 Sent Done Done false
12257 3794 Acked Read 2 Sent Done Done false
12258 3795 Acked Read 3 Sent Done Done false
12259 3796 Acked Read 6 Sent Done Done false
12260 3797 Acked Write 9 Sent Done Done false
12261 3798 Acked Flush 0 Sent Done Done false
12262 3799 Acked Read 7 Sent Done Done false
12263 3800 Acked Write 1 Sent Done Done false
12264 3801 Acked Read 9 Sent Done Done false
12265 3802 Acked Read 7 Sent Done Done false
12266 3803 Acked Flush 0 Sent Done Done false
12267 3804 Acked Read 4 Sent Done Done false
12268 3805 Acked Read 7 Sent Done Done false
12269 3806 Acked Read 7 Sent Done Done false
12270 3807 Acked Write 3 Sent Done Done false
12271 3808 Acked Write 8 Sent Done Done false
12272 3809 Acked Write 5 Sent Done Done false
12273 3810 Acked Read 9 Sent Done Done false
12274 3811 Acked Write 3 Sent Done Done false
12275 3812 Acked Write 7 Sent Done Done false
12276 3813 Acked Read 9 Sent Done Done false
12277 3814 Acked Read 3 Sent Done Done false
12278 3815 Acked Flush 0 Sent Done Done false
12279 3816 Acked Write 6 Sent Done Done false
12280 3817 Acked Read 9 Sent Done Done false
12281 3818 Acked Read 3 Sent Done Done false
12282 3819 Acked Read 9 Sent Done Done false
12283 3820 Acked Read 10 Sent Done Done false
12284 3821 Acked Write 3 Sent Done Done false
12285 3822 Acked Write 5 Sent Done Done false
12286 3823 Acked Write 9 Sent Done Done false
12287 3824 Acked Write 7 Sent Done Done false
12288 3825 Acked Read 2 Sent Done Done false
12289 3826 Acked Read 2 Sent Done Done false
12290 3827 Acked Write 7 Sent Done Done false
12291 3828 Acked Read 9 Sent Done Done false
12292 3829 Acked Flush 0 Sent Done Done false
12293 3830 Acked Read 2 Sent Done Done false
12294 3831 Acked Write 9 Sent Done Done false
12295 3832 Acked Write 6 Sent Done Done false
12296 3833 Acked Read 8 Sent Done Done false
12297 3834 Acked Write 6 Sent Done Done false
12298 3835 Acked Read 10 Sent Done Done false
12299 3836 Acked Read 5 Sent Done Done false
12300 3837 Acked Write 7 Sent Done Done false
12301 3838 Acked Write 10 Sent Done Done false
12302 3839 Acked Read 7 Sent Done Done false
12303 3840 Acked Read 5 Sent Done Done false
12304 3841 Acked Read 1 Sent Done Done false
12305 3842 Acked Write 1 Sent Done Done false
12306 3843 Acked Flush 0 Sent Done Done false
12307 3844 Acked Write 4 Sent Done Done false
12308 3845 Acked Flush 0 Sent Done Done false
12309 3846 Acked Write 1 Sent Done Done false
12310 3847 Acked Write 2 Sent Done Done false
12311 3848 Acked Flush 0 Sent Done Done false
12312 3849 Acked Read 5 Sent Done Done false
12313 3850 Acked Write 6 Sent Done Done false
12314 3851 Acked Read 3 Sent Done Done false
12315 3852 Acked Read 2 Sent Done Done false
12316 3853 Acked Read 1 Sent Done Done false
12317 3854 Acked Write 6 Sent Done Done false
12318 3855 Acked Write 8 Sent Done Done false
12319 3856 Acked Read 8 Sent Done Done false
12320 3857 Acked Write 8 Sent Done Done false
12321 3858 Acked Read 6 Sent Done Done false
12322 3859 Acked Read 7 Sent Done Done false
12323 3860 Acked Read 6 Sent Done Done false
12324 3861 Acked Write 6 Sent Done Done false
12325 3862 Acked Read 4 Sent Done Done false
12326 3863 Acked Read 9 Sent Done Done false
12327 3864 Acked Write 4 Sent Done Done false
12328 3865 Acked Read 9 Sent Done Done false
12329 3866 Acked Read 6 Sent Done Done false
12330 3867 Acked Write 8 Sent Done Done false
12331 3868 Acked Read 7 Sent Done Done false
12332 3869 Acked Read 2 Sent Done Done false
12333 3870 Acked Write 4 Sent Done Done false
12334 3871 Acked Read 9 Sent Done Done false
12335 3872 Acked Read 10 Sent Done Done false
12336 3873 Acked Write 5 Sent Done Done false
12337 3874 Acked Read 1 Sent Done Done false
12338 3875 Acked Write 2 Sent Done Done false
12339 3876 Acked Read 6 Sent Done Done false
12340 3877 Acked Read 1 Sent Done Done false
12341 3878 Acked Read 10 Sent Done Done false
12342 3879 Acked Read 3 Sent Done Done false
12343 3880 Acked Flush 0 Sent Done Done false
12344 3881 Acked Write 10 Sent Done Done false
12345 3882 Acked Write 10 Sent Done Done false
12346 3883 Acked Write 5 Sent Done Done false
12347 3884 Acked Flush 0 Sent Done Done false
12348 3885 Acked Flush 0 Sent Done Done false
12349 3886 Acked Flush 0 Sent Done Done false
12350 3887 Acked Read 6 Sent Done Done false
12351 3888 Acked Read 7 Sent Done Done false
12352 3889 Acked Read 9 Sent Done Done false
12353 3890 Acked Read 1 Sent Done Done false
12354 3891 Acked Write 3 Sent Done Done false
12355 3892 Acked Flush 0 Sent Done Done false
12356 3893 Acked Flush 0 Sent Done Done false
12357 3894 Acked Read 7 Sent Done Done false
12358 3895 Acked Write 4 Sent Done Done false
12359 3896 Acked Write 3 Sent Done Done false
12360 3897 Acked Read 6 Sent Done Done false
12361 3898 Acked Write 1 Sent Done Done false
12362 3899 Acked Write 8 Sent Done Done false
12363 3900 Acked Write 1 Sent Done Done false
12364 3901 Acked Write 2 Sent Done Done false
12365 3902 Acked Write 8 Sent Done Done false
12366 3903 Acked Read 5 Sent Done Done false
12367 3904 Acked Flush 0 Sent Done Done false
12368 3905 Acked Read 2 Sent Done Done false
12369 3906 Acked Flush 0 Sent Done Done false
12370 3907 Acked Write 7 Sent Done Done false
12371 3908 Acked Read 8 Sent Done Done false
12372 3909 Acked Read 5 Sent Done Done false
12373 3910 Acked Read 9 Sent Done Done false
12374 3911 Acked Read 4 Sent Done Done false
12375 3912 Acked Read 2 Sent Done Done false
12376 3913 Acked Read 6 Sent Done Done false
12377 3914 Acked Read 8 Sent Done Done false
12378 3915 Acked Read 5 Sent Done Done false
12379 3916 Acked Read 3 Sent Done Done false
12380 3917 Acked Read 7 Sent Done Done false
12381 3918 Acked Flush 0 Sent Done Done false
12382 3919 Acked Read 5 Sent Done Done false
12383 3920 Acked Write 5 Sent Done Done false
12384 3921 Acked Write 10 Sent Done Done false
12385 3922 Acked Read 3 Sent Done Done false
12386 3923 Acked Read 4 Sent Done Done false
12387 3924 Acked Read 7 Sent Done Done false
12388 3925 Acked Read 4 Sent Done Done false
12389 3926 Acked Read 7 Sent Done Done false
12390 3927 Acked Write 7 Sent Done Done false
12391 3928 Acked Read 4 Sent Done Done false
12392 3929 Acked Write 3 Sent Done Done false
12393 3930 Acked Flush 0 Sent Done Done false
12394 3931 Acked Read 6 Sent Done Done false
12395 3932 Acked Write 2 Sent Done Done false
12396 3933 Acked Read 9 Sent Done Done false
12397 3934 Acked Flush 0 Sent Done Done false
12398 3935 Acked Flush 0 Sent Done Done false
12399 3936 Acked Read 9 Sent Done Done false
12400 3937 Acked Read 7 Sent Done Done false
12401 3938 Acked Write 10 Sent Done Done false
12402 3939 Acked Read 8 Sent Done Done false
12403 3940 Acked Read 10 Sent Done Done false
12404 3941 Acked Read 5 Sent Done Done false
12405 3942 Acked Flush 0 Sent Done Done false
12406 3943 Acked Write 6 Sent Done Done false
12407 3944 Acked Read 8 Sent Done Done false
12408 3945 Acked Read 10 Sent Done Done false
12409 3946 Acked Read 5 Sent Done Done false
12410 3947 Acked Read 5 Sent Done Done false
12411 3948 Acked Flush 0 Sent Done Done false
12412 3949 Acked Read 6 Sent Done Done false
12413 3950 Acked Write 10 Sent Done Done false
12414 3951 Acked Flush 0 Sent Done Done false
12415 3952 Acked Read 2 Sent Done Done false
12416 3953 Acked Write 1 Sent Done Done false
12417 3954 Acked Read 2 Sent Done Done false
12418 3955 Acked Write 3 Sent Done Done false
12419 3956 Acked Read 9 Sent Done Done false
12420 3957 Acked Read 3 Sent Done Done false
12421 3958 Acked Read 5 Sent Done Done false
12422 3959 Acked Read 1 Sent Done Done false
12423 3960 Acked Read 1 Sent Done Done false
12424 3961 Acked Flush 0 Sent Done Done false
12425 3962 Acked Read 10 Sent Done Done false
12426 3963 Acked Read 2 Sent Done Done false
12427 3964 Acked Write 10 Sent Done Done false
12428 3965 Acked Write 4 Sent Done Done false
12429 3966 Acked Flush 0 Sent Done Done false
12430 3967 Acked Write 8 Sent Done Done false
12431 3968 Acked Write 4 Sent Done Done false
12432 3969 Acked Flush 0 Sent Done Done false
12433 3970 Acked Write 6 Sent Done Done false
12434 3971 Acked Read 5 Sent Done Done false
12435 3972 Acked Write 1 Sent Done Done false
12436 3973 Acked Write 6 Sent Done Done false
12437 3974 Acked Write 2 Sent Done Done false
12438 3975 Acked Write 2 Sent Done Done false
12439 3976 Acked Read 8 Sent Done Done false
12440 3977 Acked Read 9 Sent Done Done false
12441 3978 Acked Flush 0 Sent Done Done false
12442 3979 Acked Write 9 Sent Done Done false
12443 3980 Acked Write 5 Sent Done Done false
12444 3981 Acked Write 4 Sent Done Done false
12445 3982 Acked Read 4 Sent Done Done false
12446 3983 Acked Read 10 Sent Done Done false
12447 3984 Acked Write 4 Sent Done Done false
12448 3985 Acked Read 9 Sent Done Done false
12449 3986 Acked Read 6 Sent Done Done false
12450 3987 Acked Write 10 Sent Done Done false
12451 3988 Acked Write 4 Sent Done Done false
12452 3989 Acked Read 9 Sent Done Done false
12453 3990 Acked Read 3 Sent Done Done false
12454 3991 Acked Write 3 Sent Done Done false
12455 3992 Acked Read 9 Sent Done Done false
12456 3993 Acked Read 7 Sent Done Done false
12457 3994 Acked Write 10 Sent Done Done false
12458 3995 Acked Read 1 Sent Done Done false
12459 3996 Acked Write 10 Sent Done Done false
12460 3997 Acked Read 2 Sent Done Done false
12461 3998 Acked Read 3 Sent Done Done false
12462 3999 Acked Read 10 Sent Done Done false
12463 4000 Acked Read 7 Sent Done Done false
12464 4001 Acked Read 10 Sent Done Done false
12465 4002 Acked Read 3 Sent Done Done false
12466 4003 Acked Read 3 Sent Done Done false
12467 4004 Acked Read 1 Sent Done Done false
12468 4005 Acked Write 6 Sent Done Done false
12469 4006 Acked Read 1 Sent Done Done false
12470 4007 Acked Read 10 Sent Done Done false
12471 4008 Acked Read 3 Sent Done Done false
12472 4009 Acked Read 2 Sent Done Done false
12473 4010 Acked Write 8 Sent Done Done false
12474 4011 Acked Flush 0 Sent Done Done false
12475 4012 Acked Read 5 Sent Done Done false
12476 4013 Acked Flush 0 Sent Done Done false
12477 4014 Acked Write 9 Sent Done Done false
12478 4015 Acked Read 5 Sent Done Done false
12479 4016 Acked Write 10 Sent Done Done false
12480 4017 Acked Flush 0 Sent Done Done false
12481 STATES DS:0 DS:1 DS:2 TOTAL
12482 Sent 302 0 0 302
12483 Done 0 302 302 604
12484 Skipped 0 0 0 0
12485 Error 0 0 0 0
12486 Last Flush: 3715 4017 4017
12487 Downstairs last five completed: 3715 3714 3713 3712 3711
12488 Upstairs last five completed: 4017 4016 4015 4014 4013
12489 CLIENT: Up:0 ds:302 act:2
124902024-11-15T02:00:04.094ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
124912024-11-15T02:00:04.094ZWARNcrucible: ds_connection connect to 127.0.0.1:8810 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
124922024-11-15T02:00:04.094ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
124932024-11-15T02:00:04.094ZWARNcrucible: downstairs task for 0 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
124942024-11-15T02:00:04.094ZINFOcrucible: client task is exiting = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
12495 [8810] got start action
12496 [8810] I am going to restart
12497 Starting downstairs at port 8810
12498 Make output file at "/tmp/dsc/downstairs-8810.txt"
12499 [8810][0] reports Starting
12500 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1215)
12501 [8810][0] reports Running
12502 ----------------------------------------------------------------
12503 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
12504 States: Offline Active Active
12505 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
12506 3716 Acked Write 5 Sent Done Done false
12507 3717 Acked Write 5 Sent Done Done false
12508 3718 Acked Read 3 Sent Done Done false
12509 3719 Acked Read 5 Sent Done Done false
12510 3720 Acked Write 7 Sent Done Done false
12511 3721 Acked Flush 0 Sent Done Done false
12512 3722 Acked Write 2 Sent Done Done false
12513 3723 Acked Write 7 Sent Done Done false
12514 3724 Acked Write 8 Sent Done Done false
12515 3725 Acked Read 3 Sent Done Done false
12516 3726 Acked Write 10 Sent Done Done false
12517 3727 Acked Write 8 Sent Done Done false
12518 3728 Acked Write 5 Sent Done Done false
12519 3729 Acked Write 4 Sent Done Done false
12520 3730 Acked Read 7 Sent Done Done false
12521 3731 Acked Write 1 Sent Done Done false
12522 3732 Acked Read 6 Sent Done Done false
12523 3733 Acked Write 6 Sent Done Done false
12524 3734 Acked Write 7 Sent Done Done false
12525 3735 Acked Write 4 Sent Done Done false
12526 3736 Acked Flush 0 Sent Done Done false
12527 3737 Acked Write 10 Sent Done Done false
12528 3738 Acked Read 6 Sent Done Done false
12529 3739 Acked Write 5 Sent Done Done false
12530 3740 Acked Read 10 Sent Done Done false
12531 3741 Acked Read 4 Sent Done Done false
12532 3742 Acked Flush 0 Sent Done Done false
12533 3743 Acked Read 1 Sent Done Done false
12534 3744 Acked Flush 0 Sent Done Done false
12535 3745 Acked Write 1 Sent Done Done false
12536 3746 Acked Read 9 Sent Done Done false
12537 3747 Acked Write 3 Sent Done Done false
12538 3748 Acked Read 7 Sent Done Done false
12539 3749 Acked Write 2 Sent Done Done false
12540 3750 Acked Write 3 Sent Done Done false
12541 3751 Acked Write 1 Sent Done Done false
12542 3752 Acked Write 1 Sent Done Done false
12543 3753 Acked Write 5 Sent Done Done false
12544 3754 Acked Write 7 Sent Done Done false
12545 3755 Acked Read 2 Sent Done Done false
12546 3756 Acked Write 6 Sent Done Done false
12547 3757 Acked Read 8 Sent Done Done false
12548 3758 Acked Read 9 Sent Done Done false
12549 3759 Acked Read 4 Sent Done Done false
12550 3760 Acked Write 4 Sent Done Done false
12551 3761 Acked Flush 0 Sent Done Done false
12552 3762 Acked Write 10 Sent Done Done false
12553 3763 Acked Read 9 Sent Done Done false
12554 3764 Acked Flush 0 Sent Done Done false
12555 3765 Acked Write 5 Sent Done Done false
12556 3766 Acked Read 2 Sent Done Done false
12557 3767 Acked Read 6 Sent Done Done false
12558 3768 Acked Write 7 Sent Done Done false
12559 3769 Acked Write 7 Sent Done Done false
12560 3770 Acked Read 2 Sent Done Done false
12561 3771 Acked Read 4 Sent Done Done false
12562 3772 Acked Read 10 Sent Done Done false
12563 3773 Acked Write 5 Sent Done Done false
12564 3774 Acked Read 9 Sent Done Done false
12565 3775 Acked Write 9 Sent Done Done false
12566 3776 Acked Write 2 Sent Done Done false
12567 3777 Acked Write 10 Sent Done Done false
12568 3778 Acked Write 2 Sent Done Done false
12569 3779 Acked Read 7 Sent Done Done false
12570 3780 Acked Write 4 Sent Done Done false
12571 3781 Acked Read 10 Sent Done Done false
12572 3782 Acked Read 10 Sent Done Done false
12573 3783 Acked Read 6 Sent Done Done false
12574 3784 Acked Write 6 Sent Done Done false
12575 3785 Acked Write 1 Sent Done Done false
12576 3786 Acked Write 8 Sent Done Done false
12577 3787 Acked Read 2 Sent Done Done false
12578 3788 Acked Read 9 Sent Done Done false
12579 3789 Acked Read 3 Sent Done Done false
12580 3790 Acked Read 2 Sent Done Done false
12581 3791 Acked Write 9 Sent Done Done false
12582 3792 Acked Write 9 Sent Done Done false
12583 3793 Acked Read 7 Sent Done Done false
12584 3794 Acked Read 2 Sent Done Done false
12585 3795 Acked Read 3 Sent Done Done false
12586 3796 Acked Read 6 Sent Done Done false
12587 3797 Acked Write 9 Sent Done Done false
12588 3798 Acked Flush 0 Sent Done Done false
12589 3799 Acked Read 7 Sent Done Done false
12590 3800 Acked Write 1 Sent Done Done false
12591 3801 Acked Read 9 Sent Done Done false
12592 3802 Acked Read 7 Sent Done Done false
12593 3803 Acked Flush 0 Sent Done Done false
12594 3804 Acked Read 4 Sent Done Done false
12595 3805 Acked Read 7 Sent Done Done false
12596 3806 Acked Read 7 Sent Done Done false
12597 3807 Acked Write 3 Sent Done Done false
12598 3808 Acked Write 8 Sent Done Done false
12599 3809 Acked Write 5 Sent Done Done false
12600 3810 Acked Read 9 Sent Done Done false
12601 3811 Acked Write 3 Sent Done Done false
12602 3812 Acked Write 7 Sent Done Done false
12603 3813 Acked Read 9 Sent Done Done false
12604 3814 Acked Read 3 Sent Done Done false
12605 3815 Acked Flush 0 Sent Done Done false
12606 3816 Acked Write 6 Sent Done Done false
12607 3817 Acked Read 9 Sent Done Done false
12608 3818 Acked Read 3 Sent Done Done false
12609 3819 Acked Read 9 Sent Done Done false
12610 3820 Acked Read 10 Sent Done Done false
12611 3821 Acked Write 3 Sent Done Done false
12612 3822 Acked Write 5 Sent Done Done false
12613 3823 Acked Write 9 Sent Done Done false
12614 3824 Acked Write 7 Sent Done Done false
12615 3825 Acked Read 2 Sent Done Done false
12616 3826 Acked Read 2 Sent Done Done false
12617 3827 Acked Write 7 Sent Done Done false
12618 3828 Acked Read 9 Sent Done Done false
12619 3829 Acked Flush 0 Sent Done Done false
12620 3830 Acked Read 2 Sent Done Done false
12621 3831 Acked Write 9 Sent Done Done false
12622 3832 Acked Write 6 Sent Done Done false
12623 3833 Acked Read 8 Sent Done Done false
12624 3834 Acked Write 6 Sent Done Done false
12625 3835 Acked Read 10 Sent Done Done false
12626 3836 Acked Read 5 Sent Done Done false
12627 3837 Acked Write 7 Sent Done Done false
12628 3838 Acked Write 10 Sent Done Done false
12629 3839 Acked Read 7 Sent Done Done false
12630 3840 Acked Read 5 Sent Done Done false
12631 3841 Acked Read 1 Sent Done Done false
12632 3842 Acked Write 1 Sent Done Done false
12633 3843 Acked Flush 0 Sent Done Done false
12634 3844 Acked Write 4 Sent Done Done false
12635 3845 Acked Flush 0 Sent Done Done false
12636 3846 Acked Write 1 Sent Done Done false
12637 3847 Acked Write 2 Sent Done Done false
12638 3848 Acked Flush 0 Sent Done Done false
12639 3849 Acked Read 5 Sent Done Done false
12640 3850 Acked Write 6 Sent Done Done false
12641 3851 Acked Read 3 Sent Done Done false
12642 3852 Acked Read 2 Sent Done Done false
12643 3853 Acked Read 1 Sent Done Done false
12644 3854 Acked Write 6 Sent Done Done false
12645 3855 Acked Write 8 Sent Done Done false
12646 3856 Acked Read 8 Sent Done Done false
12647 3857 Acked Write 8 Sent Done Done false
12648 3858 Acked Read 6 Sent Done Done false
12649 3859 Acked Read 7 Sent Done Done false
12650 3860 Acked Read 6 Sent Done Done false
12651 3861 Acked Write 6 Sent Done Done false
12652 3862 Acked Read 4 Sent Done Done false
12653 3863 Acked Read 9 Sent Done Done false
12654 3864 Acked Write 4 Sent Done Done false
12655 3865 Acked Read 9 Sent Done Done false
12656 3866 Acked Read 6 Sent Done Done false
12657 3867 Acked Write 8 Sent Done Done false
12658 3868 Acked Read 7 Sent Done Done false
12659 3869 Acked Read 2 Sent Done Done false
12660 3870 Acked Write 4 Sent Done Done false
12661 3871 Acked Read 9 Sent Done Done false
12662 3872 Acked Read 10 Sent Done Done false
12663 3873 Acked Write 5 Sent Done Done false
12664 3874 Acked Read 1 Sent Done Done false
12665 3875 Acked Write 2 Sent Done Done false
12666 3876 Acked Read 6 Sent Done Done false
12667 3877 Acked Read 1 Sent Done Done false
12668 3878 Acked Read 10 Sent Done Done false
12669 3879 Acked Read 3 Sent Done Done false
12670 3880 Acked Flush 0 Sent Done Done false
12671 3881 Acked Write 10 Sent Done Done false
12672 3882 Acked Write 10 Sent Done Done false
12673 3883 Acked Write 5 Sent Done Done false
12674 3884 Acked Flush 0 Sent Done Done false
12675 3885 Acked Flush 0 Sent Done Done false
12676 3886 Acked Flush 0 Sent Done Done false
12677 3887 Acked Read 6 Sent Done Done false
12678 3888 Acked Read 7 Sent Done Done false
12679 3889 Acked Read 9 Sent Done Done false
12680 3890 Acked Read 1 Sent Done Done false
12681 3891 Acked Write 3 Sent Done Done false
12682 3892 Acked Flush 0 Sent Done Done false
12683 3893 Acked Flush 0 Sent Done Done false
12684 3894 Acked Read 7 Sent Done Done false
12685 3895 Acked Write 4 Sent Done Done false
12686 3896 Acked Write 3 Sent Done Done false
12687 3897 Acked Read 6 Sent Done Done false
12688 3898 Acked Write 1 Sent Done Done false
12689 3899 Acked Write 8 Sent Done Done false
12690 3900 Acked Write 1 Sent Done Done false
12691 3901 Acked Write 2 Sent Done Done false
12692 3902 Acked Write 8 Sent Done Done false
12693 3903 Acked Read 5 Sent Done Done false
12694 3904 Acked Flush 0 Sent Done Done false
12695 3905 Acked Read 2 Sent Done Done false
12696 3906 Acked Flush 0 Sent Done Done false
12697 3907 Acked Write 7 Sent Done Done false
12698 3908 Acked Read 8 Sent Done Done false
12699 3909 Acked Read 5 Sent Done Done false
12700 3910 Acked Read 9 Sent Done Done false
12701 3911 Acked Read 4 Sent Done Done false
12702 3912 Acked Read 2 Sent Done Done false
12703 3913 Acked Read 6 Sent Done Done false
12704 3914 Acked Read 8 Sent Done Done false
12705 3915 Acked Read 5 Sent Done Done false
12706 3916 Acked Read 3 Sent Done Done false
12707 3917 Acked Read 7 Sent Done Done false
12708 3918 Acked Flush 0 Sent Done Done false
12709 3919 Acked Read 5 Sent Done Done false
12710 3920 Acked Write 5 Sent Done Done false
12711 3921 Acked Write 10 Sent Done Done false
12712 3922 Acked Read 3 Sent Done Done false
12713 3923 Acked Read 4 Sent Done Done false
12714 3924 Acked Read 7 Sent Done Done false
12715 3925 Acked Read 4 Sent Done Done false
12716 3926 Acked Read 7 Sent Done Done false
12717 3927 Acked Write 7 Sent Done Done false
12718 3928 Acked Read 4 Sent Done Done false
12719 3929 Acked Write 3 Sent Done Done false
12720 3930 Acked Flush 0 Sent Done Done false
12721 3931 Acked Read 6 Sent Done Done false
12722 3932 Acked Write 2 Sent Done Done false
12723 3933 Acked Read 9 Sent Done Done false
12724 3934 Acked Flush 0 Sent Done Done false
12725 3935 Acked Flush 0 Sent Done Done false
12726 3936 Acked Read 9 Sent Done Done false
12727 3937 Acked Read 7 Sent Done Done false
12728 3938 Acked Write 10 Sent Done Done false
12729 3939 Acked Read 8 Sent Done Done false
12730 3940 Acked Read 10 Sent Done Done false
12731 3941 Acked Read 5 Sent Done Done false
12732 3942 Acked Flush 0 Sent Done Done false
12733 3943 Acked Write 6 Sent Done Done false
12734 3944 Acked Read 8 Sent Done Done false
12735 3945 Acked Read 10 Sent Done Done false
12736 3946 Acked Read 5 Sent Done Done false
12737 3947 Acked Read 5 Sent Done Done false
12738 3948 Acked Flush 0 Sent Done Done false
12739 3949 Acked Read 6 Sent Done Done false
12740 3950 Acked Write 10 Sent Done Done false
12741 3951 Acked Flush 0 Sent Done Done false
12742 3952 Acked Read 2 Sent Done Done false
12743 3953 Acked Write 1 Sent Done Done false
12744 3954 Acked Read 2 Sent Done Done false
12745 3955 Acked Write 3 Sent Done Done false
12746 3956 Acked Read 9 Sent Done Done false
12747 3957 Acked Read 3 Sent Done Done false
12748 3958 Acked Read 5 Sent Done Done false
12749 3959 Acked Read 1 Sent Done Done false
12750 3960 Acked Read 1 Sent Done Done false
12751 3961 Acked Flush 0 Sent Done Done false
12752 3962 Acked Read 10 Sent Done Done false
12753 3963 Acked Read 2 Sent Done Done false
12754 3964 Acked Write 10 Sent Done Done false
12755 3965 Acked Write 4 Sent Done Done false
12756 3966 Acked Flush 0 Sent Done Done false
12757 3967 Acked Write 8 Sent Done Done false
12758 3968 Acked Write 4 Sent Done Done false
12759 3969 Acked Flush 0 Sent Done Done false
12760 3970 Acked Write 6 Sent Done Done false
12761 3971 Acked Read 5 Sent Done Done false
12762 3972 Acked Write 1 Sent Done Done false
12763 3973 Acked Write 6 Sent Done Done false
12764 3974 Acked Write 2 Sent Done Done false
12765 3975 Acked Write 2 Sent Done Done false
12766 3976 Acked Read 8 Sent Done Done false
12767 3977 Acked Read 9 Sent Done Done false
12768 3978 Acked Flush 0 Sent Done Done false
12769 3979 Acked Write 9 Sent Done Done false
12770 3980 Acked Write 5 Sent Done Done false
12771 3981 Acked Write 4 Sent Done Done false
12772 3982 Acked Read 4 Sent Done Done false
12773 3983 Acked Read 10 Sent Done Done false
12774 3984 Acked Write 4 Sent Done Done false
12775 3985 Acked Read 9 Sent Done Done false
12776 3986 Acked Read 6 Sent Done Done false
12777 3987 Acked Write 10 Sent Done Done false
12778 3988 Acked Write 4 Sent Done Done false
12779 3989 Acked Read 9 Sent Done Done false
12780 3990 Acked Read 3 Sent Done Done false
12781 3991 Acked Write 3 Sent Done Done false
12782 3992 Acked Read 9 Sent Done Done false
12783 3993 Acked Read 7 Sent Done Done false
12784 3994 Acked Write 10 Sent Done Done false
12785 3995 Acked Read 1 Sent Done Done false
12786 3996 Acked Write 10 Sent Done Done false
12787 3997 Acked Read 2 Sent Done Done false
12788 3998 Acked Read 3 Sent Done Done false
12789 3999 Acked Read 10 Sent Done Done false
12790 4000 Acked Read 7 Sent Done Done false
12791 4001 Acked Read 10 Sent Done Done false
12792 4002 Acked Read 3 Sent Done Done false
12793 4003 Acked Read 3 Sent Done Done false
12794 4004 Acked Read 1 Sent Done Done false
12795 4005 Acked Write 6 Sent Done Done false
12796 4006 Acked Read 1 Sent Done Done false
12797 4007 Acked Read 10 Sent Done Done false
12798 4008 Acked Read 3 Sent Done Done false
12799 4009 Acked Read 2 Sent Done Done false
12800 4010 Acked Write 8 Sent Done Done false
12801 4011 Acked Flush 0 Sent Done Done false
12802 4012 Acked Read 5 Sent Done Done false
12803 4013 Acked Flush 0 Sent Done Done false
12804 4014 Acked Write 9 Sent Done Done false
12805 4015 Acked Read 5 Sent Done Done false
12806 4016 Acked Write 10 Sent Done Done false
12807 4017 Acked Flush 0 Sent Done Done false
12808 STATES DS:0 DS:1 DS:2 TOTAL
12809 Sent 302 0 0 302
12810 Done 0 302 302 604
12811 Skipped 0 0 0 0
12812 Error 0 0 0 0
12813 Last Flush: 3715 4017 4017
12814 Downstairs last five completed: 3715 3714 3713 3712 3711
12815 Upstairs last five completed: 4017 4016 4015 4014 4013
12816 CLIENT: Up:0 ds:302 act:2
12817 ----------------------------------------------------------------
12818 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
12819 States: Offline Active Active
12820 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
12821 3716 Acked Write 5 Sent Done Done false
12822 3717 Acked Write 5 Sent Done Done false
12823 3718 Acked Read 3 Sent Done Done false
12824 3719 Acked Read 5 Sent Done Done false
12825 3720 Acked Write 7 Sent Done Done false
12826 3721 Acked Flush 0 Sent Done Done false
12827 3722 Acked Write 2 Sent Done Done false
12828 3723 Acked Write 7 Sent Done Done false
12829 3724 Acked Write 8 Sent Done Done false
12830 3725 Acked Read 3 Sent Done Done false
12831 3726 Acked Write 10 Sent Done Done false
12832 3727 Acked Write 8 Sent Done Done false
12833 3728 Acked Write 5 Sent Done Done false
12834 3729 Acked Write 4 Sent Done Done false
12835 3730 Acked Read 7 Sent Done Done false
12836 3731 Acked Write 1 Sent Done Done false
12837 3732 Acked Read 6 Sent Done Done false
12838 3733 Acked Write 6 Sent Done Done false
12839 3734 Acked Write 7 Sent Done Done false
12840 3735 Acked Write 4 Sent Done Done false
12841 3736 Acked Flush 0 Sent Done Done false
12842 3737 Acked Write 10 Sent Done Done false
12843 3738 Acked Read 6 Sent Done Done false
12844 3739 Acked Write 5 Sent Done Done false
12845 3740 Acked Read 10 Sent Done Done false
12846 3741 Acked Read 4 Sent Done Done false
12847 3742 Acked Flush 0 Sent Done Done false
12848 3743 Acked Read 1 Sent Done Done false
12849 3744 Acked Flush 0 Sent Done Done false
12850 3745 Acked Write 1 Sent Done Done false
12851 3746 Acked Read 9 Sent Done Done false
12852 3747 Acked Write 3 Sent Done Done false
12853 3748 Acked Read 7 Sent Done Done false
12854 3749 Acked Write 2 Sent Done Done false
12855 3750 Acked Write 3 Sent Done Done false
12856 3751 Acked Write 1 Sent Done Done false
12857 3752 Acked Write 1 Sent Done Done false
12858 3753 Acked Write 5 Sent Done Done false
12859 3754 Acked Write 7 Sent Done Done false
12860 3755 Acked Read 2 Sent Done Done false
12861 3756 Acked Write 6 Sent Done Done false
12862 3757 Acked Read 8 Sent Done Done false
12863 3758 Acked Read 9 Sent Done Done false
12864 3759 Acked Read 4 Sent Done Done false
12865 3760 Acked Write 4 Sent Done Done false
12866 3761 Acked Flush 0 Sent Done Done false
12867 3762 Acked Write 10 Sent Done Done false
12868 3763 Acked Read 9 Sent Done Done false
12869 3764 Acked Flush 0 Sent Done Done false
12870 3765 Acked Write 5 Sent Done Done false
12871 3766 Acked Read 2 Sent Done Done false
12872 3767 Acked Read 6 Sent Done Done false
12873 3768 Acked Write 7 Sent Done Done false
12874 3769 Acked Write 7 Sent Done Done false
12875 3770 Acked Read 2 Sent Done Done false
12876 3771 Acked Read 4 Sent Done Done false
12877 3772 Acked Read 10 Sent Done Done false
12878 3773 Acked Write 5 Sent Done Done false
12879 3774 Acked Read 9 Sent Done Done false
12880 3775 Acked Write 9 Sent Done Done false
12881 3776 Acked Write 2 Sent Done Done false
12882 3777 Acked Write 10 Sent Done Done false
12883 3778 Acked Write 2 Sent Done Done false
12884 3779 Acked Read 7 Sent Done Done false
12885 3780 Acked Write 4 Sent Done Done false
12886 3781 Acked Read 10 Sent Done Done false
12887 3782 Acked Read 10 Sent Done Done false
12888 3783 Acked Read 6 Sent Done Done false
12889 3784 Acked Write 6 Sent Done Done false
12890 3785 Acked Write 1 Sent Done Done false
12891 3786 Acked Write 8 Sent Done Done false
12892 3787 Acked Read 2 Sent Done Done false
12893 3788 Acked Read 9 Sent Done Done false
12894 3789 Acked Read 3 Sent Done Done false
12895 3790 Acked Read 2 Sent Done Done false
12896 3791 Acked Write 9 Sent Done Done false
12897 3792 Acked Write 9 Sent Done Done false
12898 3793 Acked Read 7 Sent Done Done false
12899 3794 Acked Read 2 Sent Done Done false
12900 3795 Acked Read 3 Sent Done Done false
12901 3796 Acked Read 6 Sent Done Done false
12902 3797 Acked Write 9 Sent Done Done false
12903 3798 Acked Flush 0 Sent Done Done false
12904 3799 Acked Read 7 Sent Done Done false
12905 3800 Acked Write 1 Sent Done Done false
12906 3801 Acked Read 9 Sent Done Done false
12907 3802 Acked Read 7 Sent Done Done false
12908 3803 Acked Flush 0 Sent Done Done false
12909 3804 Acked Read 4 Sent Done Done false
12910 3805 Acked Read 7 Sent Done Done false
12911 3806 Acked Read 7 Sent Done Done false
12912 3807 Acked Write 3 Sent Done Done false
12913 3808 Acked Write 8 Sent Done Done false
12914 3809 Acked Write 5 Sent Done Done false
12915 3810 Acked Read 9 Sent Done Done false
12916 3811 Acked Write 3 Sent Done Done false
12917 3812 Acked Write 7 Sent Done Done false
12918 3813 Acked Read 9 Sent Done Done false
12919 3814 Acked Read 3 Sent Done Done false
12920 3815 Acked Flush 0 Sent Done Done false
12921 3816 Acked Write 6 Sent Done Done false
12922 3817 Acked Read 9 Sent Done Done false
12923 3818 Acked Read 3 Sent Done Done false
12924 3819 Acked Read 9 Sent Done Done false
12925 3820 Acked Read 10 Sent Done Done false
12926 3821 Acked Write 3 Sent Done Done false
12927 3822 Acked Write 5 Sent Done Done false
12928 3823 Acked Write 9 Sent Done Done false
12929 3824 Acked Write 7 Sent Done Done false
12930 3825 Acked Read 2 Sent Done Done false
12931 3826 Acked Read 2 Sent Done Done false
12932 3827 Acked Write 7 Sent Done Done false
12933 3828 Acked Read 9 Sent Done Done false
12934 3829 Acked Flush 0 Sent Done Done false
12935 3830 Acked Read 2 Sent Done Done false
12936 3831 Acked Write 9 Sent Done Done false
12937 3832 Acked Write 6 Sent Done Done false
12938 3833 Acked Read 8 Sent Done Done false
12939 3834 Acked Write 6 Sent Done Done false
12940 3835 Acked Read 10 Sent Done Done false
12941 3836 Acked Read 5 Sent Done Done false
12942 3837 Acked Write 7 Sent Done Done false
12943 3838 Acked Write 10 Sent Done Done false
12944 3839 Acked Read 7 Sent Done Done false
12945 3840 Acked Read 5 Sent Done Done false
12946 3841 Acked Read 1 Sent Done Done false
12947 3842 Acked Write 1 Sent Done Done false
12948 3843 Acked Flush 0 Sent Done Done false
12949 3844 Acked Write 4 Sent Done Done false
12950 3845 Acked Flush 0 Sent Done Done false
12951 3846 Acked Write 1 Sent Done Done false
12952 3847 Acked Write 2 Sent Done Done false
12953 3848 Acked Flush 0 Sent Done Done false
12954 3849 Acked Read 5 Sent Done Done false
12955 3850 Acked Write 6 Sent Done Done false
12956 3851 Acked Read 3 Sent Done Done false
12957 3852 Acked Read 2 Sent Done Done false
12958 3853 Acked Read 1 Sent Done Done false
12959 3854 Acked Write 6 Sent Done Done false
12960 3855 Acked Write 8 Sent Done Done false
12961 3856 Acked Read 8 Sent Done Done false
12962 3857 Acked Write 8 Sent Done Done false
12963 3858 Acked Read 6 Sent Done Done false
12964 3859 Acked Read 7 Sent Done Done false
12965 3860 Acked Read 6 Sent Done Done false
12966 3861 Acked Write 6 Sent Done Done false
12967 3862 Acked Read 4 Sent Done Done false
12968 3863 Acked Read 9 Sent Done Done false
12969 3864 Acked Write 4 Sent Done Done false
12970 3865 Acked Read 9 Sent Done Done false
12971 3866 Acked Read 6 Sent Done Done false
12972 3867 Acked Write 8 Sent Done Done false
12973 3868 Acked Read 7 Sent Done Done false
12974 3869 Acked Read 2 Sent Done Done false
12975 3870 Acked Write 4 Sent Done Done false
12976 3871 Acked Read 9 Sent Done Done false
12977 3872 Acked Read 10 Sent Done Done false
12978 3873 Acked Write 5 Sent Done Done false
12979 3874 Acked Read 1 Sent Done Done false
12980 3875 Acked Write 2 Sent Done Done false
12981 3876 Acked Read 6 Sent Done Done false
12982 3877 Acked Read 1 Sent Done Done false
12983 3878 Acked Read 10 Sent Done Done false
12984 3879 Acked Read 3 Sent Done Done false
12985 3880 Acked Flush 0 Sent Done Done false
12986 3881 Acked Write 10 Sent Done Done false
12987 3882 Acked Write 10 Sent Done Done false
12988 3883 Acked Write 5 Sent Done Done false
12989 3884 Acked Flush 0 Sent Done Done false
12990 3885 Acked Flush 0 Sent Done Done false
12991 3886 Acked Flush 0 Sent Done Done false
12992 3887 Acked Read 6 Sent Done Done false
12993 3888 Acked Read 7 Sent Done Done false
12994 3889 Acked Read 9 Sent Done Done false
12995 3890 Acked Read 1 Sent Done Done false
12996 3891 Acked Write 3 Sent Done Done false
12997 3892 Acked Flush 0 Sent Done Done false
12998 3893 Acked Flush 0 Sent Done Done false
12999 3894 Acked Read 7 Sent Done Done false
13000 3895 Acked Write 4 Sent Done Done false
13001 3896 Acked Write 3 Sent Done Done false
13002 3897 Acked Read 6 Sent Done Done false
13003 3898 Acked Write 1 Sent Done Done false
13004 3899 Acked Write 8 Sent Done Done false
13005 3900 Acked Write 1 Sent Done Done false
13006 3901 Acked Write 2 Sent Done Done false
13007 3902 Acked Write 8 Sent Done Done false
13008 3903 Acked Read 5 Sent Done Done false
13009 3904 Acked Flush 0 Sent Done Done false
13010 3905 Acked Read 2 Sent Done Done false
13011 3906 Acked Flush 0 Sent Done Done false
13012 3907 Acked Write 7 Sent Done Done false
13013 3908 Acked Read 8 Sent Done Done false
13014 3909 Acked Read 5 Sent Done Done false
13015 3910 Acked Read 9 Sent Done Done false
13016 3911 Acked Read 4 Sent Done Done false
13017 3912 Acked Read 2 Sent Done Done false
13018 3913 Acked Read 6 Sent Done Done false
13019 3914 Acked Read 8 Sent Done Done false
13020 3915 Acked Read 5 Sent Done Done false
13021 3916 Acked Read 3 Sent Done Done false
13022 3917 Acked Read 7 Sent Done Done false
13023 3918 Acked Flush 0 Sent Done Done false
13024 3919 Acked Read 5 Sent Done Done false
13025 3920 Acked Write 5 Sent Done Done false
13026 3921 Acked Write 10 Sent Done Done false
13027 3922 Acked Read 3 Sent Done Done false
13028 3923 Acked Read 4 Sent Done Done false
13029 3924 Acked Read 7 Sent Done Done false
13030 3925 Acked Read 4 Sent Done Done false
13031 3926 Acked Read 7 Sent Done Done false
13032 3927 Acked Write 7 Sent Done Done false
13033 3928 Acked Read 4 Sent Done Done false
13034 3929 Acked Write 3 Sent Done Done false
13035 3930 Acked Flush 0 Sent Done Done false
13036 3931 Acked Read 6 Sent Done Done false
13037 3932 Acked Write 2 Sent Done Done false
13038 3933 Acked Read 9 Sent Done Done false
13039 3934 Acked Flush 0 Sent Done Done false
13040 3935 Acked Flush 0 Sent Done Done false
13041 3936 Acked Read 9 Sent Done Done false
13042 3937 Acked Read 7 Sent Done Done false
13043 3938 Acked Write 10 Sent Done Done false
13044 3939 Acked Read 8 Sent Done Done false
13045 3940 Acked Read 10 Sent Done Done false
13046 3941 Acked Read 5 Sent Done Done false
13047 3942 Acked Flush 0 Sent Done Done false
13048 3943 Acked Write 6 Sent Done Done false
13049 3944 Acked Read 8 Sent Done Done false
13050 3945 Acked Read 10 Sent Done Done false
13051 3946 Acked Read 5 Sent Done Done false
13052 3947 Acked Read 5 Sent Done Done false
13053 3948 Acked Flush 0 Sent Done Done false
13054 3949 Acked Read 6 Sent Done Done false
13055 3950 Acked Write 10 Sent Done Done false
13056 3951 Acked Flush 0 Sent Done Done false
13057 3952 Acked Read 2 Sent Done Done false
13058 3953 Acked Write 1 Sent Done Done false
13059 3954 Acked Read 2 Sent Done Done false
13060 3955 Acked Write 3 Sent Done Done false
13061 3956 Acked Read 9 Sent Done Done false
13062 3957 Acked Read 3 Sent Done Done false
13063 3958 Acked Read 5 Sent Done Done false
13064 3959 Acked Read 1 Sent Done Done false
13065 3960 Acked Read 1 Sent Done Done false
13066 3961 Acked Flush 0 Sent Done Done false
13067 3962 Acked Read 10 Sent Done Done false
13068 3963 Acked Read 2 Sent Done Done false
13069 3964 Acked Write 10 Sent Done Done false
13070 3965 Acked Write 4 Sent Done Done false
13071 3966 Acked Flush 0 Sent Done Done false
13072 3967 Acked Write 8 Sent Done Done false
13073 3968 Acked Write 4 Sent Done Done false
13074 3969 Acked Flush 0 Sent Done Done false
13075 3970 Acked Write 6 Sent Done Done false
13076 3971 Acked Read 5 Sent Done Done false
13077 3972 Acked Write 1 Sent Done Done false
13078 3973 Acked Write 6 Sent Done Done false
13079 3974 Acked Write 2 Sent Done Done false
13080 3975 Acked Write 2 Sent Done Done false
13081 3976 Acked Read 8 Sent Done Done false
13082 3977 Acked Read 9 Sent Done Done false
13083 3978 Acked Flush 0 Sent Done Done false
13084 3979 Acked Write 9 Sent Done Done false
13085 3980 Acked Write 5 Sent Done Done false
13086 3981 Acked Write 4 Sent Done Done false
13087 3982 Acked Read 4 Sent Done Done false
13088 3983 Acked Read 10 Sent Done Done false
13089 3984 Acked Write 4 Sent Done Done false
13090 3985 Acked Read 9 Sent Done Done false
13091 3986 Acked Read 6 Sent Done Done false
13092 3987 Acked Write 10 Sent Done Done false
13093 3988 Acked Write 4 Sent Done Done false
13094 3989 Acked Read 9 Sent Done Done false
13095 3990 Acked Read 3 Sent Done Done false
13096 3991 Acked Write 3 Sent Done Done false
13097 3992 Acked Read 9 Sent Done Done false
13098 3993 Acked Read 7 Sent Done Done false
13099 3994 Acked Write 10 Sent Done Done false
13100 3995 Acked Read 1 Sent Done Done false
13101 3996 Acked Write 10 Sent Done Done false
13102 3997 Acked Read 2 Sent Done Done false
13103 3998 Acked Read 3 Sent Done Done false
13104 3999 Acked Read 10 Sent Done Done false
13105 4000 Acked Read 7 Sent Done Done false
13106 4001 Acked Read 10 Sent Done Done false
13107 4002 Acked Read 3 Sent Done Done false
13108 4003 Acked Read 3 Sent Done Done false
13109 4004 Acked Read 1 Sent Done Done false
13110 4005 Acked Write 6 Sent Done Done false
13111 4006 Acked Read 1 Sent Done Done false
13112 4007 Acked Read 10 Sent Done Done false
13113 4008 Acked Read 3 Sent Done Done false
13114 4009 Acked Read 2 Sent Done Done false
13115 4010 Acked Write 8 Sent Done Done false
13116 4011 Acked Flush 0 Sent Done Done false
13117 4012 Acked Read 5 Sent Done Done false
13118 4013 Acked Flush 0 Sent Done Done false
13119 4014 Acked Write 9 Sent Done Done false
13120 4015 Acked Read 5 Sent Done Done false
13121 4016 Acked Write 10 Sent Done Done false
13122 4017 Acked Flush 0 Sent Done Done false
13123 STATES DS:0 DS:1 DS:2 TOTAL
13124 Sent 302 0 0 302
13125 Done 0 302 302 604
13126 Skipped 0 0 0 0
13127 Error 0 0 0 0
13128 Last Flush: 3715 4017 4017
13129 Downstairs last five completed: 3715 3714 3713 3712 3711
13130 Upstairs last five completed: 4017 4016 4015 4014 4013
13131 CLIENT: Up:0 ds:302 act:2
131322024-11-15T02:00:14.096ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
131332024-11-15T02:00:14.096ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:39929) = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
131342024-11-15T02:00:14.097ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
131352024-11-15T02:00:14.097ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
131362024-11-15T02:00:14.097ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
131372024-11-15T02:00:14.097ZINFOcrucible: send last flush ID to this DS: 3715 = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
131382024-11-15T02:00:14.098ZINFOcrucible: Replied this last flush ID: 3715 = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
131392024-11-15T02:00:14.098ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
131402024-11-15T02:00:14.098ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
131412024-11-15T02:00:14.098ZINFOcrucible: [0] client re-new 302 jobs since flush 3715 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
131422024-11-15T02:00:14.098ZINFOcrucible: [0] Replayed 302 jobs since flush: 3715 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
13143 ----------------------------------------------------------------
13144 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
13145 Downstairs last five completed: 4017 4016 4015 4014 4013
13146 Upstairs last five completed: 4017 4016 4015 4014 4013
13147 CLIENT: Up:0 ds:0 act:3
13148 Replay: All jobs finished
13149 Main task has work to do, go find it
13150 got dsc Stop(0)
13151 stop 0
13152 [8810] Got stop action so:false kr:false
13153 Nov 15 02:00:14.343 INFO request completed, latency_us: 341, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 7f702e77-22a5-43ba-8672-7df134142490, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
131542024-11-15T02:00:14.344ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
13155 Nov 15 02:00:14.344 INFO request completed, latency_us: 111, response_code: 200, uri: /state/cid/0, method: GET, req_id: d195fd2f-0c30-426c-ab13-5c5672c160f1, remote_addr: 127{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2024-11-15T02:00:14.344620739Z","hostname":"w-01JCPRHT5F2MPA44KVJHM3S2PB","pid":1205,"":"io task","client":"0","":"downstairs","session_id":"971dcb2e-6891-4ecf-b367-1e04e52065ee"}
13156 .0.0.1:40008, local_addr: 127.0.0.1:9998
131572024-11-15T02:00:14.344ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
131582024-11-15T02:00:14.344ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
131592024-11-15T02:00:14.344ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
131602024-11-15T02:00:14.344ZINFOcrucible: client task is exiting = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
13161 [8810] Exited with: ExitStatus(unix_wait_status(9))
13162 [8810][0] reports Exit
13163 Nov 15 02:00:18.346 INFO request completed, latency_us: 224, response_code: 200, uri: /state/cid/0, method: GET, req_id: f3f1054a-36c2-4ac4-b38f-2c78a8d3a173, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
13164 001/300 Write block 474 len 28672 data: 3 3 4 6 5 6 6
13165 002/300 Write block 4367 len 8192 data: 3 2
13166 003/300 Read block 3146 len 40960
13167 004/300 Read block 3049 len 36864
13168 005/300 Write block 2271 len 24576 data: 4 4 5 4 3 3
13169 006/300 Write block 3156 len 12288 data: 3 3 4
13170 007/300 Read block 1436 len 36864
13171 008/300 Write block 3475 len 24576 data: 2 2 2 2 2 3
13172 009/300 Read block 3760 len 40960
13173 010/300 Read block 20 len 8192
13174 011/300 Flush
13175 012/300 Read block 2303 len 8192
13176 013/300 Write block 4309 len 20480 data: 4 3 3 5 5
13177 014/300 Flush
13178 015/300 Read block 2515 len 28672
13179 016/300 Write block 1867 len 12288 data: 3 3 4
13180 017/300 Write block 4442 len 20480 data: 4 3 4 4 4
13181 018/300 Write block 4462 len 24576 data: 2 2 2 2 2 2
13182 019/300 Read block 1467 len 12288
13183 020/300 Read block 597 len 20480
13184 021/300 Write block 3781 len 40960 data: 4 4 2 2 2 2 3 3 3 2
13185 022/300 Write block 4744 len 32768 data: 2 2 2 2 2 2 2 3
13186 023/300 Write block 1865 len 12288 data: 3 3 4
13187 024/300 Read block 240 len 12288
13188 025/300 Write block 3618 len 12288 data: 3 3 3
13189 026/300 Flush
13190 027/300 Write block 4835 len 32768 data: 5 4 4 4 4 4 5 5
13191 028/300 Write block 325 len 20480 data: 2 2 4 3 3
13192 029/300 Read block 1108 len 36864
13193 030/300 Read block 3878 len 24576
13194 031/300 Read block 3919 len 28672
13195 032/300 Read block 947 len 20480
13196 033/300 Write block 1968 len 36864 data: 2 2 2 2 2 3 3 3 2
13197 034/300 Read block 2217 len 24576
13198 035/300 Read block 1376 len 32768
13199 036/300 Write block 3941 len 40960 data: 2 2 2 2 2 2 2 2 3 3
13200 037/300 Write block 4014 len 12288 data: 3 4 3
13201 038/300 Write block 2289 len 16384 data: 3 2 2 3
13202 039/300 Write block 4242 len 4096 data: 3
13203 040/300 Read block 3865 len 4096
13204 041/300 Read block 734 len 36864
13205 042/300 Read block 4468 len 16384
13206 043/300 Write block 2345 len 12288 data: 3 3 3
13207 044/300 Read block 1072 len 12288
13208 045/300 Write block 594 len 16384 data: 5 5 3 3
13209 046/300 Read block 3292 len 28672
13210 047/300 Read block 1645 len 24576
13211 048/300 Read block 4058 len 4096
13212 049/300 Read block 2064 len 40960
13213 050/300 Read block 593 len 36864
13214 051/300 Write block 4259 len 40960 data: 5 4 3 3 4 5 6 6 4 4
13215 052/300 Read block 2338 len 12288
13216 053/300 Flush
13217 054/300 Read block 3372 len 24576
13218 055/300 Read block 4261 len 32768
13219 056/300 Read block 1190 len 40960
13220 057/300 Write block 3907 len 12288 data: 4 4 4
13221 058/300 Write block 824 len 36864 data: 3 4 5 5 4 5 3 3 2
13222 059/300 Write block 1593 len 28672 data: 3 5 4 3 3 3 3
13223 060/300 Write block 4124 len 32768 data: 5 5 5 4 4 5 5 5
13224 061/300 Read block 3397 len 8192
13225 062/300 Write block 3434 len 40960 data: 4 3 3 2 2 2 3 3 3 3
13226 063/300 Flush
13227 064/300 Read block 2287 len 12288
13228 065/300 Write block 2549 len 32768 data: 4 4 3 3 3 3 3 2
13229 066/300 Read block 3650 len 28672
13230 067/300 Read block 269 len 4096
13231 068/300 Read block 1261 len 24576
13232 069/300 Read block 3394 len 16384
13233 070/300 Read block 1857 len 28672
13234 071/300 Read block 2274 len 16384
13235 072/300 Write block 3871 len 20480 data: 4 4 3 4 5
13236 073/300 Write block 1095 len 16384 data: 3 4 4 4
13237 074/300 Read block 1348 len 8192
13238 075/300 Read block 2837 len 32768
13239 076/300 Read block 2043 len 36864
13240 077/300 Write block 3499 len 28672 data: 3 3 3 2 2 2 3
13241 078/300 Write block 2054 len 40960 data: 3 3 3 3 3 5 5 5 6 5
13242 079/300 Read block 1714 len 4096
13243 080/300 Read block 1837 len 24576
13244 081/300 Write block 3832 len 32768 data: 2 2 2 2 4 4 4 4
13245 082/300 Write block 4420 len 8192 data: 5 5
13246 083/300 Write block 107 len 16384 data: 3 3 3 2
13247 084/300 Read block 1152 len 12288
13248 085/300 Write block 1273 len 4096 data: 4
13249 086/300 Read block 3311 len 40960
13250 087/300 Read block 3007 len 4096
13251 088/300 Write block 4316 len 40960 data: 6 6 3 3 3 3 2 3 3 3
13252 089/300 Write block 631 len 8192 data: 3 3
13253 090/300 Read block 1456 len 28672
13254 091/300 Write block 989 len 40960 data: 3 3 4 4 4 4 4 5 4 5
13255 092/300 Read block 3899 len 36864
13256 093/300 Read block 1999 len 36864
13257 094/300 Write block 2237 len 24576 data: 5 5 3 3 3 3
13258 095/300 Flush
13259 096/300 Write block 1827 len 20480 data: 3 3 3 3 4
13260 097/300 Read block 505 len 36864
13261 098/300 Flush
13262 099/300 Read block 2978 len 8192
13263 100/300 Write block 355 len 12288 data: 3 3 3
13264 101/300 Read block 3969 len 12288
13265 102/300 Write block 4689 len 24576 data: 3 4 3 3 3 3
13266 103/300 Write block 3574 len 36864 data: 3 3 3 3 4 4 4 4 4
13267 104/300 Read block 4432 len 8192
13268 105/300 Write block 1498 len 36864 data: 2 2 2 2 2 2 2 2 3
13269 106/300 Write block 3313 len 4096 data: 3
13270 107/300 Write block 2745 len 28672 data: 3 3 2 3 2 2 3
13271 108/300 Read block 4012 len 32768
13272 109/300 Read block 3608 len 8192
13273 110/300 Read block 3343 len 12288
13274 111/300 Read block 3752 len 40960
13275 112/300 Write block 2025 len 8192 data: 3 3
13276 113/300 Read block 2614 len 16384
13277 114/300 Write block 4128 len 28672 data: 5 6 6 6 4 3 3
13278 115/300 Write block 4298 len 20480 data: 3 4 3 4 4
13279 116/300 Read block 3963 len 24576
13280 117/300 Read block 1937 len 8192
13281 118/300 Read block 4719 len 32768
13282 119/300 Write block 313 len 8192 data: 3 3
13283 120/300 Read block 485 len 32768
13284 121/300 Read block 2755 len 8192
13285 122/300 Read block 4021 len 20480
13286 123/300 Flush
13287 124/300 Write block 1620 len 20480 data: 3 3 3 2 2
13288 125/300 Read block 2770 len 40960
13289 126/300 Read block 612 len 32768
13290 127/300 Write block 1686 len 8192 data: 4 3
13291 128/300 Read block 2744 len 12288
13292 129/300 Write block 3170 len 40960 data: 3 3 3 3 3 2 2 3 2 3
13293 130/300 Read block 3221 len 8192
13294 131/300 Read block 1961 len 20480
13295 132/300 Write block 2584 len 32768 data: 2 2 2 3 3 3 3 4
13296 133/300 Write block 4918 len 24576 data: 5 4 5 6 7 8
13297 134/300 Flush
13298 135/300 Write block 1196 len 36864 data: 7 7 7 7 5 3 3 3 2
13299 136/300 Read block 2311 len 16384
13300 137/300 Read block 2502 len 12288
13301 138/300 Write block 1606 len 8192 data: 3 3
13302 139/300 Write block 1478 len 20480 data: 2 2 2 2 2
13303 140/300 Write block 1652 len 24576 data: 2 2 3 3 3 2
13304 141/300 Write block 4230 len 28672 data: 4 5 4 4 4 3 3
13305 142/300 Write block 1052 len 12288 data: 3 3 3
13306 143/300 Read block 1546 len 8192
13307 144/300 Write block 3803 len 32768 data: 3 3 3 3 2 2 3 3
13308 145/300 Write block 1261 len 36864 data: 3 3 2 2 2 2 3 3 3
13309 146/300 Read block 2856 len 16384
13310 147/300 Write block 291 len 24576 data: 3 4 4 4 4 3
13311 148/300 Read block 3116 len 40960
13312 149/300 Write block 4662 len 16384 data: 4 3 3 4
13313 150/300 Write block 3801 len 32768 data: 3 3 4 4 4 4 3 3
13314 151/300 Read block 1156 len 32768
13315 152/300 Write block 3257 len 8192 data: 5 5
13316 153/300 Read block 2842 len 16384
13317 154/300 Read block 1030 len 12288
13318 155/300 Read block 4481 len 36864
13319 156/300 Write block 3103 len 4096 data: 2
13320 157/300 Read block 154 len 8192
13321 158/300 Read block 2542 len 16384
13322 159/300 Read block 4291 len 36864
13323 160/300 Read block 2592 len 32768
13324 161/300 Write block 2688 len 20480 data: 3 3 2 2 2
13325 162/300 Read block 3320 len 28672
13326 163/300 Write block 2470 len 24576 data: 3 3 3 4 4 4
13327 164/300 Write block 3552 len 12288 data: 4 4 3
13328 165/300 Read block 4523 len 28672
13329 166/300 Flush
13330 167/300 Write block 3056 len 40960 data: 3 4 2 2 2 2 2 2 2 2
13331 168/300 Read block 2181 len 32768
13332 169/300 Write block 357 len 8192 data: 4 3
13333 170/300 Read block 1227 len 20480
13334 171/300 Read block 421 len 20480
13335 172/300 Read block 1877 len 24576
13336 173/300 Read block 1026 len 16384
13337 174/300 Read block 3205 len 28672
13338 175/300 Read block 1745 len 16384
13339 176/300 Read block 1626 len 40960
13340 177/300 Read block 595 len 16384
13341 178/300 Read block 1558 len 28672
13342 179/300 Write block 4590 len 8192 data: 5 5
13343 180/300 Read block 1448 len 4096
13344 181/300 Read block 357 len 4096
13345 182/300 Read block 3470 len 40960
13346 183/300 Read block 4001 len 20480
13347 184/300 Read block 2498 len 16384
13348 185/300 Write block 4701 len 32768 data: 3 3 4 5 5 6 4 5
13349 186/300 Flush
13350 187/300 Flush
13351 188/300 Write block 3840 len 12288 data: 4 4 3
13352 189/300 Read block 706 len 24576
13353 190/300 Read block 3441 len 16384
13354 191/300 Read block 1843 len 8192
13355 192/300 Read block 826 len 36864
13356 193/300 Read block 1899 len 16384
13357 194/300 Write block 4714 len 16384 data: 4 4 3 2
13358 195/300 Write block 2229 len 28672 data: 4 4 5 5 5 4 3
13359 196/300 Read block 4237 len 4096
13360 197/300 Write block 3268 len 24576 data: 2 2 2 2 2 3
13361 198/300 Write block 3569 len 20480 data: 4 3 3 3 3
13362 199/300 Write block 4941 len 36864 data: 4 3 3 3 3 3 3 3 3
13363 200/300 Write block 1324 len 20480 data: 3 3 3 3 2
13364 201/300 Flush
13365 202/300 Read block 2367 len 36864
13366 203/300 Write block 4004 len 12288 data: 3 3 3
13367 204/300 Write block 2699 len 8192 data: 4 4
13368 205/300 Read block 3032 len 8192
13369 206/300 Write block 3158 len 20480 data: 5 4 4 4 3
13370 207/300 Write block 4616 len 36864 data: 3 3 3 3 3 3 3 3 3
13371 208/300 Write block 648 len 36864 data: 2 2 2 3 3 4 4 4 4
13372 209/300 Read block 4938 len 20480
13373 210/300 Write block 4716 len 40960 data: 4 3 2 3 2 2 2 2 4 4
13374 211/300 Read block 1643 len 4096
13375 212/300 Read block 3286 len 12288
13376 213/300 Read block 46 len 16384
13377 214/300 Read block 444 len 20480
13378 215/300 Read block 4206 len 36864
13379 216/300 Read block 718 len 28672
13380 217/300 Read block 3930 len 12288
13381 218/300 Read block 506 len 36864
13382 219/300 Read block 4033 len 16384
13383 220/300 Read block 715 len 16384
13384 221/300 Read block 2143 len 36864
13385 222/300 Write block 1617 len 32768 data: 3 3 3 4 4 4 3 3
13386 223/300 Read block 3968 len 8192
13387 224/300 Read block 2992 len 4096
13388 225/300 Write block 1219 len 32768 data: 3 3 2 2 2 2 2 2
13389 226/300 Read block 1597 len 12288
13390 227/300 Read block 3989 len 20480
13391 228/300 Read block 3037 len 36864
13392 229/300 Flush
13393 230/300 Write block 1206 len 4096 data: 2
13394 231/300 Write block 3948 len 4096 data: 3
13395 232/300 Flush
13396 233/300 Read block 813 len 32768
13397 234/300 Read block 1795 len 28672
13398 235/300 Read block 1903 len 32768
13399 236/300 Read block 903 len 36864
13400 237/300 Write block 43 len 32768 data: 4 4 3 3 3 3 3 3
13401 238/300 Read block 1648 len 24576
13402 239/300 Read block 1411 len 4096
13403 240/300 Write block 4306 len 16384 data: 4 3 5 5
13404 241/300 Read block 2339 len 24576
13405 242/300 Read block 3794 len 12288
13406 243/300 Flush
13407 244/300 Write block 1366 len 40960 data: 3 3 4 4 4 4 4 4 3 3
13408 245/300 Flush
13409 246/300 Write block 3874 len 40960 data: 5 6 4 3 4 3 4 4 4 4
13410 247/300 Write block 2635 len 24576 data: 3 2 2 2 2 2
13411 248/300 Read block 1326 len 20480
13412 249/300 Read block 2885 len 4096
13413 250/300 Write block 4962 len 20480 data: 2 2 2 2 3
13414 251/300 Write block 897 len 12288 data: 3 3 3
13415 252/300 Read block 4960 len 8192
13416 253/300 Write block 4220 len 32768 data: 4 3 3 3 3 4 4 4
13417 254/300 Write block 1823 len 12288 data: 3 3 3
13418 255/300 Read block 1009 len 36864
13419 256/300 Write block 1379 len 8192 data: 4 4
13420 257/300 Read block 4342 len 20480
13421 258/300 Read block 3158 len 32768
13422 259/300 Write block 3626 len 8192 data: 2 2
13423 260/300 Read block 972 len 36864
13424 261/300 Read block 609 len 12288
13425 262/300 Read block 1883 len 8192
13426 263/300 Read block 4027 len 16384
13427 264/300 Write block 1040 len 40960 data: 2 2 2 2 2 2 2 2 2 2
13428 265/300 Read block 1752 len 8192
13429 266/300 Read block 203 len 20480
13430 267/300 Write block 1768 len 28672 data: 4 4 2 2 2 2 2
13431 268/300 Flush
13432 269/300 Flush
13433 270/300 Write block 4989 len 4096 data: 2
13434 271/300 Read block 1645 len 36864
13435 272/300 Read block 694 len 20480
13436 273/300 Read block 3181 len 16384
13437 274/300 Write block 4128 len 4096 data: 6
13438 275/300 Flush
13439 276/300 Read block 4016 len 28672
13440 277/300 Read block 402 len 20480
13441 278/300 Flush
13442 279/300 Write block 2341 len 36864 data: 5 3 3 3 4 4 4 2 2
13443 280/300 Read block 1751 len 36864
13444 281/300 Write block 4817 len 16384 data: 2 2 2 2
13445 282/300 Write block 3894 len 40960 data: 3 3 2 4 6 6 6 7 7 8
13446 283/300 Read block 3543 len 4096
13447 284/300 Write block 4021 len 8192 data: 4 4
13448 285/300 Read block 2365 len 20480
13449 286/300 Write block 2527 len 28672 data: 4 4 4 4 5 5 4
13450 287/300 Write block 2163 len 40960 data: 3 2 2 2 2 2 2 3 3 3
13451 288/300 Write block 57 len 20480 data: 3 4 3 3 3
13452 289/300 Read block 2949 len 4096
13453 290/300 Flush
13454 291/300 Flush
13455 292/300 Write block 3711 len 8192 data: 2 2
13456 293/300 Write block 2097 len 28672 data: 3 3 3 3 3 3 3
13457 294/300 Write block 3847 len 32768 data: 2 2 3 4 4 4 4 4
13458 295/300 Write block 2383 len 40960 data: 4 5 5 5 7 5 6 4 3 3
13459 296/300 Flush
13460 297/300 Read block 4835 len 12288
13461 298/300 Write block 2620 len 20480 data: 3 2 2 2 2
13462 299/300 Read block 319 len 8192
13463 300/300 Read block 3929 len 4096
13464 301/300 Write block 4452 len 40960 data: 3 2 2 2 2 2 2 2 2 2
13465 Main task has work to do, go find it
13466 got dsc Start(0)
13467 start 0
13468 Nov 15 02:00:18.582 INFO request completed, latency_us: 214, response_code: 204, uri: /start/cid/0, method: POST, req_id: 3f2f1bba-181c-49ae-8a90-716a4d469d01, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
13469 Replay: started 0, returned:Ok(())
13470 ----------------------------------------------------------------
13471 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
13472 States: Offline Active Active
13473 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
13474 4018 Acked Write 7 Sent Done Done false
13475 4019 Acked Write 2 Sent Done Done false
13476 4020 Acked Read 10 Sent Done Done false
13477 4021 Acked Read 9 Sent Done Done false
13478 4022 Acked Write 6 Sent Done Done false
13479 4023 Acked Write 3 Sent Done Done false
13480 4024 Acked Read 9 Sent Done Done false
13481 4025 Acked Write 6 Sent Done Done false
13482 4026 Acked Read 10 Sent Done Done false
13483 4027 Acked Read 2 Sent Done Done false
13484 4028 Acked Flush 0 Sent Done Done false
13485 4029 Acked Read 2 Sent Done Done false
13486 4030 Acked Write 5 Sent Done Done false
13487 4031 Acked Flush 0 Sent Done Done false
13488 4032 Acked Read 7 Sent Done Done false
13489 4033 Acked Write 3 Sent Done Done false
13490 4034 Acked Write 5 Sent Done Done false
13491 4035 Acked Write 6 Sent Done Done false
13492 4036 Acked Read 3 Sent Done Done false
13493 4037 Acked Read 5 Sent Done Done false
13494 4038 Acked Write 10 Sent Done Done false
13495 4039 Acked Write 8 Sent Done Done false
13496 4040 Acked Write 3 Sent Done Done false
13497 4041 Acked Read 3 Sent Done Done false
13498 4042 Acked Write 3 Sent Done Done false
13499 4043 Acked Flush 0 Sent Done Done false
13500 4044 Acked Write 8 Sent Done Done false
13501 4045 Acked Write 5 Sent Done Done false
13502 4046 Acked Read 9 Sent Done Done false
13503 4047 Acked Read 6 Sent Done Done false
13504 4048 Acked Read 7 Sent Done Done false
13505 4049 Acked Read 5 Sent Done Done false
13506 4050 Acked Write 9 Sent Done Done false
13507 4051 Acked Read 6 Sent Done Done false
13508 4052 Acked Read 8 Sent Done Done false
13509 4053 Acked Write 10 Sent Done Done false
13510 4054 Acked Write 3 Sent Done Done false
13511 4055 Acked Write 4 Sent Done Done false
13512 4056 Acked Write 1 Sent Done Done false
13513 4057 Acked Read 1 Sent Done Done false
13514 4058 Acked Read 9 Sent Done Done false
13515 4059 Acked Read 4 Sent Done Done false
13516 4060 Acked Write 3 Sent Done Done false
13517 4061 Acked Read 3 Sent Done Done false
13518 4062 Acked Write 4 Sent Done Done false
13519 4063 Acked Read 7 Sent Done Done false
13520 4064 Acked Read 6 Sent Done Done false
13521 4065 Acked Read 1 Sent Done Done false
13522 4066 Acked Read 10 Sent Done Done false
13523 4067 Acked Read 9 Sent Done Done false
13524 4068 Acked Write 10 Sent Done Done false
13525 4069 Acked Read 3 Sent Done Done false
13526 4070 Acked Flush 0 Sent Done Done false
13527 4071 Acked Read 6 Sent Done Done false
13528 4072 Acked Read 8 Sent Done Done false
13529 4073 Acked Read 10 Sent Done Done false
13530 4074 Acked Write 3 Sent Done Done false
13531 4075 Acked Write 9 Sent Done Done false
13532 4076 Acked Write 7 Sent Done Done false
13533 4077 Acked Write 8 Sent Done Done false
13534 4078 Acked Read 2 Sent Done Done false
13535 4079 Acked Write 10 Sent Done Done false
13536 4080 Acked Flush 0 Sent Done Done false
13537 4081 Acked Read 3 Sent Done Done false
13538 4082 Acked Write 8 Sent Done Done false
13539 4083 Acked Read 7 Sent Done Done false
13540 4084 Acked Read 1 Sent Done Done false
13541 4085 Acked Read 6 Sent Done Done false
13542 4086 Acked Read 4 Sent Done Done false
13543 4087 Acked Read 7 Sent Done Done false
13544 4088 Acked Read 4 Sent Done Done false
13545 4089 Acked Write 5 Sent Done Done false
13546 4090 Acked Write 4 Sent Done Done false
13547 4091 Acked Read 2 Sent Done Done false
13548 4092 Acked Read 8 Sent Done Done false
13549 4093 Acked Read 9 Sent Done Done false
13550 4094 Acked Write 7 Sent Done Done false
13551 4095 Acked Write 10 Sent Done Done false
13552 4096 Acked Read 1 Sent Done Done false
13553 4097 Acked Read 6 Sent Done Done false
13554 4098 Acked Write 8 Sent Done Done false
13555 4099 Acked Write 2 Sent Done Done false
13556 4100 Acked Write 4 Sent Done Done false
13557 4101 Acked Read 3 Sent Done Done false
13558 4102 Acked Write 1 Sent Done Done false
13559 4103 Acked Read 10 Sent Done Done false
13560 4104 Acked Read 1 Sent Done Done false
13561 4105 Acked Write 10 Sent Done Done false
13562 4106 Acked Write 2 Sent Done Done false
13563 4107 Acked Read 7 Sent Done Done false
13564 4108 Acked Write 10 Sent Done Done false
13565 4109 Acked Read 9 Sent Done Done false
13566 4110 Acked Read 9 Sent Done Done false
13567 4111 Acked Write 6 Sent Done Done false
13568 4112 Acked Flush 0 Sent Done Done false
13569 4113 Acked Write 5 Sent Done Done false
13570 4114 Acked Read 9 Sent Done Done false
13571 4115 Acked Flush 0 Sent Done Done false
13572 4116 Acked Read 2 Sent Done Done false
13573 4117 Acked Write 3 Sent Done Done false
13574 4118 Acked Read 3 Sent Done Done false
13575 4119 Acked Write 6 Sent Done Done false
13576 4120 Acked Write 9 Sent Done Done false
13577 4121 Acked Read 2 Sent Done Done false
13578 4122 Acked Write 9 Sent Done Done false
13579 4123 Acked Write 1 Sent Done Done false
13580 4124 Acked Write 7 Sent Done Done false
13581 4125 Acked Read 8 Sent Done Done false
13582 4126 Acked Read 2 Sent Done Done false
13583 4127 Acked Read 3 Sent Done Done false
13584 4128 Acked Read 10 Sent Done Done false
13585 4129 Acked Write 2 Sent Done Done false
13586 4130 Acked Read 4 Sent Done Done false
13587 4131 Acked Write 7 Sent Done Done false
13588 4132 Acked Write 5 Sent Done Done false
13589 4133 Acked Read 6 Sent Done Done false
13590 4134 Acked Read 2 Sent Done Done false
13591 4135 Acked Read 8 Sent Done Done false
13592 4136 Acked Write 2 Sent Done Done false
13593 4137 Acked Read 8 Sent Done Done false
13594 4138 Acked Read 2 Sent Done Done false
13595 4139 Acked Read 5 Sent Done Done false
13596 4140 Acked Flush 0 Sent Done Done false
13597 4141 Acked Write 5 Sent Done Done false
13598 4142 Acked Read 10 Sent Done Done false
13599 4143 Acked Read 8 Sent Done Done false
13600 4144 Acked Write 2 Sent Done Done false
13601 4145 Acked Read 3 Sent Done Done false
13602 4146 Acked Write 10 Sent Done Done false
13603 4147 Acked Read 2 Sent Done Done false
13604 4148 Acked Read 5 Sent Done Done false
13605 4149 Acked Write 8 Sent Done Done false
13606 4150 Acked Write 6 Sent Done Done false
13607 4151 Acked Flush 0 Sent Done Done false
13608 4152 Acked Write 9 Sent Done Done false
13609 4153 Acked Read 4 Sent Done Done false
13610 4154 Acked Read 3 Sent Done Done false
13611 4155 Acked Write 2 Sent Done Done false
13612 4156 Acked Write 5 Sent Done Done false
13613 4157 Acked Write 6 Sent Done Done false
13614 4158 Acked Write 7 Sent Done Done false
13615 4159 Acked Write 3 Sent Done Done false
13616 4160 Acked Read 2 Sent Done Done false
13617 4161 Acked Write 8 Sent Done Done false
13618 4162 Acked Write 9 Sent Done Done false
13619 4163 Acked Read 4 Sent Done Done false
13620 4164 Acked Write 6 Sent Done Done false
13621 4165 Acked Read 10 Sent Done Done false
13622 4166 Acked Write 4 Sent Done Done false
13623 4167 Acked Write 8 Sent Done Done false
13624 4168 Acked Read 8 Sent Done Done false
13625 4169 Acked Write 2 Sent Done Done false
13626 4170 Acked Read 4 Sent Done Done false
13627 4171 Acked Read 3 Sent Done Done false
13628 4172 Acked Read 9 Sent Done Done false
13629 4173 Acked Write 1 Sent Done Done false
13630 4174 Acked Read 2 Sent Done Done false
13631 4175 Acked Read 4 Sent Done Done false
13632 4176 Acked Read 9 Sent Done Done false
13633 4177 Acked Read 8 Sent Done Done false
13634 4178 Acked Write 5 Sent Done Done false
13635 4179 Acked Read 7 Sent Done Done false
13636 4180 Acked Write 6 Sent Done Done false
13637 4181 Acked Write 3 Sent Done Done false
13638 4182 Acked Read 7 Sent Done Done false
13639 4183 Acked Flush 0 Sent Done Done false
13640 4184 Acked Write 10 Sent Done Done false
13641 4185 Acked Read 8 Sent Done Done false
13642 4186 Acked Write 2 Sent Done Done false
13643 4187 Acked Read 5 Sent Done Done false
13644 4188 Acked Read 5 Sent Done Done false
13645 4189 Acked Read 6 Sent Done Done false
13646 4190 Acked Read 4 Sent Done Done false
13647 4191 Acked Read 7 Sent Done Done false
13648 4192 Acked Read 4 Sent Done Done false
13649 4193 Acked Read 10 Sent Done Done false
13650 4194 Acked Read 4 Sent Done Done false
13651 4195 Acked Read 7 Sent Done Done false
13652 4196 Acked Write 2 Sent Done Done false
13653 4197 Acked Read 1 Sent Done Done false
13654 4198 Acked Read 1 Sent Done Done false
13655 4199 Acked Read 10 Sent Done Done false
13656 4200 Acked Read 5 Sent Done Done false
13657 4201 Acked Read 4 Sent Done Done false
13658 4202 Acked Write 8 Sent Done Done false
13659 4203 Acked Flush 0 Sent Done Done false
13660 4204 Acked Flush 0 Sent Done Done false
13661 4205 Acked Write 3 Sent Done Done false
13662 4206 Acked Read 6 Sent Done Done false
13663 4207 Acked Read 4 Sent Done Done false
13664 4208 Acked Read 2 Sent Done Done false
13665 4209 Acked Read 9 Sent Done Done false
13666 4210 Acked Read 4 Sent Done Done false
13667 4211 Acked Write 4 Sent Done Done false
13668 4212 Acked Write 7 Sent Done Done false
13669 4213 Acked Read 1 Sent Done Done false
13670 4214 Acked Write 6 Sent Done Done false
13671 4215 Acked Write 5 Sent Done Done false
13672 4216 Acked Write 9 Sent Done Done false
13673 4217 Acked Write 5 Sent Done Done false
13674 4218 Acked Flush 0 Sent Done Done false
13675 4219 Acked Read 9 Sent Done Done false
13676 4220 Acked Write 3 Sent Done Done false
13677 4221 Acked Write 2 Sent Done Done false
13678 4222 Acked Read 2 Sent Done Done false
13679 4223 Acked Write 5 Sent Done Done false
13680 4224 Acked Write 9 Sent Done Done false
13681 4225 Acked Write 9 Sent Done Done false
13682 4226 Acked Read 5 Sent Done Done false
13683 4227 Acked Write 10 Sent Done Done false
13684 4228 Acked Read 1 Sent Done Done false
13685 4229 Acked Read 3 Sent Done Done false
13686 4230 Acked Read 4 Sent Done Done false
13687 4231 Acked Read 5 Sent Done Done false
13688 4232 Acked Read 9 Sent Done Done false
13689 4233 Acked Read 7 Sent Done Done false
13690 4234 Acked Read 3 Sent Done Done false
13691 4235 Acked Read 9 Sent Done Done false
13692 4236 Acked Read 4 Sent Done Done false
13693 4237 Acked Read 4 Sent Done Done false
13694 4238 Acked Read 9 Sent Done Done false
13695 4239 Acked Write 8 Sent Done Done false
13696 4240 Acked Read 2 Sent Done Done false
13697 4241 Acked Read 1 Sent Done Done false
13698 4242 Acked Write 8 Sent Done Done false
13699 4243 Acked Read 3 Sent Done Done false
13700 4244 Acked Read 5 Sent Done Done false
13701 4245 Acked Read 9 Sent Done Done false
13702 4246 Acked Flush 0 Sent Done Done false
13703 4247 Acked Write 1 Sent Done Done false
13704 4248 Acked Write 1 Sent Done Done false
13705 4249 Acked Flush 0 Sent Done Done false
13706 4250 Acked Read 8 Sent Done Done false
13707 4251 Acked Read 7 Sent Done Done false
13708 4252 Acked Read 8 Sent Done Done false
13709 4253 Acked Read 9 Sent Done Done false
13710 4254 Acked Write 8 Sent Done Done false
13711 4255 Acked Read 6 Sent Done Done false
13712 4256 Acked Read 1 Sent Done Done false
13713 4257 Acked Write 4 Sent Done Done false
13714 4258 Acked Read 6 Sent Done Done false
13715 4259 Acked Read 3 Sent Done Done false
13716 4260 Acked Flush 0 Sent Done Done false
13717 4261 Acked Write 10 Sent Done Done false
13718 4262 Acked Flush 0 Sent Done Done false
13719 4263 Acked Write 10 Sent Done Done false
13720 4264 Acked Write 6 Sent Done Done false
13721 4265 Acked Read 5 Sent Done Done false
13722 4266 Acked Read 1 Sent Done Done false
13723 4267 Acked Write 5 Sent Done Done false
13724 4268 Acked Write 3 Sent Done Done false
13725 4269 Acked Read 2 Sent Done Done false
13726 4270 Acked Write 8 Sent Done Done false
13727 4271 Acked Write 3 Sent Done Done false
13728 4272 Acked Read 9 Sent Done Done false
13729 4273 Acked Write 2 Sent Done Done false
13730 4274 Acked Read 5 Sent Done Done false
13731 4275 Acked Read 8 Sent Done Done false
13732 4276 Acked Write 2 Sent Done Done false
13733 4277 Acked Read 9 Sent Done Done false
13734 4278 Acked Read 3 Sent Done Done false
13735 4279 Acked Read 2 Sent Done Done false
13736 4280 Acked Read 4 Sent Done Done false
13737 4281 Acked Write 10 Sent Done Done false
13738 4282 Acked Read 2 Sent Done Done false
13739 4283 Acked Read 5 Sent Done Done false
13740 4284 Acked Write 7 Sent Done Done false
13741 4285 Acked Flush 0 Sent Done Done false
13742 4286 Acked Flush 0 Sent Done Done false
13743 4287 Acked Write 1 Sent Done Done false
13744 4288 Acked Read 9 Sent Done Done false
13745 4289 Acked Read 5 Sent Done Done false
13746 4290 Acked Read 4 Sent Done Done false
13747 4291 Acked Write 1 Sent Done Done false
13748 4292 Acked Flush 0 Sent Done Done false
13749 4293 Acked Read 7 Sent Done Done false
13750 4294 Acked Read 5 Sent Done Done false
13751 4295 Acked Flush 0 Sent Done Done false
13752 4296 Acked Write 9 Sent Done Done false
13753 4297 Acked Read 9 Sent Done Done false
13754 4298 Acked Write 4 Sent Done Done false
13755 4299 Acked Write 10 Sent Done Done false
13756 4300 Acked Read 1 Sent Done Done false
13757 4301 Acked Write 2 Sent Done Done false
13758 4302 Acked Read 5 Sent Done Done false
13759 4303 Acked Write 7 Sent Done Done false
13760 4304 Acked Write 10 Sent Done Done false
13761 4305 Acked Write 5 Sent Done Done false
13762 4306 Acked Read 1 Sent Done Done false
13763 4307 Acked Flush 0 Sent Done Done false
13764 4308 Acked Flush 0 Sent Done Done false
13765 4309 Acked Write 2 Sent Done Done false
13766 4310 Acked Write 7 Sent Done Done false
13767 4311 Acked Write 8 Sent Done Done false
13768 4312 Acked Write 10 Sent Done Done false
13769 4313 Acked Flush 0 Sent Done Done false
13770 4314 Acked Read 3 Sent Done Done false
13771 4315 Acked Write 5 Sent Done Done false
13772 4316 Acked Read 2 Sent Done Done false
13773 4317 Acked Read 1 Sent Done Done false
13774 4318 Acked Write 10 Sent Done Done false
13775 STATES DS:0 DS:1 DS:2 TOTAL
13776 Sent 301 0 0 301
13777 Done 0 301 301 602
13778 Skipped 0 0 0 0
13779 Error 0 0 0 0
13780 Last Flush: 4017 4313 4313
13781 Downstairs last five completed: 4017 4016 4015 4014 4013
13782 Upstairs last five completed: 4318 4317 4316 4315 4314
13783 CLIENT: Up:0 ds:301 act:2
13784 ----------------------------------------------------------------
13785 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
13786 States: Offline Active Active
13787 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
13788 4018 Acked Write 7 Sent Done Done false
13789 4019 Acked Write 2 Sent Done Done false
13790 4020 Acked Read 10 Sent Done Done false
13791 4021 Acked Read 9 Sent Done Done false
13792 4022 Acked Write 6 Sent Done Done false
13793 4023 Acked Write 3 Sent Done Done false
13794 4024 Acked Read 9 Sent Done Done false
13795 4025 Acked Write 6 Sent Done Done false
13796 4026 Acked Read 10 Sent Done Done false
13797 4027 Acked Read 2 Sent Done Done false
13798 4028 Acked Flush 0 Sent Done Done false
13799 4029 Acked Read 2 Sent Done Done false
13800 4030 Acked Write 5 Sent Done Done false
13801 4031 Acked Flush 0 Sent Done Done false
13802 4032 Acked Read 7 Sent Done Done false
13803 4033 Acked Write 3 Sent Done Done false
13804 4034 Acked Write 5 Sent Done Done false
13805 4035 Acked Write 6 Sent Done Done false
13806 4036 Acked Read 3 Sent Done Done false
13807 4037 Acked Read 5 Sent Done Done false
13808 4038 Acked Write 10 Sent Done Done false
13809 4039 Acked Write 8 Sent Done Done false
13810 4040 Acked Write 3 Sent Done Done false
13811 4041 Acked Read 3 Sent Done Done false
13812 4042 Acked Write 3 Sent Done Done false
13813 4043 Acked Flush 0 Sent Done Done false
13814 4044 Acked Write 8 Sent Done Done false
13815 4045 Acked Write 5 Sent Done Done false
13816 4046 Acked Read 9 Sent Done Done false
13817 4047 Acked Read 6 Sent Done Done false
13818 4048 Acked Read 7 Sent Done Done false
13819 4049 Acked Read 5 Sent Done Done false
13820 4050 Acked Write 9 Sent Done Done false
13821 4051 Acked Read 6 Sent Done Done false
13822 4052 Acked Read 8 Sent Done Done false
13823 4053 Acked Write 10 Sent Done Done false
13824 4054 Acked Write 3 Sent Done Done false
13825 4055 Acked Write 4 Sent Done Done false
13826 4056 Acked Write 1 Sent Done Done false
13827 4057 Acked Read 1 Sent Done Done false
13828 4058 Acked Read 9 Sent Done Done false
13829 4059 Acked Read 4 Sent Done Done false
13830 4060 Acked Write 3 Sent Done Done false
13831 4061 Acked Read 3 Sent Done Done false
13832 4062 Acked Write 4 Sent Done Done false
13833 4063 Acked Read 7 Sent Done Done false
13834 4064 Acked Read 6 Sent Done Done false
13835 4065 Acked Read 1 Sent Done Done false
13836 4066 Acked Read 10 Sent Done Done false
13837 4067 Acked Read 9 Sent Done Done false
13838 4068 Acked Write 10 Sent Done Done false
13839 4069 Acked Read 3 Sent Done Done false
13840 4070 Acked Flush 0 Sent Done Done false
13841 4071 Acked Read 6 Sent Done Done false
13842 4072 Acked Read 8 Sent Done Done false
13843 4073 Acked Read 10 Sent Done Done false
13844 4074 Acked Write 3 Sent Done Done false
13845 4075 Acked Write 9 Sent Done Done false
13846 4076 Acked Write 7 Sent Done Done false
13847 4077 Acked Write 8 Sent Done Done false
13848 4078 Acked Read 2 Sent Done Done false
13849 4079 Acked Write 10 Sent Done Done false
13850 4080 Acked Flush 0 Sent Done Done false
13851 4081 Acked Read 3 Sent Done Done false
13852 4082 Acked Write 8 Sent Done Done false
13853 4083 Acked Read 7 Sent Done Done false
13854 4084 Acked Read 1 Sent Done Done false
13855 4085 Acked Read 6 Sent Done Done false
13856 4086 Acked Read 4 Sent Done Done false
13857 4087 Acked Read 7 Sent Done Done false
13858 4088 Acked Read 4 Sent Done Done false
13859 4089 Acked Write 5 Sent Done Done false
13860 4090 Acked Write 4 Sent Done Done false
13861 4091 Acked Read 2 Sent Done Done false
13862 4092 Acked Read 8 Sent Done Done false
13863 4093 Acked Read 9 Sent Done Done false
13864 4094 Acked Write 7 Sent Done Done false
13865 4095 Acked Write 10 Sent Done Done false
13866 4096 Acked Read 1 Sent Done Done false
13867 4097 Acked Read 6 Sent Done Done false
13868 4098 Acked Write 8 Sent Done Done false
13869 4099 Acked Write 2 Sent Done Done false
13870 4100 Acked Write 4 Sent Done Done false
13871 4101 Acked Read 3 Sent Done Done false
13872 4102 Acked Write 1 Sent Done Done false
13873 4103 Acked Read 10 Sent Done Done false
13874 4104 Acked Read 1 Sent Done Done false
13875 4105 Acked Write 10 Sent Done Done false
13876 4106 Acked Write 2 Sent Done Done false
13877 4107 Acked Read 7 Sent Done Done false
13878 4108 Acked Write 10 Sent Done Done false
13879 4109 Acked Read 9 Sent Done Done false
13880 4110 Acked Read 9 Sent Done Done false
13881 4111 Acked Write 6 Sent Done Done false
13882 4112 Acked Flush 0 Sent Done Done false
13883 4113 Acked Write 5 Sent Done Done false
13884 4114 Acked Read 9 Sent Done Done false
13885 4115 Acked Flush 0 Sent Done Done false
13886 4116 Acked Read 2 Sent Done Done false
13887 4117 Acked Write 3 Sent Done Done false
13888 4118 Acked Read 3 Sent Done Done false
13889 4119 Acked Write 6 Sent Done Done false
13890 4120 Acked Write 9 Sent Done Done false
13891 4121 Acked Read 2 Sent Done Done false
13892 4122 Acked Write 9 Sent Done Done false
13893 4123 Acked Write 1 Sent Done Done false
13894 4124 Acked Write 7 Sent Done Done false
13895 4125 Acked Read 8 Sent Done Done false
13896 4126 Acked Read 2 Sent Done Done false
13897 4127 Acked Read 3 Sent Done Done false
13898 4128 Acked Read 10 Sent Done Done false
13899 4129 Acked Write 2 Sent Done Done false
13900 4130 Acked Read 4 Sent Done Done false
13901 4131 Acked Write 7 Sent Done Done false
13902 4132 Acked Write 5 Sent Done Done false
13903 4133 Acked Read 6 Sent Done Done false
13904 4134 Acked Read 2 Sent Done Done false
13905 4135 Acked Read 8 Sent Done Done false
13906 4136 Acked Write 2 Sent Done Done false
13907 4137 Acked Read 8 Sent Done Done false
13908 4138 Acked Read 2 Sent Done Done false
13909 4139 Acked Read 5 Sent Done Done false
13910 4140 Acked Flush 0 Sent Done Done false
13911 4141 Acked Write 5 Sent Done Done false
13912 4142 Acked Read 10 Sent Done Done false
13913 4143 Acked Read 8 Sent Done Done false
13914 4144 Acked Write 2 Sent Done Done false
13915 4145 Acked Read 3 Sent Done Done false
13916 4146 Acked Write 10 Sent Done Done false
13917 4147 Acked Read 2 Sent Done Done false
13918 4148 Acked Read 5 Sent Done Done false
13919 4149 Acked Write 8 Sent Done Done false
13920 4150 Acked Write 6 Sent Done Done false
13921 4151 Acked Flush 0 Sent Done Done false
13922 4152 Acked Write 9 Sent Done Done false
13923 4153 Acked Read 4 Sent Done Done false
13924 4154 Acked Read 3 Sent Done Done false
13925 4155 Acked Write 2 Sent Done Done false
13926 4156 Acked Write 5 Sent Done Done false
13927 4157 Acked Write 6 Sent Done Done false
13928 4158 Acked Write 7 Sent Done Done false
13929 4159 Acked Write 3 Sent Done Done false
13930 4160 Acked Read 2 Sent Done Done false
13931 4161 Acked Write 8 Sent Done Done false
13932 4162 Acked Write 9 Sent Done Done false
13933 4163 Acked Read 4 Sent Done Done false
13934 4164 Acked Write 6 Sent Done Done false
13935 4165 Acked Read 10 Sent Done Done false
13936 4166 Acked Write 4 Sent Done Done false
13937 4167 Acked Write 8 Sent Done Done false
13938 4168 Acked Read 8 Sent Done Done false
13939 4169 Acked Write 2 Sent Done Done false
13940 4170 Acked Read 4 Sent Done Done false
13941 4171 Acked Read 3 Sent Done Done false
13942 4172 Acked Read 9 Sent Done Done false
13943 4173 Acked Write 1 Sent Done Done false
13944 4174 Acked Read 2 Sent Done Done false
13945 4175 Acked Read 4 Sent Done Done false
13946 4176 Acked Read 9 Sent Done Done false
13947 4177 Acked Read 8 Sent Done Done false
13948 4178 Acked Write 5 Sent Done Done false
13949 4179 Acked Read 7 Sent Done Done false
13950 4180 Acked Write 6 Sent Done Done false
13951 4181 Acked Write 3 Sent Done Done false
13952 4182 Acked Read 7 Sent Done Done false
13953 4183 Acked Flush 0 Sent Done Done false
13954 4184 Acked Write 10 Sent Done Done false
13955 4185 Acked Read 8 Sent Done Done false
13956 4186 Acked Write 2 Sent Done Done false
13957 4187 Acked Read 5 Sent Done Done false
13958 4188 Acked Read 5 Sent Done Done false
13959 4189 Acked Read 6 Sent Done Done false
13960 4190 Acked Read 4 Sent Done Done false
13961 4191 Acked Read 7 Sent Done Done false
13962 4192 Acked Read 4 Sent Done Done false
13963 4193 Acked Read 10 Sent Done Done false
13964 4194 Acked Read 4 Sent Done Done false
13965 4195 Acked Read 7 Sent Done Done false
13966 4196 Acked Write 2 Sent Done Done false
13967 4197 Acked Read 1 Sent Done Done false
13968 4198 Acked Read 1 Sent Done Done false
13969 4199 Acked Read 10 Sent Done Done false
13970 4200 Acked Read 5 Sent Done Done false
13971 4201 Acked Read 4 Sent Done Done false
13972 4202 Acked Write 8 Sent Done Done false
13973 4203 Acked Flush 0 Sent Done Done false
13974 4204 Acked Flush 0 Sent Done Done false
13975 4205 Acked Write 3 Sent Done Done false
13976 4206 Acked Read 6 Sent Done Done false
13977 4207 Acked Read 4 Sent Done Done false
13978 4208 Acked Read 2 Sent Done Done false
13979 4209 Acked Read 9 Sent Done Done false
13980 4210 Acked Read 4 Sent Done Done false
13981 4211 Acked Write 4 Sent Done Done false
13982 4212 Acked Write 7 Sent Done Done false
13983 4213 Acked Read 1 Sent Done Done false
13984 4214 Acked Write 6 Sent Done Done false
13985 4215 Acked Write 5 Sent Done Done false
13986 4216 Acked Write 9 Sent Done Done false
13987 4217 Acked Write 5 Sent Done Done false
13988 4218 Acked Flush 0 Sent Done Done false
13989 4219 Acked Read 9 Sent Done Done false
13990 4220 Acked Write 3 Sent Done Done false
13991 4221 Acked Write 2 Sent Done Done false
13992 4222 Acked Read 2 Sent Done Done false
13993 4223 Acked Write 5 Sent Done Done false
13994 4224 Acked Write 9 Sent Done Done false
13995 4225 Acked Write 9 Sent Done Done false
13996 4226 Acked Read 5 Sent Done Done false
13997 4227 Acked Write 10 Sent Done Done false
13998 4228 Acked Read 1 Sent Done Done false
13999 4229 Acked Read 3 Sent Done Done false
14000 4230 Acked Read 4 Sent Done Done false
14001 4231 Acked Read 5 Sent Done Done false
14002 4232 Acked Read 9 Sent Done Done false
14003 4233 Acked Read 7 Sent Done Done false
14004 4234 Acked Read 3 Sent Done Done false
14005 4235 Acked Read 9 Sent Done Done false
14006 4236 Acked Read 4 Sent Done Done false
14007 4237 Acked Read 4 Sent Done Done false
14008 4238 Acked Read 9 Sent Done Done false
14009 4239 Acked Write 8 Sent Done Done false
14010 4240 Acked Read 2 Sent Done Done false
14011 4241 Acked Read 1 Sent Done Done false
14012 4242 Acked Write 8 Sent Done Done false
14013 4243 Acked Read 3 Sent Done Done false
14014 4244 Acked Read 5 Sent Done Done false
14015 4245 Acked Read 9 Sent Done Done false
14016 4246 Acked Flush 0 Sent Done Done false
14017 4247 Acked Write 1 Sent Done Done false
14018 4248 Acked Write 1 Sent Done Done false
14019 4249 Acked Flush 0 Sent Done Done false
14020 4250 Acked Read 8 Sent Done Done false
14021 4251 Acked Read 7 Sent Done Done false
14022 4252 Acked Read 8 Sent Done Done false
14023 4253 Acked Read 9 Sent Done Done false
14024 4254 Acked Write 8 Sent Done Done false
14025 4255 Acked Read 6 Sent Done Done false
14026 4256 Acked Read 1 Sent Done Done false
14027 4257 Acked Write 4 Sent Done Done false
14028 4258 Acked Read 6 Sent Done Done false
14029 4259 Acked Read 3 Sent Done Done false
14030 4260 Acked Flush 0 Sent Done Done false
14031 4261 Acked Write 10 Sent Done Done false
14032 4262 Acked Flush 0 Sent Done Done false
14033 4263 Acked Write 10 Sent Done Done false
14034 4264 Acked Write 6 Sent Done Done false
14035 4265 Acked Read 5 Sent Done Done false
14036 4266 Acked Read 1 Sent Done Done false
14037 4267 Acked Write 5 Sent Done Done false
14038 4268 Acked Write 3 Sent Done Done false
14039 4269 Acked Read 2 Sent Done Done false
14040 4270 Acked Write 8 Sent Done Done false
14041 4271 Acked Write 3 Sent Done Done false
14042 4272 Acked Read 9 Sent Done Done false
14043 4273 Acked Write 2 Sent Done Done false
14044 4274 Acked Read 5 Sent Done Done false
14045 4275 Acked Read 8 Sent Done Done false
14046 4276 Acked Write 2 Sent Done Done false
14047 4277 Acked Read 9 Sent Done Done false
14048 4278 Acked Read 3 Sent Done Done false
14049 4279 Acked Read 2 Sent Done Done false
14050 4280 Acked Read 4 Sent Done Done false
14051 4281 Acked Write 10 Sent Done Done false
14052 4282 Acked Read 2 Sent Done Done false
14053 4283 Acked Read 5 Sent Done Done false
14054 4284 Acked Write 7 Sent Done Done false
14055 4285 Acked Flush 0 Sent Done Done false
14056 4286 Acked Flush 0 Sent Done Done false
14057 4287 Acked Write 1 Sent Done Done false
14058 4288 Acked Read 9 Sent Done Done false
14059 4289 Acked Read 5 Sent Done Done false
14060 4290 Acked Read 4 Sent Done Done false
14061 4291 Acked Write 1 Sent Done Done false
14062 4292 Acked Flush 0 Sent Done Done false
14063 4293 Acked Read 7 Sent Done Done false
14064 4294 Acked Read 5 Sent Done Done false
14065 4295 Acked Flush 0 Sent Done Done false
14066 4296 Acked Write 9 Sent Done Done false
14067 4297 Acked Read 9 Sent Done Done false
14068 4298 Acked Write 4 Sent Done Done false
14069 4299 Acked Write 10 Sent Done Done false
14070 4300 Acked Read 1 Sent Done Done false
14071 4301 Acked Write 2 Sent Done Done false
14072 4302 Acked Read 5 Sent Done Done false
14073 4303 Acked Write 7 Sent Done Done false
14074 4304 Acked Write 10 Sent Done Done false
14075 4305 Acked Write 5 Sent Done Done false
14076 4306 Acked Read 1 Sent Done Done false
14077 4307 Acked Flush 0 Sent Done Done false
14078 4308 Acked Flush 0 Sent Done Done false
14079 4309 Acked Write 2 Sent Done Done false
14080 4310 Acked Write 7 Sent Done Done false
14081 4311 Acked Write 8 Sent Done Done false
14082 4312 Acked Write 10 Sent Done Done false
14083 4313 Acked Flush 0 Sent Done Done false
14084 4314 Acked Read 3 Sent Done Done false
14085 4315 Acked Write 5 Sent Done Done false
14086 4316 Acked Read 2 Sent Done Done false
14087 4317 Acked Read 1 Sent Done Done false
14088 4318 Acked Write 10 Sent Done Done false
14089 4319 Acked Flush 0 Sent Done Done false
14090 STATES DS:0 DS:1 DS:2 TOTAL
14091 Sent 302 0 0 302
14092 Done 0 302 302 604
14093 Skipped 0 0 0 0
14094 Error 0 0 0 0
14095 Last Flush: 4017 4319 4319
14096 Downstairs last five completed: 4017 4016 4015 4014 4013
14097 Upstairs last five completed: 4319 4318 4317 4316 4315
14098 CLIENT: Up:0 ds:302 act:2
140992024-11-15T02:00:24.346ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
141002024-11-15T02:00:24.346ZWARNcrucible: ds_connection connect to 127.0.0.1:8810 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
141012024-11-15T02:00:24.346ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
141022024-11-15T02:00:24.346ZWARNcrucible: downstairs task for 0 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
141032024-11-15T02:00:24.346ZINFOcrucible: client task is exiting = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
14104 [8810] got start action
14105 [8810] I am going to restart
14106 Starting downstairs at port 8810
14107 Make output file at "/tmp/dsc/downstairs-8810.txt"
14108 [8810][0] reports Starting
14109 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1216)
14110 [8810][0] reports Running
14111 ----------------------------------------------------------------
14112 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
14113 States: Offline Active Active
14114 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
14115 4018 Acked Write 7 Sent Done Done false
14116 4019 Acked Write 2 Sent Done Done false
14117 4020 Acked Read 10 Sent Done Done false
14118 4021 Acked Read 9 Sent Done Done false
14119 4022 Acked Write 6 Sent Done Done false
14120 4023 Acked Write 3 Sent Done Done false
14121 4024 Acked Read 9 Sent Done Done false
14122 4025 Acked Write 6 Sent Done Done false
14123 4026 Acked Read 10 Sent Done Done false
14124 4027 Acked Read 2 Sent Done Done false
14125 4028 Acked Flush 0 Sent Done Done false
14126 4029 Acked Read 2 Sent Done Done false
14127 4030 Acked Write 5 Sent Done Done false
14128 4031 Acked Flush 0 Sent Done Done false
14129 4032 Acked Read 7 Sent Done Done false
14130 4033 Acked Write 3 Sent Done Done false
14131 4034 Acked Write 5 Sent Done Done false
14132 4035 Acked Write 6 Sent Done Done false
14133 4036 Acked Read 3 Sent Done Done false
14134 4037 Acked Read 5 Sent Done Done false
14135 4038 Acked Write 10 Sent Done Done false
14136 4039 Acked Write 8 Sent Done Done false
14137 4040 Acked Write 3 Sent Done Done false
14138 4041 Acked Read 3 Sent Done Done false
14139 4042 Acked Write 3 Sent Done Done false
14140 4043 Acked Flush 0 Sent Done Done false
14141 4044 Acked Write 8 Sent Done Done false
14142 4045 Acked Write 5 Sent Done Done false
14143 4046 Acked Read 9 Sent Done Done false
14144 4047 Acked Read 6 Sent Done Done false
14145 4048 Acked Read 7 Sent Done Done false
14146 4049 Acked Read 5 Sent Done Done false
14147 4050 Acked Write 9 Sent Done Done false
14148 4051 Acked Read 6 Sent Done Done false
14149 4052 Acked Read 8 Sent Done Done false
14150 4053 Acked Write 10 Sent Done Done false
14151 4054 Acked Write 3 Sent Done Done false
14152 4055 Acked Write 4 Sent Done Done false
14153 4056 Acked Write 1 Sent Done Done false
14154 4057 Acked Read 1 Sent Done Done false
14155 4058 Acked Read 9 Sent Done Done false
14156 4059 Acked Read 4 Sent Done Done false
14157 4060 Acked Write 3 Sent Done Done false
14158 4061 Acked Read 3 Sent Done Done false
14159 4062 Acked Write 4 Sent Done Done false
14160 4063 Acked Read 7 Sent Done Done false
14161 4064 Acked Read 6 Sent Done Done false
14162 4065 Acked Read 1 Sent Done Done false
14163 4066 Acked Read 10 Sent Done Done false
14164 4067 Acked Read 9 Sent Done Done false
14165 4068 Acked Write 10 Sent Done Done false
14166 4069 Acked Read 3 Sent Done Done false
14167 4070 Acked Flush 0 Sent Done Done false
14168 4071 Acked Read 6 Sent Done Done false
14169 4072 Acked Read 8 Sent Done Done false
14170 4073 Acked Read 10 Sent Done Done false
14171 4074 Acked Write 3 Sent Done Done false
14172 4075 Acked Write 9 Sent Done Done false
14173 4076 Acked Write 7 Sent Done Done false
14174 4077 Acked Write 8 Sent Done Done false
14175 4078 Acked Read 2 Sent Done Done false
14176 4079 Acked Write 10 Sent Done Done false
14177 4080 Acked Flush 0 Sent Done Done false
14178 4081 Acked Read 3 Sent Done Done false
14179 4082 Acked Write 8 Sent Done Done false
14180 4083 Acked Read 7 Sent Done Done false
14181 4084 Acked Read 1 Sent Done Done false
14182 4085 Acked Read 6 Sent Done Done false
14183 4086 Acked Read 4 Sent Done Done false
14184 4087 Acked Read 7 Sent Done Done false
14185 4088 Acked Read 4 Sent Done Done false
14186 4089 Acked Write 5 Sent Done Done false
14187 4090 Acked Write 4 Sent Done Done false
14188 4091 Acked Read 2 Sent Done Done false
14189 4092 Acked Read 8 Sent Done Done false
14190 4093 Acked Read 9 Sent Done Done false
14191 4094 Acked Write 7 Sent Done Done false
14192 4095 Acked Write 10 Sent Done Done false
14193 4096 Acked Read 1 Sent Done Done false
14194 4097 Acked Read 6 Sent Done Done false
14195 4098 Acked Write 8 Sent Done Done false
14196 4099 Acked Write 2 Sent Done Done false
14197 4100 Acked Write 4 Sent Done Done false
14198 4101 Acked Read 3 Sent Done Done false
14199 4102 Acked Write 1 Sent Done Done false
14200 4103 Acked Read 10 Sent Done Done false
14201 4104 Acked Read 1 Sent Done Done false
14202 4105 Acked Write 10 Sent Done Done false
14203 4106 Acked Write 2 Sent Done Done false
14204 4107 Acked Read 7 Sent Done Done false
14205 4108 Acked Write 10 Sent Done Done false
14206 4109 Acked Read 9 Sent Done Done false
14207 4110 Acked Read 9 Sent Done Done false
14208 4111 Acked Write 6 Sent Done Done false
14209 4112 Acked Flush 0 Sent Done Done false
14210 4113 Acked Write 5 Sent Done Done false
14211 4114 Acked Read 9 Sent Done Done false
14212 4115 Acked Flush 0 Sent Done Done false
14213 4116 Acked Read 2 Sent Done Done false
14214 4117 Acked Write 3 Sent Done Done false
14215 4118 Acked Read 3 Sent Done Done false
14216 4119 Acked Write 6 Sent Done Done false
14217 4120 Acked Write 9 Sent Done Done false
14218 4121 Acked Read 2 Sent Done Done false
14219 4122 Acked Write 9 Sent Done Done false
14220 4123 Acked Write 1 Sent Done Done false
14221 4124 Acked Write 7 Sent Done Done false
14222 4125 Acked Read 8 Sent Done Done false
14223 4126 Acked Read 2 Sent Done Done false
14224 4127 Acked Read 3 Sent Done Done false
14225 4128 Acked Read 10 Sent Done Done false
14226 4129 Acked Write 2 Sent Done Done false
14227 4130 Acked Read 4 Sent Done Done false
14228 4131 Acked Write 7 Sent Done Done false
14229 4132 Acked Write 5 Sent Done Done false
14230 4133 Acked Read 6 Sent Done Done false
14231 4134 Acked Read 2 Sent Done Done false
14232 4135 Acked Read 8 Sent Done Done false
14233 4136 Acked Write 2 Sent Done Done false
14234 4137 Acked Read 8 Sent Done Done false
14235 4138 Acked Read 2 Sent Done Done false
14236 4139 Acked Read 5 Sent Done Done false
14237 4140 Acked Flush 0 Sent Done Done false
14238 4141 Acked Write 5 Sent Done Done false
14239 4142 Acked Read 10 Sent Done Done false
14240 4143 Acked Read 8 Sent Done Done false
14241 4144 Acked Write 2 Sent Done Done false
14242 4145 Acked Read 3 Sent Done Done false
14243 4146 Acked Write 10 Sent Done Done false
14244 4147 Acked Read 2 Sent Done Done false
14245 4148 Acked Read 5 Sent Done Done false
14246 4149 Acked Write 8 Sent Done Done false
14247 4150 Acked Write 6 Sent Done Done false
14248 4151 Acked Flush 0 Sent Done Done false
14249 4152 Acked Write 9 Sent Done Done false
14250 4153 Acked Read 4 Sent Done Done false
14251 4154 Acked Read 3 Sent Done Done false
14252 4155 Acked Write 2 Sent Done Done false
14253 4156 Acked Write 5 Sent Done Done false
14254 4157 Acked Write 6 Sent Done Done false
14255 4158 Acked Write 7 Sent Done Done false
14256 4159 Acked Write 3 Sent Done Done false
14257 4160 Acked Read 2 Sent Done Done false
14258 4161 Acked Write 8 Sent Done Done false
14259 4162 Acked Write 9 Sent Done Done false
14260 4163 Acked Read 4 Sent Done Done false
14261 4164 Acked Write 6 Sent Done Done false
14262 4165 Acked Read 10 Sent Done Done false
14263 4166 Acked Write 4 Sent Done Done false
14264 4167 Acked Write 8 Sent Done Done false
14265 4168 Acked Read 8 Sent Done Done false
14266 4169 Acked Write 2 Sent Done Done false
14267 4170 Acked Read 4 Sent Done Done false
14268 4171 Acked Read 3 Sent Done Done false
14269 4172 Acked Read 9 Sent Done Done false
14270 4173 Acked Write 1 Sent Done Done false
14271 4174 Acked Read 2 Sent Done Done false
14272 4175 Acked Read 4 Sent Done Done false
14273 4176 Acked Read 9 Sent Done Done false
14274 4177 Acked Read 8 Sent Done Done false
14275 4178 Acked Write 5 Sent Done Done false
14276 4179 Acked Read 7 Sent Done Done false
14277 4180 Acked Write 6 Sent Done Done false
14278 4181 Acked Write 3 Sent Done Done false
14279 4182 Acked Read 7 Sent Done Done false
14280 4183 Acked Flush 0 Sent Done Done false
14281 4184 Acked Write 10 Sent Done Done false
14282 4185 Acked Read 8 Sent Done Done false
14283 4186 Acked Write 2 Sent Done Done false
14284 4187 Acked Read 5 Sent Done Done false
14285 4188 Acked Read 5 Sent Done Done false
14286 4189 Acked Read 6 Sent Done Done false
14287 4190 Acked Read 4 Sent Done Done false
14288 4191 Acked Read 7 Sent Done Done false
14289 4192 Acked Read 4 Sent Done Done false
14290 4193 Acked Read 10 Sent Done Done false
14291 4194 Acked Read 4 Sent Done Done false
14292 4195 Acked Read 7 Sent Done Done false
14293 4196 Acked Write 2 Sent Done Done false
14294 4197 Acked Read 1 Sent Done Done false
14295 4198 Acked Read 1 Sent Done Done false
14296 4199 Acked Read 10 Sent Done Done false
14297 4200 Acked Read 5 Sent Done Done false
14298 4201 Acked Read 4 Sent Done Done false
14299 4202 Acked Write 8 Sent Done Done false
14300 4203 Acked Flush 0 Sent Done Done false
14301 4204 Acked Flush 0 Sent Done Done false
14302 4205 Acked Write 3 Sent Done Done false
14303 4206 Acked Read 6 Sent Done Done false
14304 4207 Acked Read 4 Sent Done Done false
14305 4208 Acked Read 2 Sent Done Done false
14306 4209 Acked Read 9 Sent Done Done false
14307 4210 Acked Read 4 Sent Done Done false
14308 4211 Acked Write 4 Sent Done Done false
14309 4212 Acked Write 7 Sent Done Done false
14310 4213 Acked Read 1 Sent Done Done false
14311 4214 Acked Write 6 Sent Done Done false
14312 4215 Acked Write 5 Sent Done Done false
14313 4216 Acked Write 9 Sent Done Done false
14314 4217 Acked Write 5 Sent Done Done false
14315 4218 Acked Flush 0 Sent Done Done false
14316 4219 Acked Read 9 Sent Done Done false
14317 4220 Acked Write 3 Sent Done Done false
14318 4221 Acked Write 2 Sent Done Done false
14319 4222 Acked Read 2 Sent Done Done false
14320 4223 Acked Write 5 Sent Done Done false
14321 4224 Acked Write 9 Sent Done Done false
14322 4225 Acked Write 9 Sent Done Done false
14323 4226 Acked Read 5 Sent Done Done false
14324 4227 Acked Write 10 Sent Done Done false
14325 4228 Acked Read 1 Sent Done Done false
14326 4229 Acked Read 3 Sent Done Done false
14327 4230 Acked Read 4 Sent Done Done false
14328 4231 Acked Read 5 Sent Done Done false
14329 4232 Acked Read 9 Sent Done Done false
14330 4233 Acked Read 7 Sent Done Done false
14331 4234 Acked Read 3 Sent Done Done false
14332 4235 Acked Read 9 Sent Done Done false
14333 4236 Acked Read 4 Sent Done Done false
14334 4237 Acked Read 4 Sent Done Done false
14335 4238 Acked Read 9 Sent Done Done false
14336 4239 Acked Write 8 Sent Done Done false
14337 4240 Acked Read 2 Sent Done Done false
14338 4241 Acked Read 1 Sent Done Done false
14339 4242 Acked Write 8 Sent Done Done false
14340 4243 Acked Read 3 Sent Done Done false
14341 4244 Acked Read 5 Sent Done Done false
14342 4245 Acked Read 9 Sent Done Done false
14343 4246 Acked Flush 0 Sent Done Done false
14344 4247 Acked Write 1 Sent Done Done false
14345 4248 Acked Write 1 Sent Done Done false
14346 4249 Acked Flush 0 Sent Done Done false
14347 4250 Acked Read 8 Sent Done Done false
14348 4251 Acked Read 7 Sent Done Done false
14349 4252 Acked Read 8 Sent Done Done false
14350 4253 Acked Read 9 Sent Done Done false
14351 4254 Acked Write 8 Sent Done Done false
14352 4255 Acked Read 6 Sent Done Done false
14353 4256 Acked Read 1 Sent Done Done false
14354 4257 Acked Write 4 Sent Done Done false
14355 4258 Acked Read 6 Sent Done Done false
14356 4259 Acked Read 3 Sent Done Done false
14357 4260 Acked Flush 0 Sent Done Done false
14358 4261 Acked Write 10 Sent Done Done false
14359 4262 Acked Flush 0 Sent Done Done false
14360 4263 Acked Write 10 Sent Done Done false
14361 4264 Acked Write 6 Sent Done Done false
14362 4265 Acked Read 5 Sent Done Done false
14363 4266 Acked Read 1 Sent Done Done false
14364 4267 Acked Write 5 Sent Done Done false
14365 4268 Acked Write 3 Sent Done Done false
14366 4269 Acked Read 2 Sent Done Done false
14367 4270 Acked Write 8 Sent Done Done false
14368 4271 Acked Write 3 Sent Done Done false
14369 4272 Acked Read 9 Sent Done Done false
14370 4273 Acked Write 2 Sent Done Done false
14371 4274 Acked Read 5 Sent Done Done false
14372 4275 Acked Read 8 Sent Done Done false
14373 4276 Acked Write 2 Sent Done Done false
14374 4277 Acked Read 9 Sent Done Done false
14375 4278 Acked Read 3 Sent Done Done false
14376 4279 Acked Read 2 Sent Done Done false
14377 4280 Acked Read 4 Sent Done Done false
14378 4281 Acked Write 10 Sent Done Done false
14379 4282 Acked Read 2 Sent Done Done false
14380 4283 Acked Read 5 Sent Done Done false
14381 4284 Acked Write 7 Sent Done Done false
14382 4285 Acked Flush 0 Sent Done Done false
14383 4286 Acked Flush 0 Sent Done Done false
14384 4287 Acked Write 1 Sent Done Done false
14385 4288 Acked Read 9 Sent Done Done false
14386 4289 Acked Read 5 Sent Done Done false
14387 4290 Acked Read 4 Sent Done Done false
14388 4291 Acked Write 1 Sent Done Done false
14389 4292 Acked Flush 0 Sent Done Done false
14390 4293 Acked Read 7 Sent Done Done false
14391 4294 Acked Read 5 Sent Done Done false
14392 4295 Acked Flush 0 Sent Done Done false
14393 4296 Acked Write 9 Sent Done Done false
14394 4297 Acked Read 9 Sent Done Done false
14395 4298 Acked Write 4 Sent Done Done false
14396 4299 Acked Write 10 Sent Done Done false
14397 4300 Acked Read 1 Sent Done Done false
14398 4301 Acked Write 2 Sent Done Done false
14399 4302 Acked Read 5 Sent Done Done false
14400 4303 Acked Write 7 Sent Done Done false
14401 4304 Acked Write 10 Sent Done Done false
14402 4305 Acked Write 5 Sent Done Done false
14403 4306 Acked Read 1 Sent Done Done false
14404 4307 Acked Flush 0 Sent Done Done false
14405 4308 Acked Flush 0 Sent Done Done false
14406 4309 Acked Write 2 Sent Done Done false
14407 4310 Acked Write 7 Sent Done Done false
14408 4311 Acked Write 8 Sent Done Done false
14409 4312 Acked Write 10 Sent Done Done false
14410 4313 Acked Flush 0 Sent Done Done false
14411 4314 Acked Read 3 Sent Done Done false
14412 4315 Acked Write 5 Sent Done Done false
14413 4316 Acked Read 2 Sent Done Done false
14414 4317 Acked Read 1 Sent Done Done false
14415 4318 Acked Write 10 Sent Done Done false
14416 4319 Acked Flush 0 Sent Done Done false
14417 STATES DS:0 DS:1 DS:2 TOTAL
14418 Sent 302 0 0 302
14419 Done 0 302 302 604
14420 Skipped 0 0 0 0
14421 Error 0 0 0 0
14422 Last Flush: 4017 4319 4319
14423 Downstairs last five completed: 4017 4016 4015 4014 4013
14424 Upstairs last five completed: 4319 4318 4317 4316 4315
14425 CLIENT: Up:0 ds:302 act:2
14426 ----------------------------------------------------------------
14427 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
14428 States: Offline Active Active
14429 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
14430 4018 Acked Write 7 Sent Done Done false
14431 4019 Acked Write 2 Sent Done Done false
14432 4020 Acked Read 10 Sent Done Done false
14433 4021 Acked Read 9 Sent Done Done false
14434 4022 Acked Write 6 Sent Done Done false
14435 4023 Acked Write 3 Sent Done Done false
14436 4024 Acked Read 9 Sent Done Done false
14437 4025 Acked Write 6 Sent Done Done false
14438 4026 Acked Read 10 Sent Done Done false
14439 4027 Acked Read 2 Sent Done Done false
14440 4028 Acked Flush 0 Sent Done Done false
14441 4029 Acked Read 2 Sent Done Done false
14442 4030 Acked Write 5 Sent Done Done false
14443 4031 Acked Flush 0 Sent Done Done false
14444 4032 Acked Read 7 Sent Done Done false
14445 4033 Acked Write 3 Sent Done Done false
14446 4034 Acked Write 5 Sent Done Done false
14447 4035 Acked Write 6 Sent Done Done false
14448 4036 Acked Read 3 Sent Done Done false
14449 4037 Acked Read 5 Sent Done Done false
14450 4038 Acked Write 10 Sent Done Done false
14451 4039 Acked Write 8 Sent Done Done false
14452 4040 Acked Write 3 Sent Done Done false
14453 4041 Acked Read 3 Sent Done Done false
14454 4042 Acked Write 3 Sent Done Done false
14455 4043 Acked Flush 0 Sent Done Done false
14456 4044 Acked Write 8 Sent Done Done false
14457 4045 Acked Write 5 Sent Done Done false
14458 4046 Acked Read 9 Sent Done Done false
14459 4047 Acked Read 6 Sent Done Done false
14460 4048 Acked Read 7 Sent Done Done false
14461 4049 Acked Read 5 Sent Done Done false
14462 4050 Acked Write 9 Sent Done Done false
14463 4051 Acked Read 6 Sent Done Done false
14464 4052 Acked Read 8 Sent Done Done false
14465 4053 Acked Write 10 Sent Done Done false
14466 4054 Acked Write 3 Sent Done Done false
14467 4055 Acked Write 4 Sent Done Done false
14468 4056 Acked Write 1 Sent Done Done false
14469 4057 Acked Read 1 Sent Done Done false
14470 4058 Acked Read 9 Sent Done Done false
14471 4059 Acked Read 4 Sent Done Done false
14472 4060 Acked Write 3 Sent Done Done false
14473 4061 Acked Read 3 Sent Done Done false
14474 4062 Acked Write 4 Sent Done Done false
14475 4063 Acked Read 7 Sent Done Done false
14476 4064 Acked Read 6 Sent Done Done false
14477 4065 Acked Read 1 Sent Done Done false
14478 4066 Acked Read 10 Sent Done Done false
14479 4067 Acked Read 9 Sent Done Done false
14480 4068 Acked Write 10 Sent Done Done false
14481 4069 Acked Read 3 Sent Done Done false
14482 4070 Acked Flush 0 Sent Done Done false
14483 4071 Acked Read 6 Sent Done Done false
14484 4072 Acked Read 8 Sent Done Done false
14485 4073 Acked Read 10 Sent Done Done false
14486 4074 Acked Write 3 Sent Done Done false
14487 4075 Acked Write 9 Sent Done Done false
14488 4076 Acked Write 7 Sent Done Done false
14489 4077 Acked Write 8 Sent Done Done false
14490 4078 Acked Read 2 Sent Done Done false
14491 4079 Acked Write 10 Sent Done Done false
14492 4080 Acked Flush 0 Sent Done Done false
14493 4081 Acked Read 3 Sent Done Done false
14494 4082 Acked Write 8 Sent Done Done false
14495 4083 Acked Read 7 Sent Done Done false
14496 4084 Acked Read 1 Sent Done Done false
14497 4085 Acked Read 6 Sent Done Done false
14498 4086 Acked Read 4 Sent Done Done false
14499 4087 Acked Read 7 Sent Done Done false
14500 4088 Acked Read 4 Sent Done Done false
14501 4089 Acked Write 5 Sent Done Done false
14502 4090 Acked Write 4 Sent Done Done false
14503 4091 Acked Read 2 Sent Done Done false
14504 4092 Acked Read 8 Sent Done Done false
14505 4093 Acked Read 9 Sent Done Done false
14506 4094 Acked Write 7 Sent Done Done false
14507 4095 Acked Write 10 Sent Done Done false
14508 4096 Acked Read 1 Sent Done Done false
14509 4097 Acked Read 6 Sent Done Done false
14510 4098 Acked Write 8 Sent Done Done false
14511 4099 Acked Write 2 Sent Done Done false
14512 4100 Acked Write 4 Sent Done Done false
14513 4101 Acked Read 3 Sent Done Done false
14514 4102 Acked Write 1 Sent Done Done false
14515 4103 Acked Read 10 Sent Done Done false
14516 4104 Acked Read 1 Sent Done Done false
14517 4105 Acked Write 10 Sent Done Done false
14518 4106 Acked Write 2 Sent Done Done false
14519 4107 Acked Read 7 Sent Done Done false
14520 4108 Acked Write 10 Sent Done Done false
14521 4109 Acked Read 9 Sent Done Done false
14522 4110 Acked Read 9 Sent Done Done false
14523 4111 Acked Write 6 Sent Done Done false
14524 4112 Acked Flush 0 Sent Done Done false
14525 4113 Acked Write 5 Sent Done Done false
14526 4114 Acked Read 9 Sent Done Done false
14527 4115 Acked Flush 0 Sent Done Done false
14528 4116 Acked Read 2 Sent Done Done false
14529 4117 Acked Write 3 Sent Done Done false
14530 4118 Acked Read 3 Sent Done Done false
14531 4119 Acked Write 6 Sent Done Done false
14532 4120 Acked Write 9 Sent Done Done false
14533 4121 Acked Read 2 Sent Done Done false
14534 4122 Acked Write 9 Sent Done Done false
14535 4123 Acked Write 1 Sent Done Done false
14536 4124 Acked Write 7 Sent Done Done false
14537 4125 Acked Read 8 Sent Done Done false
14538 4126 Acked Read 2 Sent Done Done false
14539 4127 Acked Read 3 Sent Done Done false
14540 4128 Acked Read 10 Sent Done Done false
14541 4129 Acked Write 2 Sent Done Done false
14542 4130 Acked Read 4 Sent Done Done false
14543 4131 Acked Write 7 Sent Done Done false
14544 4132 Acked Write 5 Sent Done Done false
14545 4133 Acked Read 6 Sent Done Done false
14546 4134 Acked Read 2 Sent Done Done false
14547 4135 Acked Read 8 Sent Done Done false
14548 4136 Acked Write 2 Sent Done Done false
14549 4137 Acked Read 8 Sent Done Done false
14550 4138 Acked Read 2 Sent Done Done false
14551 4139 Acked Read 5 Sent Done Done false
14552 4140 Acked Flush 0 Sent Done Done false
14553 4141 Acked Write 5 Sent Done Done false
14554 4142 Acked Read 10 Sent Done Done false
14555 4143 Acked Read 8 Sent Done Done false
14556 4144 Acked Write 2 Sent Done Done false
14557 4145 Acked Read 3 Sent Done Done false
14558 4146 Acked Write 10 Sent Done Done false
14559 4147 Acked Read 2 Sent Done Done false
14560 4148 Acked Read 5 Sent Done Done false
14561 4149 Acked Write 8 Sent Done Done false
14562 4150 Acked Write 6 Sent Done Done false
14563 4151 Acked Flush 0 Sent Done Done false
14564 4152 Acked Write 9 Sent Done Done false
14565 4153 Acked Read 4 Sent Done Done false
14566 4154 Acked Read 3 Sent Done Done false
14567 4155 Acked Write 2 Sent Done Done false
14568 4156 Acked Write 5 Sent Done Done false
14569 4157 Acked Write 6 Sent Done Done false
14570 4158 Acked Write 7 Sent Done Done false
14571 4159 Acked Write 3 Sent Done Done false
14572 4160 Acked Read 2 Sent Done Done false
14573 4161 Acked Write 8 Sent Done Done false
14574 4162 Acked Write 9 Sent Done Done false
14575 4163 Acked Read 4 Sent Done Done false
14576 4164 Acked Write 6 Sent Done Done false
14577 4165 Acked Read 10 Sent Done Done false
14578 4166 Acked Write 4 Sent Done Done false
14579 4167 Acked Write 8 Sent Done Done false
14580 4168 Acked Read 8 Sent Done Done false
14581 4169 Acked Write 2 Sent Done Done false
14582 4170 Acked Read 4 Sent Done Done false
14583 4171 Acked Read 3 Sent Done Done false
14584 4172 Acked Read 9 Sent Done Done false
14585 4173 Acked Write 1 Sent Done Done false
14586 4174 Acked Read 2 Sent Done Done false
14587 4175 Acked Read 4 Sent Done Done false
14588 4176 Acked Read 9 Sent Done Done false
14589 4177 Acked Read 8 Sent Done Done false
14590 4178 Acked Write 5 Sent Done Done false
14591 4179 Acked Read 7 Sent Done Done false
14592 4180 Acked Write 6 Sent Done Done false
14593 4181 Acked Write 3 Sent Done Done false
14594 4182 Acked Read 7 Sent Done Done false
14595 4183 Acked Flush 0 Sent Done Done false
14596 4184 Acked Write 10 Sent Done Done false
14597 4185 Acked Read 8 Sent Done Done false
14598 4186 Acked Write 2 Sent Done Done false
14599 4187 Acked Read 5 Sent Done Done false
14600 4188 Acked Read 5 Sent Done Done false
14601 4189 Acked Read 6 Sent Done Done false
14602 4190 Acked Read 4 Sent Done Done false
14603 4191 Acked Read 7 Sent Done Done false
14604 4192 Acked Read 4 Sent Done Done false
14605 4193 Acked Read 10 Sent Done Done false
14606 4194 Acked Read 4 Sent Done Done false
14607 4195 Acked Read 7 Sent Done Done false
14608 4196 Acked Write 2 Sent Done Done false
14609 4197 Acked Read 1 Sent Done Done false
14610 4198 Acked Read 1 Sent Done Done false
14611 4199 Acked Read 10 Sent Done Done false
14612 4200 Acked Read 5 Sent Done Done false
14613 4201 Acked Read 4 Sent Done Done false
14614 4202 Acked Write 8 Sent Done Done false
14615 4203 Acked Flush 0 Sent Done Done false
14616 4204 Acked Flush 0 Sent Done Done false
14617 4205 Acked Write 3 Sent Done Done false
14618 4206 Acked Read 6 Sent Done Done false
14619 4207 Acked Read 4 Sent Done Done false
14620 4208 Acked Read 2 Sent Done Done false
14621 4209 Acked Read 9 Sent Done Done false
14622 4210 Acked Read 4 Sent Done Done false
14623 4211 Acked Write 4 Sent Done Done false
14624 4212 Acked Write 7 Sent Done Done false
14625 4213 Acked Read 1 Sent Done Done false
14626 4214 Acked Write 6 Sent Done Done false
14627 4215 Acked Write 5 Sent Done Done false
14628 4216 Acked Write 9 Sent Done Done false
14629 4217 Acked Write 5 Sent Done Done false
14630 4218 Acked Flush 0 Sent Done Done false
14631 4219 Acked Read 9 Sent Done Done false
14632 4220 Acked Write 3 Sent Done Done false
14633 4221 Acked Write 2 Sent Done Done false
14634 4222 Acked Read 2 Sent Done Done false
14635 4223 Acked Write 5 Sent Done Done false
14636 4224 Acked Write 9 Sent Done Done false
14637 4225 Acked Write 9 Sent Done Done false
14638 4226 Acked Read 5 Sent Done Done false
14639 4227 Acked Write 10 Sent Done Done false
14640 4228 Acked Read 1 Sent Done Done false
14641 4229 Acked Read 3 Sent Done Done false
14642 4230 Acked Read 4 Sent Done Done false
14643 4231 Acked Read 5 Sent Done Done false
14644 4232 Acked Read 9 Sent Done Done false
14645 4233 Acked Read 7 Sent Done Done false
14646 4234 Acked Read 3 Sent Done Done false
14647 4235 Acked Read 9 Sent Done Done false
14648 4236 Acked Read 4 Sent Done Done false
14649 4237 Acked Read 4 Sent Done Done false
14650 4238 Acked Read 9 Sent Done Done false
14651 4239 Acked Write 8 Sent Done Done false
14652 4240 Acked Read 2 Sent Done Done false
14653 4241 Acked Read 1 Sent Done Done false
14654 4242 Acked Write 8 Sent Done Done false
14655 4243 Acked Read 3 Sent Done Done false
14656 4244 Acked Read 5 Sent Done Done false
14657 4245 Acked Read 9 Sent Done Done false
14658 4246 Acked Flush 0 Sent Done Done false
14659 4247 Acked Write 1 Sent Done Done false
14660 4248 Acked Write 1 Sent Done Done false
14661 4249 Acked Flush 0 Sent Done Done false
14662 4250 Acked Read 8 Sent Done Done false
14663 4251 Acked Read 7 Sent Done Done false
14664 4252 Acked Read 8 Sent Done Done false
14665 4253 Acked Read 9 Sent Done Done false
14666 4254 Acked Write 8 Sent Done Done false
14667 4255 Acked Read 6 Sent Done Done false
14668 4256 Acked Read 1 Sent Done Done false
14669 4257 Acked Write 4 Sent Done Done false
14670 4258 Acked Read 6 Sent Done Done false
14671 4259 Acked Read 3 Sent Done Done false
14672 4260 Acked Flush 0 Sent Done Done false
14673 4261 Acked Write 10 Sent Done Done false
14674 4262 Acked Flush 0 Sent Done Done false
14675 4263 Acked Write 10 Sent Done Done false
14676 4264 Acked Write 6 Sent Done Done false
14677 4265 Acked Read 5 Sent Done Done false
14678 4266 Acked Read 1 Sent Done Done false
14679 4267 Acked Write 5 Sent Done Done false
14680 4268 Acked Write 3 Sent Done Done false
14681 4269 Acked Read 2 Sent Done Done false
14682 4270 Acked Write 8 Sent Done Done false
14683 4271 Acked Write 3 Sent Done Done false
14684 4272 Acked Read 9 Sent Done Done false
14685 4273 Acked Write 2 Sent Done Done false
14686 4274 Acked Read 5 Sent Done Done false
14687 4275 Acked Read 8 Sent Done Done false
14688 4276 Acked Write 2 Sent Done Done false
14689 4277 Acked Read 9 Sent Done Done false
14690 4278 Acked Read 3 Sent Done Done false
14691 4279 Acked Read 2 Sent Done Done false
14692 4280 Acked Read 4 Sent Done Done false
14693 4281 Acked Write 10 Sent Done Done false
14694 4282 Acked Read 2 Sent Done Done false
14695 4283 Acked Read 5 Sent Done Done false
14696 4284 Acked Write 7 Sent Done Done false
14697 4285 Acked Flush 0 Sent Done Done false
14698 4286 Acked Flush 0 Sent Done Done false
14699 4287 Acked Write 1 Sent Done Done false
14700 4288 Acked Read 9 Sent Done Done false
14701 4289 Acked Read 5 Sent Done Done false
14702 4290 Acked Read 4 Sent Done Done false
14703 4291 Acked Write 1 Sent Done Done false
14704 4292 Acked Flush 0 Sent Done Done false
14705 4293 Acked Read 7 Sent Done Done false
14706 4294 Acked Read 5 Sent Done Done false
14707 4295 Acked Flush 0 Sent Done Done false
14708 4296 Acked Write 9 Sent Done Done false
14709 4297 Acked Read 9 Sent Done Done false
14710 4298 Acked Write 4 Sent Done Done false
14711 4299 Acked Write 10 Sent Done Done false
14712 4300 Acked Read 1 Sent Done Done false
14713 4301 Acked Write 2 Sent Done Done false
14714 4302 Acked Read 5 Sent Done Done false
14715 4303 Acked Write 7 Sent Done Done false
14716 4304 Acked Write 10 Sent Done Done false
14717 4305 Acked Write 5 Sent Done Done false
14718 4306 Acked Read 1 Sent Done Done false
14719 4307 Acked Flush 0 Sent Done Done false
14720 4308 Acked Flush 0 Sent Done Done false
14721 4309 Acked Write 2 Sent Done Done false
14722 4310 Acked Write 7 Sent Done Done false
14723 4311 Acked Write 8 Sent Done Done false
14724 4312 Acked Write 10 Sent Done Done false
14725 4313 Acked Flush 0 Sent Done Done false
14726 4314 Acked Read 3 Sent Done Done false
14727 4315 Acked Write 5 Sent Done Done false
14728 4316 Acked Read 2 Sent Done Done false
14729 4317 Acked Read 1 Sent Done Done false
14730 4318 Acked Write 10 Sent Done Done false
14731 4319 Acked Flush 0 Sent Done Done false
14732 STATES DS:0 DS:1 DS:2 TOTAL
14733 Sent 302 0 0 302
14734 Done 0 302 302 604
14735 Skipped 0 0 0 0
14736 Error 0 0 0 0
14737 Last Flush: 4017 4319 4319
14738 Downstairs last five completed: 4017 4016 4015 4014 4013
14739 Upstairs last five completed: 4319 4318 4317 4316 4315
14740 CLIENT: Up:0 ds:302 act:2
147412024-11-15T02:00:34.347ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
147422024-11-15T02:00:34.348ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:52011) = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
147432024-11-15T02:00:34.349ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
147442024-11-15T02:00:34.349ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
147452024-11-15T02:00:34.349ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
147462024-11-15T02:00:34.349ZINFOcrucible: send last flush ID to this DS: 4017 = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
147472024-11-15T02:00:34.350ZINFOcrucible: Replied this last flush ID: 4017 = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
147482024-11-15T02:00:34.350ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
147492024-11-15T02:00:34.350ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
147502024-11-15T02:00:34.350ZINFOcrucible: [0] client re-new 302 jobs since flush 4017 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
147512024-11-15T02:00:34.350ZINFOcrucible: [0] Replayed 302 jobs since flush: 4017 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
14752 ----------------------------------------------------------------
14753 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
14754 Downstairs last five completed: 4319 4318 4317 4316 4315
14755 Upstairs last five completed: 4319 4318 4317 4316 4315
14756 CLIENT: Up:0 ds:0 act:3
14757 Replay: All jobs finished
14758 Main task has work to do, go find it
14759 got dsc Stop(0)
14760 stop 0
14761 Nov 15 02:00:34.592 INFO [8810] Got stop action so:false kr:false
14762 request completed, latency_us: 364, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 68bf45da-23c7-4cbf-b21a-4e2f05b3a803, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
147632024-11-15T02:00:34.593ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
147642024-11-15T02:00:34.593ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
147652024-11-15T02:00:34.593ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
147662024-11-15T02:00:34.593ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
147672024-11-15T02:00:34.593ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
147682024-11-15T02:00:34.593ZINFOcrucible: client task is exiting = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
14769 Nov 15 02:00:34.593 INFO request completed, latency_us: 129, response_code: 200, uri: /state/cid/0, method: GET, req_id: 43601706-0f4d-4fe6-8d14-fd4c0aa60e49, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
14770 [8810] Exited with: ExitStatus(unix_wait_status(9))
14771 [8810][0] reports Exit
14772 Nov 15 02:00:38.596 INFO request completed, latency_us: 344, response_code: 200, uri: /state/cid/0, method: GET, req_id: 11df7ceb-4338-421b-917e-8de14bb9e7e9, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
14773 001/300 Write block 342 len 16384 data: 2 2 2 2
14774 002/300 Flush
14775 003/300 Write block 306 len 36864 data: 4 4 3 4 4 4 4 4 4
14776 004/300 Read block 1650 len 40960
14777 005/300 Write block 2020 len 8192 data: 3 2
14778 006/300 Read block 2835 len 4096
14779 007/300 Read block 1387 len 36864
14780 008/300 Write block 1365 len 8192 data: 4 4
14781 009/300 Write block 3092 len 40960 data: 3 3 3 2 2 2 2 2 2 2
14782 010/300 Write block 309 len 12288 data: 5 5 5
14783 011/300 Read block 1788 len 16384
14784 012/300 Write block 3303 len 20480 data: 3 3 3 3 3
14785 013/300 Read block 2577 len 12288
14786 014/300 Flush
14787 015/300 Read block 2671 len 16384
14788 016/300 Write block 3409 len 8192 data: 6 3
14789 017/300 Write block 1778 len 32768 data: 5 5 5 5 6 6 4 3
14790 018/300 Read block 3757 len 24576
14791 019/300 Read block 703 len 24576
14792 020/300 Read block 2819 len 8192
14793 021/300 Read block 2511 len 40960
14794 022/300 Read block 897 len 16384
14795 023/300 Read block 2732 len 20480
14796 024/300 Flush
14797 025/300 Write block 3894 len 36864 data: 4 4 3 5 7 7 7 8 8
14798 026/300 Read block 2794 len 20480
14799 027/300 Write block 1539 len 4096 data: 6
14800 028/300 Write block 3170 len 8192 data: 4 4
14801 029/300 Write block 535 len 4096 data: 4
14802 030/300 Write block 4528 len 40960 data: 7 7 7 5 5 5 3 3 3 2
14803 031/300 Write block 3708 len 36864 data: 3 4 3 3 3 2 2 2 2
14804 032/300 Read block 4076 len 16384
14805 033/300 Write block 1350 len 40960 data: 2 2 2 2 2 2 2 2 2 2
14806 034/300 Write block 4180 len 28672 data: 5 3 3 3 4 4 3
14807 035/300 Read block 4218 len 36864
14808 036/300 Read block 4596 len 20480
14809 037/300 Write block 2241 len 20480 data: 4 4 3 3 3
14810 038/300 Read block 3339 len 32768
14811 039/300 Read block 264 len 20480
14812 040/300 Flush
14813 041/300 Write block 3107 len 16384 data: 2 2 2 3
14814 042/300 Write block 930 len 32768 data: 4 4 3 3 2 2 2 3
14815 043/300 Read block 2190 len 4096
14816 044/300 Read block 445 len 40960
14817 045/300 Read block 357 len 16384
14818 046/300 Write block 3410 len 32768 data: 4 3 4 4 4 5 5 4
14819 047/300 Write block 1002 len 16384 data: 4 4 3 3
14820 048/300 Read block 343 len 12288
14821 049/300 Write block 4191 len 36864 data: 4 4 4 5 7 7 7 5 3
14822 050/300 Read block 177 len 36864
14823 051/300 Write block 3256 len 36864 data: 5 6 6 4 4 5 4 3 3
14824 052/300 Write block 991 len 20480 data: 5 5 5 5 5
14825 053/300 Read block 4558 len 20480
14826 054/300 Write block 877 len 4096 data: 2
14827 055/300 Read block 3183 len 4096
14828 056/300 Write block 3392 len 20480 data: 3 3 3 3 3
14829 057/300 Read block 1663 len 8192
14830 058/300 Flush
14831 059/300 Read block 717 len 20480
14832 060/300 Read block 2051 len 32768
14833 061/300 Read block 4608 len 12288
14834 062/300 Read block 417 len 28672
14835 063/300 Read block 1019 len 40960
14836 064/300 Write block 3870 len 8192 data: 5 5
14837 065/300 Write block 3985 len 28672 data: 2 2 3 3 3 3 2
14838 066/300 Write block 79 len 28672 data: 7 7 7 6 5 5 5
14839 067/300 Write block 3736 len 12288 data: 4 5 5
14840 068/300 Read block 3963 len 24576
14841 069/300 Read block 1105 len 28672
14842 070/300 Read block 2954 len 40960
14843 071/300 Write block 3350 len 32768 data: 5 5 6 5 5 4 3 2
14844 072/300 Write block 742 len 4096 data: 2
14845 073/300 Read block 4146 len 8192
14846 074/300 Flush
14847 075/300 Write block 501 len 32768 data: 5 5 5 5 5 4 3 3
14848 076/300 Flush
14849 077/300 Read block 1056 len 36864
14850 078/300 Read block 4046 len 4096
14851 079/300 Write block 1299 len 4096 data: 4
14852 080/300 Read block 637 len 36864
14853 081/300 Write block 2181 len 40960 data: 3 3 3 3 3 3 3 3 3 3
14854 082/300 Write block 2721 len 32768 data: 2 2 2 2 3 3 4 4
14855 083/300 Read block 120 len 12288
14856 084/300 Read block 3313 len 36864
14857 085/300 Read block 3830 len 8192
14858 086/300 Write block 3088 len 28672 data: 2 2 3 3 4 4 4
14859 087/300 Read block 3695 len 24576
14860 088/300 Read block 1889 len 36864
14861 089/300 Read block 2080 len 24576
14862 090/300 Read block 3643 len 20480
14863 091/300 Flush
14864 092/300 Write block 3952 len 4096 data: 4
14865 093/300 Read block 586 len 36864
14866 094/300 Read block 2457 len 8192
14867 095/300 Read block 241 len 40960
14868 096/300 Read block 2722 len 4096
14869 097/300 Read block 1648 len 20480
14870 098/300 Read block 3529 len 40960
14871 099/300 Read block 1022 len 16384
14872 100/300 Read block 1848 len 16384
14873 101/300 Read block 840 len 20480
14874 102/300 Write block 4316 len 36864 data: 7 7 4 4 4 4 3 4 4
14875 103/300 Write block 2765 len 28672 data: 3 3 3 3 3 4 4
14876 104/300 Read block 889 len 12288
14877 105/300 Flush
14878 106/300 Read block 3456 len 12288
14879 107/300 Flush
14880 108/300 Write block 2905 len 8192 data: 3 3
14881 109/300 Read block 1625 len 16384
14882 110/300 Write block 2336 len 28672 data: 4 4 4 5 6 6 4
14883 111/300 Read block 983 len 12288
14884 112/300 Write block 4000 len 40960 data: 2 2 2 2 4 4 4 3 3 3
14885 113/300 Read block 4785 len 24576
14886 114/300 Read block 2715 len 32768
14887 115/300 Write block 4246 len 36864 data: 4 3 3 2 3 3 4 5 5
14888 116/300 Write block 4336 len 36864 data: 3 3 4 6 5 3 2 2 2
14889 117/300 Read block 4109 len 16384
14890 118/300 Read block 82 len 40960
14891 119/300 Write block 4214 len 16384 data: 4 4 4 4
14892 120/300 Read block 3906 len 28672
14893 121/300 Write block 2204 len 4096 data: 3
14894 122/300 Read block 325 len 24576
14895 123/300 Flush
14896 124/300 Flush
14897 125/300 Write block 3401 len 36864 data: 4 4 4 4 4 4 4 6 7
14898 126/300 Read block 4477 len 4096
14899 127/300 Read block 3506 len 36864
14900 128/300 Write block 1561 len 20480 data: 6 7 7 6 6
14901 129/300 Read block 2961 len 40960
14902 130/300 Read block 1735 len 8192
14903 131/300 Read block 4837 len 4096
14904 132/300 Write block 1109 len 36864 data: 3 2 2 2 2 3 3 3 3
14905 133/300 Read block 976 len 32768
14906 134/300 Write block 2287 len 12288 data: 5 5 4
14907 135/300 Write block 3752 len 24576 data: 4 3 4 4 4 4
14908 136/300 Read block 1407 len 16384
14909 137/300 Read block 4076 len 24576
14910 138/300 Write block 2466 len 16384 data: 3 2 2 2
14911 139/300 Read block 3683 len 32768
14912 140/300 Read block 3762 len 8192
14913 141/300 Read block 2425 len 20480
14914 142/300 Write block 3820 len 8192 data: 3 3
14915 143/300 Write block 326 len 8192 data: 3 5
14916 144/300 Read block 1375 len 28672
14917 145/300 Read block 1406 len 36864
14918 146/300 Write block 2069 len 24576 data: 2 2 2 2 2 3
14919 147/300 Flush
14920 148/300 Write block 1684 len 16384 data: 4 4 5 4
14921 149/300 Write block 873 len 4096 data: 3
14922 150/300 Read block 2317 len 16384
14923 151/300 Read block 4058 len 20480
14924 152/300 Read block 3418 len 24576
14925 153/300 Write block 4247 len 20480 data: 4 4 3 4 4
14926 154/300 Write block 1722 len 8192 data: 3 3
14927 155/300 Read block 3812 len 12288
14928 156/300 Write block 3379 len 20480 data: 4 4 4 4 4
14929 157/300 Write block 4698 len 40960 data: 3 3 3 4 4 5 6 6 7 5
14930 158/300 Read block 1405 len 20480
14931 159/300 Read block 4062 len 36864
14932 160/300 Read block 47 len 20480
14933 161/300 Write block 2655 len 16384 data: 2 2 3 4
14934 162/300 Read block 564 len 12288
14935 163/300 Write block 3656 len 40960 data: 2 2 3 3 3 3 3 3 3 2
14936 164/300 Write block 242 len 36864 data: 2 2 2 2 2 2 3 3 3
14937 165/300 Write block 1362 len 8192 data: 3 3
14938 166/300 Write block 2491 len 24576 data: 4 4 3 3 3 3
14939 167/300 Write block 2463 len 8192 data: 6 6
14940 168/300 Write block 2103 len 8192 data: 4 3
14941 169/300 Write block 4234 len 24576 data: 5 4 4 4 3 3
14942 170/300 Read block 4421 len 20480
14943 171/300 Read block 2801 len 4096
14944 172/300 Write block 4669 len 20480 data: 3 3 3 2 2
14945 173/300 Read block 2131 len 8192
14946 174/300 Read block 2707 len 40960
14947 175/300 Read block 2295 len 16384
14948 176/300 Flush
14949 177/300 Write block 3730 len 4096 data: 3
14950 178/300 Write block 2877 len 8192 data: 2 2
14951 179/300 Read block 1370 len 40960
14952 180/300 Read block 574 len 8192
14953 181/300 Read block 1198 len 8192
14954 182/300 Read block 4208 len 20480
14955 183/300 Read block 3420 len 28672
14956 184/300 Write block 659 len 32768 data: 4 4 4 3 4 2 2 2
14957 185/300 Write block 820 len 36864 data: 2 2 2 3 4 5 6 6 5
14958 186/300 Write block 3049 len 8192 data: 3 3
14959 187/300 Write block 2714 len 4096 data: 2
14960 188/300 Read block 368 len 20480
14961 189/300 Flush
14962 190/300 Read block 4335 len 12288
14963 191/300 Write block 2587 len 28672 data: 4 4 4 4 5 4 3
14964 192/300 Write block 3187 len 24576 data: 6 7 7 6 6 6
14965 193/300 Read block 1642 len 32768
14966 194/300 Write block 736 len 12288 data: 3 3 3
14967 195/300 Write block 325 len 20480 data: 3 4 6 4 4
14968 196/300 Read block 4640 len 8192
14969 197/300 Read block 725 len 28672
14970 198/300 Write block 1066 len 32768 data: 3 4 4 4 3 3 3 3
14971 199/300 Write block 1328 len 36864 data: 3 2 2 2 2 2 2 2 2
14972 200/300 Write block 3456 len 12288 data: 3 3 3
14973 201/300 Write block 2842 len 32768 data: 3 3 3 3 3 4 4 3
14974 202/300 Write block 1550 len 36864 data: 6 5 4 4 3 3 3 3 5
14975 203/300 Write block 1779 len 40960 data: 6 6 6 7 7 5 4 3 4 3
14976 204/300 Flush
14977 205/300 Write block 2976 len 24576 data: 3 4 4 3 4 5
14978 206/300 Read block 3481 len 8192
14979 207/300 Write block 3196 len 12288 data: 3 2 2
14980 208/300 Read block 4181 len 16384
14981 209/300 Write block 4859 len 8192 data: 2 2
14982 210/300 Flush
14983 211/300 Read block 4689 len 36864
14984 212/300 Write block 1605 len 8192 data: 3 4
14985 213/300 Read block 2486 len 24576
14986 214/300 Read block 2004 len 16384
14987 215/300 Write block 1588 len 16384 data: 2 2 3 3
14988 216/300 Write block 556 len 8192 data: 2 3
14989 217/300 Read block 582 len 28672
14990 218/300 Write block 4843 len 20480 data: 5 4 4 4 3
14991 219/300 Write block 2273 len 8192 data: 6 5
14992 220/300 Write block 4664 len 32768 data: 4 5 2 2 2 4 4 4
14993 221/300 Read block 4029 len 16384
14994 222/300 Write block 651 len 16384 data: 4 4 5 5
14995 223/300 Write block 3769 len 28672 data: 2 3 3 3 3 3 4
14996 224/300 Read block 1345 len 24576
14997 225/300 Read block 3076 len 40960
14998 226/300 Write block 747 len 12288 data: 2 2 2
14999 227/300 Read block 3207 len 40960
15000 228/300 Write block 4958 len 4096 data: 2
15001 229/300 Read block 2382 len 28672
15002 230/300 Write block 3829 len 16384 data: 2 2 2 3
15003 231/300 Write block 4665 len 36864 data: 6 3 3 3 5 5 5 3 3
15004 232/300 Write block 4390 len 28672 data: 2 2 2 2 3 3 2
15005 233/300 Write block 2096 len 40960 data: 3 4 4 4 4 4 4 5 4 2
15006 234/300 Flush
15007 235/300 Read block 4428 len 28672
15008 236/300 Write block 4875 len 8192 data: 5 4
15009 237/300 Read block 3710 len 16384
15010 238/300 Flush
15011 239/300 Write block 2496 len 24576 data: 4 3 4 4 4 3
15012 240/300 Read block 2227 len 8192
15013 241/300 Read block 3055 len 24576
15014 242/300 Read block 3634 len 24576
15015 243/300 Write block 941 len 4096 data: 3
15016 244/300 Read block 3278 len 8192
15017 245/300 Read block 2404 len 36864
15018 246/300 Read block 1230 len 40960
15019 247/300 Read block 2136 len 32768
15020 248/300 Write block 3197 len 24576 data: 3 3 2 2 2 2
15021 249/300 Write block 2979 len 4096 data: 4
15022 250/300 Write block 4895 len 12288 data: 4 3 3
15023 251/300 Write block 1460 len 32768 data: 4 4 4 3 4 3 3 3
15024 252/300 Read block 3767 len 20480
15025 253/300 Read block 734 len 40960
15026 254/300 Flush
15027 255/300 Read block 421 len 24576
15028 256/300 Write block 4786 len 8192 data: 4 4
15029 257/300 Read block 4051 len 20480
15030 258/300 Flush
15031 259/300 Write block 2592 len 12288 data: 5 4 4
15032 260/300 Read block 3976 len 20480
15033 261/300 Read block 3814 len 24576
15034 262/300 Write block 4218 len 28672 data: 3 3 5 4 4 4 4
15035 263/300 Read block 2560 len 32768
15036 264/300 Read block 1394 len 8192
15037 265/300 Read block 1106 len 12288
15038 266/300 Write block 1145 len 16384 data: 4 5 5 5
15039 267/300 Flush
15040 268/300 Read block 3637 len 12288
15041 269/300 Read block 3719 len 16384
15042 270/300 Flush
15043 271/300 Flush
15044 272/300 Write block 2528 len 8192 data: 5 5
15045 273/300 Write block 3528 len 32768 data: 2 4 4 5 4 4 3 4
15046 274/300 Read block 2820 len 32768
15047 275/300 Write block 2218 len 16384 data: 4 3 4 4
15048 276/300 Read block 3176 len 8192
15049 277/300 Read block 4250 len 20480
15050 278/300 Flush
15051 279/300 Read block 2456 len 20480
15052 280/300 Write block 4752 len 16384 data: 3 3 3 2
15053 281/300 Read block 2447 len 24576
15054 282/300 Read block 560 len 20480
15055 283/300 Flush
15056 284/300 Read block 3810 len 36864
15057 285/300 Write block 2281 len 12288 data: 4 4 5
15058 286/300 Read block 1892 len 40960
15059 287/300 Flush
15060 288/300 Read block 1428 len 32768
15061 289/300 Read block 3136 len 8192
15062 290/300 Read block 4459 len 32768
15063 291/300 Write block 11 len 16384 data: 6 5 4 4
15064 292/300 Read block 804 len 24576
15065 293/300 Read block 471 len 20480
15066 294/300 Flush
15067 295/300 Write block 3970 len 4096 data: 4
15068 296/300 Flush
15069 297/300 Write block 3447 len 20480 data: 2 2 2 2 2
15070 298/300 Read block 2878 len 16384
15071 299/300 Read block 2801 len 4096
15072 300/300 Write block 2802 len 24576 data: 2 2 2 2 2 2
15073 301/300 Write block 1130 len 8192 data: 3 3
15074 Main task has work to do, go find it
15075 got dsc Start(0)
15076 start 0
15077 Nov 15 02:00:38.827 INFO request completed, latency_us: 264, response_code: 204, uri: /start/cid/0, method: POST, req_id: a9c0388c-76b7-4959-b959-8281945c2a41, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
15078 Replay: started 0, returned:Ok(())
15079 ----------------------------------------------------------------
15080 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
15081 States: Offline Active Active
15082 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
15083 4320 Acked Write 4 Sent Done Done false
15084 4321 Acked Flush 0 Sent Done Done false
15085 4322 Acked Write 9 Sent Done Done false
15086 4323 Acked Read 10 Sent Done Done false
15087 4324 Acked Write 2 Sent Done Done false
15088 4325 Acked Read 1 Sent Done Done false
15089 4326 Acked Read 9 Sent Done Done false
15090 4327 Acked Write 2 Sent Done Done false
15091 4328 Acked Write 10 Sent Done Done false
15092 4329 Acked Write 3 Sent Done Done false
15093 4330 Acked Read 4 Sent Done Done false
15094 4331 Acked Write 5 Sent Done Done false
15095 4332 Acked Read 3 Sent Done Done false
15096 4333 Acked Flush 0 Sent Done Done false
15097 4334 Acked Read 4 Sent Done Done false
15098 4335 Acked Write 2 Sent Done Done false
15099 4336 Acked Write 8 Sent Done Done false
15100 4337 Acked Read 6 Sent Done Done false
15101 4338 Acked Read 6 Sent Done Done false
15102 4339 Acked Read 2 Sent Done Done false
15103 4340 Acked Read 10 Sent Done Done false
15104 4341 Acked Read 4 Sent Done Done false
15105 4342 Acked Read 5 Sent Done Done false
15106 4343 Acked Flush 0 Sent Done Done false
15107 4344 Acked Write 9 Sent Done Done false
15108 4345 Acked Read 5 Sent Done Done false
15109 4346 Acked Write 1 Sent Done Done false
15110 4347 Acked Write 2 Sent Done Done false
15111 4348 Acked Write 1 Sent Done Done false
15112 4349 Acked Write 10 Sent Done Done false
15113 4350 Acked Write 9 Sent Done Done false
15114 4351 Acked Read 4 Sent Done Done false
15115 4352 Acked Write 10 Sent Done Done false
15116 4353 Acked Write 7 Sent Done Done false
15117 4354 Acked Read 9 Sent Done Done false
15118 4355 Acked Read 5 Sent Done Done false
15119 4356 Acked Write 5 Sent Done Done false
15120 4357 Acked Read 8 Sent Done Done false
15121 4358 Acked Read 5 Sent Done Done false
15122 4359 Acked Flush 0 Sent Done Done false
15123 4360 Acked Write 4 Sent Done Done false
15124 4361 Acked Write 8 Sent Done Done false
15125 4362 Acked Read 1 Sent Done Done false
15126 4363 Acked Read 10 Sent Done Done false
15127 4364 Acked Read 4 Sent Done Done false
15128 4365 Acked Write 8 Sent Done Done false
15129 4366 Acked Write 4 Sent Done Done false
15130 4367 Acked Read 3 Sent Done Done false
15131 4368 Acked Write 9 Sent Done Done false
15132 4369 Acked Read 9 Sent Done Done false
15133 4370 Acked Write 9 Sent Done Done false
15134 4371 Acked Write 5 Sent Done Done false
15135 4372 Acked Read 5 Sent Done Done false
15136 4373 Acked Write 1 Sent Done Done false
15137 4374 Acked Read 1 Sent Done Done false
15138 4375 Acked Write 5 Sent Done Done false
15139 4376 Acked Read 2 Sent Done Done false
15140 4377 Acked Flush 0 Sent Done Done false
15141 4378 Acked Read 5 Sent Done Done false
15142 4379 Acked Read 8 Sent Done Done false
15143 4380 Acked Read 3 Sent Done Done false
15144 4381 Acked Read 7 Sent Done Done false
15145 4382 Acked Read 10 Sent Done Done false
15146 4383 Acked Write 2 Sent Done Done false
15147 4384 Acked Write 7 Sent Done Done false
15148 4385 Acked Write 7 Sent Done Done false
15149 4386 Acked Write 3 Sent Done Done false
15150 4387 Acked Read 6 Sent Done Done false
15151 4388 Acked Read 7 Sent Done Done false
15152 4389 Acked Read 10 Sent Done Done false
15153 4390 Acked Write 8 Sent Done Done false
15154 4391 Acked Write 1 Sent Done Done false
15155 4392 Acked Read 2 Sent Done Done false
15156 4393 Acked Flush 0 Sent Done Done false
15157 4394 Acked Write 8 Sent Done Done false
15158 4395 Acked Flush 0 Sent Done Done false
15159 4396 Acked Read 9 Sent Done Done false
15160 4397 Acked Read 1 Sent Done Done false
15161 4398 Acked Write 1 Sent Done Done false
15162 4399 Acked Read 9 Sent Done Done false
15163 4400 Acked Write 10 Sent Done Done false
15164 4401 Acked Write 8 Sent Done Done false
15165 4402 Acked Read 3 Sent Done Done false
15166 4403 Acked Read 9 Sent Done Done false
15167 4404 Acked Read 2 Sent Done Done false
15168 4405 Acked Write 7 Sent Done Done false
15169 4406 Acked Read 6 Sent Done Done false
15170 4407 Acked Read 9 Sent Done Done false
15171 4408 Acked Read 6 Sent Done Done false
15172 4409 Acked Read 5 Sent Done Done false
15173 4410 Acked Flush 0 Sent Done Done false
15174 4411 Acked Write 1 Sent Done Done false
15175 4412 Acked Read 9 Sent Done Done false
15176 4413 Acked Read 2 Sent Done Done false
15177 4414 Acked Read 10 Sent Done Done false
15178 4415 Acked Read 1 Sent Done Done false
15179 4416 Acked Read 5 Sent Done Done false
15180 4417 Acked Read 10 Sent Done Done false
15181 4418 Acked Read 4 Sent Done Done false
15182 4419 Acked Read 4 Sent Done Done false
15183 4420 Acked Read 5 Sent Done Done false
15184 4421 Acked Write 9 Sent Done Done false
15185 4422 Acked Write 7 Sent Done Done false
15186 4423 Acked Read 3 Sent Done Done false
15187 4424 Acked Flush 0 Sent Done Done false
15188 4425 Acked Read 3 Sent Done Done false
15189 4426 Acked Flush 0 Sent Done Done false
15190 4427 Acked Write 2 Sent Done Done false
15191 4428 Acked Read 4 Sent Done Done false
15192 4429 Acked Write 7 Sent Done Done false
15193 4430 Acked Read 3 Sent Done Done false
15194 4431 Acked Write 10 Sent Done Done false
15195 4432 Acked Read 6 Sent Done Done false
15196 4433 Acked Read 8 Sent Done Done false
15197 4434 Acked Write 9 Sent Done Done false
15198 4435 Acked Write 9 Sent Done Done false
15199 4436 Acked Read 4 Sent Done Done false
15200 4437 Acked Read 10 Sent Done Done false
15201 4438 Acked Write 4 Sent Done Done false
15202 4439 Acked Read 7 Sent Done Done false
15203 4440 Acked Write 1 Sent Done Done false
15204 4441 Acked Read 6 Sent Done Done false
15205 4442 Acked Flush 0 Sent Done Done false
15206 4443 Acked Flush 0 Sent Done Done false
15207 4444 Acked Write 9 Sent Done Done false
15208 4445 Acked Read 1 Sent Done Done false
15209 4446 Acked Read 9 Sent Done Done false
15210 4447 Acked Write 5 Sent Done Done false
15211 4448 Acked Read 10 Sent Done Done false
15212 4449 Acked Read 2 Sent Done Done false
15213 4450 Acked Read 1 Sent Done Done false
15214 4451 Acked Write 9 Sent Done Done false
15215 4452 Acked Read 8 Sent Done Done false
15216 4453 Acked Write 3 Sent Done Done false
15217 4454 Acked Write 6 Sent Done Done false
15218 4455 Acked Read 4 Sent Done Done false
15219 4456 Acked Read 6 Sent Done Done false
15220 4457 Acked Write 4 Sent Done Done false
15221 4458 Acked Read 8 Sent Done Done false
15222 4459 Acked Read 2 Sent Done Done false
15223 4460 Acked Read 5 Sent Done Done false
15224 4461 Acked Write 2 Sent Done Done false
15225 4462 Acked Write 2 Sent Done Done false
15226 4463 Acked Read 7 Sent Done Done false
15227 4464 Acked Read 9 Sent Done Done false
15228 4465 Acked Write 6 Sent Done Done false
15229 4466 Acked Flush 0 Sent Done Done false
15230 4467 Acked Write 4 Sent Done Done false
15231 4468 Acked Write 1 Sent Done Done false
15232 4469 Acked Read 4 Sent Done Done false
15233 4470 Acked Read 5 Sent Done Done false
15234 4471 Acked Read 6 Sent Done Done false
15235 4472 Acked Write 5 Sent Done Done false
15236 4473 Acked Write 2 Sent Done Done false
15237 4474 Acked Read 3 Sent Done Done false
15238 4475 Acked Write 5 Sent Done Done false
15239 4476 Acked Write 10 Sent Done Done false
15240 4477 Acked Read 5 Sent Done Done false
15241 4478 Acked Read 9 Sent Done Done false
15242 4479 Acked Read 5 Sent Done Done false
15243 4480 Acked Write 4 Sent Done Done false
15244 4481 Acked Read 3 Sent Done Done false
15245 4482 Acked Write 10 Sent Done Done false
15246 4483 Acked Write 9 Sent Done Done false
15247 4484 Acked Write 2 Sent Done Done false
15248 4485 Acked Write 6 Sent Done Done false
15249 4486 Acked Write 2 Sent Done Done false
15250 4487 Acked Write 2 Sent Done Done false
15251 4488 Acked Write 6 Sent Done Done false
15252 4489 Acked Read 5 Sent Done Done false
15253 4490 Acked Read 1 Sent Done Done false
15254 4491 Acked Write 5 Sent Done Done false
15255 4492 Acked Read 2 Sent Done Done false
15256 4493 Acked Read 10 Sent Done Done false
15257 4494 Acked Read 4 Sent Done Done false
15258 4495 Acked Flush 0 Sent Done Done false
15259 4496 Acked Write 1 Sent Done Done false
15260 4497 Acked Write 2 Sent Done Done false
15261 4498 Acked Read 10 Sent Done Done false
15262 4499 Acked Read 2 Sent Done Done false
15263 4500 Acked Read 2 Sent Done Done false
15264 4501 Acked Read 5 Sent Done Done false
15265 4502 Acked Read 7 Sent Done Done false
15266 4503 Acked Write 8 Sent Done Done false
15267 4504 Acked Write 9 Sent Done Done false
15268 4505 Acked Write 2 Sent Done Done false
15269 4506 Acked Write 1 Sent Done Done false
15270 4507 Acked Read 5 Sent Done Done false
15271 4508 Acked Flush 0 Sent Done Done false
15272 4509 Acked Read 3 Sent Done Done false
15273 4510 Acked Write 7 Sent Done Done false
15274 4511 Acked Write 6 Sent Done Done false
15275 4512 Acked Read 8 Sent Done Done false
15276 4513 Acked Write 3 Sent Done Done false
15277 4514 Acked Write 5 Sent Done Done false
15278 4515 Acked Read 2 Sent Done Done false
15279 4516 Acked Read 7 Sent Done Done false
15280 4517 Acked Write 8 Sent Done Done false
15281 4518 Acked Write 9 Sent Done Done false
15282 4519 Acked Write 3 Sent Done Done false
15283 4520 Acked Write 8 Sent Done Done false
15284 4521 Acked Write 9 Sent Done Done false
15285 4522 Acked Write 10 Sent Done Done false
15286 4523 Acked Flush 0 Sent Done Done false
15287 4524 Acked Write 6 Sent Done Done false
15288 4525 Acked Read 2 Sent Done Done false
15289 4526 Acked Write 3 Sent Done Done false
15290 4527 Acked Read 4 Sent Done Done false
15291 4528 Acked Write 2 Sent Done Done false
15292 4529 Acked Flush 0 Sent Done Done false
15293 4530 Acked Read 9 Sent Done Done false
15294 4531 Acked Write 2 Sent Done Done false
15295 4532 Acked Read 6 Sent Done Done false
15296 4533 Acked Read 4 Sent Done Done false
15297 4534 Acked Write 4 Sent Done Done false
15298 4535 Acked Write 2 Sent Done Done false
15299 4536 Acked Read 7 Sent Done Done false
15300 4537 Acked Write 5 Sent Done Done false
15301 4538 Acked Write 2 Sent Done Done false
15302 4539 Acked Write 8 Sent Done Done false
15303 4540 Acked Read 4 Sent Done Done false
15304 4541 Acked Write 4 Sent Done Done false
15305 4542 Acked Write 7 Sent Done Done false
15306 4543 Acked Read 6 Sent Done Done false
15307 4544 Acked Read 10 Sent Done Done false
15308 4545 Acked Write 3 Sent Done Done false
15309 4546 Acked Read 10 Sent Done Done false
15310 4547 Acked Write 1 Sent Done Done false
15311 4548 Acked Read 7 Sent Done Done false
15312 4549 Acked Write 4 Sent Done Done false
15313 4550 Acked Write 9 Sent Done Done false
15314 4551 Acked Write 7 Sent Done Done false
15315 4552 Acked Write 10 Sent Done Done false
15316 4553 Acked Flush 0 Sent Done Done false
15317 4554 Acked Read 7 Sent Done Done false
15318 4555 Acked Write 2 Sent Done Done false
15319 4556 Acked Read 4 Sent Done Done false
15320 4557 Acked Flush 0 Sent Done Done false
15321 4558 Acked Write 6 Sent Done Done false
15322 4559 Acked Read 2 Sent Done Done false
15323 4560 Acked Read 6 Sent Done Done false
15324 4561 Acked Read 6 Sent Done Done false
15325 4562 Acked Write 1 Sent Done Done false
15326 4563 Acked Read 2 Sent Done Done false
15327 4564 Acked Read 9 Sent Done Done false
15328 4565 Acked Read 10 Sent Done Done false
15329 4566 Acked Read 8 Sent Done Done false
15330 4567 Acked Write 6 Sent Done Done false
15331 4568 Acked Write 1 Sent Done Done false
15332 4569 Acked Write 3 Sent Done Done false
15333 4570 Acked Write 8 Sent Done Done false
15334 4571 Acked Read 5 Sent Done Done false
15335 4572 Acked Read 10 Sent Done Done false
15336 4573 Acked Flush 0 Sent Done Done false
15337 4574 Acked Read 6 Sent Done Done false
15338 4575 Acked Write 2 Sent Done Done false
15339 4576 Acked Read 5 Sent Done Done false
15340 4577 Acked Flush 0 Sent Done Done false
15341 4578 Acked Write 3 Sent Done Done false
15342 4579 Acked Read 5 Sent Done Done false
15343 4580 Acked Read 6 Sent Done Done false
15344 4581 Acked Write 7 Sent Done Done false
15345 4582 Acked Read 8 Sent Done Done false
15346 4583 Acked Read 2 Sent Done Done false
15347 4584 Acked Read 3 Sent Done Done false
15348 4585 Acked Write 4 Sent Done Done false
15349 4586 Acked Flush 0 Sent Done Done false
15350 4587 Acked Read 3 Sent Done Done false
15351 4588 Acked Read 4 Sent Done Done false
15352 4589 Acked Flush 0 Sent Done Done false
15353 4590 Acked Flush 0 Sent Done Done false
15354 4591 Acked Write 2 Sent Done Done false
15355 4592 Acked Write 8 Sent Done Done false
15356 4593 Acked Read 8 Sent Done Done false
15357 4594 Acked Write 4 Sent Done Done false
15358 4595 Acked Read 2 Sent Done Done false
15359 4596 Acked Read 5 Sent Done Done false
15360 4597 Acked Flush 0 Sent Done Done false
15361 4598 Acked Read 5 Sent Done Done false
15362 4599 Acked Write 4 Sent Done Done false
15363 4600 Acked Read 6 Sent Done Done false
15364 4601 Acked Read 5 Sent Done Done false
15365 4602 Acked Flush 0 Sent Done Done false
15366 4603 Acked Read 9 Sent Done Done false
15367 4604 Acked Write 3 Sent Done Done false
15368 4605 Acked Read 10 Sent Done Done false
15369 4606 Acked Flush 0 Sent Done Done false
15370 4607 Acked Read 8 Sent Done Done false
15371 4608 Acked Read 2 Sent Done Done false
15372 4609 Acked Read 8 Sent Done Done false
15373 4610 Acked Write 4 Sent Done Done false
15374 4611 Acked Read 6 Sent Done Done false
15375 4612 Acked Read 5 Sent Done Done false
15376 4613 Acked Flush 0 Sent Done Done false
15377 4614 Acked Write 1 Sent Done Done false
15378 4615 Acked Flush 0 Sent Done Done false
15379 4616 Acked Write 5 Sent Done Done false
15380 4617 Acked Read 4 Sent Done Done false
15381 4618 Acked Read 1 Sent Done Done false
15382 4619 Acked Write 6 Sent Done Done false
15383 4620 Acked Write 2 Sent Done Done false
15384 STATES DS:0 DS:1 DS:2 TOTAL
15385 Sent 301 0 0 301
15386 Done 0 301 301 602
15387 Skipped 0 0 0 0
15388 Error 0 0 0 0
15389 Last Flush: 4319 4615 4615
15390 Downstairs last five completed: 4319 4318 4317 4316 4315
15391 Upstairs last five completed: 4620 4619 4618 4617 4616
15392 CLIENT: Up:0 ds:301 act:2
15393 ----------------------------------------------------------------
15394 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
15395 States: Offline Active Active
15396 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
15397 4320 Acked Write 4 Sent Done Done false
15398 4321 Acked Flush 0 Sent Done Done false
15399 4322 Acked Write 9 Sent Done Done false
15400 4323 Acked Read 10 Sent Done Done false
15401 4324 Acked Write 2 Sent Done Done false
15402 4325 Acked Read 1 Sent Done Done false
15403 4326 Acked Read 9 Sent Done Done false
15404 4327 Acked Write 2 Sent Done Done false
15405 4328 Acked Write 10 Sent Done Done false
15406 4329 Acked Write 3 Sent Done Done false
15407 4330 Acked Read 4 Sent Done Done false
15408 4331 Acked Write 5 Sent Done Done false
15409 4332 Acked Read 3 Sent Done Done false
15410 4333 Acked Flush 0 Sent Done Done false
15411 4334 Acked Read 4 Sent Done Done false
15412 4335 Acked Write 2 Sent Done Done false
15413 4336 Acked Write 8 Sent Done Done false
15414 4337 Acked Read 6 Sent Done Done false
15415 4338 Acked Read 6 Sent Done Done false
15416 4339 Acked Read 2 Sent Done Done false
15417 4340 Acked Read 10 Sent Done Done false
15418 4341 Acked Read 4 Sent Done Done false
15419 4342 Acked Read 5 Sent Done Done false
15420 4343 Acked Flush 0 Sent Done Done false
15421 4344 Acked Write 9 Sent Done Done false
15422 4345 Acked Read 5 Sent Done Done false
15423 4346 Acked Write 1 Sent Done Done false
15424 4347 Acked Write 2 Sent Done Done false
15425 4348 Acked Write 1 Sent Done Done false
15426 4349 Acked Write 10 Sent Done Done false
15427 4350 Acked Write 9 Sent Done Done false
15428 4351 Acked Read 4 Sent Done Done false
15429 4352 Acked Write 10 Sent Done Done false
15430 4353 Acked Write 7 Sent Done Done false
15431 4354 Acked Read 9 Sent Done Done false
15432 4355 Acked Read 5 Sent Done Done false
15433 4356 Acked Write 5 Sent Done Done false
15434 4357 Acked Read 8 Sent Done Done false
15435 4358 Acked Read 5 Sent Done Done false
15436 4359 Acked Flush 0 Sent Done Done false
15437 4360 Acked Write 4 Sent Done Done false
15438 4361 Acked Write 8 Sent Done Done false
15439 4362 Acked Read 1 Sent Done Done false
15440 4363 Acked Read 10 Sent Done Done false
15441 4364 Acked Read 4 Sent Done Done false
15442 4365 Acked Write 8 Sent Done Done false
15443 4366 Acked Write 4 Sent Done Done false
15444 4367 Acked Read 3 Sent Done Done false
15445 4368 Acked Write 9 Sent Done Done false
15446 4369 Acked Read 9 Sent Done Done false
15447 4370 Acked Write 9 Sent Done Done false
15448 4371 Acked Write 5 Sent Done Done false
15449 4372 Acked Read 5 Sent Done Done false
15450 4373 Acked Write 1 Sent Done Done false
15451 4374 Acked Read 1 Sent Done Done false
15452 4375 Acked Write 5 Sent Done Done false
15453 4376 Acked Read 2 Sent Done Done false
15454 4377 Acked Flush 0 Sent Done Done false
15455 4378 Acked Read 5 Sent Done Done false
15456 4379 Acked Read 8 Sent Done Done false
15457 4380 Acked Read 3 Sent Done Done false
15458 4381 Acked Read 7 Sent Done Done false
15459 4382 Acked Read 10 Sent Done Done false
15460 4383 Acked Write 2 Sent Done Done false
15461 4384 Acked Write 7 Sent Done Done false
15462 4385 Acked Write 7 Sent Done Done false
15463 4386 Acked Write 3 Sent Done Done false
15464 4387 Acked Read 6 Sent Done Done false
15465 4388 Acked Read 7 Sent Done Done false
15466 4389 Acked Read 10 Sent Done Done false
15467 4390 Acked Write 8 Sent Done Done false
15468 4391 Acked Write 1 Sent Done Done false
15469 4392 Acked Read 2 Sent Done Done false
15470 4393 Acked Flush 0 Sent Done Done false
15471 4394 Acked Write 8 Sent Done Done false
15472 4395 Acked Flush 0 Sent Done Done false
15473 4396 Acked Read 9 Sent Done Done false
15474 4397 Acked Read 1 Sent Done Done false
15475 4398 Acked Write 1 Sent Done Done false
15476 4399 Acked Read 9 Sent Done Done false
15477 4400 Acked Write 10 Sent Done Done false
15478 4401 Acked Write 8 Sent Done Done false
15479 4402 Acked Read 3 Sent Done Done false
15480 4403 Acked Read 9 Sent Done Done false
15481 4404 Acked Read 2 Sent Done Done false
15482 4405 Acked Write 7 Sent Done Done false
15483 4406 Acked Read 6 Sent Done Done false
15484 4407 Acked Read 9 Sent Done Done false
15485 4408 Acked Read 6 Sent Done Done false
15486 4409 Acked Read 5 Sent Done Done false
15487 4410 Acked Flush 0 Sent Done Done false
15488 4411 Acked Write 1 Sent Done Done false
15489 4412 Acked Read 9 Sent Done Done false
15490 4413 Acked Read 2 Sent Done Done false
15491 4414 Acked Read 10 Sent Done Done false
15492 4415 Acked Read 1 Sent Done Done false
15493 4416 Acked Read 5 Sent Done Done false
15494 4417 Acked Read 10 Sent Done Done false
15495 4418 Acked Read 4 Sent Done Done false
15496 4419 Acked Read 4 Sent Done Done false
15497 4420 Acked Read 5 Sent Done Done false
15498 4421 Acked Write 9 Sent Done Done false
15499 4422 Acked Write 7 Sent Done Done false
15500 4423 Acked Read 3 Sent Done Done false
15501 4424 Acked Flush 0 Sent Done Done false
15502 4425 Acked Read 3 Sent Done Done false
15503 4426 Acked Flush 0 Sent Done Done false
15504 4427 Acked Write 2 Sent Done Done false
15505 4428 Acked Read 4 Sent Done Done false
15506 4429 Acked Write 7 Sent Done Done false
15507 4430 Acked Read 3 Sent Done Done false
15508 4431 Acked Write 10 Sent Done Done false
15509 4432 Acked Read 6 Sent Done Done false
15510 4433 Acked Read 8 Sent Done Done false
15511 4434 Acked Write 9 Sent Done Done false
15512 4435 Acked Write 9 Sent Done Done false
15513 4436 Acked Read 4 Sent Done Done false
15514 4437 Acked Read 10 Sent Done Done false
15515 4438 Acked Write 4 Sent Done Done false
15516 4439 Acked Read 7 Sent Done Done false
15517 4440 Acked Write 1 Sent Done Done false
15518 4441 Acked Read 6 Sent Done Done false
15519 4442 Acked Flush 0 Sent Done Done false
15520 4443 Acked Flush 0 Sent Done Done false
15521 4444 Acked Write 9 Sent Done Done false
15522 4445 Acked Read 1 Sent Done Done false
15523 4446 Acked Read 9 Sent Done Done false
15524 4447 Acked Write 5 Sent Done Done false
15525 4448 Acked Read 10 Sent Done Done false
15526 4449 Acked Read 2 Sent Done Done false
15527 4450 Acked Read 1 Sent Done Done false
15528 4451 Acked Write 9 Sent Done Done false
15529 4452 Acked Read 8 Sent Done Done false
15530 4453 Acked Write 3 Sent Done Done false
15531 4454 Acked Write 6 Sent Done Done false
15532 4455 Acked Read 4 Sent Done Done false
15533 4456 Acked Read 6 Sent Done Done false
15534 4457 Acked Write 4 Sent Done Done false
15535 4458 Acked Read 8 Sent Done Done false
15536 4459 Acked Read 2 Sent Done Done false
15537 4460 Acked Read 5 Sent Done Done false
15538 4461 Acked Write 2 Sent Done Done false
15539 4462 Acked Write 2 Sent Done Done false
15540 4463 Acked Read 7 Sent Done Done false
15541 4464 Acked Read 9 Sent Done Done false
15542 4465 Acked Write 6 Sent Done Done false
15543 4466 Acked Flush 0 Sent Done Done false
15544 4467 Acked Write 4 Sent Done Done false
15545 4468 Acked Write 1 Sent Done Done false
15546 4469 Acked Read 4 Sent Done Done false
15547 4470 Acked Read 5 Sent Done Done false
15548 4471 Acked Read 6 Sent Done Done false
15549 4472 Acked Write 5 Sent Done Done false
15550 4473 Acked Write 2 Sent Done Done false
15551 4474 Acked Read 3 Sent Done Done false
15552 4475 Acked Write 5 Sent Done Done false
15553 4476 Acked Write 10 Sent Done Done false
15554 4477 Acked Read 5 Sent Done Done false
15555 4478 Acked Read 9 Sent Done Done false
15556 4479 Acked Read 5 Sent Done Done false
15557 4480 Acked Write 4 Sent Done Done false
15558 4481 Acked Read 3 Sent Done Done false
15559 4482 Acked Write 10 Sent Done Done false
15560 4483 Acked Write 9 Sent Done Done false
15561 4484 Acked Write 2 Sent Done Done false
15562 4485 Acked Write 6 Sent Done Done false
15563 4486 Acked Write 2 Sent Done Done false
15564 4487 Acked Write 2 Sent Done Done false
15565 4488 Acked Write 6 Sent Done Done false
15566 4489 Acked Read 5 Sent Done Done false
15567 4490 Acked Read 1 Sent Done Done false
15568 4491 Acked Write 5 Sent Done Done false
15569 4492 Acked Read 2 Sent Done Done false
15570 4493 Acked Read 10 Sent Done Done false
15571 4494 Acked Read 4 Sent Done Done false
15572 4495 Acked Flush 0 Sent Done Done false
15573 4496 Acked Write 1 Sent Done Done false
15574 4497 Acked Write 2 Sent Done Done false
15575 4498 Acked Read 10 Sent Done Done false
15576 4499 Acked Read 2 Sent Done Done false
15577 4500 Acked Read 2 Sent Done Done false
15578 4501 Acked Read 5 Sent Done Done false
15579 4502 Acked Read 7 Sent Done Done false
15580 4503 Acked Write 8 Sent Done Done false
15581 4504 Acked Write 9 Sent Done Done false
15582 4505 Acked Write 2 Sent Done Done false
15583 4506 Acked Write 1 Sent Done Done false
15584 4507 Acked Read 5 Sent Done Done false
15585 4508 Acked Flush 0 Sent Done Done false
15586 4509 Acked Read 3 Sent Done Done false
15587 4510 Acked Write 7 Sent Done Done false
15588 4511 Acked Write 6 Sent Done Done false
15589 4512 Acked Read 8 Sent Done Done false
15590 4513 Acked Write 3 Sent Done Done false
15591 4514 Acked Write 5 Sent Done Done false
15592 4515 Acked Read 2 Sent Done Done false
15593 4516 Acked Read 7 Sent Done Done false
15594 4517 Acked Write 8 Sent Done Done false
15595 4518 Acked Write 9 Sent Done Done false
15596 4519 Acked Write 3 Sent Done Done false
15597 4520 Acked Write 8 Sent Done Done false
15598 4521 Acked Write 9 Sent Done Done false
15599 4522 Acked Write 10 Sent Done Done false
15600 4523 Acked Flush 0 Sent Done Done false
15601 4524 Acked Write 6 Sent Done Done false
15602 4525 Acked Read 2 Sent Done Done false
15603 4526 Acked Write 3 Sent Done Done false
15604 4527 Acked Read 4 Sent Done Done false
15605 4528 Acked Write 2 Sent Done Done false
15606 4529 Acked Flush 0 Sent Done Done false
15607 4530 Acked Read 9 Sent Done Done false
15608 4531 Acked Write 2 Sent Done Done false
15609 4532 Acked Read 6 Sent Done Done false
15610 4533 Acked Read 4 Sent Done Done false
15611 4534 Acked Write 4 Sent Done Done false
15612 4535 Acked Write 2 Sent Done Done false
15613 4536 Acked Read 7 Sent Done Done false
15614 4537 Acked Write 5 Sent Done Done false
15615 4538 Acked Write 2 Sent Done Done false
15616 4539 Acked Write 8 Sent Done Done false
15617 4540 Acked Read 4 Sent Done Done false
15618 4541 Acked Write 4 Sent Done Done false
15619 4542 Acked Write 7 Sent Done Done false
15620 4543 Acked Read 6 Sent Done Done false
15621 4544 Acked Read 10 Sent Done Done false
15622 4545 Acked Write 3 Sent Done Done false
15623 4546 Acked Read 10 Sent Done Done false
15624 4547 Acked Write 1 Sent Done Done false
15625 4548 Acked Read 7 Sent Done Done false
15626 4549 Acked Write 4 Sent Done Done false
15627 4550 Acked Write 9 Sent Done Done false
15628 4551 Acked Write 7 Sent Done Done false
15629 4552 Acked Write 10 Sent Done Done false
15630 4553 Acked Flush 0 Sent Done Done false
15631 4554 Acked Read 7 Sent Done Done false
15632 4555 Acked Write 2 Sent Done Done false
15633 4556 Acked Read 4 Sent Done Done false
15634 4557 Acked Flush 0 Sent Done Done false
15635 4558 Acked Write 6 Sent Done Done false
15636 4559 Acked Read 2 Sent Done Done false
15637 4560 Acked Read 6 Sent Done Done false
15638 4561 Acked Read 6 Sent Done Done false
15639 4562 Acked Write 1 Sent Done Done false
15640 4563 Acked Read 2 Sent Done Done false
15641 4564 Acked Read 9 Sent Done Done false
15642 4565 Acked Read 10 Sent Done Done false
15643 4566 Acked Read 8 Sent Done Done false
15644 4567 Acked Write 6 Sent Done Done false
15645 4568 Acked Write 1 Sent Done Done false
15646 4569 Acked Write 3 Sent Done Done false
15647 4570 Acked Write 8 Sent Done Done false
15648 4571 Acked Read 5 Sent Done Done false
15649 4572 Acked Read 10 Sent Done Done false
15650 4573 Acked Flush 0 Sent Done Done false
15651 4574 Acked Read 6 Sent Done Done false
15652 4575 Acked Write 2 Sent Done Done false
15653 4576 Acked Read 5 Sent Done Done false
15654 4577 Acked Flush 0 Sent Done Done false
15655 4578 Acked Write 3 Sent Done Done false
15656 4579 Acked Read 5 Sent Done Done false
15657 4580 Acked Read 6 Sent Done Done false
15658 4581 Acked Write 7 Sent Done Done false
15659 4582 Acked Read 8 Sent Done Done false
15660 4583 Acked Read 2 Sent Done Done false
15661 4584 Acked Read 3 Sent Done Done false
15662 4585 Acked Write 4 Sent Done Done false
15663 4586 Acked Flush 0 Sent Done Done false
15664 4587 Acked Read 3 Sent Done Done false
15665 4588 Acked Read 4 Sent Done Done false
15666 4589 Acked Flush 0 Sent Done Done false
15667 4590 Acked Flush 0 Sent Done Done false
15668 4591 Acked Write 2 Sent Done Done false
15669 4592 Acked Write 8 Sent Done Done false
15670 4593 Acked Read 8 Sent Done Done false
15671 4594 Acked Write 4 Sent Done Done false
15672 4595 Acked Read 2 Sent Done Done false
15673 4596 Acked Read 5 Sent Done Done false
15674 4597 Acked Flush 0 Sent Done Done false
15675 4598 Acked Read 5 Sent Done Done false
15676 4599 Acked Write 4 Sent Done Done false
15677 4600 Acked Read 6 Sent Done Done false
15678 4601 Acked Read 5 Sent Done Done false
15679 4602 Acked Flush 0 Sent Done Done false
15680 4603 Acked Read 9 Sent Done Done false
15681 4604 Acked Write 3 Sent Done Done false
15682 4605 Acked Read 10 Sent Done Done false
15683 4606 Acked Flush 0 Sent Done Done false
15684 4607 Acked Read 8 Sent Done Done false
15685 4608 Acked Read 2 Sent Done Done false
15686 4609 Acked Read 8 Sent Done Done false
15687 4610 Acked Write 4 Sent Done Done false
15688 4611 Acked Read 6 Sent Done Done false
15689 4612 Acked Read 5 Sent Done Done false
15690 4613 Acked Flush 0 Sent Done Done false
15691 4614 Acked Write 1 Sent Done Done false
15692 4615 Acked Flush 0 Sent Done Done false
15693 4616 Acked Write 5 Sent Done Done false
15694 4617 Acked Read 4 Sent Done Done false
15695 4618 Acked Read 1 Sent Done Done false
15696 4619 Acked Write 6 Sent Done Done false
15697 4620 Acked Write 2 Sent Done Done false
15698 4621 Acked Flush 0 Sent Done Done false
15699 STATES DS:0 DS:1 DS:2 TOTAL
15700 Sent 302 0 0 302
15701 Done 0 302 302 604
15702 Skipped 0 0 0 0
15703 Error 0 0 0 0
15704 Last Flush: 4319 4621 4621
15705 Downstairs last five completed: 4319 4318 4317 4316 4315
15706 Upstairs last five completed: 4621 4620 4619 4618 4617
15707 CLIENT: Up:0 ds:302 act:2
157082024-11-15T02:00:44.594ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
157092024-11-15T02:00:44.594ZWARNcrucible: ds_connection connect to 127.0.0.1:8810 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
157102024-11-15T02:00:44.594ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
157112024-11-15T02:00:44.594ZWARNcrucible: downstairs task for 0 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
157122024-11-15T02:00:44.594ZINFOcrucible: client task is exiting = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
15713 ----------------------------------------------------------------
15714 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
15715 States: Offline Active Active
15716 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
15717 4320 Acked Write 4 Sent Done Done false
15718 4321 Acked Flush 0 Sent Done Done false
15719 4322 Acked Write 9 Sent Done Done false
15720 4323 Acked Read 10 Sent Done Done false
15721 4324 Acked Write 2 Sent Done Done false
15722 4325 Acked Read 1 Sent Done Done false
15723 4326 Acked Read 9 Sent Done Done false
15724 4327 Acked Write 2 Sent Done Done false
15725 4328 Acked Write 10 Sent Done Done false
15726 4329 Acked Write 3 Sent Done Done false
15727 4330 Acked Read 4 Sent Done Done false
15728 4331 Acked Write 5 Sent Done Done false
15729 4332 Acked Read 3 Sent Done Done false
15730 4333 Acked Flush 0 Sent Done Done false
15731 4334 Acked Read 4 Sent Done Done false
15732 4335 Acked Write 2 Sent Done Done false
15733 4336 Acked Write 8 Sent Done Done false
15734 4337 Acked Read 6 Sent Done Done false
15735 4338 Acked Read 6 Sent Done Done false
15736 4339 Acked Read 2 Sent Done Done false
15737 4340 Acked Read 10 Sent Done Done false
15738 4341 Acked Read 4 Sent Done Done false
15739 4342 Acked Read 5 Sent Done Done false
15740 4343 Acked Flush 0 Sent Done Done false
15741 4344 Acked Write 9 Sent Done Done false
15742 4345 Acked Read 5 Sent Done Done false
15743 4346 Acked Write 1 Sent Done Done false
15744 4347 Acked Write 2 Sent Done Done false
15745 4348 Acked Write 1 Sent Done Done false
15746 4349 Acked Write 10 Sent Done Done false
15747 4350 Acked Write 9 Sent Done Done false
15748 4351 Acked Read 4 Sent Done Done false
15749 4352 Acked Write 10 Sent Done Done false
15750 4353 Acked Write 7 Sent Done Done false
15751 4354 Acked Read 9 Sent Done Done false
15752 4355 Acked Read 5 Sent Done Done false
15753 4356 Acked Write 5 Sent Done Done false
15754 4357 Acked Read 8 Sent Done Done false
15755 4358 Acked Read 5 Sent Done Done false
15756 4359 Acked Flush 0 Sent Done Done false
15757 4360 Acked Write 4 Sent Done Done false
15758 4361 Acked Write 8 Sent Done Done false
15759 4362 Acked Read 1 Sent Done Done false
15760 4363 Acked Read 10 Sent Done Done false
15761 4364 Acked Read 4 Sent Done Done false
15762 4365 Acked Write 8 Sent Done Done false
15763 4366 Acked Write 4 Sent Done Done false
15764 4367 Acked Read 3 Sent Done Done false
15765 4368 Acked Write 9 Sent Done Done false
15766 4369 Acked Read 9 Sent Done Done false
15767 4370 Acked Write 9 Sent Done Done false
15768 4371 Acked Write 5 Sent Done Done false
15769 4372 Acked Read 5 Sent Done Done false
15770 4373 Acked Write 1 Sent Done Done false
15771 4374 Acked Read 1 Sent Done Done false
15772 4375 Acked Write 5 Sent Done Done false
15773 4376 Acked Read 2 Sent Done Done false
15774 4377 Acked Flush 0 Sent Done Done false
15775 4378 Acked Read 5 Sent Done Done false
15776 4379 Acked Read 8 Sent Done Done false
15777 4380 Acked Read 3 Sent Done Done false
15778 4381 Acked Read 7 Sent Done Done false
15779 4382 Acked Read 10 Sent Done Done false
15780 4383 Acked Write 2 Sent Done Done false
15781 4384 Acked Write 7 Sent Done Done false
15782 4385 Acked Write 7 Sent Done Done false
15783 4386 Acked Write 3 Sent Done Done false
15784 4387 Acked Read 6 Sent Done Done false
15785 4388 Acked Read 7 Sent Done Done false
15786 4389 Acked Read 10 Sent Done Done false
15787 4390 Acked Write 8 Sent Done Done false
15788 4391 Acked Write 1 Sent Done Done false
15789 4392 Acked Read 2 Sent Done Done false
15790 4393 Acked Flush 0 Sent Done Done false
15791 4394 Acked Write 8 Sent Done Done false
15792 4395 Acked Flush 0 Sent Done Done false
15793 4396 Acked Read 9 Sent Done Done false
15794 4397 Acked Read 1 Sent Done Done false
15795 4398 Acked Write 1 Sent Done Done false
15796 4399 Acked Read 9 Sent Done Done false
15797 4400 Acked Write 10 Sent Done Done false
15798 4401 Acked Write 8 Sent Done Done false
15799 4402 Acked Read 3 Sent Done Done false
15800 4403 Acked Read 9 Sent Done Done false
15801 4404 Acked Read 2 Sent Done Done false
15802 4405 Acked Write 7 Sent Done Done false
15803 4406 Acked Read 6 Sent Done Done false
15804 4407 Acked Read 9 Sent Done Done false
15805 4408 Acked Read 6 Sent Done Done false
15806 4409 Acked Read 5 Sent Done Done false
15807 4410 Acked Flush 0 Sent Done Done false
15808 4411 Acked Write 1 Sent Done Done false
15809 4412 Acked Read 9 Sent Done Done false
15810 4413 Acked Read 2 Sent Done Done false
15811 4414 Acked Read 10 Sent Done Done false
15812 4415 Acked Read 1 Sent Done Done false
15813 4416 Acked Read 5 Sent Done Done false
15814 4417 Acked Read 10 Sent Done Done false
15815 4418 Acked Read 4 Sent Done Done false
15816 4419 Acked Read 4 Sent Done Done false
15817 4420 Acked Read 5 Sent Done Done false
15818 4421 Acked Write 9 Sent Done Done false
15819 4422 Acked Write 7 Sent Done Done false
15820 4423 Acked Read 3 Sent Done Done false
15821 4424 Acked Flush 0 Sent Done Done false
15822 4425 Acked Read 3 Sent Done Done false
15823 4426 Acked Flush 0 Sent Done Done false
15824 4427 Acked Write 2 Sent Done Done false
15825 4428 Acked Read 4 Sent Done Done false
15826 4429 Acked Write 7 Sent Done Done false
15827 4430 Acked Read 3 Sent Done Done false
15828 4431 Acked Write 10 Sent Done Done false
15829 4432 Acked Read 6 Sent Done Done false
15830 4433 Acked Read 8 Sent Done Done false
15831 4434 Acked Write 9 Sent Done Done false
15832 4435 Acked Write 9 Sent Done Done false
15833 4436 Acked Read 4 Sent Done Done false
15834 4437 Acked Read 10 Sent Done Done false
15835 4438 Acked Write 4 Sent Done Done false
15836 4439 Acked Read 7 Sent Done Done false
15837 4440 Acked Write 1 Sent Done Done false
15838 4441 Acked Read 6 Sent Done Done false
15839 4442 Acked Flush 0 Sent Done Done false
15840 4443 Acked Flush 0 Sent Done Done false
15841 4444 Acked Write 9 Sent Done Done false
15842 4445 Acked Read 1 Sent Done Done false
15843 4446 Acked Read 9 Sent Done Done false
15844 4447 Acked Write 5 Sent Done Done false
15845 4448 Acked Read 10 Sent Done Done false
15846 4449 Acked Read 2 Sent Done Done false
15847 4450 Acked Read 1 Sent Done Done false
15848 4451 Acked Write 9 Sent Done Done false
15849 4452 Acked Read 8 Sent Done Done false
15850 4453 Acked Write 3 Sent Done Done false
15851 4454 Acked Write 6 Sent Done Done false
15852 4455 Acked Read 4 Sent Done Done false
15853 4456 Acked Read 6 Sent Done Done false
15854 4457 Acked Write 4 Sent Done Done false
15855 4458 Acked Read 8 Sent Done Done false
15856 4459 Acked Read 2 Sent Done Done false
15857 4460 Acked Read 5 Sent Done Done false
15858 4461 Acked Write 2 Sent Done Done false
15859 4462 Acked Write 2 Sent Done Done false
15860 4463 Acked Read 7 Sent Done Done false
15861 4464 Acked Read 9 Sent Done Done false
15862 4465 Acked Write 6 Sent Done Done false
15863 4466 Acked Flush 0 Sent Done Done false
15864 4467 Acked Write 4 Sent Done Done false
15865 4468 Acked Write 1 Sent Done Done false
15866 4469 Acked Read 4 Sent Done Done false
15867 4470 Acked Read 5 Sent Done Done false
15868 4471 Acked Read 6 Sent Done Done false
15869 4472 Acked Write 5 Sent Done Done false
15870 4473 Acked Write 2 Sent Done Done false
15871 4474 Acked Read 3 Sent Done Done false
15872 4475 Acked Write 5 Sent Done Done false
15873 4476 Acked Write 10 Sent Done Done false
15874 4477 Acked Read 5 Sent Done Done false
15875 4478 Acked Read 9 Sent Done Done false
15876 4479 Acked Read 5 Sent Done Done false
15877 4480 Acked Write 4 Sent Done Done false
15878 4481 Acked Read 3 Sent Done Done false
15879 4482 Acked Write 10 Sent Done Done false
15880 4483 Acked Write 9 Sent Done Done false
15881 4484 Acked Write 2 Sent Done Done false
15882 4485 Acked Write 6 Sent Done Done false
15883 4486 Acked Write 2 Sent Done Done false
15884 4487 Acked Write 2 Sent Done Done false
15885 4488 Acked Write 6 Sent Done Done false
15886 4489 Acked Read 5 Sent Done Done false
15887 4490 Acked Read 1 Sent Done Done false
15888 4491 Acked Write 5 Sent Done Done false
15889 4492 Acked Read 2 Sent Done Done false
15890 4493 Acked Read 10 Sent Done Done false
15891 4494 Acked Read 4 Sent Done Done false
15892 4495 Acked Flush 0 Sent Done Done false
15893 4496 Acked Write 1 Sent Done Done false
15894 4497 Acked Write 2 Sent Done Done false
15895 4498 Acked Read 10 Sent Done Done false
15896 4499 Acked Read 2 Sent Done Done false
15897 4500 Acked Read 2 Sent Done Done false
15898 4501 Acked Read 5 Sent Done Done false
15899 4502 Acked Read 7 Sent Done Done false
15900 4503 Acked Write 8 Sent Done Done false
15901 4504 Acked Write 9 Sent Done Done false
15902 4505 Acked Write 2 Sent Done Done false
15903 4506 Acked Write 1 Sent Done Done false
15904 4507 Acked Read 5 Sent Done Done false
15905 4508 Acked Flush 0 Sent Done Done false
15906 4509 Acked Read 3 Sent Done Done false
15907 4510 Acked Write 7 Sent Done Done false
15908 4511 Acked Write 6 Sent Done Done false
15909 4512 Acked Read 8 Sent Done Done false
15910 4513 Acked Write 3 Sent Done Done false
15911 4514 Acked Write 5 Sent Done Done false
15912 4515 Acked Read 2 Sent Done Done false
15913 4516 Acked Read 7 Sent Done Done false
15914 4517 Acked Write 8 Sent Done Done false
15915 4518 Acked Write 9 Sent Done Done false
15916 4519 Acked Write 3 Sent Done Done false
15917 4520 Acked Write 8 Sent Done Done false
15918 4521 Acked Write 9 Sent Done Done false
15919 4522 Acked Write 10 Sent Done Done false
15920 4523 Acked Flush 0 Sent Done Done false
15921 4524 Acked Write 6 Sent Done Done false
15922 4525 Acked Read 2 Sent Done Done false
15923 4526 Acked Write 3 Sent Done Done false
15924 4527 Acked Read 4 Sent Done Done false
15925 4528 Acked Write 2 Sent Done Done false
15926 4529 Acked Flush 0 Sent Done Done false
15927 4530 Acked Read 9 Sent Done Done false
15928 4531 Acked Write 2 Sent Done Done false
15929 4532 Acked Read 6 Sent Done Done false
15930 4533 Acked Read 4 Sent Done Done false
15931 4534 Acked Write 4 Sent Done Done false
15932 4535 Acked Write 2 Sent Done Done false
15933 4536 Acked Read 7 Sent Done Done false
15934 4537 Acked Write 5 Sent Done Done false
15935 4538 Acked Write 2 Sent Done Done false
15936 4539 Acked Write 8 Sent Done Done false
15937 4540 Acked Read 4 Sent Done Done false
15938 4541 Acked Write 4 Sent Done Done false
15939 4542 Acked Write 7 Sent Done Done false
15940 4543 Acked Read 6 Sent Done Done false
15941 4544 Acked Read 10 Sent Done Done false
15942 4545 Acked Write 3 Sent Done Done false
15943 4546 Acked Read 10 Sent Done Done false
15944 4547 Acked Write 1 Sent Done Done false
15945 4548 Acked Read 7 Sent Done Done false
15946 4549 Acked Write 4 Sent Done Done false
15947 4550 Acked Write 9 Sent Done Done false
15948 4551 Acked Write 7 Sent Done Done false
15949 4552 Acked Write 10 Sent Done Done false
15950 4553 Acked Flush 0 Sent Done Done false
15951 4554 Acked Read 7 Sent Done Done false
15952 4555 Acked Write 2 Sent Done Done false
15953 4556 Acked Read 4 Sent Done Done false
15954 4557 Acked Flush 0 Sent Done Done false
15955 4558 Acked Write 6 Sent Done Done false
15956 4559 Acked Read 2 Sent Done Done false
15957 4560 Acked Read 6 Sent Done Done false
15958 4561 Acked Read 6 Sent Done Done false
15959 4562 Acked Write 1 Sent Done Done false
15960 4563 Acked Read 2 Sent Done Done false
15961 4564 Acked Read 9 Sent Done Done false
15962 4565 Acked Read 10 Sent Done Done false
15963 4566 Acked Read 8 Sent Done Done false
15964 4567 Acked Write 6 Sent Done Done false
15965 4568 Acked Write 1 Sent Done Done false
15966 4569 Acked Write 3 Sent Done Done false
15967 4570 Acked Write 8 Sent Done Done false
15968 4571 Acked Read 5 Sent Done Done false
15969 4572 Acked Read 10 Sent Done Done false
15970 4573 Acked Flush 0 Sent Done Done false
15971 4574 Acked Read 6 Sent Done Done false
15972 4575 Acked Write 2 Sent Done Done false
15973 4576 Acked Read 5 Sent Done Done false
15974 4577 Acked Flush 0 Sent Done Done false
15975 4578 Acked Write 3 Sent Done Done false
15976 4579 Acked Read 5 Sent Done Done false
15977 4580 Acked Read 6 Sent Done Done false
15978 4581 Acked Write 7 Sent Done Done false
15979 4582 Acked Read 8 Sent Done Done false
15980 4583 Acked Read 2 Sent Done Done false
15981 4584 Acked Read 3 Sent Done Done false
15982 4585 Acked Write 4 Sent Done Done false
15983 4586 Acked Flush 0 Sent Done Done false
15984 4587 Acked Read 3 Sent Done Done false
15985 4588 Acked Read 4 Sent Done Done false
15986 4589 Acked Flush 0 Sent Done Done false
15987 4590 Acked Flush 0 Sent Done Done false
15988 4591 Acked Write 2 Sent Done Done false
15989 4592 Acked Write 8 Sent Done Done false
15990 4593 Acked Read 8 Sent Done Done false
15991 4594 Acked Write 4 Sent Done Done false
15992 4595 Acked Read 2 Sent Done Done false
15993 4596 Acked Read 5 Sent Done Done false
15994 4597 Acked Flush 0 Sent Done Done false
15995 4598 Acked Read 5 Sent Done Done false
15996 4599 Acked Write 4 Sent Done Done false
15997 4600 Acked Read 6 Sent Done Done false
15998 4601 Acked Read 5 Sent Done Done false
15999 4602 Acked Flush 0 Sent Done Done false
16000 4603 Acked Read 9 Sent Done Done false
16001 4604 Acked Write 3 Sent Done Done false
16002 4605 Acked Read 10 Sent Done Done false
16003 4606 Acked Flush 0 Sent Done Done false
16004 4607 Acked Read 8 Sent Done Done false
16005 4608 Acked Read 2 Sent Done Done false
16006 4609 Acked Read 8 Sent Done Done false
16007 4610 Acked Write 4 Sent Done Done false
16008 4611 Acked Read 6 Sent Done Done false
16009 4612 Acked Read 5 Sent Done Done false
16010 4613 Acked Flush 0 Sent Done Done false
16011 4614 Acked Write 1 Sent Done Done false
16012 4615 Acked Flush 0 Sent Done Done false
16013 4616 Acked Write 5 Sent Done Done false
16014 4617 Acked Read 4 Sent Done Done false
16015 4618 Acked Read 1 Sent Done Done false
16016 4619 Acked Write 6 Sent Done Done false
16017 4620 Acked Write 2 Sent Done Done false
16018 4621 Acked Flush 0 Sent Done Done false
16019 STATES DS:0 DS:1 DS:2 TOTAL
16020 Sent 302 0 0 302
16021 Done 0 302 302 604
16022 Skipped 0 0 0 0
16023 Error 0 0 0 0
16024 Last Flush: 4319 4621 4621
16025 Downstairs last five completed: 4319 4318 4317 4316 4315
16026 Upstairs last five completed: 4621 4620 4619 4618 4617
16027 CLIENT: Up:0 ds:302 act:2
16028 [8810] got start action
16029 [8810] I am going to restart
16030 Starting downstairs at port 8810
16031 Make output file at "/tmp/dsc/downstairs-8810.txt"
16032 [8810][0] reports Starting
16033 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1217)
16034 [8810][0] reports Running
16035 ----------------------------------------------------------------
16036 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
16037 States: Offline Active Active
16038 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
16039 4320 Acked Write 4 Sent Done Done false
16040 4321 Acked Flush 0 Sent Done Done false
16041 4322 Acked Write 9 Sent Done Done false
16042 4323 Acked Read 10 Sent Done Done false
16043 4324 Acked Write 2 Sent Done Done false
16044 4325 Acked Read 1 Sent Done Done false
16045 4326 Acked Read 9 Sent Done Done false
16046 4327 Acked Write 2 Sent Done Done false
16047 4328 Acked Write 10 Sent Done Done false
16048 4329 Acked Write 3 Sent Done Done false
16049 4330 Acked Read 4 Sent Done Done false
16050 4331 Acked Write 5 Sent Done Done false
16051 4332 Acked Read 3 Sent Done Done false
16052 4333 Acked Flush 0 Sent Done Done false
16053 4334 Acked Read 4 Sent Done Done false
16054 4335 Acked Write 2 Sent Done Done false
16055 4336 Acked Write 8 Sent Done Done false
16056 4337 Acked Read 6 Sent Done Done false
16057 4338 Acked Read 6 Sent Done Done false
16058 4339 Acked Read 2 Sent Done Done false
16059 4340 Acked Read 10 Sent Done Done false
16060 4341 Acked Read 4 Sent Done Done false
16061 4342 Acked Read 5 Sent Done Done false
16062 4343 Acked Flush 0 Sent Done Done false
16063 4344 Acked Write 9 Sent Done Done false
16064 4345 Acked Read 5 Sent Done Done false
16065 4346 Acked Write 1 Sent Done Done false
16066 4347 Acked Write 2 Sent Done Done false
16067 4348 Acked Write 1 Sent Done Done false
16068 4349 Acked Write 10 Sent Done Done false
16069 4350 Acked Write 9 Sent Done Done false
16070 4351 Acked Read 4 Sent Done Done false
16071 4352 Acked Write 10 Sent Done Done false
16072 4353 Acked Write 7 Sent Done Done false
16073 4354 Acked Read 9 Sent Done Done false
16074 4355 Acked Read 5 Sent Done Done false
16075 4356 Acked Write 5 Sent Done Done false
16076 4357 Acked Read 8 Sent Done Done false
16077 4358 Acked Read 5 Sent Done Done false
16078 4359 Acked Flush 0 Sent Done Done false
16079 4360 Acked Write 4 Sent Done Done false
16080 4361 Acked Write 8 Sent Done Done false
16081 4362 Acked Read 1 Sent Done Done false
16082 4363 Acked Read 10 Sent Done Done false
16083 4364 Acked Read 4 Sent Done Done false
16084 4365 Acked Write 8 Sent Done Done false
16085 4366 Acked Write 4 Sent Done Done false
16086 4367 Acked Read 3 Sent Done Done false
16087 4368 Acked Write 9 Sent Done Done false
16088 4369 Acked Read 9 Sent Done Done false
16089 4370 Acked Write 9 Sent Done Done false
16090 4371 Acked Write 5 Sent Done Done false
16091 4372 Acked Read 5 Sent Done Done false
16092 4373 Acked Write 1 Sent Done Done false
16093 4374 Acked Read 1 Sent Done Done false
16094 4375 Acked Write 5 Sent Done Done false
16095 4376 Acked Read 2 Sent Done Done false
16096 4377 Acked Flush 0 Sent Done Done false
16097 4378 Acked Read 5 Sent Done Done false
16098 4379 Acked Read 8 Sent Done Done false
16099 4380 Acked Read 3 Sent Done Done false
16100 4381 Acked Read 7 Sent Done Done false
16101 4382 Acked Read 10 Sent Done Done false
16102 4383 Acked Write 2 Sent Done Done false
16103 4384 Acked Write 7 Sent Done Done false
16104 4385 Acked Write 7 Sent Done Done false
16105 4386 Acked Write 3 Sent Done Done false
16106 4387 Acked Read 6 Sent Done Done false
16107 4388 Acked Read 7 Sent Done Done false
16108 4389 Acked Read 10 Sent Done Done false
16109 4390 Acked Write 8 Sent Done Done false
16110 4391 Acked Write 1 Sent Done Done false
16111 4392 Acked Read 2 Sent Done Done false
16112 4393 Acked Flush 0 Sent Done Done false
16113 4394 Acked Write 8 Sent Done Done false
16114 4395 Acked Flush 0 Sent Done Done false
16115 4396 Acked Read 9 Sent Done Done false
16116 4397 Acked Read 1 Sent Done Done false
16117 4398 Acked Write 1 Sent Done Done false
16118 4399 Acked Read 9 Sent Done Done false
16119 4400 Acked Write 10 Sent Done Done false
16120 4401 Acked Write 8 Sent Done Done false
16121 4402 Acked Read 3 Sent Done Done false
16122 4403 Acked Read 9 Sent Done Done false
16123 4404 Acked Read 2 Sent Done Done false
16124 4405 Acked Write 7 Sent Done Done false
16125 4406 Acked Read 6 Sent Done Done false
16126 4407 Acked Read 9 Sent Done Done false
16127 4408 Acked Read 6 Sent Done Done false
16128 4409 Acked Read 5 Sent Done Done false
16129 4410 Acked Flush 0 Sent Done Done false
16130 4411 Acked Write 1 Sent Done Done false
16131 4412 Acked Read 9 Sent Done Done false
16132 4413 Acked Read 2 Sent Done Done false
16133 4414 Acked Read 10 Sent Done Done false
16134 4415 Acked Read 1 Sent Done Done false
16135 4416 Acked Read 5 Sent Done Done false
16136 4417 Acked Read 10 Sent Done Done false
16137 4418 Acked Read 4 Sent Done Done false
16138 4419 Acked Read 4 Sent Done Done false
16139 4420 Acked Read 5 Sent Done Done false
16140 4421 Acked Write 9 Sent Done Done false
16141 4422 Acked Write 7 Sent Done Done false
16142 4423 Acked Read 3 Sent Done Done false
16143 4424 Acked Flush 0 Sent Done Done false
16144 4425 Acked Read 3 Sent Done Done false
16145 4426 Acked Flush 0 Sent Done Done false
16146 4427 Acked Write 2 Sent Done Done false
16147 4428 Acked Read 4 Sent Done Done false
16148 4429 Acked Write 7 Sent Done Done false
16149 4430 Acked Read 3 Sent Done Done false
16150 4431 Acked Write 10 Sent Done Done false
16151 4432 Acked Read 6 Sent Done Done false
16152 4433 Acked Read 8 Sent Done Done false
16153 4434 Acked Write 9 Sent Done Done false
16154 4435 Acked Write 9 Sent Done Done false
16155 4436 Acked Read 4 Sent Done Done false
16156 4437 Acked Read 10 Sent Done Done false
16157 4438 Acked Write 4 Sent Done Done false
16158 4439 Acked Read 7 Sent Done Done false
16159 4440 Acked Write 1 Sent Done Done false
16160 4441 Acked Read 6 Sent Done Done false
16161 4442 Acked Flush 0 Sent Done Done false
16162 4443 Acked Flush 0 Sent Done Done false
16163 4444 Acked Write 9 Sent Done Done false
16164 4445 Acked Read 1 Sent Done Done false
16165 4446 Acked Read 9 Sent Done Done false
16166 4447 Acked Write 5 Sent Done Done false
16167 4448 Acked Read 10 Sent Done Done false
16168 4449 Acked Read 2 Sent Done Done false
16169 4450 Acked Read 1 Sent Done Done false
16170 4451 Acked Write 9 Sent Done Done false
16171 4452 Acked Read 8 Sent Done Done false
16172 4453 Acked Write 3 Sent Done Done false
16173 4454 Acked Write 6 Sent Done Done false
16174 4455 Acked Read 4 Sent Done Done false
16175 4456 Acked Read 6 Sent Done Done false
16176 4457 Acked Write 4 Sent Done Done false
16177 4458 Acked Read 8 Sent Done Done false
16178 4459 Acked Read 2 Sent Done Done false
16179 4460 Acked Read 5 Sent Done Done false
16180 4461 Acked Write 2 Sent Done Done false
16181 4462 Acked Write 2 Sent Done Done false
16182 4463 Acked Read 7 Sent Done Done false
16183 4464 Acked Read 9 Sent Done Done false
16184 4465 Acked Write 6 Sent Done Done false
16185 4466 Acked Flush 0 Sent Done Done false
16186 4467 Acked Write 4 Sent Done Done false
16187 4468 Acked Write 1 Sent Done Done false
16188 4469 Acked Read 4 Sent Done Done false
16189 4470 Acked Read 5 Sent Done Done false
16190 4471 Acked Read 6 Sent Done Done false
16191 4472 Acked Write 5 Sent Done Done false
16192 4473 Acked Write 2 Sent Done Done false
16193 4474 Acked Read 3 Sent Done Done false
16194 4475 Acked Write 5 Sent Done Done false
16195 4476 Acked Write 10 Sent Done Done false
16196 4477 Acked Read 5 Sent Done Done false
16197 4478 Acked Read 9 Sent Done Done false
16198 4479 Acked Read 5 Sent Done Done false
16199 4480 Acked Write 4 Sent Done Done false
16200 4481 Acked Read 3 Sent Done Done false
16201 4482 Acked Write 10 Sent Done Done false
16202 4483 Acked Write 9 Sent Done Done false
16203 4484 Acked Write 2 Sent Done Done false
16204 4485 Acked Write 6 Sent Done Done false
16205 4486 Acked Write 2 Sent Done Done false
16206 4487 Acked Write 2 Sent Done Done false
16207 4488 Acked Write 6 Sent Done Done false
16208 4489 Acked Read 5 Sent Done Done false
16209 4490 Acked Read 1 Sent Done Done false
16210 4491 Acked Write 5 Sent Done Done false
16211 4492 Acked Read 2 Sent Done Done false
16212 4493 Acked Read 10 Sent Done Done false
16213 4494 Acked Read 4 Sent Done Done false
16214 4495 Acked Flush 0 Sent Done Done false
16215 4496 Acked Write 1 Sent Done Done false
16216 4497 Acked Write 2 Sent Done Done false
16217 4498 Acked Read 10 Sent Done Done false
16218 4499 Acked Read 2 Sent Done Done false
16219 4500 Acked Read 2 Sent Done Done false
16220 4501 Acked Read 5 Sent Done Done false
16221 4502 Acked Read 7 Sent Done Done false
16222 4503 Acked Write 8 Sent Done Done false
16223 4504 Acked Write 9 Sent Done Done false
16224 4505 Acked Write 2 Sent Done Done false
16225 4506 Acked Write 1 Sent Done Done false
16226 4507 Acked Read 5 Sent Done Done false
16227 4508 Acked Flush 0 Sent Done Done false
16228 4509 Acked Read 3 Sent Done Done false
16229 4510 Acked Write 7 Sent Done Done false
16230 4511 Acked Write 6 Sent Done Done false
16231 4512 Acked Read 8 Sent Done Done false
16232 4513 Acked Write 3 Sent Done Done false
16233 4514 Acked Write 5 Sent Done Done false
16234 4515 Acked Read 2 Sent Done Done false
16235 4516 Acked Read 7 Sent Done Done false
16236 4517 Acked Write 8 Sent Done Done false
16237 4518 Acked Write 9 Sent Done Done false
16238 4519 Acked Write 3 Sent Done Done false
16239 4520 Acked Write 8 Sent Done Done false
16240 4521 Acked Write 9 Sent Done Done false
16241 4522 Acked Write 10 Sent Done Done false
16242 4523 Acked Flush 0 Sent Done Done false
16243 4524 Acked Write 6 Sent Done Done false
16244 4525 Acked Read 2 Sent Done Done false
16245 4526 Acked Write 3 Sent Done Done false
16246 4527 Acked Read 4 Sent Done Done false
16247 4528 Acked Write 2 Sent Done Done false
16248 4529 Acked Flush 0 Sent Done Done false
16249 4530 Acked Read 9 Sent Done Done false
16250 4531 Acked Write 2 Sent Done Done false
16251 4532 Acked Read 6 Sent Done Done false
16252 4533 Acked Read 4 Sent Done Done false
16253 4534 Acked Write 4 Sent Done Done false
16254 4535 Acked Write 2 Sent Done Done false
16255 4536 Acked Read 7 Sent Done Done false
16256 4537 Acked Write 5 Sent Done Done false
16257 4538 Acked Write 2 Sent Done Done false
16258 4539 Acked Write 8 Sent Done Done false
16259 4540 Acked Read 4 Sent Done Done false
16260 4541 Acked Write 4 Sent Done Done false
16261 4542 Acked Write 7 Sent Done Done false
16262 4543 Acked Read 6 Sent Done Done false
16263 4544 Acked Read 10 Sent Done Done false
16264 4545 Acked Write 3 Sent Done Done false
16265 4546 Acked Read 10 Sent Done Done false
16266 4547 Acked Write 1 Sent Done Done false
16267 4548 Acked Read 7 Sent Done Done false
16268 4549 Acked Write 4 Sent Done Done false
16269 4550 Acked Write 9 Sent Done Done false
16270 4551 Acked Write 7 Sent Done Done false
16271 4552 Acked Write 10 Sent Done Done false
16272 4553 Acked Flush 0 Sent Done Done false
16273 4554 Acked Read 7 Sent Done Done false
16274 4555 Acked Write 2 Sent Done Done false
16275 4556 Acked Read 4 Sent Done Done false
16276 4557 Acked Flush 0 Sent Done Done false
16277 4558 Acked Write 6 Sent Done Done false
16278 4559 Acked Read 2 Sent Done Done false
16279 4560 Acked Read 6 Sent Done Done false
16280 4561 Acked Read 6 Sent Done Done false
16281 4562 Acked Write 1 Sent Done Done false
16282 4563 Acked Read 2 Sent Done Done false
16283 4564 Acked Read 9 Sent Done Done false
16284 4565 Acked Read 10 Sent Done Done false
16285 4566 Acked Read 8 Sent Done Done false
16286 4567 Acked Write 6 Sent Done Done false
16287 4568 Acked Write 1 Sent Done Done false
16288 4569 Acked Write 3 Sent Done Done false
16289 4570 Acked Write 8 Sent Done Done false
16290 4571 Acked Read 5 Sent Done Done false
16291 4572 Acked Read 10 Sent Done Done false
16292 4573 Acked Flush 0 Sent Done Done false
16293 4574 Acked Read 6 Sent Done Done false
16294 4575 Acked Write 2 Sent Done Done false
16295 4576 Acked Read 5 Sent Done Done false
16296 4577 Acked Flush 0 Sent Done Done false
16297 4578 Acked Write 3 Sent Done Done false
16298 4579 Acked Read 5 Sent Done Done false
16299 4580 Acked Read 6 Sent Done Done false
16300 4581 Acked Write 7 Sent Done Done false
16301 4582 Acked Read 8 Sent Done Done false
16302 4583 Acked Read 2 Sent Done Done false
16303 4584 Acked Read 3 Sent Done Done false
16304 4585 Acked Write 4 Sent Done Done false
16305 4586 Acked Flush 0 Sent Done Done false
16306 4587 Acked Read 3 Sent Done Done false
16307 4588 Acked Read 4 Sent Done Done false
16308 4589 Acked Flush 0 Sent Done Done false
16309 4590 Acked Flush 0 Sent Done Done false
16310 4591 Acked Write 2 Sent Done Done false
16311 4592 Acked Write 8 Sent Done Done false
16312 4593 Acked Read 8 Sent Done Done false
16313 4594 Acked Write 4 Sent Done Done false
16314 4595 Acked Read 2 Sent Done Done false
16315 4596 Acked Read 5 Sent Done Done false
16316 4597 Acked Flush 0 Sent Done Done false
16317 4598 Acked Read 5 Sent Done Done false
16318 4599 Acked Write 4 Sent Done Done false
16319 4600 Acked Read 6 Sent Done Done false
16320 4601 Acked Read 5 Sent Done Done false
16321 4602 Acked Flush 0 Sent Done Done false
16322 4603 Acked Read 9 Sent Done Done false
16323 4604 Acked Write 3 Sent Done Done false
16324 4605 Acked Read 10 Sent Done Done false
16325 4606 Acked Flush 0 Sent Done Done false
16326 4607 Acked Read 8 Sent Done Done false
16327 4608 Acked Read 2 Sent Done Done false
16328 4609 Acked Read 8 Sent Done Done false
16329 4610 Acked Write 4 Sent Done Done false
16330 4611 Acked Read 6 Sent Done Done false
16331 4612 Acked Read 5 Sent Done Done false
16332 4613 Acked Flush 0 Sent Done Done false
16333 4614 Acked Write 1 Sent Done Done false
16334 4615 Acked Flush 0 Sent Done Done false
16335 4616 Acked Write 5 Sent Done Done false
16336 4617 Acked Read 4 Sent Done Done false
16337 4618 Acked Read 1 Sent Done Done false
16338 4619 Acked Write 6 Sent Done Done false
16339 4620 Acked Write 2 Sent Done Done false
16340 4621 Acked Flush 0 Sent Done Done false
16341 STATES DS:0 DS:1 DS:2 TOTAL
16342 Sent 302 0 0 302
16343 Done 0 302 302 604
16344 Skipped 0 0 0 0
16345 Error 0 0 0 0
16346 Last Flush: 4319 4621 4621
16347 Downstairs last five completed: 4319 4318 4317 4316 4315
16348 Upstairs last five completed: 4621 4620 4619 4618 4617
16349 CLIENT: Up:0 ds:302 act:2
163502024-11-15T02:00:54.596ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
163512024-11-15T02:00:54.596ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:52790) = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
163522024-11-15T02:00:54.597ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
163532024-11-15T02:00:54.597ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
163542024-11-15T02:00:54.597ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
163552024-11-15T02:00:54.597ZINFOcrucible: send last flush ID to this DS: 4319 = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
163562024-11-15T02:00:54.598ZINFOcrucible: Replied this last flush ID: 4319 = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
163572024-11-15T02:00:54.598ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
163582024-11-15T02:00:54.598ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
163592024-11-15T02:00:54.598ZINFOcrucible: [0] client re-new 302 jobs since flush 4319 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
163602024-11-15T02:00:54.598ZINFOcrucible: [0] Replayed 302 jobs since flush: 4319 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
16361 ----------------------------------------------------------------
16362 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
16363 Downstairs last five completed: 4621 4620 4619 4618 4617
16364 Upstairs last five completed: 4621 4620 4619 4618 4617
16365 CLIENT: Up:0 ds:0 act:3
16366 Replay: All jobs finished
16367 Main task has work to do, go find it
16368 got dsc Stop(1)
16369 stop 1
16370 [8820] Got stop action so:false kr:false
16371 Nov 15 02:00:54.835 INFO request completed, latency_us: 462, response_code: 204, uri: /stop/cid/1, method: POST, req_id: e10585e3-6e65-4949-9626-59bec9c5aaa6, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
163722024-11-15T02:00:54.835ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
16373 Nov 15 02:00:54.836{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2024-11-15T02:00:54.836001995Z","hostname":"w-01JCPRHT5F2MPA44KVJHM3S2PB","pid":1205,"":"io task","client":"1","":"downstairs","session_id":"971dcb2e-6891-4ecf-b367-1e04e52065ee"}
16374 INFO request completed, latency_us: 189, response_code: 200, uri: /state/cid/1, method: GET, req_id: 628ccc6e-148c-4eb9-a589-30c91fe66b34, remote_addr{"msg":"downstairs task for 1 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2024-11-15T02:00:54.836053301Z","hostname":"w-01JCPRHT5F2MPA44KVJHM3S2PB","pid":1205,"session_id":"971dcb2e-6891-4ecf-b367-1e04e52065ee"}
16375 : 127.0.0.1:40008, local_addr: 127.0.0.1:9998
163762024-11-15T02:00:54.836ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
163772024-11-15T02:00:54.836ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
163782024-11-15T02:00:54.836ZINFOcrucible: client task is exiting = io task client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
16379 [8820] Exited with: ExitStatus(unix_wait_status(9))
16380 [8820][1] reports Exit
16381 Nov 15 02:00:58.838 INFO request completed, latency_us: 214, response_code: 200, uri: /state/cid/1, method: GET, req_id: 0d227873-08c9-4fc0-b330-b16138a792dd, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
16382 001/300 Read block 1768 len 20480
16383 002/300 Read block 4598 len 24576
16384 003/300 Write block 3267 len 8192 data: 2 3
16385 004/300 Write block 4096 len 40960 data: 3 3 2 2 2 3 3 3 3 3
16386 005/300 Read block 1479 len 32768
16387 006/300 Read block 348 len 16384
16388 007/300 Write block 2600 len 16384 data: 5 5 5 5
16389 008/300 Read block 3897 len 20480
16390 009/300 Read block 651 len 28672
16391 010/300 Write block 2814 len 28672 data: 2 3 3 4 5 5 4
16392 011/300 Read block 2777 len 24576
16393 012/300 Read block 451 len 8192
16394 013/300 Write block 4976 len 36864 data: 2 2 2 2 2 2 2 2 2
16395 014/300 Read block 1628 len 36864
16396 015/300 Read block 1081 len 24576
16397 016/300 Flush
16398 017/300 Read block 2054 len 20480
16399 018/300 Read block 2280 len 12288
16400 019/300 Write block 4260 len 12288 data: 5 4 4
16401 020/300 Read block 1327 len 32768
16402 021/300 Write block 2495 len 36864 data: 4 5 4 5 5 5 4 2 2
16403 022/300 Write block 4893 len 40960 data: 6 6 5 4 4 3 2 3 4 5
16404 023/300 Read block 855 len 12288
16405 024/300 Read block 3388 len 32768
16406 025/300 Read block 2651 len 12288
16407 026/300 Write block 4527 len 4096 data: 4
16408 027/300 Write block 4366 len 32768 data: 3 4 3 2 3 3 3 3
16409 028/300 Flush
16410 029/300 Read block 522 len 8192
16411 030/300 Write block 829 len 12288 data: 6 4 4
16412 031/300 Read block 741 len 24576
16413 032/300 Write block 829 len 8192 data: 7 5
16414 033/300 Write block 620 len 20480 data: 2 2 2 3 3
16415 034/300 Read block 3160 len 24576
16416 035/300 Write block 2958 len 28672 data: 5 4 2 2 2 3 2
16417 036/300 Read block 2310 len 28672
16418 037/300 Read block 3530 len 16384
16419 038/300 Read block 588 len 20480
16420 039/300 Flush
16421 040/300 Read block 4538 len 40960
16422 041/300 Read block 3618 len 40960
16423 042/300 Read block 225 len 20480
16424 043/300 Read block 1543 len 8192
16425 044/300 Read block 1532 len 16384
16426 045/300 Read block 3548 len 32768
16427 046/300 Write block 403 len 24576 data: 2 2 3 3 3 3
16428 047/300 Flush
16429 048/300 Read block 685 len 24576
16430 049/300 Write block 2471 len 20480 data: 4 4 5 5 5
16431 050/300 Write block 4305 len 20480 data: 4 5 4 6 6
16432 051/300 Read block 4620 len 24576
16433 052/300 Read block 3237 len 32768
16434 053/300 Write block 621 len 28672 data: 3 3 4 4 3 2 2
16435 054/300 Write block 3686 len 28672 data: 3 5 5 5 5 5 4
16436 055/300 Write block 4287 len 8192 data: 4 4
16437 056/300 Write block 2907 len 20480 data: 2 2 2 2 2
16438 057/300 Read block 2152 len 20480
16439 058/300 Write block 1344 len 28672 data: 4 3 3 3 2 2 3
16440 059/300 Write block 467 len 28672 data: 4 4 3 3 3 3 4
16441 060/300 Read block 1930 len 12288
16442 061/300 Write block 448 len 4096 data: 3
16443 062/300 Write block 3395 len 4096 data: 4
16444 063/300 Read block 3912 len 24576
16445 064/300 Read block 3295 len 40960
16446 065/300 Write block 1373 len 12288 data: 5 4 4
16447 066/300 Read block 1156 len 24576
16448 067/300 Read block 4980 len 4096
16449 068/300 Write block 3707 len 40960 data: 3 4 5 4 4 4 3 3 3 3
16450 069/300 Read block 4867 len 12288
16451 070/300 Flush
16452 071/300 Read block 2531 len 16384
16453 072/300 Read block 2003 len 36864
16454 073/300 Write block 4063 len 36864 data: 3 3 3 3 3 3 3 3 3
16455 074/300 Write block 4394 len 24576 data: 4 4 3 2 2 2
16456 075/300 Write block 1641 len 12288 data: 3 3 3
16457 076/300 Read block 3815 len 20480
16458 077/300 Write block 4470 len 4096 data: 3
16459 078/300 Write block 4581 len 20480 data: 3 2 2 3 4
16460 079/300 Flush
16461 080/300 Read block 3194 len 4096
16462 081/300 Read block 2088 len 36864
16463 082/300 Read block 1965 len 28672
16464 083/300 Write block 363 len 4096 data: 3
16465 084/300 Write block 3647 len 36864 data: 4 5 6 6 6 5 5 4 4
16466 085/300 Read block 560 len 36864
16467 086/300 Write block 4158 len 16384 data: 4 4 5 3
16468 087/300 Read block 2213 len 24576
16469 088/300 Flush
16470 089/300 Flush
16471 090/300 Read block 633 len 20480
16472 091/300 Flush
16473 092/300 Flush
16474 093/300 Write block 1605 len 16384 data: 4 5 4 3
16475 094/300 Read block 3119 len 28672
16476 095/300 Write block 1676 len 4096 data: 3
16477 096/300 Write block 1059 len 4096 data: 3
16478 097/300 Read block 4730 len 16384
16479 098/300 Read block 2942 len 20480
16480 099/300 Read block 345 len 28672
16481 100/300 Write block 1340 len 32768 data: 4 4 4 4 5 4 4 4
16482 101/300 Read block 1456 len 32768
16483 102/300 Read block 1285 len 8192
16484 103/300 Write block 4199 len 36864 data: 4 3 2 4 4 4 5 7 6
16485 104/300 Flush
16486 105/300 Flush
16487 106/300 Write block 4281 len 8192 data: 4 5
16488 107/300 Read block 4867 len 8192
16489 108/300 Flush
16490 109/300 Write block 3551 len 8192 data: 3 5
16491 110/300 Write block 991 len 28672 data: 6 6 6 6 6 6 5
16492 111/300 Flush
16493 112/300 Read block 3080 len 24576
16494 113/300 Flush
16495 114/300 Read block 2248 len 20480
16496 115/300 Read block 3421 len 24576
16497 116/300 Flush
16498 117/300 Write block 1954 len 32768 data: 3 3 3 4 4 4 3 3
16499 118/300 Read block 932 len 24576
16500 119/300 Read block 145 len 16384
16501 120/300 Write block 4634 len 24576 data: 3 3 2 2 2 2
16502 121/300 Write block 1874 len 40960 data: 4 4 4 3 3 3 4 5 4 4
16503 122/300 Read block 373 len 36864
16504 123/300 Write block 3118 len 4096 data: 3
16505 124/300 Read block 4938 len 36864
16506 125/300 Read block 3719 len 20480
16507 126/300 Flush
16508 127/300 Read block 4870 len 12288
16509 128/300 Read block 3793 len 20480
16510 129/300 Flush
16511 130/300 Flush
16512 131/300 Read block 4108 len 28672
16513 132/300 Read block 4670 len 28672
16514 133/300 Read block 1264 len 40960
16515 134/300 Read block 4690 len 40960
16516 135/300 Write block 2557 len 12288 data: 3 4 3
16517 136/300 Read block 488 len 4096
16518 137/300 Read block 2009 len 28672
16519 138/300 Write block 1631 len 36864 data: 3 3 3 3 3 3 3 4 4
16520 139/300 Read block 562 len 36864
16521 140/300 Read block 603 len 32768
16522 141/300 Read block 2459 len 12288
16523 142/300 Flush
16524 143/300 Read block 1245 len 16384
16525 144/300 Write block 2163 len 16384 data: 4 3 3 3
16526 145/300 Write block 2960 len 32768 data: 3 3 3 4 3 2 2 2
16527 146/300 Write block 2742 len 24576 data: 3 3 3 4 4 3
16528 147/300 Write block 3996 len 12288 data: 2 2 2
16529 148/300 Read block 2546 len 4096
16530 149/300 Read block 2734 len 28672
16531 150/300 Write block 2491 len 40960 data: 5 5 4 4 5 6 5 6 6 6
16532 151/300 Flush
16533 152/300 Read block 3805 len 28672
16534 153/300 Read block 419 len 20480
16535 154/300 Read block 581 len 40960
16536 155/300 Write block 2685 len 16384 data: 3 3 3 4
16537 156/300 Write block 3594 len 32768 data: 4 4 4 3 3 3 4 4
16538 157/300 Write block 159 len 32768 data: 3 3 3 3 4 5 5 4
16539 158/300 Write block 2473 len 40960 data: 6 6 6 4 4 3 3 2 3 5
16540 159/300 Write block 862 len 16384 data: 2 2 2 2
16541 160/300 Write block 4291 len 36864 data: 4 3 3 2 2 2 2 4 5
16542 161/300 Write block 1930 len 20480 data: 4 4 6 6 7
16543 162/300 Read block 3276 len 16384
16544 163/300 Write block 2055 len 4096 data: 4
16545 164/300 Flush
16546 165/300 Read block 4236 len 12288
16547 166/300 Read block 952 len 4096
16548 167/300 Read block 3428 len 40960
16549 168/300 Read block 1632 len 36864
16550 169/300 Write block 1012 len 12288 data: 4 5 5
16551 170/300 Read block 1993 len 40960
16552 171/300 Read block 1944 len 4096
16553 172/300 Write block 1961 len 4096 data: 4
16554 173/300 Read block 2962 len 24576
16555 174/300 Read block 1647 len 40960
16556 175/300 Read block 1517 len 24576
16557 176/300 Write block 2646 len 4096 data: 3
16558 177/300 Flush
16559 178/300 Read block 3652 len 4096
16560 179/300 Write block 3906 len 8192 data: 7 5
16561 180/300 Write block 2525 len 32768 data: 4 3 5 6 6 5 6 6
16562 181/300 Read block 1258 len 32768
16563 182/300 Read block 1776 len 20480
16564 183/300 Write block 3000 len 4096 data: 4
16565 184/300 Write block 2633 len 4096 data: 4
16566 185/300 Read block 4648 len 16384
16567 186/300 Write block 1788 len 8192 data: 4 4
16568 187/300 Flush
16569 188/300 Read block 3126 len 8192
16570 189/300 Write block 1313 len 20480 data: 3 6 6 4 5
16571 190/300 Write block 2403 len 12288 data: 3 3 3
16572 191/300 Flush
16573 192/300 Write block 80 len 20480 data: 8 8 7 6 6
16574 193/300 Write block 4730 len 36864 data: 5 4 2 2 2 2 2 2 3
16575 194/300 Read block 1667 len 24576
16576 195/300 Write block 4662 len 24576 data: 5 4 5 7 4 4
16577 196/300 Read block 1076 len 12288
16578 197/300 Flush
16579 198/300 Write block 1736 len 16384 data: 4 3 3 3
16580 199/300 Write block 255 len 20480 data: 2 3 3 4 4
16581 200/300 Write block 1837 len 24576 data: 3 3 3 3 4 4
16582 201/300 Read block 668 len 24576
16583 202/300 Flush
16584 203/300 Flush
16585 204/300 Read block 2020 len 36864
16586 205/300 Write block 2943 len 24576 data: 6 6 6 7 7 5
16587 206/300 Read block 252 len 16384
16588 207/300 Write block 2360 len 28672 data: 5 5 5 4 3 5 5
16589 208/300 Write block 260 len 40960 data: 4 4 3 3 2 2 2 2 2 2
16590 209/300 Write block 4678 len 12288 data: 3 3 3
16591 210/300 Read block 2367 len 8192
16592 211/300 Write block 4859 len 20480 data: 3 3 3 3 3
16593 212/300 Read block 2147 len 8192
16594 213/300 Write block 1282 len 28672 data: 5 5 3 3 3 2 4
16595 214/300 Read block 3957 len 28672
16596 215/300 Read block 3356 len 40960
16597 216/300 Write block 4181 len 20480 data: 4 4 4 5 5
16598 217/300 Read block 4301 len 32768
16599 218/300 Write block 3097 len 12288 data: 3 3 3
16600 219/300 Read block 3009 len 36864
16601 220/300 Flush
16602 221/300 Read block 1584 len 16384
16603 222/300 Read block 4269 len 12288
16604 223/300 Write block 4038 len 32768 data: 7 7 7 6 5 5 4 4
16605 224/300 Read block 4353 len 36864
16606 225/300 Flush
16607 226/300 Flush
16608 227/300 Read block 2895 len 32768
16609 228/300 Write block 649 len 20480 data: 3 3 5 5 6
16610 229/300 Write block 1981 len 16384 data: 3 4 4 4
16611 230/300 Read block 2164 len 16384
16612 231/300 Read block 3538 len 32768
16613 232/300 Read block 3386 len 32768
16614 233/300 Write block 4193 len 4096 data: 5
16615 234/300 Write block 1875 len 16384 data: 5 5 4 4
16616 235/300 Read block 1332 len 8192
16617 236/300 Write block 2469 len 40960 data: 3 4 5 5 7 7 7 5 5 4
16618 237/300 Read block 2906 len 40960
16619 238/300 Write block 2068 len 28672 data: 2 3 3 3 3 3 4
16620 239/300 Write block 2611 len 36864 data: 3 4 4 4 4 4 4 4 3
16621 240/300 Write block 415 len 4096 data: 4
16622 241/300 Write block 4319 len 32768 data: 5 5 5 4 5 5 4 4
16623 242/300 Write block 1524 len 20480 data: 3 3 3 3 3
16624 243/300 Flush
16625 244/300 Write block 2159 len 24576 data: 4 4 4 4 5 4
16626 245/300 Write block 4272 len 16384 data: 4 3 2 2
16627 246/300 Read block 2827 len 40960
16628 247/300 Write block 1192 len 32768 data: 3 4 5 5 8 8 8 8
16629 248/300 Read block 4693 len 36864
16630 249/300 Read block 2153 len 24576
16631 250/300 Read block 1206 len 40960
16632 251/300 Read block 2413 len 32768
16633 252/300 Write block 323 len 12288 data: 2 2 4
16634 253/300 Read block 1578 len 8192
16635 254/300 Read block 3414 len 32768
16636 255/300 Read block 2842 len 4096
16637 256/300 Flush
16638 257/300 Read block 2905 len 40960
16639 258/300 Read block 932 len 4096
16640 259/300 Write block 1229 len 32768 data: 3 3 3 2 2 2 2 2
16641 260/300 Write block 1463 len 40960 data: 4 5 4 4 4 3 3 3 3 2
16642 261/300 Read block 30 len 12288
16643 262/300 Write block 828 len 28672 data: 6 8 6 5 3 3 3
16644 263/300 Write block 4441 len 12288 data: 4 5 4
16645 264/300 Read block 1469 len 8192
16646 265/300 Read block 3095 len 28672
16647 266/300 Write block 935 len 28672 data: 3 3 4 4 3 3 4
16648 267/300 Read block 1738 len 12288
16649 268/300 Read block 4871 len 4096
16650 269/300 Write block 4664 len 20480 data: 6 8 5 5 4
16651 270/300 Read block 3902 len 40960
16652 271/300 Read block 1731 len 8192
16653 272/300 Read block 1182 len 20480
16654 273/300 Flush
16655 274/300 Read block 2601 len 24576
16656 275/300 Read block 110 len 28672
16657 276/300 Write block 4665 len 8192 data: 9 6
16658 277/300 Flush
16659 278/300 Read block 838 len 28672
16660 279/300 Write block 2473 len 28672 data: 8 8 8 6 6 5 4
16661 280/300 Read block 817 len 4096
16662 281/300 Read block 2065 len 20480
16663 282/300 Write block 2061 len 12288 data: 6 7 6
16664 283/300 Write block 849 len 24576 data: 2 2 4 4 4 4
16665 284/300 Read block 2667 len 4096
16666 285/300 Read block 2632 len 16384
16667 286/300 Read block 1644 len 12288
16668 287/300 Write block 1008 len 32768 data: 3 3 3 4 5 6 6 6
16669 288/300 Flush
16670 289/300 Read block 4051 len 32768
16671 290/300 Write block 3275 len 28672 data: 2 2 2 2 2 3 3
16672 291/300 Read block 832 len 28672
16673 292/300 Write block 1497 len 4096 data: 3
16674 293/300 Read block 4966 len 40960
16675 294/300 Read block 1264 len 20480
16676 295/300 Write block 1767 len 4096 data: 4
16677 296/300 Flush
16678 297/300 Write block 3347 len 36864 data: 3 4 5 6 6 7 6 6 5
16679 298/300 Write block 2148 len 4096 data: 4
16680 299/300 Write block 99 len 24576 data: 2 2 2 3 3 3
16681 300/300 Read block 1799 len 16384
16682 301/300 Read block 1328 len 32768
16683 Main task has work to do, go find it
16684 got dsc Start(1)
16685 start 1
16686 Nov 15 02:00:59.075 INFO request completed, latency_us: 225, response_code: 204, uri: /start/cid/1, method: POST, req_id: 25a5b6b8-8f33-455c-acf7-3fd28b0958ca, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
16687 Replay: started 1, returned:Ok(())
16688 ----------------------------------------------------------------
16689 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
16690 States: Active Offline Active
16691 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
16692 4622 Acked Read 5 Done Sent Done false
16693 4623 Acked Read 6 Done Sent Done false
16694 4624 Acked Write 2 Done Sent Done false
16695 4625 Acked Write 10 Done Sent Done false
16696 4626 Acked Read 8 Done Sent Done false
16697 4627 Acked Read 4 Done Sent Done false
16698 4628 Acked Write 4 Done Sent Done false
16699 4629 Acked Read 5 Done Sent Done false
16700 4630 Acked Read 7 Done Sent Done false
16701 4631 Acked Write 7 Done Sent Done false
16702 4632 Acked Read 6 Done Sent Done false
16703 4633 Acked Read 2 Done Sent Done false
16704 4634 Acked Write 9 Done Sent Done false
16705 4635 Acked Read 9 Done Sent Done false
16706 4636 Acked Read 6 Done Sent Done false
16707 4637 Acked Flush 0 Done Sent Done false
16708 4638 Acked Read 5 Done Sent Done false
16709 4639 Acked Read 3 Done Sent Done false
16710 4640 Acked Write 3 Done Sent Done false
16711 4641 Acked Read 8 Done Sent Done false
16712 4642 Acked Write 9 Done Sent Done false
16713 4643 Acked Write 10 Done Sent Done false
16714 4644 Acked Read 3 Done Sent Done false
16715 4645 Acked Read 8 Done Sent Done false
16716 4646 Acked Read 3 Done Sent Done false
16717 4647 Acked Write 1 Done Sent Done false
16718 4648 Acked Write 8 Done Sent Done false
16719 4649 Acked Flush 0 Done Sent Done false
16720 4650 Acked Read 2 Done Sent Done false
16721 4651 Acked Write 3 Done Sent Done false
16722 4652 Acked Read 6 Done Sent Done false
16723 4653 Acked Write 2 Done Sent Done false
16724 4654 Acked Write 5 Done Sent Done false
16725 4655 Acked Read 6 Done Sent Done false
16726 4656 Acked Write 7 Done Sent Done false
16727 4657 Acked Read 7 Done Sent Done false
16728 4658 Acked Read 4 Done Sent Done false
16729 4659 Acked Read 5 Done Sent Done false
16730 4660 Acked Flush 0 Done Sent Done false
16731 4661 Acked Read 10 Done Sent Done false
16732 4662 Acked Read 10 Done Sent Done false
16733 4663 Acked Read 5 Done Sent Done false
16734 4664 Acked Read 2 Done Sent Done false
16735 4665 Acked Read 4 Done Sent Done false
16736 4666 Acked Read 8 Done Sent Done false
16737 4667 Acked Write 6 Done Sent Done false
16738 4668 Acked Flush 0 Done Sent Done false
16739 4669 Acked Read 6 Done Sent Done false
16740 4670 Acked Write 5 Done Sent Done false
16741 4671 Acked Write 5 Done Sent Done false
16742 4672 Acked Read 6 Done Sent Done false
16743 4673 Acked Read 8 Done Sent Done false
16744 4674 Acked Write 7 Done Sent Done false
16745 4675 Acked Write 7 Done Sent Done false
16746 4676 Acked Write 2 Done Sent Done false
16747 4677 Acked Write 5 Done Sent Done false
16748 4678 Acked Read 5 Done Sent Done false
16749 4679 Acked Write 7 Done Sent Done false
16750 4680 Acked Write 7 Done Sent Done false
16751 4681 Acked Read 3 Done Sent Done false
16752 4682 Acked Write 1 Done Sent Done false
16753 4683 Acked Write 1 Done Sent Done false
16754 4684 Acked Read 6 Done Sent Done false
16755 4685 Acked Read 10 Done Sent Done false
16756 4686 Acked Write 3 Done Sent Done false
16757 4687 Acked Read 6 Done Sent Done false
16758 4688 Acked Read 1 Done Sent Done false
16759 4689 Acked Write 10 Done Sent Done false
16760 4690 Acked Read 3 Done Sent Done false
16761 4691 Acked Flush 0 Done Sent Done false
16762 4692 Acked Read 4 Done Sent Done false
16763 4693 Acked Read 9 Done Sent Done false
16764 4694 Acked Write 9 Done Sent Done false
16765 4695 Acked Write 6 Done Sent Done false
16766 4696 Acked Write 3 Done Sent Done false
16767 4697 Acked Read 5 Done Sent Done false
16768 4698 Acked Write 1 Done Sent Done false
16769 4699 Acked Write 5 Done Sent Done false
16770 4700 Acked Flush 0 Done Sent Done false
16771 4701 Acked Read 1 Done Sent Done false
16772 4702 Acked Read 9 Done Sent Done false
16773 4703 Acked Read 7 Done Sent Done false
16774 4704 Acked Write 1 Done Sent Done false
16775 4705 Acked Write 9 Done Sent Done false
16776 4706 Acked Read 9 Done Sent Done false
16777 4707 Acked Write 4 Done Sent Done false
16778 4708 Acked Read 6 Done Sent Done false
16779 4709 Acked Flush 0 Done Sent Done false
16780 4710 Acked Flush 0 Done Sent Done false
16781 4711 Acked Read 5 Done Sent Done false
16782 4712 Acked Flush 0 Done Sent Done false
16783 4713 Acked Flush 0 Done Sent Done false
16784 4714 Acked Write 4 Done Sent Done false
16785 4715 Acked Read 7 Done Sent Done false
16786 4716 Acked Write 1 Done Sent Done false
16787 4717 Acked Write 1 Done Sent Done false
16788 4718 Acked Read 4 Done Sent Done false
16789 4719 Acked Read 5 Done Sent Done false
16790 4720 Acked Read 7 Done Sent Done false
16791 4721 Acked Write 8 Done Sent Done false
16792 4722 Acked Read 8 Done Sent Done false
16793 4723 Acked Read 2 Done Sent Done false
16794 4724 Acked Write 9 Done Sent Done false
16795 4725 Acked Flush 0 Done Sent Done false
16796 4726 Acked Flush 0 Done Sent Done false
16797 4727 Acked Write 2 Done Sent Done false
16798 4728 Acked Read 2 Done Sent Done false
16799 4729 Acked Flush 0 Done Sent Done false
16800 4730 Acked Write 2 Done Sent Done false
16801 4731 Acked Write 7 Done Sent Done false
16802 4732 Acked Flush 0 Done Sent Done false
16803 4733 Acked Read 6 Done Sent Done false
16804 4734 Acked Flush 0 Done Sent Done false
16805 4735 Acked Read 5 Done Sent Done false
16806 4736 Acked Read 6 Done Sent Done false
16807 4737 Acked Flush 0 Done Sent Done false
16808 4738 Acked Write 8 Done Sent Done false
16809 4739 Acked Read 6 Done Sent Done false
16810 4740 Acked Read 4 Done Sent Done false
16811 4741 Acked Write 6 Done Sent Done false
16812 4742 Acked Write 10 Done Sent Done false
16813 4743 Acked Read 9 Done Sent Done false
16814 4744 Acked Write 1 Done Sent Done false
16815 4745 Acked Read 9 Done Sent Done false
16816 4746 Acked Read 5 Done Sent Done false
16817 4747 Acked Flush 0 Done Sent Done false
16818 4748 Acked Read 3 Done Sent Done false
16819 4749 Acked Read 5 Done Sent Done false
16820 4750 Acked Flush 0 Done Sent Done false
16821 4751 Acked Flush 0 Done Sent Done false
16822 4752 Acked Read 7 Done Sent Done false
16823 4753 Acked Read 7 Done Sent Done false
16824 4754 Acked Read 10 Done Sent Done false
16825 4755 Acked Read 10 Done Sent Done false
16826 4756 Acked Write 3 Done Sent Done false
16827 4757 Acked Read 1 Done Sent Done false
16828 4758 Acked Read 7 Done Sent Done false
16829 4759 Acked Write 9 Done Sent Done false
16830 4760 Acked Read 9 Done Sent Done false
16831 4761 Acked Read 8 Done Sent Done false
16832 4762 Acked Read 3 Done Sent Done false
16833 4763 Acked Flush 0 Done Sent Done false
16834 4764 Acked Read 4 Done Sent Done false
16835 4765 Acked Write 4 Done Sent Done false
16836 4766 Acked Write 8 Done Sent Done false
16837 4767 Acked Write 6 Done Sent Done false
16838 4768 Acked Write 3 Done Sent Done false
16839 4769 Acked Read 1 Done Sent Done false
16840 4770 Acked Read 7 Done Sent Done false
16841 4771 Acked Write 10 Done Sent Done false
16842 4772 Acked Flush 0 Done Sent Done false
16843 4773 Acked Read 7 Done Sent Done false
16844 4774 Acked Read 5 Done Sent Done false
16845 4775 Acked Read 10 Done Sent Done false
16846 4776 Acked Write 4 Done Sent Done false
16847 4777 Acked Write 8 Done Sent Done false
16848 4778 Acked Write 8 Done Sent Done false
16849 4779 Acked Write 10 Done Sent Done false
16850 4780 Acked Write 4 Done Sent Done false
16851 4781 Acked Write 9 Done Sent Done false
16852 4782 Acked Write 5 Done Sent Done false
16853 4783 Acked Read 4 Done Sent Done false
16854 4784 Acked Write 1 Done Sent Done false
16855 4785 Acked Flush 0 Done Sent Done false
16856 4786 Acked Read 3 Done Sent Done false
16857 4787 Acked Read 1 Done Sent Done false
16858 4788 Acked Read 10 Done Sent Done false
16859 4789 Acked Read 9 Done Sent Done false
16860 4790 Acked Write 3 Done Sent Done false
16861 4791 Acked Read 10 Done Sent Done false
16862 4792 Acked Read 1 Done Sent Done false
16863 4793 Acked Write 1 Done Sent Done false
16864 4794 Acked Read 6 Done Sent Done false
16865 4795 Acked Read 10 Done Sent Done false
16866 4796 Acked Read 6 Done Sent Done false
16867 4797 Acked Write 1 Done Sent Done false
16868 4798 Acked Flush 0 Done Sent Done false
16869 4799 Acked Read 1 Done Sent Done false
16870 4800 Acked Write 2 Done Sent Done false
16871 4801 Acked Write 8 Done Sent Done false
16872 4802 Acked Read 8 Done Sent Done false
16873 4803 Acked Read 5 Done Sent Done false
16874 4804 Acked Write 1 Done Sent Done false
16875 4805 Acked Write 1 Done Sent Done false
16876 4806 Acked Read 4 Done Sent Done false
16877 4807 Acked Write 2 Done Sent Done false
16878 4808 Acked Flush 0 Done Sent Done false
16879 4809 Acked Read 2 Done Sent Done false
16880 4810 Acked Write 5 Done Sent Done false
16881 4811 Acked Write 3 Done Sent Done false
16882 4812 Acked Flush 0 Done Sent Done false
16883 4813 Acked Write 5 Done Sent Done false
16884 4814 Acked Write 9 Done Sent Done false
16885 4815 Acked Read 6 Done Sent Done false
16886 4816 Acked Write 6 Done Sent Done false
16887 4817 Acked Read 3 Done Sent Done false
16888 4818 Acked Flush 0 Done Sent Done false
16889 4819 Acked Write 4 Done Sent Done false
16890 4820 Acked Write 5 Done Sent Done false
16891 4821 Acked Write 6 Done Sent Done false
16892 4822 Acked Read 6 Done Sent Done false
16893 4823 Acked Flush 0 Done Sent Done false
16894 4824 Acked Flush 0 Done Sent Done false
16895 4825 Acked Read 9 Done Sent Done false
16896 4826 Acked Write 6 Done Sent Done false
16897 4827 Acked Read 4 Done Sent Done false
16898 4828 Acked Write 7 Done Sent Done false
16899 4829 Acked Write 10 Done Sent Done false
16900 4830 Acked Write 3 Done Sent Done false
16901 4831 Acked Read 2 Done Sent Done false
16902 4832 Acked Write 5 Done Sent Done false
16903 4833 Acked Read 2 Done Sent Done false
16904 4834 Acked Write 7 Done Sent Done false
16905 4835 Acked Read 7 Done Sent Done false
16906 4836 Acked Read 10 Done Sent Done false
16907 4837 Acked Write 5 Done Sent Done false
16908 4838 Acked Read 8 Done Sent Done false
16909 4839 Acked Write 3 Done Sent Done false
16910 4840 Acked Read 9 Done Sent Done false
16911 4841 Acked Flush 0 Done Sent Done false
16912 4842 Acked Read 4 Done Sent Done false
16913 4843 Acked Read 3 Done Sent Done false
16914 4844 Acked Write 8 Done Sent Done false
16915 4845 Acked Read 9 Done Sent Done false
16916 4846 Acked Flush 0 Done Sent Done false
16917 4847 Acked Flush 0 Done Sent Done false
16918 4848 Acked Read 8 Done Sent Done false
16919 4849 Acked Write 5 Done Sent Done false
16920 4850 Acked Write 4 Done Sent Done false
16921 4851 Acked Read 4 Done Sent Done false
16922 4852 Acked Read 8 Done Sent Done false
16923 4853 Acked Read 8 Done Sent Done false
16924 4854 Acked Write 1 Done Sent Done false
16925 4855 Acked Write 4 Done Sent Done false
16926 4856 Acked Read 2 Done Sent Done false
16927 4857 Acked Write 10 Done Sent Done false
16928 4858 Acked Read 10 Done Sent Done false
16929 4859 Acked Write 7 Done Sent Done false
16930 4860 Acked Write 9 Done Sent Done false
16931 4861 Acked Write 1 Done Sent Done false
16932 4862 Acked Write 8 Done Sent Done false
16933 4863 Acked Write 5 Done Sent Done false
16934 4864 Acked Flush 0 Done Sent Done false
16935 4865 Acked Write 6 Done Sent Done false
16936 4866 Acked Write 4 Done Sent Done false
16937 4867 Acked Read 10 Done Sent Done false
16938 4868 Acked Write 8 Done Sent Done false
16939 4869 Acked Read 9 Done Sent Done false
16940 4870 Acked Read 6 Done Sent Done false
16941 4871 Acked Read 10 Done Sent Done false
16942 4872 Acked Read 8 Done Sent Done false
16943 4873 Acked Write 3 Done Sent Done false
16944 4874 Acked Read 2 Done Sent Done false
16945 4875 Acked Read 8 Done Sent Done false
16946 4876 Acked Read 1 Done Sent Done false
16947 4877 Acked Flush 0 Done Sent Done false
16948 4878 Acked Read 10 Done Sent Done false
16949 4879 Acked Read 1 Done Sent Done false
16950 4880 Acked Write 8 Done Sent Done false
16951 4881 Acked Write 10 Done Sent Done false
16952 4882 Acked Read 3 Done Sent Done false
16953 4883 Acked Write 7 Done Sent Done false
16954 4884 Acked Write 3 Done Sent Done false
16955 4885 Acked Read 2 Done Sent Done false
16956 4886 Acked Read 7 Done Sent Done false
16957 4887 Acked Write 7 Done Sent Done false
16958 4888 Acked Read 3 Done Sent Done false
16959 4889 Acked Read 1 Done Sent Done false
16960 4890 Acked Write 5 Done Sent Done false
16961 4891 Acked Read 10 Done Sent Done false
16962 4892 Acked Read 2 Done Sent Done false
16963 4893 Acked Read 5 Done Sent Done false
16964 4894 Acked Flush 0 Done Sent Done false
16965 4895 Acked Read 6 Done Sent Done false
16966 4896 Acked Read 7 Done Sent Done false
16967 4897 Acked Write 2 Done Sent Done false
16968 4898 Acked Flush 0 Done Sent Done false
16969 4899 Acked Read 7 Done Sent Done false
16970 4900 Acked Write 7 Done Sent Done false
16971 4901 Acked Read 1 Done Sent Done false
16972 4902 Acked Read 5 Done Sent Done false
16973 4903 Acked Write 3 Done Sent Done false
16974 4904 Acked Write 6 Done Sent Done false
16975 4905 Acked Read 1 Done Sent Done false
16976 4906 Acked Read 4 Done Sent Done false
16977 4907 Acked Read 3 Done Sent Done false
16978 4908 Acked Write 8 Done Sent Done false
16979 4909 Acked Flush 0 Done Sent Done false
16980 4910 Acked Read 8 Done Sent Done false
16981 4911 Acked Write 7 Done Sent Done false
16982 4912 Acked Read 7 Done Sent Done false
16983 4913 Acked Write 1 Done Sent Done false
16984 4914 Acked Read 10 Done Sent Done false
16985 4915 Acked Read 5 Done Sent Done false
16986 4916 Acked Write 1 Done Sent Done false
16987 4917 Acked Flush 0 Done Sent Done false
16988 4918 Acked Write 9 Done Sent Done false
16989 4919 Acked Write 1 Done Sent Done false
16990 4920 Acked Write 6 Done Sent Done false
16991 4921 Acked Read 4 Done Sent Done false
16992 4922 Acked Read 8 Done Sent Done false
16993 STATES DS:0 DS:1 DS:2 TOTAL
16994 Sent 0 301 0 301
16995 Done 301 0 301 602
16996 Skipped 0 0 0 0
16997 Error 0 0 0 0
16998 Last Flush: 4917 4621 4917
16999 Downstairs last five completed: 4621 4620 4619 4618 4617
17000 Upstairs last five completed: 4922 4921 4920 4919 4918
17001 CLIENT: Up:0 ds:301 act:2
17002 [8820] got start action
17003 [8820] I am going to restart
17004 Starting downstairs at port 8820
17005 Make output file at "/tmp/dsc/downstairs-8820.txt"
17006 [8820][1] reports Starting
17007 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1218)
17008 [8820][1] reports Running
17009 ----------------------------------------------------------------
17010 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
17011 States: Active Offline Active
17012 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
17013 4622 Acked Read 5 Done Sent Done false
17014 4623 Acked Read 6 Done Sent Done false
17015 4624 Acked Write 2 Done Sent Done false
17016 4625 Acked Write 10 Done Sent Done false
17017 4626 Acked Read 8 Done Sent Done false
17018 4627 Acked Read 4 Done Sent Done false
17019 4628 Acked Write 4 Done Sent Done false
17020 4629 Acked Read 5 Done Sent Done false
17021 4630 Acked Read 7 Done Sent Done false
17022 4631 Acked Write 7 Done Sent Done false
17023 4632 Acked Read 6 Done Sent Done false
17024 4633 Acked Read 2 Done Sent Done false
17025 4634 Acked Write 9 Done Sent Done false
17026 4635 Acked Read 9 Done Sent Done false
17027 4636 Acked Read 6 Done Sent Done false
17028 4637 Acked Flush 0 Done Sent Done false
17029 4638 Acked Read 5 Done Sent Done false
17030 4639 Acked Read 3 Done Sent Done false
17031 4640 Acked Write 3 Done Sent Done false
17032 4641 Acked Read 8 Done Sent Done false
17033 4642 Acked Write 9 Done Sent Done false
17034 4643 Acked Write 10 Done Sent Done false
17035 4644 Acked Read 3 Done Sent Done false
17036 4645 Acked Read 8 Done Sent Done false
17037 4646 Acked Read 3 Done Sent Done false
17038 4647 Acked Write 1 Done Sent Done false
17039 4648 Acked Write 8 Done Sent Done false
17040 4649 Acked Flush 0 Done Sent Done false
17041 4650 Acked Read 2 Done Sent Done false
17042 4651 Acked Write 3 Done Sent Done false
17043 4652 Acked Read 6 Done Sent Done false
17044 4653 Acked Write 2 Done Sent Done false
17045 4654 Acked Write 5 Done Sent Done false
17046 4655 Acked Read 6 Done Sent Done false
17047 4656 Acked Write 7 Done Sent Done false
17048 4657 Acked Read 7 Done Sent Done false
17049 4658 Acked Read 4 Done Sent Done false
17050 4659 Acked Read 5 Done Sent Done false
17051 4660 Acked Flush 0 Done Sent Done false
17052 4661 Acked Read 10 Done Sent Done false
17053 4662 Acked Read 10 Done Sent Done false
17054 4663 Acked Read 5 Done Sent Done false
17055 4664 Acked Read 2 Done Sent Done false
17056 4665 Acked Read 4 Done Sent Done false
17057 4666 Acked Read 8 Done Sent Done false
17058 4667 Acked Write 6 Done Sent Done false
17059 4668 Acked Flush 0 Done Sent Done false
17060 4669 Acked Read 6 Done Sent Done false
17061 4670 Acked Write 5 Done Sent Done false
17062 4671 Acked Write 5 Done Sent Done false
17063 4672 Acked Read 6 Done Sent Done false
17064 4673 Acked Read 8 Done Sent Done false
17065 4674 Acked Write 7 Done Sent Done false
17066 4675 Acked Write 7 Done Sent Done false
17067 4676 Acked Write 2 Done Sent Done false
17068 4677 Acked Write 5 Done Sent Done false
17069 4678 Acked Read 5 Done Sent Done false
17070 4679 Acked Write 7 Done Sent Done false
17071 4680 Acked Write 7 Done Sent Done false
17072 4681 Acked Read 3 Done Sent Done false
17073 4682 Acked Write 1 Done Sent Done false
17074 4683 Acked Write 1 Done Sent Done false
17075 4684 Acked Read 6 Done Sent Done false
17076 4685 Acked Read 10 Done Sent Done false
17077 4686 Acked Write 3 Done Sent Done false
17078 4687 Acked Read 6 Done Sent Done false
17079 4688 Acked Read 1 Done Sent Done false
17080 4689 Acked Write 10 Done Sent Done false
17081 4690 Acked Read 3 Done Sent Done false
17082 4691 Acked Flush 0 Done Sent Done false
17083 4692 Acked Read 4 Done Sent Done false
17084 4693 Acked Read 9 Done Sent Done false
17085 4694 Acked Write 9 Done Sent Done false
17086 4695 Acked Write 6 Done Sent Done false
17087 4696 Acked Write 3 Done Sent Done false
17088 4697 Acked Read 5 Done Sent Done false
17089 4698 Acked Write 1 Done Sent Done false
17090 4699 Acked Write 5 Done Sent Done false
17091 4700 Acked Flush 0 Done Sent Done false
17092 4701 Acked Read 1 Done Sent Done false
17093 4702 Acked Read 9 Done Sent Done false
17094 4703 Acked Read 7 Done Sent Done false
17095 4704 Acked Write 1 Done Sent Done false
17096 4705 Acked Write 9 Done Sent Done false
17097 4706 Acked Read 9 Done Sent Done false
17098 4707 Acked Write 4 Done Sent Done false
17099 4708 Acked Read 6 Done Sent Done false
17100 4709 Acked Flush 0 Done Sent Done false
17101 4710 Acked Flush 0 Done Sent Done false
17102 4711 Acked Read 5 Done Sent Done false
17103 4712 Acked Flush 0 Done Sent Done false
17104 4713 Acked Flush 0 Done Sent Done false
17105 4714 Acked Write 4 Done Sent Done false
17106 4715 Acked Read 7 Done Sent Done false
17107 4716 Acked Write 1 Done Sent Done false
17108 4717 Acked Write 1 Done Sent Done false
17109 4718 Acked Read 4 Done Sent Done false
17110 4719 Acked Read 5 Done Sent Done false
17111 4720 Acked Read 7 Done Sent Done false
17112 4721 Acked Write 8 Done Sent Done false
17113 4722 Acked Read 8 Done Sent Done false
17114 4723 Acked Read 2 Done Sent Done false
17115 4724 Acked Write 9 Done Sent Done false
17116 4725 Acked Flush 0 Done Sent Done false
17117 4726 Acked Flush 0 Done Sent Done false
17118 4727 Acked Write 2 Done Sent Done false
17119 4728 Acked Read 2 Done Sent Done false
17120 4729 Acked Flush 0 Done Sent Done false
17121 4730 Acked Write 2 Done Sent Done false
17122 4731 Acked Write 7 Done Sent Done false
17123 4732 Acked Flush 0 Done Sent Done false
17124 4733 Acked Read 6 Done Sent Done false
17125 4734 Acked Flush 0 Done Sent Done false
17126 4735 Acked Read 5 Done Sent Done false
17127 4736 Acked Read 6 Done Sent Done false
17128 4737 Acked Flush 0 Done Sent Done false
17129 4738 Acked Write 8 Done Sent Done false
17130 4739 Acked Read 6 Done Sent Done false
17131 4740 Acked Read 4 Done Sent Done false
17132 4741 Acked Write 6 Done Sent Done false
17133 4742 Acked Write 10 Done Sent Done false
17134 4743 Acked Read 9 Done Sent Done false
17135 4744 Acked Write 1 Done Sent Done false
17136 4745 Acked Read 9 Done Sent Done false
17137 4746 Acked Read 5 Done Sent Done false
17138 4747 Acked Flush 0 Done Sent Done false
17139 4748 Acked Read 3 Done Sent Done false
17140 4749 Acked Read 5 Done Sent Done false
17141 4750 Acked Flush 0 Done Sent Done false
17142 4751 Acked Flush 0 Done Sent Done false
17143 4752 Acked Read 7 Done Sent Done false
17144 4753 Acked Read 7 Done Sent Done false
17145 4754 Acked Read 10 Done Sent Done false
17146 4755 Acked Read 10 Done Sent Done false
17147 4756 Acked Write 3 Done Sent Done false
17148 4757 Acked Read 1 Done Sent Done false
17149 4758 Acked Read 7 Done Sent Done false
17150 4759 Acked Write 9 Done Sent Done false
17151 4760 Acked Read 9 Done Sent Done false
17152 4761 Acked Read 8 Done Sent Done false
17153 4762 Acked Read 3 Done Sent Done false
17154 4763 Acked Flush 0 Done Sent Done false
17155 4764 Acked Read 4 Done Sent Done false
17156 4765 Acked Write 4 Done Sent Done false
17157 4766 Acked Write 8 Done Sent Done false
17158 4767 Acked Write 6 Done Sent Done false
17159 4768 Acked Write 3 Done Sent Done false
17160 4769 Acked Read 1 Done Sent Done false
17161 4770 Acked Read 7 Done Sent Done false
17162 4771 Acked Write 10 Done Sent Done false
17163 4772 Acked Flush 0 Done Sent Done false
17164 4773 Acked Read 7 Done Sent Done false
17165 4774 Acked Read 5 Done Sent Done false
17166 4775 Acked Read 10 Done Sent Done false
17167 4776 Acked Write 4 Done Sent Done false
17168 4777 Acked Write 8 Done Sent Done false
17169 4778 Acked Write 8 Done Sent Done false
17170 4779 Acked Write 10 Done Sent Done false
17171 4780 Acked Write 4 Done Sent Done false
17172 4781 Acked Write 9 Done Sent Done false
17173 4782 Acked Write 5 Done Sent Done false
17174 4783 Acked Read 4 Done Sent Done false
17175 4784 Acked Write 1 Done Sent Done false
17176 4785 Acked Flush 0 Done Sent Done false
17177 4786 Acked Read 3 Done Sent Done false
17178 4787 Acked Read 1 Done Sent Done false
17179 4788 Acked Read 10 Done Sent Done false
17180 4789 Acked Read 9 Done Sent Done false
17181 4790 Acked Write 3 Done Sent Done false
17182 4791 Acked Read 10 Done Sent Done false
17183 4792 Acked Read 1 Done Sent Done false
17184 4793 Acked Write 1 Done Sent Done false
17185 4794 Acked Read 6 Done Sent Done false
17186 4795 Acked Read 10 Done Sent Done false
17187 4796 Acked Read 6 Done Sent Done false
17188 4797 Acked Write 1 Done Sent Done false
17189 4798 Acked Flush 0 Done Sent Done false
17190 4799 Acked Read 1 Done Sent Done false
17191 4800 Acked Write 2 Done Sent Done false
17192 4801 Acked Write 8 Done Sent Done false
17193 4802 Acked Read 8 Done Sent Done false
17194 4803 Acked Read 5 Done Sent Done false
17195 4804 Acked Write 1 Done Sent Done false
17196 4805 Acked Write 1 Done Sent Done false
17197 4806 Acked Read 4 Done Sent Done false
17198 4807 Acked Write 2 Done Sent Done false
17199 4808 Acked Flush 0 Done Sent Done false
17200 4809 Acked Read 2 Done Sent Done false
17201 4810 Acked Write 5 Done Sent Done false
17202 4811 Acked Write 3 Done Sent Done false
17203 4812 Acked Flush 0 Done Sent Done false
17204 4813 Acked Write 5 Done Sent Done false
17205 4814 Acked Write 9 Done Sent Done false
17206 4815 Acked Read 6 Done Sent Done false
17207 4816 Acked Write 6 Done Sent Done false
17208 4817 Acked Read 3 Done Sent Done false
17209 4818 Acked Flush 0 Done Sent Done false
17210 4819 Acked Write 4 Done Sent Done false
17211 4820 Acked Write 5 Done Sent Done false
17212 4821 Acked Write 6 Done Sent Done false
17213 4822 Acked Read 6 Done Sent Done false
17214 4823 Acked Flush 0 Done Sent Done false
17215 4824 Acked Flush 0 Done Sent Done false
17216 4825 Acked Read 9 Done Sent Done false
17217 4826 Acked Write 6 Done Sent Done false
17218 4827 Acked Read 4 Done Sent Done false
17219 4828 Acked Write 7 Done Sent Done false
17220 4829 Acked Write 10 Done Sent Done false
17221 4830 Acked Write 3 Done Sent Done false
17222 4831 Acked Read 2 Done Sent Done false
17223 4832 Acked Write 5 Done Sent Done false
17224 4833 Acked Read 2 Done Sent Done false
17225 4834 Acked Write 7 Done Sent Done false
17226 4835 Acked Read 7 Done Sent Done false
17227 4836 Acked Read 10 Done Sent Done false
17228 4837 Acked Write 5 Done Sent Done false
17229 4838 Acked Read 8 Done Sent Done false
17230 4839 Acked Write 3 Done Sent Done false
17231 4840 Acked Read 9 Done Sent Done false
17232 4841 Acked Flush 0 Done Sent Done false
17233 4842 Acked Read 4 Done Sent Done false
17234 4843 Acked Read 3 Done Sent Done false
17235 4844 Acked Write 8 Done Sent Done false
17236 4845 Acked Read 9 Done Sent Done false
17237 4846 Acked Flush 0 Done Sent Done false
17238 4847 Acked Flush 0 Done Sent Done false
17239 4848 Acked Read 8 Done Sent Done false
17240 4849 Acked Write 5 Done Sent Done false
17241 4850 Acked Write 4 Done Sent Done false
17242 4851 Acked Read 4 Done Sent Done false
17243 4852 Acked Read 8 Done Sent Done false
17244 4853 Acked Read 8 Done Sent Done false
17245 4854 Acked Write 1 Done Sent Done false
17246 4855 Acked Write 4 Done Sent Done false
17247 4856 Acked Read 2 Done Sent Done false
17248 4857 Acked Write 10 Done Sent Done false
17249 4858 Acked Read 10 Done Sent Done false
17250 4859 Acked Write 7 Done Sent Done false
17251 4860 Acked Write 9 Done Sent Done false
17252 4861 Acked Write 1 Done Sent Done false
17253 4862 Acked Write 8 Done Sent Done false
17254 4863 Acked Write 5 Done Sent Done false
17255 4864 Acked Flush 0 Done Sent Done false
17256 4865 Acked Write 6 Done Sent Done false
17257 4866 Acked Write 4 Done Sent Done false
17258 4867 Acked Read 10 Done Sent Done false
17259 4868 Acked Write 8 Done Sent Done false
17260 4869 Acked Read 9 Done Sent Done false
17261 4870 Acked Read 6 Done Sent Done false
17262 4871 Acked Read 10 Done Sent Done false
17263 4872 Acked Read 8 Done Sent Done false
17264 4873 Acked Write 3 Done Sent Done false
17265 4874 Acked Read 2 Done Sent Done false
17266 4875 Acked Read 8 Done Sent Done false
17267 4876 Acked Read 1 Done Sent Done false
17268 4877 Acked Flush 0 Done Sent Done false
17269 4878 Acked Read 10 Done Sent Done false
17270 4879 Acked Read 1 Done Sent Done false
17271 4880 Acked Write 8 Done Sent Done false
17272 4881 Acked Write 10 Done Sent Done false
17273 4882 Acked Read 3 Done Sent Done false
17274 4883 Acked Write 7 Done Sent Done false
17275 4884 Acked Write 3 Done Sent Done false
17276 4885 Acked Read 2 Done Sent Done false
17277 4886 Acked Read 7 Done Sent Done false
17278 4887 Acked Write 7 Done Sent Done false
17279 4888 Acked Read 3 Done Sent Done false
17280 4889 Acked Read 1 Done Sent Done false
17281 4890 Acked Write 5 Done Sent Done false
17282 4891 Acked Read 10 Done Sent Done false
17283 4892 Acked Read 2 Done Sent Done false
17284 4893 Acked Read 5 Done Sent Done false
17285 4894 Acked Flush 0 Done Sent Done false
17286 4895 Acked Read 6 Done Sent Done false
17287 4896 Acked Read 7 Done Sent Done false
17288 4897 Acked Write 2 Done Sent Done false
17289 4898 Acked Flush 0 Done Sent Done false
17290 4899 Acked Read 7 Done Sent Done false
17291 4900 Acked Write 7 Done Sent Done false
17292 4901 Acked Read 1 Done Sent Done false
17293 4902 Acked Read 5 Done Sent Done false
17294 4903 Acked Write 3 Done Sent Done false
17295 4904 Acked Write 6 Done Sent Done false
17296 4905 Acked Read 1 Done Sent Done false
17297 4906 Acked Read 4 Done Sent Done false
17298 4907 Acked Read 3 Done Sent Done false
17299 4908 Acked Write 8 Done Sent Done false
17300 4909 Acked Flush 0 Done Sent Done false
17301 4910 Acked Read 8 Done Sent Done false
17302 4911 Acked Write 7 Done Sent Done false
17303 4912 Acked Read 7 Done Sent Done false
17304 4913 Acked Write 1 Done Sent Done false
17305 4914 Acked Read 10 Done Sent Done false
17306 4915 Acked Read 5 Done Sent Done false
17307 4916 Acked Write 1 Done Sent Done false
17308 4917 Acked Flush 0 Done Sent Done false
17309 4918 Acked Write 9 Done Sent Done false
17310 4919 Acked Write 1 Done Sent Done false
17311 4920 Acked Write 6 Done Sent Done false
17312 4921 Acked Read 4 Done Sent Done false
17313 4922 Acked Read 8 Done Sent Done false
17314 4923 Acked Flush 0 Done Sent Done false
17315 STATES DS:0 DS:1 DS:2 TOTAL
17316 Sent 0 302 0 302
17317 Done 302 0 302 604
17318 Skipped 0 0 0 0
17319 Error 0 0 0 0
17320 Last Flush: 4923 4621 4923
17321 Downstairs last five completed: 4621 4620 4619 4618 4617
17322 Upstairs last five completed: 4923 4922 4921 4920 4919
17323 CLIENT: Up:0 ds:302 act:2
173242024-11-15T02:01:04.837ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
173252024-11-15T02:01:04.837ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:36459) = io task client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
173262024-11-15T02:01:04.838ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
173272024-11-15T02:01:04.839ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
173282024-11-15T02:01:04.839ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
173292024-11-15T02:01:04.839ZINFOcrucible: send last flush ID to this DS: 4621 = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
173302024-11-15T02:01:04.839ZINFOcrucible: Replied this last flush ID: 4621 = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
173312024-11-15T02:01:04.839ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
173322024-11-15T02:01:04.839ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
173332024-11-15T02:01:04.839ZINFOcrucible: [1] client re-new 302 jobs since flush 4621 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
173342024-11-15T02:01:04.839ZINFOcrucible: [1] Replayed 302 jobs since flush: 4621 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
17335 ----------------------------------------------------------------
17336 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
17337 Downstairs last five completed: 4923 4922 4921 4920 4919
17338 Upstairs last five completed: 4923 4922 4921 4920 4919
17339 CLIENT: Up:0 ds:0 act:3
17340 Replay: All jobs finished
17341 Main task has work to do, go find it
17342 got dsc Stop(0)
17343 stop 0
17344 Nov 15 02:01:07.080 INFO request completed[8810] Got stop action so:false kr:false
17345 , latency_us: 343, response_code: 204, uri: /stop/cid/0, method: POST, req_id: d321aa41-9c4b-4669-bb41-39eb0d3170e9, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
17346 Nov 15 02:01:07.081 INFO request completed, latency_us: 132, response_code: 200, uri: /state/cid/0, method: GET, req_id: 617fcdd9-18b6-46da-9a15-a79034c07cc1, remote_addr: 127.0.0{"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2024-11-15T02:01:07.08134358Z","hostname":"w-01JCPRHT5F2MPA44KVJHM3S2PB","pid":1205,"":"io task","client":"0","":"downstairs","session_id":"971dcb2e-6891-4ecf-b367-1e04e52065ee"}
17347 .1:40008, local_addr: 127.0.0.1:9998
173482024-11-15T02:01:07.081ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
173492024-11-15T02:01:07.081ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
173502024-11-15T02:01:07.081ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
173512024-11-15T02:01:07.081ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
173522024-11-15T02:01:07.081ZINFOcrucible: client task is exiting = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
17353 [8810] Exited with: ExitStatus(unix_wait_status(9))
17354 [8810][0] reports Exit
17355 Nov 15 02:01:11.084 INFO request completed, latency_us: 217, response_code: 200, uri: /state/cid/0, method: GET, req_id: 50ad22f7-91f6-46d0-8dfa-ed83e5804a3f, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
17356 001/300 Read block 32 len 32768
17357 002/300 Write block 2130 len 40960 data: 6 5 4 4 4 4 4 4 4 3
17358 003/300 Write block 3639 len 28672 data: 3 2 2 2 2 2 2
17359 004/300 Flush
17360 005/300 Write block 2564 len 28672 data: 4 4 6 5 3 4 4
17361 006/300 Write block 3527 len 28672 data: 2 3 5 5 6 5 5
17362 007/300 Flush
17363 008/300 Read block 1586 len 16384
17364 009/300 Flush
17365 010/300 Write block 3788 len 28672 data: 4 4 3 2 2 2 3
17366 011/300 Read block 1138 len 32768
17367 012/300 Write block 47 len 20480 data: 4 4 4 4 3
17368 013/300 Read block 753 len 36864
17369 014/300 Read block 2067 len 4096
17370 015/300 Write block 709 len 28672 data: 5 5 5 5 3 3 3
17371 016/300 Flush
17372 017/300 Read block 3308 len 24576
17373 018/300 Read block 2205 len 28672
17374 019/300 Read block 1455 len 4096
17375 020/300 Read block 1845 len 40960
17376 021/300 Write block 4363 len 12288 data: 2 2 2
17377 022/300 Read block 1646 len 32768
17378 023/300 Read block 3441 len 40960
17379 024/300 Write block 3033 len 24576 data: 4 5 4 4 4 4
17380 025/300 Write block 1581 len 36864 data: 2 2 2 2 2 2 2 3 3
17381 026/300 Read block 1570 len 40960
17382 027/300 Read block 1283 len 28672
17383 028/300 Read block 387 len 24576
17384 029/300 Read block 3664 len 20480
17385 030/300 Write block 4902 len 20480 data: 6 6 6 4 4
17386 031/300 Read block 661 len 16384
17387 032/300 Write block 2647 len 8192 data: 4 4
17388 033/300 Read block 2226 len 4096
17389 034/300 Read block 396 len 32768
17390 035/300 Read block 2801 len 36864
17391 036/300 Write block 1911 len 4096 data: 3
17392 037/300 Read block 3917 len 8192
17393 038/300 Read block 4709 len 20480
17394 039/300 Flush
17395 040/300 Read block 1893 len 12288
17396 041/300 Write block 4036 len 8192 data: 4 6
17397 042/300 Read block 971 len 40960
17398 043/300 Write block 1471 len 12288 data: 4 3 2
17399 044/300 Flush
17400 045/300 Write block 4813 len 36864 data: 2 2 2 2 3 3 3 3 2
17401 046/300 Read block 2163 len 16384
17402 047/300 Flush
17403 048/300 Flush
17404 049/300 Write block 358 len 40960 data: 4 4 4 3 3 4 2 2 2 2
17405 050/300 Read block 2889 len 24576
17406 051/300 Read block 2220 len 40960
17407 052/300 Read block 4611 len 12288
17408 053/300 Read block 4810 len 32768
17409 054/300 Write block 1535 len 12288 data: 3 3 3
17410 055/300 Write block 3683 len 20480 data: 4 3 3 4 6
17411 056/300 Write block 1120 len 12288 data: 5 6 5
17412 057/300 Read block 1208 len 20480
17413 058/300 Write block 118 len 16384 data: 2 2 2 3
17414 059/300 Read block 3921 len 32768
17415 060/300 Read block 2669 len 40960
17416 061/300 Write block 1568 len 8192 data: 3 3
17417 062/300 Write block 1600 len 4096 data: 3
17418 063/300 Read block 2992 len 8192
17419 064/300 Write block 1355 len 28672 data: 3 3 3 3 3 2 2
17420 065/300 Read block 1011 len 28672
17421 066/300 Write block 3987 len 32768 data: 4 4 4 4 3 2 2 2
17422 067/300 Write block 618 len 32768 data: 2 2 3 4 4 5 5 4
17423 068/300 Read block 4312 len 8192
17424 069/300 Read block 3020 len 8192
17425 070/300 Read block 1699 len 20480
17426 071/300 Flush
17427 072/300 Read block 1651 len 16384
17428 073/300 Read block 1971 len 36864
17429 074/300 Read block 1478 len 40960
17430 075/300 Write block 1900 len 36864 data: 5 5 4 5 5 5 3 3 3
17431 076/300 Write block 1246 len 24576 data: 4 4 4 4 4 4
17432 077/300 Read block 2185 len 32768
17433 078/300 Write block 3654 len 12288 data: 5 5 3
17434 079/300 Read block 73 len 28672
17435 080/300 Write block 3219 len 8192 data: 5 5
17436 081/300 Write block 159 len 20480 data: 4 4 4 4 5
17437 082/300 Write block 2780 len 12288 data: 4 4 4
17438 083/300 Read block 3914 len 32768
17439 084/300 Read block 4149 len 24576
17440 085/300 Read block 3201 len 12288
17441 086/300 Write block 1161 len 4096 data: 2
17442 087/300 Read block 4604 len 4096
17443 088/300 Read block 3013 len 8192
17444 089/300 Write block 4852 len 24576 data: 2 2 2 2 2 2
17445 090/300 Write block 1640 len 36864 data: 3 4 4 4 3 2 2 2 2
17446 091/300 Read block 177 len 20480
17447 092/300 Read block 624 len 36864
17448 093/300 Write block 2228 len 28672 data: 3 5 5 6 6 6 5
17449 094/300 Read block 3238 len 4096
17450 095/300 Write block 670 len 28672 data: 4 4 4 4 3 3 3
17451 096/300 Write block 1691 len 40960 data: 4 4 4 3 3 3 3 5 5 4
17452 097/300 Flush
17453 098/300 Write block 2761 len 16384 data: 4 3 3 3
17454 099/300 Write block 3138 len 36864 data: 3 2 3 3 3 3 3 3 3
17455 100/300 Write block 4657 len 28672 data: 2 3 3 4 4 6 5
17456 101/300 Write block 4189 len 24576 data: 4 5 5 5 6 6
17457 102/300 Write block 2466 len 40960 data: 4 3 3 4 5 6 6 9 9 9
17458 103/300 Read block 4677 len 40960
17459 104/300 Read block 3074 len 16384
17460 105/300 Read block 593 len 28672
17461 106/300 Read block 1593 len 12288
17462 107/300 Read block 4312 len 36864
17463 108/300 Write block 205 len 28672 data: 4 3 3 3 3 3 3
17464 109/300 Read block 3834 len 8192
17465 110/300 Write block 2421 len 16384 data: 2 2 2 2
17466 111/300 Write block 758 len 20480 data: 6 6 4 4 4
17467 112/300 Read block 2728 len 36864
17468 113/300 Read block 3076 len 40960
17469 114/300 Read block 2129 len 16384
17470 115/300 Read block 2174 len 16384
17471 116/300 Read block 675 len 24576
17472 117/300 Write block 794 len 32768 data: 4 4 3 3 3 3 3 3
17473 118/300 Write block 3530 len 28672 data: 6 7 6 6 4 5 4
17474 119/300 Write block 2166 len 36864 data: 4 3 3 3 4 4 4 4 4
17475 120/300 Read block 3530 len 20480
17476 121/300 Read block 2554 len 8192
17477 122/300 Write block 847 len 24576 data: 3 3 3 3 5 5
17478 123/300 Write block 1651 len 28672 data: 2 3 3 4 4 4 3
17479 124/300 Read block 1571 len 28672
17480 125/300 Write block 2978 len 40960 data: 5 5 5 6 5 6 6 6 6 4
17481 126/300 Read block 3595 len 8192
17482 127/300 Read block 1053 len 32768
17483 128/300 Read block 1267 len 40960
17484 129/300 Read block 4260 len 36864
17485 130/300 Write block 3243 len 36864 data: 3 3 5 5 5 5 5 5 6
17486 131/300 Read block 3957 len 16384
17487 132/300 Read block 3565 len 16384
17488 133/300 Flush
17489 134/300 Read block 1583 len 4096
17490 135/300 Write block 264 len 8192 data: 3 3
17491 136/300 Read block 4159 len 12288
17492 137/300 Write block 388 len 28672 data: 5 5 3 5 4 4 4
17493 138/300 Write block 2018 len 20480 data: 3 3 4 3 2
17494 139/300 Read block 86 len 40960
17495 140/300 Write block 2301 len 16384 data: 2 2 2 2
17496 141/300 Write block 890 len 28672 data: 3 3 3 2 3 3 4
17497 142/300 Write block 619 len 40960 data: 3 4 5 5 6 6 5 3 3 2
17498 143/300 Read block 490 len 4096
17499 144/300 Read block 3508 len 16384
17500 145/300 Write block 2307 len 24576 data: 3 4 4 3 3 3
17501 146/300 Read block 2058 len 32768
17502 147/300 Read block 1470 len 40960
17503 148/300 Read block 1575 len 12288
17504 149/300 Read block 1875 len 20480
17505 150/300 Read block 4612 len 32768
17506 151/300 Write block 307 len 32768 data: 5 4 6 6 6 5 5 5
17507 152/300 Write block 2136 len 40960 data: 5 5 5 4 3 3 2 3 3 3
17508 153/300 Read block 4985 len 16384
17509 154/300 Write block 2476 len 4096 data: 7
17510 155/300 Read block 2595 len 16384
17511 156/300 Write block 4756 len 4096 data: 3
17512 157/300 Read block 3651 len 24576
17513 158/300 Write block 2725 len 28672 data: 4 4 5 5 4 4 4
17514 159/300 Flush
17515 160/300 Read block 981 len 16384
17516 161/300 Read block 3532 len 4096
17517 162/300 Read block 1369 len 28672
17518 163/300 Write block 3152 len 36864 data: 4 3 3 3 4 4 6 5 5
17519 164/300 Read block 49 len 36864
17520 165/300 Read block 1133 len 8192
17521 166/300 Read block 2856 len 40960
17522 167/300 Write block 2220 len 8192 data: 5 5
17523 168/300 Write block 4682 len 16384 data: 3 4 6 5
17524 169/300 Flush
17525 170/300 Write block 3008 len 32768 data: 3 3 3 3 3 3 3 3
17526 171/300 Write block 2421 len 12288 data: 3 3 3
17527 172/300 Write block 1936 len 40960 data: 4 4 5 5 5 5 4 3 2 2
17528 173/300 Read block 2870 len 24576
17529 174/300 Write block 96 len 32768 data: 3 3 3 3 3 3 4 4
17530 175/300 Write block 1054 len 24576 data: 4 3 3 3 3 4
17531 176/300 Read block 39 len 20480
17532 177/300 Read block 3963 len 20480
17533 178/300 Write block 2339 len 32768 data: 6 7 7 5 4 4 5 5
17534 179/300 Write block 1625 len 40960 data: 2 2 2 3 3 3 4 4 4 4
17535 180/300 Write block 306 len 36864 data: 5 6 5 7 7 7 6 6 6
17536 181/300 Read block 1938 len 40960
17537 182/300 Write block 3923 len 8192 data: 6 6
17538 183/300 Read block 440 len 12288
17539 184/300 Read block 1004 len 36864
17540 185/300 Write block 4817 len 40960 data: 4 4 4 4 3 2 2 2 3 3
17541 186/300 Write block 2115 len 16384 data: 2 2 3 2
17542 187/300 Read block 3711 len 24576
17543 188/300 Write block 3622 len 24576 data: 3 2 2 2 3 3
17544 189/300 Write block 4284 len 20480 data: 4 4 4 5 5
17545 190/300 Write block 3545 len 36864 data: 3 3 2 3 3 3 4 6 5
17546 191/300 Read block 1495 len 4096
17547 192/300 Read block 713 len 28672
17548 193/300 Read block 1171 len 28672
17549 194/300 Read block 1152 len 4096
17550 195/300 Write block 3004 len 32768 data: 3 3 3 3 4 4 4 4
17551 196/300 Read block 4756 len 8192
17552 197/300 Read block 2432 len 36864
17553 198/300 Write block 3757 len 16384 data: 5 5 5 5
17554 199/300 Read block 1009 len 24576
17555 200/300 Read block 2474 len 40960
17556 201/300 Write block 2753 len 32768 data: 3 3 3 3 4 4 4 4
17557 202/300 Write block 1084 len 24576 data: 2 2 2 3 2 2
17558 203/300 Write block 1638 len 32768 data: 5 5 4 5 5 5 4 3
17559 204/300 Flush
17560 205/300 Write block 3146 len 8192 data: 4 3
17561 206/300 Read block 3321 len 12288
17562 207/300 Flush
17563 208/300 Read block 2454 len 8192
17564 209/300 Read block 1930 len 8192
17565 210/300 Write block 1251 len 24576 data: 5 4 3 3 3 3
17566 211/300 Read block 2827 len 24576
17567 212/300 Read block 2261 len 8192
17568 213/300 Write block 1453 len 16384 data: 3 2 4 3
17569 214/300 Flush
17570 215/300 Write block 1821 len 24576 data: 3 4 4 4 4 3
17571 216/300 Write block 4058 len 32768 data: 2 2 2 2 2 4 4 4
17572 217/300 Write block 3832 len 36864 data: 4 3 3 3 5 5 5 5 5
17573 218/300 Write block 2620 len 16384 data: 4 3 3 3
17574 219/300 Read block 1620 len 4096
17575 220/300 Read block 4529 len 32768
17576 221/300 Write block 1235 len 28672 data: 3 3 2 2 2 2 3
17577 222/300 Read block 4865 len 36864
17578 223/300 Read block 2978 len 36864
17579 224/300 Read block 2849 len 8192
17580 225/300 Write block 4223 len 40960 data: 5 5 5 5 5 5 6 5 6 5
17581 226/300 Read block 2698 len 20480
17582 227/300 Read block 345 len 16384
17583 228/300 Read block 225 len 8192
17584 229/300 Write block 3324 len 28672 data: 3 2 2 3 3 3 3
17585 230/300 Read block 1226 len 28672
17586 231/300 Read block 3662 len 28672
17587 232/300 Read block 2886 len 20480
17588 233/300 Read block 3363 len 24576
17589 234/300 Write block 2104 len 40960 data: 5 3 2 2 2 2 2 2 2 2
17590 235/300 Flush
17591 236/300 Read block 1893 len 16384
17592 237/300 Read block 2904 len 28672
17593 238/300 Read block 4722 len 20480
17594 239/300 Write block 4615 len 32768 data: 3 4 4 4 4 4 4 4
17595 240/300 Read block 3795 len 16384
17596 241/300 Read block 4211 len 40960
17597 242/300 Write block 1243 len 24576 data: 3 3 4 5 5 5
17598 243/300 Read block 749 len 24576
17599 244/300 Read block 1952 len 32768
17600 245/300 Read block 2348 len 24576
17601 246/300 Read block 3995 len 28672
17602 247/300 Write block 4419 len 24576 data: 5 6 6 5 5 4
17603 248/300 Flush
17604 249/300 Write block 2223 len 12288 data: 4 4 4
17605 250/300 Write block 2029 len 4096 data: 2
17606 251/300 Write block 2301 len 36864 data: 3 3 3 3 2 2 4 5 5
17607 252/300 Read block 3117 len 12288
17608 253/300 Read block 4838 len 20480
17609 254/300 Write block 3196 len 32768 data: 4 4 4 3 3 3 3 2
17610 255/300 Read block 4091 len 32768
17611 256/300 Read block 92 len 36864
17612 257/300 Read block 3825 len 28672
17613 258/300 Read block 105 len 36864
17614 259/300 Write block 4116 len 4096 data: 2
17615 260/300 Read block 2995 len 20480
17616 261/300 Write block 3412 len 20480 data: 5 5 5 6 6
17617 262/300 Read block 2008 len 4096
17618 263/300 Write block 2657 len 20480 data: 4 5 4 4 3
17619 264/300 Flush
17620 265/300 Read block 3862 len 24576
17621 266/300 Write block 4586 len 36864 data: 4 4 3 5 6 6 4 4 3
17622 267/300 Read block 4532 len 16384
17623 268/300 Read block 3886 len 24576
17624 269/300 Read block 1122 len 40960
17625 270/300 Read block 1536 len 16384
17626 271/300 Read block 2176 len 40960
17627 272/300 Read block 3001 len 12288
17628 273/300 Read block 536 len 8192
17629 274/300 Write block 4037 len 12288 data: 7 8 8
17630 275/300 Write block 4372 len 12288 data: 4 4 3
17631 276/300 Write block 2834 len 36864 data: 3 3 4 4 3 2 2 2 4
17632 277/300 Write block 4139 len 20480 data: 2 4 3 4 4
17633 278/300 Write block 2344 len 36864 data: 5 6 6 5 3 3 3 2 2
17634 279/300 Write block 4876 len 12288 data: 5 4 2
17635 280/300 Write block 1137 len 32768 data: 2 2 2 3 3 3 4 4
17636 281/300 Read block 3022 len 8192
17637 282/300 Flush
17638 283/300 Write block 1807 len 12288 data: 3 2 2
17639 284/300 Write block 3976 len 36864 data: 2 2 3 2 2 2 2 2 2
17640 285/300 Write block 4297 len 24576 data: 3 5 6 4 5 5
17641 286/300 Flush
17642 287/300 Write block 2380 len 24576 data: 3 3 3 5 6 6
17643 288/300 Write block 1953 len 24576 data: 3 4 4 4 5 5
17644 289/300 Flush
17645 290/300 Read block 3839 len 36864
17646 291/300 Read block 1514 len 16384
17647 292/300 Write block 1417 len 16384 data: 2 2 2 3
17648 293/300 Flush
17649 294/300 Write block 852 len 20480 data: 6 5 5 4 4
17650 295/300 Read block 2594 len 36864
17651 296/300 Write block 2563 len 20480 data: 5 5 5 7 6
17652 297/300 Write block 674 len 32768 data: 4 4 4 4 5 5 5 5
17653 298/300 Read block 3409 len 12288
17654 299/300 Write block 2810 len 40960 data: 2 2 2 2 3 4 4 5 6 6
17655 300/300 Read block 3510 len 40960
17656 301/300 Write block 2272 len 4096 data: 5
17657 Main task has work to do, go find it
17658 got dsc Start(0)
17659 start 0
17660 Nov 15 02:01:11.328 INFO request completed, latency_us: 216, response_code: 204, uri: /start/cid/0, method: POST, req_id: 35c89022-d64f-433f-8243-3723983d8ce2, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
17661 Replay: started 0, returned:Ok(())
17662 ----------------------------------------------------------------
17663 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
17664 States: Offline Active Active
17665 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
17666 4924 Acked Read 8 Sent Done Done false
17667 4925 Acked Write 10 Sent Done Done false
17668 4926 Acked Write 7 Sent Done Done false
17669 4927 Acked Flush 0 Sent Done Done false
17670 4928 Acked Write 7 Sent Done Done false
17671 4929 Acked Write 7 Sent Done Done false
17672 4930 Acked Flush 0 Sent Done Done false
17673 4931 Acked Read 4 Sent Done Done false
17674 4932 Acked Flush 0 Sent Done Done false
17675 4933 Acked Write 7 Sent Done Done false
17676 4934 Acked Read 8 Sent Done Done false
17677 4935 Acked Write 5 Sent Done Done false
17678 4936 Acked Read 9 Sent Done Done false
17679 4937 Acked Read 1 Sent Done Done false
17680 4938 Acked Write 7 Sent Done Done false
17681 4939 Acked Flush 0 Sent Done Done false
17682 4940 Acked Read 6 Sent Done Done false
17683 4941 Acked Read 7 Sent Done Done false
17684 4942 Acked Read 1 Sent Done Done false
17685 4943 Acked Read 10 Sent Done Done false
17686 4944 Acked Write 3 Sent Done Done false
17687 4945 Acked Read 8 Sent Done Done false
17688 4946 Acked Read 10 Sent Done Done false
17689 4947 Acked Write 6 Sent Done Done false
17690 4948 Acked Write 9 Sent Done Done false
17691 4949 Acked Read 10 Sent Done Done false
17692 4950 Acked Read 7 Sent Done Done false
17693 4951 Acked Read 6 Sent Done Done false
17694 4952 Acked Read 5 Sent Done Done false
17695 4953 Acked Write 5 Sent Done Done false
17696 4954 Acked Read 4 Sent Done Done false
17697 4955 Acked Write 2 Sent Done Done false
17698 4956 Acked Read 1 Sent Done Done false
17699 4957 Acked Read 8 Sent Done Done false
17700 4958 Acked Read 9 Sent Done Done false
17701 4959 Acked Write 1 Sent Done Done false
17702 4960 Acked Read 2 Sent Done Done false
17703 4961 Acked Read 5 Sent Done Done false
17704 4962 Acked Flush 0 Sent Done Done false
17705 4963 Acked Read 3 Sent Done Done false
17706 4964 Acked Write 2 Sent Done Done false
17707 4965 Acked Read 10 Sent Done Done false
17708 4966 Acked Write 3 Sent Done Done false
17709 4967 Acked Flush 0 Sent Done Done false
17710 4968 Acked Write 9 Sent Done Done false
17711 4969 Acked Read 4 Sent Done Done false
17712 4970 Acked Flush 0 Sent Done Done false
17713 4971 Acked Flush 0 Sent Done Done false
17714 4972 Acked Write 10 Sent Done Done false
17715 4973 Acked Read 6 Sent Done Done false
17716 4974 Acked Read 10 Sent Done Done false
17717 4975 Acked Read 3 Sent Done Done false
17718 4976 Acked Read 8 Sent Done Done false
17719 4977 Acked Write 3 Sent Done Done false
17720 4978 Acked Write 5 Sent Done Done false
17721 4979 Acked Write 3 Sent Done Done false
17722 4980 Acked Read 5 Sent Done Done false
17723 4981 Acked Write 4 Sent Done Done false
17724 4982 Acked Read 8 Sent Done Done false
17725 4983 Acked Read 10 Sent Done Done false
17726 4984 Acked Write 2 Sent Done Done false
17727 4985 Acked Write 1 Sent Done Done false
17728 4986 Acked Read 2 Sent Done Done false
17729 4987 Acked Write 7 Sent Done Done false
17730 4988 Acked Read 7 Sent Done Done false
17731 4989 Acked Write 8 Sent Done Done false
17732 4990 Acked Write 8 Sent Done Done false
17733 4991 Acked Read 2 Sent Done Done false
17734 4992 Acked Read 2 Sent Done Done false
17735 4993 Acked Read 5 Sent Done Done false
17736 4994 Acked Flush 0 Sent Done Done false
17737 4995 Acked Read 4 Sent Done Done false
17738 4996 Acked Read 9 Sent Done Done false
17739 4997 Acked Read 10 Sent Done Done false
17740 4998 Acked Write 9 Sent Done Done false
17741 4999 Acked Write 6 Sent Done Done false
17742 5000 Acked Read 8 Sent Done Done false
17743 5001 Acked Write 3 Sent Done Done false
17744 5002 Acked Read 7 Sent Done Done false
17745 5003 Acked Write 2 Sent Done Done false
17746 5004 Acked Write 5 Sent Done Done false
17747 5005 Acked Write 3 Sent Done Done false
17748 5006 Acked Read 8 Sent Done Done false
17749 5007 Acked Read 6 Sent Done Done false
17750 5008 Acked Read 3 Sent Done Done false
17751 5009 Acked Write 1 Sent Done Done false
17752 5010 Acked Read 1 Sent Done Done false
17753 5011 Acked Read 2 Sent Done Done false
17754 5012 Acked Write 6 Sent Done Done false
17755 5013 Acked Write 9 Sent Done Done false
17756 5014 Acked Read 5 Sent Done Done false
17757 5015 Acked Read 9 Sent Done Done false
17758 5016 Acked Write 7 Sent Done Done false
17759 5017 Acked Read 1 Sent Done Done false
17760 5018 Acked Write 7 Sent Done Done false
17761 5019 Acked Write 10 Sent Done Done false
17762 5020 Acked Flush 0 Sent Done Done false
17763 5021 Acked Write 4 Sent Done Done false
17764 5022 Acked Write 9 Sent Done Done false
17765 5023 Acked Write 7 Sent Done Done false
17766 5024 Acked Write 6 Sent Done Done false
17767 5025 Acked Write 10 Sent Done Done false
17768 5026 Acked Read 10 Sent Done Done false
17769 5027 Acked Read 4 Sent Done Done false
17770 5028 Acked Read 7 Sent Done Done false
17771 5029 Acked Read 3 Sent Done Done false
17772 5030 Acked Read 9 Sent Done Done false
17773 5031 Acked Write 7 Sent Done Done false
17774 5032 Acked Read 2 Sent Done Done false
17775 5033 Acked Write 4 Sent Done Done false
17776 5034 Acked Write 5 Sent Done Done false
17777 5035 Acked Read 9 Sent Done Done false
17778 5036 Acked Read 10 Sent Done Done false
17779 5037 Acked Read 4 Sent Done Done false
17780 5038 Acked Read 4 Sent Done Done false
17781 5039 Acked Read 6 Sent Done Done false
17782 5040 Acked Write 8 Sent Done Done false
17783 5041 Acked Write 7 Sent Done Done false
17784 5042 Acked Write 9 Sent Done Done false
17785 5043 Acked Read 5 Sent Done Done false
17786 5044 Acked Read 2 Sent Done Done false
17787 5045 Acked Write 6 Sent Done Done false
17788 5046 Acked Write 7 Sent Done Done false
17789 5047 Acked Read 7 Sent Done Done false
17790 5048 Acked Write 10 Sent Done Done false
17791 5049 Acked Read 2 Sent Done Done false
17792 5050 Acked Read 8 Sent Done Done false
17793 5051 Acked Read 10 Sent Done Done false
17794 5052 Acked Read 9 Sent Done Done false
17795 5053 Acked Write 9 Sent Done Done false
17796 5054 Acked Read 4 Sent Done Done false
17797 5055 Acked Read 4 Sent Done Done false
17798 5056 Acked Flush 0 Sent Done Done false
17799 5057 Acked Read 1 Sent Done Done false
17800 5058 Acked Write 2 Sent Done Done false
17801 5059 Acked Read 3 Sent Done Done false
17802 5060 Acked Write 7 Sent Done Done false
17803 5061 Acked Write 5 Sent Done Done false
17804 5062 Acked Read 10 Sent Done Done false
17805 5063 Acked Write 4 Sent Done Done false
17806 5064 Acked Write 7 Sent Done Done false
17807 5065 Acked Write 10 Sent Done Done false
17808 5066 Acked Read 1 Sent Done Done false
17809 5067 Acked Read 4 Sent Done Done false
17810 5068 Acked Write 6 Sent Done Done false
17811 5069 Acked Read 8 Sent Done Done false
17812 5070 Acked Read 10 Sent Done Done false
17813 5071 Acked Read 3 Sent Done Done false
17814 5072 Acked Read 5 Sent Done Done false
17815 5073 Acked Read 8 Sent Done Done false
17816 5074 Acked Write 8 Sent Done Done false
17817 5075 Acked Write 10 Sent Done Done false
17818 5076 Acked Read 4 Sent Done Done false
17819 5077 Acked Write 1 Sent Done Done false
17820 5078 Acked Read 4 Sent Done Done false
17821 5079 Acked Write 1 Sent Done Done false
17822 5080 Acked Read 6 Sent Done Done false
17823 5081 Acked Write 7 Sent Done Done false
17824 5082 Acked Flush 0 Sent Done Done false
17825 5083 Acked Read 4 Sent Done Done false
17826 5084 Acked Read 1 Sent Done Done false
17827 5085 Acked Read 7 Sent Done Done false
17828 5086 Acked Write 9 Sent Done Done false
17829 5087 Acked Read 9 Sent Done Done false
17830 5088 Acked Read 2 Sent Done Done false
17831 5089 Acked Read 10 Sent Done Done false
17832 5090 Acked Write 2 Sent Done Done false
17833 5091 Acked Write 4 Sent Done Done false
17834 5092 Acked Flush 0 Sent Done Done false
17835 5093 Acked Write 8 Sent Done Done false
17836 5094 Acked Write 3 Sent Done Done false
17837 5095 Acked Write 10 Sent Done Done false
17838 5096 Acked Read 6 Sent Done Done false
17839 5097 Acked Write 8 Sent Done Done false
17840 5098 Acked Write 6 Sent Done Done false
17841 5099 Acked Read 5 Sent Done Done false
17842 5100 Acked Read 5 Sent Done Done false
17843 5101 Acked Write 8 Sent Done Done false
17844 5102 Acked Write 10 Sent Done Done false
17845 5103 Acked Write 9 Sent Done Done false
17846 5104 Acked Read 10 Sent Done Done false
17847 5105 Acked Write 2 Sent Done Done false
17848 5106 Acked Read 3 Sent Done Done false
17849 5107 Acked Read 9 Sent Done Done false
17850 5108 Acked Write 10 Sent Done Done false
17851 5109 Acked Write 4 Sent Done Done false
17852 5110 Acked Read 6 Sent Done Done false
17853 5111 Acked Write 6 Sent Done Done false
17854 5112 Acked Write 5 Sent Done Done false
17855 5113 Acked Write 9 Sent Done Done false
17856 5114 Acked Read 1 Sent Done Done false
17857 5115 Acked Read 7 Sent Done Done false
17858 5116 Acked Read 7 Sent Done Done false
17859 5117 Acked Read 1 Sent Done Done false
17860 5118 Acked Write 8 Sent Done Done false
17861 5119 Acked Read 2 Sent Done Done false
17862 5120 Acked Read 9 Sent Done Done false
17863 5121 Acked Write 4 Sent Done Done false
17864 5122 Acked Read 6 Sent Done Done false
17865 5123 Acked Read 10 Sent Done Done false
17866 5124 Acked Write 8 Sent Done Done false
17867 5125 Acked Write 6 Sent Done Done false
17868 5126 Acked Write 8 Sent Done Done false
17869 5127 Acked Flush 0 Sent Done Done false
17870 5128 Acked Write 2 Sent Done Done false
17871 5129 Acked Read 3 Sent Done Done false
17872 5130 Acked Flush 0 Sent Done Done false
17873 5131 Acked Read 2 Sent Done Done false
17874 5132 Acked Read 2 Sent Done Done false
17875 5133 Acked Write 6 Sent Done Done false
17876 5134 Acked Read 6 Sent Done Done false
17877 5135 Acked Read 2 Sent Done Done false
17878 5136 Acked Write 4 Sent Done Done false
17879 5137 Acked Flush 0 Sent Done Done false
17880 5138 Acked Write 6 Sent Done Done false
17881 5139 Acked Write 8 Sent Done Done false
17882 5140 Acked Write 9 Sent Done Done false
17883 5141 Acked Write 4 Sent Done Done false
17884 5142 Acked Read 1 Sent Done Done false
17885 5143 Acked Read 8 Sent Done Done false
17886 5144 Acked Write 7 Sent Done Done false
17887 5145 Acked Read 9 Sent Done Done false
17888 5146 Acked Read 9 Sent Done Done false
17889 5147 Acked Read 2 Sent Done Done false
17890 5148 Acked Write 10 Sent Done Done false
17891 5149 Acked Read 5 Sent Done Done false
17892 5150 Acked Read 4 Sent Done Done false
17893 5151 Acked Read 2 Sent Done Done false
17894 5152 Acked Write 7 Sent Done Done false
17895 5153 Acked Read 7 Sent Done Done false
17896 5154 Acked Read 7 Sent Done Done false
17897 5155 Acked Read 5 Sent Done Done false
17898 5156 Acked Read 6 Sent Done Done false
17899 5157 Acked Write 10 Sent Done Done false
17900 5158 Acked Flush 0 Sent Done Done false
17901 5159 Acked Read 4 Sent Done Done false
17902 5160 Acked Read 7 Sent Done Done false
17903 5161 Acked Read 5 Sent Done Done false
17904 5162 Acked Write 8 Sent Done Done false
17905 5163 Acked Read 4 Sent Done Done false
17906 5164 Acked Read 10 Sent Done Done false
17907 5165 Acked Write 6 Sent Done Done false
17908 5166 Acked Read 6 Sent Done Done false
17909 5167 Acked Read 8 Sent Done Done false
17910 5168 Acked Read 6 Sent Done Done false
17911 5169 Acked Read 7 Sent Done Done false
17912 5170 Acked Write 6 Sent Done Done false
17913 5171 Acked Flush 0 Sent Done Done false
17914 5172 Acked Write 3 Sent Done Done false
17915 5173 Acked Write 1 Sent Done Done false
17916 5174 Acked Write 9 Sent Done Done false
17917 5175 Acked Read 3 Sent Done Done false
17918 5176 Acked Read 5 Sent Done Done false
17919 5177 Acked Write 8 Sent Done Done false
17920 5178 Acked Read 8 Sent Done Done false
17921 5179 Acked Read 9 Sent Done Done false
17922 5180 Acked Read 7 Sent Done Done false
17923 5181 Acked Read 9 Sent Done Done false
17924 5182 Acked Write 1 Sent Done Done false
17925 5183 Acked Read 5 Sent Done Done false
17926 5184 Acked Write 5 Sent Done Done false
17927 5185 Acked Read 1 Sent Done Done false
17928 5186 Acked Write 5 Sent Done Done false
17929 5187 Acked Flush 0 Sent Done Done false
17930 5188 Acked Read 6 Sent Done Done false
17931 5189 Acked Write 9 Sent Done Done false
17932 5190 Acked Read 4 Sent Done Done false
17933 5191 Acked Read 6 Sent Done Done false
17934 5192 Acked Read 10 Sent Done Done false
17935 5193 Acked Read 4 Sent Done Done false
17936 5194 Acked Read 10 Sent Done Done false
17937 5195 Acked Read 3 Sent Done Done false
17938 5196 Acked Read 2 Sent Done Done false
17939 5197 Acked Write 3 Sent Done Done false
17940 5198 Acked Write 3 Sent Done Done false
17941 5199 Acked Write 9 Sent Done Done false
17942 5200 Acked Write 5 Sent Done Done false
17943 5201 Acked Write 9 Sent Done Done false
17944 5202 Acked Write 3 Sent Done Done false
17945 5203 Acked Write 8 Sent Done Done false
17946 5204 Acked Read 2 Sent Done Done false
17947 5205 Acked Flush 0 Sent Done Done false
17948 5206 Acked Write 3 Sent Done Done false
17949 5207 Acked Write 9 Sent Done Done false
17950 5208 Acked Write 6 Sent Done Done false
17951 5209 Acked Flush 0 Sent Done Done false
17952 5210 Acked Write 6 Sent Done Done false
17953 5211 Acked Write 6 Sent Done Done false
17954 5212 Acked Flush 0 Sent Done Done false
17955 5213 Acked Read 9 Sent Done Done false
17956 5214 Acked Read 4 Sent Done Done false
17957 5215 Acked Write 4 Sent Done Done false
17958 5216 Acked Flush 0 Sent Done Done false
17959 5217 Acked Write 5 Sent Done Done false
17960 5218 Acked Read 9 Sent Done Done false
17961 5219 Acked Write 5 Sent Done Done false
17962 5220 Acked Write 8 Sent Done Done false
17963 5221 Acked Read 3 Sent Done Done false
17964 5222 Acked Write 10 Sent Done Done false
17965 5223 Acked Read 10 Sent Done Done false
17966 5224 Acked Write 1 Sent Done Done false
17967 STATES DS:0 DS:1 DS:2 TOTAL
17968 Sent 301 0 0 301
17969 Done 0 301 301 602
17970 Skipped 0 0 0 0
17971 Error 0 0 0 0
17972 Last Flush: 4923 5216 5216
17973 Downstairs last five completed: 4923 4922 4921 4920 4919
17974 Upstairs last five completed: 5224 5223 5222 5221 5220
17975 CLIENT: Up:0 ds:301 act:2
17976 [8810] got start action
17977 [8810] I am going to restart
17978 Starting downstairs at port 8810
17979 Make output file at "/tmp/dsc/downstairs-8810.txt"
17980 [8810][0] reports Starting
17981 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1219)
17982 [8810][0] reports Running
17983 ----------------------------------------------------------------
17984 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
17985 States: Offline Active Active
17986 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
17987 4924 Acked Read 8 Sent Done Done false
17988 4925 Acked Write 10 Sent Done Done false
17989 4926 Acked Write 7 Sent Done Done false
17990 4927 Acked Flush 0 Sent Done Done false
17991 4928 Acked Write 7 Sent Done Done false
17992 4929 Acked Write 7 Sent Done Done false
17993 4930 Acked Flush 0 Sent Done Done false
17994 4931 Acked Read 4 Sent Done Done false
17995 4932 Acked Flush 0 Sent Done Done false
17996 4933 Acked Write 7 Sent Done Done false
17997 4934 Acked Read 8 Sent Done Done false
17998 4935 Acked Write 5 Sent Done Done false
17999 4936 Acked Read 9 Sent Done Done false
18000 4937 Acked Read 1 Sent Done Done false
18001 4938 Acked Write 7 Sent Done Done false
18002 4939 Acked Flush 0 Sent Done Done false
18003 4940 Acked Read 6 Sent Done Done false
18004 4941 Acked Read 7 Sent Done Done false
18005 4942 Acked Read 1 Sent Done Done false
18006 4943 Acked Read 10 Sent Done Done false
18007 4944 Acked Write 3 Sent Done Done false
18008 4945 Acked Read 8 Sent Done Done false
18009 4946 Acked Read 10 Sent Done Done false
18010 4947 Acked Write 6 Sent Done Done false
18011 4948 Acked Write 9 Sent Done Done false
18012 4949 Acked Read 10 Sent Done Done false
18013 4950 Acked Read 7 Sent Done Done false
18014 4951 Acked Read 6 Sent Done Done false
18015 4952 Acked Read 5 Sent Done Done false
18016 4953 Acked Write 5 Sent Done Done false
18017 4954 Acked Read 4 Sent Done Done false
18018 4955 Acked Write 2 Sent Done Done false
18019 4956 Acked Read 1 Sent Done Done false
18020 4957 Acked Read 8 Sent Done Done false
18021 4958 Acked Read 9 Sent Done Done false
18022 4959 Acked Write 1 Sent Done Done false
18023 4960 Acked Read 2 Sent Done Done false
18024 4961 Acked Read 5 Sent Done Done false
18025 4962 Acked Flush 0 Sent Done Done false
18026 4963 Acked Read 3 Sent Done Done false
18027 4964 Acked Write 2 Sent Done Done false
18028 4965 Acked Read 10 Sent Done Done false
18029 4966 Acked Write 3 Sent Done Done false
18030 4967 Acked Flush 0 Sent Done Done false
18031 4968 Acked Write 9 Sent Done Done false
18032 4969 Acked Read 4 Sent Done Done false
18033 4970 Acked Flush 0 Sent Done Done false
18034 4971 Acked Flush 0 Sent Done Done false
18035 4972 Acked Write 10 Sent Done Done false
18036 4973 Acked Read 6 Sent Done Done false
18037 4974 Acked Read 10 Sent Done Done false
18038 4975 Acked Read 3 Sent Done Done false
18039 4976 Acked Read 8 Sent Done Done false
18040 4977 Acked Write 3 Sent Done Done false
18041 4978 Acked Write 5 Sent Done Done false
18042 4979 Acked Write 3 Sent Done Done false
18043 4980 Acked Read 5 Sent Done Done false
18044 4981 Acked Write 4 Sent Done Done false
18045 4982 Acked Read 8 Sent Done Done false
18046 4983 Acked Read 10 Sent Done Done false
18047 4984 Acked Write 2 Sent Done Done false
18048 4985 Acked Write 1 Sent Done Done false
18049 4986 Acked Read 2 Sent Done Done false
18050 4987 Acked Write 7 Sent Done Done false
18051 4988 Acked Read 7 Sent Done Done false
18052 4989 Acked Write 8 Sent Done Done false
18053 4990 Acked Write 8 Sent Done Done false
18054 4991 Acked Read 2 Sent Done Done false
18055 4992 Acked Read 2 Sent Done Done false
18056 4993 Acked Read 5 Sent Done Done false
18057 4994 Acked Flush 0 Sent Done Done false
18058 4995 Acked Read 4 Sent Done Done false
18059 4996 Acked Read 9 Sent Done Done false
18060 4997 Acked Read 10 Sent Done Done false
18061 4998 Acked Write 9 Sent Done Done false
18062 4999 Acked Write 6 Sent Done Done false
18063 5000 Acked Read 8 Sent Done Done false
18064 5001 Acked Write 3 Sent Done Done false
18065 5002 Acked Read 7 Sent Done Done false
18066 5003 Acked Write 2 Sent Done Done false
18067 5004 Acked Write 5 Sent Done Done false
18068 5005 Acked Write 3 Sent Done Done false
18069 5006 Acked Read 8 Sent Done Done false
18070 5007 Acked Read 6 Sent Done Done false
18071 5008 Acked Read 3 Sent Done Done false
18072 5009 Acked Write 1 Sent Done Done false
18073 5010 Acked Read 1 Sent Done Done false
18074 5011 Acked Read 2 Sent Done Done false
18075 5012 Acked Write 6 Sent Done Done false
18076 5013 Acked Write 9 Sent Done Done false
18077 5014 Acked Read 5 Sent Done Done false
18078 5015 Acked Read 9 Sent Done Done false
18079 5016 Acked Write 7 Sent Done Done false
18080 5017 Acked Read 1 Sent Done Done false
18081 5018 Acked Write 7 Sent Done Done false
18082 5019 Acked Write 10 Sent Done Done false
18083 5020 Acked Flush 0 Sent Done Done false
18084 5021 Acked Write 4 Sent Done Done false
18085 5022 Acked Write 9 Sent Done Done false
18086 5023 Acked Write 7 Sent Done Done false
18087 5024 Acked Write 6 Sent Done Done false
18088 5025 Acked Write 10 Sent Done Done false
18089 5026 Acked Read 10 Sent Done Done false
18090 5027 Acked Read 4 Sent Done Done false
18091 5028 Acked Read 7 Sent Done Done false
18092 5029 Acked Read 3 Sent Done Done false
18093 5030 Acked Read 9 Sent Done Done false
18094 5031 Acked Write 7 Sent Done Done false
18095 5032 Acked Read 2 Sent Done Done false
18096 5033 Acked Write 4 Sent Done Done false
18097 5034 Acked Write 5 Sent Done Done false
18098 5035 Acked Read 9 Sent Done Done false
18099 5036 Acked Read 10 Sent Done Done false
18100 5037 Acked Read 4 Sent Done Done false
18101 5038 Acked Read 4 Sent Done Done false
18102 5039 Acked Read 6 Sent Done Done false
18103 5040 Acked Write 8 Sent Done Done false
18104 5041 Acked Write 7 Sent Done Done false
18105 5042 Acked Write 9 Sent Done Done false
18106 5043 Acked Read 5 Sent Done Done false
18107 5044 Acked Read 2 Sent Done Done false
18108 5045 Acked Write 6 Sent Done Done false
18109 5046 Acked Write 7 Sent Done Done false
18110 5047 Acked Read 7 Sent Done Done false
18111 5048 Acked Write 10 Sent Done Done false
18112 5049 Acked Read 2 Sent Done Done false
18113 5050 Acked Read 8 Sent Done Done false
18114 5051 Acked Read 10 Sent Done Done false
18115 5052 Acked Read 9 Sent Done Done false
18116 5053 Acked Write 9 Sent Done Done false
18117 5054 Acked Read 4 Sent Done Done false
18118 5055 Acked Read 4 Sent Done Done false
18119 5056 Acked Flush 0 Sent Done Done false
18120 5057 Acked Read 1 Sent Done Done false
18121 5058 Acked Write 2 Sent Done Done false
18122 5059 Acked Read 3 Sent Done Done false
18123 5060 Acked Write 7 Sent Done Done false
18124 5061 Acked Write 5 Sent Done Done false
18125 5062 Acked Read 10 Sent Done Done false
18126 5063 Acked Write 4 Sent Done Done false
18127 5064 Acked Write 7 Sent Done Done false
18128 5065 Acked Write 10 Sent Done Done false
18129 5066 Acked Read 1 Sent Done Done false
18130 5067 Acked Read 4 Sent Done Done false
18131 5068 Acked Write 6 Sent Done Done false
18132 5069 Acked Read 8 Sent Done Done false
18133 5070 Acked Read 10 Sent Done Done false
18134 5071 Acked Read 3 Sent Done Done false
18135 5072 Acked Read 5 Sent Done Done false
18136 5073 Acked Read 8 Sent Done Done false
18137 5074 Acked Write 8 Sent Done Done false
18138 5075 Acked Write 10 Sent Done Done false
18139 5076 Acked Read 4 Sent Done Done false
18140 5077 Acked Write 1 Sent Done Done false
18141 5078 Acked Read 4 Sent Done Done false
18142 5079 Acked Write 1 Sent Done Done false
18143 5080 Acked Read 6 Sent Done Done false
18144 5081 Acked Write 7 Sent Done Done false
18145 5082 Acked Flush 0 Sent Done Done false
18146 5083 Acked Read 4 Sent Done Done false
18147 5084 Acked Read 1 Sent Done Done false
18148 5085 Acked Read 7 Sent Done Done false
18149 5086 Acked Write 9 Sent Done Done false
18150 5087 Acked Read 9 Sent Done Done false
18151 5088 Acked Read 2 Sent Done Done false
18152 5089 Acked Read 10 Sent Done Done false
18153 5090 Acked Write 2 Sent Done Done false
18154 5091 Acked Write 4 Sent Done Done false
18155 5092 Acked Flush 0 Sent Done Done false
18156 5093 Acked Write 8 Sent Done Done false
18157 5094 Acked Write 3 Sent Done Done false
18158 5095 Acked Write 10 Sent Done Done false
18159 5096 Acked Read 6 Sent Done Done false
18160 5097 Acked Write 8 Sent Done Done false
18161 5098 Acked Write 6 Sent Done Done false
18162 5099 Acked Read 5 Sent Done Done false
18163 5100 Acked Read 5 Sent Done Done false
18164 5101 Acked Write 8 Sent Done Done false
18165 5102 Acked Write 10 Sent Done Done false
18166 5103 Acked Write 9 Sent Done Done false
18167 5104 Acked Read 10 Sent Done Done false
18168 5105 Acked Write 2 Sent Done Done false
18169 5106 Acked Read 3 Sent Done Done false
18170 5107 Acked Read 9 Sent Done Done false
18171 5108 Acked Write 10 Sent Done Done false
18172 5109 Acked Write 4 Sent Done Done false
18173 5110 Acked Read 6 Sent Done Done false
18174 5111 Acked Write 6 Sent Done Done false
18175 5112 Acked Write 5 Sent Done Done false
18176 5113 Acked Write 9 Sent Done Done false
18177 5114 Acked Read 1 Sent Done Done false
18178 5115 Acked Read 7 Sent Done Done false
18179 5116 Acked Read 7 Sent Done Done false
18180 5117 Acked Read 1 Sent Done Done false
18181 5118 Acked Write 8 Sent Done Done false
18182 5119 Acked Read 2 Sent Done Done false
18183 5120 Acked Read 9 Sent Done Done false
18184 5121 Acked Write 4 Sent Done Done false
18185 5122 Acked Read 6 Sent Done Done false
18186 5123 Acked Read 10 Sent Done Done false
18187 5124 Acked Write 8 Sent Done Done false
18188 5125 Acked Write 6 Sent Done Done false
18189 5126 Acked Write 8 Sent Done Done false
18190 5127 Acked Flush 0 Sent Done Done false
18191 5128 Acked Write 2 Sent Done Done false
18192 5129 Acked Read 3 Sent Done Done false
18193 5130 Acked Flush 0 Sent Done Done false
18194 5131 Acked Read 2 Sent Done Done false
18195 5132 Acked Read 2 Sent Done Done false
18196 5133 Acked Write 6 Sent Done Done false
18197 5134 Acked Read 6 Sent Done Done false
18198 5135 Acked Read 2 Sent Done Done false
18199 5136 Acked Write 4 Sent Done Done false
18200 5137 Acked Flush 0 Sent Done Done false
18201 5138 Acked Write 6 Sent Done Done false
18202 5139 Acked Write 8 Sent Done Done false
18203 5140 Acked Write 9 Sent Done Done false
18204 5141 Acked Write 4 Sent Done Done false
18205 5142 Acked Read 1 Sent Done Done false
18206 5143 Acked Read 8 Sent Done Done false
18207 5144 Acked Write 7 Sent Done Done false
18208 5145 Acked Read 9 Sent Done Done false
18209 5146 Acked Read 9 Sent Done Done false
18210 5147 Acked Read 2 Sent Done Done false
18211 5148 Acked Write 10 Sent Done Done false
18212 5149 Acked Read 5 Sent Done Done false
18213 5150 Acked Read 4 Sent Done Done false
18214 5151 Acked Read 2 Sent Done Done false
18215 5152 Acked Write 7 Sent Done Done false
18216 5153 Acked Read 7 Sent Done Done false
18217 5154 Acked Read 7 Sent Done Done false
18218 5155 Acked Read 5 Sent Done Done false
18219 5156 Acked Read 6 Sent Done Done false
18220 5157 Acked Write 10 Sent Done Done false
18221 5158 Acked Flush 0 Sent Done Done false
18222 5159 Acked Read 4 Sent Done Done false
18223 5160 Acked Read 7 Sent Done Done false
18224 5161 Acked Read 5 Sent Done Done false
18225 5162 Acked Write 8 Sent Done Done false
18226 5163 Acked Read 4 Sent Done Done false
18227 5164 Acked Read 10 Sent Done Done false
18228 5165 Acked Write 6 Sent Done Done false
18229 5166 Acked Read 6 Sent Done Done false
18230 5167 Acked Read 8 Sent Done Done false
18231 5168 Acked Read 6 Sent Done Done false
18232 5169 Acked Read 7 Sent Done Done false
18233 5170 Acked Write 6 Sent Done Done false
18234 5171 Acked Flush 0 Sent Done Done false
18235 5172 Acked Write 3 Sent Done Done false
18236 5173 Acked Write 1 Sent Done Done false
18237 5174 Acked Write 9 Sent Done Done false
18238 5175 Acked Read 3 Sent Done Done false
18239 5176 Acked Read 5 Sent Done Done false
18240 5177 Acked Write 8 Sent Done Done false
18241 5178 Acked Read 8 Sent Done Done false
18242 5179 Acked Read 9 Sent Done Done false
18243 5180 Acked Read 7 Sent Done Done false
18244 5181 Acked Read 9 Sent Done Done false
18245 5182 Acked Write 1 Sent Done Done false
18246 5183 Acked Read 5 Sent Done Done false
18247 5184 Acked Write 5 Sent Done Done false
18248 5185 Acked Read 1 Sent Done Done false
18249 5186 Acked Write 5 Sent Done Done false
18250 5187 Acked Flush 0 Sent Done Done false
18251 5188 Acked Read 6 Sent Done Done false
18252 5189 Acked Write 9 Sent Done Done false
18253 5190 Acked Read 4 Sent Done Done false
18254 5191 Acked Read 6 Sent Done Done false
18255 5192 Acked Read 10 Sent Done Done false
18256 5193 Acked Read 4 Sent Done Done false
18257 5194 Acked Read 10 Sent Done Done false
18258 5195 Acked Read 3 Sent Done Done false
18259 5196 Acked Read 2 Sent Done Done false
18260 5197 Acked Write 3 Sent Done Done false
18261 5198 Acked Write 3 Sent Done Done false
18262 5199 Acked Write 9 Sent Done Done false
18263 5200 Acked Write 5 Sent Done Done false
18264 5201 Acked Write 9 Sent Done Done false
18265 5202 Acked Write 3 Sent Done Done false
18266 5203 Acked Write 8 Sent Done Done false
18267 5204 Acked Read 2 Sent Done Done false
18268 5205 Acked Flush 0 Sent Done Done false
18269 5206 Acked Write 3 Sent Done Done false
18270 5207 Acked Write 9 Sent Done Done false
18271 5208 Acked Write 6 Sent Done Done false
18272 5209 Acked Flush 0 Sent Done Done false
18273 5210 Acked Write 6 Sent Done Done false
18274 5211 Acked Write 6 Sent Done Done false
18275 5212 Acked Flush 0 Sent Done Done false
18276 5213 Acked Read 9 Sent Done Done false
18277 5214 Acked Read 4 Sent Done Done false
18278 5215 Acked Write 4 Sent Done Done false
18279 5216 Acked Flush 0 Sent Done Done false
18280 5217 Acked Write 5 Sent Done Done false
18281 5218 Acked Read 9 Sent Done Done false
18282 5219 Acked Write 5 Sent Done Done false
18283 5220 Acked Write 8 Sent Done Done false
18284 5221 Acked Read 3 Sent Done Done false
18285 5222 Acked Write 10 Sent Done Done false
18286 5223 Acked Read 10 Sent Done Done false
18287 5224 Acked Write 1 Sent Done Done false
18288 5225 Acked Flush 0 Sent Done Done false
18289 STATES DS:0 DS:1 DS:2 TOTAL
18290 Sent 302 0 0 302
18291 Done 0 302 302 604
18292 Skipped 0 0 0 0
18293 Error 0 0 0 0
18294 Last Flush: 4923 5225 5225
18295 Downstairs last five completed: 4923 4922 4921 4920 4919
18296 Upstairs last five completed: 5225 5224 5223 5222 5221
18297 CLIENT: Up:0 ds:302 act:2
182982024-11-15T02:01:17.082ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
182992024-11-15T02:01:17.082ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:64409) = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
183002024-11-15T02:01:17.083ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
183012024-11-15T02:01:17.084ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
183022024-11-15T02:01:17.084ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
183032024-11-15T02:01:17.084ZINFOcrucible: send last flush ID to this DS: 4923 = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
183042024-11-15T02:01:17.084ZINFOcrucible: Replied this last flush ID: 4923 = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
183052024-11-15T02:01:17.084ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
183062024-11-15T02:01:17.084ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
183072024-11-15T02:01:17.084ZINFOcrucible: [0] client re-new 302 jobs since flush 4923 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
183082024-11-15T02:01:17.084ZINFOcrucible: [0] Replayed 302 jobs since flush: 4923 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
18309 ----------------------------------------------------------------
18310 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
18311 Downstairs last five completed: 5225 5224 5223 5222 5221
18312 Upstairs last five completed: 5225 5224 5223 5222 5221
18313 CLIENT: Up:0 ds:0 act:3
18314 Replay: All jobs finished
18315 Main task has work to do, go find it
18316 got dsc Stop(1)
18317 stop 1
18318 Nov 15 02:01:19.334 INFO request completed, latency_us: 363, response_code: 204, uri: /stop/cid/1[8820] Got stop action so:false kr:false
18319 , method: POST, req_id: 6888620e-4183-4dde-a95d-62036b9e2669, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
183202024-11-15T02:01:19.335ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
18321 Nov 15 02:01:19.335 INFO request completed, {"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2024-11-15T02:01:19.33510951Z","hostname":"w-01JCPRHT5F2MPA44KVJHM3S2PB","pid":1205,"":"io task","client":"1","":"downstairs","session_id":"971dcb2e-6891-4ecf-b367-1e04e52065ee"}
18322 latency_us: 177, response_code: 200, uri: /state/cid/1, method: GET{"msg":"downstairs task for 1 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2024-11-15T02:01:19.335158297Z","hostname":"w-01JCPRHT5F2MPA44KVJHM3S2PB","pid":1205,"session_id":"971dcb2e-6891-4ecf-b367-1e04e52065ee"}
18323 , req_id: e382c3e2-ad7e-4124-aedc-77bc912fcd48, remote_addr: 127.0.0.1:40008, local_addr{"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-15T02:01:19.335199494Z","hostname":"w-01JCPRHT5F2MPA44KVJHM3S2PB","pid":1205,"client":"1","":"downstairs","session_id":"971dcb2e-6891-4ecf-b367-1e04e52065ee"}
18324 : 127.0.0.1:9998
183252024-11-15T02:01:19.335ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
183262024-11-15T02:01:19.335ZINFOcrucible: client task is exiting = io task client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
18327 [8820] Exited with: ExitStatus(unix_wait_status(9))
18328 [8820][1] reports Exit
18329 Nov 15 02:01:23.338 INFO request completed, latency_us: 234, response_code: 200, uri: /state/cid/1, method: GET, req_id: 08827605-9bf4-47e1-ba48-41ab5a45aa04, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
18330 001/300 Read block 4247 len 36864
18331 002/300 Write block 4315 len 4096 data: 5
18332 003/300 Read block 2925 len 4096
18333 004/300 Read block 3909 len 8192
18334 005/300 Read block 4590 len 32768
18335 006/300 Read block 2556 len 16384
18336 007/300 Read block 1003 len 40960
18337 008/300 Write block 1003 len 36864 data: 5 4 4 3 3 4 4 4 5
18338 009/300 Write block 556 len 28672 data: 3 4 3 3 3 3 3
18339 010/300 Read block 2454 len 40960
18340 011/300 Write block 4570 len 12288 data: 3 2 2
18341 012/300 Write block 4614 len 28672 data: 3 4 5 5 5 5 5
18342 013/300 Write block 2358 len 20480 data: 2 3 6 6 6
18343 014/300 Flush
18344 015/300 Write block 3835 len 28672 data: 4 6 6 6 6 6 5
18345 016/300 Flush
18346 017/300 Write block 4729 len 36864 data: 4 6 5 3 3 3 3 3 3
18347 018/300 Read block 2696 len 36864
18348 019/300 Flush
18349 020/300 Write block 1602 len 20480 data: 3 3 3 5 6
18350 021/300 Read block 4203 len 4096
18351 022/300 Read block 228 len 12288
18352 023/300 Write block 2967 len 40960 data: 3 2 2 2 4 4 4 4 4 4
18353 024/300 Read block 1382 len 40960
18354 025/300 Read block 181 len 8192
18355 026/300 Read block 641 len 32768
18356 027/300 Read block 2506 len 4096
18357 028/300 Read block 1540 len 12288
18358 029/300 Read block 1145 len 12288
18359 030/300 Flush
18360 031/300 Read block 3651 len 28672
18361 032/300 Write block 3498 len 40960 data: 3 4 4 4 3 3 3 4 4 4
18362 033/300 Write block 4970 len 4096 data: 2
18363 034/300 Read block 2871 len 24576
18364 035/300 Read block 463 len 40960
18365 036/300 Read block 3284 len 8192
18366 037/300 Read block 4310 len 20480
18367 038/300 Write block 4857 len 40960 data: 3 2 4 4 4 4 4 3 4 4
18368 039/300 Write block 3867 len 4096 data: 5
18369 040/300 Write block 3112 len 4096 data: 3
18370 041/300 Read block 4569 len 12288
18371 042/300 Write block 3260 len 36864 data: 5 6 5 4 4 3 2 3 4
18372 043/300 Flush
18373 044/300 Write block 3197 len 12288 data: 5 5 4
18374 045/300 Read block 3655 len 4096
18375 046/300 Read block 4846 len 4096
18376 047/300 Flush
18377 048/300 Write block 621 len 16384 data: 6 6 7 7
18378 049/300 Read block 2314 len 16384
18379 050/300 Write block 1884 len 20480 data: 3 4 4 4 5
18380 051/300 Write block 3909 len 32768 data: 5 2 2 3 3 3 2 2
18381 052/300 Write block 321 len 8192 data: 4 4
18382 053/300 Write block 2987 len 40960 data: 5 4 3 3 4 4 2 2 2 2
18383 054/300 Read block 3410 len 4096
18384 055/300 Read block 2573 len 8192
18385 056/300 Read block 2018 len 12288
18386 057/300 Write block 207 len 40960 data: 4 4 4 4 4 3 3 4 4 4
18387 058/300 Read block 3403 len 32768
18388 059/300 Write block 2294 len 24576 data: 3 3 2 2 2 2
18389 060/300 Read block 4659 len 36864
18390 061/300 Write block 4178 len 40960 data: 6 6 6 5 5 5 6 6 4 3
18391 062/300 Write block 912 len 20480 data: 2 2 2 2 2
18392 063/300 Read block 4187 len 4096
18393 064/300 Write block 1919 len 16384 data: 3 3 3 3
18394 065/300 Write block 1140 len 40960 data: 4 4 4 5 5 5 6 6 6 4
18395 066/300 Flush
18396 067/300 Write block 4306 len 40960 data: 6 5 7 7 4 4 6 6 5 6
18397 068/300 Write block 3862 len 40960 data: 4 4 5 5 5 6 6 6 6 6
18398 069/300 Read block 1780 len 16384
18399 070/300 Read block 2992 len 24576
18400 071/300 Read block 1135 len 20480
18401 072/300 Write block 526 len 12288 data: 4 5 3
18402 073/300 Write block 4801 len 8192 data: 4 3
18403 074/300 Write block 3595 len 4096 data: 5
18404 075/300 Flush
18405 076/300 Write block 917 len 28672 data: 2 3 3 4 3 3 3
18406 077/300 Read block 365 len 8192
18407 078/300 Read block 3293 len 20480
18408 079/300 Read block 4581 len 4096
18409 080/300 Read block 468 len 28672
18410 081/300 Read block 2789 len 24576
18411 082/300 Write block 2239 len 8192 data: 4 4
18412 083/300 Read block 4377 len 40960
18413 084/300 Write block 3811 len 16384 data: 3 2 2 2
18414 085/300 Write block 3690 len 40960 data: 6 6 5 4 4 4 4 3 4 4
18415 086/300 Write block 3485 len 16384 data: 3 3 3 4
18416 087/300 Write block 1033 len 4096 data: 3
18417 088/300 Read block 2 len 24576
18418 089/300 Write block 57 len 28672 data: 4 5 4 4 4 3 3
18419 090/300 Flush
18420 091/300 Read block 4481 len 32768
18421 092/300 Write block 1835 len 16384 data: 2 2 4 4
18422 093/300 Flush
18423 094/300 Write block 4219 len 12288 data: 4 6 5
18424 095/300 Flush
18425 096/300 Read block 1320 len 32768
18426 097/300 Read block 2158 len 36864
18427 098/300 Read block 22 len 28672
18428 099/300 Write block 26 len 36864 data: 3 3 3 3 3 4 4 4 4
18429 100/300 Read block 1961 len 12288
18430 101/300 Flush
18431 102/300 Write block 906 len 20480 data: 5 4 4 5 3
18432 103/300 Write block 1688 len 4096 data: 3
18433 104/300 Write block 2967 len 8192 data: 4 3
18434 105/300 Read block 3702 len 12288
18435 106/300 Write block 3358 len 20480 data: 2 2 2 4 3
18436 107/300 Write block 670 len 4096 data: 5
18437 108/300 Read block 4453 len 4096
18438 109/300 Read block 2336 len 36864
18439 110/300 Read block 4666 len 40960
18440 111/300 Read block 153 len 12288
18441 112/300 Write block 4253 len 20480 data: 6 6 4 4 4
18442 113/300 Read block 2817 len 16384
18443 114/300 Write block 2304 len 28672 data: 4 3 3 5 6 6 4
18444 115/300 Write block 2389 len 28672 data: 7 5 4 4 4 4 2
18445 116/300 Flush
18446 117/300 Write block 4266 len 12288 data: 7 5 5
18447 118/300 Write block 1816 len 24576 data: 3 2 2 2 3 4
18448 119/300 Read block 3628 len 16384
18449 120/300 Flush
18450 121/300 Read block 328 len 4096
18451 122/300 Read block 4934 len 20480
18452 123/300 Write block 930 len 8192 data: 5 5
18453 124/300 Write block 3811 len 8192 data: 4 3
18454 125/300 Write block 23 len 24576 data: 3 3 3 4 4 4
18455 126/300 Read block 3532 len 32768
18456 127/300 Write block 1446 len 8192 data: 4 3
18457 128/300 Read block 2246 len 20480
18458 129/300 Read block 3769 len 24576
18459 130/300 Read block 3319 len 28672
18460 131/300 Read block 4671 len 36864
18461 132/300 Flush
18462 133/300 Read block 1204 len 12288
18463 134/300 Read block 611 len 4096
18464 135/300 Read block 957 len 36864
18465 136/300 Read block 1575 len 4096
18466 137/300 Flush
18467 138/300 Write block 4 len 20480 data: 4 3 5 5 4
18468 139/300 Read block 505 len 24576
18469 140/300 Write block 1297 len 20480 data: 4 4 5 4 4
18470 141/300 Read block 1306 len 8192
18471 142/300 Read block 839 len 4096
18472 143/300 Write block 1647 len 40960 data: 3 3 2 2 3 4 4 5 5 5
18473 144/300 Read block 4589 len 36864
18474 145/300 Write block 2886 len 32768 data: 3 3 4 4 4 3 3 3
18475 146/300 Flush
18476 147/300 Read block 852 len 28672
18477 148/300 Write block 1065 len 36864 data: 3 4 5 5 5 4 4 4 4
18478 149/300 Write block 3919 len 12288 data: 3 4 4
18479 150/300 Read block 3432 len 4096
18480 151/300 Write block 693 len 32768 data: 3 2 2 2 2 2 4 5
18481 152/300 Read block 1054 len 28672
18482 153/300 Write block 3962 len 12288 data: 3 2 2
18483 154/300 Read block 324 len 28672
18484 155/300 Write block 767 len 40960 data: 4 4 2 2 3 4 4 4 3 2
18485 156/300 Read block 1021 len 12288
18486 157/300 Read block 1309 len 32768
18487 158/300 Write block 4696 len 32768 data: 3 2 4 4 4 5 5 6
18488 159/300 Read block 4917 len 32768
18489 160/300 Read block 3834 len 36864
18490 161/300 Flush
18491 162/300 Read block 4513 len 40960
18492 163/300 Write block 184 len 36864 data: 2 3 2 2 2 2 2 4 4
18493 164/300 Write block 2297 len 32768 data: 3 3 3 2 4 4 4 5
18494 165/300 Write block 483 len 12288 data: 4 4 5
18495 166/300 Write block 2152 len 32768 data: 6 4 4 5 4 4 4 5
18496 167/300 Read block 3462 len 16384
18497 168/300 Write block 3242 len 32768 data: 2 4 4 6 6 6 6 6
18498 169/300 Write block 3269 len 24576 data: 3 3 3 3 4 2
18499 170/300 Read block 4341 len 20480
18500 171/300 Read block 3794 len 24576
18501 172/300 Write block 1372 len 40960 data: 5 6 5 5 4 4 4 5 5 5
18502 173/300 Read block 4285 len 24576
18503 174/300 Read block 3592 len 32768
18504 175/300 Read block 4609 len 8192
18505 176/300 Read block 1722 len 36864
18506 177/300 Read block 876 len 28672
18507 178/300 Read block 3052 len 28672
18508 179/300 Write block 2230 len 4096 data: 6
18509 180/300 Read block 1099 len 36864
18510 181/300 Read block 747 len 20480
18511 182/300 Read block 4523 len 16384
18512 183/300 Read block 1768 len 20480
18513 184/300 Write block 789 len 40960 data: 6 5 4 3 4 5 5 4 4 4
18514 185/300 Write block 2315 len 28672 data: 3 3 2 4 5 5 5
18515 186/300 Read block 1535 len 24576
18516 187/300 Write block 1034 len 12288 data: 3 3 3
18517 188/300 Write block 451 len 20480 data: 3 3 3 3 2
18518 189/300 Read block 154 len 40960
18519 190/300 Write block 3510 len 32768 data: 2 2 2 2 2 2 2 3
18520 191/300 Flush
18521 192/300 Read block 380 len 20480
18522 193/300 Read block 3312 len 36864
18523 194/300 Write block 4606 len 40960 data: 3 3 4 3 3 2 4 3 4 5
18524 195/300 Read block 386 len 4096
18525 196/300 Read block 3952 len 16384
18526 197/300 Read block 4680 len 24576
18527 198/300 Flush
18528 199/300 Write block 1747 len 8192 data: 5 3
18529 200/300 Flush
18530 201/300 Read block 2532 len 8192
18531 202/300 Read block 3392 len 4096
18532 203/300 Read block 1393 len 40960
18533 204/300 Read block 1952 len 36864
18534 205/300 Read block 551 len 36864
18535 206/300 Write block 2441 len 4096 data: 3
18536 207/300 Write block 3971 len 40960 data: 4 4 3 3 2 3 3 4 3 3
18537 208/300 Read block 1723 len 8192
18538 209/300 Read block 236 len 40960
18539 210/300 Write block 2909 len 8192 data: 3 3
18540 211/300 Read block 1028 len 12288
18541 212/300 Flush
18542 213/300 Write block 4091 len 20480 data: 3 3 3 3 3
18543 214/300 Read block 927 len 20480
18544 215/300 Read block 238 len 8192
18545 216/300 Write block 2574 len 12288 data: 4 4 6
18546 217/300 Read block 2331 len 4096
18547 218/300 Read block 576 len 28672
18548 219/300 Read block 4109 len 8192
18549 220/300 Flush
18550 221/300 Read block 1912 len 4096
18551 222/300 Read block 1743 len 24576
18552 223/300 Read block 4236 len 16384
18553 224/300 Write block 2655 len 4096 data: 3
18554 225/300 Flush
18555 226/300 Read block 4068 len 24576
18556 227/300 Write block 997 len 32768 data: 6 6 5 5 5 5 6 5
18557 228/300 Read block 1322 len 12288
18558 229/300 Write block 3093 len 4096 data: 5
18559 230/300 Write block 3850 len 8192 data: 5 5
18560 231/300 Read block 3282 len 8192
18561 232/300 Flush
18562 233/300 Read block 3178 len 24576
18563 234/300 Read block 969 len 12288
18564 235/300 Read block 4346 len 28672
18565 236/300 Write block 2590 len 4096 data: 5
18566 237/300 Read block 1666 len 32768
18567 238/300 Write block 4199 len 28672 data: 5 4 3 5 5 5 6
18568 239/300 Read block 3502 len 40960
18569 240/300 Read block 4672 len 16384
18570 241/300 Read block 2616 len 32768
18571 242/300 Write block 4719 len 24576 data: 4 3 3 3 3 5
18572 243/300 Read block 1392 len 32768
18573 244/300 Write block 375 len 8192 data: 3 3
18574 245/300 Flush
18575 246/300 Read block 2565 len 28672
18576 247/300 Read block 1086 len 16384
18577 248/300 Read block 2512 len 12288
18578 249/300 Write block 1138 len 40960 data: 3 3 5 5 5 6 6 6 7 7
18579 250/300 Read block 3868 len 16384
18580 251/300 Write block 1788 len 4096 data: 5
18581 252/300 Flush
18582 253/300 Write block 4536 len 32768 data: 4 3 3 3 3 3 3 3
18583 254/300 Read block 2654 len 20480
18584 255/300 Flush
18585 256/300 Flush
18586 257/300 Read block 1392 len 12288
18587 258/300 Flush
18588 259/300 Read block 716 len 16384
18589 260/300 Read block 3848 len 40960
18590 261/300 Write block 3160 len 16384 data: 6 5 4 3
18591 262/300 Write block 3706 len 32768 data: 3 4 5 6 5 5 5 4
18592 263/300 Write block 4586 len 32768 data: 5 5 4 6 7 7 5 5
18593 264/300 Read block 1019 len 20480
18594 265/300 Write block 4235 len 36864 data: 5 5 5 4 4 2 2 4 3
18595 266/300 Read block 3688 len 40960
18596 267/300 Write block 3495 len 28672 data: 2 2 3 4 5 5 5
18597 268/300 Read block 1461 len 16384
18598 269/300 Flush
18599 270/300 Read block 2978 len 32768
18600 271/300 Read block 881 len 12288
18601 272/300 Write block 4774 len 40960 data: 4 4 5 4 6 5 5 5 5 4
18602 273/300 Read block 1435 len 12288
18603 274/300 Write block 234 len 16384 data: 4 4 5 3
18604 275/300 Read block 3158 len 36864
18605 276/300 Read block 122 len 4096
18606 277/300 Write block 1405 len 40960 data: 4 4 4 4 3 4 4 4 4 3
18607 278/300 Read block 587 len 8192
18608 279/300 Read block 4299 len 24576
18609 280/300 Read block 2367 len 32768
18610 281/300 Read block 3217 len 12288
18611 282/300 Write block 3761 len 16384 data: 5 5 4 4
18612 283/300 Read block 2869 len 28672
18613 284/300 Read block 1952 len 16384
18614 285/300 Read block 1571 len 12288
18615 286/300 Flush
18616 287/300 Write block 4178 len 32768 data: 7 7 7 6 6 6 7 7
18617 288/300 Read block 1988 len 40960
18618 289/300 Read block 4473 len 8192
18619 290/300 Read block 3199 len 16384
18620 291/300 Read block 1339 len 28672
18621 292/300 Read block 3527 len 36864
18622 293/300 Flush
18623 294/300 Flush
18624 295/300 Write block 2849 len 32768 data: 4 2 2 2 2 2 2 3
18625 296/300 Write block 4241 len 4096 data: 3
18626 297/300 Read block 4507 len 4096
18627 298/300 Read block 3845 len 40960
18628 299/300 Write block 4667 len 4096 data: 6
18629 300/300 Read block 4165 len 12288
18630 301/300 Write block 326 len 32768 data: 5 7 5 5 3 4 4 5
18631 Main task has work to do, go find it
18632 got dsc Start(1)
18633 start 1
18634 Nov 15 02:01:23.577 INFO request completed, latency_us: 226, response_code: 204, uri: /start/cid/1, method: POST, req_id: 5875e094-f022-4316-b73a-131aa7d14677, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
18635 Replay: started 1, returned:Ok(())
18636 ----------------------------------------------------------------
18637 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
18638 States: Active Offline Active
18639 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
18640 5226 Acked Read 9 Done Sent Done false
18641 5227 Acked Write 1 Done Sent Done false
18642 5228 Acked Read 1 Done Sent Done false
18643 5229 Acked Read 2 Done Sent Done false
18644 5230 Acked Read 8 Done Sent Done false
18645 5231 Acked Read 4 Done Sent Done false
18646 5232 Acked Read 10 Done Sent Done false
18647 5233 Acked Write 9 Done Sent Done false
18648 5234 Acked Write 7 Done Sent Done false
18649 5235 Acked Read 10 Done Sent Done false
18650 5236 Acked Write 3 Done Sent Done false
18651 5237 Acked Write 7 Done Sent Done false
18652 5238 Acked Write 5 Done Sent Done false
18653 5239 Acked Flush 0 Done Sent Done false
18654 5240 Acked Write 7 Done Sent Done false
18655 5241 Acked Flush 0 Done Sent Done false
18656 5242 Acked Write 9 Done Sent Done false
18657 5243 Acked Read 9 Done Sent Done false
18658 5244 Acked Flush 0 Done Sent Done false
18659 5245 Acked Write 5 Done Sent Done false
18660 5246 Acked Read 1 Done Sent Done false
18661 5247 Acked Read 3 Done Sent Done false
18662 5248 Acked Write 10 Done Sent Done false
18663 5249 Acked Read 10 Done Sent Done false
18664 5250 Acked Read 2 Done Sent Done false
18665 5251 Acked Read 8 Done Sent Done false
18666 5252 Acked Read 1 Done Sent Done false
18667 5253 Acked Read 3 Done Sent Done false
18668 5254 Acked Read 3 Done Sent Done false
18669 5255 Acked Flush 0 Done Sent Done false
18670 5256 Acked Read 7 Done Sent Done false
18671 5257 Acked Write 10 Done Sent Done false
18672 5258 Acked Write 1 Done Sent Done false
18673 5259 Acked Read 6 Done Sent Done false
18674 5260 Acked Read 10 Done Sent Done false
18675 5261 Acked Read 2 Done Sent Done false
18676 5262 Acked Read 5 Done Sent Done false
18677 5263 Acked Write 10 Done Sent Done false
18678 5264 Acked Write 1 Done Sent Done false
18679 5265 Acked Write 1 Done Sent Done false
18680 5266 Acked Read 3 Done Sent Done false
18681 5267 Acked Write 9 Done Sent Done false
18682 5268 Acked Flush 0 Done Sent Done false
18683 5269 Acked Write 3 Done Sent Done false
18684 5270 Acked Read 1 Done Sent Done false
18685 5271 Acked Read 1 Done Sent Done false
18686 5272 Acked Flush 0 Done Sent Done false
18687 5273 Acked Write 4 Done Sent Done false
18688 5274 Acked Read 4 Done Sent Done false
18689 5275 Acked Write 5 Done Sent Done false
18690 5276 Acked Write 8 Done Sent Done false
18691 5277 Acked Write 2 Done Sent Done false
18692 5278 Acked Write 10 Done Sent Done false
18693 5279 Acked Read 1 Done Sent Done false
18694 5280 Acked Read 2 Done Sent Done false
18695 5281 Acked Read 3 Done Sent Done false
18696 5282 Acked Write 10 Done Sent Done false
18697 5283 Acked Read 8 Done Sent Done false
18698 5284 Acked Write 6 Done Sent Done false
18699 5285 Acked Read 9 Done Sent Done false
18700 5286 Acked Write 10 Done Sent Done false
18701 5287 Acked Write 5 Done Sent Done false
18702 5288 Acked Read 1 Done Sent Done false
18703 5289 Acked Write 4 Done Sent Done false
18704 5290 Acked Write 10 Done Sent Done false
18705 5291 Acked Flush 0 Done Sent Done false
18706 5292 Acked Write 10 Done Sent Done false
18707 5293 Acked Write 10 Done Sent Done false
18708 5294 Acked Read 4 Done Sent Done false
18709 5295 Acked Read 6 Done Sent Done false
18710 5296 Acked Read 5 Done Sent Done false
18711 5297 Acked Write 3 Done Sent Done false
18712 5298 Acked Write 2 Done Sent Done false
18713 5299 Acked Write 1 Done Sent Done false
18714 5300 Acked Flush 0 Done Sent Done false
18715 5301 Acked Write 7 Done Sent Done false
18716 5302 Acked Read 2 Done Sent Done false
18717 5303 Acked Read 5 Done Sent Done false
18718 5304 Acked Read 1 Done Sent Done false
18719 5305 Acked Read 7 Done Sent Done false
18720 5306 Acked Read 6 Done Sent Done false
18721 5307 Acked Write 2 Done Sent Done false
18722 5308 Acked Read 10 Done Sent Done false
18723 5309 Acked Write 4 Done Sent Done false
18724 5310 Acked Write 10 Done Sent Done false
18725 5311 Acked Write 4 Done Sent Done false
18726 5312 Acked Write 1 Done Sent Done false
18727 5313 Acked Read 6 Done Sent Done false
18728 5314 Acked Write 7 Done Sent Done false
18729 5315 Acked Flush 0 Done Sent Done false
18730 5316 Acked Read 8 Done Sent Done false
18731 5317 Acked Write 4 Done Sent Done false
18732 5318 Acked Flush 0 Done Sent Done false
18733 5319 Acked Write 3 Done Sent Done false
18734 5320 Acked Flush 0 Done Sent Done false
18735 5321 Acked Read 8 Done Sent Done false
18736 5322 Acked Read 9 Done Sent Done false
18737 5323 Acked Read 7 Done Sent Done false
18738 5324 Acked Write 9 Done Sent Done false
18739 5325 Acked Read 3 Done Sent Done false
18740 5326 Acked Flush 0 Done Sent Done false
18741 5327 Acked Write 5 Done Sent Done false
18742 5328 Acked Write 1 Done Sent Done false
18743 5329 Acked Write 2 Done Sent Done false
18744 5330 Acked Read 3 Done Sent Done false
18745 5331 Acked Write 5 Done Sent Done false
18746 5332 Acked Write 1 Done Sent Done false
18747 5333 Acked Read 1 Done Sent Done false
18748 5334 Acked Read 9 Done Sent Done false
18749 5335 Acked Read 10 Done Sent Done false
18750 5336 Acked Read 3 Done Sent Done false
18751 5337 Acked Write 5 Done Sent Done false
18752 5338 Acked Read 4 Done Sent Done false
18753 5339 Acked Write 7 Done Sent Done false
18754 5340 Acked Write 7 Done Sent Done false
18755 5341 Acked Flush 0 Done Sent Done false
18756 5342 Acked Write 3 Done Sent Done false
18757 5343 Acked Write 6 Done Sent Done false
18758 5344 Acked Read 4 Done Sent Done false
18759 5345 Acked Flush 0 Done Sent Done false
18760 5346 Acked Read 1 Done Sent Done false
18761 5347 Acked Read 5 Done Sent Done false
18762 5348 Acked Write 2 Done Sent Done false
18763 5349 Acked Write 2 Done Sent Done false
18764 5350 Acked Write 6 Done Sent Done false
18765 5351 Acked Read 8 Done Sent Done false
18766 5352 Acked Write 2 Done Sent Done false
18767 5353 Acked Read 5 Done Sent Done false
18768 5354 Acked Read 6 Done Sent Done false
18769 5355 Acked Read 7 Done Sent Done false
18770 5356 Acked Read 9 Done Sent Done false
18771 5357 Acked Flush 0 Done Sent Done false
18772 5358 Acked Read 3 Done Sent Done false
18773 5359 Acked Read 1 Done Sent Done false
18774 5360 Acked Read 9 Done Sent Done false
18775 5361 Acked Read 1 Done Sent Done false
18776 5362 Acked Flush 0 Done Sent Done false
18777 5363 Acked Write 5 Done Sent Done false
18778 5364 Acked Read 6 Done Sent Done false
18779 5365 Acked Write 5 Done Sent Done false
18780 5366 Acked Read 2 Done Sent Done false
18781 5367 Acked Read 1 Done Sent Done false
18782 5368 Acked Write 10 Done Sent Done false
18783 5369 Acked Read 9 Done Sent Done false
18784 5370 Acked Write 8 Done Sent Done false
18785 5371 Acked Flush 0 Done Sent Done false
18786 5372 Acked Read 7 Done Sent Done false
18787 5373 Acked Write 9 Done Sent Done false
18788 5374 Acked Write 3 Done Sent Done false
18789 5375 Acked Read 1 Done Sent Done false
18790 5376 Acked Write 8 Done Sent Done false
18791 5377 Acked Read 7 Done Sent Done false
18792 5378 Acked Write 3 Done Sent Done false
18793 5379 Acked Read 7 Done Sent Done false
18794 5380 Acked Write 10 Done Sent Done false
18795 5381 Acked Read 3 Done Sent Done false
18796 5382 Acked Read 8 Done Sent Done false
18797 5383 Acked Write 8 Done Sent Done false
18798 5384 Acked Read 8 Done Sent Done false
18799 5385 Acked Read 9 Done Sent Done false
18800 5386 Acked Flush 0 Done Sent Done false
18801 5387 Acked Read 10 Done Sent Done false
18802 5388 Acked Write 9 Done Sent Done false
18803 5389 Acked Write 8 Done Sent Done false
18804 5390 Acked Write 3 Done Sent Done false
18805 5391 Acked Write 8 Done Sent Done false
18806 5392 Acked Read 4 Done Sent Done false
18807 5393 Acked Write 8 Done Sent Done false
18808 5394 Acked Write 6 Done Sent Done false
18809 5395 Acked Read 5 Done Sent Done false
18810 5396 Acked Read 6 Done Sent Done false
18811 5397 Acked Write 10 Done Sent Done false
18812 5398 Acked Read 6 Done Sent Done false
18813 5399 Acked Read 8 Done Sent Done false
18814 5400 Acked Read 2 Done Sent Done false
18815 5401 Acked Read 9 Done Sent Done false
18816 5402 Acked Read 7 Done Sent Done false
18817 5403 Acked Read 7 Done Sent Done false
18818 5404 Acked Write 1 Done Sent Done false
18819 5405 Acked Read 9 Done Sent Done false
18820 5406 Acked Read 5 Done Sent Done false
18821 5407 Acked Read 4 Done Sent Done false
18822 5408 Acked Read 5 Done Sent Done false
18823 5409 Acked Write 10 Done Sent Done false
18824 5410 Acked Write 7 Done Sent Done false
18825 5411 Acked Read 6 Done Sent Done false
18826 5412 Acked Write 3 Done Sent Done false
18827 5413 Acked Write 5 Done Sent Done false
18828 5414 Acked Read 10 Done Sent Done false
18829 5415 Acked Write 8 Done Sent Done false
18830 5416 Acked Flush 0 Done Sent Done false
18831 5417 Acked Read 5 Done Sent Done false
18832 5418 Acked Read 9 Done Sent Done false
18833 5419 Acked Write 10 Done Sent Done false
18834 5420 Acked Read 1 Done Sent Done false
18835 5421 Acked Read 4 Done Sent Done false
18836 5422 Acked Read 6 Done Sent Done false
18837 5423 Acked Flush 0 Done Sent Done false
18838 5424 Acked Write 2 Done Sent Done false
18839 5425 Acked Flush 0 Done Sent Done false
18840 5426 Acked Read 2 Done Sent Done false
18841 5427 Acked Read 1 Done Sent Done false
18842 5428 Acked Read 10 Done Sent Done false
18843 5429 Acked Read 9 Done Sent Done false
18844 5430 Acked Read 9 Done Sent Done false
18845 5431 Acked Write 1 Done Sent Done false
18846 5432 Acked Write 10 Done Sent Done false
18847 5433 Acked Read 2 Done Sent Done false
18848 5434 Acked Read 10 Done Sent Done false
18849 5435 Acked Write 2 Done Sent Done false
18850 5436 Acked Read 3 Done Sent Done false
18851 5437 Acked Flush 0 Done Sent Done false
18852 5438 Acked Write 5 Done Sent Done false
18853 5439 Acked Read 5 Done Sent Done false
18854 5440 Acked Read 2 Done Sent Done false
18855 5441 Acked Write 3 Done Sent Done false
18856 5442 Acked Read 1 Done Sent Done false
18857 5443 Acked Read 7 Done Sent Done false
18858 5444 Acked Read 2 Done Sent Done false
18859 5445 Acked Flush 0 Done Sent Done false
18860 5446 Acked Read 1 Done Sent Done false
18861 5447 Acked Read 6 Done Sent Done false
18862 5448 Acked Read 4 Done Sent Done false
18863 5449 Acked Write 1 Done Sent Done false
18864 5450 Acked Flush 0 Done Sent Done false
18865 5451 Acked Read 6 Done Sent Done false
18866 5452 Acked Write 8 Done Sent Done false
18867 5453 Acked Read 3 Done Sent Done false
18868 5454 Acked Write 1 Done Sent Done false
18869 5455 Acked Write 2 Done Sent Done false
18870 5456 Acked Read 2 Done Sent Done false
18871 5457 Acked Flush 0 Done Sent Done false
18872 5458 Acked Read 6 Done Sent Done false
18873 5459 Acked Read 3 Done Sent Done false
18874 5460 Acked Read 7 Done Sent Done false
18875 5461 Acked Write 1 Done Sent Done false
18876 5462 Acked Read 8 Done Sent Done false
18877 5463 Acked Write 7 Done Sent Done false
18878 5464 Acked Read 10 Done Sent Done false
18879 5465 Acked Read 4 Done Sent Done false
18880 5466 Acked Read 8 Done Sent Done false
18881 5467 Acked Write 6 Done Sent Done false
18882 5468 Acked Read 8 Done Sent Done false
18883 5469 Acked Write 2 Done Sent Done false
18884 5470 Acked Flush 0 Done Sent Done false
18885 5471 Acked Read 7 Done Sent Done false
18886 5472 Acked Read 4 Done Sent Done false
18887 5473 Acked Read 3 Done Sent Done false
18888 5474 Acked Write 10 Done Sent Done false
18889 5475 Acked Read 4 Done Sent Done false
18890 5476 Acked Write 1 Done Sent Done false
18891 5477 Acked Flush 0 Done Sent Done false
18892 5478 Acked Write 8 Done Sent Done false
18893 5479 Acked Read 5 Done Sent Done false
18894 5480 Acked Flush 0 Done Sent Done false
18895 5481 Acked Flush 0 Done Sent Done false
18896 5482 Acked Read 3 Done Sent Done false
18897 5483 Acked Flush 0 Done Sent Done false
18898 5484 Acked Read 4 Done Sent Done false
18899 5485 Acked Read 10 Done Sent Done false
18900 5486 Acked Write 4 Done Sent Done false
18901 5487 Acked Write 8 Done Sent Done false
18902 5488 Acked Write 8 Done Sent Done false
18903 5489 Acked Read 5 Done Sent Done false
18904 5490 Acked Write 9 Done Sent Done false
18905 5491 Acked Read 10 Done Sent Done false
18906 5492 Acked Write 7 Done Sent Done false
18907 5493 Acked Read 4 Done Sent Done false
18908 5494 Acked Flush 0 Done Sent Done false
18909 5495 Acked Read 8 Done Sent Done false
18910 5496 Acked Read 3 Done Sent Done false
18911 5497 Acked Write 10 Done Sent Done false
18912 5498 Acked Read 3 Done Sent Done false
18913 5499 Acked Write 4 Done Sent Done false
18914 5500 Acked Read 9 Done Sent Done false
18915 5501 Acked Read 1 Done Sent Done false
18916 5502 Acked Write 10 Done Sent Done false
18917 5503 Acked Read 2 Done Sent Done false
18918 5504 Acked Read 6 Done Sent Done false
18919 5505 Acked Read 8 Done Sent Done false
18920 5506 Acked Read 3 Done Sent Done false
18921 5507 Acked Write 4 Done Sent Done false
18922 5508 Acked Read 7 Done Sent Done false
18923 5509 Acked Read 4 Done Sent Done false
18924 5510 Acked Read 3 Done Sent Done false
18925 5511 Acked Flush 0 Done Sent Done false
18926 5512 Acked Write 8 Done Sent Done false
18927 5513 Acked Read 10 Done Sent Done false
18928 5514 Acked Read 2 Done Sent Done false
18929 5515 Acked Read 4 Done Sent Done false
18930 5516 Acked Read 7 Done Sent Done false
18931 5517 Acked Read 9 Done Sent Done false
18932 5518 Acked Flush 0 Done Sent Done false
18933 5519 Acked Flush 0 Done Sent Done false
18934 5520 Acked Write 8 Done Sent Done false
18935 5521 Acked Write 1 Done Sent Done false
18936 5522 Acked Read 1 Done Sent Done false
18937 5523 Acked Read 10 Done Sent Done false
18938 5524 Acked Write 1 Done Sent Done false
18939 5525 Acked Read 3 Done Sent Done false
18940 5526 Acked Write 8 Done Sent Done false
18941 STATES DS:0 DS:1 DS:2 TOTAL
18942 Sent 0 301 0 301
18943 Done 301 0 301 602
18944 Skipped 0 0 0 0
18945 Error 0 0 0 0
18946 Last Flush: 5519 5225 5519
18947 Downstairs last five completed: 5225 5224 5223 5222 5221
18948 Upstairs last five completed: 5526 5525 5524 5523 5522
18949 CLIENT: Up:0 ds:301 act:2
18950 [8820] got start action
18951 [8820] I am going to restart
18952 Starting downstairs at port 8820
18953 Make output file at "/tmp/dsc/downstairs-8820.txt"
18954 [8820][1] reports Starting
18955 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1220)
18956 [8820][1] reports Running
18957 ----------------------------------------------------------------
18958 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
18959 States: Active Offline Active
18960 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
18961 5226 Acked Read 9 Done Sent Done false
18962 5227 Acked Write 1 Done Sent Done false
18963 5228 Acked Read 1 Done Sent Done false
18964 5229 Acked Read 2 Done Sent Done false
18965 5230 Acked Read 8 Done Sent Done false
18966 5231 Acked Read 4 Done Sent Done false
18967 5232 Acked Read 10 Done Sent Done false
18968 5233 Acked Write 9 Done Sent Done false
18969 5234 Acked Write 7 Done Sent Done false
18970 5235 Acked Read 10 Done Sent Done false
18971 5236 Acked Write 3 Done Sent Done false
18972 5237 Acked Write 7 Done Sent Done false
18973 5238 Acked Write 5 Done Sent Done false
18974 5239 Acked Flush 0 Done Sent Done false
18975 5240 Acked Write 7 Done Sent Done false
18976 5241 Acked Flush 0 Done Sent Done false
18977 5242 Acked Write 9 Done Sent Done false
18978 5243 Acked Read 9 Done Sent Done false
18979 5244 Acked Flush 0 Done Sent Done false
18980 5245 Acked Write 5 Done Sent Done false
18981 5246 Acked Read 1 Done Sent Done false
18982 5247 Acked Read 3 Done Sent Done false
18983 5248 Acked Write 10 Done Sent Done false
18984 5249 Acked Read 10 Done Sent Done false
18985 5250 Acked Read 2 Done Sent Done false
18986 5251 Acked Read 8 Done Sent Done false
18987 5252 Acked Read 1 Done Sent Done false
18988 5253 Acked Read 3 Done Sent Done false
18989 5254 Acked Read 3 Done Sent Done false
18990 5255 Acked Flush 0 Done Sent Done false
18991 5256 Acked Read 7 Done Sent Done false
18992 5257 Acked Write 10 Done Sent Done false
18993 5258 Acked Write 1 Done Sent Done false
18994 5259 Acked Read 6 Done Sent Done false
18995 5260 Acked Read 10 Done Sent Done false
18996 5261 Acked Read 2 Done Sent Done false
18997 5262 Acked Read 5 Done Sent Done false
18998 5263 Acked Write 10 Done Sent Done false
18999 5264 Acked Write 1 Done Sent Done false
19000 5265 Acked Write 1 Done Sent Done false
19001 5266 Acked Read 3 Done Sent Done false
19002 5267 Acked Write 9 Done Sent Done false
19003 5268 Acked Flush 0 Done Sent Done false
19004 5269 Acked Write 3 Done Sent Done false
19005 5270 Acked Read 1 Done Sent Done false
19006 5271 Acked Read 1 Done Sent Done false
19007 5272 Acked Flush 0 Done Sent Done false
19008 5273 Acked Write 4 Done Sent Done false
19009 5274 Acked Read 4 Done Sent Done false
19010 5275 Acked Write 5 Done Sent Done false
19011 5276 Acked Write 8 Done Sent Done false
19012 5277 Acked Write 2 Done Sent Done false
19013 5278 Acked Write 10 Done Sent Done false
19014 5279 Acked Read 1 Done Sent Done false
19015 5280 Acked Read 2 Done Sent Done false
19016 5281 Acked Read 3 Done Sent Done false
19017 5282 Acked Write 10 Done Sent Done false
19018 5283 Acked Read 8 Done Sent Done false
19019 5284 Acked Write 6 Done Sent Done false
19020 5285 Acked Read 9 Done Sent Done false
19021 5286 Acked Write 10 Done Sent Done false
19022 5287 Acked Write 5 Done Sent Done false
19023 5288 Acked Read 1 Done Sent Done false
19024 5289 Acked Write 4 Done Sent Done false
19025 5290 Acked Write 10 Done Sent Done false
19026 5291 Acked Flush 0 Done Sent Done false
19027 5292 Acked Write 10 Done Sent Done false
19028 5293 Acked Write 10 Done Sent Done false
19029 5294 Acked Read 4 Done Sent Done false
19030 5295 Acked Read 6 Done Sent Done false
19031 5296 Acked Read 5 Done Sent Done false
19032 5297 Acked Write 3 Done Sent Done false
19033 5298 Acked Write 2 Done Sent Done false
19034 5299 Acked Write 1 Done Sent Done false
19035 5300 Acked Flush 0 Done Sent Done false
19036 5301 Acked Write 7 Done Sent Done false
19037 5302 Acked Read 2 Done Sent Done false
19038 5303 Acked Read 5 Done Sent Done false
19039 5304 Acked Read 1 Done Sent Done false
19040 5305 Acked Read 7 Done Sent Done false
19041 5306 Acked Read 6 Done Sent Done false
19042 5307 Acked Write 2 Done Sent Done false
19043 5308 Acked Read 10 Done Sent Done false
19044 5309 Acked Write 4 Done Sent Done false
19045 5310 Acked Write 10 Done Sent Done false
19046 5311 Acked Write 4 Done Sent Done false
19047 5312 Acked Write 1 Done Sent Done false
19048 5313 Acked Read 6 Done Sent Done false
19049 5314 Acked Write 7 Done Sent Done false
19050 5315 Acked Flush 0 Done Sent Done false
19051 5316 Acked Read 8 Done Sent Done false
19052 5317 Acked Write 4 Done Sent Done false
19053 5318 Acked Flush 0 Done Sent Done false
19054 5319 Acked Write 3 Done Sent Done false
19055 5320 Acked Flush 0 Done Sent Done false
19056 5321 Acked Read 8 Done Sent Done false
19057 5322 Acked Read 9 Done Sent Done false
19058 5323 Acked Read 7 Done Sent Done false
19059 5324 Acked Write 9 Done Sent Done false
19060 5325 Acked Read 3 Done Sent Done false
19061 5326 Acked Flush 0 Done Sent Done false
19062 5327 Acked Write 5 Done Sent Done false
19063 5328 Acked Write 1 Done Sent Done false
19064 5329 Acked Write 2 Done Sent Done false
19065 5330 Acked Read 3 Done Sent Done false
19066 5331 Acked Write 5 Done Sent Done false
19067 5332 Acked Write 1 Done Sent Done false
19068 5333 Acked Read 1 Done Sent Done false
19069 5334 Acked Read 9 Done Sent Done false
19070 5335 Acked Read 10 Done Sent Done false
19071 5336 Acked Read 3 Done Sent Done false
19072 5337 Acked Write 5 Done Sent Done false
19073 5338 Acked Read 4 Done Sent Done false
19074 5339 Acked Write 7 Done Sent Done false
19075 5340 Acked Write 7 Done Sent Done false
19076 5341 Acked Flush 0 Done Sent Done false
19077 5342 Acked Write 3 Done Sent Done false
19078 5343 Acked Write 6 Done Sent Done false
19079 5344 Acked Read 4 Done Sent Done false
19080 5345 Acked Flush 0 Done Sent Done false
19081 5346 Acked Read 1 Done Sent Done false
19082 5347 Acked Read 5 Done Sent Done false
19083 5348 Acked Write 2 Done Sent Done false
19084 5349 Acked Write 2 Done Sent Done false
19085 5350 Acked Write 6 Done Sent Done false
19086 5351 Acked Read 8 Done Sent Done false
19087 5352 Acked Write 2 Done Sent Done false
19088 5353 Acked Read 5 Done Sent Done false
19089 5354 Acked Read 6 Done Sent Done false
19090 5355 Acked Read 7 Done Sent Done false
19091 5356 Acked Read 9 Done Sent Done false
19092 5357 Acked Flush 0 Done Sent Done false
19093 5358 Acked Read 3 Done Sent Done false
19094 5359 Acked Read 1 Done Sent Done false
19095 5360 Acked Read 9 Done Sent Done false
19096 5361 Acked Read 1 Done Sent Done false
19097 5362 Acked Flush 0 Done Sent Done false
19098 5363 Acked Write 5 Done Sent Done false
19099 5364 Acked Read 6 Done Sent Done false
19100 5365 Acked Write 5 Done Sent Done false
19101 5366 Acked Read 2 Done Sent Done false
19102 5367 Acked Read 1 Done Sent Done false
19103 5368 Acked Write 10 Done Sent Done false
19104 5369 Acked Read 9 Done Sent Done false
19105 5370 Acked Write 8 Done Sent Done false
19106 5371 Acked Flush 0 Done Sent Done false
19107 5372 Acked Read 7 Done Sent Done false
19108 5373 Acked Write 9 Done Sent Done false
19109 5374 Acked Write 3 Done Sent Done false
19110 5375 Acked Read 1 Done Sent Done false
19111 5376 Acked Write 8 Done Sent Done false
19112 5377 Acked Read 7 Done Sent Done false
19113 5378 Acked Write 3 Done Sent Done false
19114 5379 Acked Read 7 Done Sent Done false
19115 5380 Acked Write 10 Done Sent Done false
19116 5381 Acked Read 3 Done Sent Done false
19117 5382 Acked Read 8 Done Sent Done false
19118 5383 Acked Write 8 Done Sent Done false
19119 5384 Acked Read 8 Done Sent Done false
19120 5385 Acked Read 9 Done Sent Done false
19121 5386 Acked Flush 0 Done Sent Done false
19122 5387 Acked Read 10 Done Sent Done false
19123 5388 Acked Write 9 Done Sent Done false
19124 5389 Acked Write 8 Done Sent Done false
19125 5390 Acked Write 3 Done Sent Done false
19126 5391 Acked Write 8 Done Sent Done false
19127 5392 Acked Read 4 Done Sent Done false
19128 5393 Acked Write 8 Done Sent Done false
19129 5394 Acked Write 6 Done Sent Done false
19130 5395 Acked Read 5 Done Sent Done false
19131 5396 Acked Read 6 Done Sent Done false
19132 5397 Acked Write 10 Done Sent Done false
19133 5398 Acked Read 6 Done Sent Done false
19134 5399 Acked Read 8 Done Sent Done false
19135 5400 Acked Read 2 Done Sent Done false
19136 5401 Acked Read 9 Done Sent Done false
19137 5402 Acked Read 7 Done Sent Done false
19138 5403 Acked Read 7 Done Sent Done false
19139 5404 Acked Write 1 Done Sent Done false
19140 5405 Acked Read 9 Done Sent Done false
19141 5406 Acked Read 5 Done Sent Done false
19142 5407 Acked Read 4 Done Sent Done false
19143 5408 Acked Read 5 Done Sent Done false
19144 5409 Acked Write 10 Done Sent Done false
19145 5410 Acked Write 7 Done Sent Done false
19146 5411 Acked Read 6 Done Sent Done false
19147 5412 Acked Write 3 Done Sent Done false
19148 5413 Acked Write 5 Done Sent Done false
19149 5414 Acked Read 10 Done Sent Done false
19150 5415 Acked Write 8 Done Sent Done false
19151 5416 Acked Flush 0 Done Sent Done false
19152 5417 Acked Read 5 Done Sent Done false
19153 5418 Acked Read 9 Done Sent Done false
19154 5419 Acked Write 10 Done Sent Done false
19155 5420 Acked Read 1 Done Sent Done false
19156 5421 Acked Read 4 Done Sent Done false
19157 5422 Acked Read 6 Done Sent Done false
19158 5423 Acked Flush 0 Done Sent Done false
19159 5424 Acked Write 2 Done Sent Done false
19160 5425 Acked Flush 0 Done Sent Done false
19161 5426 Acked Read 2 Done Sent Done false
19162 5427 Acked Read 1 Done Sent Done false
19163 5428 Acked Read 10 Done Sent Done false
19164 5429 Acked Read 9 Done Sent Done false
19165 5430 Acked Read 9 Done Sent Done false
19166 5431 Acked Write 1 Done Sent Done false
19167 5432 Acked Write 10 Done Sent Done false
19168 5433 Acked Read 2 Done Sent Done false
19169 5434 Acked Read 10 Done Sent Done false
19170 5435 Acked Write 2 Done Sent Done false
19171 5436 Acked Read 3 Done Sent Done false
19172 5437 Acked Flush 0 Done Sent Done false
19173 5438 Acked Write 5 Done Sent Done false
19174 5439 Acked Read 5 Done Sent Done false
19175 5440 Acked Read 2 Done Sent Done false
19176 5441 Acked Write 3 Done Sent Done false
19177 5442 Acked Read 1 Done Sent Done false
19178 5443 Acked Read 7 Done Sent Done false
19179 5444 Acked Read 2 Done Sent Done false
19180 5445 Acked Flush 0 Done Sent Done false
19181 5446 Acked Read 1 Done Sent Done false
19182 5447 Acked Read 6 Done Sent Done false
19183 5448 Acked Read 4 Done Sent Done false
19184 5449 Acked Write 1 Done Sent Done false
19185 5450 Acked Flush 0 Done Sent Done false
19186 5451 Acked Read 6 Done Sent Done false
19187 5452 Acked Write 8 Done Sent Done false
19188 5453 Acked Read 3 Done Sent Done false
19189 5454 Acked Write 1 Done Sent Done false
19190 5455 Acked Write 2 Done Sent Done false
19191 5456 Acked Read 2 Done Sent Done false
19192 5457 Acked Flush 0 Done Sent Done false
19193 5458 Acked Read 6 Done Sent Done false
19194 5459 Acked Read 3 Done Sent Done false
19195 5460 Acked Read 7 Done Sent Done false
19196 5461 Acked Write 1 Done Sent Done false
19197 5462 Acked Read 8 Done Sent Done false
19198 5463 Acked Write 7 Done Sent Done false
19199 5464 Acked Read 10 Done Sent Done false
19200 5465 Acked Read 4 Done Sent Done false
19201 5466 Acked Read 8 Done Sent Done false
19202 5467 Acked Write 6 Done Sent Done false
19203 5468 Acked Read 8 Done Sent Done false
19204 5469 Acked Write 2 Done Sent Done false
19205 5470 Acked Flush 0 Done Sent Done false
19206 5471 Acked Read 7 Done Sent Done false
19207 5472 Acked Read 4 Done Sent Done false
19208 5473 Acked Read 3 Done Sent Done false
19209 5474 Acked Write 10 Done Sent Done false
19210 5475 Acked Read 4 Done Sent Done false
19211 5476 Acked Write 1 Done Sent Done false
19212 5477 Acked Flush 0 Done Sent Done false
19213 5478 Acked Write 8 Done Sent Done false
19214 5479 Acked Read 5 Done Sent Done false
19215 5480 Acked Flush 0 Done Sent Done false
19216 5481 Acked Flush 0 Done Sent Done false
19217 5482 Acked Read 3 Done Sent Done false
19218 5483 Acked Flush 0 Done Sent Done false
19219 5484 Acked Read 4 Done Sent Done false
19220 5485 Acked Read 10 Done Sent Done false
19221 5486 Acked Write 4 Done Sent Done false
19222 5487 Acked Write 8 Done Sent Done false
19223 5488 Acked Write 8 Done Sent Done false
19224 5489 Acked Read 5 Done Sent Done false
19225 5490 Acked Write 9 Done Sent Done false
19226 5491 Acked Read 10 Done Sent Done false
19227 5492 Acked Write 7 Done Sent Done false
19228 5493 Acked Read 4 Done Sent Done false
19229 5494 Acked Flush 0 Done Sent Done false
19230 5495 Acked Read 8 Done Sent Done false
19231 5496 Acked Read 3 Done Sent Done false
19232 5497 Acked Write 10 Done Sent Done false
19233 5498 Acked Read 3 Done Sent Done false
19234 5499 Acked Write 4 Done Sent Done false
19235 5500 Acked Read 9 Done Sent Done false
19236 5501 Acked Read 1 Done Sent Done false
19237 5502 Acked Write 10 Done Sent Done false
19238 5503 Acked Read 2 Done Sent Done false
19239 5504 Acked Read 6 Done Sent Done false
19240 5505 Acked Read 8 Done Sent Done false
19241 5506 Acked Read 3 Done Sent Done false
19242 5507 Acked Write 4 Done Sent Done false
19243 5508 Acked Read 7 Done Sent Done false
19244 5509 Acked Read 4 Done Sent Done false
19245 5510 Acked Read 3 Done Sent Done false
19246 5511 Acked Flush 0 Done Sent Done false
19247 5512 Acked Write 8 Done Sent Done false
19248 5513 Acked Read 10 Done Sent Done false
19249 5514 Acked Read 2 Done Sent Done false
19250 5515 Acked Read 4 Done Sent Done false
19251 5516 Acked Read 7 Done Sent Done false
19252 5517 Acked Read 9 Done Sent Done false
19253 5518 Acked Flush 0 Done Sent Done false
19254 5519 Acked Flush 0 Done Sent Done false
19255 5520 Acked Write 8 Done Sent Done false
19256 5521 Acked Write 1 Done Sent Done false
19257 5522 Acked Read 1 Done Sent Done false
19258 5523 Acked Read 10 Done Sent Done false
19259 5524 Acked Write 1 Done Sent Done false
19260 5525 Acked Read 3 Done Sent Done false
19261 5526 Acked Write 8 Done Sent Done false
19262 5527 Acked Flush 0 Done Sent Done false
19263 STATES DS:0 DS:1 DS:2 TOTAL
19264 Sent 0 302 0 302
19265 Done 302 0 302 604
19266 Skipped 0 0 0 0
19267 Error 0 0 0 0
19268 Last Flush: 5527 5225 5527
19269 Downstairs last five completed: 5225 5224 5223 5222 5221
19270 Upstairs last five completed: 5527 5526 5525 5524 5523
19271 CLIENT: Up:0 ds:302 act:2
192722024-11-15T02:01:29.336ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
192732024-11-15T02:01:29.336ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:44401) = io task client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
192742024-11-15T02:01:29.337ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
192752024-11-15T02:01:29.338ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
192762024-11-15T02:01:29.338ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
192772024-11-15T02:01:29.338ZINFOcrucible: send last flush ID to this DS: 5225 = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
192782024-11-15T02:01:29.338ZINFOcrucible: Replied this last flush ID: 5225 = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
192792024-11-15T02:01:29.338ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
192802024-11-15T02:01:29.338ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
192812024-11-15T02:01:29.338ZINFOcrucible: [1] client re-new 302 jobs since flush 5225 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
192822024-11-15T02:01:29.338ZINFOcrucible: [1] Replayed 302 jobs since flush: 5225 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
19283 ----------------------------------------------------------------
19284 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
19285 Downstairs last five completed: 5527 5526 5525 5524 5523
19286 Upstairs last five completed: 5527 5526 5525 5524 5523
19287 CLIENT: Up:0 ds:0 act:3
19288 Replay: All jobs finished
19289 Main task has work to do, go find it
19290 got dsc Stop(2)
19291 stop 2
19292 Nov 15 02:01:31.582 INFO request completed, latency_us: 350, response_code:[8830] Got stop action so:false kr:false
19293 204, uri: /stop/cid/2, method: POST, req_id: 6130c86c-87d8-41ab-8e34-98b73dc324d6, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
19294 Nov 15 02:01:31.583 INFO request completed, latency_us: 152, {"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2024-11-15T02:01:31.58346222Z","hostname":"w-01JCPRHT5F2MPA44KVJHM3S2PB","pid":1205,"":"io task","client":"2","":"downstairs","session_id":"971dcb2e-6891-4ecf-b367-1e04e52065ee"}
19295 response_code: 200, uri: /state/cid/2, method: GET, req_id: 128d2371-9093-4048-bec0-a1716abfb683, remote_addr: 127.0.0.1:40008, local_addr{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2024-11-15T02:01:31.583538815Z","hostname":"w-01JCPRHT5F2MPA44KVJHM3S2PB","pid":1205,"":"io task","client":"2","":"downstairs","session_id":"971dcb2e-6891-4ecf-b367-1e04e52065ee"}
19296 : 127.0.0.1:9998
192972024-11-15T02:01:31.583ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
192982024-11-15T02:01:31.583ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
192992024-11-15T02:01:31.583ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
193002024-11-15T02:01:31.583ZINFOcrucible: client task is exiting = io task client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
19301 [8830] Exited with: ExitStatus(unix_wait_status(9))
19302 [8830][2] reports Exit
19303 Nov 15 02:01:35.585 INFO request completed, latency_us: 269, response_code: 200, uri: /state/cid/2, method: GET, req_id: 78449216-644c-4b02-abc4-0da9fd540843, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
19304 001/300 Read block 394 len 4096
19305 002/300 Read block 2624 len 28672
19306 003/300 Write block 4601 len 36864 data: 2 2 2 3 3 4 4 5 4
19307 004/300 Write block 2232 len 12288 data: 7 7 6
19308 005/300 Write block 180 len 12288 data: 3 3 3
19309 006/300 Write block 4173 len 12288 data: 4 5 6
19310 007/300 Write block 2448 len 36864 data: 6 6 6 6 5 7 7 7 6
19311 008/300 Read block 3537 len 24576
19312 009/300 Read block 2644 len 12288
19313 010/300 Write block 623 len 40960 data: 8 8 6 4 4 3 2 3 4 4
19314 011/300 Write block 3439 len 40960 data: 3 4 4 4 4 4 4 4 3 3
19315 012/300 Read block 4094 len 40960
19316 013/300 Read block 3369 len 12288
19317 014/300 Read block 4352 len 40960
19318 015/300 Flush
19319 016/300 Read block 2229 len 20480
19320 017/300 Flush
19321 018/300 Read block 3681 len 40960
19322 019/300 Write block 1130 len 28672 data: 4 4 3 2 2 2 2
19323 020/300 Read block 315 len 16384
19324 021/300 Read block 1788 len 4096
19325 022/300 Write block 4931 len 8192 data: 2 2
19326 023/300 Read block 914 len 12288
19327 024/300 Write block 877 len 36864 data: 3 3 3 3 3 3 3 3 2
19328 025/300 Write block 1210 len 4096 data: 3
19329 026/300 Read block 3926 len 8192
19330 027/300 Write block 2401 len 8192 data: 3 3
19331 028/300 Read block 3119 len 24576
19332 029/300 Write block 1226 len 12288 data: 3 3 3
19333 030/300 Write block 4073 len 20480 data: 3 3 3 3 3
19334 031/300 Read block 2736 len 32768
19335 032/300 Write block 4330 len 24576 data: 5 6 6 5 4 4
19336 033/300 Read block 1325 len 20480
19337 034/300 Write block 4045 len 28672 data: 5 5 4 2 2 2 2
19338 035/300 Write block 4089 len 32768 data: 2 2 4 4 4 4 4 4
19339 036/300 Read block 4835 len 28672
19340 037/300 Read block 4032 len 16384
19341 038/300 Flush
19342 039/300 Read block 1715 len 4096
19343 040/300 Read block 2110 len 20480
19344 041/300 Read block 2381 len 32768
19345 042/300 Read block 4850 len 16384
19346 043/300 Write block 3920 len 20480 data: 5 5 5 7 7
19347 044/300 Write block 3046 len 24576 data: 4 3 3 4 4 4
19348 045/300 Read block 901 len 4096
19349 046/300 Read block 618 len 36864
19350 047/300 Read block 2410 len 4096
19351 048/300 Read block 4309 len 4096
19352 049/300 Flush
19353 050/300 Read block 1312 len 24576
19354 051/300 Write block 790 len 36864 data: 6 5 4 5 6 6 5 5 5
19355 052/300 Flush
19356 053/300 Read block 2862 len 28672
19357 054/300 Flush
19358 055/300 Read block 1222 len 20480
19359 056/300 Read block 3761 len 36864
19360 057/300 Write block 3373 len 36864 data: 5 5 4 5 4 3 5 5 5
19361 058/300 Read block 350 len 32768
19362 059/300 Flush
19363 060/300 Read block 4620 len 40960
19364 061/300 Flush
19365 062/300 Read block 4315 len 8192
19366 063/300 Read block 3873 len 16384
19367 064/300 Read block 1001 len 40960
19368 065/300 Write block 4652 len 36864 data: 4 3 3 3 2 3 4 4 5
19369 066/300 Read block 1987 len 32768
19370 067/300 Read block 2184 len 12288
19371 068/300 Read block 3239 len 20480
19372 069/300 Read block 10 len 8192
19373 070/300 Write block 4179 len 28672 data: 8 8 7 7 7 8 8
19374 071/300 Write block 4326 len 4096 data: 5
19375 072/300 Flush
19376 073/300 Read block 4954 len 4096
19377 074/300 Read block 151 len 40960
19378 075/300 Write block 104 len 32768 data: 4 3 2 4 4 4 3 2
19379 076/300 Read block 1218 len 12288
19380 077/300 Write block 3807 len 12288 data: 4 4 4
19381 078/300 Write block 322 len 24576 data: 5 3 3 5 6 8
19382 079/300 Read block 119 len 24576
19383 080/300 Write block 2848 len 28672 data: 5 5 3 3 3 3 3
19384 081/300 Write block 4456 len 28672 data: 3 3 3 3 3 3 3
19385 082/300 Read block 1765 len 36864
19386 083/300 Read block 4147 len 32768
19387 084/300 Read block 3802 len 16384
19388 085/300 Read block 1946 len 4096
19389 086/300 Read block 3031 len 20480
19390 087/300 Write block 4612 len 12288 data: 5 4 5
19391 088/300 Write block 661 len 28672 data: 5 4 5 3 3 3 5
19392 089/300 Write block 306 len 4096 data: 6
19393 090/300 Write block 3526 len 32768 data: 3 3 4 6 7 8 7 7
19394 091/300 Flush
19395 092/300 Read block 2021 len 36864
19396 093/300 Write block 461 len 24576 data: 5 5 5 4 4 4
19397 094/300 Write block 3284 len 12288 data: 4 3 3
19398 095/300 Write block 2782 len 8192 data: 5 4
19399 096/300 Flush
19400 097/300 Read block 4532 len 8192
19401 098/300 Read block 280 len 8192
19402 099/300 Read block 416 len 32768
19403 100/300 Write block 4563 len 4096 data: 3
19404 101/300 Write block 1528 len 32768 data: 4 3 4 4 4 4 2 4
19405 102/300 Read block 3083 len 4096
19406 103/300 Write block 4743 len 36864 data: 3 3 3 3 3 3 3 3 4
19407 104/300 Read block 1899 len 20480
19408 105/300 Read block 3920 len 12288
19409 106/300 Write block 1038 len 16384 data: 2 2 3 3
19410 107/300 Read block 4171 len 40960
19411 108/300 Flush
19412 109/300 Write block 2789 len 12288 data: 2 3 3
19413 110/300 Read block 2230 len 32768
19414 111/300 Read block 4091 len 16384
19415 112/300 Write block 1052 len 12288 data: 4 4 5
19416 113/300 Read block 84 len 12288
19417 114/300 Read block 239 len 16384
19418 115/300 Write block 2209 len 12288 data: 4 3 3
19419 116/300 Write block 4811 len 32768 data: 3 3 3 3 3 3 5 5
19420 117/300 Write block 4829 len 36864 data: 5 5 4 4 4 5 6 5 5
19421 118/300 Read block 259 len 32768
19422 119/300 Read block 295 len 4096
19423 120/300 Read block 4364 len 40960
19424 121/300 Read block 4191 len 36864
19425 122/300 Flush
19426 123/300 Read block 3365 len 24576
19427 124/300 Read block 1921 len 4096
19428 125/300 Read block 3715 len 12288
19429 126/300 Read block 4749 len 28672
19430 127/300 Write block 2808 len 36864 data: 2 2 3 3 3 3 4 5 5
19431 128/300 Read block 4839 len 4096
19432 129/300 Write block 3148 len 20480 data: 3 3 4 4 5
19433 130/300 Read block 450 len 12288
19434 131/300 Write block 476 len 4096 data: 5
19435 132/300 Write block 3763 len 24576 data: 5 5 3 3 2 2
19436 133/300 Read block 4676 len 4096
19437 134/300 Read block 4473 len 4096
19438 135/300 Read block 789 len 4096
19439 136/300 Write block 4665 len 36864 data: 10 7 7 5 6 6 6 4 4
19440 137/300 Read block 4433 len 36864
19441 138/300 Read block 4658 len 32768
19442 139/300 Flush
19443 140/300 Flush
19444 141/300 Read block 3025 len 32768
19445 142/300 Read block 2624 len 24576
19446 143/300 Write block 2086 len 16384 data: 2 2 2 2
19447 144/300 Write block 4890 len 36864 data: 6 5 5 7 7 6 5 5 4
19448 145/300 Read block 2431 len 28672
19449 146/300 Read block 1250 len 36864
19450 147/300 Write block 2402 len 32768 data: 4 4 4 4 3 2 2 2
19451 148/300 Read block 1816 len 20480
19452 149/300 Read block 3451 len 8192
19453 150/300 Write block 1391 len 8192 data: 2 2
19454 151/300 Read block 761 len 40960
19455 152/300 Write block 1684 len 20480 data: 5 5 6 5 4
19456 153/300 Write block 108 len 24576 data: 5 5 4 3 2 2
19457 154/300 Write block 3103 len 24576 data: 3 2 2 2 3 3
19458 155/300 Read block 4173 len 12288
19459 156/300 Read block 3041 len 16384
19460 157/300 Write block 808 len 28672 data: 2 2 2 2 2 3 3
19461 158/300 Write block 4640 len 32768 data: 2 2 2 4 3 3 5 4
19462 159/300 Read block 2974 len 8192
19463 160/300 Write block 4197 len 32768 data: 8 6 6 5 4 6 6 6
19464 161/300 Write block 1269 len 20480 data: 4 3 4 4 5
19465 162/300 Write block 4826 len 16384 data: 4 4 4 6
19466 163/300 Write block 1841 len 20480 data: 5 5 4 5 6
19467 164/300 Write block 3575 len 8192 data: 4 4
19468 165/300 Flush
19469 166/300 Read block 2070 len 24576
19470 167/300 Read block 3541 len 12288
19471 168/300 Read block 2718 len 16384
19472 169/300 Write block 1642 len 24576 data: 6 6 5 4 3 4
19473 170/300 Read block 4991 len 24576
19474 171/300 Read block 4589 len 4096
19475 172/300 Flush
19476 173/300 Read block 842 len 32768
19477 174/300 Read block 218 len 32768
19478 175/300 Flush
19479 176/300 Flush
19480 177/300 Read block 4003 len 32768
19481 178/300 Write block 736 len 40960 data: 4 4 4 4 3 3 3 2 2 2
19482 179/300 Write block 3116 len 8192 data: 2 3
19483 180/300 Flush
19484 181/300 Read block 3126 len 8192
19485 182/300 Write block 3252 len 8192 data: 6 6
19486 183/300 Write block 160 len 8192 data: 5 5
19487 184/300 Read block 4885 len 12288
19488 185/300 Read block 4613 len 32768
19489 186/300 Flush
19490 187/300 Write block 2785 len 28672 data: 4 3 3 2 3 4 4
19491 188/300 Write block 734 len 4096 data: 3
19492 189/300 Read block 1581 len 28672
19493 190/300 Flush
19494 191/300 Write block 4149 len 40960 data: 4 4 4 3 3 3 4 4 4 5
19495 192/300 Read block 667 len 16384
19496 193/300 Read block 89 len 24576
19497 194/300 Read block 1102 len 24576
19498 195/300 Read block 2712 len 8192
19499 196/300 Write block 4353 len 40960 data: 3 3 4 4 3 3 3 3 3 2
19500 197/300 Write block 2305 len 40960 data: 4 4 6 7 7 5 4 4 3 3
19501 198/300 Read block 4311 len 36864
19502 199/300 Write block 2087 len 28672 data: 3 3 3 2 2 2 2
19503 200/300 Read block 4178 len 36864
19504 201/300 Read block 3875 len 28672
19505 202/300 Read block 621 len 12288
19506 203/300 Read block 3107 len 40960
19507 204/300 Read block 273 len 12288
19508 205/300 Read block 3484 len 4096
19509 206/300 Write block 1792 len 12288 data: 3 3 3
19510 207/300 Write block 1691 len 12288 data: 5 5 5
19511 208/300 Read block 3911 len 32768
19512 209/300 Read block 599 len 8192
19513 210/300 Write block 3232 len 4096 data: 2
19514 211/300 Write block 3655 len 32768 data: 6 4 3 4 4 4 4 4
19515 212/300 Write block 232 len 36864 data: 7 5 5 5 6 4 3 3 3
19516 213/300 Write block 4407 len 24576 data: 3 3 3 4 2 2
19517 214/300 Flush
19518 215/300 Write block 4079 len 36864 data: 3 3 4 2 2 2 2 2 2
19519 216/300 Flush
19520 217/300 Read block 289 len 4096
19521 218/300 Write block 4075 len 24576 data: 4 4 4 3 4 4
19522 219/300 Read block 3429 len 4096
19523 220/300 Write block 4240 len 16384 data: 3 4 5 4
19524 221/300 Write block 2397 len 36864 data: 2 2 2 3 4 5 5 5 5
19525 222/300 Read block 4972 len 12288
19526 223/300 Read block 2051 len 8192
19527 224/300 Read block 696 len 40960
19528 225/300 Read block 380 len 28672
19529 226/300 Write block 3236 len 12288 data: 2 2 2
19530 227/300 Write block 260 len 36864 data: 5 5 4 4 4 4 3 3 3
19531 228/300 Write block 1295 len 20480 data: 4 4 5 5 6
19532 229/300 Read block 2303 len 20480
19533 230/300 Write block 617 len 28672 data: 2 3 4 5 7 7 9
19534 231/300 Write block 3548 len 40960 data: 4 4 4 5 7 6 4 4 4 4
19535 232/300 Read block 3226 len 8192
19536 233/300 Write block 3489 len 16384 data: 4 4 3 2
19537 234/300 Write block 3994 len 8192 data: 3 2
19538 235/300 Write block 2541 len 16384 data: 2 4 4 5
19539 236/300 Flush
19540 237/300 Read block 974 len 24576
19541 238/300 Read block 4787 len 28672
19542 239/300 Read block 4101 len 12288
19543 240/300 Write block 3696 len 28672 data: 5 4 5 5 4 4 4
19544 241/300 Read block 4961 len 12288
19545 242/300 Write block 4749 len 8192 data: 4 4
19546 243/300 Write block 1371 len 4096 data: 5
19547 244/300 Write block 1175 len 8192 data: 5 5
19548 245/300 Flush
19549 246/300 Write block 2780 len 8192 data: 5 5
19550 247/300 Write block 1375 len 32768 data: 6 5 5 5 6 6 6 4
19551 248/300 Flush
19552 249/300 Read block 1650 len 20480
19553 250/300 Write block 1828 len 28672 data: 4 4 4 5 4 3 3
19554 251/300 Read block 358 len 12288
19555 252/300 Write block 4909 len 20480 data: 2 2 2 2 3
19556 253/300 Write block 3003 len 8192 data: 3 4
19557 254/300 Read block 1294 len 20480
19558 255/300 Flush
19559 256/300 Read block 929 len 20480
19560 257/300 Write block 2141 len 40960 data: 4 3 4 4 4 3 3 5 5 6
19561 258/300 Flush
19562 259/300 Write block 1838 len 8192 data: 5 4
19563 260/300 Read block 2450 len 8192
19564 261/300 Read block 4119 len 12288
19565 262/300 Read block 917 len 32768
19566 263/300 Write block 3865 len 12288 data: 6 6 7
19567 264/300 Write block 4065 len 24576 data: 5 4 4 4 4 4
19568 265/300 Read block 1712 len 36864
19569 266/300 Read block 3113 len 12288
19570 267/300 Read block 3932 len 24576
19571 268/300 Write block 246 len 32768 data: 3 3 4 4 4 3 3 3
19572 269/300 Read block 2443 len 32768
19573 270/300 Write block 399 len 16384 data: 3 3 2 2
19574 271/300 Read block 4962 len 16384
19575 272/300 Read block 2114 len 28672
19576 273/300 Write block 3226 len 32768 data: 2 2 2 2 2 2 3 2
19577 274/300 Read block 1270 len 20480
19578 275/300 Write block 3979 len 12288 data: 4 4 3
19579 276/300 Read block 175 len 32768
19580 277/300 Write block 4270 len 20480 data: 4 4 5 4 3
19581 278/300 Read block 3578 len 4096
19582 279/300 Read block 1281 len 8192
19583 280/300 Read block 1579 len 36864
19584 281/300 Read block 3409 len 4096
19585 282/300 Read block 311 len 12288
19586 283/300 Read block 2597 len 28672
19587 284/300 Read block 4293 len 32768
19588 285/300 Read block 4626 len 24576
19589 286/300 Read block 2667 len 32768
19590 287/300 Flush
19591 288/300 Read block 958 len 40960
19592 289/300 Read block 3502 len 12288
19593 290/300 Read block 4854 len 8192
19594 291/300 Write block 1666 len 40960 data: 9 8 7 7 6 4 3 3 2 3
19595 292/300 Read block 523 len 24576
19596 293/300 Write block 85 len 12288 data: 6 5 6
19597 294/300 Read block 2199 len 32768
19598 295/300 Write block 2365 len 4096 data: 6
19599 296/300 Write block 3648 len 40960 data: 6 7 7 7 6 6 6 7 5 4
19600 297/300 Read block 2982 len 20480
19601 298/300 Write block 3536 len 20480 data: 5 4 5 4 3
19602 299/300 Read block 2173 len 16384
19603 300/300 Read block 996 len 24576
19604 301/300 Read block 124 len 32768
19605 Main task has work to do, go find it
19606 got dsc Start(2)
19607 start 2
19608 Nov 15 02:01:35.821 INFO request completed, latency_us: 237, response_code: 204, uri: /start/cid/2, method: POST, req_id: 6ad15fd6-4c0d-45f4-b732-e973b6d773ac, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
19609 Replay: started 2, returned:Ok(())
19610 ----------------------------------------------------------------
19611 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
19612 States: Active Active Offline
19613 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
19614 5528 Acked Read 1 Done Done Sent false
19615 5529 Acked Read 7 Done Done Sent false
19616 5530 Acked Write 9 Done Done Sent false
19617 5531 Acked Write 3 Done Done Sent false
19618 5532 Acked Write 3 Done Done Sent false
19619 5533 Acked Write 3 Done Done Sent false
19620 5534 Acked Write 9 Done Done Sent false
19621 5535 Acked Read 6 Done Done Sent false
19622 5536 Acked Read 3 Done Done Sent false
19623 5537 Acked Write 10 Done Done Sent false
19624 5538 Acked Write 10 Done Done Sent false
19625 5539 Acked Read 10 Done Done Sent false
19626 5540 Acked Read 3 Done Done Sent false
19627 5541 Acked Read 10 Done Done Sent false
19628 5542 Acked Flush 0 Done Done Sent false
19629 5543 Acked Read 5 Done Done Sent false
19630 5544 Acked Flush 0 Done Done Sent false
19631 5545 Acked Read 10 Done Done Sent false
19632 5546 Acked Write 7 Done Done Sent false
19633 5547 Acked Read 4 Done Done Sent false
19634 5548 Acked Read 1 Done Done Sent false
19635 5549 Acked Write 2 Done Done Sent false
19636 5550 Acked Read 3 Done Done Sent false
19637 5551 Acked Write 9 Done Done Sent false
19638 5552 Acked Write 1 Done Done Sent false
19639 5553 Acked Read 2 Done Done Sent false
19640 5554 Acked Write 2 Done Done Sent false
19641 5555 Acked Read 6 Done Done Sent false
19642 5556 Acked Write 3 Done Done Sent false
19643 5557 Acked Write 5 Done Done Sent false
19644 5558 Acked Read 8 Done Done Sent false
19645 5559 Acked Write 6 Done Done Sent false
19646 5560 Acked Read 5 Done Done Sent false
19647 5561 Acked Write 7 Done Done Sent false
19648 5562 Acked Write 8 Done Done Sent false
19649 5563 Acked Read 7 Done Done Sent false
19650 5564 Acked Read 4 Done Done Sent false
19651 5565 Acked Flush 0 Done Done Sent false
19652 5566 Acked Read 1 Done Done Sent false
19653 5567 Acked Read 5 Done Done Sent false
19654 5568 Acked Read 8 Done Done Sent false
19655 5569 Acked Read 4 Done Done Sent false
19656 5570 Acked Write 5 Done Done Sent false
19657 5571 Acked Write 6 Done Done Sent false
19658 5572 Acked Read 1 Done Done Sent false
19659 5573 Acked Read 9 Done Done Sent false
19660 5574 Acked Read 1 Done Done Sent false
19661 5575 Acked Read 1 Done Done Sent false
19662 5576 Acked Flush 0 Done Done Sent false
19663 5577 Acked Read 6 Done Done Sent false
19664 5578 Acked Write 9 Done Done Sent false
19665 5579 Acked Flush 0 Done Done Sent false
19666 5580 Acked Read 7 Done Done Sent false
19667 5581 Acked Flush 0 Done Done Sent false
19668 5582 Acked Read 5 Done Done Sent false
19669 5583 Acked Read 9 Done Done Sent false
19670 5584 Acked Write 9 Done Done Sent false
19671 5585 Acked Read 8 Done Done Sent false
19672 5586 Acked Flush 0 Done Done Sent false
19673 5587 Acked Read 10 Done Done Sent false
19674 5588 Acked Flush 0 Done Done Sent false
19675 5589 Acked Read 2 Done Done Sent false
19676 5590 Acked Read 4 Done Done Sent false
19677 5591 Acked Read 10 Done Done Sent false
19678 5592 Acked Write 9 Done Done Sent false
19679 5593 Acked Read 8 Done Done Sent false
19680 5594 Acked Read 3 Done Done Sent false
19681 5595 Acked Read 5 Done Done Sent false
19682 5596 Acked Read 2 Done Done Sent false
19683 5597 Acked Write 7 Done Done Sent false
19684 5598 Acked Write 1 Done Done Sent false
19685 5599 Acked Flush 0 Done Done Sent false
19686 5600 Acked Read 1 Done Done Sent false
19687 5601 Acked Read 10 Done Done Sent false
19688 5602 Acked Write 8 Done Done Sent false
19689 5603 Acked Read 3 Done Done Sent false
19690 5604 Acked Write 3 Done Done Sent false
19691 5605 Acked Write 6 Done Done Sent false
19692 5606 Acked Read 6 Done Done Sent false
19693 5607 Acked Write 7 Done Done Sent false
19694 5608 Acked Write 7 Done Done Sent false
19695 5609 Acked Read 9 Done Done Sent false
19696 5610 Acked Read 8 Done Done Sent false
19697 5611 Acked Read 4 Done Done Sent false
19698 5612 Acked Read 1 Done Done Sent false
19699 5613 Acked Read 5 Done Done Sent false
19700 5614 Acked Write 3 Done Done Sent false
19701 5615 Acked Write 7 Done Done Sent false
19702 5616 Acked Write 1 Done Done Sent false
19703 5617 Acked Write 8 Done Done Sent false
19704 5618 Acked Flush 0 Done Done Sent false
19705 5619 Acked Read 9 Done Done Sent false
19706 5620 Acked Write 6 Done Done Sent false
19707 5621 Acked Write 3 Done Done Sent false
19708 5622 Acked Write 2 Done Done Sent false
19709 5623 Acked Flush 0 Done Done Sent false
19710 5624 Acked Read 2 Done Done Sent false
19711 5625 Acked Read 2 Done Done Sent false
19712 5626 Acked Read 8 Done Done Sent false
19713 5627 Acked Write 1 Done Done Sent false
19714 5628 Acked Write 8 Done Done Sent false
19715 5629 Acked Read 1 Done Done Sent false
19716 5630 Acked Write 9 Done Done Sent false
19717 5631 Acked Read 5 Done Done Sent false
19718 5632 Acked Read 3 Done Done Sent false
19719 5633 Acked Write 4 Done Done Sent false
19720 5634 Acked Read 10 Done Done Sent false
19721 5635 Acked Flush 0 Done Done Sent false
19722 5636 Acked Write 3 Done Done Sent false
19723 5637 Acked Read 8 Done Done Sent false
19724 5638 Acked Read 4 Done Done Sent false
19725 5639 Acked Write 3 Done Done Sent false
19726 5640 Acked Read 3 Done Done Sent false
19727 5641 Acked Read 4 Done Done Sent false
19728 5642 Acked Write 3 Done Done Sent false
19729 5643 Acked Write 8 Done Done Sent false
19730 5644 Acked Write 9 Done Done Sent false
19731 5645 Acked Read 8 Done Done Sent false
19732 5646 Acked Read 1 Done Done Sent false
19733 5647 Acked Read 10 Done Done Sent false
19734 5648 Acked Read 9 Done Done Sent false
19735 5649 Acked Flush 0 Done Done Sent false
19736 5650 Acked Read 6 Done Done Sent false
19737 5651 Acked Read 1 Done Done Sent false
19738 5652 Acked Read 3 Done Done Sent false
19739 5653 Acked Read 7 Done Done Sent false
19740 5654 Acked Write 9 Done Done Sent false
19741 5655 Acked Read 1 Done Done Sent false
19742 5656 Acked Write 5 Done Done Sent false
19743 5657 Acked Read 3 Done Done Sent false
19744 5658 Acked Write 1 Done Done Sent false
19745 5659 Acked Write 6 Done Done Sent false
19746 5660 Acked Read 1 Done Done Sent false
19747 5661 Acked Read 1 Done Done Sent false
19748 5662 Acked Read 1 Done Done Sent false
19749 5663 Acked Write 9 Done Done Sent false
19750 5664 Acked Read 9 Done Done Sent false
19751 5665 Acked Read 8 Done Done Sent false
19752 5666 Acked Flush 0 Done Done Sent false
19753 5667 Acked Flush 0 Done Done Sent false
19754 5668 Acked Read 8 Done Done Sent false
19755 5669 Acked Read 6 Done Done Sent false
19756 5670 Acked Write 4 Done Done Sent false
19757 5671 Acked Write 9 Done Done Sent false
19758 5672 Acked Read 7 Done Done Sent false
19759 5673 Acked Read 9 Done Done Sent false
19760 5674 Acked Write 8 Done Done Sent false
19761 5675 Acked Read 5 Done Done Sent false
19762 5676 Acked Read 2 Done Done Sent false
19763 5677 Acked Write 2 Done Done Sent false
19764 5678 Acked Read 10 Done Done Sent false
19765 5679 Acked Write 5 Done Done Sent false
19766 5680 Acked Write 6 Done Done Sent false
19767 5681 Acked Write 6 Done Done Sent false
19768 5682 Acked Read 3 Done Done Sent false
19769 5683 Acked Read 4 Done Done Sent false
19770 5684 Acked Write 7 Done Done Sent false
19771 5685 Acked Write 8 Done Done Sent false
19772 5686 Acked Read 2 Done Done Sent false
19773 5687 Acked Write 8 Done Done Sent false
19774 5688 Acked Write 5 Done Done Sent false
19775 5689 Acked Write 4 Done Done Sent false
19776 5690 Acked Write 5 Done Done Sent false
19777 5691 Acked Write 2 Done Done Sent false
19778 5692 Acked Flush 0 Done Done Sent false
19779 5693 Acked Read 6 Done Done Sent false
19780 5694 Acked Read 3 Done Done Sent false
19781 5695 Acked Read 4 Done Done Sent false
19782 5696 Acked Write 6 Done Done Sent false
19783 5697 Acked Read 6 Done Done Sent false
19784 5698 Acked Read 1 Done Done Sent false
19785 5699 Acked Flush 0 Done Done Sent false
19786 5700 Acked Read 8 Done Done Sent false
19787 5701 Acked Read 8 Done Done Sent false
19788 5702 Acked Flush 0 Done Done Sent false
19789 5703 Acked Flush 0 Done Done Sent false
19790 5704 Acked Read 8 Done Done Sent false
19791 5705 Acked Write 10 Done Done Sent false
19792 5706 Acked Write 2 Done Done Sent false
19793 5707 Acked Flush 0 Done Done Sent false
19794 5708 Acked Read 2 Done Done Sent false
19795 5709 Acked Write 2 Done Done Sent false
19796 5710 Acked Write 2 Done Done Sent false
19797 5711 Acked Read 3 Done Done Sent false
19798 5712 Acked Read 8 Done Done Sent false
19799 5713 Acked Flush 0 Done Done Sent false
19800 5714 Acked Write 7 Done Done Sent false
19801 5715 Acked Write 1 Done Done Sent false
19802 5716 Acked Read 7 Done Done Sent false
19803 5717 Acked Flush 0 Done Done Sent false
19804 5718 Acked Write 10 Done Done Sent false
19805 5719 Acked Read 4 Done Done Sent false
19806 5720 Acked Read 6 Done Done Sent false
19807 5721 Acked Read 6 Done Done Sent false
19808 5722 Acked Read 2 Done Done Sent false
19809 5723 Acked Write 10 Done Done Sent false
19810 5724 Acked Write 10 Done Done Sent false
19811 5725 Acked Read 9 Done Done Sent false
19812 5726 Acked Write 7 Done Done Sent false
19813 5727 Acked Read 9 Done Done Sent false
19814 5728 Acked Read 7 Done Done Sent false
19815 5729 Acked Read 3 Done Done Sent false
19816 5730 Acked Read 10 Done Done Sent false
19817 5731 Acked Read 3 Done Done Sent false
19818 5732 Acked Read 1 Done Done Sent false
19819 5733 Acked Write 3 Done Done Sent false
19820 5734 Acked Write 3 Done Done Sent false
19821 5735 Acked Read 8 Done Done Sent false
19822 5736 Acked Read 2 Done Done Sent false
19823 5737 Acked Write 1 Done Done Sent false
19824 5738 Acked Write 8 Done Done Sent false
19825 5739 Acked Write 9 Done Done Sent false
19826 5740 Acked Write 6 Done Done Sent false
19827 5741 Acked Flush 0 Done Done Sent false
19828 5742 Acked Write 9 Done Done Sent false
19829 5743 Acked Flush 0 Done Done Sent false
19830 5744 Acked Read 1 Done Done Sent false
19831 5745 Acked Write 6 Done Done Sent false
19832 5746 Acked Read 1 Done Done Sent false
19833 5747 Acked Write 4 Done Done Sent false
19834 5748 Acked Write 9 Done Done Sent false
19835 5749 Acked Read 3 Done Done Sent false
19836 5750 Acked Read 2 Done Done Sent false
19837 5751 Acked Read 10 Done Done Sent false
19838 5752 Acked Read 7 Done Done Sent false
19839 5753 Acked Write 3 Done Done Sent false
19840 5754 Acked Write 9 Done Done Sent false
19841 5755 Acked Write 5 Done Done Sent false
19842 5756 Acked Read 5 Done Done Sent false
19843 5757 Acked Write 7 Done Done Sent false
19844 5758 Acked Write 10 Done Done Sent false
19845 5759 Acked Read 2 Done Done Sent false
19846 5760 Acked Write 4 Done Done Sent false
19847 5761 Acked Write 2 Done Done Sent false
19848 5762 Acked Write 4 Done Done Sent false
19849 5763 Acked Flush 0 Done Done Sent false
19850 5764 Acked Read 6 Done Done Sent false
19851 5765 Acked Read 7 Done Done Sent false
19852 5766 Acked Read 3 Done Done Sent false
19853 5767 Acked Write 7 Done Done Sent false
19854 5768 Acked Read 3 Done Done Sent false
19855 5769 Acked Write 2 Done Done Sent false
19856 5770 Acked Write 1 Done Done Sent false
19857 5771 Acked Write 2 Done Done Sent false
19858 5772 Acked Flush 0 Done Done Sent false
19859 5773 Acked Write 2 Done Done Sent false
19860 5774 Acked Write 8 Done Done Sent false
19861 5775 Acked Flush 0 Done Done Sent false
19862 5776 Acked Read 5 Done Done Sent false
19863 5777 Acked Write 7 Done Done Sent false
19864 5778 Acked Read 3 Done Done Sent false
19865 5779 Acked Write 5 Done Done Sent false
19866 5780 Acked Write 2 Done Done Sent false
19867 5781 Acked Read 5 Done Done Sent false
19868 5782 Acked Flush 0 Done Done Sent false
19869 5783 Acked Read 5 Done Done Sent false
19870 5784 Acked Write 10 Done Done Sent false
19871 5785 Acked Flush 0 Done Done Sent false
19872 5786 Acked Write 2 Done Done Sent false
19873 5787 Acked Read 2 Done Done Sent false
19874 5788 Acked Read 3 Done Done Sent false
19875 5789 Acked Read 8 Done Done Sent false
19876 5790 Acked Write 3 Done Done Sent false
19877 5791 Acked Write 6 Done Done Sent false
19878 5792 Acked Read 9 Done Done Sent false
19879 5793 Acked Read 3 Done Done Sent false
19880 5794 Acked Read 6 Done Done Sent false
19881 5795 Acked Write 8 Done Done Sent false
19882 5796 Acked Read 8 Done Done Sent false
19883 5797 Acked Write 4 Done Done Sent false
19884 5798 Acked Read 4 Done Done Sent false
19885 5799 Acked Read 7 Done Done Sent false
19886 5800 Acked Write 8 Done Done Sent false
19887 5801 Acked Read 5 Done Done Sent false
19888 5802 Acked Write 3 Done Done Sent false
19889 5803 Acked Read 8 Done Done Sent false
19890 5804 Acked Write 5 Done Done Sent false
19891 5805 Acked Read 1 Done Done Sent false
19892 5806 Acked Read 2 Done Done Sent false
19893 5807 Acked Read 9 Done Done Sent false
19894 5808 Acked Read 1 Done Done Sent false
19895 5809 Acked Read 3 Done Done Sent false
19896 5810 Acked Read 7 Done Done Sent false
19897 5811 Acked Read 8 Done Done Sent false
19898 5812 Acked Read 6 Done Done Sent false
19899 5813 Acked Read 8 Done Done Sent false
19900 5814 Acked Flush 0 Done Done Sent false
19901 5815 Acked Read 10 Done Done Sent false
19902 5816 Acked Read 3 Done Done Sent false
19903 5817 Acked Read 2 Done Done Sent false
19904 5818 Acked Write 10 Done Done Sent false
19905 5819 Acked Read 6 Done Done Sent false
19906 5820 Acked Write 3 Done Done Sent false
19907 5821 Acked Read 8 Done Done Sent false
19908 5822 Acked Write 1 Done Done Sent false
19909 5823 Acked Write 10 Done Done Sent false
19910 5824 Acked Read 5 Done Done Sent false
19911 5825 Acked Write 5 Done Done Sent false
19912 5826 Acked Read 4 Done Done Sent false
19913 5827 Acked Read 6 Done Done Sent false
19914 5828 Acked Read 8 Done Done Sent false
19915 STATES DS:0 DS:1 DS:2 TOTAL
19916 Sent 0 0 301 301
19917 Done 301 301 0 602
19918 Skipped 0 0 0 0
19919 Error 0 0 0 0
19920 Last Flush: 5814 5814 5527
19921 Downstairs last five completed: 5527 5526 5525 5524 5523
19922 Upstairs last five completed: 5828 5827 5826 5825 5824
19923 CLIENT: Up:0 ds:301 act:2
19924 ----------------------------------------------------------------
19925 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
19926 States: Active Active Offline
19927 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
19928 5528 Acked Read 1 Done Done Sent false
19929 5529 Acked Read 7 Done Done Sent false
19930 5530 Acked Write 9 Done Done Sent false
19931 5531 Acked Write 3 Done Done Sent false
19932 5532 Acked Write 3 Done Done Sent false
19933 5533 Acked Write 3 Done Done Sent false
19934 5534 Acked Write 9 Done Done Sent false
19935 5535 Acked Read 6 Done Done Sent false
19936 5536 Acked Read 3 Done Done Sent false
19937 5537 Acked Write 10 Done Done Sent false
19938 5538 Acked Write 10 Done Done Sent false
19939 5539 Acked Read 10 Done Done Sent false
19940 5540 Acked Read 3 Done Done Sent false
19941 5541 Acked Read 10 Done Done Sent false
19942 5542 Acked Flush 0 Done Done Sent false
19943 5543 Acked Read 5 Done Done Sent false
19944 5544 Acked Flush 0 Done Done Sent false
19945 5545 Acked Read 10 Done Done Sent false
19946 5546 Acked Write 7 Done Done Sent false
19947 5547 Acked Read 4 Done Done Sent false
19948 5548 Acked Read 1 Done Done Sent false
19949 5549 Acked Write 2 Done Done Sent false
19950 5550 Acked Read 3 Done Done Sent false
19951 5551 Acked Write 9 Done Done Sent false
19952 5552 Acked Write 1 Done Done Sent false
19953 5553 Acked Read 2 Done Done Sent false
19954 5554 Acked Write 2 Done Done Sent false
19955 5555 Acked Read 6 Done Done Sent false
19956 5556 Acked Write 3 Done Done Sent false
19957 5557 Acked Write 5 Done Done Sent false
19958 5558 Acked Read 8 Done Done Sent false
19959 5559 Acked Write 6 Done Done Sent false
19960 5560 Acked Read 5 Done Done Sent false
19961 5561 Acked Write 7 Done Done Sent false
19962 5562 Acked Write 8 Done Done Sent false
19963 5563 Acked Read 7 Done Done Sent false
19964 5564 Acked Read 4 Done Done Sent false
19965 5565 Acked Flush 0 Done Done Sent false
19966 5566 Acked Read 1 Done Done Sent false
19967 5567 Acked Read 5 Done Done Sent false
19968 5568 Acked Read 8 Done Done Sent false
19969 5569 Acked Read 4 Done Done Sent false
19970 5570 Acked Write 5 Done Done Sent false
19971 5571 Acked Write 6 Done Done Sent false
19972 5572 Acked Read 1 Done Done Sent false
19973 5573 Acked Read 9 Done Done Sent false
19974 5574 Acked Read 1 Done Done Sent false
19975 5575 Acked Read 1 Done Done Sent false
19976 5576 Acked Flush 0 Done Done Sent false
19977 5577 Acked Read 6 Done Done Sent false
19978 5578 Acked Write 9 Done Done Sent false
19979 5579 Acked Flush 0 Done Done Sent false
19980 5580 Acked Read 7 Done Done Sent false
19981 5581 Acked Flush 0 Done Done Sent false
19982 5582 Acked Read 5 Done Done Sent false
19983 5583 Acked Read 9 Done Done Sent false
19984 5584 Acked Write 9 Done Done Sent false
19985 5585 Acked Read 8 Done Done Sent false
19986 5586 Acked Flush 0 Done Done Sent false
19987 5587 Acked Read 10 Done Done Sent false
19988 5588 Acked Flush 0 Done Done Sent false
19989 5589 Acked Read 2 Done Done Sent false
19990 5590 Acked Read 4 Done Done Sent false
19991 5591 Acked Read 10 Done Done Sent false
19992 5592 Acked Write 9 Done Done Sent false
19993 5593 Acked Read 8 Done Done Sent false
19994 5594 Acked Read 3 Done Done Sent false
19995 5595 Acked Read 5 Done Done Sent false
19996 5596 Acked Read 2 Done Done Sent false
19997 5597 Acked Write 7 Done Done Sent false
19998 5598 Acked Write 1 Done Done Sent false
19999 5599 Acked Flush 0 Done Done Sent false
20000 5600 Acked Read 1 Done Done Sent false
20001 5601 Acked Read 10 Done Done Sent false
20002 5602 Acked Write 8 Done Done Sent false
20003 5603 Acked Read 3 Done Done Sent false
20004 5604 Acked Write 3 Done Done Sent false
20005 5605 Acked Write 6 Done Done Sent false
20006 5606 Acked Read 6 Done Done Sent false
20007 5607 Acked Write 7 Done Done Sent false
20008 5608 Acked Write 7 Done Done Sent false
20009 5609 Acked Read 9 Done Done Sent false
20010 5610 Acked Read 8 Done Done Sent false
20011 5611 Acked Read 4 Done Done Sent false
20012 5612 Acked Read 1 Done Done Sent false
20013 5613 Acked Read 5 Done Done Sent false
20014 5614 Acked Write 3 Done Done Sent false
20015 5615 Acked Write 7 Done Done Sent false
20016 5616 Acked Write 1 Done Done Sent false
20017 5617 Acked Write 8 Done Done Sent false
20018 5618 Acked Flush 0 Done Done Sent false
20019 5619 Acked Read 9 Done Done Sent false
20020 5620 Acked Write 6 Done Done Sent false
20021 5621 Acked Write 3 Done Done Sent false
20022 5622 Acked Write 2 Done Done Sent false
20023 5623 Acked Flush 0 Done Done Sent false
20024 5624 Acked Read 2 Done Done Sent false
20025 5625 Acked Read 2 Done Done Sent false
20026 5626 Acked Read 8 Done Done Sent false
20027 5627 Acked Write 1 Done Done Sent false
20028 5628 Acked Write 8 Done Done Sent false
20029 5629 Acked Read 1 Done Done Sent false
20030 5630 Acked Write 9 Done Done Sent false
20031 5631 Acked Read 5 Done Done Sent false
20032 5632 Acked Read 3 Done Done Sent false
20033 5633 Acked Write 4 Done Done Sent false
20034 5634 Acked Read 10 Done Done Sent false
20035 5635 Acked Flush 0 Done Done Sent false
20036 5636 Acked Write 3 Done Done Sent false
20037 5637 Acked Read 8 Done Done Sent false
20038 5638 Acked Read 4 Done Done Sent false
20039 5639 Acked Write 3 Done Done Sent false
20040 5640 Acked Read 3 Done Done Sent false
20041 5641 Acked Read 4 Done Done Sent false
20042 5642 Acked Write 3 Done Done Sent false
20043 5643 Acked Write 8 Done Done Sent false
20044 5644 Acked Write 9 Done Done Sent false
20045 5645 Acked Read 8 Done Done Sent false
20046 5646 Acked Read 1 Done Done Sent false
20047 5647 Acked Read 10 Done Done Sent false
20048 5648 Acked Read 9 Done Done Sent false
20049 5649 Acked Flush 0 Done Done Sent false
20050 5650 Acked Read 6 Done Done Sent false
20051 5651 Acked Read 1 Done Done Sent false
20052 5652 Acked Read 3 Done Done Sent false
20053 5653 Acked Read 7 Done Done Sent false
20054 5654 Acked Write 9 Done Done Sent false
20055 5655 Acked Read 1 Done Done Sent false
20056 5656 Acked Write 5 Done Done Sent false
20057 5657 Acked Read 3 Done Done Sent false
20058 5658 Acked Write 1 Done Done Sent false
20059 5659 Acked Write 6 Done Done Sent false
20060 5660 Acked Read 1 Done Done Sent false
20061 5661 Acked Read 1 Done Done Sent false
20062 5662 Acked Read 1 Done Done Sent false
20063 5663 Acked Write 9 Done Done Sent false
20064 5664 Acked Read 9 Done Done Sent false
20065 5665 Acked Read 8 Done Done Sent false
20066 5666 Acked Flush 0 Done Done Sent false
20067 5667 Acked Flush 0 Done Done Sent false
20068 5668 Acked Read 8 Done Done Sent false
20069 5669 Acked Read 6 Done Done Sent false
20070 5670 Acked Write 4 Done Done Sent false
20071 5671 Acked Write 9 Done Done Sent false
20072 5672 Acked Read 7 Done Done Sent false
20073 5673 Acked Read 9 Done Done Sent false
20074 5674 Acked Write 8 Done Done Sent false
20075 5675 Acked Read 5 Done Done Sent false
20076 5676 Acked Read 2 Done Done Sent false
20077 5677 Acked Write 2 Done Done Sent false
20078 5678 Acked Read 10 Done Done Sent false
20079 5679 Acked Write 5 Done Done Sent false
20080 5680 Acked Write 6 Done Done Sent false
20081 5681 Acked Write 6 Done Done Sent false
20082 5682 Acked Read 3 Done Done Sent false
20083 5683 Acked Read 4 Done Done Sent false
20084 5684 Acked Write 7 Done Done Sent false
20085 5685 Acked Write 8 Done Done Sent false
20086 5686 Acked Read 2 Done Done Sent false
20087 5687 Acked Write 8 Done Done Sent false
20088 5688 Acked Write 5 Done Done Sent false
20089 5689 Acked Write 4 Done Done Sent false
20090 5690 Acked Write 5 Done Done Sent false
20091 5691 Acked Write 2 Done Done Sent false
20092 5692 Acked Flush 0 Done Done Sent false
20093 5693 Acked Read 6 Done Done Sent false
20094 5694 Acked Read 3 Done Done Sent false
20095 5695 Acked Read 4 Done Done Sent false
20096 5696 Acked Write 6 Done Done Sent false
20097 5697 Acked Read 6 Done Done Sent false
20098 5698 Acked Read 1 Done Done Sent false
20099 5699 Acked Flush 0 Done Done Sent false
20100 5700 Acked Read 8 Done Done Sent false
20101 5701 Acked Read 8 Done Done Sent false
20102 5702 Acked Flush 0 Done Done Sent false
20103 5703 Acked Flush 0 Done Done Sent false
20104 5704 Acked Read 8 Done Done Sent false
20105 5705 Acked Write 10 Done Done Sent false
20106 5706 Acked Write 2 Done Done Sent false
20107 5707 Acked Flush 0 Done Done Sent false
20108 5708 Acked Read 2 Done Done Sent false
20109 5709 Acked Write 2 Done Done Sent false
20110 5710 Acked Write 2 Done Done Sent false
20111 5711 Acked Read 3 Done Done Sent false
20112 5712 Acked Read 8 Done Done Sent false
20113 5713 Acked Flush 0 Done Done Sent false
20114 5714 Acked Write 7 Done Done Sent false
20115 5715 Acked Write 1 Done Done Sent false
20116 5716 Acked Read 7 Done Done Sent false
20117 5717 Acked Flush 0 Done Done Sent false
20118 5718 Acked Write 10 Done Done Sent false
20119 5719 Acked Read 4 Done Done Sent false
20120 5720 Acked Read 6 Done Done Sent false
20121 5721 Acked Read 6 Done Done Sent false
20122 5722 Acked Read 2 Done Done Sent false
20123 5723 Acked Write 10 Done Done Sent false
20124 5724 Acked Write 10 Done Done Sent false
20125 5725 Acked Read 9 Done Done Sent false
20126 5726 Acked Write 7 Done Done Sent false
20127 5727 Acked Read 9 Done Done Sent false
20128 5728 Acked Read 7 Done Done Sent false
20129 5729 Acked Read 3 Done Done Sent false
20130 5730 Acked Read 10 Done Done Sent false
20131 5731 Acked Read 3 Done Done Sent false
20132 5732 Acked Read 1 Done Done Sent false
20133 5733 Acked Write 3 Done Done Sent false
20134 5734 Acked Write 3 Done Done Sent false
20135 5735 Acked Read 8 Done Done Sent false
20136 5736 Acked Read 2 Done Done Sent false
20137 5737 Acked Write 1 Done Done Sent false
20138 5738 Acked Write 8 Done Done Sent false
20139 5739 Acked Write 9 Done Done Sent false
20140 5740 Acked Write 6 Done Done Sent false
20141 5741 Acked Flush 0 Done Done Sent false
20142 5742 Acked Write 9 Done Done Sent false
20143 5743 Acked Flush 0 Done Done Sent false
20144 5744 Acked Read 1 Done Done Sent false
20145 5745 Acked Write 6 Done Done Sent false
20146 5746 Acked Read 1 Done Done Sent false
20147 5747 Acked Write 4 Done Done Sent false
20148 5748 Acked Write 9 Done Done Sent false
20149 5749 Acked Read 3 Done Done Sent false
20150 5750 Acked Read 2 Done Done Sent false
20151 5751 Acked Read 10 Done Done Sent false
20152 5752 Acked Read 7 Done Done Sent false
20153 5753 Acked Write 3 Done Done Sent false
20154 5754 Acked Write 9 Done Done Sent false
20155 5755 Acked Write 5 Done Done Sent false
20156 5756 Acked Read 5 Done Done Sent false
20157 5757 Acked Write 7 Done Done Sent false
20158 5758 Acked Write 10 Done Done Sent false
20159 5759 Acked Read 2 Done Done Sent false
20160 5760 Acked Write 4 Done Done Sent false
20161 5761 Acked Write 2 Done Done Sent false
20162 5762 Acked Write 4 Done Done Sent false
20163 5763 Acked Flush 0 Done Done Sent false
20164 5764 Acked Read 6 Done Done Sent false
20165 5765 Acked Read 7 Done Done Sent false
20166 5766 Acked Read 3 Done Done Sent false
20167 5767 Acked Write 7 Done Done Sent false
20168 5768 Acked Read 3 Done Done Sent false
20169 5769 Acked Write 2 Done Done Sent false
20170 5770 Acked Write 1 Done Done Sent false
20171 5771 Acked Write 2 Done Done Sent false
20172 5772 Acked Flush 0 Done Done Sent false
20173 5773 Acked Write 2 Done Done Sent false
20174 5774 Acked Write 8 Done Done Sent false
20175 5775 Acked Flush 0 Done Done Sent false
20176 5776 Acked Read 5 Done Done Sent false
20177 5777 Acked Write 7 Done Done Sent false
20178 5778 Acked Read 3 Done Done Sent false
20179 5779 Acked Write 5 Done Done Sent false
20180 5780 Acked Write 2 Done Done Sent false
20181 5781 Acked Read 5 Done Done Sent false
20182 5782 Acked Flush 0 Done Done Sent false
20183 5783 Acked Read 5 Done Done Sent false
20184 5784 Acked Write 10 Done Done Sent false
20185 5785 Acked Flush 0 Done Done Sent false
20186 5786 Acked Write 2 Done Done Sent false
20187 5787 Acked Read 2 Done Done Sent false
20188 5788 Acked Read 3 Done Done Sent false
20189 5789 Acked Read 8 Done Done Sent false
20190 5790 Acked Write 3 Done Done Sent false
20191 5791 Acked Write 6 Done Done Sent false
20192 5792 Acked Read 9 Done Done Sent false
20193 5793 Acked Read 3 Done Done Sent false
20194 5794 Acked Read 6 Done Done Sent false
20195 5795 Acked Write 8 Done Done Sent false
20196 5796 Acked Read 8 Done Done Sent false
20197 5797 Acked Write 4 Done Done Sent false
20198 5798 Acked Read 4 Done Done Sent false
20199 5799 Acked Read 7 Done Done Sent false
20200 5800 Acked Write 8 Done Done Sent false
20201 5801 Acked Read 5 Done Done Sent false
20202 5802 Acked Write 3 Done Done Sent false
20203 5803 Acked Read 8 Done Done Sent false
20204 5804 Acked Write 5 Done Done Sent false
20205 5805 Acked Read 1 Done Done Sent false
20206 5806 Acked Read 2 Done Done Sent false
20207 5807 Acked Read 9 Done Done Sent false
20208 5808 Acked Read 1 Done Done Sent false
20209 5809 Acked Read 3 Done Done Sent false
20210 5810 Acked Read 7 Done Done Sent false
20211 5811 Acked Read 8 Done Done Sent false
20212 5812 Acked Read 6 Done Done Sent false
20213 5813 Acked Read 8 Done Done Sent false
20214 5814 Acked Flush 0 Done Done Sent false
20215 5815 Acked Read 10 Done Done Sent false
20216 5816 Acked Read 3 Done Done Sent false
20217 5817 Acked Read 2 Done Done Sent false
20218 5818 Acked Write 10 Done Done Sent false
20219 5819 Acked Read 6 Done Done Sent false
20220 5820 Acked Write 3 Done Done Sent false
20221 5821 Acked Read 8 Done Done Sent false
20222 5822 Acked Write 1 Done Done Sent false
20223 5823 Acked Write 10 Done Done Sent false
20224 5824 Acked Read 5 Done Done Sent false
20225 5825 Acked Write 5 Done Done Sent false
20226 5826 Acked Read 4 Done Done Sent false
20227 5827 Acked Read 6 Done Done Sent false
20228 5828 Acked Read 8 Done Done Sent false
20229 5829 Acked Flush 0 Done Done Sent false
20230 STATES DS:0 DS:1 DS:2 TOTAL
20231 Sent 0 0 302 302
20232 Done 302 302 0 604
20233 Skipped 0 0 0 0
20234 Error 0 0 0 0
20235 Last Flush: 5829 5829 5527
20236 Downstairs last five completed: 5527 5526 5525 5524 5523
20237 Upstairs last five completed: 5829 5828 5827 5826 5825
20238 CLIENT: Up:0 ds:302 act:2
202392024-11-15T02:01:41.584ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
202402024-11-15T02:01:41.584ZWARNcrucible: ds_connection connect to 127.0.0.1:8830 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
202412024-11-15T02:01:41.585ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
202422024-11-15T02:01:41.585ZWARNcrucible: downstairs task for 2 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
202432024-11-15T02:01:41.585ZINFOcrucible: client task is exiting = io task client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
20244 [8830] got start action
20245 [8830] I am going to restart
20246 Starting downstairs at port 8830
20247 Make output file at "/tmp/dsc/downstairs-8830.txt"
20248 [8830][2] reports Starting
20249 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1221)
20250 [8830][2] reports Running
20251 ----------------------------------------------------------------
20252 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
20253 States: Active Active Offline
20254 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
20255 5528 Acked Read 1 Done Done Sent false
20256 5529 Acked Read 7 Done Done Sent false
20257 5530 Acked Write 9 Done Done Sent false
20258 5531 Acked Write 3 Done Done Sent false
20259 5532 Acked Write 3 Done Done Sent false
20260 5533 Acked Write 3 Done Done Sent false
20261 5534 Acked Write 9 Done Done Sent false
20262 5535 Acked Read 6 Done Done Sent false
20263 5536 Acked Read 3 Done Done Sent false
20264 5537 Acked Write 10 Done Done Sent false
20265 5538 Acked Write 10 Done Done Sent false
20266 5539 Acked Read 10 Done Done Sent false
20267 5540 Acked Read 3 Done Done Sent false
20268 5541 Acked Read 10 Done Done Sent false
20269 5542 Acked Flush 0 Done Done Sent false
20270 5543 Acked Read 5 Done Done Sent false
20271 5544 Acked Flush 0 Done Done Sent false
20272 5545 Acked Read 10 Done Done Sent false
20273 5546 Acked Write 7 Done Done Sent false
20274 5547 Acked Read 4 Done Done Sent false
20275 5548 Acked Read 1 Done Done Sent false
20276 5549 Acked Write 2 Done Done Sent false
20277 5550 Acked Read 3 Done Done Sent false
20278 5551 Acked Write 9 Done Done Sent false
20279 5552 Acked Write 1 Done Done Sent false
20280 5553 Acked Read 2 Done Done Sent false
20281 5554 Acked Write 2 Done Done Sent false
20282 5555 Acked Read 6 Done Done Sent false
20283 5556 Acked Write 3 Done Done Sent false
20284 5557 Acked Write 5 Done Done Sent false
20285 5558 Acked Read 8 Done Done Sent false
20286 5559 Acked Write 6 Done Done Sent false
20287 5560 Acked Read 5 Done Done Sent false
20288 5561 Acked Write 7 Done Done Sent false
20289 5562 Acked Write 8 Done Done Sent false
20290 5563 Acked Read 7 Done Done Sent false
20291 5564 Acked Read 4 Done Done Sent false
20292 5565 Acked Flush 0 Done Done Sent false
20293 5566 Acked Read 1 Done Done Sent false
20294 5567 Acked Read 5 Done Done Sent false
20295 5568 Acked Read 8 Done Done Sent false
20296 5569 Acked Read 4 Done Done Sent false
20297 5570 Acked Write 5 Done Done Sent false
20298 5571 Acked Write 6 Done Done Sent false
20299 5572 Acked Read 1 Done Done Sent false
20300 5573 Acked Read 9 Done Done Sent false
20301 5574 Acked Read 1 Done Done Sent false
20302 5575 Acked Read 1 Done Done Sent false
20303 5576 Acked Flush 0 Done Done Sent false
20304 5577 Acked Read 6 Done Done Sent false
20305 5578 Acked Write 9 Done Done Sent false
20306 5579 Acked Flush 0 Done Done Sent false
20307 5580 Acked Read 7 Done Done Sent false
20308 5581 Acked Flush 0 Done Done Sent false
20309 5582 Acked Read 5 Done Done Sent false
20310 5583 Acked Read 9 Done Done Sent false
20311 5584 Acked Write 9 Done Done Sent false
20312 5585 Acked Read 8 Done Done Sent false
20313 5586 Acked Flush 0 Done Done Sent false
20314 5587 Acked Read 10 Done Done Sent false
20315 5588 Acked Flush 0 Done Done Sent false
20316 5589 Acked Read 2 Done Done Sent false
20317 5590 Acked Read 4 Done Done Sent false
20318 5591 Acked Read 10 Done Done Sent false
20319 5592 Acked Write 9 Done Done Sent false
20320 5593 Acked Read 8 Done Done Sent false
20321 5594 Acked Read 3 Done Done Sent false
20322 5595 Acked Read 5 Done Done Sent false
20323 5596 Acked Read 2 Done Done Sent false
20324 5597 Acked Write 7 Done Done Sent false
20325 5598 Acked Write 1 Done Done Sent false
20326 5599 Acked Flush 0 Done Done Sent false
20327 5600 Acked Read 1 Done Done Sent false
20328 5601 Acked Read 10 Done Done Sent false
20329 5602 Acked Write 8 Done Done Sent false
20330 5603 Acked Read 3 Done Done Sent false
20331 5604 Acked Write 3 Done Done Sent false
20332 5605 Acked Write 6 Done Done Sent false
20333 5606 Acked Read 6 Done Done Sent false
20334 5607 Acked Write 7 Done Done Sent false
20335 5608 Acked Write 7 Done Done Sent false
20336 5609 Acked Read 9 Done Done Sent false
20337 5610 Acked Read 8 Done Done Sent false
20338 5611 Acked Read 4 Done Done Sent false
20339 5612 Acked Read 1 Done Done Sent false
20340 5613 Acked Read 5 Done Done Sent false
20341 5614 Acked Write 3 Done Done Sent false
20342 5615 Acked Write 7 Done Done Sent false
20343 5616 Acked Write 1 Done Done Sent false
20344 5617 Acked Write 8 Done Done Sent false
20345 5618 Acked Flush 0 Done Done Sent false
20346 5619 Acked Read 9 Done Done Sent false
20347 5620 Acked Write 6 Done Done Sent false
20348 5621 Acked Write 3 Done Done Sent false
20349 5622 Acked Write 2 Done Done Sent false
20350 5623 Acked Flush 0 Done Done Sent false
20351 5624 Acked Read 2 Done Done Sent false
20352 5625 Acked Read 2 Done Done Sent false
20353 5626 Acked Read 8 Done Done Sent false
20354 5627 Acked Write 1 Done Done Sent false
20355 5628 Acked Write 8 Done Done Sent false
20356 5629 Acked Read 1 Done Done Sent false
20357 5630 Acked Write 9 Done Done Sent false
20358 5631 Acked Read 5 Done Done Sent false
20359 5632 Acked Read 3 Done Done Sent false
20360 5633 Acked Write 4 Done Done Sent false
20361 5634 Acked Read 10 Done Done Sent false
20362 5635 Acked Flush 0 Done Done Sent false
20363 5636 Acked Write 3 Done Done Sent false
20364 5637 Acked Read 8 Done Done Sent false
20365 5638 Acked Read 4 Done Done Sent false
20366 5639 Acked Write 3 Done Done Sent false
20367 5640 Acked Read 3 Done Done Sent false
20368 5641 Acked Read 4 Done Done Sent false
20369 5642 Acked Write 3 Done Done Sent false
20370 5643 Acked Write 8 Done Done Sent false
20371 5644 Acked Write 9 Done Done Sent false
20372 5645 Acked Read 8 Done Done Sent false
20373 5646 Acked Read 1 Done Done Sent false
20374 5647 Acked Read 10 Done Done Sent false
20375 5648 Acked Read 9 Done Done Sent false
20376 5649 Acked Flush 0 Done Done Sent false
20377 5650 Acked Read 6 Done Done Sent false
20378 5651 Acked Read 1 Done Done Sent false
20379 5652 Acked Read 3 Done Done Sent false
20380 5653 Acked Read 7 Done Done Sent false
20381 5654 Acked Write 9 Done Done Sent false
20382 5655 Acked Read 1 Done Done Sent false
20383 5656 Acked Write 5 Done Done Sent false
20384 5657 Acked Read 3 Done Done Sent false
20385 5658 Acked Write 1 Done Done Sent false
20386 5659 Acked Write 6 Done Done Sent false
20387 5660 Acked Read 1 Done Done Sent false
20388 5661 Acked Read 1 Done Done Sent false
20389 5662 Acked Read 1 Done Done Sent false
20390 5663 Acked Write 9 Done Done Sent false
20391 5664 Acked Read 9 Done Done Sent false
20392 5665 Acked Read 8 Done Done Sent false
20393 5666 Acked Flush 0 Done Done Sent false
20394 5667 Acked Flush 0 Done Done Sent false
20395 5668 Acked Read 8 Done Done Sent false
20396 5669 Acked Read 6 Done Done Sent false
20397 5670 Acked Write 4 Done Done Sent false
20398 5671 Acked Write 9 Done Done Sent false
20399 5672 Acked Read 7 Done Done Sent false
20400 5673 Acked Read 9 Done Done Sent false
20401 5674 Acked Write 8 Done Done Sent false
20402 5675 Acked Read 5 Done Done Sent false
20403 5676 Acked Read 2 Done Done Sent false
20404 5677 Acked Write 2 Done Done Sent false
20405 5678 Acked Read 10 Done Done Sent false
20406 5679 Acked Write 5 Done Done Sent false
20407 5680 Acked Write 6 Done Done Sent false
20408 5681 Acked Write 6 Done Done Sent false
20409 5682 Acked Read 3 Done Done Sent false
20410 5683 Acked Read 4 Done Done Sent false
20411 5684 Acked Write 7 Done Done Sent false
20412 5685 Acked Write 8 Done Done Sent false
20413 5686 Acked Read 2 Done Done Sent false
20414 5687 Acked Write 8 Done Done Sent false
20415 5688 Acked Write 5 Done Done Sent false
20416 5689 Acked Write 4 Done Done Sent false
20417 5690 Acked Write 5 Done Done Sent false
20418 5691 Acked Write 2 Done Done Sent false
20419 5692 Acked Flush 0 Done Done Sent false
20420 5693 Acked Read 6 Done Done Sent false
20421 5694 Acked Read 3 Done Done Sent false
20422 5695 Acked Read 4 Done Done Sent false
20423 5696 Acked Write 6 Done Done Sent false
20424 5697 Acked Read 6 Done Done Sent false
20425 5698 Acked Read 1 Done Done Sent false
20426 5699 Acked Flush 0 Done Done Sent false
20427 5700 Acked Read 8 Done Done Sent false
20428 5701 Acked Read 8 Done Done Sent false
20429 5702 Acked Flush 0 Done Done Sent false
20430 5703 Acked Flush 0 Done Done Sent false
20431 5704 Acked Read 8 Done Done Sent false
20432 5705 Acked Write 10 Done Done Sent false
20433 5706 Acked Write 2 Done Done Sent false
20434 5707 Acked Flush 0 Done Done Sent false
20435 5708 Acked Read 2 Done Done Sent false
20436 5709 Acked Write 2 Done Done Sent false
20437 5710 Acked Write 2 Done Done Sent false
20438 5711 Acked Read 3 Done Done Sent false
20439 5712 Acked Read 8 Done Done Sent false
20440 5713 Acked Flush 0 Done Done Sent false
20441 5714 Acked Write 7 Done Done Sent false
20442 5715 Acked Write 1 Done Done Sent false
20443 5716 Acked Read 7 Done Done Sent false
20444 5717 Acked Flush 0 Done Done Sent false
20445 5718 Acked Write 10 Done Done Sent false
20446 5719 Acked Read 4 Done Done Sent false
20447 5720 Acked Read 6 Done Done Sent false
20448 5721 Acked Read 6 Done Done Sent false
20449 5722 Acked Read 2 Done Done Sent false
20450 5723 Acked Write 10 Done Done Sent false
20451 5724 Acked Write 10 Done Done Sent false
20452 5725 Acked Read 9 Done Done Sent false
20453 5726 Acked Write 7 Done Done Sent false
20454 5727 Acked Read 9 Done Done Sent false
20455 5728 Acked Read 7 Done Done Sent false
20456 5729 Acked Read 3 Done Done Sent false
20457 5730 Acked Read 10 Done Done Sent false
20458 5731 Acked Read 3 Done Done Sent false
20459 5732 Acked Read 1 Done Done Sent false
20460 5733 Acked Write 3 Done Done Sent false
20461 5734 Acked Write 3 Done Done Sent false
20462 5735 Acked Read 8 Done Done Sent false
20463 5736 Acked Read 2 Done Done Sent false
20464 5737 Acked Write 1 Done Done Sent false
20465 5738 Acked Write 8 Done Done Sent false
20466 5739 Acked Write 9 Done Done Sent false
20467 5740 Acked Write 6 Done Done Sent false
20468 5741 Acked Flush 0 Done Done Sent false
20469 5742 Acked Write 9 Done Done Sent false
20470 5743 Acked Flush 0 Done Done Sent false
20471 5744 Acked Read 1 Done Done Sent false
20472 5745 Acked Write 6 Done Done Sent false
20473 5746 Acked Read 1 Done Done Sent false
20474 5747 Acked Write 4 Done Done Sent false
20475 5748 Acked Write 9 Done Done Sent false
20476 5749 Acked Read 3 Done Done Sent false
20477 5750 Acked Read 2 Done Done Sent false
20478 5751 Acked Read 10 Done Done Sent false
20479 5752 Acked Read 7 Done Done Sent false
20480 5753 Acked Write 3 Done Done Sent false
20481 5754 Acked Write 9 Done Done Sent false
20482 5755 Acked Write 5 Done Done Sent false
20483 5756 Acked Read 5 Done Done Sent false
20484 5757 Acked Write 7 Done Done Sent false
20485 5758 Acked Write 10 Done Done Sent false
20486 5759 Acked Read 2 Done Done Sent false
20487 5760 Acked Write 4 Done Done Sent false
20488 5761 Acked Write 2 Done Done Sent false
20489 5762 Acked Write 4 Done Done Sent false
20490 5763 Acked Flush 0 Done Done Sent false
20491 5764 Acked Read 6 Done Done Sent false
20492 5765 Acked Read 7 Done Done Sent false
20493 5766 Acked Read 3 Done Done Sent false
20494 5767 Acked Write 7 Done Done Sent false
20495 5768 Acked Read 3 Done Done Sent false
20496 5769 Acked Write 2 Done Done Sent false
20497 5770 Acked Write 1 Done Done Sent false
20498 5771 Acked Write 2 Done Done Sent false
20499 5772 Acked Flush 0 Done Done Sent false
20500 5773 Acked Write 2 Done Done Sent false
20501 5774 Acked Write 8 Done Done Sent false
20502 5775 Acked Flush 0 Done Done Sent false
20503 5776 Acked Read 5 Done Done Sent false
20504 5777 Acked Write 7 Done Done Sent false
20505 5778 Acked Read 3 Done Done Sent false
20506 5779 Acked Write 5 Done Done Sent false
20507 5780 Acked Write 2 Done Done Sent false
20508 5781 Acked Read 5 Done Done Sent false
20509 5782 Acked Flush 0 Done Done Sent false
20510 5783 Acked Read 5 Done Done Sent false
20511 5784 Acked Write 10 Done Done Sent false
20512 5785 Acked Flush 0 Done Done Sent false
20513 5786 Acked Write 2 Done Done Sent false
20514 5787 Acked Read 2 Done Done Sent false
20515 5788 Acked Read 3 Done Done Sent false
20516 5789 Acked Read 8 Done Done Sent false
20517 5790 Acked Write 3 Done Done Sent false
20518 5791 Acked Write 6 Done Done Sent false
20519 5792 Acked Read 9 Done Done Sent false
20520 5793 Acked Read 3 Done Done Sent false
20521 5794 Acked Read 6 Done Done Sent false
20522 5795 Acked Write 8 Done Done Sent false
20523 5796 Acked Read 8 Done Done Sent false
20524 5797 Acked Write 4 Done Done Sent false
20525 5798 Acked Read 4 Done Done Sent false
20526 5799 Acked Read 7 Done Done Sent false
20527 5800 Acked Write 8 Done Done Sent false
20528 5801 Acked Read 5 Done Done Sent false
20529 5802 Acked Write 3 Done Done Sent false
20530 5803 Acked Read 8 Done Done Sent false
20531 5804 Acked Write 5 Done Done Sent false
20532 5805 Acked Read 1 Done Done Sent false
20533 5806 Acked Read 2 Done Done Sent false
20534 5807 Acked Read 9 Done Done Sent false
20535 5808 Acked Read 1 Done Done Sent false
20536 5809 Acked Read 3 Done Done Sent false
20537 5810 Acked Read 7 Done Done Sent false
20538 5811 Acked Read 8 Done Done Sent false
20539 5812 Acked Read 6 Done Done Sent false
20540 5813 Acked Read 8 Done Done Sent false
20541 5814 Acked Flush 0 Done Done Sent false
20542 5815 Acked Read 10 Done Done Sent false
20543 5816 Acked Read 3 Done Done Sent false
20544 5817 Acked Read 2 Done Done Sent false
20545 5818 Acked Write 10 Done Done Sent false
20546 5819 Acked Read 6 Done Done Sent false
20547 5820 Acked Write 3 Done Done Sent false
20548 5821 Acked Read 8 Done Done Sent false
20549 5822 Acked Write 1 Done Done Sent false
20550 5823 Acked Write 10 Done Done Sent false
20551 5824 Acked Read 5 Done Done Sent false
20552 5825 Acked Write 5 Done Done Sent false
20553 5826 Acked Read 4 Done Done Sent false
20554 5827 Acked Read 6 Done Done Sent false
20555 5828 Acked Read 8 Done Done Sent false
20556 5829 Acked Flush 0 Done Done Sent false
20557 STATES DS:0 DS:1 DS:2 TOTAL
20558 Sent 0 0 302 302
20559 Done 302 302 0 604
20560 Skipped 0 0 0 0
20561 Error 0 0 0 0
20562 Last Flush: 5829 5829 5527
20563 Downstairs last five completed: 5527 5526 5525 5524 5523
20564 Upstairs last five completed: 5829 5828 5827 5826 5825
20565 CLIENT: Up:0 ds:302 act:2
20566 ----------------------------------------------------------------
20567 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
20568 States: Active Active Offline
20569 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
20570 5528 Acked Read 1 Done Done Sent false
20571 5529 Acked Read 7 Done Done Sent false
20572 5530 Acked Write 9 Done Done Sent false
20573 5531 Acked Write 3 Done Done Sent false
20574 5532 Acked Write 3 Done Done Sent false
20575 5533 Acked Write 3 Done Done Sent false
20576 5534 Acked Write 9 Done Done Sent false
20577 5535 Acked Read 6 Done Done Sent false
20578 5536 Acked Read 3 Done Done Sent false
20579 5537 Acked Write 10 Done Done Sent false
20580 5538 Acked Write 10 Done Done Sent false
20581 5539 Acked Read 10 Done Done Sent false
20582 5540 Acked Read 3 Done Done Sent false
20583 5541 Acked Read 10 Done Done Sent false
20584 5542 Acked Flush 0 Done Done Sent false
20585 5543 Acked Read 5 Done Done Sent false
20586 5544 Acked Flush 0 Done Done Sent false
20587 5545 Acked Read 10 Done Done Sent false
20588 5546 Acked Write 7 Done Done Sent false
20589 5547 Acked Read 4 Done Done Sent false
20590 5548 Acked Read 1 Done Done Sent false
20591 5549 Acked Write 2 Done Done Sent false
20592 5550 Acked Read 3 Done Done Sent false
20593 5551 Acked Write 9 Done Done Sent false
20594 5552 Acked Write 1 Done Done Sent false
20595 5553 Acked Read 2 Done Done Sent false
20596 5554 Acked Write 2 Done Done Sent false
20597 5555 Acked Read 6 Done Done Sent false
20598 5556 Acked Write 3 Done Done Sent false
20599 5557 Acked Write 5 Done Done Sent false
20600 5558 Acked Read 8 Done Done Sent false
20601 5559 Acked Write 6 Done Done Sent false
20602 5560 Acked Read 5 Done Done Sent false
20603 5561 Acked Write 7 Done Done Sent false
20604 5562 Acked Write 8 Done Done Sent false
20605 5563 Acked Read 7 Done Done Sent false
20606 5564 Acked Read 4 Done Done Sent false
20607 5565 Acked Flush 0 Done Done Sent false
20608 5566 Acked Read 1 Done Done Sent false
20609 5567 Acked Read 5 Done Done Sent false
20610 5568 Acked Read 8 Done Done Sent false
20611 5569 Acked Read 4 Done Done Sent false
20612 5570 Acked Write 5 Done Done Sent false
20613 5571 Acked Write 6 Done Done Sent false
20614 5572 Acked Read 1 Done Done Sent false
20615 5573 Acked Read 9 Done Done Sent false
20616 5574 Acked Read 1 Done Done Sent false
20617 5575 Acked Read 1 Done Done Sent false
20618 5576 Acked Flush 0 Done Done Sent false
20619 5577 Acked Read 6 Done Done Sent false
20620 5578 Acked Write 9 Done Done Sent false
20621 5579 Acked Flush 0 Done Done Sent false
20622 5580 Acked Read 7 Done Done Sent false
20623 5581 Acked Flush 0 Done Done Sent false
20624 5582 Acked Read 5 Done Done Sent false
20625 5583 Acked Read 9 Done Done Sent false
20626 5584 Acked Write 9 Done Done Sent false
20627 5585 Acked Read 8 Done Done Sent false
20628 5586 Acked Flush 0 Done Done Sent false
20629 5587 Acked Read 10 Done Done Sent false
20630 5588 Acked Flush 0 Done Done Sent false
20631 5589 Acked Read 2 Done Done Sent false
20632 5590 Acked Read 4 Done Done Sent false
20633 5591 Acked Read 10 Done Done Sent false
20634 5592 Acked Write 9 Done Done Sent false
20635 5593 Acked Read 8 Done Done Sent false
20636 5594 Acked Read 3 Done Done Sent false
20637 5595 Acked Read 5 Done Done Sent false
20638 5596 Acked Read 2 Done Done Sent false
20639 5597 Acked Write 7 Done Done Sent false
20640 5598 Acked Write 1 Done Done Sent false
20641 5599 Acked Flush 0 Done Done Sent false
20642 5600 Acked Read 1 Done Done Sent false
20643 5601 Acked Read 10 Done Done Sent false
20644 5602 Acked Write 8 Done Done Sent false
20645 5603 Acked Read 3 Done Done Sent false
20646 5604 Acked Write 3 Done Done Sent false
20647 5605 Acked Write 6 Done Done Sent false
20648 5606 Acked Read 6 Done Done Sent false
20649 5607 Acked Write 7 Done Done Sent false
20650 5608 Acked Write 7 Done Done Sent false
20651 5609 Acked Read 9 Done Done Sent false
20652 5610 Acked Read 8 Done Done Sent false
20653 5611 Acked Read 4 Done Done Sent false
20654 5612 Acked Read 1 Done Done Sent false
20655 5613 Acked Read 5 Done Done Sent false
20656 5614 Acked Write 3 Done Done Sent false
20657 5615 Acked Write 7 Done Done Sent false
20658 5616 Acked Write 1 Done Done Sent false
20659 5617 Acked Write 8 Done Done Sent false
20660 5618 Acked Flush 0 Done Done Sent false
20661 5619 Acked Read 9 Done Done Sent false
20662 5620 Acked Write 6 Done Done Sent false
20663 5621 Acked Write 3 Done Done Sent false
20664 5622 Acked Write 2 Done Done Sent false
20665 5623 Acked Flush 0 Done Done Sent false
20666 5624 Acked Read 2 Done Done Sent false
20667 5625 Acked Read 2 Done Done Sent false
20668 5626 Acked Read 8 Done Done Sent false
20669 5627 Acked Write 1 Done Done Sent false
20670 5628 Acked Write 8 Done Done Sent false
20671 5629 Acked Read 1 Done Done Sent false
20672 5630 Acked Write 9 Done Done Sent false
20673 5631 Acked Read 5 Done Done Sent false
20674 5632 Acked Read 3 Done Done Sent false
20675 5633 Acked Write 4 Done Done Sent false
20676 5634 Acked Read 10 Done Done Sent false
20677 5635 Acked Flush 0 Done Done Sent false
20678 5636 Acked Write 3 Done Done Sent false
20679 5637 Acked Read 8 Done Done Sent false
20680 5638 Acked Read 4 Done Done Sent false
20681 5639 Acked Write 3 Done Done Sent false
20682 5640 Acked Read 3 Done Done Sent false
20683 5641 Acked Read 4 Done Done Sent false
20684 5642 Acked Write 3 Done Done Sent false
20685 5643 Acked Write 8 Done Done Sent false
20686 5644 Acked Write 9 Done Done Sent false
20687 5645 Acked Read 8 Done Done Sent false
20688 5646 Acked Read 1 Done Done Sent false
20689 5647 Acked Read 10 Done Done Sent false
20690 5648 Acked Read 9 Done Done Sent false
20691 5649 Acked Flush 0 Done Done Sent false
20692 5650 Acked Read 6 Done Done Sent false
20693 5651 Acked Read 1 Done Done Sent false
20694 5652 Acked Read 3 Done Done Sent false
20695 5653 Acked Read 7 Done Done Sent false
20696 5654 Acked Write 9 Done Done Sent false
20697 5655 Acked Read 1 Done Done Sent false
20698 5656 Acked Write 5 Done Done Sent false
20699 5657 Acked Read 3 Done Done Sent false
20700 5658 Acked Write 1 Done Done Sent false
20701 5659 Acked Write 6 Done Done Sent false
20702 5660 Acked Read 1 Done Done Sent false
20703 5661 Acked Read 1 Done Done Sent false
20704 5662 Acked Read 1 Done Done Sent false
20705 5663 Acked Write 9 Done Done Sent false
20706 5664 Acked Read 9 Done Done Sent false
20707 5665 Acked Read 8 Done Done Sent false
20708 5666 Acked Flush 0 Done Done Sent false
20709 5667 Acked Flush 0 Done Done Sent false
20710 5668 Acked Read 8 Done Done Sent false
20711 5669 Acked Read 6 Done Done Sent false
20712 5670 Acked Write 4 Done Done Sent false
20713 5671 Acked Write 9 Done Done Sent false
20714 5672 Acked Read 7 Done Done Sent false
20715 5673 Acked Read 9 Done Done Sent false
20716 5674 Acked Write 8 Done Done Sent false
20717 5675 Acked Read 5 Done Done Sent false
20718 5676 Acked Read 2 Done Done Sent false
20719 5677 Acked Write 2 Done Done Sent false
20720 5678 Acked Read 10 Done Done Sent false
20721 5679 Acked Write 5 Done Done Sent false
20722 5680 Acked Write 6 Done Done Sent false
20723 5681 Acked Write 6 Done Done Sent false
20724 5682 Acked Read 3 Done Done Sent false
20725 5683 Acked Read 4 Done Done Sent false
20726 5684 Acked Write 7 Done Done Sent false
20727 5685 Acked Write 8 Done Done Sent false
20728 5686 Acked Read 2 Done Done Sent false
20729 5687 Acked Write 8 Done Done Sent false
20730 5688 Acked Write 5 Done Done Sent false
20731 5689 Acked Write 4 Done Done Sent false
20732 5690 Acked Write 5 Done Done Sent false
20733 5691 Acked Write 2 Done Done Sent false
20734 5692 Acked Flush 0 Done Done Sent false
20735 5693 Acked Read 6 Done Done Sent false
20736 5694 Acked Read 3 Done Done Sent false
20737 5695 Acked Read 4 Done Done Sent false
20738 5696 Acked Write 6 Done Done Sent false
20739 5697 Acked Read 6 Done Done Sent false
20740 5698 Acked Read 1 Done Done Sent false
20741 5699 Acked Flush 0 Done Done Sent false
20742 5700 Acked Read 8 Done Done Sent false
20743 5701 Acked Read 8 Done Done Sent false
20744 5702 Acked Flush 0 Done Done Sent false
20745 5703 Acked Flush 0 Done Done Sent false
20746 5704 Acked Read 8 Done Done Sent false
20747 5705 Acked Write 10 Done Done Sent false
20748 5706 Acked Write 2 Done Done Sent false
20749 5707 Acked Flush 0 Done Done Sent false
20750 5708 Acked Read 2 Done Done Sent false
20751 5709 Acked Write 2 Done Done Sent false
20752 5710 Acked Write 2 Done Done Sent false
20753 5711 Acked Read 3 Done Done Sent false
20754 5712 Acked Read 8 Done Done Sent false
20755 5713 Acked Flush 0 Done Done Sent false
20756 5714 Acked Write 7 Done Done Sent false
20757 5715 Acked Write 1 Done Done Sent false
20758 5716 Acked Read 7 Done Done Sent false
20759 5717 Acked Flush 0 Done Done Sent false
20760 5718 Acked Write 10 Done Done Sent false
20761 5719 Acked Read 4 Done Done Sent false
20762 5720 Acked Read 6 Done Done Sent false
20763 5721 Acked Read 6 Done Done Sent false
20764 5722 Acked Read 2 Done Done Sent false
20765 5723 Acked Write 10 Done Done Sent false
20766 5724 Acked Write 10 Done Done Sent false
20767 5725 Acked Read 9 Done Done Sent false
20768 5726 Acked Write 7 Done Done Sent false
20769 5727 Acked Read 9 Done Done Sent false
20770 5728 Acked Read 7 Done Done Sent false
20771 5729 Acked Read 3 Done Done Sent false
20772 5730 Acked Read 10 Done Done Sent false
20773 5731 Acked Read 3 Done Done Sent false
20774 5732 Acked Read 1 Done Done Sent false
20775 5733 Acked Write 3 Done Done Sent false
20776 5734 Acked Write 3 Done Done Sent false
20777 5735 Acked Read 8 Done Done Sent false
20778 5736 Acked Read 2 Done Done Sent false
20779 5737 Acked Write 1 Done Done Sent false
20780 5738 Acked Write 8 Done Done Sent false
20781 5739 Acked Write 9 Done Done Sent false
20782 5740 Acked Write 6 Done Done Sent false
20783 5741 Acked Flush 0 Done Done Sent false
20784 5742 Acked Write 9 Done Done Sent false
20785 5743 Acked Flush 0 Done Done Sent false
20786 5744 Acked Read 1 Done Done Sent false
20787 5745 Acked Write 6 Done Done Sent false
20788 5746 Acked Read 1 Done Done Sent false
20789 5747 Acked Write 4 Done Done Sent false
20790 5748 Acked Write 9 Done Done Sent false
20791 5749 Acked Read 3 Done Done Sent false
20792 5750 Acked Read 2 Done Done Sent false
20793 5751 Acked Read 10 Done Done Sent false
20794 5752 Acked Read 7 Done Done Sent false
20795 5753 Acked Write 3 Done Done Sent false
20796 5754 Acked Write 9 Done Done Sent false
20797 5755 Acked Write 5 Done Done Sent false
20798 5756 Acked Read 5 Done Done Sent false
20799 5757 Acked Write 7 Done Done Sent false
20800 5758 Acked Write 10 Done Done Sent false
20801 5759 Acked Read 2 Done Done Sent false
20802 5760 Acked Write 4 Done Done Sent false
20803 5761 Acked Write 2 Done Done Sent false
20804 5762 Acked Write 4 Done Done Sent false
20805 5763 Acked Flush 0 Done Done Sent false
20806 5764 Acked Read 6 Done Done Sent false
20807 5765 Acked Read 7 Done Done Sent false
20808 5766 Acked Read 3 Done Done Sent false
20809 5767 Acked Write 7 Done Done Sent false
20810 5768 Acked Read 3 Done Done Sent false
20811 5769 Acked Write 2 Done Done Sent false
20812 5770 Acked Write 1 Done Done Sent false
20813 5771 Acked Write 2 Done Done Sent false
20814 5772 Acked Flush 0 Done Done Sent false
20815 5773 Acked Write 2 Done Done Sent false
20816 5774 Acked Write 8 Done Done Sent false
20817 5775 Acked Flush 0 Done Done Sent false
20818 5776 Acked Read 5 Done Done Sent false
20819 5777 Acked Write 7 Done Done Sent false
20820 5778 Acked Read 3 Done Done Sent false
20821 5779 Acked Write 5 Done Done Sent false
20822 5780 Acked Write 2 Done Done Sent false
20823 5781 Acked Read 5 Done Done Sent false
20824 5782 Acked Flush 0 Done Done Sent false
20825 5783 Acked Read 5 Done Done Sent false
20826 5784 Acked Write 10 Done Done Sent false
20827 5785 Acked Flush 0 Done Done Sent false
20828 5786 Acked Write 2 Done Done Sent false
20829 5787 Acked Read 2 Done Done Sent false
20830 5788 Acked Read 3 Done Done Sent false
20831 5789 Acked Read 8 Done Done Sent false
20832 5790 Acked Write 3 Done Done Sent false
20833 5791 Acked Write 6 Done Done Sent false
20834 5792 Acked Read 9 Done Done Sent false
20835 5793 Acked Read 3 Done Done Sent false
20836 5794 Acked Read 6 Done Done Sent false
20837 5795 Acked Write 8 Done Done Sent false
20838 5796 Acked Read 8 Done Done Sent false
20839 5797 Acked Write 4 Done Done Sent false
20840 5798 Acked Read 4 Done Done Sent false
20841 5799 Acked Read 7 Done Done Sent false
20842 5800 Acked Write 8 Done Done Sent false
20843 5801 Acked Read 5 Done Done Sent false
20844 5802 Acked Write 3 Done Done Sent false
20845 5803 Acked Read 8 Done Done Sent false
20846 5804 Acked Write 5 Done Done Sent false
20847 5805 Acked Read 1 Done Done Sent false
20848 5806 Acked Read 2 Done Done Sent false
20849 5807 Acked Read 9 Done Done Sent false
20850 5808 Acked Read 1 Done Done Sent false
20851 5809 Acked Read 3 Done Done Sent false
20852 5810 Acked Read 7 Done Done Sent false
20853 5811 Acked Read 8 Done Done Sent false
20854 5812 Acked Read 6 Done Done Sent false
20855 5813 Acked Read 8 Done Done Sent false
20856 5814 Acked Flush 0 Done Done Sent false
20857 5815 Acked Read 10 Done Done Sent false
20858 5816 Acked Read 3 Done Done Sent false
20859 5817 Acked Read 2 Done Done Sent false
20860 5818 Acked Write 10 Done Done Sent false
20861 5819 Acked Read 6 Done Done Sent false
20862 5820 Acked Write 3 Done Done Sent false
20863 5821 Acked Read 8 Done Done Sent false
20864 5822 Acked Write 1 Done Done Sent false
20865 5823 Acked Write 10 Done Done Sent false
20866 5824 Acked Read 5 Done Done Sent false
20867 5825 Acked Write 5 Done Done Sent false
20868 5826 Acked Read 4 Done Done Sent false
20869 5827 Acked Read 6 Done Done Sent false
20870 5828 Acked Read 8 Done Done Sent false
20871 5829 Acked Flush 0 Done Done Sent false
20872 STATES DS:0 DS:1 DS:2 TOTAL
20873 Sent 0 0 302 302
20874 Done 302 302 0 604
20875 Skipped 0 0 0 0
20876 Error 0 0 0 0
20877 Last Flush: 5829 5829 5527
20878 Downstairs last five completed: 5527 5526 5525 5524 5523
20879 Upstairs last five completed: 5829 5828 5827 5826 5825
20880 CLIENT: Up:0 ds:302 act:2
208812024-11-15T02:01:51.585ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
208822024-11-15T02:01:51.586ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:61296) = io task client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
208832024-11-15T02:01:51.587ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
208842024-11-15T02:01:51.588ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
208852024-11-15T02:01:51.588ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
208862024-11-15T02:01:51.588ZINFOcrucible: send last flush ID to this DS: 5527 = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
208872024-11-15T02:01:51.588ZINFOcrucible: Replied this last flush ID: 5527 = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
208882024-11-15T02:01:51.588ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
208892024-11-15T02:01:51.588ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
208902024-11-15T02:01:51.588ZINFOcrucible: [2] client re-new 302 jobs since flush 5527 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
208912024-11-15T02:01:51.588ZINFOcrucible: [2] Replayed 302 jobs since flush: 5527 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
20892 ----------------------------------------------------------------
20893 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
20894 Downstairs last five completed: 5829 5828 5827 5826 5825
20895 Upstairs last five completed: 5829 5828 5827 5826 5825
20896 CLIENT: Up:0 ds:0 act:3
20897 Replay: All jobs finished
20898 Main task has work to do, go find it
20899 got dsc Stop(2)
20900 stop 2
20901 [8830] Got stop action so:false kr:false
20902 Nov 15 02:01:51.831 INFO request completed, latency_us: 329, response_code: 204, uri: /stop/cid/2, method: POST, req_id: e7385ad9-e4d3-4381-bbff-c12343781ea9, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
209032024-11-15T02:01:51.832ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
209042024-11-15T02:01:51.832ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
209052024-11-15T02:01:51.832ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
209062024-11-15T02:01:51.832ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
209072024-11-15T02:01:51.832ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
20908 Nov 15 02:01:51.832 INFO request completed{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2024-11-15T02:01:51.832751791Z","hostname":"w-01JCPRHT5F2MPA44KVJHM3S2PB","pid":1205,"":"io task","client":"2","":"downstairs","session_id":"971dcb2e-6891-4ecf-b367-1e04e52065ee"}
20909 , latency_us: 110, response_code: 200, uri: /state/cid/2, method: GET, req_id: 8487ea2d-0cd4-41b4-85b8-518050baff94, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
20910 [8830] Exited with: ExitStatus(unix_wait_status(9))
20911 [8830][2] reports Exit
20912 Nov 15 02:01:55.834 INFO request completed, latency_us: 192, response_code: 200, uri: /state/cid/2, method: GET, req_id: 415f779e-8197-40a0-b709-4dcb475eff0c, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
20913 001/300 Write block 3569 len 40960 data: 5 4 4 4 4 4 5 5 4 5
20914 002/300 Write block 2454 len 16384 data: 8 8 7 7
20915 003/300 Write block 532 len 40960 data: 4 4 5 5 3 3 3 3 3 3
20916 004/300 Write block 2594 len 24576 data: 5 5 6 6 6 6
20917 005/300 Read block 1243 len 36864
20918 006/300 Write block 4070 len 36864 data: 5 4 2 4 4 5 5 5 4
20919 007/300 Write block 1081 len 28672 data: 2 2 3 3 3 3 4
20920 008/300 Write block 437 len 4096 data: 2
20921 009/300 Read block 1245 len 28672
20922 010/300 Read block 618 len 40960
20923 011/300 Write block 4901 len 20480 data: 5 7 7 7 5
20924 012/300 Read block 4687 len 20480
20925 013/300 Read block 3644 len 20480
20926 014/300 Read block 3149 len 16384
20927 015/300 Read block 4959 len 20480
20928 016/300 Read block 4029 len 24576
20929 017/300 Write block 250 len 4096 data: 5
20930 018/300 Read block 2935 len 4096
20931 019/300 Write block 3147 len 32768 data: 4 4 4 5 5 6 4 4
20932 020/300 Write block 533 len 24576 data: 5 6 6 4 4 4
20933 021/300 Read block 937 len 36864
20934 022/300 Flush
20935 023/300 Read block 156 len 32768
20936 024/300 Write block 352 len 16384 data: 2 3 3 4
20937 025/300 Read block 7 len 28672
20938 026/300 Read block 4264 len 16384
20939 027/300 Read block 1984 len 12288
20940 028/300 Flush
20941 029/300 Read block 3292 len 24576
20942 030/300 Flush
20943 031/300 Write block 1597 len 28672 data: 4 4 4 4 3 4 4
20944 032/300 Read block 115 len 24576
20945 033/300 Write block 1276 len 40960 data: 5 5 5 4 5 5 6 6 4 4
20946 034/300 Read block 433 len 24576
20947 035/300 Read block 3196 len 36864
20948 036/300 Read block 311 len 20480
20949 037/300 Write block 2766 len 12288 data: 4 4 4
20950 038/300 Write block 4884 len 24576 data: 3 3 4 4 3 4
20951 039/300 Write block 1893 len 20480 data: 6 6 5 5 6
20952 040/300 Read block 612 len 24576
20953 041/300 Flush
20954 042/300 Flush
20955 043/300 Read block 3083 len 20480
20956 044/300 Write block 4463 len 12288 data: 3 3 3
20957 045/300 Read block 3008 len 24576
20958 046/300 Read block 4347 len 20480
20959 047/300 Read block 1354 len 8192
20960 048/300 Read block 3700 len 4096
20961 049/300 Read block 3308 len 32768
20962 050/300 Write block 3058 len 12288 data: 3 3 3
20963 051/300 Write block 2722 len 8192 data: 3 3
20964 052/300 Write block 1951 len 28672 data: 3 3 4 5 5 5 6
20965 053/300 Read block 4096 len 12288
20966 054/300 Write block 1692 len 32768 data: 6 6 4 4 4 4 6 6
20967 055/300 Read block 1594 len 40960
20968 056/300 Flush
20969 057/300 Read block 760 len 16384
20970 058/300 Write block 3697 len 28672 data: 5 6 6 5 5 5 4
20971 059/300 Write block 2220 len 20480 data: 6 6 3 5 5
20972 060/300 Write block 2710 len 28672 data: 2 2 2 2 3 2 2
20973 061/300 Flush
20974 062/300 Write block 1634 len 24576 data: 5 4 4 4 6 6
20975 063/300 Flush
20976 064/300 Write block 1207 len 16384 data: 2 3 3 4
20977 065/300 Read block 2656 len 24576
20978 066/300 Read block 643 len 24576
20979 067/300 Read block 2044 len 16384
20980 068/300 Read block 2328 len 28672
20981 069/300 Read block 301 len 36864
20982 070/300 Read block 2644 len 24576
20983 071/300 Write block 1182 len 32768 data: 2 2 2 2 2 5 4 3
20984 072/300 Read block 1711 len 8192
20985 073/300 Write block 38 len 24576 data: 5 3 4 4 4 5
20986 074/300 Read block 3449 len 16384
20987 075/300 Flush
20988 076/300 Write block 2885 len 36864 data: 2 4 4 5 5 5 4 4 4
20989 077/300 Read block 74 len 8192
20990 078/300 Write block 20 len 32768 data: 4 3 3 4 4 4 5 5
20991 079/300 Flush
20992 080/300 Read block 967 len 16384
20993 081/300 Read block 1583 len 32768
20994 082/300 Read block 4124 len 12288
20995 083/300 Write block 1325 len 4096 data: 4
20996 084/300 Write block 1990 len 36864 data: 3 3 2 2 2 2 2 3 3
20997 085/300 Flush
20998 086/300 Read block 3743 len 24576
20999 087/300 Write block 490 len 40960 data: 4 4 3 3 3 3 3 4 4 4
21000 088/300 Read block 529 len 40960
21001 089/300 Read block 364 len 24576
21002 090/300 Read block 1002 len 16384
21003 091/300 Write block 4444 len 20480 data: 5 5 5 4 4
21004 092/300 Read block 3472 len 40960
21005 093/300 Write block 2527 len 24576 data: 6 7 7 6 7 7
21006 094/300 Write block 757 len 8192 data: 5 7
21007 095/300 Write block 3489 len 32768 data: 5 5 4 3 2 2 3 3
21008 096/300 Write block 507 len 28672 data: 4 4 3 3 3 3 4
21009 097/300 Read block 736 len 28672
21010 098/300 Write block 513 len 36864 data: 5 3 2 2 2 2 2 2 2
21011 099/300 Write block 2982 len 8192 data: 6 7
21012 100/300 Read block 4776 len 4096
21013 101/300 Read block 3573 len 12288
21014 102/300 Write block 2852 len 20480 data: 4 4 4 3 4
21015 103/300 Write block 3248 len 8192 data: 7 7
21016 104/300 Write block 795 len 20480 data: 7 6 6 6 4
21017 105/300 Write block 4888 len 8192 data: 4 5
21018 106/300 Read block 1413 len 12288
21019 107/300 Write block 2753 len 12288 data: 4 4 4
21020 108/300 Read block 227 len 28672
21021 109/300 Flush
21022 110/300 Write block 2039 len 16384 data: 3 2 2 2
21023 111/300 Write block 2264 len 12288 data: 4 4 4
21024 112/300 Read block 2516 len 40960
21025 113/300 Read block 3905 len 16384
21026 114/300 Read block 1368 len 24576
21027 115/300 Read block 2503 len 12288
21028 116/300 Write block 2408 len 32768 data: 3 3 2 2 2 2 2 2
21029 117/300 Flush
21030 118/300 Flush
21031 119/300 Write block 228 len 8192 data: 5 7
21032 120/300 Read block 443 len 20480
21033 121/300 Read block 3680 len 20480
21034 122/300 Read block 3519 len 36864
21035 123/300 Write block 4003 len 12288 data: 3 5 5
21036 124/300 Flush
21037 125/300 Read block 2507 len 36864
21038 126/300 Write block 4543 len 8192 data: 4 3
21039 127/300 Write block 4211 len 4096 data: 3
21040 128/300 Read block 1613 len 12288
21041 129/300 Write block 1758 len 8192 data: 2 3
21042 130/300 Write block 3989 len 20480 data: 5 5 4 3 3
21043 131/300 Write block 200 len 16384 data: 4 4 3 4
21044 132/300 Write block 2659 len 32768 data: 5 5 4 3 3 3 2 2
21045 133/300 Read block 4778 len 36864
21046 134/300 Write block 995 len 8192 data: 7 7
21047 135/300 Read block 2459 len 28672
21048 136/300 Read block 1774 len 4096
21049 137/300 Read block 3616 len 12288
21050 138/300 Write block 2435 len 32768 data: 3 2 2 2 2 2 4 3
21051 139/300 Write block 1322 len 8192 data: 3 3
21052 140/300 Read block 327 len 28672
21053 141/300 Read block 198 len 20480
21054 142/300 Write block 4694 len 12288 data: 4 3 4
21055 143/300 Write block 3141 len 20480 data: 4 4 4 4 4
21056 144/300 Read block 1387 len 40960
21057 145/300 Read block 9 len 4096
21058 146/300 Read block 4303 len 24576
21059 147/300 Write block 2525 len 40960 data: 5 4 7 8 8 7 8 8 5 4
21060 148/300 Write block 2878 len 24576 data: 3 2 2 2 2 2
21061 149/300 Write block 2902 len 16384 data: 3 3 3 4
21062 150/300 Write block 2528 len 36864 data: 9 9 8 9 9 6 5 4 4
21063 151/300 Read block 3372 len 32768
21064 152/300 Read block 2115 len 24576
21065 153/300 Write block 665 len 28672 data: 4 4 6 5 4 6 5
21066 154/300 Read block 4790 len 40960
21067 155/300 Read block 3114 len 24576
21068 156/300 Write block 648 len 32768 data: 3 4 4 6 6 7 6 5
21069 157/300 Write block 4592 len 20480 data: 6 6 4 3 3
21070 158/300 Read block 4457 len 40960
21071 159/300 Read block 1196 len 12288
21072 160/300 Write block 2120 len 20480 data: 2 2 2 2 2
21073 161/300 Read block 2354 len 36864
21074 162/300 Write block 4173 len 4096 data: 5
21075 163/300 Write block 4965 len 24576 data: 3 4 2 2 2 3
21076 164/300 Write block 1790 len 28672 data: 4 3 4 4 4 3 3
21077 165/300 Read block 4023 len 40960
21078 166/300 Read block 1205 len 28672
21079 167/300 Flush
21080 168/300 Read block 2751 len 12288
21081 169/300 Read block 1006 len 24576
21082 170/300 Read block 4104 len 24576
21083 171/300 Flush
21084 172/300 Write block 3081 len 4096 data: 9
21085 173/300 Write block 986 len 36864 data: 3 3 3 4 4 7 7 7 7
21086 174/300 Read block 3539 len 12288
21087 175/300 Write block 873 len 36864 data: 4 2 2 2 4 4 4 4 4
21088 176/300 Read block 1912 len 24576
21089 177/300 Read block 190 len 20480
21090 178/300 Read block 2175 len 8192
21091 179/300 Read block 4799 len 16384
21092 180/300 Read block 4781 len 24576
21093 181/300 Read block 3948 len 32768
21094 182/300 Read block 3019 len 12288
21095 183/300 Write block 2077 len 12288 data: 4 5 5
21096 184/300 Write block 877 len 8192 data: 5 5
21097 185/300 Write block 3012 len 16384 data: 4 4 4 4
21098 186/300 Write block 1906 len 4096 data: 4
21099 187/300 Read block 2290 len 12288
21100 188/300 Read block 2016 len 40960
21101 189/300 Read block 3063 len 4096
21102 190/300 Read block 1108 len 4096
21103 191/300 Read block 2071 len 8192
21104 192/300 Write block 3219 len 4096 data: 6
21105 193/300 Flush
21106 194/300 Write block 144 len 4096 data: 2
21107 195/300 Flush
21108 196/300 Write block 3214 len 20480 data: 2 2 2 2 4
21109 197/300 Flush
21110 198/300 Read block 354 len 16384
21111 199/300 Read block 1617 len 40960
21112 200/300 Read block 3673 len 24576
21113 201/300 Read block 4389 len 28672
21114 202/300 Write block 4966 len 28672 data: 5 3 3 3 4 2 2
21115 203/300 Read block 4242 len 4096
21116 204/300 Read block 180 len 8192
21117 205/300 Write block 4884 len 16384 data: 4 4 5 5
21118 206/300 Read block 3529 len 24576
21119 207/300 Read block 4582 len 16384
21120 208/300 Read block 4562 len 4096
21121 209/300 Write block 2372 len 20480 data: 3 3 4 3 2
21122 210/300 Write block 133 len 40960 data: 3 3 4 4 3 3 3 3 3 2
21123 211/300 Write block 3050 len 40960 data: 5 5 3 3 3 3 4 5 4 4
21124 212/300 Read block 1580 len 24576
21125 213/300 Write block 2275 len 16384 data: 4 4 3 4
21126 214/300 Read block 599 len 20480
21127 215/300 Read block 1041 len 12288
21128 216/300 Read block 4737 len 36864
21129 217/300 Write block 3220 len 28672 data: 6 4 4 3 3 3 3
21130 218/300 Write block 4701 len 40960 data: 6 6 7 7 7 8 6 6 6 6
21131 219/300 Read block 273 len 32768
21132 220/300 Read block 2478 len 20480
21133 221/300 Write block 2170 len 16384 data: 5 5 5 5
21134 222/300 Write block 4899 len 12288 data: 3 4 6
21135 223/300 Write block 2015 len 16384 data: 3 3 3 4
21136 224/300 Flush
21137 225/300 Read block 1079 len 24576
21138 226/300 Write block 1998 len 24576 data: 4 2 3 4 4 5
21139 227/300 Read block 2080 len 8192
21140 228/300 Read block 1854 len 32768
21141 229/300 Write block 170 len 4096 data: 5
21142 230/300 Write block 4205 len 8192 data: 7 8
21143 231/300 Read block 462 len 8192
21144 232/300 Write block 505 len 20480 data: 6 5 5 5 4
21145 233/300 Write block 873 len 24576 data: 5 3 3 3 6 6
21146 234/300 Read block 3202 len 24576
21147 235/300 Write block 4037 len 40960 data: 8 9 9 8 7 6 6 5 6 6
21148 236/300 Read block 2696 len 32768
21149 237/300 Flush
21150 238/300 Write block 69 len 28672 data: 5 4 5 5 4 4 4
21151 239/300 Read block 230 len 36864
21152 240/300 Read block 4041 len 24576
21153 241/300 Read block 3371 len 36864
21154 242/300 Flush
21155 243/300 Read block 1283 len 20480
21156 244/300 Write block 3591 len 32768 data: 5 4 4 5 6 5 4 4
21157 245/300 Write block 785 len 12288 data: 7 7 7
21158 246/300 Write block 4017 len 32768 data: 3 4 3 4 5 5 4 4
21159 247/300 Read block 605 len 28672
21160 248/300 Read block 1601 len 12288
21161 249/300 Flush
21162 250/300 Read block 2052 len 28672
21163 251/300 Read block 260 len 36864
21164 252/300 Write block 554 len 16384 data: 2 2 4 5
21165 253/300 Read block 4699 len 32768
21166 254/300 Read block 3936 len 12288
21167 255/300 Write block 1631 len 40960 data: 5 5 5 6 5 5 5 7 7 5
21168 256/300 Write block 959 len 8192 data: 2 2
21169 257/300 Write block 2024 len 20480 data: 3 4 4 3 2
21170 258/300 Read block 655 len 4096
21171 259/300 Read block 2912 len 4096
21172 260/300 Read block 4751 len 16384
21173 261/300 Write block 418 len 8192 data: 5 6
21174 262/300 Write block 4776 len 12288 data: 6 5 7
21175 263/300 Read block 3984 len 4096
21176 264/300 Read block 3561 len 28672
21177 265/300 Write block 2869 len 32768 data: 4 4 4 4 4 3 3 2
21178 266/300 Write block 2886 len 12288 data: 5 5 6
21179 267/300 Read block 888 len 36864
21180 268/300 Read block 1116 len 4096
21181 269/300 Write block 2474 len 20480 data: 10 10 8 7 6
21182 270/300 Write block 3392 len 20480 data: 4 4 4 5 4
21183 271/300 Write block 3925 len 32768 data: 5 5 5 4 3 2 2 3
21184 272/300 Read block 2478 len 40960
21185 273/300 Write block 2006 len 8192 data: 4 3
21186 274/300 Flush
21187 275/300 Write block 3343 len 40960 data: 4 3 3 3 4 5 6 7 7 8
21188 276/300 Write block 54 len 28672 data: 4 3 3 5 6 5 5
21189 277/300 Write block 1769 len 40960 data: 5 3 3 3 3 3 2 5 5 6
21190 278/300 Read block 304 len 40960
21191 279/300 Read block 2244 len 4096
21192 280/300 Write block 3333 len 20480 data: 3 3 4 3 3
21193 281/300 Read block 3323 len 40960
21194 282/300 Read block 2327 len 36864
21195 283/300 Flush
21196 284/300 Write block 2332 len 24576 data: 3 3 2 2 5 5
21197 285/300 Read block 2014 len 40960
21198 286/300 Read block 4797 len 40960
21199 287/300 Write block 557 len 28672 data: 6 4 4 4 4 4 3
21200 288/300 Write block 1417 len 12288 data: 3 3 3
21201 289/300 Read block 2668 len 4096
21202 290/300 Write block 902 len 32768 data: 4 4 4 5 6 5 5 6
21203 291/300 Write block 4656 len 32768 data: 3 4 5 5 6 5 7 6
21204 292/300 Flush
21205 293/300 Write block 3657 len 28672 data: 5 5 5 5 5 5 4
21206 294/300 Read block 1456 len 12288
21207 295/300 Flush
21208 296/300 Write block 3448 len 20480 data: 4 3 3 3 3
21209 297/300 Read block 4120 len 40960
21210 298/300 Flush
21211 299/300 Write block 3429 len 16384 data: 2 2 2 2
21212 300/300 Read block 352 len 12288
21213 301/300 Write block 779 len 28672 data: 3 3 5 5 7 7 8
21214 Main task has work to do, go find it
21215 got dsc Start(2)
21216 start 2
21217 Nov 15 02:01:56.068 INFO request completed, latency_us: 214, response_code: 204, uri: /start/cid/2, method: POST, req_id: c9951d6c-6232-4d7d-b437-98b628840a9a, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
21218 Replay: started 2, returned:Ok(())
21219 ----------------------------------------------------------------
21220 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
21221 States: Active Active Offline
21222 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
21223 5830 Acked Write 10 Done Done Sent false
21224 5831 Acked Write 4 Done Done Sent false
21225 5832 Acked Write 10 Done Done Sent false
21226 5833 Acked Write 6 Done Done Sent false
21227 5834 Acked Read 9 Done Done Sent false
21228 5835 Acked Write 9 Done Done Sent false
21229 5836 Acked Write 7 Done Done Sent false
21230 5837 Acked Write 1 Done Done Sent false
21231 5838 Acked Read 7 Done Done Sent false
21232 5839 Acked Read 10 Done Done Sent false
21233 5840 Acked Write 5 Done Done Sent false
21234 5841 Acked Read 5 Done Done Sent false
21235 5842 Acked Read 5 Done Done Sent false
21236 5843 Acked Read 4 Done Done Sent false
21237 5844 Acked Read 5 Done Done Sent false
21238 5845 Acked Read 6 Done Done Sent false
21239 5846 Acked Write 1 Done Done Sent false
21240 5847 Acked Read 1 Done Done Sent false
21241 5848 Acked Write 8 Done Done Sent false
21242 5849 Acked Write 6 Done Done Sent false
21243 5850 Acked Read 9 Done Done Sent false
21244 5851 Acked Flush 0 Done Done Sent false
21245 5852 Acked Read 8 Done Done Sent false
21246 5853 Acked Write 4 Done Done Sent false
21247 5854 Acked Read 7 Done Done Sent false
21248 5855 Acked Read 4 Done Done Sent false
21249 5856 Acked Read 3 Done Done Sent false
21250 5857 Acked Flush 0 Done Done Sent false
21251 5858 Acked Read 6 Done Done Sent false
21252 5859 Acked Flush 0 Done Done Sent false
21253 5860 Acked Write 7 Done Done Sent false
21254 5861 Acked Read 6 Done Done Sent false
21255 5862 Acked Write 10 Done Done Sent false
21256 5863 Acked Read 6 Done Done Sent false
21257 5864 Acked Read 9 Done Done Sent false
21258 5865 Acked Read 5 Done Done Sent false
21259 5866 Acked Write 3 Done Done Sent false
21260 5867 Acked Write 6 Done Done Sent false
21261 5868 Acked Write 5 Done Done Sent false
21262 5869 Acked Read 6 Done Done Sent false
21263 5870 Acked Flush 0 Done Done Sent false
21264 5871 Acked Flush 0 Done Done Sent false
21265 5872 Acked Read 5 Done Done Sent false
21266 5873 Acked Write 3 Done Done Sent false
21267 5874 Acked Read 6 Done Done Sent false
21268 5875 Acked Read 5 Done Done Sent false
21269 5876 Acked Read 2 Done Done Sent false
21270 5877 Acked Read 1 Done Done Sent false
21271 5878 Acked Read 8 Done Done Sent false
21272 5879 Acked Write 3 Done Done Sent false
21273 5880 Acked Write 2 Done Done Sent false
21274 5881 Acked Write 7 Done Done Sent false
21275 5882 Acked Read 3 Done Done Sent false
21276 5883 Acked Write 8 Done Done Sent false
21277 5884 Acked Read 10 Done Done Sent false
21278 5885 Acked Flush 0 Done Done Sent false
21279 5886 Acked Read 4 Done Done Sent false
21280 5887 Acked Write 7 Done Done Sent false
21281 5888 Acked Write 5 Done Done Sent false
21282 5889 Acked Write 7 Done Done Sent false
21283 5890 Acked Flush 0 Done Done Sent false
21284 5891 Acked Write 6 Done Done Sent false
21285 5892 Acked Flush 0 Done Done Sent false
21286 5893 Acked Write 4 Done Done Sent false
21287 5894 Acked Read 6 Done Done Sent false
21288 5895 Acked Read 6 Done Done Sent false
21289 5896 Acked Read 4 Done Done Sent false
21290 5897 Acked Read 7 Done Done Sent false
21291 5898 Acked Read 9 Done Done Sent false
21292 5899 Acked Read 6 Done Done Sent false
21293 5900 Acked Write 8 Done Done Sent false
21294 5901 Acked Read 2 Done Done Sent false
21295 5902 Acked Write 6 Done Done Sent false
21296 5903 Acked Read 4 Done Done Sent false
21297 5904 Acked Flush 0 Done Done Sent false
21298 5905 Acked Write 9 Done Done Sent false
21299 5906 Acked Read 2 Done Done Sent false
21300 5907 Acked Write 8 Done Done Sent false
21301 5908 Acked Flush 0 Done Done Sent false
21302 5909 Acked Read 4 Done Done Sent false
21303 5910 Acked Read 8 Done Done Sent false
21304 5911 Acked Read 3 Done Done Sent false
21305 5912 Acked Write 1 Done Done Sent false
21306 5913 Acked Write 9 Done Done Sent false
21307 5914 Acked Flush 0 Done Done Sent false
21308 5915 Acked Read 6 Done Done Sent false
21309 5916 Acked Write 10 Done Done Sent false
21310 5917 Acked Read 10 Done Done Sent false
21311 5918 Acked Read 6 Done Done Sent false
21312 5919 Acked Read 4 Done Done Sent false
21313 5920 Acked Write 5 Done Done Sent false
21314 5921 Acked Read 10 Done Done Sent false
21315 5922 Acked Write 6 Done Done Sent false
21316 5923 Acked Write 2 Done Done Sent false
21317 5924 Acked Write 8 Done Done Sent false
21318 5925 Acked Write 7 Done Done Sent false
21319 5926 Acked Read 7 Done Done Sent false
21320 5927 Acked Write 9 Done Done Sent false
21321 5928 Acked Write 2 Done Done Sent false
21322 5929 Acked Read 1 Done Done Sent false
21323 5930 Acked Read 3 Done Done Sent false
21324 5931 Acked Write 5 Done Done Sent false
21325 5932 Acked Write 2 Done Done Sent false
21326 5933 Acked Write 5 Done Done Sent false
21327 5934 Acked Write 2 Done Done Sent false
21328 5935 Acked Read 3 Done Done Sent false
21329 5936 Acked Write 3 Done Done Sent false
21330 5937 Acked Read 7 Done Done Sent false
21331 5938 Acked Flush 0 Done Done Sent false
21332 5939 Acked Write 4 Done Done Sent false
21333 5940 Acked Write 3 Done Done Sent false
21334 5941 Acked Read 10 Done Done Sent false
21335 5942 Acked Read 4 Done Done Sent false
21336 5943 Acked Read 6 Done Done Sent false
21337 5944 Acked Read 3 Done Done Sent false
21338 5945 Acked Write 8 Done Done Sent false
21339 5946 Acked Flush 0 Done Done Sent false
21340 5947 Acked Flush 0 Done Done Sent false
21341 5948 Acked Write 2 Done Done Sent false
21342 5949 Acked Read 5 Done Done Sent false
21343 5950 Acked Read 5 Done Done Sent false
21344 5951 Acked Read 9 Done Done Sent false
21345 5952 Acked Write 3 Done Done Sent false
21346 5953 Acked Flush 0 Done Done Sent false
21347 5954 Acked Read 9 Done Done Sent false
21348 5955 Acked Write 2 Done Done Sent false
21349 5956 Acked Write 1 Done Done Sent false
21350 5957 Acked Read 3 Done Done Sent false
21351 5958 Acked Write 2 Done Done Sent false
21352 5959 Acked Write 5 Done Done Sent false
21353 5960 Acked Write 4 Done Done Sent false
21354 5961 Acked Write 8 Done Done Sent false
21355 5962 Acked Read 9 Done Done Sent false
21356 5963 Acked Write 2 Done Done Sent false
21357 5964 Acked Read 7 Done Done Sent false
21358 5965 Acked Read 1 Done Done Sent false
21359 5966 Acked Read 3 Done Done Sent false
21360 5967 Acked Write 8 Done Done Sent false
21361 5968 Acked Write 2 Done Done Sent false
21362 5969 Acked Read 7 Done Done Sent false
21363 5970 Acked Read 5 Done Done Sent false
21364 5971 Acked Write 3 Done Done Sent false
21365 5972 Acked Write 5 Done Done Sent false
21366 5973 Acked Read 10 Done Done Sent false
21367 5974 Acked Read 1 Done Done Sent false
21368 5975 Acked Read 6 Done Done Sent false
21369 5976 Acked Write 10 Done Done Sent false
21370 5977 Acked Write 6 Done Done Sent false
21371 5978 Acked Write 4 Done Done Sent false
21372 5979 Acked Write 9 Done Done Sent false
21373 5980 Acked Read 8 Done Done Sent false
21374 5981 Acked Read 6 Done Done Sent false
21375 5982 Acked Write 7 Done Done Sent false
21376 5983 Acked Read 10 Done Done Sent false
21377 5984 Acked Read 6 Done Done Sent false
21378 5985 Acked Write 8 Done Done Sent false
21379 5986 Acked Write 5 Done Done Sent false
21380 5987 Acked Read 10 Done Done Sent false
21381 5988 Acked Read 3 Done Done Sent false
21382 5989 Acked Write 5 Done Done Sent false
21383 5990 Acked Read 9 Done Done Sent false
21384 5991 Acked Write 1 Done Done Sent false
21385 5992 Acked Write 6 Done Done Sent false
21386 5993 Acked Write 7 Done Done Sent false
21387 5994 Acked Read 10 Done Done Sent false
21388 5995 Acked Read 7 Done Done Sent false
21389 5996 Acked Flush 0 Done Done Sent false
21390 5997 Acked Read 3 Done Done Sent false
21391 5998 Acked Read 6 Done Done Sent false
21392 5999 Acked Read 6 Done Done Sent false
21393 6000 Acked Flush 0 Done Done Sent false
21394 6001 Acked Write 1 Done Done Sent false
21395 6002 Acked Write 9 Done Done Sent false
21396 6003 Acked Read 3 Done Done Sent false
21397 6004 Acked Write 9 Done Done Sent false
21398 6005 Acked Read 6 Done Done Sent false
21399 6006 Acked Read 5 Done Done Sent false
21400 6007 Acked Read 2 Done Done Sent false
21401 6008 Acked Read 4 Done Done Sent false
21402 6009 Acked Read 6 Done Done Sent false
21403 6010 Acked Read 8 Done Done Sent false
21404 6011 Acked Read 3 Done Done Sent false
21405 6012 Acked Write 3 Done Done Sent false
21406 6013 Acked Write 2 Done Done Sent false
21407 6014 Acked Write 4 Done Done Sent false
21408 6015 Acked Write 1 Done Done Sent false
21409 6016 Acked Read 3 Done Done Sent false
21410 6017 Acked Read 10 Done Done Sent false
21411 6018 Acked Read 1 Done Done Sent false
21412 6019 Acked Read 1 Done Done Sent false
21413 6020 Acked Read 2 Done Done Sent false
21414 6021 Acked Write 1 Done Done Sent false
21415 6022 Acked Flush 0 Done Done Sent false
21416 6023 Acked Write 1 Done Done Sent false
21417 6024 Acked Flush 0 Done Done Sent false
21418 6025 Acked Write 5 Done Done Sent false
21419 6026 Acked Flush 0 Done Done Sent false
21420 6027 Acked Read 4 Done Done Sent false
21421 6028 Acked Read 10 Done Done Sent false
21422 6029 Acked Read 6 Done Done Sent false
21423 6030 Acked Read 7 Done Done Sent false
21424 6031 Acked Write 7 Done Done Sent false
21425 6032 Acked Read 1 Done Done Sent false
21426 6033 Acked Read 2 Done Done Sent false
21427 6034 Acked Write 4 Done Done Sent false
21428 6035 Acked Read 6 Done Done Sent false
21429 6036 Acked Read 4 Done Done Sent false
21430 6037 Acked Read 1 Done Done Sent false
21431 6038 Acked Write 5 Done Done Sent false
21432 6039 Acked Write 10 Done Done Sent false
21433 6040 Acked Write 10 Done Done Sent false
21434 6041 Acked Read 6 Done Done Sent false
21435 6042 Acked Write 4 Done Done Sent false
21436 6043 Acked Read 5 Done Done Sent false
21437 6044 Acked Read 3 Done Done Sent false
21438 6045 Acked Read 9 Done Done Sent false
21439 6046 Acked Write 7 Done Done Sent false
21440 6047 Acked Write 10 Done Done Sent false
21441 6048 Acked Read 8 Done Done Sent false
21442 6049 Acked Read 5 Done Done Sent false
21443 6050 Acked Write 4 Done Done Sent false
21444 6051 Acked Write 3 Done Done Sent false
21445 6052 Acked Write 4 Done Done Sent false
21446 6053 Acked Flush 0 Done Done Sent false
21447 6054 Acked Read 6 Done Done Sent false
21448 6055 Acked Write 6 Done Done Sent false
21449 6056 Acked Read 2 Done Done Sent false
21450 6057 Acked Read 8 Done Done Sent false
21451 6058 Acked Write 1 Done Done Sent false
21452 6059 Acked Write 2 Done Done Sent false
21453 6060 Acked Read 2 Done Done Sent false
21454 6061 Acked Write 5 Done Done Sent false
21455 6062 Acked Write 6 Done Done Sent false
21456 6063 Acked Read 6 Done Done Sent false
21457 6064 Acked Write 10 Done Done Sent false
21458 6065 Acked Read 8 Done Done Sent false
21459 6066 Acked Flush 0 Done Done Sent false
21460 6067 Acked Write 7 Done Done Sent false
21461 6068 Acked Read 9 Done Done Sent false
21462 6069 Acked Read 6 Done Done Sent false
21463 6070 Acked Read 9 Done Done Sent false
21464 6071 Acked Flush 0 Done Done Sent false
21465 6072 Acked Read 5 Done Done Sent false
21466 6073 Acked Write 8 Done Done Sent false
21467 6074 Acked Write 3 Done Done Sent false
21468 6075 Acked Write 8 Done Done Sent false
21469 6076 Acked Read 7 Done Done Sent false
21470 6077 Acked Read 3 Done Done Sent false
21471 6078 Acked Flush 0 Done Done Sent false
21472 6079 Acked Read 7 Done Done Sent false
21473 6080 Acked Read 9 Done Done Sent false
21474 6081 Acked Write 4 Done Done Sent false
21475 6082 Acked Read 8 Done Done Sent false
21476 6083 Acked Read 3 Done Done Sent false
21477 6084 Acked Write 10 Done Done Sent false
21478 6085 Acked Write 2 Done Done Sent false
21479 6086 Acked Write 5 Done Done Sent false
21480 6087 Acked Read 1 Done Done Sent false
21481 6088 Acked Read 1 Done Done Sent false
21482 6089 Acked Read 4 Done Done Sent false
21483 6090 Acked Write 2 Done Done Sent false
21484 6091 Acked Write 3 Done Done Sent false
21485 6092 Acked Read 1 Done Done Sent false
21486 6093 Acked Read 7 Done Done Sent false
21487 6094 Acked Write 8 Done Done Sent false
21488 6095 Acked Write 3 Done Done Sent false
21489 6096 Acked Read 9 Done Done Sent false
21490 6097 Acked Read 1 Done Done Sent false
21491 6098 Acked Write 5 Done Done Sent false
21492 6099 Acked Write 5 Done Done Sent false
21493 6100 Acked Write 8 Done Done Sent false
21494 6101 Acked Read 10 Done Done Sent false
21495 6102 Acked Write 2 Done Done Sent false
21496 6103 Acked Flush 0 Done Done Sent false
21497 6104 Acked Write 10 Done Done Sent false
21498 6105 Acked Write 7 Done Done Sent false
21499 6106 Acked Write 10 Done Done Sent false
21500 6107 Acked Read 10 Done Done Sent false
21501 6108 Acked Read 1 Done Done Sent false
21502 6109 Acked Write 5 Done Done Sent false
21503 6110 Acked Read 10 Done Done Sent false
21504 6111 Acked Read 9 Done Done Sent false
21505 6112 Acked Flush 0 Done Done Sent false
21506 6113 Acked Write 6 Done Done Sent false
21507 6114 Acked Read 10 Done Done Sent false
21508 6115 Acked Read 10 Done Done Sent false
21509 6116 Acked Write 7 Done Done Sent false
21510 6117 Acked Write 3 Done Done Sent false
21511 6118 Acked Read 1 Done Done Sent false
21512 6119 Acked Write 8 Done Done Sent false
21513 6120 Acked Write 8 Done Done Sent false
21514 6121 Acked Flush 0 Done Done Sent false
21515 6122 Acked Write 7 Done Done Sent false
21516 6123 Acked Read 3 Done Done Sent false
21517 6124 Acked Flush 0 Done Done Sent false
21518 6125 Acked Write 5 Done Done Sent false
21519 6126 Acked Read 10 Done Done Sent false
21520 6127 Acked Flush 0 Done Done Sent false
21521 6128 Acked Write 4 Done Done Sent false
21522 6129 Acked Read 3 Done Done Sent false
21523 6130 Acked Write 7 Done Done Sent false
21524 STATES DS:0 DS:1 DS:2 TOTAL
21525 Sent 0 0 301 301
21526 Done 301 301 0 602
21527 Skipped 0 0 0 0
21528 Error 0 0 0 0
21529 Last Flush: 6127 6127 5829
21530 Downstairs last five completed: 5829 5828 5827 5826 5825
21531 Upstairs last five completed: 6130 6129 6128 6127 6126
21532 CLIENT: Up:0 ds:301 act:2
21533 [8830] got start action
21534 [8830] I am going to restart
21535 Starting downstairs at port 8830
21536 Make output file at "/tmp/dsc/downstairs-8830.txt"
21537 [8830][2] reports Starting
21538 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1222)
21539 [8830][2] reports Running
21540 ----------------------------------------------------------------
21541 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
21542 States: Active Active Offline
21543 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
21544 5830 Acked Write 10 Done Done Sent false
21545 5831 Acked Write 4 Done Done Sent false
21546 5832 Acked Write 10 Done Done Sent false
21547 5833 Acked Write 6 Done Done Sent false
21548 5834 Acked Read 9 Done Done Sent false
21549 5835 Acked Write 9 Done Done Sent false
21550 5836 Acked Write 7 Done Done Sent false
21551 5837 Acked Write 1 Done Done Sent false
21552 5838 Acked Read 7 Done Done Sent false
21553 5839 Acked Read 10 Done Done Sent false
21554 5840 Acked Write 5 Done Done Sent false
21555 5841 Acked Read 5 Done Done Sent false
21556 5842 Acked Read 5 Done Done Sent false
21557 5843 Acked Read 4 Done Done Sent false
21558 5844 Acked Read 5 Done Done Sent false
21559 5845 Acked Read 6 Done Done Sent false
21560 5846 Acked Write 1 Done Done Sent false
21561 5847 Acked Read 1 Done Done Sent false
21562 5848 Acked Write 8 Done Done Sent false
21563 5849 Acked Write 6 Done Done Sent false
21564 5850 Acked Read 9 Done Done Sent false
21565 5851 Acked Flush 0 Done Done Sent false
21566 5852 Acked Read 8 Done Done Sent false
21567 5853 Acked Write 4 Done Done Sent false
21568 5854 Acked Read 7 Done Done Sent false
21569 5855 Acked Read 4 Done Done Sent false
21570 5856 Acked Read 3 Done Done Sent false
21571 5857 Acked Flush 0 Done Done Sent false
21572 5858 Acked Read 6 Done Done Sent false
21573 5859 Acked Flush 0 Done Done Sent false
21574 5860 Acked Write 7 Done Done Sent false
21575 5861 Acked Read 6 Done Done Sent false
21576 5862 Acked Write 10 Done Done Sent false
21577 5863 Acked Read 6 Done Done Sent false
21578 5864 Acked Read 9 Done Done Sent false
21579 5865 Acked Read 5 Done Done Sent false
21580 5866 Acked Write 3 Done Done Sent false
21581 5867 Acked Write 6 Done Done Sent false
21582 5868 Acked Write 5 Done Done Sent false
21583 5869 Acked Read 6 Done Done Sent false
21584 5870 Acked Flush 0 Done Done Sent false
21585 5871 Acked Flush 0 Done Done Sent false
21586 5872 Acked Read 5 Done Done Sent false
21587 5873 Acked Write 3 Done Done Sent false
21588 5874 Acked Read 6 Done Done Sent false
21589 5875 Acked Read 5 Done Done Sent false
21590 5876 Acked Read 2 Done Done Sent false
21591 5877 Acked Read 1 Done Done Sent false
21592 5878 Acked Read 8 Done Done Sent false
21593 5879 Acked Write 3 Done Done Sent false
21594 5880 Acked Write 2 Done Done Sent false
21595 5881 Acked Write 7 Done Done Sent false
21596 5882 Acked Read 3 Done Done Sent false
21597 5883 Acked Write 8 Done Done Sent false
21598 5884 Acked Read 10 Done Done Sent false
21599 5885 Acked Flush 0 Done Done Sent false
21600 5886 Acked Read 4 Done Done Sent false
21601 5887 Acked Write 7 Done Done Sent false
21602 5888 Acked Write 5 Done Done Sent false
21603 5889 Acked Write 7 Done Done Sent false
21604 5890 Acked Flush 0 Done Done Sent false
21605 5891 Acked Write 6 Done Done Sent false
21606 5892 Acked Flush 0 Done Done Sent false
21607 5893 Acked Write 4 Done Done Sent false
21608 5894 Acked Read 6 Done Done Sent false
21609 5895 Acked Read 6 Done Done Sent false
21610 5896 Acked Read 4 Done Done Sent false
21611 5897 Acked Read 7 Done Done Sent false
21612 5898 Acked Read 9 Done Done Sent false
21613 5899 Acked Read 6 Done Done Sent false
21614 5900 Acked Write 8 Done Done Sent false
21615 5901 Acked Read 2 Done Done Sent false
21616 5902 Acked Write 6 Done Done Sent false
21617 5903 Acked Read 4 Done Done Sent false
21618 5904 Acked Flush 0 Done Done Sent false
21619 5905 Acked Write 9 Done Done Sent false
21620 5906 Acked Read 2 Done Done Sent false
21621 5907 Acked Write 8 Done Done Sent false
21622 5908 Acked Flush 0 Done Done Sent false
21623 5909 Acked Read 4 Done Done Sent false
21624 5910 Acked Read 8 Done Done Sent false
21625 5911 Acked Read 3 Done Done Sent false
21626 5912 Acked Write 1 Done Done Sent false
21627 5913 Acked Write 9 Done Done Sent false
21628 5914 Acked Flush 0 Done Done Sent false
21629 5915 Acked Read 6 Done Done Sent false
21630 5916 Acked Write 10 Done Done Sent false
21631 5917 Acked Read 10 Done Done Sent false
21632 5918 Acked Read 6 Done Done Sent false
21633 5919 Acked Read 4 Done Done Sent false
21634 5920 Acked Write 5 Done Done Sent false
21635 5921 Acked Read 10 Done Done Sent false
21636 5922 Acked Write 6 Done Done Sent false
21637 5923 Acked Write 2 Done Done Sent false
21638 5924 Acked Write 8 Done Done Sent false
21639 5925 Acked Write 7 Done Done Sent false
21640 5926 Acked Read 7 Done Done Sent false
21641 5927 Acked Write 9 Done Done Sent false
21642 5928 Acked Write 2 Done Done Sent false
21643 5929 Acked Read 1 Done Done Sent false
21644 5930 Acked Read 3 Done Done Sent false
21645 5931 Acked Write 5 Done Done Sent false
21646 5932 Acked Write 2 Done Done Sent false
21647 5933 Acked Write 5 Done Done Sent false
21648 5934 Acked Write 2 Done Done Sent false
21649 5935 Acked Read 3 Done Done Sent false
21650 5936 Acked Write 3 Done Done Sent false
21651 5937 Acked Read 7 Done Done Sent false
21652 5938 Acked Flush 0 Done Done Sent false
21653 5939 Acked Write 4 Done Done Sent false
21654 5940 Acked Write 3 Done Done Sent false
21655 5941 Acked Read 10 Done Done Sent false
21656 5942 Acked Read 4 Done Done Sent false
21657 5943 Acked Read 6 Done Done Sent false
21658 5944 Acked Read 3 Done Done Sent false
21659 5945 Acked Write 8 Done Done Sent false
21660 5946 Acked Flush 0 Done Done Sent false
21661 5947 Acked Flush 0 Done Done Sent false
21662 5948 Acked Write 2 Done Done Sent false
21663 5949 Acked Read 5 Done Done Sent false
21664 5950 Acked Read 5 Done Done Sent false
21665 5951 Acked Read 9 Done Done Sent false
21666 5952 Acked Write 3 Done Done Sent false
21667 5953 Acked Flush 0 Done Done Sent false
21668 5954 Acked Read 9 Done Done Sent false
21669 5955 Acked Write 2 Done Done Sent false
21670 5956 Acked Write 1 Done Done Sent false
21671 5957 Acked Read 3 Done Done Sent false
21672 5958 Acked Write 2 Done Done Sent false
21673 5959 Acked Write 5 Done Done Sent false
21674 5960 Acked Write 4 Done Done Sent false
21675 5961 Acked Write 8 Done Done Sent false
21676 5962 Acked Read 9 Done Done Sent false
21677 5963 Acked Write 2 Done Done Sent false
21678 5964 Acked Read 7 Done Done Sent false
21679 5965 Acked Read 1 Done Done Sent false
21680 5966 Acked Read 3 Done Done Sent false
21681 5967 Acked Write 8 Done Done Sent false
21682 5968 Acked Write 2 Done Done Sent false
21683 5969 Acked Read 7 Done Done Sent false
21684 5970 Acked Read 5 Done Done Sent false
21685 5971 Acked Write 3 Done Done Sent false
21686 5972 Acked Write 5 Done Done Sent false
21687 5973 Acked Read 10 Done Done Sent false
21688 5974 Acked Read 1 Done Done Sent false
21689 5975 Acked Read 6 Done Done Sent false
21690 5976 Acked Write 10 Done Done Sent false
21691 5977 Acked Write 6 Done Done Sent false
21692 5978 Acked Write 4 Done Done Sent false
21693 5979 Acked Write 9 Done Done Sent false
21694 5980 Acked Read 8 Done Done Sent false
21695 5981 Acked Read 6 Done Done Sent false
21696 5982 Acked Write 7 Done Done Sent false
21697 5983 Acked Read 10 Done Done Sent false
21698 5984 Acked Read 6 Done Done Sent false
21699 5985 Acked Write 8 Done Done Sent false
21700 5986 Acked Write 5 Done Done Sent false
21701 5987 Acked Read 10 Done Done Sent false
21702 5988 Acked Read 3 Done Done Sent false
21703 5989 Acked Write 5 Done Done Sent false
21704 5990 Acked Read 9 Done Done Sent false
21705 5991 Acked Write 1 Done Done Sent false
21706 5992 Acked Write 6 Done Done Sent false
21707 5993 Acked Write 7 Done Done Sent false
21708 5994 Acked Read 10 Done Done Sent false
21709 5995 Acked Read 7 Done Done Sent false
21710 5996 Acked Flush 0 Done Done Sent false
21711 5997 Acked Read 3 Done Done Sent false
21712 5998 Acked Read 6 Done Done Sent false
21713 5999 Acked Read 6 Done Done Sent false
21714 6000 Acked Flush 0 Done Done Sent false
21715 6001 Acked Write 1 Done Done Sent false
21716 6002 Acked Write 9 Done Done Sent false
21717 6003 Acked Read 3 Done Done Sent false
21718 6004 Acked Write 9 Done Done Sent false
21719 6005 Acked Read 6 Done Done Sent false
21720 6006 Acked Read 5 Done Done Sent false
21721 6007 Acked Read 2 Done Done Sent false
21722 6008 Acked Read 4 Done Done Sent false
21723 6009 Acked Read 6 Done Done Sent false
21724 6010 Acked Read 8 Done Done Sent false
21725 6011 Acked Read 3 Done Done Sent false
21726 6012 Acked Write 3 Done Done Sent false
21727 6013 Acked Write 2 Done Done Sent false
21728 6014 Acked Write 4 Done Done Sent false
21729 6015 Acked Write 1 Done Done Sent false
21730 6016 Acked Read 3 Done Done Sent false
21731 6017 Acked Read 10 Done Done Sent false
21732 6018 Acked Read 1 Done Done Sent false
21733 6019 Acked Read 1 Done Done Sent false
21734 6020 Acked Read 2 Done Done Sent false
21735 6021 Acked Write 1 Done Done Sent false
21736 6022 Acked Flush 0 Done Done Sent false
21737 6023 Acked Write 1 Done Done Sent false
21738 6024 Acked Flush 0 Done Done Sent false
21739 6025 Acked Write 5 Done Done Sent false
21740 6026 Acked Flush 0 Done Done Sent false
21741 6027 Acked Read 4 Done Done Sent false
21742 6028 Acked Read 10 Done Done Sent false
21743 6029 Acked Read 6 Done Done Sent false
21744 6030 Acked Read 7 Done Done Sent false
21745 6031 Acked Write 7 Done Done Sent false
21746 6032 Acked Read 1 Done Done Sent false
21747 6033 Acked Read 2 Done Done Sent false
21748 6034 Acked Write 4 Done Done Sent false
21749 6035 Acked Read 6 Done Done Sent false
21750 6036 Acked Read 4 Done Done Sent false
21751 6037 Acked Read 1 Done Done Sent false
21752 6038 Acked Write 5 Done Done Sent false
21753 6039 Acked Write 10 Done Done Sent false
21754 6040 Acked Write 10 Done Done Sent false
21755 6041 Acked Read 6 Done Done Sent false
21756 6042 Acked Write 4 Done Done Sent false
21757 6043 Acked Read 5 Done Done Sent false
21758 6044 Acked Read 3 Done Done Sent false
21759 6045 Acked Read 9 Done Done Sent false
21760 6046 Acked Write 7 Done Done Sent false
21761 6047 Acked Write 10 Done Done Sent false
21762 6048 Acked Read 8 Done Done Sent false
21763 6049 Acked Read 5 Done Done Sent false
21764 6050 Acked Write 4 Done Done Sent false
21765 6051 Acked Write 3 Done Done Sent false
21766 6052 Acked Write 4 Done Done Sent false
21767 6053 Acked Flush 0 Done Done Sent false
21768 6054 Acked Read 6 Done Done Sent false
21769 6055 Acked Write 6 Done Done Sent false
21770 6056 Acked Read 2 Done Done Sent false
21771 6057 Acked Read 8 Done Done Sent false
21772 6058 Acked Write 1 Done Done Sent false
21773 6059 Acked Write 2 Done Done Sent false
21774 6060 Acked Read 2 Done Done Sent false
21775 6061 Acked Write 5 Done Done Sent false
21776 6062 Acked Write 6 Done Done Sent false
21777 6063 Acked Read 6 Done Done Sent false
21778 6064 Acked Write 10 Done Done Sent false
21779 6065 Acked Read 8 Done Done Sent false
21780 6066 Acked Flush 0 Done Done Sent false
21781 6067 Acked Write 7 Done Done Sent false
21782 6068 Acked Read 9 Done Done Sent false
21783 6069 Acked Read 6 Done Done Sent false
21784 6070 Acked Read 9 Done Done Sent false
21785 6071 Acked Flush 0 Done Done Sent false
21786 6072 Acked Read 5 Done Done Sent false
21787 6073 Acked Write 8 Done Done Sent false
21788 6074 Acked Write 3 Done Done Sent false
21789 6075 Acked Write 8 Done Done Sent false
21790 6076 Acked Read 7 Done Done Sent false
21791 6077 Acked Read 3 Done Done Sent false
21792 6078 Acked Flush 0 Done Done Sent false
21793 6079 Acked Read 7 Done Done Sent false
21794 6080 Acked Read 9 Done Done Sent false
21795 6081 Acked Write 4 Done Done Sent false
21796 6082 Acked Read 8 Done Done Sent false
21797 6083 Acked Read 3 Done Done Sent false
21798 6084 Acked Write 10 Done Done Sent false
21799 6085 Acked Write 2 Done Done Sent false
21800 6086 Acked Write 5 Done Done Sent false
21801 6087 Acked Read 1 Done Done Sent false
21802 6088 Acked Read 1 Done Done Sent false
21803 6089 Acked Read 4 Done Done Sent false
21804 6090 Acked Write 2 Done Done Sent false
21805 6091 Acked Write 3 Done Done Sent false
21806 6092 Acked Read 1 Done Done Sent false
21807 6093 Acked Read 7 Done Done Sent false
21808 6094 Acked Write 8 Done Done Sent false
21809 6095 Acked Write 3 Done Done Sent false
21810 6096 Acked Read 9 Done Done Sent false
21811 6097 Acked Read 1 Done Done Sent false
21812 6098 Acked Write 5 Done Done Sent false
21813 6099 Acked Write 5 Done Done Sent false
21814 6100 Acked Write 8 Done Done Sent false
21815 6101 Acked Read 10 Done Done Sent false
21816 6102 Acked Write 2 Done Done Sent false
21817 6103 Acked Flush 0 Done Done Sent false
21818 6104 Acked Write 10 Done Done Sent false
21819 6105 Acked Write 7 Done Done Sent false
21820 6106 Acked Write 10 Done Done Sent false
21821 6107 Acked Read 10 Done Done Sent false
21822 6108 Acked Read 1 Done Done Sent false
21823 6109 Acked Write 5 Done Done Sent false
21824 6110 Acked Read 10 Done Done Sent false
21825 6111 Acked Read 9 Done Done Sent false
21826 6112 Acked Flush 0 Done Done Sent false
21827 6113 Acked Write 6 Done Done Sent false
21828 6114 Acked Read 10 Done Done Sent false
21829 6115 Acked Read 10 Done Done Sent false
21830 6116 Acked Write 7 Done Done Sent false
21831 6117 Acked Write 3 Done Done Sent false
21832 6118 Acked Read 1 Done Done Sent false
21833 6119 Acked Write 8 Done Done Sent false
21834 6120 Acked Write 8 Done Done Sent false
21835 6121 Acked Flush 0 Done Done Sent false
21836 6122 Acked Write 7 Done Done Sent false
21837 6123 Acked Read 3 Done Done Sent false
21838 6124 Acked Flush 0 Done Done Sent false
21839 6125 Acked Write 5 Done Done Sent false
21840 6126 Acked Read 10 Done Done Sent false
21841 6127 Acked Flush 0 Done Done Sent false
21842 6128 Acked Write 4 Done Done Sent false
21843 6129 Acked Read 3 Done Done Sent false
21844 6130 Acked Write 7 Done Done Sent false
21845 6131 Acked Flush 0 Done Done Sent false
21846 STATES DS:0 DS:1 DS:2 TOTAL
21847 Sent 0 0 302 302
21848 Done 302 302 0 604
21849 Skipped 0 0 0 0
21850 Error 0 0 0 0
21851 Last Flush: 6131 6131 5829
21852 Downstairs last five completed: 5829 5828 5827 5826 5825
21853 Upstairs last five completed: 6131 6130 6129 6128 6127
21854 CLIENT: Up:0 ds:302 act:2
218552024-11-15T02:02:01.834ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
218562024-11-15T02:02:01.834ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:51876) = io task client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
218572024-11-15T02:02:01.836ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
218582024-11-15T02:02:01.836ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
218592024-11-15T02:02:01.836ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
218602024-11-15T02:02:01.836ZINFOcrucible: send last flush ID to this DS: 5829 = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
218612024-11-15T02:02:01.836ZINFOcrucible: Replied this last flush ID: 5829 = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
218622024-11-15T02:02:01.836ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
218632024-11-15T02:02:01.837ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
218642024-11-15T02:02:01.837ZINFOcrucible: [2] client re-new 302 jobs since flush 5829 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
218652024-11-15T02:02:01.837ZINFOcrucible: [2] Replayed 302 jobs since flush: 5829 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
21866 ----------------------------------------------------------------
21867 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
21868 Downstairs last five completed: 6131 6130 6129 6128 6127
21869 Upstairs last five completed: 6131 6130 6129 6128 6127
21870 CLIENT: Up:0 ds:0 act:3
21871 Replay: All jobs finished
21872 Main task has work to do, go find it
21873 got dsc Stop(2)
21874 stop 2
21875 Nov 15 02:02:04.073 [8830] Got stop action so:false kr:false
21876 INFO request completed, latency_us: 505, response_code: 204, uri: /stop/cid/2, method: POST, req_id: dbe396ab-e887-4fcd-a1d6-53d059ccef3e, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
218772024-11-15T02:02:04.074ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
218782024-11-15T02:02:04.074ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
218792024-11-15T02:02:04.074ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
218802024-11-15T02:02:04.074ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
218812024-11-15T02:02:04.074ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
21882 Nov 15 02:02:04.074 INFO request completed, latency_us:{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2024-11-15T02:02:04.074887284Z","hostname":"w-01JCPRHT5F2MPA44KVJHM3S2PB","pid":1205,"":"io task","client":"2","":"downstairs","session_id":"971dcb2e-6891-4ecf-b367-1e04e52065ee"}
21883 160, response_code: 200, uri: /state/cid/2, method: GET, req_id: 0c265254-6b04-42b5-994b-f28cd827bda7, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
21884 [8830] Exited with: ExitStatus(unix_wait_status(9))
21885 [8830][2] reports Exit
21886 Nov 15 02:02:08.076 INFO request completed, latency_us: 217, response_code: 200, uri: /state/cid/2, method: GET, req_id: c7712de8-1d74-4365-9c7f-ba79a1ecc275, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
21887 001/300 Read block 681 len 36864
21888 002/300 Write block 2167 len 12288 data: 4 4 4
21889 003/300 Read block 3047 len 28672
21890 004/300 Write block 172 len 4096 data: 6
21891 005/300 Read block 3526 len 40960
21892 006/300 Write block 4863 len 28672 data: 5 4 5 5 4 2 2
21893 007/300 Write block 2825 len 40960 data: 4 4 3 3 3 2 2 2 3 4
21894 008/300 Flush
21895 009/300 Flush
21896 010/300 Write block 1712 len 24576 data: 3 3 2 2 2 3
21897 011/300 Write block 2455 len 36864 data: 9 8 8 6 6 7 7 6 7
21898 012/300 Write block 1982 len 24576 data: 5 5 5 4 5 4
21899 013/300 Flush
21900 014/300 Read block 427 len 20480
21901 015/300 Write block 528 len 16384 data: 4 3 5 5
21902 016/300 Write block 2123 len 16384 data: 3 3 3 3
21903 017/300 Write block 4427 len 32768 data: 4 3 3 3 3 3 3 2
21904 018/300 Read block 2497 len 36864
21905 019/300 Read block 2776 len 20480
21906 020/300 Write block 108 len 40960 data: 6 6 5 4 3 3 2 2 3 3
21907 021/300 Write block 2371 len 4096 data: 2
21908 022/300 Write block 4500 len 4096 data: 3
21909 023/300 Write block 430 len 16384 data: 4 4 4 4
21910 024/300 Read block 1052 len 4096
21911 025/300 Write block 2375 len 8192 data: 4 3
21912 026/300 Flush
21913 027/300 Write block 2856 len 28672 data: 5 3 3 4 4 3 2
21914 028/300 Flush
21915 029/300 Read block 4841 len 36864
21916 030/300 Write block 998 len 20480 data: 7 6 6 6 6
21917 031/300 Flush
21918 032/300 Write block 1938 len 16384 data: 6 6 6 6
21919 033/300 Write block 801 len 24576 data: 4 3 3 2 2 2
21920 034/300 Read block 2446 len 28672
21921 035/300 Write block 3490 len 36864 data: 6 5 4 3 3 4 4 4 5
21922 036/300 Read block 2873 len 40960
21923 037/300 Read block 1513 len 40960
21924 038/300 Read block 1589 len 32768
21925 039/300 Read block 153 len 8192
21926 040/300 Read block 616 len 12288
21927 041/300 Read block 3937 len 12288
21928 042/300 Write block 1055 len 16384 data: 4 4 4 4
21929 043/300 Write block 2260 len 36864 data: 2 3 4 4 5 5 5 4 2
21930 044/300 Read block 731 len 24576
21931 045/300 Write block 3630 len 40960 data: 3 4 4 2 2 2 2 2 2 4
21932 046/300 Flush
21933 047/300 Write block 859 len 36864 data: 3 2 2 3 3 3 3 2 2
21934 048/300 Write block 4205 len 12288 data: 8 9 7
21935 049/300 Read block 2849 len 20480
21936 050/300 Read block 1854 len 36864
21937 051/300 Write block 696 len 12288 data: 3 3 3
21938 052/300 Write block 2314 len 28672 data: 4 4 4 3 5 6 6
21939 053/300 Read block 3289 len 12288
21940 054/300 Read block 1634 len 16384
21941 055/300 Read block 1839 len 28672
21942 056/300 Flush
21943 057/300 Write block 617 len 8192 data: 3 4
21944 058/300 Read block 2481 len 28672
21945 059/300 Write block 4892 len 24576 data: 6 8 8 7 6 6
21946 060/300 Read block 239 len 8192
21947 061/300 Read block 2518 len 8192
21948 062/300 Read block 2483 len 20480
21949 063/300 Read block 4929 len 32768
21950 064/300 Write block 70 len 24576 data: 5 6 6 5 5 5
21951 065/300 Read block 3707 len 32768
21952 066/300 Read block 4956 len 20480
21953 067/300 Write block 2949 len 8192 data: 4 4
21954 068/300 Flush
21955 069/300 Read block 888 len 36864
21956 070/300 Read block 2280 len 16384
21957 071/300 Read block 886 len 40960
21958 072/300 Read block 2360 len 16384
21959 073/300 Read block 973 len 20480
21960 074/300 Read block 2580 len 16384
21961 075/300 Read block 4703 len 12288
21962 076/300 Read block 1378 len 8192
21963 077/300 Read block 2498 len 32768
21964 078/300 Read block 714 len 16384
21965 079/300 Read block 1383 len 20480
21966 080/300 Write block 4298 len 8192 data: 6 7
21967 081/300 Write block 1809 len 16384 data: 3 2 3 3
21968 082/300 Write block 4172 len 32768 data: 3 6 6 7 6 6 8 9
21969 083/300 Write block 4488 len 20480 data: 3 3 4 4 5
21970 084/300 Write block 3263 len 16384 data: 5 5 4 3
21971 085/300 Read block 2207 len 4096
21972 086/300 Read block 3445 len 28672
21973 087/300 Read block 1514 len 32768
21974 088/300 Write block 4616 len 20480 data: 6 6 6 6 6
21975 089/300 Write block 3441 len 20480 data: 5 5 5 5 5
21976 090/300 Read block 189 len 36864
21977 091/300 Read block 2667 len 20480
21978 092/300 Write block 525 len 8192 data: 3 5
21979 093/300 Write block 1644 len 36864 data: 6 5 4 5 4 3 3 4 5
21980 094/300 Read block 2665 len 4096
21981 095/300 Write block 4652 len 8192 data: 5 4
21982 096/300 Write block 2874 len 36864 data: 4 4 3 3 4 3 3 3 3
21983 097/300 Read block 1311 len 28672
21984 098/300 Read block 2863 len 24576
21985 099/300 Write block 4133 len 24576 data: 4 4 3 3 2 2
21986 100/300 Read block 3791 len 12288
21987 101/300 Read block 4900 len 24576
21988 102/300 Read block 3451 len 12288
21989 103/300 Flush
21990 104/300 Flush
21991 105/300 Read block 1408 len 28672
21992 106/300 Write block 498 len 20480 data: 5 5 4 6 6
21993 107/300 Read block 329 len 32768
21994 108/300 Write block 3890 len 36864 data: 3 3 3 3 5 5 4 6 8
21995 109/300 Write block 1214 len 20480 data: 3 3 3 2 3
21996 110/300 Read block 3011 len 8192
21997 111/300 Write block 3377 len 40960 data: 5 4 6 6 6 5 5 4 3 3
21998 112/300 Read block 829 len 32768
21999 113/300 Flush
22000 114/300 Write block 3274 len 12288 data: 3 3 3
22001 115/300 Read block 3504 len 28672
22002 116/300 Write block 3811 len 20480 data: 5 4 3 3 2
22003 117/300 Write block 381 len 36864 data: 3 3 3 3 3 3 3 6 6
22004 118/300 Write block 343 len 32768 data: 3 3 3 2 2 2 2 2
22005 119/300 Read block 357 len 16384
22006 120/300 Flush
22007 121/300 Read block 1 len 28672
22008 122/300 Write block 997 len 24576 data: 7 8 7 7 7 7
22009 123/300 Write block 716 len 36864 data: 3 4 4 2 2 2 2 2 2
22010 124/300 Flush
22011 125/300 Read block 864 len 20480
22012 126/300 Read block 2163 len 20480
22013 127/300 Read block 146 len 12288
22014 128/300 Read block 1970 len 4096
22015 129/300 Write block 2490 len 12288 data: 4 6 6
22016 130/300 Read block 4390 len 20480
22017 131/300 Read block 1881 len 16384
22018 132/300 Read block 2161 len 24576
22019 133/300 Read block 3511 len 4096
22020 134/300 Read block 3 len 40960
22021 135/300 Read block 110 len 16384
22022 136/300 Flush
22023 137/300 Read block 3020 len 20480
22024 138/300 Read block 4909 len 28672
22025 139/300 Read block 769 len 16384
22026 140/300 Read block 2563 len 36864
22027 141/300 Read block 3459 len 20480
22028 142/300 Read block 2887 len 28672
22029 143/300 Write block 2990 len 4096 data: 4
22030 144/300 Read block 669 len 20480
22031 145/300 Read block 2752 len 16384
22032 146/300 Write block 3881 len 32768 data: 5 5 5 5 6 5 5 5
22033 147/300 Read block 3647 len 16384
22034 148/300 Write block 2240 len 12288 data: 5 5 5
22035 149/300 Read block 740 len 32768
22036 150/300 Read block 457 len 16384
22037 151/300 Read block 3127 len 36864
22038 152/300 Read block 3727 len 4096
22039 153/300 Read block 3329 len 20480
22040 154/300 Read block 980 len 36864
22041 155/300 Write block 4043 len 28672 data: 7 6 7 7 5 3 3
22042 156/300 Write block 4453 len 36864 data: 3 3 3 4 4 4 4 4 4
22043 157/300 Read block 4776 len 40960
22044 158/300 Write block 2734 len 40960 data: 3 2 2 3 3 4 4 4 4 4
22045 159/300 Write block 362 len 24576 data: 4 5 3 3 3 3
22046 160/300 Write block 4093 len 24576 data: 5 5 5 5 4 3
22047 161/300 Flush
22048 162/300 Read block 2994 len 36864
22049 163/300 Flush
22050 164/300 Read block 220 len 8192
22051 165/300 Write block 4822 len 24576 data: 3 3 3 4 5 5
22052 166/300 Read block 3007 len 4096
22053 167/300 Write block 4580 len 28672 data: 3 4 3 3 4 5 6
22054 168/300 Write block 1915 len 8192 data: 3 3
22055 169/300 Read block 2141 len 20480
22056 170/300 Flush
22057 171/300 Write block 4661 len 4096 data: 6
22058 172/300 Read block 2047 len 28672
22059 173/300 Read block 1874 len 32768
22060 174/300 Read block 4382 len 40960
22061 175/300 Write block 3690 len 12288 data: 7 7 6
22062 176/300 Read block 4150 len 28672
22063 177/300 Write block 2488 len 32768 data: 4 3 5 7 7 5 5 6
22064 178/300 Write block 4920 len 12288 data: 6 7 8
22065 179/300 Flush
22066 180/300 Write block 1355 len 36864 data: 4 4 4 4 4 3 3 4 4
22067 181/300 Read block 1744 len 40960
22068 182/300 Read block 2978 len 4096
22069 183/300 Write block 1912 len 36864 data: 3 2 3 4 4 3 3 4 4
22070 184/300 Read block 2626 len 24576
22071 185/300 Read block 2291 len 16384
22072 186/300 Write block 1835 len 28672 data: 3 3 5 6 5 4 6
22073 187/300 Write block 4636 len 12288 data: 3 3 3
22074 188/300 Read block 3507 len 4096
22075 189/300 Read block 1329 len 8192
22076 190/300 Read block 187 len 24576
22077 191/300 Read block 3137 len 20480
22078 192/300 Read block 1199 len 24576
22079 193/300 Read block 4809 len 40960
22080 194/300 Write block 4875 len 36864 data: 6 6 5 3 2 2 2 3 3
22081 195/300 Read block 457 len 24576
22082 196/300 Read block 163 len 4096
22083 197/300 Read block 4741 len 12288
22084 198/300 Write block 1969 len 4096 data: 3
22085 199/300 Read block 471 len 4096
22086 200/300 Read block 3441 len 28672
22087 201/300 Read block 2966 len 28672
22088 202/300 Read block 825 len 16384
22089 203/300 Read block 4612 len 40960
22090 204/300 Read block 467 len 32768
22091 205/300 Flush
22092 206/300 Read block 3048 len 8192
22093 207/300 Read block 4304 len 20480
22094 208/300 Write block 822 len 4096 data: 3
22095 209/300 Flush
22096 210/300 Read block 3360 len 16384
22097 211/300 Read block 2389 len 20480
22098 212/300 Write block 3463 len 28672 data: 2 2 2 2 3 4 3
22099 213/300 Read block 3790 len 36864
22100 214/300 Write block 459 len 24576 data: 3 4 6 6 6 5
22101 215/300 Read block 4781 len 4096
22102 216/300 Write block 459 len 12288 data: 4 5 7
22103 217/300 Read block 4371 len 40960
22104 218/300 Flush
22105 219/300 Write block 3035 len 40960 data: 5 5 5 5 5 5 4 4 4 3
22106 220/300 Read block 3450 len 12288
22107 221/300 Write block 3898 len 16384 data: 9 8 8 9
22108 222/300 Read block 1462 len 40960
22109 223/300 Read block 4924 len 16384
22110 224/300 Write block 1758 len 36864 data: 3 4 3 3 3 4 4 2 4
22111 225/300 Read block 4638 len 32768
22112 226/300 Write block 3271 len 40960 data: 4 4 5 4 4 4 3 3 3 4
22113 227/300 Read block 4865 len 32768
22114 228/300 Read block 3486 len 28672
22115 229/300 Write block 1114 len 8192 data: 4 4
22116 230/300 Read block 866 len 36864
22117 231/300 Write block 2153 len 8192 data: 5 5
22118 232/300 Read block 4232 len 16384
22119 233/300 Write block 654 len 4096 data: 7
22120 234/300 Write block 718 len 40960 data: 5 3 3 3 3 3 3 2 2 3
22121 235/300 Read block 3427 len 28672
22122 236/300 Write block 4833 len 4096 data: 5
22123 237/300 Read block 2765 len 4096
22124 238/300 Read block 4844 len 24576
22125 239/300 Read block 1890 len 40960
22126 240/300 Read block 2991 len 20480
22127 241/300 Flush
22128 242/300 Write block 10 len 28672 data: 6 7 6 5 5 3 2
22129 243/300 Flush
22130 244/300 Write block 2606 len 28672 data: 3 3 3 3 3 4 5
22131 245/300 Read block 194 len 24576
22132 246/300 Write block 1673 len 36864 data: 4 3 4 4 3 4 4 4 5
22133 247/300 Write block 3143 len 24576 data: 5 5 5 5 5 5
22134 248/300 Read block 2153 len 28672
22135 249/300 Read block 4296 len 28672
22136 250/300 Write block 610 len 20480 data: 2 2 2 2 2
22137 251/300 Read block 1936 len 16384
22138 252/300 Flush
22139 253/300 Read block 2584 len 40960
22140 254/300 Read block 4632 len 20480
22141 255/300 Write block 3660 len 32768 data: 6 6 6 5 4 3 3 4
22142 256/300 Write block 438 len 16384 data: 2 2 2 3
22143 257/300 Read block 2076 len 24576
22144 258/300 Flush
22145 259/300 Write block 2538 len 24576 data: 3 3 2 3 5 5
22146 260/300 Read block 4657 len 40960
22147 261/300 Write block 4334 len 28672 data: 5 5 4 4 5 7 6
22148 262/300 Flush
22149 263/300 Write block 4395 len 16384 data: 5 4 3 3
22150 264/300 Write block 1630 len 40960 data: 4 6 6 6 7 6 6 6 8 8
22151 265/300 Write block 4102 len 28672 data: 4 4 4 4 4 4 5
22152 266/300 Read block 1895 len 12288
22153 267/300 Write block 2403 len 28672 data: 6 6 6 4 3 4 4
22154 268/300 Write block 92 len 24576 data: 4 3 5 3 4 4
22155 269/300 Write block 2824 len 40960 data: 6 5 5 4 4 4 3 3 3 4
22156 270/300 Read block 1061 len 36864
22157 271/300 Write block 606 len 20480 data: 3 2 2 2 3
22158 272/300 Read block 2195 len 12288
22159 273/300 Read block 1843 len 40960
22160 274/300 Write block 1068 len 24576 data: 6 6 5 5 5 5
22161 275/300 Read block 2048 len 36864
22162 276/300 Read block 888 len 16384
22163 277/300 Write block 962 len 36864 data: 3 4 4 4 4 5 4 2 2
22164 278/300 Flush
22165 279/300 Write block 4321 len 8192 data: 6 5
22166 280/300 Read block 1197 len 28672
22167 281/300 Read block 3374 len 36864
22168 282/300 Write block 3102 len 4096 data: 2
22169 283/300 Read block 2955 len 8192
22170 284/300 Read block 2119 len 4096
22171 285/300 Write block 1667 len 16384 data: 9 8 8 7
22172 286/300 Read block 2664 len 32768
22173 287/300 Read block 173 len 12288
22174 288/300 Read block 4029 len 36864
22175 289/300 Read block 659 len 4096
22176 290/300 Flush
22177 291/300 Read block 2088 len 20480
22178 292/300 Read block 1580 len 28672
22179 293/300 Read block 2157 len 40960
22180 294/300 Write block 4601 len 28672 data: 3 3 3 4 4 5 5
22181 295/300 Read block 762 len 40960
22182 296/300 Read block 2485 len 36864
22183 297/300 Read block 2581 len 36864
22184 298/300 Read block 490 len 20480
22185 299/300 Write block 3253 len 28672 data: 7 6 5 6 7 7 5
22186 300/300 Read block 3755 len 40960
22187 301/300 Read block 435 len 28672
22188 Main task has work to do, go find it
22189 got dsc Start(2)
22190 start 2
22191 Nov 15 02:02:08.326 INFO request completed, latency_us: 218, response_code: 204, uri: /start/cid/2, method: POST, req_id: 2e740b56-08ec-4449-9f1f-e31168abd151, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
22192 Replay: started 2, returned:Ok(())
22193 ----------------------------------------------------------------
22194 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
22195 States: Active Active Offline
22196 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
22197 6132 Acked Read 9 Done Done Sent false
22198 6133 Acked Write 3 Done Done Sent false
22199 6134 Acked Read 7 Done Done Sent false
22200 6135 Acked Write 1 Done Done Sent false
22201 6136 Acked Read 10 Done Done Sent false
22202 6137 Acked Write 7 Done Done Sent false
22203 6138 Acked Write 10 Done Done Sent false
22204 6139 Acked Flush 0 Done Done Sent false
22205 6140 Acked Flush 0 Done Done Sent false
22206 6141 Acked Write 6 Done Done Sent false
22207 6142 Acked Write 9 Done Done Sent false
22208 6143 Acked Write 6 Done Done Sent false
22209 6144 Acked Flush 0 Done Done Sent false
22210 6145 Acked Read 5 Done Done Sent false
22211 6146 Acked Write 4 Done Done Sent false
22212 6147 Acked Write 4 Done Done Sent false
22213 6148 Acked Write 8 Done Done Sent false
22214 6149 Acked Read 9 Done Done Sent false
22215 6150 Acked Read 5 Done Done Sent false
22216 6151 Acked Write 10 Done Done Sent false
22217 6152 Acked Write 1 Done Done Sent false
22218 6153 Acked Write 1 Done Done Sent false
22219 6154 Acked Write 4 Done Done Sent false
22220 6155 Acked Read 1 Done Done Sent false
22221 6156 Acked Write 2 Done Done Sent false
22222 6157 Acked Flush 0 Done Done Sent false
22223 6158 Acked Write 7 Done Done Sent false
22224 6159 Acked Flush 0 Done Done Sent false
22225 6160 Acked Read 9 Done Done Sent false
22226 6161 Acked Write 5 Done Done Sent false
22227 6162 Acked Flush 0 Done Done Sent false
22228 6163 Acked Write 4 Done Done Sent false
22229 6164 Acked Write 6 Done Done Sent false
22230 6165 Acked Read 7 Done Done Sent false
22231 6166 Acked Write 9 Done Done Sent false
22232 6167 Acked Read 10 Done Done Sent false
22233 6168 Acked Read 10 Done Done Sent false
22234 6169 Acked Read 8 Done Done Sent false
22235 6170 Acked Read 2 Done Done Sent false
22236 6171 Acked Read 3 Done Done Sent false
22237 6172 Acked Read 3 Done Done Sent false
22238 6173 Acked Write 4 Done Done Sent false
22239 6174 Acked Write 9 Done Done Sent false
22240 6175 Acked Read 6 Done Done Sent false
22241 6176 Acked Write 10 Done Done Sent false
22242 6177 Acked Flush 0 Done Done Sent false
22243 6178 Acked Write 9 Done Done Sent false
22244 6179 Acked Write 3 Done Done Sent false
22245 6180 Acked Read 5 Done Done Sent false
22246 6181 Acked Read 9 Done Done Sent false
22247 6182 Acked Write 3 Done Done Sent false
22248 6183 Acked Write 7 Done Done Sent false
22249 6184 Acked Read 3 Done Done Sent false
22250 6185 Acked Read 4 Done Done Sent false
22251 6186 Acked Read 7 Done Done Sent false
22252 6187 Acked Flush 0 Done Done Sent false
22253 6188 Acked Write 2 Done Done Sent false
22254 6189 Acked Read 7 Done Done Sent false
22255 6190 Acked Write 6 Done Done Sent false
22256 6191 Acked Read 2 Done Done Sent false
22257 6192 Acked Read 2 Done Done Sent false
22258 6193 Acked Read 5 Done Done Sent false
22259 6194 Acked Read 8 Done Done Sent false
22260 6195 Acked Write 6 Done Done Sent false
22261 6196 Acked Read 8 Done Done Sent false
22262 6197 Acked Read 5 Done Done Sent false
22263 6198 Acked Write 2 Done Done Sent false
22264 6199 Acked Flush 0 Done Done Sent false
22265 6200 Acked Read 9 Done Done Sent false
22266 6201 Acked Read 4 Done Done Sent false
22267 6202 Acked Read 10 Done Done Sent false
22268 6203 Acked Read 4 Done Done Sent false
22269 6204 Acked Read 5 Done Done Sent false
22270 6205 Acked Read 4 Done Done Sent false
22271 6206 Acked Read 3 Done Done Sent false
22272 6207 Acked Read 2 Done Done Sent false
22273 6208 Acked Read 8 Done Done Sent false
22274 6209 Acked Read 4 Done Done Sent false
22275 6210 Acked Read 5 Done Done Sent false
22276 6211 Acked Write 2 Done Done Sent false
22277 6212 Acked Write 4 Done Done Sent false
22278 6213 Acked Write 8 Done Done Sent false
22279 6214 Acked Write 5 Done Done Sent false
22280 6215 Acked Write 4 Done Done Sent false
22281 6216 Acked Read 1 Done Done Sent false
22282 6217 Acked Read 7 Done Done Sent false
22283 6218 Acked Read 8 Done Done Sent false
22284 6219 Acked Write 5 Done Done Sent false
22285 6220 Acked Write 5 Done Done Sent false
22286 6221 Acked Read 9 Done Done Sent false
22287 6222 Acked Read 5 Done Done Sent false
22288 6223 Acked Write 2 Done Done Sent false
22289 6224 Acked Write 9 Done Done Sent false
22290 6225 Acked Read 1 Done Done Sent false
22291 6226 Acked Write 2 Done Done Sent false
22292 6227 Acked Write 9 Done Done Sent false
22293 6228 Acked Read 7 Done Done Sent false
22294 6229 Acked Read 6 Done Done Sent false
22295 6230 Acked Write 6 Done Done Sent false
22296 6231 Acked Read 3 Done Done Sent false
22297 6232 Acked Read 6 Done Done Sent false
22298 6233 Acked Read 3 Done Done Sent false
22299 6234 Acked Flush 0 Done Done Sent false
22300 6235 Acked Flush 0 Done Done Sent false
22301 6236 Acked Read 7 Done Done Sent false
22302 6237 Acked Write 5 Done Done Sent false
22303 6238 Acked Read 8 Done Done Sent false
22304 6239 Acked Write 9 Done Done Sent false
22305 6240 Acked Write 5 Done Done Sent false
22306 6241 Acked Read 2 Done Done Sent false
22307 6242 Acked Write 10 Done Done Sent false
22308 6243 Acked Read 8 Done Done Sent false
22309 6244 Acked Flush 0 Done Done Sent false
22310 6245 Acked Write 3 Done Done Sent false
22311 6246 Acked Read 7 Done Done Sent false
22312 6247 Acked Write 5 Done Done Sent false
22313 6248 Acked Write 9 Done Done Sent false
22314 6249 Acked Write 8 Done Done Sent false
22315 6250 Acked Read 4 Done Done Sent false
22316 6251 Acked Flush 0 Done Done Sent false
22317 6252 Acked Read 7 Done Done Sent false
22318 6253 Acked Write 6 Done Done Sent false
22319 6254 Acked Write 9 Done Done Sent false
22320 6255 Acked Flush 0 Done Done Sent false
22321 6256 Acked Read 5 Done Done Sent false
22322 6257 Acked Read 5 Done Done Sent false
22323 6258 Acked Read 3 Done Done Sent false
22324 6259 Acked Read 1 Done Done Sent false
22325 6260 Acked Write 3 Done Done Sent false
22326 6261 Acked Read 5 Done Done Sent false
22327 6262 Acked Read 4 Done Done Sent false
22328 6263 Acked Read 6 Done Done Sent false
22329 6264 Acked Read 1 Done Done Sent false
22330 6265 Acked Read 10 Done Done Sent false
22331 6266 Acked Read 4 Done Done Sent false
22332 6267 Acked Flush 0 Done Done Sent false
22333 6268 Acked Read 5 Done Done Sent false
22334 6269 Acked Read 7 Done Done Sent false
22335 6270 Acked Read 4 Done Done Sent false
22336 6271 Acked Read 9 Done Done Sent false
22337 6272 Acked Read 5 Done Done Sent false
22338 6273 Acked Read 7 Done Done Sent false
22339 6274 Acked Write 1 Done Done Sent false
22340 6275 Acked Read 5 Done Done Sent false
22341 6276 Acked Read 4 Done Done Sent false
22342 6277 Acked Write 8 Done Done Sent false
22343 6278 Acked Read 4 Done Done Sent false
22344 6279 Acked Write 3 Done Done Sent false
22345 6280 Acked Read 8 Done Done Sent false
22346 6281 Acked Read 4 Done Done Sent false
22347 6282 Acked Read 9 Done Done Sent false
22348 6283 Acked Read 1 Done Done Sent false
22349 6284 Acked Read 5 Done Done Sent false
22350 6285 Acked Read 9 Done Done Sent false
22351 6286 Acked Write 7 Done Done Sent false
22352 6287 Acked Write 9 Done Done Sent false
22353 6288 Acked Read 10 Done Done Sent false
22354 6289 Acked Write 10 Done Done Sent false
22355 6290 Acked Write 6 Done Done Sent false
22356 6291 Acked Write 6 Done Done Sent false
22357 6292 Acked Flush 0 Done Done Sent false
22358 6293 Acked Read 9 Done Done Sent false
22359 6294 Acked Flush 0 Done Done Sent false
22360 6295 Acked Read 2 Done Done Sent false
22361 6296 Acked Write 6 Done Done Sent false
22362 6297 Acked Read 1 Done Done Sent false
22363 6298 Acked Write 7 Done Done Sent false
22364 6299 Acked Write 2 Done Done Sent false
22365 6300 Acked Read 5 Done Done Sent false
22366 6301 Acked Flush 0 Done Done Sent false
22367 6302 Acked Write 1 Done Done Sent false
22368 6303 Acked Read 7 Done Done Sent false
22369 6304 Acked Read 8 Done Done Sent false
22370 6305 Acked Read 10 Done Done Sent false
22371 6306 Acked Write 3 Done Done Sent false
22372 6307 Acked Read 7 Done Done Sent false
22373 6308 Acked Write 8 Done Done Sent false
22374 6309 Acked Write 3 Done Done Sent false
22375 6310 Acked Flush 0 Done Done Sent false
22376 6311 Acked Write 9 Done Done Sent false
22377 6312 Acked Read 10 Done Done Sent false
22378 6313 Acked Read 1 Done Done Sent false
22379 6314 Acked Write 9 Done Done Sent false
22380 6315 Acked Read 6 Done Done Sent false
22381 6316 Acked Read 4 Done Done Sent false
22382 6317 Acked Write 7 Done Done Sent false
22383 6318 Acked Write 3 Done Done Sent false
22384 6319 Acked Read 1 Done Done Sent false
22385 6320 Acked Read 2 Done Done Sent false
22386 6321 Acked Read 6 Done Done Sent false
22387 6322 Acked Read 5 Done Done Sent false
22388 6323 Acked Read 6 Done Done Sent false
22389 6324 Acked Read 10 Done Done Sent false
22390 6325 Acked Write 9 Done Done Sent false
22391 6326 Acked Read 6 Done Done Sent false
22392 6327 Acked Read 1 Done Done Sent false
22393 6328 Acked Read 3 Done Done Sent false
22394 6329 Acked Write 1 Done Done Sent false
22395 6330 Acked Read 1 Done Done Sent false
22396 6331 Acked Read 7 Done Done Sent false
22397 6332 Acked Read 7 Done Done Sent false
22398 6333 Acked Read 4 Done Done Sent false
22399 6334 Acked Read 10 Done Done Sent false
22400 6335 Acked Read 8 Done Done Sent false
22401 6336 Acked Flush 0 Done Done Sent false
22402 6337 Acked Read 2 Done Done Sent false
22403 6338 Acked Read 5 Done Done Sent false
22404 6339 Acked Write 1 Done Done Sent false
22405 6340 Acked Flush 0 Done Done Sent false
22406 6341 Acked Read 4 Done Done Sent false
22407 6342 Acked Read 5 Done Done Sent false
22408 6343 Acked Write 7 Done Done Sent false
22409 6344 Acked Read 9 Done Done Sent false
22410 6345 Acked Write 6 Done Done Sent false
22411 6346 Acked Read 1 Done Done Sent false
22412 6347 Acked Write 3 Done Done Sent false
22413 6348 Acked Read 10 Done Done Sent false
22414 6349 Acked Flush 0 Done Done Sent false
22415 6350 Acked Write 10 Done Done Sent false
22416 6351 Acked Read 3 Done Done Sent false
22417 6352 Acked Write 4 Done Done Sent false
22418 6353 Acked Read 10 Done Done Sent false
22419 6354 Acked Read 4 Done Done Sent false
22420 6355 Acked Write 9 Done Done Sent false
22421 6356 Acked Read 8 Done Done Sent false
22422 6357 Acked Write 10 Done Done Sent false
22423 6358 Acked Read 8 Done Done Sent false
22424 6359 Acked Read 7 Done Done Sent false
22425 6360 Acked Write 2 Done Done Sent false
22426 6361 Acked Read 9 Done Done Sent false
22427 6362 Acked Write 2 Done Done Sent false
22428 6363 Acked Read 4 Done Done Sent false
22429 6364 Acked Write 1 Done Done Sent false
22430 6365 Acked Write 10 Done Done Sent false
22431 6366 Acked Read 7 Done Done Sent false
22432 6367 Acked Write 1 Done Done Sent false
22433 6368 Acked Read 1 Done Done Sent false
22434 6369 Acked Read 6 Done Done Sent false
22435 6370 Acked Read 10 Done Done Sent false
22436 6371 Acked Read 5 Done Done Sent false
22437 6372 Acked Flush 0 Done Done Sent false
22438 6373 Acked Write 7 Done Done Sent false
22439 6374 Acked Flush 0 Done Done Sent false
22440 6375 Acked Write 7 Done Done Sent false
22441 6376 Acked Read 6 Done Done Sent false
22442 6377 Acked Write 9 Done Done Sent false
22443 6378 Acked Write 6 Done Done Sent false
22444 6379 Acked Read 7 Done Done Sent false
22445 6380 Acked Read 7 Done Done Sent false
22446 6381 Acked Write 5 Done Done Sent false
22447 6382 Acked Read 4 Done Done Sent false
22448 6383 Acked Flush 0 Done Done Sent false
22449 6384 Acked Read 10 Done Done Sent false
22450 6385 Acked Read 5 Done Done Sent false
22451 6386 Acked Write 8 Done Done Sent false
22452 6387 Acked Write 4 Done Done Sent false
22453 6388 Acked Read 6 Done Done Sent false
22454 6389 Acked Flush 0 Done Done Sent false
22455 6390 Acked Write 6 Done Done Sent false
22456 6391 Acked Read 10 Done Done Sent false
22457 6392 Acked Write 7 Done Done Sent false
22458 6393 Acked Flush 0 Done Done Sent false
22459 6394 Acked Write 4 Done Done Sent false
22460 6395 Acked Write 10 Done Done Sent false
22461 6396 Acked Write 7 Done Done Sent false
22462 6397 Acked Read 3 Done Done Sent false
22463 6398 Acked Write 7 Done Done Sent false
22464 6399 Acked Write 6 Done Done Sent false
22465 6400 Acked Write 10 Done Done Sent false
22466 6401 Acked Read 9 Done Done Sent false
22467 6402 Acked Write 5 Done Done Sent false
22468 6403 Acked Read 3 Done Done Sent false
22469 6404 Acked Read 10 Done Done Sent false
22470 6405 Acked Write 6 Done Done Sent false
22471 6406 Acked Read 9 Done Done Sent false
22472 6407 Acked Read 4 Done Done Sent false
22473 6408 Acked Write 9 Done Done Sent false
22474 6409 Acked Flush 0 Done Done Sent false
22475 6410 Acked Write 2 Done Done Sent false
22476 6411 Acked Read 7 Done Done Sent false
22477 6412 Acked Read 9 Done Done Sent false
22478 6413 Acked Write 1 Done Done Sent false
22479 6414 Acked Read 2 Done Done Sent false
22480 6415 Acked Read 1 Done Done Sent false
22481 6416 Acked Write 4 Done Done Sent false
22482 6417 Acked Read 8 Done Done Sent false
22483 6418 Acked Read 3 Done Done Sent false
22484 6419 Acked Read 9 Done Done Sent false
22485 6420 Acked Read 1 Done Done Sent false
22486 6421 Acked Flush 0 Done Done Sent false
22487 6422 Acked Read 5 Done Done Sent false
22488 6423 Acked Read 7 Done Done Sent false
22489 6424 Acked Read 10 Done Done Sent false
22490 6425 Acked Write 7 Done Done Sent false
22491 6426 Acked Read 10 Done Done Sent false
22492 6427 Acked Read 9 Done Done Sent false
22493 6428 Acked Read 9 Done Done Sent false
22494 6429 Acked Read 5 Done Done Sent false
22495 6430 Acked Write 7 Done Done Sent false
22496 6431 Acked Read 10 Done Done Sent false
22497 6432 Acked Read 7 Done Done Sent false
22498 STATES DS:0 DS:1 DS:2 TOTAL
22499 Sent 0 0 301 301
22500 Done 301 301 0 602
22501 Skipped 0 0 0 0
22502 Error 0 0 0 0
22503 Last Flush: 6421 6421 6131
22504 Downstairs last five completed: 6131 6130 6129 6128 6127
22505 Upstairs last five completed: 6432 6431 6430 6429 6428
22506 CLIENT: Up:0 ds:301 act:2
22507 ----------------------------------------------------------------
22508 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
22509 States: Active Active Offline
22510 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
22511 6132 Acked Read 9 Done Done Sent false
22512 6133 Acked Write 3 Done Done Sent false
22513 6134 Acked Read 7 Done Done Sent false
22514 6135 Acked Write 1 Done Done Sent false
22515 6136 Acked Read 10 Done Done Sent false
22516 6137 Acked Write 7 Done Done Sent false
22517 6138 Acked Write 10 Done Done Sent false
22518 6139 Acked Flush 0 Done Done Sent false
22519 6140 Acked Flush 0 Done Done Sent false
22520 6141 Acked Write 6 Done Done Sent false
22521 6142 Acked Write 9 Done Done Sent false
22522 6143 Acked Write 6 Done Done Sent false
22523 6144 Acked Flush 0 Done Done Sent false
22524 6145 Acked Read 5 Done Done Sent false
22525 6146 Acked Write 4 Done Done Sent false
22526 6147 Acked Write 4 Done Done Sent false
22527 6148 Acked Write 8 Done Done Sent false
22528 6149 Acked Read 9 Done Done Sent false
22529 6150 Acked Read 5 Done Done Sent false
22530 6151 Acked Write 10 Done Done Sent false
22531 6152 Acked Write 1 Done Done Sent false
22532 6153 Acked Write 1 Done Done Sent false
22533 6154 Acked Write 4 Done Done Sent false
22534 6155 Acked Read 1 Done Done Sent false
22535 6156 Acked Write 2 Done Done Sent false
22536 6157 Acked Flush 0 Done Done Sent false
22537 6158 Acked Write 7 Done Done Sent false
22538 6159 Acked Flush 0 Done Done Sent false
22539 6160 Acked Read 9 Done Done Sent false
22540 6161 Acked Write 5 Done Done Sent false
22541 6162 Acked Flush 0 Done Done Sent false
22542 6163 Acked Write 4 Done Done Sent false
22543 6164 Acked Write 6 Done Done Sent false
22544 6165 Acked Read 7 Done Done Sent false
22545 6166 Acked Write 9 Done Done Sent false
22546 6167 Acked Read 10 Done Done Sent false
22547 6168 Acked Read 10 Done Done Sent false
22548 6169 Acked Read 8 Done Done Sent false
22549 6170 Acked Read 2 Done Done Sent false
22550 6171 Acked Read 3 Done Done Sent false
22551 6172 Acked Read 3 Done Done Sent false
22552 6173 Acked Write 4 Done Done Sent false
22553 6174 Acked Write 9 Done Done Sent false
22554 6175 Acked Read 6 Done Done Sent false
22555 6176 Acked Write 10 Done Done Sent false
22556 6177 Acked Flush 0 Done Done Sent false
22557 6178 Acked Write 9 Done Done Sent false
22558 6179 Acked Write 3 Done Done Sent false
22559 6180 Acked Read 5 Done Done Sent false
22560 6181 Acked Read 9 Done Done Sent false
22561 6182 Acked Write 3 Done Done Sent false
22562 6183 Acked Write 7 Done Done Sent false
22563 6184 Acked Read 3 Done Done Sent false
22564 6185 Acked Read 4 Done Done Sent false
22565 6186 Acked Read 7 Done Done Sent false
22566 6187 Acked Flush 0 Done Done Sent false
22567 6188 Acked Write 2 Done Done Sent false
22568 6189 Acked Read 7 Done Done Sent false
22569 6190 Acked Write 6 Done Done Sent false
22570 6191 Acked Read 2 Done Done Sent false
22571 6192 Acked Read 2 Done Done Sent false
22572 6193 Acked Read 5 Done Done Sent false
22573 6194 Acked Read 8 Done Done Sent false
22574 6195 Acked Write 6 Done Done Sent false
22575 6196 Acked Read 8 Done Done Sent false
22576 6197 Acked Read 5 Done Done Sent false
22577 6198 Acked Write 2 Done Done Sent false
22578 6199 Acked Flush 0 Done Done Sent false
22579 6200 Acked Read 9 Done Done Sent false
22580 6201 Acked Read 4 Done Done Sent false
22581 6202 Acked Read 10 Done Done Sent false
22582 6203 Acked Read 4 Done Done Sent false
22583 6204 Acked Read 5 Done Done Sent false
22584 6205 Acked Read 4 Done Done Sent false
22585 6206 Acked Read 3 Done Done Sent false
22586 6207 Acked Read 2 Done Done Sent false
22587 6208 Acked Read 8 Done Done Sent false
22588 6209 Acked Read 4 Done Done Sent false
22589 6210 Acked Read 5 Done Done Sent false
22590 6211 Acked Write 2 Done Done Sent false
22591 6212 Acked Write 4 Done Done Sent false
22592 6213 Acked Write 8 Done Done Sent false
22593 6214 Acked Write 5 Done Done Sent false
22594 6215 Acked Write 4 Done Done Sent false
22595 6216 Acked Read 1 Done Done Sent false
22596 6217 Acked Read 7 Done Done Sent false
22597 6218 Acked Read 8 Done Done Sent false
22598 6219 Acked Write 5 Done Done Sent false
22599 6220 Acked Write 5 Done Done Sent false
22600 6221 Acked Read 9 Done Done Sent false
22601 6222 Acked Read 5 Done Done Sent false
22602 6223 Acked Write 2 Done Done Sent false
22603 6224 Acked Write 9 Done Done Sent false
22604 6225 Acked Read 1 Done Done Sent false
22605 6226 Acked Write 2 Done Done Sent false
22606 6227 Acked Write 9 Done Done Sent false
22607 6228 Acked Read 7 Done Done Sent false
22608 6229 Acked Read 6 Done Done Sent false
22609 6230 Acked Write 6 Done Done Sent false
22610 6231 Acked Read 3 Done Done Sent false
22611 6232 Acked Read 6 Done Done Sent false
22612 6233 Acked Read 3 Done Done Sent false
22613 6234 Acked Flush 0 Done Done Sent false
22614 6235 Acked Flush 0 Done Done Sent false
22615 6236 Acked Read 7 Done Done Sent false
22616 6237 Acked Write 5 Done Done Sent false
22617 6238 Acked Read 8 Done Done Sent false
22618 6239 Acked Write 9 Done Done Sent false
22619 6240 Acked Write 5 Done Done Sent false
22620 6241 Acked Read 2 Done Done Sent false
22621 6242 Acked Write 10 Done Done Sent false
22622 6243 Acked Read 8 Done Done Sent false
22623 6244 Acked Flush 0 Done Done Sent false
22624 6245 Acked Write 3 Done Done Sent false
22625 6246 Acked Read 7 Done Done Sent false
22626 6247 Acked Write 5 Done Done Sent false
22627 6248 Acked Write 9 Done Done Sent false
22628 6249 Acked Write 8 Done Done Sent false
22629 6250 Acked Read 4 Done Done Sent false
22630 6251 Acked Flush 0 Done Done Sent false
22631 6252 Acked Read 7 Done Done Sent false
22632 6253 Acked Write 6 Done Done Sent false
22633 6254 Acked Write 9 Done Done Sent false
22634 6255 Acked Flush 0 Done Done Sent false
22635 6256 Acked Read 5 Done Done Sent false
22636 6257 Acked Read 5 Done Done Sent false
22637 6258 Acked Read 3 Done Done Sent false
22638 6259 Acked Read 1 Done Done Sent false
22639 6260 Acked Write 3 Done Done Sent false
22640 6261 Acked Read 5 Done Done Sent false
22641 6262 Acked Read 4 Done Done Sent false
22642 6263 Acked Read 6 Done Done Sent false
22643 6264 Acked Read 1 Done Done Sent false
22644 6265 Acked Read 10 Done Done Sent false
22645 6266 Acked Read 4 Done Done Sent false
22646 6267 Acked Flush 0 Done Done Sent false
22647 6268 Acked Read 5 Done Done Sent false
22648 6269 Acked Read 7 Done Done Sent false
22649 6270 Acked Read 4 Done Done Sent false
22650 6271 Acked Read 9 Done Done Sent false
22651 6272 Acked Read 5 Done Done Sent false
22652 6273 Acked Read 7 Done Done Sent false
22653 6274 Acked Write 1 Done Done Sent false
22654 6275 Acked Read 5 Done Done Sent false
22655 6276 Acked Read 4 Done Done Sent false
22656 6277 Acked Write 8 Done Done Sent false
22657 6278 Acked Read 4 Done Done Sent false
22658 6279 Acked Write 3 Done Done Sent false
22659 6280 Acked Read 8 Done Done Sent false
22660 6281 Acked Read 4 Done Done Sent false
22661 6282 Acked Read 9 Done Done Sent false
22662 6283 Acked Read 1 Done Done Sent false
22663 6284 Acked Read 5 Done Done Sent false
22664 6285 Acked Read 9 Done Done Sent false
22665 6286 Acked Write 7 Done Done Sent false
22666 6287 Acked Write 9 Done Done Sent false
22667 6288 Acked Read 10 Done Done Sent false
22668 6289 Acked Write 10 Done Done Sent false
22669 6290 Acked Write 6 Done Done Sent false
22670 6291 Acked Write 6 Done Done Sent false
22671 6292 Acked Flush 0 Done Done Sent false
22672 6293 Acked Read 9 Done Done Sent false
22673 6294 Acked Flush 0 Done Done Sent false
22674 6295 Acked Read 2 Done Done Sent false
22675 6296 Acked Write 6 Done Done Sent false
22676 6297 Acked Read 1 Done Done Sent false
22677 6298 Acked Write 7 Done Done Sent false
22678 6299 Acked Write 2 Done Done Sent false
22679 6300 Acked Read 5 Done Done Sent false
22680 6301 Acked Flush 0 Done Done Sent false
22681 6302 Acked Write 1 Done Done Sent false
22682 6303 Acked Read 7 Done Done Sent false
22683 6304 Acked Read 8 Done Done Sent false
22684 6305 Acked Read 10 Done Done Sent false
22685 6306 Acked Write 3 Done Done Sent false
22686 6307 Acked Read 7 Done Done Sent false
22687 6308 Acked Write 8 Done Done Sent false
22688 6309 Acked Write 3 Done Done Sent false
22689 6310 Acked Flush 0 Done Done Sent false
22690 6311 Acked Write 9 Done Done Sent false
22691 6312 Acked Read 10 Done Done Sent false
22692 6313 Acked Read 1 Done Done Sent false
22693 6314 Acked Write 9 Done Done Sent false
22694 6315 Acked Read 6 Done Done Sent false
22695 6316 Acked Read 4 Done Done Sent false
22696 6317 Acked Write 7 Done Done Sent false
22697 6318 Acked Write 3 Done Done Sent false
22698 6319 Acked Read 1 Done Done Sent false
22699 6320 Acked Read 2 Done Done Sent false
22700 6321 Acked Read 6 Done Done Sent false
22701 6322 Acked Read 5 Done Done Sent false
22702 6323 Acked Read 6 Done Done Sent false
22703 6324 Acked Read 10 Done Done Sent false
22704 6325 Acked Write 9 Done Done Sent false
22705 6326 Acked Read 6 Done Done Sent false
22706 6327 Acked Read 1 Done Done Sent false
22707 6328 Acked Read 3 Done Done Sent false
22708 6329 Acked Write 1 Done Done Sent false
22709 6330 Acked Read 1 Done Done Sent false
22710 6331 Acked Read 7 Done Done Sent false
22711 6332 Acked Read 7 Done Done Sent false
22712 6333 Acked Read 4 Done Done Sent false
22713 6334 Acked Read 10 Done Done Sent false
22714 6335 Acked Read 8 Done Done Sent false
22715 6336 Acked Flush 0 Done Done Sent false
22716 6337 Acked Read 2 Done Done Sent false
22717 6338 Acked Read 5 Done Done Sent false
22718 6339 Acked Write 1 Done Done Sent false
22719 6340 Acked Flush 0 Done Done Sent false
22720 6341 Acked Read 4 Done Done Sent false
22721 6342 Acked Read 5 Done Done Sent false
22722 6343 Acked Write 7 Done Done Sent false
22723 6344 Acked Read 9 Done Done Sent false
22724 6345 Acked Write 6 Done Done Sent false
22725 6346 Acked Read 1 Done Done Sent false
22726 6347 Acked Write 3 Done Done Sent false
22727 6348 Acked Read 10 Done Done Sent false
22728 6349 Acked Flush 0 Done Done Sent false
22729 6350 Acked Write 10 Done Done Sent false
22730 6351 Acked Read 3 Done Done Sent false
22731 6352 Acked Write 4 Done Done Sent false
22732 6353 Acked Read 10 Done Done Sent false
22733 6354 Acked Read 4 Done Done Sent false
22734 6355 Acked Write 9 Done Done Sent false
22735 6356 Acked Read 8 Done Done Sent false
22736 6357 Acked Write 10 Done Done Sent false
22737 6358 Acked Read 8 Done Done Sent false
22738 6359 Acked Read 7 Done Done Sent false
22739 6360 Acked Write 2 Done Done Sent false
22740 6361 Acked Read 9 Done Done Sent false
22741 6362 Acked Write 2 Done Done Sent false
22742 6363 Acked Read 4 Done Done Sent false
22743 6364 Acked Write 1 Done Done Sent false
22744 6365 Acked Write 10 Done Done Sent false
22745 6366 Acked Read 7 Done Done Sent false
22746 6367 Acked Write 1 Done Done Sent false
22747 6368 Acked Read 1 Done Done Sent false
22748 6369 Acked Read 6 Done Done Sent false
22749 6370 Acked Read 10 Done Done Sent false
22750 6371 Acked Read 5 Done Done Sent false
22751 6372 Acked Flush 0 Done Done Sent false
22752 6373 Acked Write 7 Done Done Sent false
22753 6374 Acked Flush 0 Done Done Sent false
22754 6375 Acked Write 7 Done Done Sent false
22755 6376 Acked Read 6 Done Done Sent false
22756 6377 Acked Write 9 Done Done Sent false
22757 6378 Acked Write 6 Done Done Sent false
22758 6379 Acked Read 7 Done Done Sent false
22759 6380 Acked Read 7 Done Done Sent false
22760 6381 Acked Write 5 Done Done Sent false
22761 6382 Acked Read 4 Done Done Sent false
22762 6383 Acked Flush 0 Done Done Sent false
22763 6384 Acked Read 10 Done Done Sent false
22764 6385 Acked Read 5 Done Done Sent false
22765 6386 Acked Write 8 Done Done Sent false
22766 6387 Acked Write 4 Done Done Sent false
22767 6388 Acked Read 6 Done Done Sent false
22768 6389 Acked Flush 0 Done Done Sent false
22769 6390 Acked Write 6 Done Done Sent false
22770 6391 Acked Read 10 Done Done Sent false
22771 6392 Acked Write 7 Done Done Sent false
22772 6393 Acked Flush 0 Done Done Sent false
22773 6394 Acked Write 4 Done Done Sent false
22774 6395 Acked Write 10 Done Done Sent false
22775 6396 Acked Write 7 Done Done Sent false
22776 6397 Acked Read 3 Done Done Sent false
22777 6398 Acked Write 7 Done Done Sent false
22778 6399 Acked Write 6 Done Done Sent false
22779 6400 Acked Write 10 Done Done Sent false
22780 6401 Acked Read 9 Done Done Sent false
22781 6402 Acked Write 5 Done Done Sent false
22782 6403 Acked Read 3 Done Done Sent false
22783 6404 Acked Read 10 Done Done Sent false
22784 6405 Acked Write 6 Done Done Sent false
22785 6406 Acked Read 9 Done Done Sent false
22786 6407 Acked Read 4 Done Done Sent false
22787 6408 Acked Write 9 Done Done Sent false
22788 6409 Acked Flush 0 Done Done Sent false
22789 6410 Acked Write 2 Done Done Sent false
22790 6411 Acked Read 7 Done Done Sent false
22791 6412 Acked Read 9 Done Done Sent false
22792 6413 Acked Write 1 Done Done Sent false
22793 6414 Acked Read 2 Done Done Sent false
22794 6415 Acked Read 1 Done Done Sent false
22795 6416 Acked Write 4 Done Done Sent false
22796 6417 Acked Read 8 Done Done Sent false
22797 6418 Acked Read 3 Done Done Sent false
22798 6419 Acked Read 9 Done Done Sent false
22799 6420 Acked Read 1 Done Done Sent false
22800 6421 Acked Flush 0 Done Done Sent false
22801 6422 Acked Read 5 Done Done Sent false
22802 6423 Acked Read 7 Done Done Sent false
22803 6424 Acked Read 10 Done Done Sent false
22804 6425 Acked Write 7 Done Done Sent false
22805 6426 Acked Read 10 Done Done Sent false
22806 6427 Acked Read 9 Done Done Sent false
22807 6428 Acked Read 9 Done Done Sent false
22808 6429 Acked Read 5 Done Done Sent false
22809 6430 Acked Write 7 Done Done Sent false
22810 6431 Acked Read 10 Done Done Sent false
22811 6432 Acked Read 7 Done Done Sent false
22812 6433 Acked Flush 0 Done Done Sent false
22813 STATES DS:0 DS:1 DS:2 TOTAL
22814 Sent 0 0 302 302
22815 Done 302 302 0 604
22816 Skipped 0 0 0 0
22817 Error 0 0 0 0
22818 Last Flush: 6433 6433 6131
22819 Downstairs last five completed: 6131 6130 6129 6128 6127
22820 Upstairs last five completed: 6433 6432 6431 6430 6429
22821 CLIENT: Up:0 ds:302 act:2
228222024-11-15T02:02:14.076ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
228232024-11-15T02:02:14.076ZWARNcrucible: ds_connection connect to 127.0.0.1:8830 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
228242024-11-15T02:02:14.076ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
228252024-11-15T02:02:14.076ZWARNcrucible: downstairs task for 2 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
228262024-11-15T02:02:14.076ZINFOcrucible: client task is exiting = io task client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
22827 [8830] got start action
22828 [8830] I am going to restart
22829 Starting downstairs at port 8830
22830 Make output file at "/tmp/dsc/downstairs-8830.txt"
22831 [8830][2] reports Starting
22832 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1223)
22833 [8830][2] reports Running
22834 ----------------------------------------------------------------
22835 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
22836 States: Active Active Offline
22837 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
22838 6132 Acked Read 9 Done Done Sent false
22839 6133 Acked Write 3 Done Done Sent false
22840 6134 Acked Read 7 Done Done Sent false
22841 6135 Acked Write 1 Done Done Sent false
22842 6136 Acked Read 10 Done Done Sent false
22843 6137 Acked Write 7 Done Done Sent false
22844 6138 Acked Write 10 Done Done Sent false
22845 6139 Acked Flush 0 Done Done Sent false
22846 6140 Acked Flush 0 Done Done Sent false
22847 6141 Acked Write 6 Done Done Sent false
22848 6142 Acked Write 9 Done Done Sent false
22849 6143 Acked Write 6 Done Done Sent false
22850 6144 Acked Flush 0 Done Done Sent false
22851 6145 Acked Read 5 Done Done Sent false
22852 6146 Acked Write 4 Done Done Sent false
22853 6147 Acked Write 4 Done Done Sent false
22854 6148 Acked Write 8 Done Done Sent false
22855 6149 Acked Read 9 Done Done Sent false
22856 6150 Acked Read 5 Done Done Sent false
22857 6151 Acked Write 10 Done Done Sent false
22858 6152 Acked Write 1 Done Done Sent false
22859 6153 Acked Write 1 Done Done Sent false
22860 6154 Acked Write 4 Done Done Sent false
22861 6155 Acked Read 1 Done Done Sent false
22862 6156 Acked Write 2 Done Done Sent false
22863 6157 Acked Flush 0 Done Done Sent false
22864 6158 Acked Write 7 Done Done Sent false
22865 6159 Acked Flush 0 Done Done Sent false
22866 6160 Acked Read 9 Done Done Sent false
22867 6161 Acked Write 5 Done Done Sent false
22868 6162 Acked Flush 0 Done Done Sent false
22869 6163 Acked Write 4 Done Done Sent false
22870 6164 Acked Write 6 Done Done Sent false
22871 6165 Acked Read 7 Done Done Sent false
22872 6166 Acked Write 9 Done Done Sent false
22873 6167 Acked Read 10 Done Done Sent false
22874 6168 Acked Read 10 Done Done Sent false
22875 6169 Acked Read 8 Done Done Sent false
22876 6170 Acked Read 2 Done Done Sent false
22877 6171 Acked Read 3 Done Done Sent false
22878 6172 Acked Read 3 Done Done Sent false
22879 6173 Acked Write 4 Done Done Sent false
22880 6174 Acked Write 9 Done Done Sent false
22881 6175 Acked Read 6 Done Done Sent false
22882 6176 Acked Write 10 Done Done Sent false
22883 6177 Acked Flush 0 Done Done Sent false
22884 6178 Acked Write 9 Done Done Sent false
22885 6179 Acked Write 3 Done Done Sent false
22886 6180 Acked Read 5 Done Done Sent false
22887 6181 Acked Read 9 Done Done Sent false
22888 6182 Acked Write 3 Done Done Sent false
22889 6183 Acked Write 7 Done Done Sent false
22890 6184 Acked Read 3 Done Done Sent false
22891 6185 Acked Read 4 Done Done Sent false
22892 6186 Acked Read 7 Done Done Sent false
22893 6187 Acked Flush 0 Done Done Sent false
22894 6188 Acked Write 2 Done Done Sent false
22895 6189 Acked Read 7 Done Done Sent false
22896 6190 Acked Write 6 Done Done Sent false
22897 6191 Acked Read 2 Done Done Sent false
22898 6192 Acked Read 2 Done Done Sent false
22899 6193 Acked Read 5 Done Done Sent false
22900 6194 Acked Read 8 Done Done Sent false
22901 6195 Acked Write 6 Done Done Sent false
22902 6196 Acked Read 8 Done Done Sent false
22903 6197 Acked Read 5 Done Done Sent false
22904 6198 Acked Write 2 Done Done Sent false
22905 6199 Acked Flush 0 Done Done Sent false
22906 6200 Acked Read 9 Done Done Sent false
22907 6201 Acked Read 4 Done Done Sent false
22908 6202 Acked Read 10 Done Done Sent false
22909 6203 Acked Read 4 Done Done Sent false
22910 6204 Acked Read 5 Done Done Sent false
22911 6205 Acked Read 4 Done Done Sent false
22912 6206 Acked Read 3 Done Done Sent false
22913 6207 Acked Read 2 Done Done Sent false
22914 6208 Acked Read 8 Done Done Sent false
22915 6209 Acked Read 4 Done Done Sent false
22916 6210 Acked Read 5 Done Done Sent false
22917 6211 Acked Write 2 Done Done Sent false
22918 6212 Acked Write 4 Done Done Sent false
22919 6213 Acked Write 8 Done Done Sent false
22920 6214 Acked Write 5 Done Done Sent false
22921 6215 Acked Write 4 Done Done Sent false
22922 6216 Acked Read 1 Done Done Sent false
22923 6217 Acked Read 7 Done Done Sent false
22924 6218 Acked Read 8 Done Done Sent false
22925 6219 Acked Write 5 Done Done Sent false
22926 6220 Acked Write 5 Done Done Sent false
22927 6221 Acked Read 9 Done Done Sent false
22928 6222 Acked Read 5 Done Done Sent false
22929 6223 Acked Write 2 Done Done Sent false
22930 6224 Acked Write 9 Done Done Sent false
22931 6225 Acked Read 1 Done Done Sent false
22932 6226 Acked Write 2 Done Done Sent false
22933 6227 Acked Write 9 Done Done Sent false
22934 6228 Acked Read 7 Done Done Sent false
22935 6229 Acked Read 6 Done Done Sent false
22936 6230 Acked Write 6 Done Done Sent false
22937 6231 Acked Read 3 Done Done Sent false
22938 6232 Acked Read 6 Done Done Sent false
22939 6233 Acked Read 3 Done Done Sent false
22940 6234 Acked Flush 0 Done Done Sent false
22941 6235 Acked Flush 0 Done Done Sent false
22942 6236 Acked Read 7 Done Done Sent false
22943 6237 Acked Write 5 Done Done Sent false
22944 6238 Acked Read 8 Done Done Sent false
22945 6239 Acked Write 9 Done Done Sent false
22946 6240 Acked Write 5 Done Done Sent false
22947 6241 Acked Read 2 Done Done Sent false
22948 6242 Acked Write 10 Done Done Sent false
22949 6243 Acked Read 8 Done Done Sent false
22950 6244 Acked Flush 0 Done Done Sent false
22951 6245 Acked Write 3 Done Done Sent false
22952 6246 Acked Read 7 Done Done Sent false
22953 6247 Acked Write 5 Done Done Sent false
22954 6248 Acked Write 9 Done Done Sent false
22955 6249 Acked Write 8 Done Done Sent false
22956 6250 Acked Read 4 Done Done Sent false
22957 6251 Acked Flush 0 Done Done Sent false
22958 6252 Acked Read 7 Done Done Sent false
22959 6253 Acked Write 6 Done Done Sent false
22960 6254 Acked Write 9 Done Done Sent false
22961 6255 Acked Flush 0 Done Done Sent false
22962 6256 Acked Read 5 Done Done Sent false
22963 6257 Acked Read 5 Done Done Sent false
22964 6258 Acked Read 3 Done Done Sent false
22965 6259 Acked Read 1 Done Done Sent false
22966 6260 Acked Write 3 Done Done Sent false
22967 6261 Acked Read 5 Done Done Sent false
22968 6262 Acked Read 4 Done Done Sent false
22969 6263 Acked Read 6 Done Done Sent false
22970 6264 Acked Read 1 Done Done Sent false
22971 6265 Acked Read 10 Done Done Sent false
22972 6266 Acked Read 4 Done Done Sent false
22973 6267 Acked Flush 0 Done Done Sent false
22974 6268 Acked Read 5 Done Done Sent false
22975 6269 Acked Read 7 Done Done Sent false
22976 6270 Acked Read 4 Done Done Sent false
22977 6271 Acked Read 9 Done Done Sent false
22978 6272 Acked Read 5 Done Done Sent false
22979 6273 Acked Read 7 Done Done Sent false
22980 6274 Acked Write 1 Done Done Sent false
22981 6275 Acked Read 5 Done Done Sent false
22982 6276 Acked Read 4 Done Done Sent false
22983 6277 Acked Write 8 Done Done Sent false
22984 6278 Acked Read 4 Done Done Sent false
22985 6279 Acked Write 3 Done Done Sent false
22986 6280 Acked Read 8 Done Done Sent false
22987 6281 Acked Read 4 Done Done Sent false
22988 6282 Acked Read 9 Done Done Sent false
22989 6283 Acked Read 1 Done Done Sent false
22990 6284 Acked Read 5 Done Done Sent false
22991 6285 Acked Read 9 Done Done Sent false
22992 6286 Acked Write 7 Done Done Sent false
22993 6287 Acked Write 9 Done Done Sent false
22994 6288 Acked Read 10 Done Done Sent false
22995 6289 Acked Write 10 Done Done Sent false
22996 6290 Acked Write 6 Done Done Sent false
22997 6291 Acked Write 6 Done Done Sent false
22998 6292 Acked Flush 0 Done Done Sent false
22999 6293 Acked Read 9 Done Done Sent false
23000 6294 Acked Flush 0 Done Done Sent false
23001 6295 Acked Read 2 Done Done Sent false
23002 6296 Acked Write 6 Done Done Sent false
23003 6297 Acked Read 1 Done Done Sent false
23004 6298 Acked Write 7 Done Done Sent false
23005 6299 Acked Write 2 Done Done Sent false
23006 6300 Acked Read 5 Done Done Sent false
23007 6301 Acked Flush 0 Done Done Sent false
23008 6302 Acked Write 1 Done Done Sent false
23009 6303 Acked Read 7 Done Done Sent false
23010 6304 Acked Read 8 Done Done Sent false
23011 6305 Acked Read 10 Done Done Sent false
23012 6306 Acked Write 3 Done Done Sent false
23013 6307 Acked Read 7 Done Done Sent false
23014 6308 Acked Write 8 Done Done Sent false
23015 6309 Acked Write 3 Done Done Sent false
23016 6310 Acked Flush 0 Done Done Sent false
23017 6311 Acked Write 9 Done Done Sent false
23018 6312 Acked Read 10 Done Done Sent false
23019 6313 Acked Read 1 Done Done Sent false
23020 6314 Acked Write 9 Done Done Sent false
23021 6315 Acked Read 6 Done Done Sent false
23022 6316 Acked Read 4 Done Done Sent false
23023 6317 Acked Write 7 Done Done Sent false
23024 6318 Acked Write 3 Done Done Sent false
23025 6319 Acked Read 1 Done Done Sent false
23026 6320 Acked Read 2 Done Done Sent false
23027 6321 Acked Read 6 Done Done Sent false
23028 6322 Acked Read 5 Done Done Sent false
23029 6323 Acked Read 6 Done Done Sent false
23030 6324 Acked Read 10 Done Done Sent false
23031 6325 Acked Write 9 Done Done Sent false
23032 6326 Acked Read 6 Done Done Sent false
23033 6327 Acked Read 1 Done Done Sent false
23034 6328 Acked Read 3 Done Done Sent false
23035 6329 Acked Write 1 Done Done Sent false
23036 6330 Acked Read 1 Done Done Sent false
23037 6331 Acked Read 7 Done Done Sent false
23038 6332 Acked Read 7 Done Done Sent false
23039 6333 Acked Read 4 Done Done Sent false
23040 6334 Acked Read 10 Done Done Sent false
23041 6335 Acked Read 8 Done Done Sent false
23042 6336 Acked Flush 0 Done Done Sent false
23043 6337 Acked Read 2 Done Done Sent false
23044 6338 Acked Read 5 Done Done Sent false
23045 6339 Acked Write 1 Done Done Sent false
23046 6340 Acked Flush 0 Done Done Sent false
23047 6341 Acked Read 4 Done Done Sent false
23048 6342 Acked Read 5 Done Done Sent false
23049 6343 Acked Write 7 Done Done Sent false
23050 6344 Acked Read 9 Done Done Sent false
23051 6345 Acked Write 6 Done Done Sent false
23052 6346 Acked Read 1 Done Done Sent false
23053 6347 Acked Write 3 Done Done Sent false
23054 6348 Acked Read 10 Done Done Sent false
23055 6349 Acked Flush 0 Done Done Sent false
23056 6350 Acked Write 10 Done Done Sent false
23057 6351 Acked Read 3 Done Done Sent false
23058 6352 Acked Write 4 Done Done Sent false
23059 6353 Acked Read 10 Done Done Sent false
23060 6354 Acked Read 4 Done Done Sent false
23061 6355 Acked Write 9 Done Done Sent false
23062 6356 Acked Read 8 Done Done Sent false
23063 6357 Acked Write 10 Done Done Sent false
23064 6358 Acked Read 8 Done Done Sent false
23065 6359 Acked Read 7 Done Done Sent false
23066 6360 Acked Write 2 Done Done Sent false
23067 6361 Acked Read 9 Done Done Sent false
23068 6362 Acked Write 2 Done Done Sent false
23069 6363 Acked Read 4 Done Done Sent false
23070 6364 Acked Write 1 Done Done Sent false
23071 6365 Acked Write 10 Done Done Sent false
23072 6366 Acked Read 7 Done Done Sent false
23073 6367 Acked Write 1 Done Done Sent false
23074 6368 Acked Read 1 Done Done Sent false
23075 6369 Acked Read 6 Done Done Sent false
23076 6370 Acked Read 10 Done Done Sent false
23077 6371 Acked Read 5 Done Done Sent false
23078 6372 Acked Flush 0 Done Done Sent false
23079 6373 Acked Write 7 Done Done Sent false
23080 6374 Acked Flush 0 Done Done Sent false
23081 6375 Acked Write 7 Done Done Sent false
23082 6376 Acked Read 6 Done Done Sent false
23083 6377 Acked Write 9 Done Done Sent false
23084 6378 Acked Write 6 Done Done Sent false
23085 6379 Acked Read 7 Done Done Sent false
23086 6380 Acked Read 7 Done Done Sent false
23087 6381 Acked Write 5 Done Done Sent false
23088 6382 Acked Read 4 Done Done Sent false
23089 6383 Acked Flush 0 Done Done Sent false
23090 6384 Acked Read 10 Done Done Sent false
23091 6385 Acked Read 5 Done Done Sent false
23092 6386 Acked Write 8 Done Done Sent false
23093 6387 Acked Write 4 Done Done Sent false
23094 6388 Acked Read 6 Done Done Sent false
23095 6389 Acked Flush 0 Done Done Sent false
23096 6390 Acked Write 6 Done Done Sent false
23097 6391 Acked Read 10 Done Done Sent false
23098 6392 Acked Write 7 Done Done Sent false
23099 6393 Acked Flush 0 Done Done Sent false
23100 6394 Acked Write 4 Done Done Sent false
23101 6395 Acked Write 10 Done Done Sent false
23102 6396 Acked Write 7 Done Done Sent false
23103 6397 Acked Read 3 Done Done Sent false
23104 6398 Acked Write 7 Done Done Sent false
23105 6399 Acked Write 6 Done Done Sent false
23106 6400 Acked Write 10 Done Done Sent false
23107 6401 Acked Read 9 Done Done Sent false
23108 6402 Acked Write 5 Done Done Sent false
23109 6403 Acked Read 3 Done Done Sent false
23110 6404 Acked Read 10 Done Done Sent false
23111 6405 Acked Write 6 Done Done Sent false
23112 6406 Acked Read 9 Done Done Sent false
23113 6407 Acked Read 4 Done Done Sent false
23114 6408 Acked Write 9 Done Done Sent false
23115 6409 Acked Flush 0 Done Done Sent false
23116 6410 Acked Write 2 Done Done Sent false
23117 6411 Acked Read 7 Done Done Sent false
23118 6412 Acked Read 9 Done Done Sent false
23119 6413 Acked Write 1 Done Done Sent false
23120 6414 Acked Read 2 Done Done Sent false
23121 6415 Acked Read 1 Done Done Sent false
23122 6416 Acked Write 4 Done Done Sent false
23123 6417 Acked Read 8 Done Done Sent false
23124 6418 Acked Read 3 Done Done Sent false
23125 6419 Acked Read 9 Done Done Sent false
23126 6420 Acked Read 1 Done Done Sent false
23127 6421 Acked Flush 0 Done Done Sent false
23128 6422 Acked Read 5 Done Done Sent false
23129 6423 Acked Read 7 Done Done Sent false
23130 6424 Acked Read 10 Done Done Sent false
23131 6425 Acked Write 7 Done Done Sent false
23132 6426 Acked Read 10 Done Done Sent false
23133 6427 Acked Read 9 Done Done Sent false
23134 6428 Acked Read 9 Done Done Sent false
23135 6429 Acked Read 5 Done Done Sent false
23136 6430 Acked Write 7 Done Done Sent false
23137 6431 Acked Read 10 Done Done Sent false
23138 6432 Acked Read 7 Done Done Sent false
23139 6433 Acked Flush 0 Done Done Sent false
23140 STATES DS:0 DS:1 DS:2 TOTAL
23141 Sent 0 0 302 302
23142 Done 302 302 0 604
23143 Skipped 0 0 0 0
23144 Error 0 0 0 0
23145 Last Flush: 6433 6433 6131
23146 Downstairs last five completed: 6131 6130 6129 6128 6127
23147 Upstairs last five completed: 6433 6432 6431 6430 6429
23148 CLIENT: Up:0 ds:302 act:2
23149 ----------------------------------------------------------------
23150 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
23151 States: Active Active Offline
23152 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
23153 6132 Acked Read 9 Done Done Sent false
23154 6133 Acked Write 3 Done Done Sent false
23155 6134 Acked Read 7 Done Done Sent false
23156 6135 Acked Write 1 Done Done Sent false
23157 6136 Acked Read 10 Done Done Sent false
23158 6137 Acked Write 7 Done Done Sent false
23159 6138 Acked Write 10 Done Done Sent false
23160 6139 Acked Flush 0 Done Done Sent false
23161 6140 Acked Flush 0 Done Done Sent false
23162 6141 Acked Write 6 Done Done Sent false
23163 6142 Acked Write 9 Done Done Sent false
23164 6143 Acked Write 6 Done Done Sent false
23165 6144 Acked Flush 0 Done Done Sent false
23166 6145 Acked Read 5 Done Done Sent false
23167 6146 Acked Write 4 Done Done Sent false
23168 6147 Acked Write 4 Done Done Sent false
23169 6148 Acked Write 8 Done Done Sent false
23170 6149 Acked Read 9 Done Done Sent false
23171 6150 Acked Read 5 Done Done Sent false
23172 6151 Acked Write 10 Done Done Sent false
23173 6152 Acked Write 1 Done Done Sent false
23174 6153 Acked Write 1 Done Done Sent false
23175 6154 Acked Write 4 Done Done Sent false
23176 6155 Acked Read 1 Done Done Sent false
23177 6156 Acked Write 2 Done Done Sent false
23178 6157 Acked Flush 0 Done Done Sent false
23179 6158 Acked Write 7 Done Done Sent false
23180 6159 Acked Flush 0 Done Done Sent false
23181 6160 Acked Read 9 Done Done Sent false
23182 6161 Acked Write 5 Done Done Sent false
23183 6162 Acked Flush 0 Done Done Sent false
23184 6163 Acked Write 4 Done Done Sent false
23185 6164 Acked Write 6 Done Done Sent false
23186 6165 Acked Read 7 Done Done Sent false
23187 6166 Acked Write 9 Done Done Sent false
23188 6167 Acked Read 10 Done Done Sent false
23189 6168 Acked Read 10 Done Done Sent false
23190 6169 Acked Read 8 Done Done Sent false
23191 6170 Acked Read 2 Done Done Sent false
23192 6171 Acked Read 3 Done Done Sent false
23193 6172 Acked Read 3 Done Done Sent false
23194 6173 Acked Write 4 Done Done Sent false
23195 6174 Acked Write 9 Done Done Sent false
23196 6175 Acked Read 6 Done Done Sent false
23197 6176 Acked Write 10 Done Done Sent false
23198 6177 Acked Flush 0 Done Done Sent false
23199 6178 Acked Write 9 Done Done Sent false
23200 6179 Acked Write 3 Done Done Sent false
23201 6180 Acked Read 5 Done Done Sent false
23202 6181 Acked Read 9 Done Done Sent false
23203 6182 Acked Write 3 Done Done Sent false
23204 6183 Acked Write 7 Done Done Sent false
23205 6184 Acked Read 3 Done Done Sent false
23206 6185 Acked Read 4 Done Done Sent false
23207 6186 Acked Read 7 Done Done Sent false
23208 6187 Acked Flush 0 Done Done Sent false
23209 6188 Acked Write 2 Done Done Sent false
23210 6189 Acked Read 7 Done Done Sent false
23211 6190 Acked Write 6 Done Done Sent false
23212 6191 Acked Read 2 Done Done Sent false
23213 6192 Acked Read 2 Done Done Sent false
23214 6193 Acked Read 5 Done Done Sent false
23215 6194 Acked Read 8 Done Done Sent false
23216 6195 Acked Write 6 Done Done Sent false
23217 6196 Acked Read 8 Done Done Sent false
23218 6197 Acked Read 5 Done Done Sent false
23219 6198 Acked Write 2 Done Done Sent false
23220 6199 Acked Flush 0 Done Done Sent false
23221 6200 Acked Read 9 Done Done Sent false
23222 6201 Acked Read 4 Done Done Sent false
23223 6202 Acked Read 10 Done Done Sent false
23224 6203 Acked Read 4 Done Done Sent false
23225 6204 Acked Read 5 Done Done Sent false
23226 6205 Acked Read 4 Done Done Sent false
23227 6206 Acked Read 3 Done Done Sent false
23228 6207 Acked Read 2 Done Done Sent false
23229 6208 Acked Read 8 Done Done Sent false
23230 6209 Acked Read 4 Done Done Sent false
23231 6210 Acked Read 5 Done Done Sent false
23232 6211 Acked Write 2 Done Done Sent false
23233 6212 Acked Write 4 Done Done Sent false
23234 6213 Acked Write 8 Done Done Sent false
23235 6214 Acked Write 5 Done Done Sent false
23236 6215 Acked Write 4 Done Done Sent false
23237 6216 Acked Read 1 Done Done Sent false
23238 6217 Acked Read 7 Done Done Sent false
23239 6218 Acked Read 8 Done Done Sent false
23240 6219 Acked Write 5 Done Done Sent false
23241 6220 Acked Write 5 Done Done Sent false
23242 6221 Acked Read 9 Done Done Sent false
23243 6222 Acked Read 5 Done Done Sent false
23244 6223 Acked Write 2 Done Done Sent false
23245 6224 Acked Write 9 Done Done Sent false
23246 6225 Acked Read 1 Done Done Sent false
23247 6226 Acked Write 2 Done Done Sent false
23248 6227 Acked Write 9 Done Done Sent false
23249 6228 Acked Read 7 Done Done Sent false
23250 6229 Acked Read 6 Done Done Sent false
23251 6230 Acked Write 6 Done Done Sent false
23252 6231 Acked Read 3 Done Done Sent false
23253 6232 Acked Read 6 Done Done Sent false
23254 6233 Acked Read 3 Done Done Sent false
23255 6234 Acked Flush 0 Done Done Sent false
23256 6235 Acked Flush 0 Done Done Sent false
23257 6236 Acked Read 7 Done Done Sent false
23258 6237 Acked Write 5 Done Done Sent false
23259 6238 Acked Read 8 Done Done Sent false
23260 6239 Acked Write 9 Done Done Sent false
23261 6240 Acked Write 5 Done Done Sent false
23262 6241 Acked Read 2 Done Done Sent false
23263 6242 Acked Write 10 Done Done Sent false
23264 6243 Acked Read 8 Done Done Sent false
23265 6244 Acked Flush 0 Done Done Sent false
23266 6245 Acked Write 3 Done Done Sent false
23267 6246 Acked Read 7 Done Done Sent false
23268 6247 Acked Write 5 Done Done Sent false
23269 6248 Acked Write 9 Done Done Sent false
23270 6249 Acked Write 8 Done Done Sent false
23271 6250 Acked Read 4 Done Done Sent false
23272 6251 Acked Flush 0 Done Done Sent false
23273 6252 Acked Read 7 Done Done Sent false
23274 6253 Acked Write 6 Done Done Sent false
23275 6254 Acked Write 9 Done Done Sent false
23276 6255 Acked Flush 0 Done Done Sent false
23277 6256 Acked Read 5 Done Done Sent false
23278 6257 Acked Read 5 Done Done Sent false
23279 6258 Acked Read 3 Done Done Sent false
23280 6259 Acked Read 1 Done Done Sent false
23281 6260 Acked Write 3 Done Done Sent false
23282 6261 Acked Read 5 Done Done Sent false
23283 6262 Acked Read 4 Done Done Sent false
23284 6263 Acked Read 6 Done Done Sent false
23285 6264 Acked Read 1 Done Done Sent false
23286 6265 Acked Read 10 Done Done Sent false
23287 6266 Acked Read 4 Done Done Sent false
23288 6267 Acked Flush 0 Done Done Sent false
23289 6268 Acked Read 5 Done Done Sent false
23290 6269 Acked Read 7 Done Done Sent false
23291 6270 Acked Read 4 Done Done Sent false
23292 6271 Acked Read 9 Done Done Sent false
23293 6272 Acked Read 5 Done Done Sent false
23294 6273 Acked Read 7 Done Done Sent false
23295 6274 Acked Write 1 Done Done Sent false
23296 6275 Acked Read 5 Done Done Sent false
23297 6276 Acked Read 4 Done Done Sent false
23298 6277 Acked Write 8 Done Done Sent false
23299 6278 Acked Read 4 Done Done Sent false
23300 6279 Acked Write 3 Done Done Sent false
23301 6280 Acked Read 8 Done Done Sent false
23302 6281 Acked Read 4 Done Done Sent false
23303 6282 Acked Read 9 Done Done Sent false
23304 6283 Acked Read 1 Done Done Sent false
23305 6284 Acked Read 5 Done Done Sent false
23306 6285 Acked Read 9 Done Done Sent false
23307 6286 Acked Write 7 Done Done Sent false
23308 6287 Acked Write 9 Done Done Sent false
23309 6288 Acked Read 10 Done Done Sent false
23310 6289 Acked Write 10 Done Done Sent false
23311 6290 Acked Write 6 Done Done Sent false
23312 6291 Acked Write 6 Done Done Sent false
23313 6292 Acked Flush 0 Done Done Sent false
23314 6293 Acked Read 9 Done Done Sent false
23315 6294 Acked Flush 0 Done Done Sent false
23316 6295 Acked Read 2 Done Done Sent false
23317 6296 Acked Write 6 Done Done Sent false
23318 6297 Acked Read 1 Done Done Sent false
23319 6298 Acked Write 7 Done Done Sent false
23320 6299 Acked Write 2 Done Done Sent false
23321 6300 Acked Read 5 Done Done Sent false
23322 6301 Acked Flush 0 Done Done Sent false
23323 6302 Acked Write 1 Done Done Sent false
23324 6303 Acked Read 7 Done Done Sent false
23325 6304 Acked Read 8 Done Done Sent false
23326 6305 Acked Read 10 Done Done Sent false
23327 6306 Acked Write 3 Done Done Sent false
23328 6307 Acked Read 7 Done Done Sent false
23329 6308 Acked Write 8 Done Done Sent false
23330 6309 Acked Write 3 Done Done Sent false
23331 6310 Acked Flush 0 Done Done Sent false
23332 6311 Acked Write 9 Done Done Sent false
23333 6312 Acked Read 10 Done Done Sent false
23334 6313 Acked Read 1 Done Done Sent false
23335 6314 Acked Write 9 Done Done Sent false
23336 6315 Acked Read 6 Done Done Sent false
23337 6316 Acked Read 4 Done Done Sent false
23338 6317 Acked Write 7 Done Done Sent false
23339 6318 Acked Write 3 Done Done Sent false
23340 6319 Acked Read 1 Done Done Sent false
23341 6320 Acked Read 2 Done Done Sent false
23342 6321 Acked Read 6 Done Done Sent false
23343 6322 Acked Read 5 Done Done Sent false
23344 6323 Acked Read 6 Done Done Sent false
23345 6324 Acked Read 10 Done Done Sent false
23346 6325 Acked Write 9 Done Done Sent false
23347 6326 Acked Read 6 Done Done Sent false
23348 6327 Acked Read 1 Done Done Sent false
23349 6328 Acked Read 3 Done Done Sent false
23350 6329 Acked Write 1 Done Done Sent false
23351 6330 Acked Read 1 Done Done Sent false
23352 6331 Acked Read 7 Done Done Sent false
23353 6332 Acked Read 7 Done Done Sent false
23354 6333 Acked Read 4 Done Done Sent false
23355 6334 Acked Read 10 Done Done Sent false
23356 6335 Acked Read 8 Done Done Sent false
23357 6336 Acked Flush 0 Done Done Sent false
23358 6337 Acked Read 2 Done Done Sent false
23359 6338 Acked Read 5 Done Done Sent false
23360 6339 Acked Write 1 Done Done Sent false
23361 6340 Acked Flush 0 Done Done Sent false
23362 6341 Acked Read 4 Done Done Sent false
23363 6342 Acked Read 5 Done Done Sent false
23364 6343 Acked Write 7 Done Done Sent false
23365 6344 Acked Read 9 Done Done Sent false
23366 6345 Acked Write 6 Done Done Sent false
23367 6346 Acked Read 1 Done Done Sent false
23368 6347 Acked Write 3 Done Done Sent false
23369 6348 Acked Read 10 Done Done Sent false
23370 6349 Acked Flush 0 Done Done Sent false
23371 6350 Acked Write 10 Done Done Sent false
23372 6351 Acked Read 3 Done Done Sent false
23373 6352 Acked Write 4 Done Done Sent false
23374 6353 Acked Read 10 Done Done Sent false
23375 6354 Acked Read 4 Done Done Sent false
23376 6355 Acked Write 9 Done Done Sent false
23377 6356 Acked Read 8 Done Done Sent false
23378 6357 Acked Write 10 Done Done Sent false
23379 6358 Acked Read 8 Done Done Sent false
23380 6359 Acked Read 7 Done Done Sent false
23381 6360 Acked Write 2 Done Done Sent false
23382 6361 Acked Read 9 Done Done Sent false
23383 6362 Acked Write 2 Done Done Sent false
23384 6363 Acked Read 4 Done Done Sent false
23385 6364 Acked Write 1 Done Done Sent false
23386 6365 Acked Write 10 Done Done Sent false
23387 6366 Acked Read 7 Done Done Sent false
23388 6367 Acked Write 1 Done Done Sent false
23389 6368 Acked Read 1 Done Done Sent false
23390 6369 Acked Read 6 Done Done Sent false
23391 6370 Acked Read 10 Done Done Sent false
23392 6371 Acked Read 5 Done Done Sent false
23393 6372 Acked Flush 0 Done Done Sent false
23394 6373 Acked Write 7 Done Done Sent false
23395 6374 Acked Flush 0 Done Done Sent false
23396 6375 Acked Write 7 Done Done Sent false
23397 6376 Acked Read 6 Done Done Sent false
23398 6377 Acked Write 9 Done Done Sent false
23399 6378 Acked Write 6 Done Done Sent false
23400 6379 Acked Read 7 Done Done Sent false
23401 6380 Acked Read 7 Done Done Sent false
23402 6381 Acked Write 5 Done Done Sent false
23403 6382 Acked Read 4 Done Done Sent false
23404 6383 Acked Flush 0 Done Done Sent false
23405 6384 Acked Read 10 Done Done Sent false
23406 6385 Acked Read 5 Done Done Sent false
23407 6386 Acked Write 8 Done Done Sent false
23408 6387 Acked Write 4 Done Done Sent false
23409 6388 Acked Read 6 Done Done Sent false
23410 6389 Acked Flush 0 Done Done Sent false
23411 6390 Acked Write 6 Done Done Sent false
23412 6391 Acked Read 10 Done Done Sent false
23413 6392 Acked Write 7 Done Done Sent false
23414 6393 Acked Flush 0 Done Done Sent false
23415 6394 Acked Write 4 Done Done Sent false
23416 6395 Acked Write 10 Done Done Sent false
23417 6396 Acked Write 7 Done Done Sent false
23418 6397 Acked Read 3 Done Done Sent false
23419 6398 Acked Write 7 Done Done Sent false
23420 6399 Acked Write 6 Done Done Sent false
23421 6400 Acked Write 10 Done Done Sent false
23422 6401 Acked Read 9 Done Done Sent false
23423 6402 Acked Write 5 Done Done Sent false
23424 6403 Acked Read 3 Done Done Sent false
23425 6404 Acked Read 10 Done Done Sent false
23426 6405 Acked Write 6 Done Done Sent false
23427 6406 Acked Read 9 Done Done Sent false
23428 6407 Acked Read 4 Done Done Sent false
23429 6408 Acked Write 9 Done Done Sent false
23430 6409 Acked Flush 0 Done Done Sent false
23431 6410 Acked Write 2 Done Done Sent false
23432 6411 Acked Read 7 Done Done Sent false
23433 6412 Acked Read 9 Done Done Sent false
23434 6413 Acked Write 1 Done Done Sent false
23435 6414 Acked Read 2 Done Done Sent false
23436 6415 Acked Read 1 Done Done Sent false
23437 6416 Acked Write 4 Done Done Sent false
23438 6417 Acked Read 8 Done Done Sent false
23439 6418 Acked Read 3 Done Done Sent false
23440 6419 Acked Read 9 Done Done Sent false
23441 6420 Acked Read 1 Done Done Sent false
23442 6421 Acked Flush 0 Done Done Sent false
23443 6422 Acked Read 5 Done Done Sent false
23444 6423 Acked Read 7 Done Done Sent false
23445 6424 Acked Read 10 Done Done Sent false
23446 6425 Acked Write 7 Done Done Sent false
23447 6426 Acked Read 10 Done Done Sent false
23448 6427 Acked Read 9 Done Done Sent false
23449 6428 Acked Read 9 Done Done Sent false
23450 6429 Acked Read 5 Done Done Sent false
23451 6430 Acked Write 7 Done Done Sent false
23452 6431 Acked Read 10 Done Done Sent false
23453 6432 Acked Read 7 Done Done Sent false
23454 6433 Acked Flush 0 Done Done Sent false
23455 STATES DS:0 DS:1 DS:2 TOTAL
23456 Sent 0 0 302 302
23457 Done 302 302 0 604
23458 Skipped 0 0 0 0
23459 Error 0 0 0 0
23460 Last Flush: 6433 6433 6131
23461 Downstairs last five completed: 6131 6130 6129 6128 6127
23462 Upstairs last five completed: 6433 6432 6431 6430 6429
23463 CLIENT: Up:0 ds:302 act:2
234642024-11-15T02:02:24.078ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
234652024-11-15T02:02:24.078ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:45691) = io task client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
234662024-11-15T02:02:24.079ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
234672024-11-15T02:02:24.080ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
234682024-11-15T02:02:24.080ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
234692024-11-15T02:02:24.080ZINFOcrucible: send last flush ID to this DS: 6131 = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
234702024-11-15T02:02:24.080ZINFOcrucible: Replied this last flush ID: 6131 = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
234712024-11-15T02:02:24.080ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
234722024-11-15T02:02:24.080ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
234732024-11-15T02:02:24.080ZINFOcrucible: [2] client re-new 302 jobs since flush 6131 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
234742024-11-15T02:02:24.080ZINFOcrucible: [2] Replayed 302 jobs since flush: 6131 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
23475 ----------------------------------------------------------------
23476 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
23477 Downstairs last five completed: 6433 6432 6431 6430 6429
23478 Upstairs last five completed: 6433 6432 6431 6430 6429
23479 CLIENT: Up:0 ds:0 act:3
23480 Replay: All jobs finished
23481 Main task has work to do, go find it
23482 got dsc Stop(0)
23483 stop 0
23484 Nov 15 02:02:24.336 INFO request completed[8810] Got stop action so:false kr:false
23485 , latency_us: 373, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 4b519ef6-c730-40eb-a712-b7191edd6861, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
234862024-11-15T02:02:24.337ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
234872024-11-15T02:02:24.337ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
234882024-11-15T02:02:24.337ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
234892024-11-15T02:02:24.337ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
234902024-11-15T02:02:24.337ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
23491 Nov 15 02:02:24.337 INFO request completed, latency_us: 131, response_code: 200, uri: /state/cid/0, method: GET, req_id: 6d319fbc-e875-4af2-ac28-606823a14162, remote_addr: 127.0.{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2024-11-15T02:02:24.337501269Z","hostname":"w-01JCPRHT5F2MPA44KVJHM3S2PB","pid":1205,"":"io task","client":"0","":"downstairs","session_id":"971dcb2e-6891-4ecf-b367-1e04e52065ee"}
23492 0.1:40008, local_addr: 127.0.0.1:9998
23493 [8810] Exited with: ExitStatus(unix_wait_status(9))
23494 [8810][0] reports Exit
23495 Nov 15 02:02:28.339 INFO request completed, latency_us: 218, response_code: 200, uri: /state/cid/0, method: GET, req_id: 972eebb3-fcf9-4d2e-9e1a-e9dba7ea1ffb, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
23496 001/300 Read block 3830 len 4096
23497 002/300 Read block 2454 len 24576
23498 003/300 Write block 3962 len 12288 data: 4 3 3
23499 004/300 Read block 4409 len 4096
23500 005/300 Read block 1541 len 24576
23501 006/300 Read block 879 len 8192
23502 007/300 Write block 2390 len 32768 data: 6 5 5 5 5 3 2 3
23503 008/300 Read block 3747 len 28672
23504 009/300 Write block 4419 len 40960 data: 6 7 7 6 6 5 4 4 5 4
23505 010/300 Write block 3771 len 4096 data: 4
23506 011/300 Read block 1908 len 4096
23507 012/300 Read block 2912 len 12288
23508 013/300 Read block 2684 len 40960
23509 014/300 Write block 2064 len 4096 data: 4
23510 015/300 Write block 3806 len 8192 data: 5 5
23511 016/300 Write block 3959 len 32768 data: 6 5 5 5 4 4 2 2
23512 017/300 Write block 1835 len 8192 data: 4 4
23513 018/300 Write block 4205 len 32768 data: 9 10 8 5 6 4 4 2
23514 019/300 Read block 564 len 20480
23515 020/300 Write block 1019 len 4096 data: 5
23516 021/300 Read block 1582 len 36864
23517 022/300 Flush
23518 023/300 Read block 4660 len 24576
23519 024/300 Read block 4466 len 28672
23520 025/300 Write block 4419 len 24576 data: 7 8 8 7 7 6
23521 026/300 Flush
23522 027/300 Write block 562 len 8192 data: 5 4
23523 028/300 Read block 3018 len 32768
23524 029/300 Write block 3485 len 40960 data: 4 4 4 5 6 7 6 5 4 4
23525 030/300 Read block 3492 len 36864
23526 031/300 Read block 3627 len 28672
23527 032/300 Write block 1654 len 40960 data: 6 6 6 4 5 5 6 7 7 9
23528 033/300 Write block 4873 len 36864 data: 3 4 7 7 6 4 3 3 3
23529 034/300 Read block 3645 len 32768
23530 035/300 Read block 4109 len 28672
23531 036/300 Flush
23532 037/300 Read block 1338 len 12288
23533 038/300 Flush
23534 039/300 Read block 482 len 4096
23535 040/300 Write block 3440 len 28672 data: 5 6 6 6 6 6 5
23536 041/300 Read block 2421 len 40960
23537 042/300 Write block 3565 len 28672 data: 3 3 3 4 6 5 5
23538 043/300 Read block 3886 len 20480
23539 044/300 Write block 3364 len 20480 data: 3 3 2 2 3
23540 045/300 Write block 4518 len 16384 data: 4 3 3 4
23541 046/300 Flush
23542 047/300 Write block 4203 len 32768 data: 7 7 10 11 9 6 7 5
23543 048/300 Write block 3764 len 28672 data: 6 4 4 3 3 3 4
23544 049/300 Write block 157 len 24576 data: 3 3 5 6 6 5
23545 050/300 Read block 1112 len 24576
23546 051/300 Read block 4341 len 12288
23547 052/300 Read block 4889 len 4096
23548 053/300 Read block 4629 len 40960
23549 054/300 Read block 2555 len 36864
23550 055/300 Write block 2558 len 36864 data: 5 4 2 5 5 6 6 6 8
23551 056/300 Write block 3330 len 40960 data: 4 3 3 4 4 5 4 4 3 3
23552 057/300 Flush
23553 058/300 Write block 3010 len 8192 data: 5 5
23554 059/300 Read block 481 len 4096
23555 060/300 Write block 3095 len 8192 data: 3 3
23556 061/300 Write block 2132 len 8192 data: 5 5
23557 062/300 Read block 4612 len 12288
23558 063/300 Read block 1841 len 12288
23559 064/300 Read block 4933 len 8192
23560 065/300 Read block 1070 len 4096
23561 066/300 Write block 809 len 20480 data: 3 3 3 3 4
23562 067/300 Read block 2324 len 16384
23563 068/300 Read block 4253 len 32768
23564 069/300 Write block 1364 len 20480 data: 3 5 5 4 5
23565 070/300 Write block 2172 len 40960 data: 6 6 5 4 4 4 3 3 3 4
23566 071/300 Read block 1043 len 24576
23567 072/300 Read block 1461 len 28672
23568 073/300 Read block 3380 len 40960
23569 074/300 Write block 1952 len 16384 data: 4 5 6 6
23570 075/300 Flush
23571 076/300 Read block 2433 len 16384
23572 077/300 Read block 1195 len 4096
23573 078/300 Read block 4570 len 4096
23574 079/300 Write block 315 len 32768 data: 3 3 3 3 3 3 5 6
23575 080/300 Write block 2335 len 12288 data: 3 6 6
23576 081/300 Read block 2620 len 4096
23577 082/300 Write block 1370 len 4096 data: 5
23578 083/300 Read block 2212 len 16384
23579 084/300 Read block 4172 len 16384
23580 085/300 Read block 984 len 20480
23581 086/300 Write block 4464 len 24576 data: 4 4 3 3 2 2
23582 087/300 Write block 375 len 4096 data: 4
23583 088/300 Write block 1572 len 20480 data: 3 3 3 2 2
23584 089/300 Write block 2881 len 40960 data: 4 4 3 2 3 6 6 7 6 6
23585 090/300 Read block 3683 len 8192
23586 091/300 Write block 608 len 32768 data: 3 3 4 3 3 3 3 2
23587 092/300 Write block 3732 len 40960 data: 3 3 3 4 5 6 6 6 5 5
23588 093/300 Read block 3644 len 4096
23589 094/300 Write block 1349 len 40960 data: 3 4 3 3 3 3 5 5 5 5
23590 095/300 Read block 2759 len 24576
23591 096/300 Read block 4500 len 36864
23592 097/300 Flush
23593 098/300 Read block 1800 len 4096
23594 099/300 Read block 3588 len 28672
23595 100/300 Write block 2669 len 12288 data: 2 2 2
23596 101/300 Read block 4563 len 8192
23597 102/300 Write block 2280 len 36864 data: 3 5 5 6 7 8 7 6 6
23598 103/300 Read block 1276 len 16384
23599 104/300 Read block 2584 len 16384
23600 105/300 Read block 3467 len 24576
23601 106/300 Write block 570 len 20480 data: 3 3 3 3 3
23602 107/300 Read block 1867 len 20480
23603 108/300 Write block 1538 len 24576 data: 3 7 6 6 5 6
23604 109/300 Write block 4079 len 28672 data: 5 5 5 3 3 3 3
23605 110/300 Write block 4315 len 32768 data: 7 8 8 5 6 6 7 6
23606 111/300 Write block 4701 len 32768 data: 7 7 8 8 8 9 7 7
23607 112/300 Read block 2200 len 32768
23608 113/300 Write block 3140 len 4096 data: 4
23609 114/300 Write block 184 len 28672 data: 3 4 3 3 3 3 3
23610 115/300 Read block 2714 len 32768
23611 116/300 Write block 54 len 40960 data: 5 4 4 6 7 6 6 5 4 4
23612 117/300 Read block 3661 len 28672
23613 118/300 Read block 2509 len 32768
23614 119/300 Flush
23615 120/300 Read block 4128 len 24576
23616 121/300 Read block 660 len 12288
23617 122/300 Write block 4472 len 32768 data: 3 4 3 3 3 3 3 2
23618 123/300 Read block 989 len 12288
23619 124/300 Read block 3067 len 36864
23620 125/300 Write block 3339 len 8192 data: 4 3
23621 126/300 Write block 3210 len 8192 data: 2 2
23622 127/300 Flush
23623 128/300 Read block 1536 len 16384
23624 129/300 Write block 3087 len 12288 data: 2 3 3
23625 130/300 Write block 4310 len 40960 data: 5 5 7 7 6 8 9 9 6 7
23626 131/300 Read block 4893 len 36864
23627 132/300 Write block 2105 len 8192 data: 4 3
23628 133/300 Write block 2233 len 4096 data: 8
23629 134/300 Read block 3341 len 4096
23630 135/300 Read block 3489 len 16384
23631 136/300 Read block 4929 len 4096
23632 137/300 Write block 3214 len 24576 data: 3 3 3 3 5 7
23633 138/300 Write block 1349 len 12288 data: 4 5 4
23634 139/300 Read block 2120 len 36864
23635 140/300 Write block 328 len 40960 data: 6 6 4 5 5 6 4 4 3 4
23636 141/300 Write block 1588 len 4096 data: 4
23637 142/300 Read block 434 len 40960
23638 143/300 Write block 4862 len 12288 data: 5 6 5
23639 144/300 Write block 4548 len 16384 data: 2 2 2 3
23640 145/300 Write block 227 len 4096 data: 3
23641 146/300 Read block 4374 len 8192
23642 147/300 Write block 786 len 24576 data: 8 8 7 7 7 6
23643 148/300 Read block 1923 len 4096
23644 149/300 Write block 2142 len 32768 data: 4 5 5 5 4 4 6 6
23645 150/300 Write block 2324 len 20480 data: 6 7 4 4 4
23646 151/300 Read block 1748 len 32768
23647 152/300 Read block 1694 len 24576
23648 153/300 Flush
23649 154/300 Read block 578 len 20480
23650 155/300 Read block 4067 len 16384
23651 156/300 Write block 2020 len 36864 data: 5 4 3 2 4 5 5 4 3
23652 157/300 Write block 999 len 24576 data: 8 8 8 8 7 6
23653 158/300 Write block 3355 len 20480 data: 6 4 3 3 3
23654 159/300 Write block 2716 len 20480 data: 3 2 2 2 2
23655 160/300 Read block 2037 len 8192
23656 161/300 Read block 2863 len 32768
23657 162/300 Read block 4005 len 4096
23658 163/300 Write block 875 len 32768 data: 4 4 7 7 5 5 5 4
23659 164/300 Flush
23660 165/300 Read block 3116 len 12288
23661 166/300 Write block 1679 len 28672 data: 5 5 6 4 4 6 6
23662 167/300 Read block 4947 len 40960
23663 168/300 Read block 486 len 16384
23664 169/300 Write block 3465 len 16384 data: 3 3 4 5
23665 170/300 Write block 950 len 16384 data: 2 2 2 2
23666 171/300 Read block 1142 len 4096
23667 172/300 Write block 333 len 36864 data: 7 5 5 4 5 3 3 2 2
23668 173/300 Read block 1320 len 12288
23669 174/300 Read block 2440 len 28672
23670 175/300 Write block 2215 len 40960 data: 4 5 4 5 4 7 7 4 6 6
23671 176/300 Read block 4779 len 32768
23672 177/300 Write block 6 len 24576 data: 6 6 5 5 7 8
23673 178/300 Write block 2373 len 4096 data: 4
23674 179/300 Write block 2060 len 40960 data: 6 7 8 7 5 3 2 2 3 4
23675 180/300 Read block 4109 len 32768
23676 181/300 Write block 3887 len 12288 data: 6 6 3
23677 182/300 Read block 3938 len 8192
23678 183/300 Read block 1098 len 20480
23679 184/300 Read block 2327 len 40960
23680 185/300 Read block 101 len 40960
23681 186/300 Write block 2021 len 36864 data: 5 4 3 5 6 6 5 4 3
23682 187/300 Write block 3391 len 12288 data: 2 5 5
23683 188/300 Read block 2140 len 28672
23684 189/300 Write block 2025 len 12288 data: 7 7 6
23685 190/300 Read block 1593 len 4096
23686 191/300 Write block 3368 len 32768 data: 4 4 4 3 5 6 6 5
23687 192/300 Write block 2411 len 36864 data: 3 3 3 3 3 2 2 2 2
23688 193/300 Read block 3362 len 16384
23689 194/300 Read block 1579 len 4096
23690 195/300 Read block 4558 len 28672
23691 196/300 Read block 1253 len 4096
23692 197/300 Flush
23693 198/300 Write block 2678 len 20480 data: 3 3 3 3 4
23694 199/300 Write block 3098 len 20480 data: 4 4 3 3 3
23695 200/300 Write block 3591 len 36864 data: 6 5 5 6 7 6 5 5 4
23696 201/300 Read block 1975 len 4096
23697 202/300 Write block 3221 len 24576 data: 5 5 4 4 4 4
23698 203/300 Flush
23699 204/300 Read block 835 len 4096
23700 205/300 Read block 4252 len 20480
23701 206/300 Read block 2932 len 20480
23702 207/300 Write block 2918 len 20480 data: 3 3 3 3 3
23703 208/300 Read block 2386 len 36864
23704 209/300 Flush
23705 210/300 Read block 425 len 28672
23706 211/300 Read block 4256 len 36864
23707 212/300 Read block 2862 len 32768
23708 213/300 Read block 1762 len 36864
23709 214/300 Flush
23710 215/300 Read block 2987 len 24576
23711 216/300 Read block 3980 len 12288
23712 217/300 Read block 1744 len 16384
23713 218/300 Read block 3621 len 12288
23714 219/300 Write block 1508 len 36864 data: 3 3 4 4 4 5 5 5 4
23715 220/300 Read block 2175 len 16384
23716 221/300 Write block 999 len 12288 data: 9 9 9
23717 222/300 Read block 188 len 20480
23718 223/300 Read block 3196 len 8192
23719 224/300 Flush
23720 225/300 Write block 3368 len 8192 data: 5 5
23721 226/300 Read block 2555 len 28672
23722 227/300 Read block 3317 len 40960
23723 228/300 Write block 864 len 12288 data: 4 4 3
23724 229/300 Write block 4401 len 40960 data: 3 3 3 3 2 2 4 4 4 5
23725 230/300 Read block 636 len 20480
23726 231/300 Read block 3966 len 8192
23727 232/300 Read block 3133 len 36864
23728 233/300 Write block 4600 len 40960 data: 3 4 4 4 5 5 6 6 6 5
23729 234/300 Read block 233 len 12288
23730 235/300 Read block 3977 len 12288
23731 236/300 Write block 677 len 4096 data: 5
23732 237/300 Write block 3865 len 16384 data: 7 7 8 7
23733 238/300 Read block 3333 len 8192
23734 239/300 Read block 35 len 16384
23735 240/300 Read block 950 len 4096
23736 241/300 Write block 1207 len 20480 data: 3 4 4 5 4
23737 242/300 Write block 1907 len 12288 data: 4 4 2
23738 243/300 Flush
23739 244/300 Write block 1087 len 4096 data: 5
23740 245/300 Write block 1085 len 4096 data: 4
23741 246/300 Read block 3204 len 40960
23742 247/300 Read block 4045 len 8192
23743 248/300 Read block 2365 len 32768
23744 249/300 Write block 460 len 12288 data: 6 8 7
23745 250/300 Write block 2597 len 40960 data: 7 7 7 6 6 6 6 3 3 4
23746 251/300 Flush
23747 252/300 Write block 2332 len 12288 data: 4 4 3
23748 253/300 Read block 2291 len 4096
23749 254/300 Write block 4523 len 4096 data: 4
23750 255/300 Read block 1365 len 28672
23751 256/300 Write block 4683 len 12288 data: 5 7 6
23752 257/300 Read block 4030 len 4096
23753 258/300 Write block 3163 len 28672 data: 4 3 2 3 3 3 3
23754 259/300 Flush
23755 260/300 Write block 4655 len 20480 data: 4 4 5 6 6
23756 261/300 Write block 3664 len 36864 data: 5 4 4 5 3 4 4 4 4
23757 262/300 Read block 3301 len 40960
23758 263/300 Write block 2300 len 12288 data: 3 5 5
23759 264/300 Flush
23760 265/300 Flush
23761 266/300 Read block 4125 len 12288
23762 267/300 Read block 4449 len 32768
23763 268/300 Read block 3288 len 12288
23764 269/300 Read block 3927 len 24576
23765 270/300 Read block 4000 len 40960
23766 271/300 Read block 1521 len 16384
23767 272/300 Read block 1539 len 28672
23768 273/300 Read block 4970 len 40960
23769 274/300 Read block 896 len 12288
23770 275/300 Write block 3938 len 12288 data: 4 4 3
23771 276/300 Write block 572 len 36864 data: 4 4 4 3 3 4 4 3 2
23772 277/300 Read block 1380 len 16384
23773 278/300 Read block 1715 len 28672
23774 279/300 Write block 4251 len 36864 data: 5 5 7 7 5 5 5 5 6
23775 280/300 Write block 1429 len 8192 data: 2 2
23776 281/300 Read block 1036 len 12288
23777 282/300 Write block 3430 len 36864 data: 3 3 3 2 5 4 4 3 3
23778 283/300 Flush
23779 284/300 Write block 3396 len 12288 data: 5 2 3
23780 285/300 Read block 218 len 24576
23781 286/300 Flush
23782 287/300 Read block 3790 len 20480
23783 288/300 Read block 1636 len 4096
23784 289/300 Read block 788 len 16384
23785 290/300 Write block 4888 len 20480 data: 5 6 7 6 7
23786 291/300 Write block 307 len 12288 data: 7 6 8
23787 292/300 Write block 811 len 4096 data: 4
23788 293/300 Read block 2667 len 8192
23789 294/300 Read block 58 len 24576
23790 295/300 Read block 71 len 36864
23791 296/300 Read block 669 len 24576
23792 297/300 Flush
23793 298/300 Write block 862 len 16384 data: 4 4 5 5
23794 299/300 Read block 1602 len 12288
23795 300/300 Read block 3649 len 8192
23796 301/300 Write block 2403 len 20480 data: 7 7 7 5 4
23797 Main task has work to do, go find it
23798 got dsc Start(0)
23799 start 0
23800 Nov 15 02:02:28.569 INFO request completed, latency_us: 252, response_code: 204, uri: /start/cid/0, method: POST, req_id: 3af9a2a7-7e52-4131-b881-615803cf661a, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
23801 Replay: started 0, returned:Ok(())
23802 ----------------------------------------------------------------
23803 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
23804 States: Offline Active Active
23805 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
23806 6434 Acked Read 1 Sent Done Done false
23807 6435 Acked Read 6 Sent Done Done false
23808 6436 Acked Write 3 Sent Done Done false
23809 6437 Acked Read 1 Sent Done Done false
23810 6438 Acked Read 6 Sent Done Done false
23811 6439 Acked Read 2 Sent Done Done false
23812 6440 Acked Write 8 Sent Done Done false
23813 6441 Acked Read 7 Sent Done Done false
23814 6442 Acked Write 10 Sent Done Done false
23815 6443 Acked Write 1 Sent Done Done false
23816 6444 Acked Read 1 Sent Done Done false
23817 6445 Acked Read 3 Sent Done Done false
23818 6446 Acked Read 10 Sent Done Done false
23819 6447 Acked Write 1 Sent Done Done false
23820 6448 Acked Write 2 Sent Done Done false
23821 6449 Acked Write 8 Sent Done Done false
23822 6450 Acked Write 2 Sent Done Done false
23823 6451 Acked Write 8 Sent Done Done false
23824 6452 Acked Read 5 Sent Done Done false
23825 6453 Acked Write 1 Sent Done Done false
23826 6454 Acked Read 9 Sent Done Done false
23827 6455 Acked Flush 0 Sent Done Done false
23828 6456 Acked Read 6 Sent Done Done false
23829 6457 Acked Read 7 Sent Done Done false
23830 6458 Acked Write 6 Sent Done Done false
23831 6459 Acked Flush 0 Sent Done Done false
23832 6460 Acked Write 2 Sent Done Done false
23833 6461 Acked Read 8 Sent Done Done false
23834 6462 Acked Write 10 Sent Done Done false
23835 6463 Acked Read 9 Sent Done Done false
23836 6464 Acked Read 7 Sent Done Done false
23837 6465 Acked Write 10 Sent Done Done false
23838 6466 Acked Write 9 Sent Done Done false
23839 6467 Acked Read 8 Sent Done Done false
23840 6468 Acked Read 7 Sent Done Done false
23841 6469 Acked Flush 0 Sent Done Done false
23842 6470 Acked Read 3 Sent Done Done false
23843 6471 Acked Flush 0 Sent Done Done false
23844 6472 Acked Read 1 Sent Done Done false
23845 6473 Acked Write 7 Sent Done Done false
23846 6474 Acked Read 10 Sent Done Done false
23847 6475 Acked Write 7 Sent Done Done false
23848 6476 Acked Read 5 Sent Done Done false
23849 6477 Acked Write 5 Sent Done Done false
23850 6478 Acked Write 4 Sent Done Done false
23851 6479 Acked Flush 0 Sent Done Done false
23852 6480 Acked Write 8 Sent Done Done false
23853 6481 Acked Write 7 Sent Done Done false
23854 6482 Acked Write 6 Sent Done Done false
23855 6483 Acked Read 6 Sent Done Done false
23856 6484 Acked Read 3 Sent Done Done false
23857 6485 Acked Read 1 Sent Done Done false
23858 6486 Acked Read 10 Sent Done Done false
23859 6487 Acked Read 9 Sent Done Done false
23860 6488 Acked Write 9 Sent Done Done false
23861 6489 Acked Write 10 Sent Done Done false
23862 6490 Acked Flush 0 Sent Done Done false
23863 6491 Acked Write 2 Sent Done Done false
23864 6492 Acked Read 1 Sent Done Done false
23865 6493 Acked Write 2 Sent Done Done false
23866 6494 Acked Write 2 Sent Done Done false
23867 6495 Acked Read 3 Sent Done Done false
23868 6496 Acked Read 3 Sent Done Done false
23869 6497 Acked Read 2 Sent Done Done false
23870 6498 Acked Read 1 Sent Done Done false
23871 6499 Acked Write 5 Sent Done Done false
23872 6500 Acked Read 4 Sent Done Done false
23873 6501 Acked Read 8 Sent Done Done false
23874 6502 Acked Write 5 Sent Done Done false
23875 6503 Acked Write 10 Sent Done Done false
23876 6504 Acked Read 6 Sent Done Done false
23877 6505 Acked Read 7 Sent Done Done false
23878 6506 Acked Read 10 Sent Done Done false
23879 6507 Acked Write 4 Sent Done Done false
23880 6508 Acked Flush 0 Sent Done Done false
23881 6509 Acked Read 4 Sent Done Done false
23882 6510 Acked Read 1 Sent Done Done false
23883 6511 Acked Read 1 Sent Done Done false
23884 6512 Acked Write 8 Sent Done Done false
23885 6513 Acked Write 3 Sent Done Done false
23886 6514 Acked Read 1 Sent Done Done false
23887 6515 Acked Write 1 Sent Done Done false
23888 6516 Acked Read 4 Sent Done Done false
23889 6517 Acked Read 4 Sent Done Done false
23890 6518 Acked Read 5 Sent Done Done false
23891 6519 Acked Write 6 Sent Done Done false
23892 6520 Acked Write 1 Sent Done Done false
23893 6521 Acked Write 5 Sent Done Done false
23894 6522 Acked Write 10 Sent Done Done false
23895 6523 Acked Read 2 Sent Done Done false
23896 6524 Acked Write 8 Sent Done Done false
23897 6525 Acked Write 10 Sent Done Done false
23898 6526 Acked Read 1 Sent Done Done false
23899 6527 Acked Write 10 Sent Done Done false
23900 6528 Acked Read 6 Sent Done Done false
23901 6529 Acked Read 9 Sent Done Done false
23902 6530 Acked Flush 0 Sent Done Done false
23903 6531 Acked Read 1 Sent Done Done false
23904 6532 Acked Read 7 Sent Done Done false
23905 6533 Acked Write 3 Sent Done Done false
23906 6534 Acked Read 2 Sent Done Done false
23907 6535 Acked Write 9 Sent Done Done false
23908 6536 Acked Read 4 Sent Done Done false
23909 6537 Acked Read 4 Sent Done Done false
23910 6538 Acked Read 6 Sent Done Done false
23911 6539 Acked Write 5 Sent Done Done false
23912 6540 Acked Read 5 Sent Done Done false
23913 6541 Acked Write 6 Sent Done Done false
23914 6542 Acked Write 7 Sent Done Done false
23915 6543 Acked Write 8 Sent Done Done false
23916 6544 Acked Write 8 Sent Done Done false
23917 6545 Acked Read 8 Sent Done Done false
23918 6546 Acked Write 1 Sent Done Done false
23919 6547 Acked Write 7 Sent Done Done false
23920 6548 Acked Read 8 Sent Done Done false
23921 6549 Acked Write 10 Sent Done Done false
23922 6550 Acked Read 7 Sent Done Done false
23923 6551 Acked Read 8 Sent Done Done false
23924 6552 Acked Flush 0 Sent Done Done false
23925 6553 Acked Read 6 Sent Done Done false
23926 6554 Acked Read 3 Sent Done Done false
23927 6555 Acked Write 8 Sent Done Done false
23928 6556 Acked Read 3 Sent Done Done false
23929 6557 Acked Read 9 Sent Done Done false
23930 6558 Acked Write 2 Sent Done Done false
23931 6559 Acked Write 2 Sent Done Done false
23932 6560 Acked Flush 0 Sent Done Done false
23933 6561 Acked Read 4 Sent Done Done false
23934 6562 Acked Write 3 Sent Done Done false
23935 6563 Acked Write 10 Sent Done Done false
23936 6564 Acked Read 9 Sent Done Done false
23937 6565 Acked Write 2 Sent Done Done false
23938 6566 Acked Write 1 Sent Done Done false
23939 6567 Acked Read 1 Sent Done Done false
23940 6568 Acked Read 4 Sent Done Done false
23941 6569 Acked Read 1 Sent Done Done false
23942 6570 Acked Write 6 Sent Done Done false
23943 6571 Acked Write 3 Sent Done Done false
23944 6572 Acked Read 9 Sent Done Done false
23945 6573 Acked Write 10 Sent Done Done false
23946 6574 Acked Write 1 Sent Done Done false
23947 6575 Acked Read 10 Sent Done Done false
23948 6576 Acked Write 3 Sent Done Done false
23949 6577 Acked Write 4 Sent Done Done false
23950 6578 Acked Write 1 Sent Done Done false
23951 6579 Acked Read 2 Sent Done Done false
23952 6580 Acked Write 6 Sent Done Done false
23953 6581 Acked Read 1 Sent Done Done false
23954 6582 Acked Write 8 Sent Done Done false
23955 6583 Acked Write 5 Sent Done Done false
23956 6584 Acked Read 8 Sent Done Done false
23957 6585 Acked Read 6 Sent Done Done false
23958 6586 Acked Flush 0 Sent Done Done false
23959 6587 Acked Read 5 Sent Done Done false
23960 6588 Acked Read 4 Sent Done Done false
23961 6589 Acked Write 9 Sent Done Done false
23962 6590 Acked Write 6 Sent Done Done false
23963 6591 Acked Write 5 Sent Done Done false
23964 6592 Acked Write 5 Sent Done Done false
23965 6593 Acked Read 2 Sent Done Done false
23966 6594 Acked Read 8 Sent Done Done false
23967 6595 Acked Read 1 Sent Done Done false
23968 6596 Acked Write 8 Sent Done Done false
23969 6597 Acked Flush 0 Sent Done Done false
23970 6598 Acked Read 3 Sent Done Done false
23971 6599 Acked Write 7 Sent Done Done false
23972 6600 Acked Read 10 Sent Done Done false
23973 6601 Acked Read 4 Sent Done Done false
23974 6602 Acked Write 4 Sent Done Done false
23975 6603 Acked Write 4 Sent Done Done false
23976 6604 Acked Read 1 Sent Done Done false
23977 6605 Acked Write 9 Sent Done Done false
23978 6606 Acked Read 3 Sent Done Done false
23979 6607 Acked Read 7 Sent Done Done false
23980 6608 Acked Write 10 Sent Done Done false
23981 6609 Acked Read 8 Sent Done Done false
23982 6610 Acked Write 6 Sent Done Done false
23983 6611 Acked Write 1 Sent Done Done false
23984 6612 Acked Write 10 Sent Done Done false
23985 6613 Acked Read 8 Sent Done Done false
23986 6614 Acked Write 3 Sent Done Done false
23987 6615 Acked Read 2 Sent Done Done false
23988 6616 Acked Read 5 Sent Done Done false
23989 6617 Acked Read 10 Sent Done Done false
23990 6618 Acked Read 10 Sent Done Done false
23991 6619 Acked Write 9 Sent Done Done false
23992 6620 Acked Write 3 Sent Done Done false
23993 6621 Acked Read 7 Sent Done Done false
23994 6622 Acked Write 3 Sent Done Done false
23995 6623 Acked Read 1 Sent Done Done false
23996 6624 Acked Write 8 Sent Done Done false
23997 6625 Acked Write 9 Sent Done Done false
23998 6626 Acked Read 4 Sent Done Done false
23999 6627 Acked Read 1 Sent Done Done false
24000 6628 Acked Read 7 Sent Done Done false
24001 6629 Acked Read 1 Sent Done Done false
24002 6630 Acked Flush 0 Sent Done Done false
24003 6631 Acked Write 5 Sent Done Done false
24004 6632 Acked Write 5 Sent Done Done false
24005 6633 Acked Write 9 Sent Done Done false
24006 6634 Acked Read 1 Sent Done Done false
24007 6635 Acked Write 6 Sent Done Done false
24008 6636 Acked Flush 0 Sent Done Done false
24009 6637 Acked Read 1 Sent Done Done false
24010 6638 Acked Read 5 Sent Done Done false
24011 6639 Acked Read 5 Sent Done Done false
24012 6640 Acked Write 5 Sent Done Done false
24013 6641 Acked Read 9 Sent Done Done false
24014 6642 Acked Flush 0 Sent Done Done false
24015 6643 Acked Read 7 Sent Done Done false
24016 6644 Acked Read 9 Sent Done Done false
24017 6645 Acked Read 8 Sent Done Done false
24018 6646 Acked Read 9 Sent Done Done false
24019 6647 Acked Flush 0 Sent Done Done false
24020 6648 Acked Read 6 Sent Done Done false
24021 6649 Acked Read 3 Sent Done Done false
24022 6650 Acked Read 4 Sent Done Done false
24023 6651 Acked Read 3 Sent Done Done false
24024 6652 Acked Write 9 Sent Done Done false
24025 6653 Acked Read 4 Sent Done Done false
24026 6654 Acked Write 3 Sent Done Done false
24027 6655 Acked Read 5 Sent Done Done false
24028 6656 Acked Read 2 Sent Done Done false
24029 6657 Acked Flush 0 Sent Done Done false
24030 6658 Acked Write 2 Sent Done Done false
24031 6659 Acked Read 7 Sent Done Done false
24032 6660 Acked Read 10 Sent Done Done false
24033 6661 Acked Write 3 Sent Done Done false
24034 6662 Acked Write 10 Sent Done Done false
24035 6663 Acked Read 5 Sent Done Done false
24036 6664 Acked Read 2 Sent Done Done false
24037 6665 Acked Read 9 Sent Done Done false
24038 6666 Acked Write 10 Sent Done Done false
24039 6667 Acked Read 3 Sent Done Done false
24040 6668 Acked Read 3 Sent Done Done false
24041 6669 Acked Write 1 Sent Done Done false
24042 6670 Acked Write 4 Sent Done Done false
24043 6671 Acked Read 2 Sent Done Done false
24044 6672 Acked Read 4 Sent Done Done false
24045 6673 Acked Read 1 Sent Done Done false
24046 6674 Acked Write 5 Sent Done Done false
24047 6675 Acked Write 3 Sent Done Done false
24048 6676 Acked Flush 0 Sent Done Done false
24049 6677 Acked Write 1 Sent Done Done false
24050 6678 Acked Write 1 Sent Done Done false
24051 6679 Acked Read 10 Sent Done Done false
24052 6680 Acked Read 2 Sent Done Done false
24053 6681 Acked Read 8 Sent Done Done false
24054 6682 Acked Write 3 Sent Done Done false
24055 6683 Acked Write 10 Sent Done Done false
24056 6684 Acked Flush 0 Sent Done Done false
24057 6685 Acked Write 3 Sent Done Done false
24058 6686 Acked Read 1 Sent Done Done false
24059 6687 Acked Write 1 Sent Done Done false
24060 6688 Acked Read 7 Sent Done Done false
24061 6689 Acked Write 3 Sent Done Done false
24062 6690 Acked Read 1 Sent Done Done false
24063 6691 Acked Write 7 Sent Done Done false
24064 6692 Acked Flush 0 Sent Done Done false
24065 6693 Acked Write 5 Sent Done Done false
24066 6694 Acked Write 9 Sent Done Done false
24067 6695 Acked Read 10 Sent Done Done false
24068 6696 Acked Write 3 Sent Done Done false
24069 6697 Acked Flush 0 Sent Done Done false
24070 6698 Acked Flush 0 Sent Done Done false
24071 6699 Acked Read 3 Sent Done Done false
24072 6700 Acked Read 8 Sent Done Done false
24073 6701 Acked Read 3 Sent Done Done false
24074 6702 Acked Read 6 Sent Done Done false
24075 6703 Acked Read 10 Sent Done Done false
24076 6704 Acked Read 4 Sent Done Done false
24077 6705 Acked Read 7 Sent Done Done false
24078 6706 Acked Read 10 Sent Done Done false
24079 6707 Acked Read 3 Sent Done Done false
24080 6708 Acked Write 3 Sent Done Done false
24081 6709 Acked Write 9 Sent Done Done false
24082 6710 Acked Read 4 Sent Done Done false
24083 6711 Acked Read 7 Sent Done Done false
24084 6712 Acked Write 9 Sent Done Done false
24085 6713 Acked Write 2 Sent Done Done false
24086 6714 Acked Read 3 Sent Done Done false
24087 6715 Acked Write 9 Sent Done Done false
24088 6716 Acked Flush 0 Sent Done Done false
24089 6717 Acked Write 3 Sent Done Done false
24090 6718 Acked Read 6 Sent Done Done false
24091 6719 Acked Flush 0 Sent Done Done false
24092 6720 Acked Read 5 Sent Done Done false
24093 6721 Acked Read 1 Sent Done Done false
24094 6722 Acked Read 4 Sent Done Done false
24095 6723 Acked Write 5 Sent Done Done false
24096 6724 Acked Write 3 Sent Done Done false
24097 6725 Acked Write 1 Sent Done Done false
24098 6726 Acked Read 2 Sent Done Done false
24099 6727 Acked Read 6 Sent Done Done false
24100 6728 Acked Read 9 Sent Done Done false
24101 6729 Acked Read 6 Sent Done Done false
24102 6730 Acked Flush 0 Sent Done Done false
24103 6731 Acked Write 4 Sent Done Done false
24104 6732 Acked Read 3 Sent Done Done false
24105 6733 Acked Read 2 Sent Done Done false
24106 6734 Acked Write 5 Sent Done Done false
24107 STATES DS:0 DS:1 DS:2 TOTAL
24108 Sent 301 0 0 301
24109 Done 0 301 301 602
24110 Skipped 0 0 0 0
24111 Error 0 0 0 0
24112 Last Flush: 6433 6730 6730
24113 Downstairs last five completed: 6433 6432 6431 6430 6429
24114 Upstairs last five completed: 6734 6733 6732 6731 6730
24115 CLIENT: Up:0 ds:301 act:2
24116 ----------------------------------------------------------------
24117 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
24118 States: Offline Active Active
24119 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
24120 6434 Acked Read 1 Sent Done Done false
24121 6435 Acked Read 6 Sent Done Done false
24122 6436 Acked Write 3 Sent Done Done false
24123 6437 Acked Read 1 Sent Done Done false
24124 6438 Acked Read 6 Sent Done Done false
24125 6439 Acked Read 2 Sent Done Done false
24126 6440 Acked Write 8 Sent Done Done false
24127 6441 Acked Read 7 Sent Done Done false
24128 6442 Acked Write 10 Sent Done Done false
24129 6443 Acked Write 1 Sent Done Done false
24130 6444 Acked Read 1 Sent Done Done false
24131 6445 Acked Read 3 Sent Done Done false
24132 6446 Acked Read 10 Sent Done Done false
24133 6447 Acked Write 1 Sent Done Done false
24134 6448 Acked Write 2 Sent Done Done false
24135 6449 Acked Write 8 Sent Done Done false
24136 6450 Acked Write 2 Sent Done Done false
24137 6451 Acked Write 8 Sent Done Done false
24138 6452 Acked Read 5 Sent Done Done false
24139 6453 Acked Write 1 Sent Done Done false
24140 6454 Acked Read 9 Sent Done Done false
24141 6455 Acked Flush 0 Sent Done Done false
24142 6456 Acked Read 6 Sent Done Done false
24143 6457 Acked Read 7 Sent Done Done false
24144 6458 Acked Write 6 Sent Done Done false
24145 6459 Acked Flush 0 Sent Done Done false
24146 6460 Acked Write 2 Sent Done Done false
24147 6461 Acked Read 8 Sent Done Done false
24148 6462 Acked Write 10 Sent Done Done false
24149 6463 Acked Read 9 Sent Done Done false
24150 6464 Acked Read 7 Sent Done Done false
24151 6465 Acked Write 10 Sent Done Done false
24152 6466 Acked Write 9 Sent Done Done false
24153 6467 Acked Read 8 Sent Done Done false
24154 6468 Acked Read 7 Sent Done Done false
24155 6469 Acked Flush 0 Sent Done Done false
24156 6470 Acked Read 3 Sent Done Done false
24157 6471 Acked Flush 0 Sent Done Done false
24158 6472 Acked Read 1 Sent Done Done false
24159 6473 Acked Write 7 Sent Done Done false
24160 6474 Acked Read 10 Sent Done Done false
24161 6475 Acked Write 7 Sent Done Done false
24162 6476 Acked Read 5 Sent Done Done false
24163 6477 Acked Write 5 Sent Done Done false
24164 6478 Acked Write 4 Sent Done Done false
24165 6479 Acked Flush 0 Sent Done Done false
24166 6480 Acked Write 8 Sent Done Done false
24167 6481 Acked Write 7 Sent Done Done false
24168 6482 Acked Write 6 Sent Done Done false
24169 6483 Acked Read 6 Sent Done Done false
24170 6484 Acked Read 3 Sent Done Done false
24171 6485 Acked Read 1 Sent Done Done false
24172 6486 Acked Read 10 Sent Done Done false
24173 6487 Acked Read 9 Sent Done Done false
24174 6488 Acked Write 9 Sent Done Done false
24175 6489 Acked Write 10 Sent Done Done false
24176 6490 Acked Flush 0 Sent Done Done false
24177 6491 Acked Write 2 Sent Done Done false
24178 6492 Acked Read 1 Sent Done Done false
24179 6493 Acked Write 2 Sent Done Done false
24180 6494 Acked Write 2 Sent Done Done false
24181 6495 Acked Read 3 Sent Done Done false
24182 6496 Acked Read 3 Sent Done Done false
24183 6497 Acked Read 2 Sent Done Done false
24184 6498 Acked Read 1 Sent Done Done false
24185 6499 Acked Write 5 Sent Done Done false
24186 6500 Acked Read 4 Sent Done Done false
24187 6501 Acked Read 8 Sent Done Done false
24188 6502 Acked Write 5 Sent Done Done false
24189 6503 Acked Write 10 Sent Done Done false
24190 6504 Acked Read 6 Sent Done Done false
24191 6505 Acked Read 7 Sent Done Done false
24192 6506 Acked Read 10 Sent Done Done false
24193 6507 Acked Write 4 Sent Done Done false
24194 6508 Acked Flush 0 Sent Done Done false
24195 6509 Acked Read 4 Sent Done Done false
24196 6510 Acked Read 1 Sent Done Done false
24197 6511 Acked Read 1 Sent Done Done false
24198 6512 Acked Write 8 Sent Done Done false
24199 6513 Acked Write 3 Sent Done Done false
24200 6514 Acked Read 1 Sent Done Done false
24201 6515 Acked Write 1 Sent Done Done false
24202 6516 Acked Read 4 Sent Done Done false
24203 6517 Acked Read 4 Sent Done Done false
24204 6518 Acked Read 5 Sent Done Done false
24205 6519 Acked Write 6 Sent Done Done false
24206 6520 Acked Write 1 Sent Done Done false
24207 6521 Acked Write 5 Sent Done Done false
24208 6522 Acked Write 10 Sent Done Done false
24209 6523 Acked Read 2 Sent Done Done false
24210 6524 Acked Write 8 Sent Done Done false
24211 6525 Acked Write 10 Sent Done Done false
24212 6526 Acked Read 1 Sent Done Done false
24213 6527 Acked Write 10 Sent Done Done false
24214 6528 Acked Read 6 Sent Done Done false
24215 6529 Acked Read 9 Sent Done Done false
24216 6530 Acked Flush 0 Sent Done Done false
24217 6531 Acked Read 1 Sent Done Done false
24218 6532 Acked Read 7 Sent Done Done false
24219 6533 Acked Write 3 Sent Done Done false
24220 6534 Acked Read 2 Sent Done Done false
24221 6535 Acked Write 9 Sent Done Done false
24222 6536 Acked Read 4 Sent Done Done false
24223 6537 Acked Read 4 Sent Done Done false
24224 6538 Acked Read 6 Sent Done Done false
24225 6539 Acked Write 5 Sent Done Done false
24226 6540 Acked Read 5 Sent Done Done false
24227 6541 Acked Write 6 Sent Done Done false
24228 6542 Acked Write 7 Sent Done Done false
24229 6543 Acked Write 8 Sent Done Done false
24230 6544 Acked Write 8 Sent Done Done false
24231 6545 Acked Read 8 Sent Done Done false
24232 6546 Acked Write 1 Sent Done Done false
24233 6547 Acked Write 7 Sent Done Done false
24234 6548 Acked Read 8 Sent Done Done false
24235 6549 Acked Write 10 Sent Done Done false
24236 6550 Acked Read 7 Sent Done Done false
24237 6551 Acked Read 8 Sent Done Done false
24238 6552 Acked Flush 0 Sent Done Done false
24239 6553 Acked Read 6 Sent Done Done false
24240 6554 Acked Read 3 Sent Done Done false
24241 6555 Acked Write 8 Sent Done Done false
24242 6556 Acked Read 3 Sent Done Done false
24243 6557 Acked Read 9 Sent Done Done false
24244 6558 Acked Write 2 Sent Done Done false
24245 6559 Acked Write 2 Sent Done Done false
24246 6560 Acked Flush 0 Sent Done Done false
24247 6561 Acked Read 4 Sent Done Done false
24248 6562 Acked Write 3 Sent Done Done false
24249 6563 Acked Write 10 Sent Done Done false
24250 6564 Acked Read 9 Sent Done Done false
24251 6565 Acked Write 2 Sent Done Done false
24252 6566 Acked Write 1 Sent Done Done false
24253 6567 Acked Read 1 Sent Done Done false
24254 6568 Acked Read 4 Sent Done Done false
24255 6569 Acked Read 1 Sent Done Done false
24256 6570 Acked Write 6 Sent Done Done false
24257 6571 Acked Write 3 Sent Done Done false
24258 6572 Acked Read 9 Sent Done Done false
24259 6573 Acked Write 10 Sent Done Done false
24260 6574 Acked Write 1 Sent Done Done false
24261 6575 Acked Read 10 Sent Done Done false
24262 6576 Acked Write 3 Sent Done Done false
24263 6577 Acked Write 4 Sent Done Done false
24264 6578 Acked Write 1 Sent Done Done false
24265 6579 Acked Read 2 Sent Done Done false
24266 6580 Acked Write 6 Sent Done Done false
24267 6581 Acked Read 1 Sent Done Done false
24268 6582 Acked Write 8 Sent Done Done false
24269 6583 Acked Write 5 Sent Done Done false
24270 6584 Acked Read 8 Sent Done Done false
24271 6585 Acked Read 6 Sent Done Done false
24272 6586 Acked Flush 0 Sent Done Done false
24273 6587 Acked Read 5 Sent Done Done false
24274 6588 Acked Read 4 Sent Done Done false
24275 6589 Acked Write 9 Sent Done Done false
24276 6590 Acked Write 6 Sent Done Done false
24277 6591 Acked Write 5 Sent Done Done false
24278 6592 Acked Write 5 Sent Done Done false
24279 6593 Acked Read 2 Sent Done Done false
24280 6594 Acked Read 8 Sent Done Done false
24281 6595 Acked Read 1 Sent Done Done false
24282 6596 Acked Write 8 Sent Done Done false
24283 6597 Acked Flush 0 Sent Done Done false
24284 6598 Acked Read 3 Sent Done Done false
24285 6599 Acked Write 7 Sent Done Done false
24286 6600 Acked Read 10 Sent Done Done false
24287 6601 Acked Read 4 Sent Done Done false
24288 6602 Acked Write 4 Sent Done Done false
24289 6603 Acked Write 4 Sent Done Done false
24290 6604 Acked Read 1 Sent Done Done false
24291 6605 Acked Write 9 Sent Done Done false
24292 6606 Acked Read 3 Sent Done Done false
24293 6607 Acked Read 7 Sent Done Done false
24294 6608 Acked Write 10 Sent Done Done false
24295 6609 Acked Read 8 Sent Done Done false
24296 6610 Acked Write 6 Sent Done Done false
24297 6611 Acked Write 1 Sent Done Done false
24298 6612 Acked Write 10 Sent Done Done false
24299 6613 Acked Read 8 Sent Done Done false
24300 6614 Acked Write 3 Sent Done Done false
24301 6615 Acked Read 2 Sent Done Done false
24302 6616 Acked Read 5 Sent Done Done false
24303 6617 Acked Read 10 Sent Done Done false
24304 6618 Acked Read 10 Sent Done Done false
24305 6619 Acked Write 9 Sent Done Done false
24306 6620 Acked Write 3 Sent Done Done false
24307 6621 Acked Read 7 Sent Done Done false
24308 6622 Acked Write 3 Sent Done Done false
24309 6623 Acked Read 1 Sent Done Done false
24310 6624 Acked Write 8 Sent Done Done false
24311 6625 Acked Write 9 Sent Done Done false
24312 6626 Acked Read 4 Sent Done Done false
24313 6627 Acked Read 1 Sent Done Done false
24314 6628 Acked Read 7 Sent Done Done false
24315 6629 Acked Read 1 Sent Done Done false
24316 6630 Acked Flush 0 Sent Done Done false
24317 6631 Acked Write 5 Sent Done Done false
24318 6632 Acked Write 5 Sent Done Done false
24319 6633 Acked Write 9 Sent Done Done false
24320 6634 Acked Read 1 Sent Done Done false
24321 6635 Acked Write 6 Sent Done Done false
24322 6636 Acked Flush 0 Sent Done Done false
24323 6637 Acked Read 1 Sent Done Done false
24324 6638 Acked Read 5 Sent Done Done false
24325 6639 Acked Read 5 Sent Done Done false
24326 6640 Acked Write 5 Sent Done Done false
24327 6641 Acked Read 9 Sent Done Done false
24328 6642 Acked Flush 0 Sent Done Done false
24329 6643 Acked Read 7 Sent Done Done false
24330 6644 Acked Read 9 Sent Done Done false
24331 6645 Acked Read 8 Sent Done Done false
24332 6646 Acked Read 9 Sent Done Done false
24333 6647 Acked Flush 0 Sent Done Done false
24334 6648 Acked Read 6 Sent Done Done false
24335 6649 Acked Read 3 Sent Done Done false
24336 6650 Acked Read 4 Sent Done Done false
24337 6651 Acked Read 3 Sent Done Done false
24338 6652 Acked Write 9 Sent Done Done false
24339 6653 Acked Read 4 Sent Done Done false
24340 6654 Acked Write 3 Sent Done Done false
24341 6655 Acked Read 5 Sent Done Done false
24342 6656 Acked Read 2 Sent Done Done false
24343 6657 Acked Flush 0 Sent Done Done false
24344 6658 Acked Write 2 Sent Done Done false
24345 6659 Acked Read 7 Sent Done Done false
24346 6660 Acked Read 10 Sent Done Done false
24347 6661 Acked Write 3 Sent Done Done false
24348 6662 Acked Write 10 Sent Done Done false
24349 6663 Acked Read 5 Sent Done Done false
24350 6664 Acked Read 2 Sent Done Done false
24351 6665 Acked Read 9 Sent Done Done false
24352 6666 Acked Write 10 Sent Done Done false
24353 6667 Acked Read 3 Sent Done Done false
24354 6668 Acked Read 3 Sent Done Done false
24355 6669 Acked Write 1 Sent Done Done false
24356 6670 Acked Write 4 Sent Done Done false
24357 6671 Acked Read 2 Sent Done Done false
24358 6672 Acked Read 4 Sent Done Done false
24359 6673 Acked Read 1 Sent Done Done false
24360 6674 Acked Write 5 Sent Done Done false
24361 6675 Acked Write 3 Sent Done Done false
24362 6676 Acked Flush 0 Sent Done Done false
24363 6677 Acked Write 1 Sent Done Done false
24364 6678 Acked Write 1 Sent Done Done false
24365 6679 Acked Read 10 Sent Done Done false
24366 6680 Acked Read 2 Sent Done Done false
24367 6681 Acked Read 8 Sent Done Done false
24368 6682 Acked Write 3 Sent Done Done false
24369 6683 Acked Write 10 Sent Done Done false
24370 6684 Acked Flush 0 Sent Done Done false
24371 6685 Acked Write 3 Sent Done Done false
24372 6686 Acked Read 1 Sent Done Done false
24373 6687 Acked Write 1 Sent Done Done false
24374 6688 Acked Read 7 Sent Done Done false
24375 6689 Acked Write 3 Sent Done Done false
24376 6690 Acked Read 1 Sent Done Done false
24377 6691 Acked Write 7 Sent Done Done false
24378 6692 Acked Flush 0 Sent Done Done false
24379 6693 Acked Write 5 Sent Done Done false
24380 6694 Acked Write 9 Sent Done Done false
24381 6695 Acked Read 10 Sent Done Done false
24382 6696 Acked Write 3 Sent Done Done false
24383 6697 Acked Flush 0 Sent Done Done false
24384 6698 Acked Flush 0 Sent Done Done false
24385 6699 Acked Read 3 Sent Done Done false
24386 6700 Acked Read 8 Sent Done Done false
24387 6701 Acked Read 3 Sent Done Done false
24388 6702 Acked Read 6 Sent Done Done false
24389 6703 Acked Read 10 Sent Done Done false
24390 6704 Acked Read 4 Sent Done Done false
24391 6705 Acked Read 7 Sent Done Done false
24392 6706 Acked Read 10 Sent Done Done false
24393 6707 Acked Read 3 Sent Done Done false
24394 6708 Acked Write 3 Sent Done Done false
24395 6709 Acked Write 9 Sent Done Done false
24396 6710 Acked Read 4 Sent Done Done false
24397 6711 Acked Read 7 Sent Done Done false
24398 6712 Acked Write 9 Sent Done Done false
24399 6713 Acked Write 2 Sent Done Done false
24400 6714 Acked Read 3 Sent Done Done false
24401 6715 Acked Write 9 Sent Done Done false
24402 6716 Acked Flush 0 Sent Done Done false
24403 6717 Acked Write 3 Sent Done Done false
24404 6718 Acked Read 6 Sent Done Done false
24405 6719 Acked Flush 0 Sent Done Done false
24406 6720 Acked Read 5 Sent Done Done false
24407 6721 Acked Read 1 Sent Done Done false
24408 6722 Acked Read 4 Sent Done Done false
24409 6723 Acked Write 5 Sent Done Done false
24410 6724 Acked Write 3 Sent Done Done false
24411 6725 Acked Write 1 Sent Done Done false
24412 6726 Acked Read 2 Sent Done Done false
24413 6727 Acked Read 6 Sent Done Done false
24414 6728 Acked Read 9 Sent Done Done false
24415 6729 Acked Read 6 Sent Done Done false
24416 6730 Acked Flush 0 Sent Done Done false
24417 6731 Acked Write 4 Sent Done Done false
24418 6732 Acked Read 3 Sent Done Done false
24419 6733 Acked Read 2 Sent Done Done false
24420 6734 Acked Write 5 Sent Done Done false
24421 6735 Acked Flush 0 Sent Done Done false
24422 STATES DS:0 DS:1 DS:2 TOTAL
24423 Sent 302 0 0 302
24424 Done 0 302 302 604
24425 Skipped 0 0 0 0
24426 Error 0 0 0 0
24427 Last Flush: 6433 6735 6735
24428 Downstairs last five completed: 6433 6432 6431 6430 6429
24429 Upstairs last five completed: 6735 6734 6733 6732 6731
24430 CLIENT: Up:0 ds:302 act:2
244312024-11-15T02:02:34.339ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
244322024-11-15T02:02:34.339ZWARNcrucible: ds_connection connect to 127.0.0.1:8810 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
244332024-11-15T02:02:34.339ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
244342024-11-15T02:02:34.339ZWARNcrucible: downstairs task for 0 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
244352024-11-15T02:02:34.339ZINFOcrucible: client task is exiting = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
24436 [8810] got start action
24437 [8810] I am going to restart
24438 Starting downstairs at port 8810
24439 Make output file at "/tmp/dsc/downstairs-8810.txt"
24440 [8810][0] reports Starting
24441 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1224)
24442 [8810][0] reports Running
24443 ----------------------------------------------------------------
24444 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
24445 States: Offline Active Active
24446 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
24447 6434 Acked Read 1 Sent Done Done false
24448 6435 Acked Read 6 Sent Done Done false
24449 6436 Acked Write 3 Sent Done Done false
24450 6437 Acked Read 1 Sent Done Done false
24451 6438 Acked Read 6 Sent Done Done false
24452 6439 Acked Read 2 Sent Done Done false
24453 6440 Acked Write 8 Sent Done Done false
24454 6441 Acked Read 7 Sent Done Done false
24455 6442 Acked Write 10 Sent Done Done false
24456 6443 Acked Write 1 Sent Done Done false
24457 6444 Acked Read 1 Sent Done Done false
24458 6445 Acked Read 3 Sent Done Done false
24459 6446 Acked Read 10 Sent Done Done false
24460 6447 Acked Write 1 Sent Done Done false
24461 6448 Acked Write 2 Sent Done Done false
24462 6449 Acked Write 8 Sent Done Done false
24463 6450 Acked Write 2 Sent Done Done false
24464 6451 Acked Write 8 Sent Done Done false
24465 6452 Acked Read 5 Sent Done Done false
24466 6453 Acked Write 1 Sent Done Done false
24467 6454 Acked Read 9 Sent Done Done false
24468 6455 Acked Flush 0 Sent Done Done false
24469 6456 Acked Read 6 Sent Done Done false
24470 6457 Acked Read 7 Sent Done Done false
24471 6458 Acked Write 6 Sent Done Done false
24472 6459 Acked Flush 0 Sent Done Done false
24473 6460 Acked Write 2 Sent Done Done false
24474 6461 Acked Read 8 Sent Done Done false
24475 6462 Acked Write 10 Sent Done Done false
24476 6463 Acked Read 9 Sent Done Done false
24477 6464 Acked Read 7 Sent Done Done false
24478 6465 Acked Write 10 Sent Done Done false
24479 6466 Acked Write 9 Sent Done Done false
24480 6467 Acked Read 8 Sent Done Done false
24481 6468 Acked Read 7 Sent Done Done false
24482 6469 Acked Flush 0 Sent Done Done false
24483 6470 Acked Read 3 Sent Done Done false
24484 6471 Acked Flush 0 Sent Done Done false
24485 6472 Acked Read 1 Sent Done Done false
24486 6473 Acked Write 7 Sent Done Done false
24487 6474 Acked Read 10 Sent Done Done false
24488 6475 Acked Write 7 Sent Done Done false
24489 6476 Acked Read 5 Sent Done Done false
24490 6477 Acked Write 5 Sent Done Done false
24491 6478 Acked Write 4 Sent Done Done false
24492 6479 Acked Flush 0 Sent Done Done false
24493 6480 Acked Write 8 Sent Done Done false
24494 6481 Acked Write 7 Sent Done Done false
24495 6482 Acked Write 6 Sent Done Done false
24496 6483 Acked Read 6 Sent Done Done false
24497 6484 Acked Read 3 Sent Done Done false
24498 6485 Acked Read 1 Sent Done Done false
24499 6486 Acked Read 10 Sent Done Done false
24500 6487 Acked Read 9 Sent Done Done false
24501 6488 Acked Write 9 Sent Done Done false
24502 6489 Acked Write 10 Sent Done Done false
24503 6490 Acked Flush 0 Sent Done Done false
24504 6491 Acked Write 2 Sent Done Done false
24505 6492 Acked Read 1 Sent Done Done false
24506 6493 Acked Write 2 Sent Done Done false
24507 6494 Acked Write 2 Sent Done Done false
24508 6495 Acked Read 3 Sent Done Done false
24509 6496 Acked Read 3 Sent Done Done false
24510 6497 Acked Read 2 Sent Done Done false
24511 6498 Acked Read 1 Sent Done Done false
24512 6499 Acked Write 5 Sent Done Done false
24513 6500 Acked Read 4 Sent Done Done false
24514 6501 Acked Read 8 Sent Done Done false
24515 6502 Acked Write 5 Sent Done Done false
24516 6503 Acked Write 10 Sent Done Done false
24517 6504 Acked Read 6 Sent Done Done false
24518 6505 Acked Read 7 Sent Done Done false
24519 6506 Acked Read 10 Sent Done Done false
24520 6507 Acked Write 4 Sent Done Done false
24521 6508 Acked Flush 0 Sent Done Done false
24522 6509 Acked Read 4 Sent Done Done false
24523 6510 Acked Read 1 Sent Done Done false
24524 6511 Acked Read 1 Sent Done Done false
24525 6512 Acked Write 8 Sent Done Done false
24526 6513 Acked Write 3 Sent Done Done false
24527 6514 Acked Read 1 Sent Done Done false
24528 6515 Acked Write 1 Sent Done Done false
24529 6516 Acked Read 4 Sent Done Done false
24530 6517 Acked Read 4 Sent Done Done false
24531 6518 Acked Read 5 Sent Done Done false
24532 6519 Acked Write 6 Sent Done Done false
24533 6520 Acked Write 1 Sent Done Done false
24534 6521 Acked Write 5 Sent Done Done false
24535 6522 Acked Write 10 Sent Done Done false
24536 6523 Acked Read 2 Sent Done Done false
24537 6524 Acked Write 8 Sent Done Done false
24538 6525 Acked Write 10 Sent Done Done false
24539 6526 Acked Read 1 Sent Done Done false
24540 6527 Acked Write 10 Sent Done Done false
24541 6528 Acked Read 6 Sent Done Done false
24542 6529 Acked Read 9 Sent Done Done false
24543 6530 Acked Flush 0 Sent Done Done false
24544 6531 Acked Read 1 Sent Done Done false
24545 6532 Acked Read 7 Sent Done Done false
24546 6533 Acked Write 3 Sent Done Done false
24547 6534 Acked Read 2 Sent Done Done false
24548 6535 Acked Write 9 Sent Done Done false
24549 6536 Acked Read 4 Sent Done Done false
24550 6537 Acked Read 4 Sent Done Done false
24551 6538 Acked Read 6 Sent Done Done false
24552 6539 Acked Write 5 Sent Done Done false
24553 6540 Acked Read 5 Sent Done Done false
24554 6541 Acked Write 6 Sent Done Done false
24555 6542 Acked Write 7 Sent Done Done false
24556 6543 Acked Write 8 Sent Done Done false
24557 6544 Acked Write 8 Sent Done Done false
24558 6545 Acked Read 8 Sent Done Done false
24559 6546 Acked Write 1 Sent Done Done false
24560 6547 Acked Write 7 Sent Done Done false
24561 6548 Acked Read 8 Sent Done Done false
24562 6549 Acked Write 10 Sent Done Done false
24563 6550 Acked Read 7 Sent Done Done false
24564 6551 Acked Read 8 Sent Done Done false
24565 6552 Acked Flush 0 Sent Done Done false
24566 6553 Acked Read 6 Sent Done Done false
24567 6554 Acked Read 3 Sent Done Done false
24568 6555 Acked Write 8 Sent Done Done false
24569 6556 Acked Read 3 Sent Done Done false
24570 6557 Acked Read 9 Sent Done Done false
24571 6558 Acked Write 2 Sent Done Done false
24572 6559 Acked Write 2 Sent Done Done false
24573 6560 Acked Flush 0 Sent Done Done false
24574 6561 Acked Read 4 Sent Done Done false
24575 6562 Acked Write 3 Sent Done Done false
24576 6563 Acked Write 10 Sent Done Done false
24577 6564 Acked Read 9 Sent Done Done false
24578 6565 Acked Write 2 Sent Done Done false
24579 6566 Acked Write 1 Sent Done Done false
24580 6567 Acked Read 1 Sent Done Done false
24581 6568 Acked Read 4 Sent Done Done false
24582 6569 Acked Read 1 Sent Done Done false
24583 6570 Acked Write 6 Sent Done Done false
24584 6571 Acked Write 3 Sent Done Done false
24585 6572 Acked Read 9 Sent Done Done false
24586 6573 Acked Write 10 Sent Done Done false
24587 6574 Acked Write 1 Sent Done Done false
24588 6575 Acked Read 10 Sent Done Done false
24589 6576 Acked Write 3 Sent Done Done false
24590 6577 Acked Write 4 Sent Done Done false
24591 6578 Acked Write 1 Sent Done Done false
24592 6579 Acked Read 2 Sent Done Done false
24593 6580 Acked Write 6 Sent Done Done false
24594 6581 Acked Read 1 Sent Done Done false
24595 6582 Acked Write 8 Sent Done Done false
24596 6583 Acked Write 5 Sent Done Done false
24597 6584 Acked Read 8 Sent Done Done false
24598 6585 Acked Read 6 Sent Done Done false
24599 6586 Acked Flush 0 Sent Done Done false
24600 6587 Acked Read 5 Sent Done Done false
24601 6588 Acked Read 4 Sent Done Done false
24602 6589 Acked Write 9 Sent Done Done false
24603 6590 Acked Write 6 Sent Done Done false
24604 6591 Acked Write 5 Sent Done Done false
24605 6592 Acked Write 5 Sent Done Done false
24606 6593 Acked Read 2 Sent Done Done false
24607 6594 Acked Read 8 Sent Done Done false
24608 6595 Acked Read 1 Sent Done Done false
24609 6596 Acked Write 8 Sent Done Done false
24610 6597 Acked Flush 0 Sent Done Done false
24611 6598 Acked Read 3 Sent Done Done false
24612 6599 Acked Write 7 Sent Done Done false
24613 6600 Acked Read 10 Sent Done Done false
24614 6601 Acked Read 4 Sent Done Done false
24615 6602 Acked Write 4 Sent Done Done false
24616 6603 Acked Write 4 Sent Done Done false
24617 6604 Acked Read 1 Sent Done Done false
24618 6605 Acked Write 9 Sent Done Done false
24619 6606 Acked Read 3 Sent Done Done false
24620 6607 Acked Read 7 Sent Done Done false
24621 6608 Acked Write 10 Sent Done Done false
24622 6609 Acked Read 8 Sent Done Done false
24623 6610 Acked Write 6 Sent Done Done false
24624 6611 Acked Write 1 Sent Done Done false
24625 6612 Acked Write 10 Sent Done Done false
24626 6613 Acked Read 8 Sent Done Done false
24627 6614 Acked Write 3 Sent Done Done false
24628 6615 Acked Read 2 Sent Done Done false
24629 6616 Acked Read 5 Sent Done Done false
24630 6617 Acked Read 10 Sent Done Done false
24631 6618 Acked Read 10 Sent Done Done false
24632 6619 Acked Write 9 Sent Done Done false
24633 6620 Acked Write 3 Sent Done Done false
24634 6621 Acked Read 7 Sent Done Done false
24635 6622 Acked Write 3 Sent Done Done false
24636 6623 Acked Read 1 Sent Done Done false
24637 6624 Acked Write 8 Sent Done Done false
24638 6625 Acked Write 9 Sent Done Done false
24639 6626 Acked Read 4 Sent Done Done false
24640 6627 Acked Read 1 Sent Done Done false
24641 6628 Acked Read 7 Sent Done Done false
24642 6629 Acked Read 1 Sent Done Done false
24643 6630 Acked Flush 0 Sent Done Done false
24644 6631 Acked Write 5 Sent Done Done false
24645 6632 Acked Write 5 Sent Done Done false
24646 6633 Acked Write 9 Sent Done Done false
24647 6634 Acked Read 1 Sent Done Done false
24648 6635 Acked Write 6 Sent Done Done false
24649 6636 Acked Flush 0 Sent Done Done false
24650 6637 Acked Read 1 Sent Done Done false
24651 6638 Acked Read 5 Sent Done Done false
24652 6639 Acked Read 5 Sent Done Done false
24653 6640 Acked Write 5 Sent Done Done false
24654 6641 Acked Read 9 Sent Done Done false
24655 6642 Acked Flush 0 Sent Done Done false
24656 6643 Acked Read 7 Sent Done Done false
24657 6644 Acked Read 9 Sent Done Done false
24658 6645 Acked Read 8 Sent Done Done false
24659 6646 Acked Read 9 Sent Done Done false
24660 6647 Acked Flush 0 Sent Done Done false
24661 6648 Acked Read 6 Sent Done Done false
24662 6649 Acked Read 3 Sent Done Done false
24663 6650 Acked Read 4 Sent Done Done false
24664 6651 Acked Read 3 Sent Done Done false
24665 6652 Acked Write 9 Sent Done Done false
24666 6653 Acked Read 4 Sent Done Done false
24667 6654 Acked Write 3 Sent Done Done false
24668 6655 Acked Read 5 Sent Done Done false
24669 6656 Acked Read 2 Sent Done Done false
24670 6657 Acked Flush 0 Sent Done Done false
24671 6658 Acked Write 2 Sent Done Done false
24672 6659 Acked Read 7 Sent Done Done false
24673 6660 Acked Read 10 Sent Done Done false
24674 6661 Acked Write 3 Sent Done Done false
24675 6662 Acked Write 10 Sent Done Done false
24676 6663 Acked Read 5 Sent Done Done false
24677 6664 Acked Read 2 Sent Done Done false
24678 6665 Acked Read 9 Sent Done Done false
24679 6666 Acked Write 10 Sent Done Done false
24680 6667 Acked Read 3 Sent Done Done false
24681 6668 Acked Read 3 Sent Done Done false
24682 6669 Acked Write 1 Sent Done Done false
24683 6670 Acked Write 4 Sent Done Done false
24684 6671 Acked Read 2 Sent Done Done false
24685 6672 Acked Read 4 Sent Done Done false
24686 6673 Acked Read 1 Sent Done Done false
24687 6674 Acked Write 5 Sent Done Done false
24688 6675 Acked Write 3 Sent Done Done false
24689 6676 Acked Flush 0 Sent Done Done false
24690 6677 Acked Write 1 Sent Done Done false
24691 6678 Acked Write 1 Sent Done Done false
24692 6679 Acked Read 10 Sent Done Done false
24693 6680 Acked Read 2 Sent Done Done false
24694 6681 Acked Read 8 Sent Done Done false
24695 6682 Acked Write 3 Sent Done Done false
24696 6683 Acked Write 10 Sent Done Done false
24697 6684 Acked Flush 0 Sent Done Done false
24698 6685 Acked Write 3 Sent Done Done false
24699 6686 Acked Read 1 Sent Done Done false
24700 6687 Acked Write 1 Sent Done Done false
24701 6688 Acked Read 7 Sent Done Done false
24702 6689 Acked Write 3 Sent Done Done false
24703 6690 Acked Read 1 Sent Done Done false
24704 6691 Acked Write 7 Sent Done Done false
24705 6692 Acked Flush 0 Sent Done Done false
24706 6693 Acked Write 5 Sent Done Done false
24707 6694 Acked Write 9 Sent Done Done false
24708 6695 Acked Read 10 Sent Done Done false
24709 6696 Acked Write 3 Sent Done Done false
24710 6697 Acked Flush 0 Sent Done Done false
24711 6698 Acked Flush 0 Sent Done Done false
24712 6699 Acked Read 3 Sent Done Done false
24713 6700 Acked Read 8 Sent Done Done false
24714 6701 Acked Read 3 Sent Done Done false
24715 6702 Acked Read 6 Sent Done Done false
24716 6703 Acked Read 10 Sent Done Done false
24717 6704 Acked Read 4 Sent Done Done false
24718 6705 Acked Read 7 Sent Done Done false
24719 6706 Acked Read 10 Sent Done Done false
24720 6707 Acked Read 3 Sent Done Done false
24721 6708 Acked Write 3 Sent Done Done false
24722 6709 Acked Write 9 Sent Done Done false
24723 6710 Acked Read 4 Sent Done Done false
24724 6711 Acked Read 7 Sent Done Done false
24725 6712 Acked Write 9 Sent Done Done false
24726 6713 Acked Write 2 Sent Done Done false
24727 6714 Acked Read 3 Sent Done Done false
24728 6715 Acked Write 9 Sent Done Done false
24729 6716 Acked Flush 0 Sent Done Done false
24730 6717 Acked Write 3 Sent Done Done false
24731 6718 Acked Read 6 Sent Done Done false
24732 6719 Acked Flush 0 Sent Done Done false
24733 6720 Acked Read 5 Sent Done Done false
24734 6721 Acked Read 1 Sent Done Done false
24735 6722 Acked Read 4 Sent Done Done false
24736 6723 Acked Write 5 Sent Done Done false
24737 6724 Acked Write 3 Sent Done Done false
24738 6725 Acked Write 1 Sent Done Done false
24739 6726 Acked Read 2 Sent Done Done false
24740 6727 Acked Read 6 Sent Done Done false
24741 6728 Acked Read 9 Sent Done Done false
24742 6729 Acked Read 6 Sent Done Done false
24743 6730 Acked Flush 0 Sent Done Done false
24744 6731 Acked Write 4 Sent Done Done false
24745 6732 Acked Read 3 Sent Done Done false
24746 6733 Acked Read 2 Sent Done Done false
24747 6734 Acked Write 5 Sent Done Done false
24748 6735 Acked Flush 0 Sent Done Done false
24749 STATES DS:0 DS:1 DS:2 TOTAL
24750 Sent 302 0 0 302
24751 Done 0 302 302 604
24752 Skipped 0 0 0 0
24753 Error 0 0 0 0
24754 Last Flush: 6433 6735 6735
24755 Downstairs last five completed: 6433 6432 6431 6430 6429
24756 Upstairs last five completed: 6735 6734 6733 6732 6731
24757 CLIENT: Up:0 ds:302 act:2
24758 ----------------------------------------------------------------
24759 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
24760 States: Offline Active Active
24761 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
24762 6434 Acked Read 1 Sent Done Done false
24763 6435 Acked Read 6 Sent Done Done false
24764 6436 Acked Write 3 Sent Done Done false
24765 6437 Acked Read 1 Sent Done Done false
24766 6438 Acked Read 6 Sent Done Done false
24767 6439 Acked Read 2 Sent Done Done false
24768 6440 Acked Write 8 Sent Done Done false
24769 6441 Acked Read 7 Sent Done Done false
24770 6442 Acked Write 10 Sent Done Done false
24771 6443 Acked Write 1 Sent Done Done false
24772 6444 Acked Read 1 Sent Done Done false
24773 6445 Acked Read 3 Sent Done Done false
24774 6446 Acked Read 10 Sent Done Done false
24775 6447 Acked Write 1 Sent Done Done false
24776 6448 Acked Write 2 Sent Done Done false
24777 6449 Acked Write 8 Sent Done Done false
24778 6450 Acked Write 2 Sent Done Done false
24779 6451 Acked Write 8 Sent Done Done false
24780 6452 Acked Read 5 Sent Done Done false
24781 6453 Acked Write 1 Sent Done Done false
24782 6454 Acked Read 9 Sent Done Done false
24783 6455 Acked Flush 0 Sent Done Done false
24784 6456 Acked Read 6 Sent Done Done false
24785 6457 Acked Read 7 Sent Done Done false
24786 6458 Acked Write 6 Sent Done Done false
24787 6459 Acked Flush 0 Sent Done Done false
24788 6460 Acked Write 2 Sent Done Done false
24789 6461 Acked Read 8 Sent Done Done false
24790 6462 Acked Write 10 Sent Done Done false
24791 6463 Acked Read 9 Sent Done Done false
24792 6464 Acked Read 7 Sent Done Done false
24793 6465 Acked Write 10 Sent Done Done false
24794 6466 Acked Write 9 Sent Done Done false
24795 6467 Acked Read 8 Sent Done Done false
24796 6468 Acked Read 7 Sent Done Done false
24797 6469 Acked Flush 0 Sent Done Done false
24798 6470 Acked Read 3 Sent Done Done false
24799 6471 Acked Flush 0 Sent Done Done false
24800 6472 Acked Read 1 Sent Done Done false
24801 6473 Acked Write 7 Sent Done Done false
24802 6474 Acked Read 10 Sent Done Done false
24803 6475 Acked Write 7 Sent Done Done false
24804 6476 Acked Read 5 Sent Done Done false
24805 6477 Acked Write 5 Sent Done Done false
24806 6478 Acked Write 4 Sent Done Done false
24807 6479 Acked Flush 0 Sent Done Done false
24808 6480 Acked Write 8 Sent Done Done false
24809 6481 Acked Write 7 Sent Done Done false
24810 6482 Acked Write 6 Sent Done Done false
24811 6483 Acked Read 6 Sent Done Done false
24812 6484 Acked Read 3 Sent Done Done false
24813 6485 Acked Read 1 Sent Done Done false
24814 6486 Acked Read 10 Sent Done Done false
24815 6487 Acked Read 9 Sent Done Done false
24816 6488 Acked Write 9 Sent Done Done false
24817 6489 Acked Write 10 Sent Done Done false
24818 6490 Acked Flush 0 Sent Done Done false
24819 6491 Acked Write 2 Sent Done Done false
24820 6492 Acked Read 1 Sent Done Done false
24821 6493 Acked Write 2 Sent Done Done false
24822 6494 Acked Write 2 Sent Done Done false
24823 6495 Acked Read 3 Sent Done Done false
24824 6496 Acked Read 3 Sent Done Done false
24825 6497 Acked Read 2 Sent Done Done false
24826 6498 Acked Read 1 Sent Done Done false
24827 6499 Acked Write 5 Sent Done Done false
24828 6500 Acked Read 4 Sent Done Done false
24829 6501 Acked Read 8 Sent Done Done false
24830 6502 Acked Write 5 Sent Done Done false
24831 6503 Acked Write 10 Sent Done Done false
24832 6504 Acked Read 6 Sent Done Done false
24833 6505 Acked Read 7 Sent Done Done false
24834 6506 Acked Read 10 Sent Done Done false
24835 6507 Acked Write 4 Sent Done Done false
24836 6508 Acked Flush 0 Sent Done Done false
24837 6509 Acked Read 4 Sent Done Done false
24838 6510 Acked Read 1 Sent Done Done false
24839 6511 Acked Read 1 Sent Done Done false
24840 6512 Acked Write 8 Sent Done Done false
24841 6513 Acked Write 3 Sent Done Done false
24842 6514 Acked Read 1 Sent Done Done false
24843 6515 Acked Write 1 Sent Done Done false
24844 6516 Acked Read 4 Sent Done Done false
24845 6517 Acked Read 4 Sent Done Done false
24846 6518 Acked Read 5 Sent Done Done false
24847 6519 Acked Write 6 Sent Done Done false
24848 6520 Acked Write 1 Sent Done Done false
24849 6521 Acked Write 5 Sent Done Done false
24850 6522 Acked Write 10 Sent Done Done false
24851 6523 Acked Read 2 Sent Done Done false
24852 6524 Acked Write 8 Sent Done Done false
24853 6525 Acked Write 10 Sent Done Done false
24854 6526 Acked Read 1 Sent Done Done false
24855 6527 Acked Write 10 Sent Done Done false
24856 6528 Acked Read 6 Sent Done Done false
24857 6529 Acked Read 9 Sent Done Done false
24858 6530 Acked Flush 0 Sent Done Done false
24859 6531 Acked Read 1 Sent Done Done false
24860 6532 Acked Read 7 Sent Done Done false
24861 6533 Acked Write 3 Sent Done Done false
24862 6534 Acked Read 2 Sent Done Done false
24863 6535 Acked Write 9 Sent Done Done false
24864 6536 Acked Read 4 Sent Done Done false
24865 6537 Acked Read 4 Sent Done Done false
24866 6538 Acked Read 6 Sent Done Done false
24867 6539 Acked Write 5 Sent Done Done false
24868 6540 Acked Read 5 Sent Done Done false
24869 6541 Acked Write 6 Sent Done Done false
24870 6542 Acked Write 7 Sent Done Done false
24871 6543 Acked Write 8 Sent Done Done false
24872 6544 Acked Write 8 Sent Done Done false
24873 6545 Acked Read 8 Sent Done Done false
24874 6546 Acked Write 1 Sent Done Done false
24875 6547 Acked Write 7 Sent Done Done false
24876 6548 Acked Read 8 Sent Done Done false
24877 6549 Acked Write 10 Sent Done Done false
24878 6550 Acked Read 7 Sent Done Done false
24879 6551 Acked Read 8 Sent Done Done false
24880 6552 Acked Flush 0 Sent Done Done false
24881 6553 Acked Read 6 Sent Done Done false
24882 6554 Acked Read 3 Sent Done Done false
24883 6555 Acked Write 8 Sent Done Done false
24884 6556 Acked Read 3 Sent Done Done false
24885 6557 Acked Read 9 Sent Done Done false
24886 6558 Acked Write 2 Sent Done Done false
24887 6559 Acked Write 2 Sent Done Done false
24888 6560 Acked Flush 0 Sent Done Done false
24889 6561 Acked Read 4 Sent Done Done false
24890 6562 Acked Write 3 Sent Done Done false
24891 6563 Acked Write 10 Sent Done Done false
24892 6564 Acked Read 9 Sent Done Done false
24893 6565 Acked Write 2 Sent Done Done false
24894 6566 Acked Write 1 Sent Done Done false
24895 6567 Acked Read 1 Sent Done Done false
24896 6568 Acked Read 4 Sent Done Done false
24897 6569 Acked Read 1 Sent Done Done false
24898 6570 Acked Write 6 Sent Done Done false
24899 6571 Acked Write 3 Sent Done Done false
24900 6572 Acked Read 9 Sent Done Done false
24901 6573 Acked Write 10 Sent Done Done false
24902 6574 Acked Write 1 Sent Done Done false
24903 6575 Acked Read 10 Sent Done Done false
24904 6576 Acked Write 3 Sent Done Done false
24905 6577 Acked Write 4 Sent Done Done false
24906 6578 Acked Write 1 Sent Done Done false
24907 6579 Acked Read 2 Sent Done Done false
24908 6580 Acked Write 6 Sent Done Done false
24909 6581 Acked Read 1 Sent Done Done false
24910 6582 Acked Write 8 Sent Done Done false
24911 6583 Acked Write 5 Sent Done Done false
24912 6584 Acked Read 8 Sent Done Done false
24913 6585 Acked Read 6 Sent Done Done false
24914 6586 Acked Flush 0 Sent Done Done false
24915 6587 Acked Read 5 Sent Done Done false
24916 6588 Acked Read 4 Sent Done Done false
24917 6589 Acked Write 9 Sent Done Done false
24918 6590 Acked Write 6 Sent Done Done false
24919 6591 Acked Write 5 Sent Done Done false
24920 6592 Acked Write 5 Sent Done Done false
24921 6593 Acked Read 2 Sent Done Done false
24922 6594 Acked Read 8 Sent Done Done false
24923 6595 Acked Read 1 Sent Done Done false
24924 6596 Acked Write 8 Sent Done Done false
24925 6597 Acked Flush 0 Sent Done Done false
24926 6598 Acked Read 3 Sent Done Done false
24927 6599 Acked Write 7 Sent Done Done false
24928 6600 Acked Read 10 Sent Done Done false
24929 6601 Acked Read 4 Sent Done Done false
24930 6602 Acked Write 4 Sent Done Done false
24931 6603 Acked Write 4 Sent Done Done false
24932 6604 Acked Read 1 Sent Done Done false
24933 6605 Acked Write 9 Sent Done Done false
24934 6606 Acked Read 3 Sent Done Done false
24935 6607 Acked Read 7 Sent Done Done false
24936 6608 Acked Write 10 Sent Done Done false
24937 6609 Acked Read 8 Sent Done Done false
24938 6610 Acked Write 6 Sent Done Done false
24939 6611 Acked Write 1 Sent Done Done false
24940 6612 Acked Write 10 Sent Done Done false
24941 6613 Acked Read 8 Sent Done Done false
24942 6614 Acked Write 3 Sent Done Done false
24943 6615 Acked Read 2 Sent Done Done false
24944 6616 Acked Read 5 Sent Done Done false
24945 6617 Acked Read 10 Sent Done Done false
24946 6618 Acked Read 10 Sent Done Done false
24947 6619 Acked Write 9 Sent Done Done false
24948 6620 Acked Write 3 Sent Done Done false
24949 6621 Acked Read 7 Sent Done Done false
24950 6622 Acked Write 3 Sent Done Done false
24951 6623 Acked Read 1 Sent Done Done false
24952 6624 Acked Write 8 Sent Done Done false
24953 6625 Acked Write 9 Sent Done Done false
24954 6626 Acked Read 4 Sent Done Done false
24955 6627 Acked Read 1 Sent Done Done false
24956 6628 Acked Read 7 Sent Done Done false
24957 6629 Acked Read 1 Sent Done Done false
24958 6630 Acked Flush 0 Sent Done Done false
24959 6631 Acked Write 5 Sent Done Done false
24960 6632 Acked Write 5 Sent Done Done false
24961 6633 Acked Write 9 Sent Done Done false
24962 6634 Acked Read 1 Sent Done Done false
24963 6635 Acked Write 6 Sent Done Done false
24964 6636 Acked Flush 0 Sent Done Done false
24965 6637 Acked Read 1 Sent Done Done false
24966 6638 Acked Read 5 Sent Done Done false
24967 6639 Acked Read 5 Sent Done Done false
24968 6640 Acked Write 5 Sent Done Done false
24969 6641 Acked Read 9 Sent Done Done false
24970 6642 Acked Flush 0 Sent Done Done false
24971 6643 Acked Read 7 Sent Done Done false
24972 6644 Acked Read 9 Sent Done Done false
24973 6645 Acked Read 8 Sent Done Done false
24974 6646 Acked Read 9 Sent Done Done false
24975 6647 Acked Flush 0 Sent Done Done false
24976 6648 Acked Read 6 Sent Done Done false
24977 6649 Acked Read 3 Sent Done Done false
24978 6650 Acked Read 4 Sent Done Done false
24979 6651 Acked Read 3 Sent Done Done false
24980 6652 Acked Write 9 Sent Done Done false
24981 6653 Acked Read 4 Sent Done Done false
24982 6654 Acked Write 3 Sent Done Done false
24983 6655 Acked Read 5 Sent Done Done false
24984 6656 Acked Read 2 Sent Done Done false
24985 6657 Acked Flush 0 Sent Done Done false
24986 6658 Acked Write 2 Sent Done Done false
24987 6659 Acked Read 7 Sent Done Done false
24988 6660 Acked Read 10 Sent Done Done false
24989 6661 Acked Write 3 Sent Done Done false
24990 6662 Acked Write 10 Sent Done Done false
24991 6663 Acked Read 5 Sent Done Done false
24992 6664 Acked Read 2 Sent Done Done false
24993 6665 Acked Read 9 Sent Done Done false
24994 6666 Acked Write 10 Sent Done Done false
24995 6667 Acked Read 3 Sent Done Done false
24996 6668 Acked Read 3 Sent Done Done false
24997 6669 Acked Write 1 Sent Done Done false
24998 6670 Acked Write 4 Sent Done Done false
24999 6671 Acked Read 2 Sent Done Done false
25000 6672 Acked Read 4 Sent Done Done false
25001 6673 Acked Read 1 Sent Done Done false
25002 6674 Acked Write 5 Sent Done Done false
25003 6675 Acked Write 3 Sent Done Done false
25004 6676 Acked Flush 0 Sent Done Done false
25005 6677 Acked Write 1 Sent Done Done false
25006 6678 Acked Write 1 Sent Done Done false
25007 6679 Acked Read 10 Sent Done Done false
25008 6680 Acked Read 2 Sent Done Done false
25009 6681 Acked Read 8 Sent Done Done false
25010 6682 Acked Write 3 Sent Done Done false
25011 6683 Acked Write 10 Sent Done Done false
25012 6684 Acked Flush 0 Sent Done Done false
25013 6685 Acked Write 3 Sent Done Done false
25014 6686 Acked Read 1 Sent Done Done false
25015 6687 Acked Write 1 Sent Done Done false
25016 6688 Acked Read 7 Sent Done Done false
25017 6689 Acked Write 3 Sent Done Done false
25018 6690 Acked Read 1 Sent Done Done false
25019 6691 Acked Write 7 Sent Done Done false
25020 6692 Acked Flush 0 Sent Done Done false
25021 6693 Acked Write 5 Sent Done Done false
25022 6694 Acked Write 9 Sent Done Done false
25023 6695 Acked Read 10 Sent Done Done false
25024 6696 Acked Write 3 Sent Done Done false
25025 6697 Acked Flush 0 Sent Done Done false
25026 6698 Acked Flush 0 Sent Done Done false
25027 6699 Acked Read 3 Sent Done Done false
25028 6700 Acked Read 8 Sent Done Done false
25029 6701 Acked Read 3 Sent Done Done false
25030 6702 Acked Read 6 Sent Done Done false
25031 6703 Acked Read 10 Sent Done Done false
25032 6704 Acked Read 4 Sent Done Done false
25033 6705 Acked Read 7 Sent Done Done false
25034 6706 Acked Read 10 Sent Done Done false
25035 6707 Acked Read 3 Sent Done Done false
25036 6708 Acked Write 3 Sent Done Done false
25037 6709 Acked Write 9 Sent Done Done false
25038 6710 Acked Read 4 Sent Done Done false
25039 6711 Acked Read 7 Sent Done Done false
25040 6712 Acked Write 9 Sent Done Done false
25041 6713 Acked Write 2 Sent Done Done false
25042 6714 Acked Read 3 Sent Done Done false
25043 6715 Acked Write 9 Sent Done Done false
25044 6716 Acked Flush 0 Sent Done Done false
25045 6717 Acked Write 3 Sent Done Done false
25046 6718 Acked Read 6 Sent Done Done false
25047 6719 Acked Flush 0 Sent Done Done false
25048 6720 Acked Read 5 Sent Done Done false
25049 6721 Acked Read 1 Sent Done Done false
25050 6722 Acked Read 4 Sent Done Done false
25051 6723 Acked Write 5 Sent Done Done false
25052 6724 Acked Write 3 Sent Done Done false
25053 6725 Acked Write 1 Sent Done Done false
25054 6726 Acked Read 2 Sent Done Done false
25055 6727 Acked Read 6 Sent Done Done false
25056 6728 Acked Read 9 Sent Done Done false
25057 6729 Acked Read 6 Sent Done Done false
25058 6730 Acked Flush 0 Sent Done Done false
25059 6731 Acked Write 4 Sent Done Done false
25060 6732 Acked Read 3 Sent Done Done false
25061 6733 Acked Read 2 Sent Done Done false
25062 6734 Acked Write 5 Sent Done Done false
25063 6735 Acked Flush 0 Sent Done Done false
25064 STATES DS:0 DS:1 DS:2 TOTAL
25065 Sent 302 0 0 302
25066 Done 0 302 302 604
25067 Skipped 0 0 0 0
25068 Error 0 0 0 0
25069 Last Flush: 6433 6735 6735
25070 Downstairs last five completed: 6433 6432 6431 6430 6429
25071 Upstairs last five completed: 6735 6734 6733 6732 6731
25072 CLIENT: Up:0 ds:302 act:2
250732024-11-15T02:02:44.341ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
250742024-11-15T02:02:44.341ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:34093) = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
250752024-11-15T02:02:44.342ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
250762024-11-15T02:02:44.343ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
250772024-11-15T02:02:44.343ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
250782024-11-15T02:02:44.343ZINFOcrucible: send last flush ID to this DS: 6433 = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
250792024-11-15T02:02:44.343ZINFOcrucible: Replied this last flush ID: 6433 = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
250802024-11-15T02:02:44.343ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
250812024-11-15T02:02:44.343ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
250822024-11-15T02:02:44.343ZINFOcrucible: [0] client re-new 302 jobs since flush 6433 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
250832024-11-15T02:02:44.343ZINFOcrucible: [0] Replayed 302 jobs since flush: 6433 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
25084 ----------------------------------------------------------------
25085 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
25086 Downstairs last five completed: 6735 6734 6733 6732 6731
25087 Upstairs last five completed: 6735 6734 6733 6732 6731
25088 CLIENT: Up:0 ds:0 act:3
25089 Replay: All jobs finished
25090 Main task has work to do, go find it
25091 got dsc Stop(1)
25092 stop 1
25093 Nov 15 02:02:44.580[8820] Got stop action so:false kr:false
25094 INFO request completed, latency_us: 356, response_code: 204, uri: /stop/cid/1, method: POST, req_id: a4484c17-7411-4b1e-a7bb-27f60a813b48, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
25095 Nov 15 02:02:44.580 INFO request completed, {"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2024-11-15T02:02:44.580919718Z","hostname":"w-01JCPRHT5F2MPA44KVJHM3S2PB","pid":1205,"":"io task","client":"1","":"downstairs","session_id":"971dcb2e-6891-4ecf-b367-1e04e52065ee"}
25096 latency_us: 136, response_code: 200, {"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2024-11-15T02:02:44.581002533Z","hostname":"w-01JCPRHT5F2MPA44KVJHM3S2PB","pid":1205,"":"io task","client":"1","":"downstairs","session_id":"971dcb2e-6891-4ecf-b367-1e04e52065ee"}
25097 uri: /state/cid/1, method: GET{"msg":"downstairs task for 1 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2024-11-15T02:02:44.581042161Z","hostname":"w-01JCPRHT5F2MPA44KVJHM3S2PB","pid":1205,"session_id":"971dcb2e-6891-4ecf-b367-1e04e52065ee"}
25098 , req_id: 22073475-ec6a-490e-9574-55491b6c0d4c, remote_addr: 127{"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-15T02:02:44.58106672Z","hostname":"w-01JCPRHT5F2MPA44KVJHM3S2PB","pid":1205,"client":"1","":"downstairs","session_id":"971dcb2e-6891-4ecf-b367-1e04e52065ee"}
25099 .0.0.1:40008, local_addr:{"msg":"[1] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-15T02:02:44.581082879Z","hostname":"w-01JCPRHT5F2MPA44KVJHM3S2PB","pid":1205,"client":"1","":"downstairs","session_id":"971dcb2e-6891-4ecf-b367-1e04e52065ee"}
25100 127.{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2024-11-15T02:02:44.581139295Z","hostname":"w-01JCPRHT5F2MPA44KVJHM3S2PB","pid":1205,"":"io task","client":"1","":"downstairs","session_id":"971dcb2e-6891-4ecf-b367-1e04e52065ee"}
25101 0.0.1:9998
25102 [8820] Exited with: ExitStatus(unix_wait_status(9))
25103 [8820][1] reports Exit
25104 Nov 15 02:02:48.583 INFO request completed, latency_us: 262, response_code: 200, uri: /state/cid/1, method: GET, req_id: 86075b6b-9172-4e22-ba81-e5fd03787c79, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
25105 001/300 Write block 1470 len 40960 data: 4 5 4 3 2 2 2 2 3 3
25106 002/300 Write block 2942 len 24576 data: 5 7 7 7 8 8
25107 003/300 Write block 4861 len 16384 data: 5 6 7 6
25108 004/300 Write block 2686 len 20480 data: 4 4 5 4 3
25109 005/300 Write block 2319 len 4096 data: 7
25110 006/300 Write block 2227 len 12288 data: 4 4 6
25111 007/300 Read block 3214 len 24576
25112 008/300 Write block 4138 len 16384 data: 3 3 5 4
25113 009/300 Read block 1989 len 40960
25114 010/300 Read block 1191 len 40960
25115 011/300 Read block 3332 len 12288
25116 012/300 Write block 679 len 36864 data: 6 6 6 5 5 4 3 2 2
25117 013/300 Read block 3033 len 16384
25118 014/300 Write block 3870 len 24576 data: 7 7 5 4 6 7
25119 015/300 Read block 2182 len 12288
25120 016/300 Write block 3225 len 32768 data: 5 5 3 3 3 3 3 4
25121 017/300 Read block 2354 len 28672
25122 018/300 Write block 3238 len 12288 data: 3 2 2
25123 019/300 Flush
25124 020/300 Read block 1714 len 32768
25125 021/300 Write block 811 len 36864 data: 5 4 5 4 3 3 3 2 2
25126 022/300 Read block 1929 len 32768
25127 023/300 Write block 3807 len 12288 data: 6 5 5
25128 024/300 Write block 2561 len 8192 data: 6 6
25129 025/300 Write block 4957 len 40960 data: 3 3 2 2 2 3 3 3 4 6
25130 026/300 Read block 2848 len 28672
25131 027/300 Write block 4478 len 36864 data: 4 3 3 3 3 3 3 3 3
25132 028/300 Read block 1517 len 20480
25133 029/300 Write block 4649 len 12288 data: 4 4 4
25134 030/300 Read block 3870 len 16384
25135 031/300 Read block 2356 len 20480
25136 032/300 Read block 3487 len 12288
25137 033/300 Read block 920 len 8192
25138 034/300 Write block 595 len 24576 data: 6 4 4 3 4 4
25139 035/300 Read block 215 len 40960
25140 036/300 Read block 3062 len 8192
25141 037/300 Read block 3494 len 28672
25142 038/300 Read block 4006 len 4096
25143 039/300 Flush
25144 040/300 Write block 2759 len 32768 data: 5 5 5 4 4 4 4 5
25145 041/300 Write block 2254 len 20480 data: 4 4 3 3 3
25146 042/300 Read block 3171 len 20480
25147 043/300 Read block 2582 len 20480
25148 044/300 Read block 4385 len 40960
25149 045/300 Read block 4112 len 28672
25150 046/300 Read block 3231 len 12288
25151 047/300 Read block 1055 len 12288
25152 048/300 Write block 3026 len 12288 data: 2 2 2
25153 049/300 Write block 1471 len 16384 data: 6 5 4 3
25154 050/300 Read block 4863 len 40960
25155 051/300 Write block 2871 len 16384 data: 5 5 5 5
25156 052/300 Read block 2656 len 4096
25157 053/300 Read block 3619 len 28672
25158 054/300 Read block 4459 len 24576
25159 055/300 Flush
25160 056/300 Write block 4025 len 4096 data: 4
25161 057/300 Read block 2483 len 16384
25162 058/300 Write block 1241 len 40960 data: 4 3 4 4 5 6 6 6 5 5
25163 059/300 Write block 4456 len 28672 data: 5 5 5 5 5 5 4
25164 060/300 Read block 4611 len 24576
25165 061/300 Read block 3433 len 8192
25166 062/300 Read block 863 len 24576
25167 063/300 Read block 1698 len 40960
25168 064/300 Read block 96 len 32768
25169 065/300 Read block 2139 len 16384
25170 066/300 Write block 2118 len 8192 data: 3 2
25171 067/300 Read block 4442 len 20480
25172 068/300 Flush
25173 069/300 Read block 57 len 16384
25174 070/300 Write block 4267 len 12288 data: 6 6 4
25175 071/300 Write block 4896 len 20480 data: 7 7 5 4 5
25176 072/300 Write block 4172 len 40960 data: 4 7 7 8 7 7 9 10 9 8
25177 073/300 Read block 1247 len 28672
25178 074/300 Write block 3973 len 36864 data: 4 4 3 4 4 5 5 5 4
25179 075/300 Write block 4998 len 8192 data: 2 2
25180 076/300 Read block 4050 len 8192
25181 077/300 Read block 2161 len 16384
25182 078/300 Read block 3482 len 8192
25183 079/300 Write block 547 len 8192 data: 4 4
25184 080/300 Read block 131 len 40960
25185 081/300 Flush
25186 082/300 Flush
25187 083/300 Write block 4594 len 36864 data: 5 4 4 3 2 3 4 5 5
25188 084/300 Flush
25189 085/300 Read block 3301 len 32768
25190 086/300 Read block 4728 len 20480
25191 087/300 Read block 4576 len 20480
25192 088/300 Write block 3962 len 16384 data: 6 5 5 3
25193 089/300 Write block 4383 len 8192 data: 2 2
25194 090/300 Flush
25195 091/300 Write block 436 len 36864 data: 2 3 3 3 3 4 3 3 3
25196 092/300 Read block 2021 len 24576
25197 093/300 Write block 75 len 20480 data: 6 4 4 5 8
25198 094/300 Write block 575 len 12288 data: 4 4 5
25199 095/300 Flush
25200 096/300 Read block 541 len 28672
25201 097/300 Read block 378 len 4096
25202 098/300 Write block 2666 len 4096 data: 3
25203 099/300 Read block 3907 len 20480
25204 100/300 Write block 4075 len 12288 data: 6 6 6
25205 101/300 Read block 4307 len 4096
25206 102/300 Write block 2027 len 40960 data: 7 5 4 3 3 3 3 3 3 3
25207 103/300 Read block 853 len 36864
25208 104/300 Write block 4900 len 8192 data: 6 7
25209 105/300 Read block 2516 len 20480
25210 106/300 Flush
25211 107/300 Read block 26 len 4096
25212 108/300 Write block 605 len 8192 data: 3 4
25213 109/300 Write block 2522 len 24576 data: 3 3 3 6 5 8
25214 110/300 Read block 4103 len 12288
25215 111/300 Read block 103 len 20480
25216 112/300 Read block 741 len 20480
25217 113/300 Read block 3422 len 8192
25218 114/300 Read block 3430 len 20480
25219 115/300 Read block 4141 len 24576
25220 116/300 Read block 1660 len 4096
25221 117/300 Read block 2350 len 4096
25222 118/300 Read block 3595 len 20480
25223 119/300 Flush
25224 120/300 Read block 1038 len 36864
25225 121/300 Read block 4417 len 8192
25226 122/300 Read block 4695 len 24576
25227 123/300 Write block 19 len 32768 data: 3 5 4 4 5 5 5 6
25228 124/300 Write block 969 len 16384 data: 3 3 2 2
25229 125/300 Write block 1257 len 8192 data: 3 4
25230 126/300 Read block 3912 len 40960
25231 127/300 Write block 2085 len 8192 data: 2 3
25232 128/300 Write block 4650 len 28672 data: 5 5 6 5 4 5 5
25233 129/300 Write block 4793 len 28672 data: 4 5 5 5 5 5 5
25234 130/300 Read block 780 len 28672
25235 131/300 Read block 1428 len 20480
25236 132/300 Read block 3496 len 24576
25237 133/300 Flush
25238 134/300 Read block 132 len 16384
25239 135/300 Read block 1721 len 28672
25240 136/300 Write block 3868 len 28672 data: 8 7 8 8 6 5 7
25241 137/300 Read block 4931 len 32768
25242 138/300 Write block 3323 len 32768 data: 3 4 3 3 4 4 4 5
25243 139/300 Write block 687 len 8192 data: 3 2
25244 140/300 Read block 2686 len 20480
25245 141/300 Flush
25246 142/300 Read block 510 len 20480
25247 143/300 Write block 1450 len 40960 data: 3 4 3 4 3 5 4 3 4 4
25248 144/300 Write block 3573 len 12288 data: 5 5 6
25249 145/300 Write block 1003 len 4096 data: 8
25250 146/300 Read block 4151 len 40960
25251 147/300 Read block 4821 len 24576
25252 148/300 Flush
25253 149/300 Flush
25254 150/300 Read block 1940 len 8192
25255 151/300 Read block 3003 len 8192
25256 152/300 Read block 3105 len 20480
25257 153/300 Write block 406 len 4096 data: 4
25258 154/300 Read block 2405 len 4096
25259 155/300 Read block 2711 len 20480
25260 156/300 Read block 590 len 12288
25261 157/300 Write block 588 len 24576 data: 5 5 5 6 6 6
25262 158/300 Read block 2424 len 8192
25263 159/300 Write block 4140 len 8192 data: 6 5
25264 160/300 Read block 4565 len 20480
25265 161/300 Write block 4269 len 32768 data: 5 5 5 6 5 4 3 3
25266 162/300 Write block 2244 len 36864 data: 4 4 3 3 3 3 2 2 3
25267 163/300 Write block 3249 len 40960 data: 8 6 7 7 8 7 6 7 8 8
25268 164/300 Read block 4097 len 16384
25269 165/300 Flush
25270 166/300 Write block 3081 len 20480 data: 10 5 4 4 4
25271 167/300 Write block 3324 len 24576 data: 5 4 4 5 5 5
25272 168/300 Write block 3648 len 24576 data: 7 8 8 8 7 7
25273 169/300 Write block 2457 len 32768 data: 9 7 7 8 8 7 8 7
25274 170/300 Write block 685 len 40960 data: 4 3 4 3 2 3 3 3 4 3
25275 171/300 Read block 3561 len 8192
25276 172/300 Write block 1853 len 16384 data: 4 4 4 4
25277 173/300 Write block 1185 len 16384 data: 3 3 6 5
25278 174/300 Read block 4892 len 4096
25279 175/300 Write block 2727 len 8192 data: 6 6
25280 176/300 Write block 425 len 4096 data: 2
25281 177/300 Write block 4873 len 24576 data: 4 5 8 8 7 5
25282 178/300 Read block 3977 len 32768
25283 179/300 Read block 1658 len 36864
25284 180/300 Read block 3657 len 4096
25285 181/300 Write block 3530 len 32768 data: 8 9 8 8 5 6 6 5
25286 182/300 Write block 2877 len 4096 data: 4
25287 183/300 Read block 3316 len 20480
25288 184/300 Read block 2586 len 28672
25289 185/300 Read block 3084 len 16384
25290 186/300 Read block 3221 len 4096
25291 187/300 Read block 4389 len 4096
25292 188/300 Write block 3862 len 8192 data: 5 5
25293 189/300 Read block 436 len 28672
25294 190/300 Read block 1369 len 32768
25295 191/300 Read block 944 len 24576
25296 192/300 Read block 2691 len 20480
25297 193/300 Read block 3876 len 8192
25298 194/300 Read block 462 len 16384
25299 195/300 Read block 3161 len 16384
25300 196/300 Flush
25301 197/300 Write block 2887 len 40960 data: 7 8 7 7 5 5 5 4 3 4
25302 198/300 Read block 1986 len 12288
25303 199/300 Read block 957 len 24576
25304 200/300 Read block 230 len 24576
25305 201/300 Write block 16 len 32768 data: 3 2 2 4 6 5 5 6
25306 202/300 Flush
25307 203/300 Write block 4193 len 4096 data: 7
25308 204/300 Write block 3332 len 40960 data: 4 5 5 6 5 5 4 5 4 2
25309 205/300 Read block 4079 len 12288
25310 206/300 Read block 2225 len 24576
25311 207/300 Flush
25312 208/300 Write block 3277 len 40960 data: 4 4 4 5 4 4 4 5 4 4
25313 209/300 Write block 2444 len 24576 data: 3 4 4 5 7 7
25314 210/300 Write block 2613 len 40960 data: 5 5 5 5 5 5 4 5 4 4
25315 211/300 Read block 1182 len 24576
25316 212/300 Write block 367 len 40960 data: 4 2 2 2 2 2 2 2 5 4
25317 213/300 Write block 696 len 24576 data: 4 4 4 5 6 6
25318 214/300 Read block 4499 len 28672
25319 215/300 Read block 2949 len 28672
25320 216/300 Write block 492 len 16384 data: 4 4 4 4
25321 217/300 Write block 1401 len 32768 data: 2 2 2 3 5 5 5 5
25322 218/300 Read block 3442 len 16384
25323 219/300 Write block 1154 len 12288 data: 2 2 2
25324 220/300 Read block 500 len 20480
25325 221/300 Read block 3069 len 40960
25326 222/300 Read block 1607 len 4096
25327 223/300 Write block 3910 len 4096 data: 3
25328 224/300 Read block 2796 len 12288
25329 225/300 Read block 756 len 12288
25330 226/300 Flush
25331 227/300 Write block 852 len 32768 data: 7 6 6 5 5 4 4 4
25332 228/300 Write block 2685 len 24576 data: 4 5 5 6 5 4
25333 229/300 Read block 138 len 4096
25334 230/300 Read block 942 len 24576
25335 231/300 Read block 1941 len 40960
25336 232/300 Write block 3775 len 20480 data: 5 4 3 3 3
25337 233/300 Write block 297 len 16384 data: 3 2 2 3
25338 234/300 Read block 1361 len 12288
25339 235/300 Read block 3038 len 8192
25340 236/300 Read block 4257 len 28672
25341 237/300 Read block 1639 len 8192
25342 238/300 Read block 2302 len 16384
25343 239/300 Read block 1909 len 4096
25344 240/300 Read block 3256 len 40960
25345 241/300 Read block 1696 len 20480
25346 242/300 Flush
25347 243/300 Read block 258 len 16384
25348 244/300 Read block 3827 len 24576
25349 245/300 Read block 391 len 36864
25350 246/300 Read block 2777 len 24576
25351 247/300 Read block 1671 len 24576
25352 248/300 Write block 4002 len 12288 data: 3 4 6
25353 249/300 Write block 4196 len 36864 data: 8 9 7 7 6 5 7 8 8
25354 250/300 Read block 4139 len 24576
25355 251/300 Write block 907 len 32768 data: 6 6 7 4 3 3 3 3
25356 252/300 Read block 3612 len 40960
25357 253/300 Read block 1192 len 36864
25358 254/300 Write block 3780 len 32768 data: 3 5 5 3 3 3 3 4
25359 255/300 Read block 3478 len 32768
25360 256/300 Read block 2312 len 24576
25361 257/300 Write block 1834 len 28672 data: 4 5 5 6 7 6 5
25362 258/300 Read block 3805 len 28672
25363 259/300 Read block 4678 len 12288
25364 260/300 Read block 4193 len 24576
25365 261/300 Read block 3462 len 36864
25366 262/300 Write block 2830 len 24576 data: 4 4 4 5 5 4
25367 263/300 Write block 2827 len 16384 data: 5 5 5 5
25368 264/300 Write block 4467 len 16384 data: 4 3 3 4
25369 265/300 Flush
25370 266/300 Read block 4783 len 16384
25371 267/300 Write block 2042 len 32768 data: 3 2 2 2 2 2 2 2
25372 268/300 Read block 305 len 28672
25373 269/300 Write block 4549 len 24576 data: 3 3 4 3 3 3
25374 270/300 Flush
25375 271/300 Write block 3649 len 20480 data: 9 9 9 8 8
25376 272/300 Read block 3839 len 36864
25377 273/300 Write block 1015 len 12288 data: 7 6 6
25378 274/300 Write block 3842 len 24576 data: 4 3 2 2 2 3
25379 275/300 Read block 264 len 36864
25380 276/300 Read block 2089 len 16384
25381 277/300 Write block 3354 len 8192 data: 7 7
25382 278/300 Read block 23 len 32768
25383 279/300 Write block 1072 len 24576 data: 6 6 3 4 2 2
25384 280/300 Read block 4472 len 36864
25385 281/300 Read block 1410 len 36864
25386 282/300 Read block 667 len 4096
25387 283/300 Read block 1492 len 16384
25388 284/300 Flush
25389 285/300 Read block 3918 len 28672
25390 286/300 Read block 2945 len 32768
25391 287/300 Read block 3817 len 24576
25392 288/300 Write block 3669 len 28672 data: 5 5 5 5 4 5 5
25393 289/300 Write block 567 len 36864 data: 2 3 2 4 4 5 5 5 5
25394 290/300 Write block 2092 len 24576 data: 3 3 2 2 4 5
25395 291/300 Read block 901 len 32768
25396 292/300 Flush
25397 293/300 Read block 1878 len 20480
25398 294/300 Write block 227 len 24576 data: 4 6 8 7 7 8
25399 295/300 Flush
25400 296/300 Write block 2868 len 12288 data: 3 5 5
25401 297/300 Read block 3613 len 8192
25402 298/300 Read block 942 len 36864
25403 299/300 Read block 1986 len 12288
25404 300/300 Read block 2650 len 24576
25405 301/300 Read block 3265 len 4096
25406 Main task has work to do, go find it
25407 got dsc Start(1)
25408 start 1
25409 Nov 15 02:02:48.823 INFO request completed, latency_us: 211, response_code: 204, uri: /start/cid/1, method: POST, req_id: a7fc185f-186e-4538-85a2-cce6cf856629, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
25410 Replay: started 1, returned:Ok(())
25411 ----------------------------------------------------------------
25412 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
25413 States: Active Offline Active
25414 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
25415 6736 Acked Write 10 Done Sent Done false
25416 6737 Acked Write 6 Done Sent Done false
25417 6738 Acked Write 4 Done Sent Done false
25418 6739 Acked Write 5 Done Sent Done false
25419 6740 Acked Write 1 Done Sent Done false
25420 6741 Acked Write 3 Done Sent Done false
25421 6742 Acked Read 6 Done Sent Done false
25422 6743 Acked Write 4 Done Sent Done false
25423 6744 Acked Read 10 Done Sent Done false
25424 6745 Acked Read 10 Done Sent Done false
25425 6746 Acked Read 3 Done Sent Done false
25426 6747 Acked Write 9 Done Sent Done false
25427 6748 Acked Read 4 Done Sent Done false
25428 6749 Acked Write 6 Done Sent Done false
25429 6750 Acked Read 3 Done Sent Done false
25430 6751 Acked Write 8 Done Sent Done false
25431 6752 Acked Read 7 Done Sent Done false
25432 6753 Acked Write 3 Done Sent Done false
25433 6754 Acked Flush 0 Done Sent Done false
25434 6755 Acked Read 8 Done Sent Done false
25435 6756 Acked Write 9 Done Sent Done false
25436 6757 Acked Read 8 Done Sent Done false
25437 6758 Acked Write 3 Done Sent Done false
25438 6759 Acked Write 2 Done Sent Done false
25439 6760 Acked Write 10 Done Sent Done false
25440 6761 Acked Read 7 Done Sent Done false
25441 6762 Acked Write 9 Done Sent Done false
25442 6763 Acked Read 5 Done Sent Done false
25443 6764 Acked Write 3 Done Sent Done false
25444 6765 Acked Read 4 Done Sent Done false
25445 6766 Acked Read 5 Done Sent Done false
25446 6767 Acked Read 3 Done Sent Done false
25447 6768 Acked Read 2 Done Sent Done false
25448 6769 Acked Write 6 Done Sent Done false
25449 6770 Acked Read 10 Done Sent Done false
25450 6771 Acked Read 2 Done Sent Done false
25451 6772 Acked Read 7 Done Sent Done false
25452 6773 Acked Read 1 Done Sent Done false
25453 6774 Acked Flush 0 Done Sent Done false
25454 6775 Acked Write 8 Done Sent Done false
25455 6776 Acked Write 5 Done Sent Done false
25456 6777 Acked Read 5 Done Sent Done false
25457 6778 Acked Read 5 Done Sent Done false
25458 6779 Acked Read 10 Done Sent Done false
25459 6780 Acked Read 7 Done Sent Done false
25460 6781 Acked Read 3 Done Sent Done false
25461 6782 Acked Read 3 Done Sent Done false
25462 6783 Acked Write 3 Done Sent Done false
25463 6784 Acked Write 4 Done Sent Done false
25464 6785 Acked Read 10 Done Sent Done false
25465 6786 Acked Write 4 Done Sent Done false
25466 6787 Acked Read 1 Done Sent Done false
25467 6788 Acked Read 7 Done Sent Done false
25468 6789 Acked Read 6 Done Sent Done false
25469 6790 Acked Flush 0 Done Sent Done false
25470 6791 Acked Write 1 Done Sent Done false
25471 6792 Acked Read 4 Done Sent Done false
25472 6793 Acked Write 10 Done Sent Done false
25473 6794 Acked Write 7 Done Sent Done false
25474 6795 Acked Read 6 Done Sent Done false
25475 6796 Acked Read 2 Done Sent Done false
25476 6797 Acked Read 6 Done Sent Done false
25477 6798 Acked Read 10 Done Sent Done false
25478 6799 Acked Read 8 Done Sent Done false
25479 6800 Acked Read 4 Done Sent Done false
25480 6801 Acked Write 2 Done Sent Done false
25481 6802 Acked Read 5 Done Sent Done false
25482 6803 Acked Flush 0 Done Sent Done false
25483 6804 Acked Read 4 Done Sent Done false
25484 6805 Acked Write 3 Done Sent Done false
25485 6806 Acked Write 5 Done Sent Done false
25486 6807 Acked Write 10 Done Sent Done false
25487 6808 Acked Read 7 Done Sent Done false
25488 6809 Acked Write 9 Done Sent Done false
25489 6810 Acked Write 2 Done Sent Done false
25490 6811 Acked Read 2 Done Sent Done false
25491 6812 Acked Read 4 Done Sent Done false
25492 6813 Acked Read 2 Done Sent Done false
25493 6814 Acked Write 2 Done Sent Done false
25494 6815 Acked Read 10 Done Sent Done false
25495 6816 Acked Flush 0 Done Sent Done false
25496 6817 Acked Flush 0 Done Sent Done false
25497 6818 Acked Write 9 Done Sent Done false
25498 6819 Acked Flush 0 Done Sent Done false
25499 6820 Acked Read 8 Done Sent Done false
25500 6821 Acked Read 5 Done Sent Done false
25501 6822 Acked Read 5 Done Sent Done false
25502 6823 Acked Write 4 Done Sent Done false
25503 6824 Acked Write 2 Done Sent Done false
25504 6825 Acked Flush 0 Done Sent Done false
25505 6826 Acked Write 9 Done Sent Done false
25506 6827 Acked Read 6 Done Sent Done false
25507 6828 Acked Write 5 Done Sent Done false
25508 6829 Acked Write 3 Done Sent Done false
25509 6830 Acked Flush 0 Done Sent Done false
25510 6831 Acked Read 7 Done Sent Done false
25511 6832 Acked Read 1 Done Sent Done false
25512 6833 Acked Write 1 Done Sent Done false
25513 6834 Acked Read 5 Done Sent Done false
25514 6835 Acked Write 3 Done Sent Done false
25515 6836 Acked Read 1 Done Sent Done false
25516 6837 Acked Write 10 Done Sent Done false
25517 6838 Acked Read 9 Done Sent Done false
25518 6839 Acked Write 2 Done Sent Done false
25519 6840 Acked Read 5 Done Sent Done false
25520 6841 Acked Flush 0 Done Sent Done false
25521 6842 Acked Read 1 Done Sent Done false
25522 6843 Acked Write 2 Done Sent Done false
25523 6844 Acked Write 6 Done Sent Done false
25524 6845 Acked Read 3 Done Sent Done false
25525 6846 Acked Read 5 Done Sent Done false
25526 6847 Acked Read 5 Done Sent Done false
25527 6848 Acked Read 2 Done Sent Done false
25528 6849 Acked Read 5 Done Sent Done false
25529 6850 Acked Read 6 Done Sent Done false
25530 6851 Acked Read 1 Done Sent Done false
25531 6852 Acked Read 1 Done Sent Done false
25532 6853 Acked Read 5 Done Sent Done false
25533 6854 Acked Flush 0 Done Sent Done false
25534 6855 Acked Read 9 Done Sent Done false
25535 6856 Acked Read 2 Done Sent Done false
25536 6857 Acked Read 6 Done Sent Done false
25537 6858 Acked Write 8 Done Sent Done false
25538 6859 Acked Write 4 Done Sent Done false
25539 6860 Acked Write 2 Done Sent Done false
25540 6861 Acked Read 10 Done Sent Done false
25541 6862 Acked Write 2 Done Sent Done false
25542 6863 Acked Write 7 Done Sent Done false
25543 6864 Acked Write 7 Done Sent Done false
25544 6865 Acked Read 7 Done Sent Done false
25545 6866 Acked Read 5 Done Sent Done false
25546 6867 Acked Read 6 Done Sent Done false
25547 6868 Acked Flush 0 Done Sent Done false
25548 6869 Acked Read 4 Done Sent Done false
25549 6870 Acked Read 7 Done Sent Done false
25550 6871 Acked Write 7 Done Sent Done false
25551 6872 Acked Read 8 Done Sent Done false
25552 6873 Acked Write 8 Done Sent Done false
25553 6874 Acked Write 2 Done Sent Done false
25554 6875 Acked Read 5 Done Sent Done false
25555 6876 Acked Flush 0 Done Sent Done false
25556 6877 Acked Read 5 Done Sent Done false
25557 6878 Acked Write 10 Done Sent Done false
25558 6879 Acked Write 3 Done Sent Done false
25559 6880 Acked Write 1 Done Sent Done false
25560 6881 Acked Read 10 Done Sent Done false
25561 6882 Acked Read 6 Done Sent Done false
25562 6883 Acked Flush 0 Done Sent Done false
25563 6884 Acked Flush 0 Done Sent Done false
25564 6885 Acked Read 2 Done Sent Done false
25565 6886 Acked Read 2 Done Sent Done false
25566 6887 Acked Read 5 Done Sent Done false
25567 6888 Acked Write 1 Done Sent Done false
25568 6889 Acked Read 1 Done Sent Done false
25569 6890 Acked Read 5 Done Sent Done false
25570 6891 Acked Read 3 Done Sent Done false
25571 6892 Acked Write 6 Done Sent Done false
25572 6893 Acked Read 2 Done Sent Done false
25573 6894 Acked Write 2 Done Sent Done false
25574 6895 Acked Read 5 Done Sent Done false
25575 6896 Acked Write 8 Done Sent Done false
25576 6897 Acked Write 9 Done Sent Done false
25577 6898 Acked Write 10 Done Sent Done false
25578 6899 Acked Read 4 Done Sent Done false
25579 6900 Acked Flush 0 Done Sent Done false
25580 6901 Acked Write 5 Done Sent Done false
25581 6902 Acked Write 6 Done Sent Done false
25582 6903 Acked Write 6 Done Sent Done false
25583 6904 Acked Write 8 Done Sent Done false
25584 6905 Acked Write 10 Done Sent Done false
25585 6906 Acked Read 2 Done Sent Done false
25586 6907 Acked Write 4 Done Sent Done false
25587 6908 Acked Write 4 Done Sent Done false
25588 6909 Acked Read 1 Done Sent Done false
25589 6910 Acked Write 2 Done Sent Done false
25590 6911 Acked Write 1 Done Sent Done false
25591 6912 Acked Write 6 Done Sent Done false
25592 6913 Acked Read 8 Done Sent Done false
25593 6914 Acked Read 9 Done Sent Done false
25594 6915 Acked Read 1 Done Sent Done false
25595 6916 Acked Write 8 Done Sent Done false
25596 6917 Acked Write 1 Done Sent Done false
25597 6918 Acked Read 5 Done Sent Done false
25598 6919 Acked Read 7 Done Sent Done false
25599 6920 Acked Read 4 Done Sent Done false
25600 6921 Acked Read 1 Done Sent Done false
25601 6922 Acked Read 1 Done Sent Done false
25602 6923 Acked Write 2 Done Sent Done false
25603 6924 Acked Read 7 Done Sent Done false
25604 6925 Acked Read 8 Done Sent Done false
25605 6926 Acked Read 6 Done Sent Done false
25606 6927 Acked Read 5 Done Sent Done false
25607 6928 Acked Read 2 Done Sent Done false
25608 6929 Acked Read 4 Done Sent Done false
25609 6930 Acked Read 4 Done Sent Done false
25610 6931 Acked Flush 0 Done Sent Done false
25611 6932 Acked Write 10 Done Sent Done false
25612 6933 Acked Read 3 Done Sent Done false
25613 6934 Acked Read 6 Done Sent Done false
25614 6935 Acked Read 6 Done Sent Done false
25615 6936 Acked Write 8 Done Sent Done false
25616 6937 Acked Flush 0 Done Sent Done false
25617 6938 Acked Write 1 Done Sent Done false
25618 6939 Acked Write 10 Done Sent Done false
25619 6940 Acked Read 3 Done Sent Done false
25620 6941 Acked Read 6 Done Sent Done false
25621 6942 Acked Flush 0 Done Sent Done false
25622 6943 Acked Write 10 Done Sent Done false
25623 6944 Acked Write 6 Done Sent Done false
25624 6945 Acked Write 10 Done Sent Done false
25625 6946 Acked Read 6 Done Sent Done false
25626 6947 Acked Write 10 Done Sent Done false
25627 6948 Acked Write 6 Done Sent Done false
25628 6949 Acked Read 7 Done Sent Done false
25629 6950 Acked Read 7 Done Sent Done false
25630 6951 Acked Write 4 Done Sent Done false
25631 6952 Acked Write 8 Done Sent Done false
25632 6953 Acked Read 4 Done Sent Done false
25633 6954 Acked Write 3 Done Sent Done false
25634 6955 Acked Read 5 Done Sent Done false
25635 6956 Acked Read 10 Done Sent Done false
25636 6957 Acked Read 1 Done Sent Done false
25637 6958 Acked Write 1 Done Sent Done false
25638 6959 Acked Read 3 Done Sent Done false
25639 6960 Acked Read 3 Done Sent Done false
25640 6961 Acked Flush 0 Done Sent Done false
25641 6962 Acked Write 8 Done Sent Done false
25642 6963 Acked Write 6 Done Sent Done false
25643 6964 Acked Read 1 Done Sent Done false
25644 6965 Acked Read 6 Done Sent Done false
25645 6966 Acked Read 10 Done Sent Done false
25646 6967 Acked Write 5 Done Sent Done false
25647 6968 Acked Write 4 Done Sent Done false
25648 6969 Acked Read 3 Done Sent Done false
25649 6970 Acked Read 2 Done Sent Done false
25650 6971 Acked Read 7 Done Sent Done false
25651 6972 Acked Read 2 Done Sent Done false
25652 6973 Acked Read 4 Done Sent Done false
25653 6974 Acked Read 1 Done Sent Done false
25654 6975 Acked Read 10 Done Sent Done false
25655 6976 Acked Read 5 Done Sent Done false
25656 6977 Acked Flush 0 Done Sent Done false
25657 6978 Acked Read 4 Done Sent Done false
25658 6979 Acked Read 6 Done Sent Done false
25659 6980 Acked Read 9 Done Sent Done false
25660 6981 Acked Read 6 Done Sent Done false
25661 6982 Acked Read 6 Done Sent Done false
25662 6983 Acked Write 3 Done Sent Done false
25663 6984 Acked Write 9 Done Sent Done false
25664 6985 Acked Read 6 Done Sent Done false
25665 6986 Acked Write 8 Done Sent Done false
25666 6987 Acked Read 10 Done Sent Done false
25667 6988 Acked Read 9 Done Sent Done false
25668 6989 Acked Write 8 Done Sent Done false
25669 6990 Acked Read 8 Done Sent Done false
25670 6991 Acked Read 6 Done Sent Done false
25671 6992 Acked Write 7 Done Sent Done false
25672 6993 Acked Read 7 Done Sent Done false
25673 6994 Acked Read 3 Done Sent Done false
25674 6995 Acked Read 6 Done Sent Done false
25675 6996 Acked Read 9 Done Sent Done false
25676 6997 Acked Write 6 Done Sent Done false
25677 6998 Acked Write 4 Done Sent Done false
25678 6999 Acked Write 4 Done Sent Done false
25679 7000 Acked Flush 0 Done Sent Done false
25680 7001 Acked Read 4 Done Sent Done false
25681 7002 Acked Write 8 Done Sent Done false
25682 7003 Acked Read 7 Done Sent Done false
25683 7004 Acked Write 6 Done Sent Done false
25684 7005 Acked Flush 0 Done Sent Done false
25685 7006 Acked Write 5 Done Sent Done false
25686 7007 Acked Read 9 Done Sent Done false
25687 7008 Acked Write 3 Done Sent Done false
25688 7009 Acked Write 6 Done Sent Done false
25689 7010 Acked Read 9 Done Sent Done false
25690 7011 Acked Read 4 Done Sent Done false
25691 7012 Acked Write 2 Done Sent Done false
25692 7013 Acked Read 8 Done Sent Done false
25693 7014 Acked Write 6 Done Sent Done false
25694 7015 Acked Read 9 Done Sent Done false
25695 7016 Acked Read 9 Done Sent Done false
25696 7017 Acked Read 1 Done Sent Done false
25697 7018 Acked Read 4 Done Sent Done false
25698 7019 Acked Flush 0 Done Sent Done false
25699 7020 Acked Read 7 Done Sent Done false
25700 7021 Acked Read 8 Done Sent Done false
25701 7022 Acked Read 6 Done Sent Done false
25702 7023 Acked Write 7 Done Sent Done false
25703 7024 Acked Write 9 Done Sent Done false
25704 7025 Acked Write 6 Done Sent Done false
25705 7026 Acked Read 8 Done Sent Done false
25706 7027 Acked Flush 0 Done Sent Done false
25707 7028 Acked Read 5 Done Sent Done false
25708 7029 Acked Write 6 Done Sent Done false
25709 7030 Acked Flush 0 Done Sent Done false
25710 7031 Acked Write 3 Done Sent Done false
25711 7032 Acked Read 2 Done Sent Done false
25712 7033 Acked Read 9 Done Sent Done false
25713 7034 Acked Read 3 Done Sent Done false
25714 7035 Acked Read 6 Done Sent Done false
25715 7036 Acked Read 1 Done Sent Done false
25716 STATES DS:0 DS:1 DS:2 TOTAL
25717 Sent 0 301 0 301
25718 Done 301 0 301 602
25719 Skipped 0 0 0 0
25720 Error 0 0 0 0
25721 Last Flush: 7030 6735 7030
25722 Downstairs last five completed: 6735 6734 6733 6732 6731
25723 Upstairs last five completed: 7036 7035 7034 7033 7032
25724 CLIENT: Up:0 ds:301 act:2
25725 [8820] got start action
25726 [8820] I am going to restart
25727 Starting downstairs at port 8820
25728 Make output file at "/tmp/dsc/downstairs-8820.txt"
25729 [8820][1] reports Starting
25730 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1225)
25731 [8820][1] reports Running
25732 ----------------------------------------------------------------
25733 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
25734 States: Active Offline Active
25735 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
25736 6736 Acked Write 10 Done Sent Done false
25737 6737 Acked Write 6 Done Sent Done false
25738 6738 Acked Write 4 Done Sent Done false
25739 6739 Acked Write 5 Done Sent Done false
25740 6740 Acked Write 1 Done Sent Done false
25741 6741 Acked Write 3 Done Sent Done false
25742 6742 Acked Read 6 Done Sent Done false
25743 6743 Acked Write 4 Done Sent Done false
25744 6744 Acked Read 10 Done Sent Done false
25745 6745 Acked Read 10 Done Sent Done false
25746 6746 Acked Read 3 Done Sent Done false
25747 6747 Acked Write 9 Done Sent Done false
25748 6748 Acked Read 4 Done Sent Done false
25749 6749 Acked Write 6 Done Sent Done false
25750 6750 Acked Read 3 Done Sent Done false
25751 6751 Acked Write 8 Done Sent Done false
25752 6752 Acked Read 7 Done Sent Done false
25753 6753 Acked Write 3 Done Sent Done false
25754 6754 Acked Flush 0 Done Sent Done false
25755 6755 Acked Read 8 Done Sent Done false
25756 6756 Acked Write 9 Done Sent Done false
25757 6757 Acked Read 8 Done Sent Done false
25758 6758 Acked Write 3 Done Sent Done false
25759 6759 Acked Write 2 Done Sent Done false
25760 6760 Acked Write 10 Done Sent Done false
25761 6761 Acked Read 7 Done Sent Done false
25762 6762 Acked Write 9 Done Sent Done false
25763 6763 Acked Read 5 Done Sent Done false
25764 6764 Acked Write 3 Done Sent Done false
25765 6765 Acked Read 4 Done Sent Done false
25766 6766 Acked Read 5 Done Sent Done false
25767 6767 Acked Read 3 Done Sent Done false
25768 6768 Acked Read 2 Done Sent Done false
25769 6769 Acked Write 6 Done Sent Done false
25770 6770 Acked Read 10 Done Sent Done false
25771 6771 Acked Read 2 Done Sent Done false
25772 6772 Acked Read 7 Done Sent Done false
25773 6773 Acked Read 1 Done Sent Done false
25774 6774 Acked Flush 0 Done Sent Done false
25775 6775 Acked Write 8 Done Sent Done false
25776 6776 Acked Write 5 Done Sent Done false
25777 6777 Acked Read 5 Done Sent Done false
25778 6778 Acked Read 5 Done Sent Done false
25779 6779 Acked Read 10 Done Sent Done false
25780 6780 Acked Read 7 Done Sent Done false
25781 6781 Acked Read 3 Done Sent Done false
25782 6782 Acked Read 3 Done Sent Done false
25783 6783 Acked Write 3 Done Sent Done false
25784 6784 Acked Write 4 Done Sent Done false
25785 6785 Acked Read 10 Done Sent Done false
25786 6786 Acked Write 4 Done Sent Done false
25787 6787 Acked Read 1 Done Sent Done false
25788 6788 Acked Read 7 Done Sent Done false
25789 6789 Acked Read 6 Done Sent Done false
25790 6790 Acked Flush 0 Done Sent Done false
25791 6791 Acked Write 1 Done Sent Done false
25792 6792 Acked Read 4 Done Sent Done false
25793 6793 Acked Write 10 Done Sent Done false
25794 6794 Acked Write 7 Done Sent Done false
25795 6795 Acked Read 6 Done Sent Done false
25796 6796 Acked Read 2 Done Sent Done false
25797 6797 Acked Read 6 Done Sent Done false
25798 6798 Acked Read 10 Done Sent Done false
25799 6799 Acked Read 8 Done Sent Done false
25800 6800 Acked Read 4 Done Sent Done false
25801 6801 Acked Write 2 Done Sent Done false
25802 6802 Acked Read 5 Done Sent Done false
25803 6803 Acked Flush 0 Done Sent Done false
25804 6804 Acked Read 4 Done Sent Done false
25805 6805 Acked Write 3 Done Sent Done false
25806 6806 Acked Write 5 Done Sent Done false
25807 6807 Acked Write 10 Done Sent Done false
25808 6808 Acked Read 7 Done Sent Done false
25809 6809 Acked Write 9 Done Sent Done false
25810 6810 Acked Write 2 Done Sent Done false
25811 6811 Acked Read 2 Done Sent Done false
25812 6812 Acked Read 4 Done Sent Done false
25813 6813 Acked Read 2 Done Sent Done false
25814 6814 Acked Write 2 Done Sent Done false
25815 6815 Acked Read 10 Done Sent Done false
25816 6816 Acked Flush 0 Done Sent Done false
25817 6817 Acked Flush 0 Done Sent Done false
25818 6818 Acked Write 9 Done Sent Done false
25819 6819 Acked Flush 0 Done Sent Done false
25820 6820 Acked Read 8 Done Sent Done false
25821 6821 Acked Read 5 Done Sent Done false
25822 6822 Acked Read 5 Done Sent Done false
25823 6823 Acked Write 4 Done Sent Done false
25824 6824 Acked Write 2 Done Sent Done false
25825 6825 Acked Flush 0 Done Sent Done false
25826 6826 Acked Write 9 Done Sent Done false
25827 6827 Acked Read 6 Done Sent Done false
25828 6828 Acked Write 5 Done Sent Done false
25829 6829 Acked Write 3 Done Sent Done false
25830 6830 Acked Flush 0 Done Sent Done false
25831 6831 Acked Read 7 Done Sent Done false
25832 6832 Acked Read 1 Done Sent Done false
25833 6833 Acked Write 1 Done Sent Done false
25834 6834 Acked Read 5 Done Sent Done false
25835 6835 Acked Write 3 Done Sent Done false
25836 6836 Acked Read 1 Done Sent Done false
25837 6837 Acked Write 10 Done Sent Done false
25838 6838 Acked Read 9 Done Sent Done false
25839 6839 Acked Write 2 Done Sent Done false
25840 6840 Acked Read 5 Done Sent Done false
25841 6841 Acked Flush 0 Done Sent Done false
25842 6842 Acked Read 1 Done Sent Done false
25843 6843 Acked Write 2 Done Sent Done false
25844 6844 Acked Write 6 Done Sent Done false
25845 6845 Acked Read 3 Done Sent Done false
25846 6846 Acked Read 5 Done Sent Done false
25847 6847 Acked Read 5 Done Sent Done false
25848 6848 Acked Read 2 Done Sent Done false
25849 6849 Acked Read 5 Done Sent Done false
25850 6850 Acked Read 6 Done Sent Done false
25851 6851 Acked Read 1 Done Sent Done false
25852 6852 Acked Read 1 Done Sent Done false
25853 6853 Acked Read 5 Done Sent Done false
25854 6854 Acked Flush 0 Done Sent Done false
25855 6855 Acked Read 9 Done Sent Done false
25856 6856 Acked Read 2 Done Sent Done false
25857 6857 Acked Read 6 Done Sent Done false
25858 6858 Acked Write 8 Done Sent Done false
25859 6859 Acked Write 4 Done Sent Done false
25860 6860 Acked Write 2 Done Sent Done false
25861 6861 Acked Read 10 Done Sent Done false
25862 6862 Acked Write 2 Done Sent Done false
25863 6863 Acked Write 7 Done Sent Done false
25864 6864 Acked Write 7 Done Sent Done false
25865 6865 Acked Read 7 Done Sent Done false
25866 6866 Acked Read 5 Done Sent Done false
25867 6867 Acked Read 6 Done Sent Done false
25868 6868 Acked Flush 0 Done Sent Done false
25869 6869 Acked Read 4 Done Sent Done false
25870 6870 Acked Read 7 Done Sent Done false
25871 6871 Acked Write 7 Done Sent Done false
25872 6872 Acked Read 8 Done Sent Done false
25873 6873 Acked Write 8 Done Sent Done false
25874 6874 Acked Write 2 Done Sent Done false
25875 6875 Acked Read 5 Done Sent Done false
25876 6876 Acked Flush 0 Done Sent Done false
25877 6877 Acked Read 5 Done Sent Done false
25878 6878 Acked Write 10 Done Sent Done false
25879 6879 Acked Write 3 Done Sent Done false
25880 6880 Acked Write 1 Done Sent Done false
25881 6881 Acked Read 10 Done Sent Done false
25882 6882 Acked Read 6 Done Sent Done false
25883 6883 Acked Flush 0 Done Sent Done false
25884 6884 Acked Flush 0 Done Sent Done false
25885 6885 Acked Read 2 Done Sent Done false
25886 6886 Acked Read 2 Done Sent Done false
25887 6887 Acked Read 5 Done Sent Done false
25888 6888 Acked Write 1 Done Sent Done false
25889 6889 Acked Read 1 Done Sent Done false
25890 6890 Acked Read 5 Done Sent Done false
25891 6891 Acked Read 3 Done Sent Done false
25892 6892 Acked Write 6 Done Sent Done false
25893 6893 Acked Read 2 Done Sent Done false
25894 6894 Acked Write 2 Done Sent Done false
25895 6895 Acked Read 5 Done Sent Done false
25896 6896 Acked Write 8 Done Sent Done false
25897 6897 Acked Write 9 Done Sent Done false
25898 6898 Acked Write 10 Done Sent Done false
25899 6899 Acked Read 4 Done Sent Done false
25900 6900 Acked Flush 0 Done Sent Done false
25901 6901 Acked Write 5 Done Sent Done false
25902 6902 Acked Write 6 Done Sent Done false
25903 6903 Acked Write 6 Done Sent Done false
25904 6904 Acked Write 8 Done Sent Done false
25905 6905 Acked Write 10 Done Sent Done false
25906 6906 Acked Read 2 Done Sent Done false
25907 6907 Acked Write 4 Done Sent Done false
25908 6908 Acked Write 4 Done Sent Done false
25909 6909 Acked Read 1 Done Sent Done false
25910 6910 Acked Write 2 Done Sent Done false
25911 6911 Acked Write 1 Done Sent Done false
25912 6912 Acked Write 6 Done Sent Done false
25913 6913 Acked Read 8 Done Sent Done false
25914 6914 Acked Read 9 Done Sent Done false
25915 6915 Acked Read 1 Done Sent Done false
25916 6916 Acked Write 8 Done Sent Done false
25917 6917 Acked Write 1 Done Sent Done false
25918 6918 Acked Read 5 Done Sent Done false
25919 6919 Acked Read 7 Done Sent Done false
25920 6920 Acked Read 4 Done Sent Done false
25921 6921 Acked Read 1 Done Sent Done false
25922 6922 Acked Read 1 Done Sent Done false
25923 6923 Acked Write 2 Done Sent Done false
25924 6924 Acked Read 7 Done Sent Done false
25925 6925 Acked Read 8 Done Sent Done false
25926 6926 Acked Read 6 Done Sent Done false
25927 6927 Acked Read 5 Done Sent Done false
25928 6928 Acked Read 2 Done Sent Done false
25929 6929 Acked Read 4 Done Sent Done false
25930 6930 Acked Read 4 Done Sent Done false
25931 6931 Acked Flush 0 Done Sent Done false
25932 6932 Acked Write 10 Done Sent Done false
25933 6933 Acked Read 3 Done Sent Done false
25934 6934 Acked Read 6 Done Sent Done false
25935 6935 Acked Read 6 Done Sent Done false
25936 6936 Acked Write 8 Done Sent Done false
25937 6937 Acked Flush 0 Done Sent Done false
25938 6938 Acked Write 1 Done Sent Done false
25939 6939 Acked Write 10 Done Sent Done false
25940 6940 Acked Read 3 Done Sent Done false
25941 6941 Acked Read 6 Done Sent Done false
25942 6942 Acked Flush 0 Done Sent Done false
25943 6943 Acked Write 10 Done Sent Done false
25944 6944 Acked Write 6 Done Sent Done false
25945 6945 Acked Write 10 Done Sent Done false
25946 6946 Acked Read 6 Done Sent Done false
25947 6947 Acked Write 10 Done Sent Done false
25948 6948 Acked Write 6 Done Sent Done false
25949 6949 Acked Read 7 Done Sent Done false
25950 6950 Acked Read 7 Done Sent Done false
25951 6951 Acked Write 4 Done Sent Done false
25952 6952 Acked Write 8 Done Sent Done false
25953 6953 Acked Read 4 Done Sent Done false
25954 6954 Acked Write 3 Done Sent Done false
25955 6955 Acked Read 5 Done Sent Done false
25956 6956 Acked Read 10 Done Sent Done false
25957 6957 Acked Read 1 Done Sent Done false
25958 6958 Acked Write 1 Done Sent Done false
25959 6959 Acked Read 3 Done Sent Done false
25960 6960 Acked Read 3 Done Sent Done false
25961 6961 Acked Flush 0 Done Sent Done false
25962 6962 Acked Write 8 Done Sent Done false
25963 6963 Acked Write 6 Done Sent Done false
25964 6964 Acked Read 1 Done Sent Done false
25965 6965 Acked Read 6 Done Sent Done false
25966 6966 Acked Read 10 Done Sent Done false
25967 6967 Acked Write 5 Done Sent Done false
25968 6968 Acked Write 4 Done Sent Done false
25969 6969 Acked Read 3 Done Sent Done false
25970 6970 Acked Read 2 Done Sent Done false
25971 6971 Acked Read 7 Done Sent Done false
25972 6972 Acked Read 2 Done Sent Done false
25973 6973 Acked Read 4 Done Sent Done false
25974 6974 Acked Read 1 Done Sent Done false
25975 6975 Acked Read 10 Done Sent Done false
25976 6976 Acked Read 5 Done Sent Done false
25977 6977 Acked Flush 0 Done Sent Done false
25978 6978 Acked Read 4 Done Sent Done false
25979 6979 Acked Read 6 Done Sent Done false
25980 6980 Acked Read 9 Done Sent Done false
25981 6981 Acked Read 6 Done Sent Done false
25982 6982 Acked Read 6 Done Sent Done false
25983 6983 Acked Write 3 Done Sent Done false
25984 6984 Acked Write 9 Done Sent Done false
25985 6985 Acked Read 6 Done Sent Done false
25986 6986 Acked Write 8 Done Sent Done false
25987 6987 Acked Read 10 Done Sent Done false
25988 6988 Acked Read 9 Done Sent Done false
25989 6989 Acked Write 8 Done Sent Done false
25990 6990 Acked Read 8 Done Sent Done false
25991 6991 Acked Read 6 Done Sent Done false
25992 6992 Acked Write 7 Done Sent Done false
25993 6993 Acked Read 7 Done Sent Done false
25994 6994 Acked Read 3 Done Sent Done false
25995 6995 Acked Read 6 Done Sent Done false
25996 6996 Acked Read 9 Done Sent Done false
25997 6997 Acked Write 6 Done Sent Done false
25998 6998 Acked Write 4 Done Sent Done false
25999 6999 Acked Write 4 Done Sent Done false
26000 7000 Acked Flush 0 Done Sent Done false
26001 7001 Acked Read 4 Done Sent Done false
26002 7002 Acked Write 8 Done Sent Done false
26003 7003 Acked Read 7 Done Sent Done false
26004 7004 Acked Write 6 Done Sent Done false
26005 7005 Acked Flush 0 Done Sent Done false
26006 7006 Acked Write 5 Done Sent Done false
26007 7007 Acked Read 9 Done Sent Done false
26008 7008 Acked Write 3 Done Sent Done false
26009 7009 Acked Write 6 Done Sent Done false
26010 7010 Acked Read 9 Done Sent Done false
26011 7011 Acked Read 4 Done Sent Done false
26012 7012 Acked Write 2 Done Sent Done false
26013 7013 Acked Read 8 Done Sent Done false
26014 7014 Acked Write 6 Done Sent Done false
26015 7015 Acked Read 9 Done Sent Done false
26016 7016 Acked Read 9 Done Sent Done false
26017 7017 Acked Read 1 Done Sent Done false
26018 7018 Acked Read 4 Done Sent Done false
26019 7019 Acked Flush 0 Done Sent Done false
26020 7020 Acked Read 7 Done Sent Done false
26021 7021 Acked Read 8 Done Sent Done false
26022 7022 Acked Read 6 Done Sent Done false
26023 7023 Acked Write 7 Done Sent Done false
26024 7024 Acked Write 9 Done Sent Done false
26025 7025 Acked Write 6 Done Sent Done false
26026 7026 Acked Read 8 Done Sent Done false
26027 7027 Acked Flush 0 Done Sent Done false
26028 7028 Acked Read 5 Done Sent Done false
26029 7029 Acked Write 6 Done Sent Done false
26030 7030 Acked Flush 0 Done Sent Done false
26031 7031 Acked Write 3 Done Sent Done false
26032 7032 Acked Read 2 Done Sent Done false
26033 7033 Acked Read 9 Done Sent Done false
26034 7034 Acked Read 3 Done Sent Done false
26035 7035 Acked Read 6 Done Sent Done false
26036 7036 Acked Read 1 Done Sent Done false
26037 7037 Acked Flush 0 Done Sent Done false
26038 STATES DS:0 DS:1 DS:2 TOTAL
26039 Sent 0 302 0 302
26040 Done 302 0 302 604
26041 Skipped 0 0 0 0
26042 Error 0 0 0 0
26043 Last Flush: 7037 6735 7037
26044 Downstairs last five completed: 6735 6734 6733 6732 6731
26045 Upstairs last five completed: 7037 7036 7035 7034 7033
26046 CLIENT: Up:0 ds:302 act:2
260472024-11-15T02:02:54.581ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
260482024-11-15T02:02:54.582ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:33861) = io task client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
260492024-11-15T02:02:54.583ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
260502024-11-15T02:02:54.583ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
260512024-11-15T02:02:54.583ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
260522024-11-15T02:02:54.583ZINFOcrucible: send last flush ID to this DS: 6735 = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
260532024-11-15T02:02:54.584ZINFOcrucible: Replied this last flush ID: 6735 = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
260542024-11-15T02:02:54.584ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
260552024-11-15T02:02:54.584ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
260562024-11-15T02:02:54.584ZINFOcrucible: [1] client re-new 302 jobs since flush 6735 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
260572024-11-15T02:02:54.584ZINFOcrucible: [1] Replayed 302 jobs since flush: 6735 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
26058 ----------------------------------------------------------------
26059 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
26060 Downstairs last five completed: 7037 7036 7035 7034 7033
26061 Upstairs last five completed: 7037 7036 7035 7034 7033
26062 CLIENT: Up:0 ds:0 act:3
26063 Replay: All jobs finished
26064 Main task has work to do, go find it
26065 got dsc Stop(2)
26066 stop 2
26067 [8830] Got stop action so:false kr:false
26068 Nov 15 02:02:56.829 INFO request completed, latency_us: 364, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 9ace13c8-d242-48a8-967d-59833b56677a, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
260692024-11-15T02:02:56.830ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
260702024-11-15T02:02:56.830ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
260712024-11-15T02:02:56.830ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
260722024-11-15T02:02:56.830ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
260732024-11-15T02:02:56.830ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
26074 Nov 15 02:02:56.830 INFO request completed, latency_us: 171, response_code: 200, uri: /state/cid/2, method: GET, req_id: 6f91c966-587e-4d32-92ef-a9fd14a0fd0f, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
260752024-11-15T02:02:56.830ZINFOcrucible: client task is exiting = io task client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
26076 [8830] Exited with: ExitStatus(unix_wait_status(9))
26077 [8830][2] reports Exit
26078 Nov 15 02:03:00.832 INFO request completed, latency_us: 202, response_code: 200, uri: /state/cid/2, method: GET, req_id: 67066535-1ad6-4bb5-b64f-411f407a41d1, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
26079 001/300 Write block 3232 len 12288 data: 5 3 2
26080 002/300 Read block 679 len 36864
26081 003/300 Write block 1326 len 8192 data: 4 4
26082 004/300 Flush
26083 005/300 Write block 2075 len 28672 data: 3 4 5 6 6 5 5
26084 006/300 Write block 291 len 16384 data: 4 5 5 5
26085 007/300 Flush
26086 008/300 Write block 4572 len 28672 data: 3 2 2 2 2 3 3
26087 009/300 Read block 4445 len 4096
26088 010/300 Read block 3838 len 20480
26089 011/300 Read block 1929 len 28672
26090 012/300 Read block 3630 len 28672
26091 013/300 Flush
26092 014/300 Read block 192 len 4096
26093 015/300 Read block 4220 len 24576
26094 016/300 Write block 4860 len 24576 data: 5 6 7 8 7 6
26095 017/300 Read block 4899 len 20480
26096 018/300 Read block 2071 len 40960
26097 019/300 Write block 2477 len 4096 data: 8
26098 020/300 Write block 532 len 16384 data: 5 6 7 7
26099 021/300 Write block 462 len 12288 data: 8 7 6
26100 022/300 Read block 846 len 16384
26101 023/300 Read block 3615 len 24576
26102 024/300 Write block 1737 len 12288 data: 4 4 4
26103 025/300 Write block 4949 len 16384 data: 4 3 3 2
26104 026/300 Read block 3097 len 24576
26105 027/300 Write block 2803 len 8192 data: 3 3
26106 028/300 Read block 2888 len 24576
26107 029/300 Flush
26108 030/300 Write block 3690 len 8192 data: 8 8
26109 031/300 Write block 1995 len 8192 data: 3 3
26110 032/300 Write block 4814 len 12288 data: 4 4 4
26111 033/300 Read block 4401 len 24576
26112 034/300 Write block 4798 len 8192 data: 6 6
26113 035/300 Read block 2783 len 24576
26114 036/300 Read block 160 len 8192
26115 037/300 Flush
26116 038/300 Write block 47 len 36864 data: 5 5 5 5 4 3 2 6 5
26117 039/300 Read block 1118 len 12288
26118 040/300 Flush
26119 041/300 Read block 2540 len 12288
26120 042/300 Write block 1463 len 12288 data: 5 6 5
26121 043/300 Read block 1353 len 28672
26122 044/300 Write block 1449 len 16384 data: 3 4 5 4
26123 045/300 Write block 1732 len 4096 data: 4
26124 046/300 Read block 3947 len 16384
26125 047/300 Flush
26126 048/300 Read block 3080 len 24576
26127 049/300 Flush
26128 050/300 Write block 323 len 16384 data: 4 4 6 7
26129 051/300 Write block 1985 len 36864 data: 5 6 5 4 3 4 4 3 3
26130 052/300 Read block 688 len 4096
26131 053/300 Write block 4259 len 32768 data: 7 6 5 5 5 6 7 8
26132 054/300 Write block 422 len 28672 data: 5 4 2 3 2 3 3
26133 055/300 Read block 1366 len 12288
26134 056/300 Write block 2243 len 40960 data: 4 5 5 4 4 4 4 3 3 4
26135 057/300 Write block 4487 len 36864 data: 3 4 4 5 5 6 6 6 6
26136 058/300 Read block 1364 len 24576
26137 059/300 Flush
26138 060/300 Read block 2584 len 32768
26139 061/300 Read block 3882 len 36864
26140 062/300 Read block 61 len 4096
26141 063/300 Write block 1059 len 40960 data: 5 3 3 2 3 3 4 5 6 7
26142 064/300 Read block 2573 len 28672
26143 065/300 Read block 2869 len 12288
26144 066/300 Read block 728 len 24576
26145 067/300 Read block 2980 len 28672
26146 068/300 Write block 3250 len 24576 data: 7 8 8 9 8 7
26147 069/300 Read block 4419 len 16384
26148 070/300 Write block 2262 len 32768 data: 5 5 6 6 6 5 3 3
26149 071/300 Write block 4692 len 20480 data: 4 4 5 4 5
26150 072/300 Read block 676 len 24576
26151 073/300 Write block 4529 len 40960 data: 8 8 6 6 6 4 4 5 4 4
26152 074/300 Read block 1072 len 20480
26153 075/300 Write block 1762 len 20480 data: 4 5 5 3 5
26154 076/300 Read block 3 len 8192
26155 077/300 Read block 691 len 28672
26156 078/300 Read block 1197 len 16384
26157 079/300 Read block 450 len 20480
26158 080/300 Read block 2186 len 24576
26159 081/300 Write block 4891 len 8192 data: 7 8
26160 082/300 Read block 3897 len 40960
26161 083/300 Read block 1442 len 36864
26162 084/300 Read block 491 len 12288
26163 085/300 Write block 1583 len 20480 data: 3 3 3 3 3
26164 086/300 Read block 2764 len 40960
26165 087/300 Write block 3355 len 8192 data: 8 5
26166 088/300 Write block 194 len 36864 data: 4 4 4 4 4 4 5 5 4
26167 089/300 Read block 960 len 28672
26168 090/300 Write block 3437 len 12288 data: 4 4 4
26169 091/300 Write block 4136 len 32768 data: 4 3 4 4 7 6 5 5
26170 092/300 Read block 389 len 4096
26171 093/300 Read block 403 len 16384
26172 094/300 Read block 2733 len 20480
26173 095/300 Read block 3322 len 40960
26174 096/300 Read block 1477 len 16384
26175 097/300 Read block 4292 len 8192
26176 098/300 Read block 3752 len 12288
26177 099/300 Read block 1668 len 4096
26178 100/300 Write block 4378 len 24576 data: 2 2 2 2 2 3
26179 101/300 Read block 31 len 28672
26180 102/300 Write block 628 len 36864 data: 4 3 4 5 5 3 3 3 3
26181 103/300 Read block 1764 len 8192
26182 104/300 Read block 2739 len 12288
26183 105/300 Write block 514 len 8192 data: 4 3
26184 106/300 Read block 843 len 28672
26185 107/300 Write block 539 len 40960 data: 4 4 4 3 4 4 4 4 5 5
26186 108/300 Write block 3839 len 24576 data: 7 7 6 5 4 3
26187 109/300 Read block 3521 len 28672
26188 110/300 Write block 4366 len 32768 data: 4 5 4 3 4 4 5 5
26189 111/300 Read block 291 len 16384
26190 112/300 Read block 4781 len 16384
26191 113/300 Read block 797 len 4096
26192 114/300 Write block 991 len 16384 data: 8 8 8 8
26193 115/300 Write block 861 len 8192 data: 3 5
26194 116/300 Read block 1770 len 36864
26195 117/300 Flush
26196 118/300 Write block 3149 len 4096 data: 5
26197 119/300 Read block 952 len 28672
26198 120/300 Read block 4074 len 36864
26199 121/300 Read block 3122 len 4096
26200 122/300 Read block 4336 len 12288
26201 123/300 Read block 2976 len 32768
26202 124/300 Write block 2959 len 28672 data: 5 4 4 4 5 4 3
26203 125/300 Write block 4039 len 20480 data: 10 9 8 7 8
26204 126/300 Write block 4008 len 32768 data: 4 4 4 4 4 3 4 5
26205 127/300 Read block 3844 len 8192
26206 128/300 Read block 795 len 40960
26207 129/300 Write block 2171 len 8192 data: 6 7
26208 130/300 Write block 1354 len 20480 data: 4 6 6 6 6
26209 131/300 Read block 1277 len 8192
26210 132/300 Read block 4033 len 28672
26211 133/300 Write block 430 len 8192 data: 5 5
26212 134/300 Read block 3500 len 36864
26213 135/300 Write block 1095 len 12288 data: 4 5 5
26214 136/300 Read block 711 len 12288
26215 137/300 Write block 1225 len 20480 data: 3 4 4 4 4
26216 138/300 Read block 1404 len 20480
26217 139/300 Write block 4534 len 40960 data: 5 5 6 5 5 4 4 4 4 5
26218 140/300 Flush
26219 141/300 Write block 1195 len 16384 data: 6 9 9 9
26220 142/300 Write block 4679 len 28672 data: 4 4 3 4 6 8 7
26221 143/300 Write block 4987 len 16384 data: 2 2 3 2
26222 144/300 Write block 3162 len 8192 data: 5 5
26223 145/300 Read block 636 len 4096
26224 146/300 Write block 491 len 24576 data: 5 5 5 5 5 4
26225 147/300 Read block 1501 len 16384
26226 148/300 Read block 1706 len 40960
26227 149/300 Read block 1812 len 36864
26228 150/300 Flush
26229 151/300 Read block 814 len 24576
26230 152/300 Write block 395 len 32768 data: 4 3 3 3 4 4 3 3
26231 153/300 Write block 1657 len 32768 data: 5 6 6 7 8 8 10 10
26232 154/300 Read block 1029 len 4096
26233 155/300 Read block 1861 len 4096
26234 156/300 Write block 1358 len 32768 data: 7 5 4 4 5 5 4 6
26235 157/300 Read block 4582 len 16384
26236 158/300 Write block 1973 len 20480 data: 4 4 4 3 2
26237 159/300 Read block 2330 len 24576
26238 160/300 Read block 2884 len 8192
26239 161/300 Read block 3258 len 40960
26240 162/300 Read block 1286 len 4096
26241 163/300 Read block 4290 len 20480
26242 164/300 Write block 4837 len 8192 data: 6 5
26243 165/300 Flush
26244 166/300 Read block 3767 len 8192
26245 167/300 Flush
26246 168/300 Read block 2599 len 36864
26247 169/300 Read block 2426 len 8192
26248 170/300 Read block 920 len 40960
26249 171/300 Write block 4719 len 8192 data: 5 4
26250 172/300 Read block 3952 len 32768
26251 173/300 Write block 3464 len 8192 data: 3 4
26252 174/300 Read block 1902 len 12288
26253 175/300 Flush
26254 176/300 Write block 2783 len 8192 data: 5 4
26255 177/300 Write block 3802 len 28672 data: 4 5 5 5 6 7 6
26256 178/300 Read block 1016 len 40960
26257 179/300 Read block 1949 len 40960
26258 180/300 Write block 1429 len 28672 data: 3 3 2 2 2 3 3
26259 181/300 Flush
26260 182/300 Write block 3250 len 24576 data: 8 9 9 10 9 8
26261 183/300 Read block 4505 len 16384
26262 184/300 Write block 4318 len 16384 data: 7 8 7 8
26263 185/300 Read block 1165 len 16384
26264 186/300 Read block 776 len 20480
26265 187/300 Write block 4754 len 40960 data: 4 3 4 3 2 2 2 2 2 2
26266 188/300 Read block 604 len 8192
26267 189/300 Read block 1348 len 4096
26268 190/300 Read block 2614 len 40960
26269 191/300 Flush
26270 192/300 Write block 4567 len 40960 data: 4 4 5 4 3 4 3 3 3 3
26271 193/300 Write block 161 len 16384 data: 7 6 6 6
26272 194/300 Read block 1800 len 40960
26273 195/300 Write block 4774 len 40960 data: 5 5 7 6 8 6 6 6 6 5
26274 196/300 Read block 2255 len 4096
26275 197/300 Write block 3365 len 12288 data: 4 3 3
26276 198/300 Write block 4148 len 24576 data: 3 5 5 5 4 4
26277 199/300 Read block 3924 len 40960
26278 200/300 Write block 3338 len 16384 data: 5 6 5 3
26279 201/300 Read block 3658 len 24576
26280 202/300 Read block 4739 len 4096
26281 203/300 Write block 3583 len 20480 data: 3 3 3 2 2
26282 204/300 Read block 2043 len 40960
26283 205/300 Read block 1259 len 24576
26284 206/300 Write block 4755 len 40960 data: 4 5 4 3 3 3 3 3 3 2
26285 207/300 Read block 4683 len 8192
26286 208/300 Read block 4152 len 4096
26287 209/300 Read block 522 len 16384
26288 210/300 Read block 644 len 20480
26289 211/300 Read block 1620 len 8192
26290 212/300 Write block 3203 len 40960 data: 3 2 2 2 2 2 2 3 3 2
26291 213/300 Flush
26292 214/300 Read block 4852 len 20480
26293 215/300 Write block 1849 len 28672 data: 4 3 2 3 5 5 5
26294 216/300 Write block 98 len 8192 data: 4 4
26295 217/300 Read block 1251 len 32768
26296 218/300 Write block 2794 len 20480 data: 2 2 2 2 2
26297 219/300 Read block 3425 len 32768
26298 220/300 Write block 4125 len 28672 data: 6 6 5 7 7 7 7
26299 221/300 Read block 2768 len 36864
26300 222/300 Read block 2447 len 40960
26301 223/300 Write block 3049 len 4096 data: 5
26302 224/300 Flush
26303 225/300 Read block 1904 len 20480
26304 226/300 Write block 4335 len 4096 data: 6
26305 227/300 Write block 970 len 4096 data: 4
26306 228/300 Flush
26307 229/300 Write block 443 len 24576 data: 4 4 4 3 3 4
26308 230/300 Write block 4285 len 20480 data: 5 5 6 6 5
26309 231/300 Read block 3215 len 28672
26310 232/300 Read block 4208 len 32768
26311 233/300 Read block 1408 len 16384
26312 234/300 Write block 570 len 4096 data: 5
26313 235/300 Read block 415 len 12288
26314 236/300 Read block 4217 len 36864
26315 237/300 Read block 1476 len 8192
26316 238/300 Read block 1221 len 8192
26317 239/300 Write block 1517 len 40960 data: 4 3 3 3 3 2 2 4 4 4
26318 240/300 Write block 1234 len 8192 data: 3 4
26319 241/300 Read block 443 len 4096
26320 242/300 Write block 4000 len 4096 data: 3
26321 243/300 Read block 353 len 28672
26322 244/300 Read block 3772 len 20480
26323 245/300 Write block 1174 len 12288 data: 5 6 6
26324 246/300 Read block 4920 len 16384
26325 247/300 Flush
26326 248/300 Write block 4765 len 16384 data: 2 2 3 3
26327 249/300 Write block 1678 len 28672 data: 5 6 6 7 5 5 7
26328 250/300 Read block 3941 len 20480
26329 251/300 Read block 4731 len 8192
26330 252/300 Read block 4069 len 36864
26331 253/300 Read block 2635 len 24576
26332 254/300 Read block 4024 len 20480
26333 255/300 Write block 1154 len 24576 data: 3 3 3 3 3 3
26334 256/300 Read block 4463 len 8192
26335 257/300 Write block 4311 len 36864 data: 6 8 8 7 9 10 10 8 9
26336 258/300 Read block 2244 len 36864
26337 259/300 Read block 830 len 12288
26338 260/300 Flush
26339 261/300 Write block 4655 len 16384 data: 6 6 6 7
26340 262/300 Write block 360 len 32768 data: 5 4 5 6 4 4 4 5
26341 263/300 Flush
26342 264/300 Write block 886 len 28672 data: 2 2 3 3 4 4 4
26343 265/300 Read block 4271 len 20480
26344 266/300 Write block 232 len 40960 data: 9 6 6 6 7 5 4 4 4 3
26345 267/300 Flush
26346 268/300 Read block 4171 len 12288
26347 269/300 Read block 2717 len 40960
26348 270/300 Read block 1922 len 4096
26349 271/300 Flush
26350 272/300 Write block 4063 len 20480 data: 5 5 6 5 5
26351 273/300 Flush
26352 274/300 Write block 3179 len 20480 data: 4 4 4 4 5
26353 275/300 Read block 2908 len 20480
26354 276/300 Read block 3139 len 24576
26355 277/300 Flush
26356 278/300 Read block 1613 len 20480
26357 279/300 Read block 3132 len 4096
26358 280/300 Write block 2308 len 28672 data: 8 8 6 5 5 4 5
26359 281/300 Write block 4012 len 36864 data: 5 4 5 6 4 4 5 4 5
26360 282/300 Read block 3734 len 20480
26361 283/300 Write block 4434 len 36864 data: 3 5 5 4 4 4 4 5 6
26362 284/300 Write block 1464 len 8192 data: 7 6
26363 285/300 Write block 808 len 16384 data: 3 4 4 6
26364 286/300 Read block 4030 len 12288
26365 287/300 Write block 3027 len 4096 data: 3
26366 288/300 Write block 984 len 24576 data: 2 3 4 4 4 5
26367 289/300 Read block 3440 len 12288
26368 290/300 Read block 2742 len 40960
26369 291/300 Read block 3552 len 36864
26370 292/300 Write block 747 len 16384 data: 3 3 3 2
26371 293/300 Write block 969 len 16384 data: 4 5 3 3
26372 294/300 Flush
26373 295/300 Read block 3215 len 4096
26374 296/300 Write block 3815 len 8192 data: 3 2
26375 297/300 Write block 843 len 24576 data: 6 6 5 4 4 4
26376 298/300 Flush
26377 299/300 Write block 2816 len 32768 data: 6 6 7 7 5 5 6 6
26378 300/300 Read block 854 len 24576
26379 301/300 Read block 3982 len 24576
26380 Main task has work to do, go find it
26381 got dsc Start(2)
26382 start 2
26383 Nov 15 02:03:01.067 INFO request completed, latency_us: 213, response_code: 204, uri: /start/cid/2, method: POST, req_id: ddca2a7e-6421-496c-a030-fc95e7b0e2a6, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
26384 Replay: started 2, returned:Ok(())
26385 ----------------------------------------------------------------
26386 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
26387 States: Active Active Offline
26388 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
26389 7038 Acked Write 3 Done Done Sent false
26390 7039 Acked Read 9 Done Done Sent false
26391 7040 Acked Write 2 Done Done Sent false
26392 7041 Acked Flush 0 Done Done Sent false
26393 7042 Acked Write 7 Done Done Sent false
26394 7043 Acked Write 4 Done Done Sent false
26395 7044 Acked Flush 0 Done Done Sent false
26396 7045 Acked Write 7 Done Done Sent false
26397 7046 Acked Read 1 Done Done Sent false
26398 7047 Acked Read 5 Done Done Sent false
26399 7048 Acked Read 7 Done Done Sent false
26400 7049 Acked Read 7 Done Done Sent false
26401 7050 Acked Flush 0 Done Done Sent false
26402 7051 Acked Read 1 Done Done Sent false
26403 7052 Acked Read 6 Done Done Sent false
26404 7053 Acked Write 6 Done Done Sent false
26405 7054 Acked Read 5 Done Done Sent false
26406 7055 Acked Read 10 Done Done Sent false
26407 7056 Acked Write 1 Done Done Sent false
26408 7057 Acked Write 4 Done Done Sent false
26409 7058 Acked Write 3 Done Done Sent false
26410 7059 Acked Read 4 Done Done Sent false
26411 7060 Acked Read 6 Done Done Sent false
26412 7061 Acked Write 3 Done Done Sent false
26413 7062 Acked Write 4 Done Done Sent false
26414 7063 Acked Read 6 Done Done Sent false
26415 7064 Acked Write 2 Done Done Sent false
26416 7065 Acked Read 6 Done Done Sent false
26417 7066 Acked Flush 0 Done Done Sent false
26418 7067 Acked Write 2 Done Done Sent false
26419 7068 Acked Write 2 Done Done Sent false
26420 7069 Acked Write 3 Done Done Sent false
26421 7070 Acked Read 6 Done Done Sent false
26422 7071 Acked Write 2 Done Done Sent false
26423 7072 Acked Read 6 Done Done Sent false
26424 7073 Acked Read 2 Done Done Sent false
26425 7074 Acked Flush 0 Done Done Sent false
26426 7075 Acked Write 9 Done Done Sent false
26427 7076 Acked Read 3 Done Done Sent false
26428 7077 Acked Flush 0 Done Done Sent false
26429 7078 Acked Read 3 Done Done Sent false
26430 7079 Acked Write 3 Done Done Sent false
26431 7080 Acked Read 7 Done Done Sent false
26432 7081 Acked Write 4 Done Done Sent false
26433 7082 Acked Write 1 Done Done Sent false
26434 7083 Acked Read 4 Done Done Sent false
26435 7084 Acked Flush 0 Done Done Sent false
26436 7085 Acked Read 6 Done Done Sent false
26437 7086 Acked Flush 0 Done Done Sent false
26438 7087 Acked Write 4 Done Done Sent false
26439 7088 Acked Write 9 Done Done Sent false
26440 7089 Acked Read 1 Done Done Sent false
26441 7090 Acked Write 8 Done Done Sent false
26442 7091 Acked Write 7 Done Done Sent false
26443 7092 Acked Read 3 Done Done Sent false
26444 7093 Acked Write 10 Done Done Sent false
26445 7094 Acked Write 9 Done Done Sent false
26446 7095 Acked Read 6 Done Done Sent false
26447 7096 Acked Flush 0 Done Done Sent false
26448 7097 Acked Read 8 Done Done Sent false
26449 7098 Acked Read 9 Done Done Sent false
26450 7099 Acked Read 1 Done Done Sent false
26451 7100 Acked Write 10 Done Done Sent false
26452 7101 Acked Read 7 Done Done Sent false
26453 7102 Acked Read 3 Done Done Sent false
26454 7103 Acked Read 6 Done Done Sent false
26455 7104 Acked Read 7 Done Done Sent false
26456 7105 Acked Write 6 Done Done Sent false
26457 7106 Acked Read 4 Done Done Sent false
26458 7107 Acked Write 8 Done Done Sent false
26459 7108 Acked Write 5 Done Done Sent false
26460 7109 Acked Read 6 Done Done Sent false
26461 7110 Acked Write 10 Done Done Sent false
26462 7111 Acked Read 5 Done Done Sent false
26463 7112 Acked Write 5 Done Done Sent false
26464 7113 Acked Read 2 Done Done Sent false
26465 7114 Acked Read 7 Done Done Sent false
26466 7115 Acked Read 4 Done Done Sent false
26467 7116 Acked Read 5 Done Done Sent false
26468 7117 Acked Read 6 Done Done Sent false
26469 7118 Acked Write 2 Done Done Sent false
26470 7119 Acked Read 10 Done Done Sent false
26471 7120 Acked Read 9 Done Done Sent false
26472 7121 Acked Read 3 Done Done Sent false
26473 7122 Acked Write 5 Done Done Sent false
26474 7123 Acked Read 10 Done Done Sent false
26475 7124 Acked Write 2 Done Done Sent false
26476 7125 Acked Write 9 Done Done Sent false
26477 7126 Acked Read 7 Done Done Sent false
26478 7127 Acked Write 3 Done Done Sent false
26479 7128 Acked Write 8 Done Done Sent false
26480 7129 Acked Read 1 Done Done Sent false
26481 7130 Acked Read 4 Done Done Sent false
26482 7131 Acked Read 5 Done Done Sent false
26483 7132 Acked Read 10 Done Done Sent false
26484 7133 Acked Read 4 Done Done Sent false
26485 7134 Acked Read 2 Done Done Sent false
26486 7135 Acked Read 3 Done Done Sent false
26487 7136 Acked Read 1 Done Done Sent false
26488 7137 Acked Write 6 Done Done Sent false
26489 7138 Acked Read 7 Done Done Sent false
26490 7139 Acked Write 9 Done Done Sent false
26491 7140 Acked Read 2 Done Done Sent false
26492 7141 Acked Read 3 Done Done Sent false
26493 7142 Acked Write 2 Done Done Sent false
26494 7143 Acked Read 7 Done Done Sent false
26495 7144 Acked Write 10 Done Done Sent false
26496 7145 Acked Write 6 Done Done Sent false
26497 7146 Acked Read 7 Done Done Sent false
26498 7147 Acked Write 8 Done Done Sent false
26499 7148 Acked Read 4 Done Done Sent false
26500 7149 Acked Read 4 Done Done Sent false
26501 7150 Acked Read 1 Done Done Sent false
26502 7151 Acked Write 4 Done Done Sent false
26503 7152 Acked Write 2 Done Done Sent false
26504 7153 Acked Read 9 Done Done Sent false
26505 7154 Acked Flush 0 Done Done Sent false
26506 7155 Acked Write 1 Done Done Sent false
26507 7156 Acked Read 7 Done Done Sent false
26508 7157 Acked Read 9 Done Done Sent false
26509 7158 Acked Read 1 Done Done Sent false
26510 7159 Acked Read 3 Done Done Sent false
26511 7160 Acked Read 8 Done Done Sent false
26512 7161 Acked Write 7 Done Done Sent false
26513 7162 Acked Write 5 Done Done Sent false
26514 7163 Acked Write 8 Done Done Sent false
26515 7164 Acked Read 2 Done Done Sent false
26516 7165 Acked Read 10 Done Done Sent false
26517 7166 Acked Write 2 Done Done Sent false
26518 7167 Acked Write 5 Done Done Sent false
26519 7168 Acked Read 2 Done Done Sent false
26520 7169 Acked Read 7 Done Done Sent false
26521 7170 Acked Write 2 Done Done Sent false
26522 7171 Acked Read 9 Done Done Sent false
26523 7172 Acked Write 3 Done Done Sent false
26524 7173 Acked Read 3 Done Done Sent false
26525 7174 Acked Write 5 Done Done Sent false
26526 7175 Acked Read 5 Done Done Sent false
26527 7176 Acked Write 10 Done Done Sent false
26528 7177 Acked Flush 0 Done Done Sent false
26529 7178 Acked Write 4 Done Done Sent false
26530 7179 Acked Write 7 Done Done Sent false
26531 7180 Acked Write 4 Done Done Sent false
26532 7181 Acked Write 2 Done Done Sent false
26533 7182 Acked Read 1 Done Done Sent false
26534 7183 Acked Write 6 Done Done Sent false
26535 7184 Acked Read 4 Done Done Sent false
26536 7185 Acked Read 10 Done Done Sent false
26537 7186 Acked Read 9 Done Done Sent false
26538 7187 Acked Flush 0 Done Done Sent false
26539 7188 Acked Read 6 Done Done Sent false
26540 7189 Acked Write 8 Done Done Sent false
26541 7190 Acked Write 8 Done Done Sent false
26542 7191 Acked Read 1 Done Done Sent false
26543 7192 Acked Read 1 Done Done Sent false
26544 7193 Acked Write 8 Done Done Sent false
26545 7194 Acked Read 4 Done Done Sent false
26546 7195 Acked Write 5 Done Done Sent false
26547 7196 Acked Read 6 Done Done Sent false
26548 7197 Acked Read 2 Done Done Sent false
26549 7198 Acked Read 10 Done Done Sent false
26550 7199 Acked Read 1 Done Done Sent false
26551 7200 Acked Read 5 Done Done Sent false
26552 7201 Acked Write 2 Done Done Sent false
26553 7202 Acked Flush 0 Done Done Sent false
26554 7203 Acked Read 2 Done Done Sent false
26555 7204 Acked Flush 0 Done Done Sent false
26556 7205 Acked Read 9 Done Done Sent false
26557 7206 Acked Read 2 Done Done Sent false
26558 7207 Acked Read 10 Done Done Sent false
26559 7208 Acked Write 2 Done Done Sent false
26560 7209 Acked Read 8 Done Done Sent false
26561 7210 Acked Write 2 Done Done Sent false
26562 7211 Acked Read 3 Done Done Sent false
26563 7212 Acked Flush 0 Done Done Sent false
26564 7213 Acked Write 2 Done Done Sent false
26565 7214 Acked Write 7 Done Done Sent false
26566 7215 Acked Read 10 Done Done Sent false
26567 7216 Acked Read 10 Done Done Sent false
26568 7217 Acked Write 7 Done Done Sent false
26569 7218 Acked Flush 0 Done Done Sent false
26570 7219 Acked Write 6 Done Done Sent false
26571 7220 Acked Read 4 Done Done Sent false
26572 7221 Acked Write 4 Done Done Sent false
26573 7222 Acked Read 4 Done Done Sent false
26574 7223 Acked Read 5 Done Done Sent false
26575 7224 Acked Write 10 Done Done Sent false
26576 7225 Acked Read 2 Done Done Sent false
26577 7226 Acked Read 1 Done Done Sent false
26578 7227 Acked Read 10 Done Done Sent false
26579 7228 Acked Flush 0 Done Done Sent false
26580 7229 Acked Write 10 Done Done Sent false
26581 7230 Acked Write 4 Done Done Sent false
26582 7231 Acked Read 10 Done Done Sent false
26583 7232 Acked Write 10 Done Done Sent false
26584 7233 Acked Read 1 Done Done Sent false
26585 7234 Acked Write 3 Done Done Sent false
26586 7235 Acked Write 6 Done Done Sent false
26587 7236 Acked Read 10 Done Done Sent false
26588 7237 Acked Write 4 Done Done Sent false
26589 7238 Acked Read 6 Done Done Sent false
26590 7239 Acked Read 1 Done Done Sent false
26591 7240 Acked Write 5 Done Done Sent false
26592 7241 Acked Read 10 Done Done Sent false
26593 7242 Acked Read 6 Done Done Sent false
26594 7243 Acked Write 10 Done Done Sent false
26595 7244 Acked Read 2 Done Done Sent false
26596 7245 Acked Read 1 Done Done Sent false
26597 7246 Acked Read 4 Done Done Sent false
26598 7247 Acked Read 5 Done Done Sent false
26599 7248 Acked Read 2 Done Done Sent false
26600 7249 Acked Write 10 Done Done Sent false
26601 7250 Acked Flush 0 Done Done Sent false
26602 7251 Acked Read 5 Done Done Sent false
26603 7252 Acked Write 7 Done Done Sent false
26604 7253 Acked Write 2 Done Done Sent false
26605 7254 Acked Read 8 Done Done Sent false
26606 7255 Acked Write 5 Done Done Sent false
26607 7256 Acked Read 8 Done Done Sent false
26608 7257 Acked Write 7 Done Done Sent false
26609 7258 Acked Read 9 Done Done Sent false
26610 7259 Acked Read 10 Done Done Sent false
26611 7260 Acked Write 1 Done Done Sent false
26612 7261 Acked Flush 0 Done Done Sent false
26613 7262 Acked Read 5 Done Done Sent false
26614 7263 Acked Write 1 Done Done Sent false
26615 7264 Acked Write 1 Done Done Sent false
26616 7265 Acked Flush 0 Done Done Sent false
26617 7266 Acked Write 6 Done Done Sent false
26618 7267 Acked Write 5 Done Done Sent false
26619 7268 Acked Read 7 Done Done Sent false
26620 7269 Acked Read 8 Done Done Sent false
26621 7270 Acked Read 4 Done Done Sent false
26622 7271 Acked Write 1 Done Done Sent false
26623 7272 Acked Read 3 Done Done Sent false
26624 7273 Acked Read 9 Done Done Sent false
26625 7274 Acked Read 2 Done Done Sent false
26626 7275 Acked Read 2 Done Done Sent false
26627 7276 Acked Write 10 Done Done Sent false
26628 7277 Acked Write 2 Done Done Sent false
26629 7278 Acked Read 1 Done Done Sent false
26630 7279 Acked Write 1 Done Done Sent false
26631 7280 Acked Read 7 Done Done Sent false
26632 7281 Acked Read 5 Done Done Sent false
26633 7282 Acked Write 3 Done Done Sent false
26634 7283 Acked Read 4 Done Done Sent false
26635 7284 Acked Flush 0 Done Done Sent false
26636 7285 Acked Write 4 Done Done Sent false
26637 7286 Acked Write 7 Done Done Sent false
26638 7287 Acked Read 5 Done Done Sent false
26639 7288 Acked Read 2 Done Done Sent false
26640 7289 Acked Read 9 Done Done Sent false
26641 7290 Acked Read 6 Done Done Sent false
26642 7291 Acked Read 5 Done Done Sent false
26643 7292 Acked Write 6 Done Done Sent false
26644 7293 Acked Read 2 Done Done Sent false
26645 7294 Acked Write 9 Done Done Sent false
26646 7295 Acked Read 9 Done Done Sent false
26647 7296 Acked Read 3 Done Done Sent false
26648 7297 Acked Flush 0 Done Done Sent false
26649 7298 Acked Write 4 Done Done Sent false
26650 7299 Acked Write 8 Done Done Sent false
26651 7300 Acked Flush 0 Done Done Sent false
26652 7301 Acked Write 7 Done Done Sent false
26653 7302 Acked Read 5 Done Done Sent false
26654 7303 Acked Write 10 Done Done Sent false
26655 7304 Acked Flush 0 Done Done Sent false
26656 7305 Acked Read 3 Done Done Sent false
26657 7306 Acked Read 10 Done Done Sent false
26658 7307 Acked Read 1 Done Done Sent false
26659 7308 Acked Flush 0 Done Done Sent false
26660 7309 Acked Write 5 Done Done Sent false
26661 7310 Acked Flush 0 Done Done Sent false
26662 7311 Acked Write 5 Done Done Sent false
26663 7312 Acked Read 5 Done Done Sent false
26664 7313 Acked Read 6 Done Done Sent false
26665 7314 Acked Flush 0 Done Done Sent false
26666 7315 Acked Read 5 Done Done Sent false
26667 7316 Acked Read 1 Done Done Sent false
26668 7317 Acked Write 7 Done Done Sent false
26669 7318 Acked Write 9 Done Done Sent false
26670 7319 Acked Read 5 Done Done Sent false
26671 7320 Acked Write 9 Done Done Sent false
26672 7321 Acked Write 2 Done Done Sent false
26673 7322 Acked Write 4 Done Done Sent false
26674 7323 Acked Read 3 Done Done Sent false
26675 7324 Acked Write 1 Done Done Sent false
26676 7325 Acked Write 6 Done Done Sent false
26677 7326 Acked Read 3 Done Done Sent false
26678 7327 Acked Read 10 Done Done Sent false
26679 7328 Acked Read 9 Done Done Sent false
26680 7329 Acked Write 4 Done Done Sent false
26681 7330 Acked Write 4 Done Done Sent false
26682 7331 Acked Flush 0 Done Done Sent false
26683 7332 Acked Read 1 Done Done Sent false
26684 7333 Acked Write 2 Done Done Sent false
26685 7334 Acked Write 6 Done Done Sent false
26686 7335 Acked Flush 0 Done Done Sent false
26687 7336 Acked Write 8 Done Done Sent false
26688 7337 Acked Read 6 Done Done Sent false
26689 7338 Acked Read 6 Done Done Sent false
26690 STATES DS:0 DS:1 DS:2 TOTAL
26691 Sent 0 0 301 301
26692 Done 301 301 0 602
26693 Skipped 0 0 0 0
26694 Error 0 0 0 0
26695 Last Flush: 7335 7335 7037
26696 Downstairs last five completed: 7037 7036 7035 7034 7033
26697 Upstairs last five completed: 7338 7337 7336 7335 7334
26698 CLIENT: Up:0 ds:301 act:2
26699 [8830] got start action
26700 [8830] I am going to restart
26701 Starting downstairs at port 8830
26702 Make output file at "/tmp/dsc/downstairs-8830.txt"
26703 [8830][2] reports Starting
26704 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1226)
26705 [8830][2] reports Running
26706 ----------------------------------------------------------------
26707 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
26708 States: Active Active Offline
26709 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
26710 7038 Acked Write 3 Done Done Sent false
26711 7039 Acked Read 9 Done Done Sent false
26712 7040 Acked Write 2 Done Done Sent false
26713 7041 Acked Flush 0 Done Done Sent false
26714 7042 Acked Write 7 Done Done Sent false
26715 7043 Acked Write 4 Done Done Sent false
26716 7044 Acked Flush 0 Done Done Sent false
26717 7045 Acked Write 7 Done Done Sent false
26718 7046 Acked Read 1 Done Done Sent false
26719 7047 Acked Read 5 Done Done Sent false
26720 7048 Acked Read 7 Done Done Sent false
26721 7049 Acked Read 7 Done Done Sent false
26722 7050 Acked Flush 0 Done Done Sent false
26723 7051 Acked Read 1 Done Done Sent false
26724 7052 Acked Read 6 Done Done Sent false
26725 7053 Acked Write 6 Done Done Sent false
26726 7054 Acked Read 5 Done Done Sent false
26727 7055 Acked Read 10 Done Done Sent false
26728 7056 Acked Write 1 Done Done Sent false
26729 7057 Acked Write 4 Done Done Sent false
26730 7058 Acked Write 3 Done Done Sent false
26731 7059 Acked Read 4 Done Done Sent false
26732 7060 Acked Read 6 Done Done Sent false
26733 7061 Acked Write 3 Done Done Sent false
26734 7062 Acked Write 4 Done Done Sent false
26735 7063 Acked Read 6 Done Done Sent false
26736 7064 Acked Write 2 Done Done Sent false
26737 7065 Acked Read 6 Done Done Sent false
26738 7066 Acked Flush 0 Done Done Sent false
26739 7067 Acked Write 2 Done Done Sent false
26740 7068 Acked Write 2 Done Done Sent false
26741 7069 Acked Write 3 Done Done Sent false
26742 7070 Acked Read 6 Done Done Sent false
26743 7071 Acked Write 2 Done Done Sent false
26744 7072 Acked Read 6 Done Done Sent false
26745 7073 Acked Read 2 Done Done Sent false
26746 7074 Acked Flush 0 Done Done Sent false
26747 7075 Acked Write 9 Done Done Sent false
26748 7076 Acked Read 3 Done Done Sent false
26749 7077 Acked Flush 0 Done Done Sent false
26750 7078 Acked Read 3 Done Done Sent false
26751 7079 Acked Write 3 Done Done Sent false
26752 7080 Acked Read 7 Done Done Sent false
26753 7081 Acked Write 4 Done Done Sent false
26754 7082 Acked Write 1 Done Done Sent false
26755 7083 Acked Read 4 Done Done Sent false
26756 7084 Acked Flush 0 Done Done Sent false
26757 7085 Acked Read 6 Done Done Sent false
26758 7086 Acked Flush 0 Done Done Sent false
26759 7087 Acked Write 4 Done Done Sent false
26760 7088 Acked Write 9 Done Done Sent false
26761 7089 Acked Read 1 Done Done Sent false
26762 7090 Acked Write 8 Done Done Sent false
26763 7091 Acked Write 7 Done Done Sent false
26764 7092 Acked Read 3 Done Done Sent false
26765 7093 Acked Write 10 Done Done Sent false
26766 7094 Acked Write 9 Done Done Sent false
26767 7095 Acked Read 6 Done Done Sent false
26768 7096 Acked Flush 0 Done Done Sent false
26769 7097 Acked Read 8 Done Done Sent false
26770 7098 Acked Read 9 Done Done Sent false
26771 7099 Acked Read 1 Done Done Sent false
26772 7100 Acked Write 10 Done Done Sent false
26773 7101 Acked Read 7 Done Done Sent false
26774 7102 Acked Read 3 Done Done Sent false
26775 7103 Acked Read 6 Done Done Sent false
26776 7104 Acked Read 7 Done Done Sent false
26777 7105 Acked Write 6 Done Done Sent false
26778 7106 Acked Read 4 Done Done Sent false
26779 7107 Acked Write 8 Done Done Sent false
26780 7108 Acked Write 5 Done Done Sent false
26781 7109 Acked Read 6 Done Done Sent false
26782 7110 Acked Write 10 Done Done Sent false
26783 7111 Acked Read 5 Done Done Sent false
26784 7112 Acked Write 5 Done Done Sent false
26785 7113 Acked Read 2 Done Done Sent false
26786 7114 Acked Read 7 Done Done Sent false
26787 7115 Acked Read 4 Done Done Sent false
26788 7116 Acked Read 5 Done Done Sent false
26789 7117 Acked Read 6 Done Done Sent false
26790 7118 Acked Write 2 Done Done Sent false
26791 7119 Acked Read 10 Done Done Sent false
26792 7120 Acked Read 9 Done Done Sent false
26793 7121 Acked Read 3 Done Done Sent false
26794 7122 Acked Write 5 Done Done Sent false
26795 7123 Acked Read 10 Done Done Sent false
26796 7124 Acked Write 2 Done Done Sent false
26797 7125 Acked Write 9 Done Done Sent false
26798 7126 Acked Read 7 Done Done Sent false
26799 7127 Acked Write 3 Done Done Sent false
26800 7128 Acked Write 8 Done Done Sent false
26801 7129 Acked Read 1 Done Done Sent false
26802 7130 Acked Read 4 Done Done Sent false
26803 7131 Acked Read 5 Done Done Sent false
26804 7132 Acked Read 10 Done Done Sent false
26805 7133 Acked Read 4 Done Done Sent false
26806 7134 Acked Read 2 Done Done Sent false
26807 7135 Acked Read 3 Done Done Sent false
26808 7136 Acked Read 1 Done Done Sent false
26809 7137 Acked Write 6 Done Done Sent false
26810 7138 Acked Read 7 Done Done Sent false
26811 7139 Acked Write 9 Done Done Sent false
26812 7140 Acked Read 2 Done Done Sent false
26813 7141 Acked Read 3 Done Done Sent false
26814 7142 Acked Write 2 Done Done Sent false
26815 7143 Acked Read 7 Done Done Sent false
26816 7144 Acked Write 10 Done Done Sent false
26817 7145 Acked Write 6 Done Done Sent false
26818 7146 Acked Read 7 Done Done Sent false
26819 7147 Acked Write 8 Done Done Sent false
26820 7148 Acked Read 4 Done Done Sent false
26821 7149 Acked Read 4 Done Done Sent false
26822 7150 Acked Read 1 Done Done Sent false
26823 7151 Acked Write 4 Done Done Sent false
26824 7152 Acked Write 2 Done Done Sent false
26825 7153 Acked Read 9 Done Done Sent false
26826 7154 Acked Flush 0 Done Done Sent false
26827 7155 Acked Write 1 Done Done Sent false
26828 7156 Acked Read 7 Done Done Sent false
26829 7157 Acked Read 9 Done Done Sent false
26830 7158 Acked Read 1 Done Done Sent false
26831 7159 Acked Read 3 Done Done Sent false
26832 7160 Acked Read 8 Done Done Sent false
26833 7161 Acked Write 7 Done Done Sent false
26834 7162 Acked Write 5 Done Done Sent false
26835 7163 Acked Write 8 Done Done Sent false
26836 7164 Acked Read 2 Done Done Sent false
26837 7165 Acked Read 10 Done Done Sent false
26838 7166 Acked Write 2 Done Done Sent false
26839 7167 Acked Write 5 Done Done Sent false
26840 7168 Acked Read 2 Done Done Sent false
26841 7169 Acked Read 7 Done Done Sent false
26842 7170 Acked Write 2 Done Done Sent false
26843 7171 Acked Read 9 Done Done Sent false
26844 7172 Acked Write 3 Done Done Sent false
26845 7173 Acked Read 3 Done Done Sent false
26846 7174 Acked Write 5 Done Done Sent false
26847 7175 Acked Read 5 Done Done Sent false
26848 7176 Acked Write 10 Done Done Sent false
26849 7177 Acked Flush 0 Done Done Sent false
26850 7178 Acked Write 4 Done Done Sent false
26851 7179 Acked Write 7 Done Done Sent false
26852 7180 Acked Write 4 Done Done Sent false
26853 7181 Acked Write 2 Done Done Sent false
26854 7182 Acked Read 1 Done Done Sent false
26855 7183 Acked Write 6 Done Done Sent false
26856 7184 Acked Read 4 Done Done Sent false
26857 7185 Acked Read 10 Done Done Sent false
26858 7186 Acked Read 9 Done Done Sent false
26859 7187 Acked Flush 0 Done Done Sent false
26860 7188 Acked Read 6 Done Done Sent false
26861 7189 Acked Write 8 Done Done Sent false
26862 7190 Acked Write 8 Done Done Sent false
26863 7191 Acked Read 1 Done Done Sent false
26864 7192 Acked Read 1 Done Done Sent false
26865 7193 Acked Write 8 Done Done Sent false
26866 7194 Acked Read 4 Done Done Sent false
26867 7195 Acked Write 5 Done Done Sent false
26868 7196 Acked Read 6 Done Done Sent false
26869 7197 Acked Read 2 Done Done Sent false
26870 7198 Acked Read 10 Done Done Sent false
26871 7199 Acked Read 1 Done Done Sent false
26872 7200 Acked Read 5 Done Done Sent false
26873 7201 Acked Write 2 Done Done Sent false
26874 7202 Acked Flush 0 Done Done Sent false
26875 7203 Acked Read 2 Done Done Sent false
26876 7204 Acked Flush 0 Done Done Sent false
26877 7205 Acked Read 9 Done Done Sent false
26878 7206 Acked Read 2 Done Done Sent false
26879 7207 Acked Read 10 Done Done Sent false
26880 7208 Acked Write 2 Done Done Sent false
26881 7209 Acked Read 8 Done Done Sent false
26882 7210 Acked Write 2 Done Done Sent false
26883 7211 Acked Read 3 Done Done Sent false
26884 7212 Acked Flush 0 Done Done Sent false
26885 7213 Acked Write 2 Done Done Sent false
26886 7214 Acked Write 7 Done Done Sent false
26887 7215 Acked Read 10 Done Done Sent false
26888 7216 Acked Read 10 Done Done Sent false
26889 7217 Acked Write 7 Done Done Sent false
26890 7218 Acked Flush 0 Done Done Sent false
26891 7219 Acked Write 6 Done Done Sent false
26892 7220 Acked Read 4 Done Done Sent false
26893 7221 Acked Write 4 Done Done Sent false
26894 7222 Acked Read 4 Done Done Sent false
26895 7223 Acked Read 5 Done Done Sent false
26896 7224 Acked Write 10 Done Done Sent false
26897 7225 Acked Read 2 Done Done Sent false
26898 7226 Acked Read 1 Done Done Sent false
26899 7227 Acked Read 10 Done Done Sent false
26900 7228 Acked Flush 0 Done Done Sent false
26901 7229 Acked Write 10 Done Done Sent false
26902 7230 Acked Write 4 Done Done Sent false
26903 7231 Acked Read 10 Done Done Sent false
26904 7232 Acked Write 10 Done Done Sent false
26905 7233 Acked Read 1 Done Done Sent false
26906 7234 Acked Write 3 Done Done Sent false
26907 7235 Acked Write 6 Done Done Sent false
26908 7236 Acked Read 10 Done Done Sent false
26909 7237 Acked Write 4 Done Done Sent false
26910 7238 Acked Read 6 Done Done Sent false
26911 7239 Acked Read 1 Done Done Sent false
26912 7240 Acked Write 5 Done Done Sent false
26913 7241 Acked Read 10 Done Done Sent false
26914 7242 Acked Read 6 Done Done Sent false
26915 7243 Acked Write 10 Done Done Sent false
26916 7244 Acked Read 2 Done Done Sent false
26917 7245 Acked Read 1 Done Done Sent false
26918 7246 Acked Read 4 Done Done Sent false
26919 7247 Acked Read 5 Done Done Sent false
26920 7248 Acked Read 2 Done Done Sent false
26921 7249 Acked Write 10 Done Done Sent false
26922 7250 Acked Flush 0 Done Done Sent false
26923 7251 Acked Read 5 Done Done Sent false
26924 7252 Acked Write 7 Done Done Sent false
26925 7253 Acked Write 2 Done Done Sent false
26926 7254 Acked Read 8 Done Done Sent false
26927 7255 Acked Write 5 Done Done Sent false
26928 7256 Acked Read 8 Done Done Sent false
26929 7257 Acked Write 7 Done Done Sent false
26930 7258 Acked Read 9 Done Done Sent false
26931 7259 Acked Read 10 Done Done Sent false
26932 7260 Acked Write 1 Done Done Sent false
26933 7261 Acked Flush 0 Done Done Sent false
26934 7262 Acked Read 5 Done Done Sent false
26935 7263 Acked Write 1 Done Done Sent false
26936 7264 Acked Write 1 Done Done Sent false
26937 7265 Acked Flush 0 Done Done Sent false
26938 7266 Acked Write 6 Done Done Sent false
26939 7267 Acked Write 5 Done Done Sent false
26940 7268 Acked Read 7 Done Done Sent false
26941 7269 Acked Read 8 Done Done Sent false
26942 7270 Acked Read 4 Done Done Sent false
26943 7271 Acked Write 1 Done Done Sent false
26944 7272 Acked Read 3 Done Done Sent false
26945 7273 Acked Read 9 Done Done Sent false
26946 7274 Acked Read 2 Done Done Sent false
26947 7275 Acked Read 2 Done Done Sent false
26948 7276 Acked Write 10 Done Done Sent false
26949 7277 Acked Write 2 Done Done Sent false
26950 7278 Acked Read 1 Done Done Sent false
26951 7279 Acked Write 1 Done Done Sent false
26952 7280 Acked Read 7 Done Done Sent false
26953 7281 Acked Read 5 Done Done Sent false
26954 7282 Acked Write 3 Done Done Sent false
26955 7283 Acked Read 4 Done Done Sent false
26956 7284 Acked Flush 0 Done Done Sent false
26957 7285 Acked Write 4 Done Done Sent false
26958 7286 Acked Write 7 Done Done Sent false
26959 7287 Acked Read 5 Done Done Sent false
26960 7288 Acked Read 2 Done Done Sent false
26961 7289 Acked Read 9 Done Done Sent false
26962 7290 Acked Read 6 Done Done Sent false
26963 7291 Acked Read 5 Done Done Sent false
26964 7292 Acked Write 6 Done Done Sent false
26965 7293 Acked Read 2 Done Done Sent false
26966 7294 Acked Write 9 Done Done Sent false
26967 7295 Acked Read 9 Done Done Sent false
26968 7296 Acked Read 3 Done Done Sent false
26969 7297 Acked Flush 0 Done Done Sent false
26970 7298 Acked Write 4 Done Done Sent false
26971 7299 Acked Write 8 Done Done Sent false
26972 7300 Acked Flush 0 Done Done Sent false
26973 7301 Acked Write 7 Done Done Sent false
26974 7302 Acked Read 5 Done Done Sent false
26975 7303 Acked Write 10 Done Done Sent false
26976 7304 Acked Flush 0 Done Done Sent false
26977 7305 Acked Read 3 Done Done Sent false
26978 7306 Acked Read 10 Done Done Sent false
26979 7307 Acked Read 1 Done Done Sent false
26980 7308 Acked Flush 0 Done Done Sent false
26981 7309 Acked Write 5 Done Done Sent false
26982 7310 Acked Flush 0 Done Done Sent false
26983 7311 Acked Write 5 Done Done Sent false
26984 7312 Acked Read 5 Done Done Sent false
26985 7313 Acked Read 6 Done Done Sent false
26986 7314 Acked Flush 0 Done Done Sent false
26987 7315 Acked Read 5 Done Done Sent false
26988 7316 Acked Read 1 Done Done Sent false
26989 7317 Acked Write 7 Done Done Sent false
26990 7318 Acked Write 9 Done Done Sent false
26991 7319 Acked Read 5 Done Done Sent false
26992 7320 Acked Write 9 Done Done Sent false
26993 7321 Acked Write 2 Done Done Sent false
26994 7322 Acked Write 4 Done Done Sent false
26995 7323 Acked Read 3 Done Done Sent false
26996 7324 Acked Write 1 Done Done Sent false
26997 7325 Acked Write 6 Done Done Sent false
26998 7326 Acked Read 3 Done Done Sent false
26999 7327 Acked Read 10 Done Done Sent false
27000 7328 Acked Read 9 Done Done Sent false
27001 7329 Acked Write 4 Done Done Sent false
27002 7330 Acked Write 4 Done Done Sent false
27003 7331 Acked Flush 0 Done Done Sent false
27004 7332 Acked Read 1 Done Done Sent false
27005 7333 Acked Write 2 Done Done Sent false
27006 7334 Acked Write 6 Done Done Sent false
27007 7335 Acked Flush 0 Done Done Sent false
27008 7336 Acked Write 8 Done Done Sent false
27009 7337 Acked Read 6 Done Done Sent false
27010 7338 Acked Read 6 Done Done Sent false
27011 7339 Acked Flush 0 Done Done Sent false
27012 STATES DS:0 DS:1 DS:2 TOTAL
27013 Sent 0 0 302 302
27014 Done 302 302 0 604
27015 Skipped 0 0 0 0
27016 Error 0 0 0 0
27017 Last Flush: 7339 7339 7037
27018 Downstairs last five completed: 7037 7036 7035 7034 7033
27019 Upstairs last five completed: 7339 7338 7337 7336 7335
27020 CLIENT: Up:0 ds:302 act:2
270212024-11-15T02:03:06.832ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
270222024-11-15T02:03:06.832ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:60592) = io task client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
270232024-11-15T02:03:06.833ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
270242024-11-15T02:03:06.834ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
270252024-11-15T02:03:06.834ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
270262024-11-15T02:03:06.834ZINFOcrucible: send last flush ID to this DS: 7037 = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
270272024-11-15T02:03:06.834ZINFOcrucible: Replied this last flush ID: 7037 = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
270282024-11-15T02:03:06.834ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
270292024-11-15T02:03:06.834ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
270302024-11-15T02:03:06.834ZINFOcrucible: [2] client re-new 302 jobs since flush 7037 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
270312024-11-15T02:03:06.834ZINFOcrucible: [2] Replayed 302 jobs since flush: 7037 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
27032 ----------------------------------------------------------------
27033 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
27034 Downstairs last five completed: 7339 7338 7337 7336 7335
27035 Upstairs last five completed: 7339 7338 7337 7336 7335
27036 CLIENT: Up:0 ds:0 act:3
27037 Replay: All jobs finished
27038 Main task has work to do, go find it
27039 got dsc Stop(1)
27040 stop 1
27041 [8820] Got stop action so:false kr:false
27042 Nov 15 02:03:09.072 INFO request completed, latency_us: 341, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 549cab0c-b623-458b-9b10-fd2323e1cf71, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
27043 Nov 15 02:03:09.073 INFO request completed, latency_us: 124, {"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2024-11-15T02:03:09.07324432Z","hostname":"w-01JCPRHT5F2MPA44KVJHM3S2PB","pid":1205,"":"io task","client":"1","":"downstairs","session_id":"971dcb2e-6891-4ecf-b367-1e04e52065ee"}
27044 response_code: 200, uri: /state/cid/1, method: GET, req_id: 5c0d7a78-480d-4f2b-8db9-abdf82b4535d, remote_addr: 127.0.0.1:40008{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2024-11-15T02:03:09.073326305Z","hostname":"w-01JCPRHT5F2MPA44KVJHM3S2PB","pid":1205,"":"io task","client":"1","":"downstairs","session_id":"971dcb2e-6891-4ecf-b367-1e04e52065ee"}
27045 , local_addr: 127.0.0.1:9998
270462024-11-15T02:03:09.073ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
270472024-11-15T02:03:09.073ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
270482024-11-15T02:03:09.073ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
270492024-11-15T02:03:09.073ZINFOcrucible: client task is exiting = io task client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
27050 [8820] Exited with: ExitStatus(unix_wait_status(9))
27051 [8820][1] reports Exit
27052 Nov 15 02:03:13.075 INFO request completed, latency_us: 213, response_code: 200, uri: /state/cid/1, method: GET, req_id: 7c3046b8-6502-41ef-a87a-5afd21b6366d, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
27053 001/300 Write block 2983 len 12288 data: 8 7 7
27054 002/300 Read block 3988 len 4096
27055 003/300 Write block 3815 len 8192 data: 4 3
27056 004/300 Write block 2414 len 32768 data: 4 4 3 3 3 3 2 4
27057 005/300 Write block 1076 len 16384 data: 3 3 2 2
27058 006/300 Read block 1504 len 20480
27059 007/300 Read block 4134 len 20480
27060 008/300 Read block 4645 len 8192
27061 009/300 Write block 1914 len 20480 data: 4 5 5 4 4
27062 010/300 Write block 2954 len 20480 data: 7 6 6 6 6
27063 011/300 Flush
27064 012/300 Write block 3620 len 12288 data: 4 3 4
27065 013/300 Flush
27066 014/300 Write block 2979 len 28672 data: 6 6 7 7 9 8 8
27067 015/300 Read block 724 len 8192
27068 016/300 Write block 2537 len 8192 data: 3 4
27069 017/300 Write block 3434 len 40960 data: 6 5 5 5 5 5 6 7 7 7
27070 018/300 Flush
27071 019/300 Read block 268 len 28672
27072 020/300 Write block 1817 len 40960 data: 3 3 3 4 5 5 5 5 5 4
27073 021/300 Read block 3230 len 8192
27074 022/300 Write block 3671 len 40960 data: 6 6 5 6 6 5 5 5 4 4
27075 023/300 Write block 3676 len 40960 data: 6 6 6 5 5 3 4 5 4 4
27076 024/300 Write block 1396 len 4096 data: 2
27077 025/300 Write block 877 len 16384 data: 8 8 6 6
27078 026/300 Write block 2311 len 36864 data: 6 6 5 6 5 5 4 6 8
27079 027/300 Write block 4065 len 20480 data: 7 6 6 5 5
27080 028/300 Read block 2524 len 32768
27081 029/300 Read block 2811 len 16384
27082 030/300 Write block 3317 len 20480 data: 3 3 3 3 3
27083 031/300 Read block 799 len 4096
27084 032/300 Write block 3930 len 28672 data: 3 3 4 3 3 4 5
27085 033/300 Read block 877 len 40960
27086 034/300 Flush
27087 035/300 Write block 3164 len 20480 data: 4 3 4 4 4
27088 036/300 Read block 59 len 32768
27089 037/300 Write block 81 len 32768 data: 9 8 7 7 7 6 7 7
27090 038/300 Read block 2445 len 24576
27091 039/300 Read block 3448 len 16384
27092 040/300 Write block 4954 len 20480 data: 2 3 4 4 4
27093 041/300 Read block 232 len 40960
27094 042/300 Write block 4389 len 16384 data: 2 3 3 3
27095 043/300 Flush
27096 044/300 Write block 3966 len 8192 data: 3 3
27097 045/300 Write block 2810 len 28672 data: 4 4 4 4 5 6 7
27098 046/300 Write block 3671 len 4096 data: 7
27099 047/300 Read block 3785 len 8192
27100 048/300 Flush
27101 049/300 Flush
27102 050/300 Write block 211 len 28672 data: 5 4 4 5 5 5 4
27103 051/300 Read block 4001 len 8192
27104 052/300 Write block 4389 len 4096 data: 3
27105 053/300 Read block 525 len 36864
27106 054/300 Read block 2205 len 8192
27107 055/300 Write block 316 len 36864 data: 4 4 4 4 4 6 7 5 5
27108 056/300 Write block 605 len 4096 data: 4
27109 057/300 Read block 3809 len 12288
27110 058/300 Read block 632 len 4096
27111 059/300 Write block 1757 len 24576 data: 2 4 5 4 4 5
27112 060/300 Read block 4862 len 16384
27113 061/300 Read block 766 len 36864
27114 062/300 Read block 817 len 36864
27115 063/300 Read block 3261 len 16384
27116 064/300 Write block 2514 len 20480 data: 3 4 5 2 2
27117 065/300 Write block 415 len 16384 data: 5 5 5 6
27118 066/300 Read block 1660 len 12288
27119 067/300 Read block 2330 len 8192
27120 068/300 Write block 451 len 8192 data: 4 4
27121 069/300 Read block 2078 len 24576
27122 070/300 Read block 3136 len 36864
27123 071/300 Read block 2131 len 4096
27124 072/300 Write block 1339 len 12288 data: 4 5 5
27125 073/300 Write block 784 len 36864 data: 8 9 9 9 8 8 8 7 5
27126 074/300 Write block 3646 len 16384 data: 4 5 8 10
27127 075/300 Read block 2517 len 36864
27128 076/300 Read block 4634 len 8192
27129 077/300 Write block 294 len 28672 data: 6 5 4 4 3 3 4
27130 078/300 Read block 1861 len 32768
27131 079/300 Write block 3224 len 24576 data: 5 6 6 4 4 4
27132 080/300 Read block 2818 len 40960
27133 081/300 Read block 838 len 40960
27134 082/300 Read block 3662 len 40960
27135 083/300 Flush
27136 084/300 Read block 3892 len 20480
27137 085/300 Read block 1446 len 4096
27138 086/300 Flush
27139 087/300 Flush
27140 088/300 Write block 1964 len 4096 data: 2
27141 089/300 Read block 4317 len 20480
27142 090/300 Write block 1755 len 40960 data: 2 2 3 5 6 5 5 6 6 6
27143 091/300 Write block 4860 len 40960 data: 6 7 8 9 8 7 6 5 3 3
27144 092/300 Write block 1292 len 24576 data: 3 3 4 5 5 6
27145 093/300 Write block 2271 len 16384 data: 5 6 7 6
27146 094/300 Write block 2757 len 24576 data: 5 5 6 6 6 5
27147 095/300 Read block 1680 len 24576
27148 096/300 Flush
27149 097/300 Write block 1879 len 12288 data: 4 5 6
27150 098/300 Read block 552 len 20480
27151 099/300 Read block 3644 len 12288
27152 100/300 Read block 4373 len 32768
27153 101/300 Write block 3400 len 12288 data: 4 5 5
27154 102/300 Write block 1806 len 8192 data: 4 4
27155 103/300 Flush
27156 104/300 Write block 4273 len 32768 data: 6 5 4 4 3 4 3 4
27157 105/300 Read block 595 len 16384
27158 106/300 Read block 3220 len 40960
27159 107/300 Read block 1637 len 24576
27160 108/300 Flush
27161 109/300 Read block 3146 len 36864
27162 110/300 Write block 1772 len 20480 data: 4 4 4 3 6
27163 111/300 Write block 64 len 36864 data: 3 4 5 5 5 6 6 7 7
27164 112/300 Read block 2923 len 32768
27165 113/300 Read block 369 len 16384
27166 114/300 Write block 1624 len 20480 data: 4 3 3 3 4
27167 115/300 Write block 3852 len 40960 data: 5 5 5 2 2 2 2 3 4 4
27168 116/300 Write block 3499 len 24576 data: 6 6 6 4 4 4
27169 117/300 Write block 1090 len 36864 data: 2 2 2 2 2 5 6 6 5
27170 118/300 Read block 3399 len 8192
27171 119/300 Write block 4119 len 40960 data: 2 3 3 5 4 6 7 7 6 8
27172 120/300 Read block 4300 len 32768
27173 121/300 Read block 3885 len 36864
27174 122/300 Read block 4974 len 12288
27175 123/300 Read block 2793 len 32768
27176 124/300 Read block 638 len 8192
27177 125/300 Write block 2859 len 4096 data: 5
27178 126/300 Read block 2069 len 20480
27179 127/300 Write block 2677 len 28672 data: 3 4 4 4 4 5 3
27180 128/300 Read block 4327 len 12288
27181 129/300 Write block 3270 len 28672 data: 4 5 5 6 5 5 5
27182 130/300 Read block 3046 len 4096
27183 131/300 Read block 4881 len 16384
27184 132/300 Write block 2021 len 36864 data: 6 5 4 6 8 8 8 6 5
27185 133/300 Read block 2493 len 4096
27186 134/300 Read block 1841 len 20480
27187 135/300 Read block 4024 len 28672
27188 136/300 Write block 3495 len 24576 data: 5 5 5 6 7 7
27189 137/300 Read block 162 len 36864
27190 138/300 Read block 4047 len 8192
27191 139/300 Read block 4199 len 24576
27192 140/300 Flush
27193 141/300 Write block 4782 len 28672 data: 7 6 5 3 5 5 3
27194 142/300 Read block 4123 len 32768
27195 143/300 Flush
27196 144/300 Read block 17 len 20480
27197 145/300 Read block 343 len 20480
27198 146/300 Write block 4555 len 8192 data: 4 4
27199 147/300 Read block 1701 len 28672
27200 148/300 Read block 3434 len 28672
27201 149/300 Write block 69 len 24576 data: 7 7 8 8 6 6
27202 150/300 Read block 1218 len 40960
27203 151/300 Write block 4884 len 16384 data: 5 5 6 6
27204 152/300 Read block 2312 len 12288
27205 153/300 Write block 34 len 36864 data: 5 4 4 4 6 4 5 5 5
27206 154/300 Read block 423 len 4096
27207 155/300 Flush
27208 156/300 Write block 1816 len 28672 data: 4 4 4 4 5 6 6
27209 157/300 Read block 4539 len 16384
27210 158/300 Write block 520 len 24576 data: 3 3 3 3 3 4
27211 159/300 Write block 4629 len 24576 data: 2 2 2 3 3 4
27212 160/300 Flush
27213 161/300 Write block 4058 len 28672 data: 3 3 3 3 3 6 6
27214 162/300 Read block 3167 len 20480
27215 163/300 Read block 2875 len 16384
27216 164/300 Write block 3822 len 4096 data: 3
27217 165/300 Write block 3463 len 28672 data: 3 4 5 4 5 6 4
27218 166/300 Read block 2390 len 28672
27219 167/300 Write block 3469 len 32768 data: 5 3 3 3 3 2 3 3
27220 168/300 Flush
27221 169/300 Read block 2731 len 24576
27222 170/300 Read block 3886 len 8192
27223 171/300 Read block 4758 len 16384
27224 172/300 Read block 1766 len 12288
27225 173/300 Write block 1871 len 4096 data: 4
27226 174/300 Read block 2164 len 12288
27227 175/300 Write block 1700 len 16384 data: 5 5 5 4
27228 176/300 Flush
27229 177/300 Read block 3433 len 20480
27230 178/300 Read block 1869 len 28672
27231 179/300 Write block 849 len 8192 data: 4 4
27232 180/300 Flush
27233 181/300 Read block 1514 len 40960
27234 182/300 Read block 39 len 32768
27235 183/300 Read block 3009 len 16384
27236 184/300 Read block 686 len 32768
27237 185/300 Flush
27238 186/300 Write block 3330 len 20480 data: 6 4 5 6 6
27239 187/300 Flush
27240 188/300 Flush
27241 189/300 Flush
27242 190/300 Write block 661 len 20480 data: 6 5 6 4 5
27243 191/300 Flush
27244 192/300 Write block 2043 len 20480 data: 3 3 3 3 3
27245 193/300 Read block 4233 len 8192
27246 194/300 Read block 1364 len 36864
27247 195/300 Read block 4405 len 36864
27248 196/300 Read block 311 len 24576
27249 197/300 Write block 1267 len 12288 data: 4 4 5
27250 198/300 Write block 1363 len 28672 data: 6 5 7 6 5 6 5
27251 199/300 Read block 2998 len 24576
27252 200/300 Flush
27253 201/300 Flush
27254 202/300 Read block 981 len 28672
27255 203/300 Write block 1180 len 16384 data: 2 2 3 3
27256 204/300 Read block 2414 len 40960
27257 205/300 Read block 3547 len 40960
27258 206/300 Write block 4713 len 4096 data: 4
27259 207/300 Read block 30 len 32768
27260 208/300 Flush
27261 209/300 Write block 1188 len 36864 data: 6 4 2 2 4 5 6 7 10
27262 210/300 Write block 1405 len 4096 data: 6
27263 211/300 Write block 3879 len 4096 data: 4
27264 212/300 Flush
27265 213/300 Write block 3679 len 16384 data: 6 6 4 5
27266 214/300 Read block 2898 len 4096
27267 215/300 Flush
27268 216/300 Read block 3519 len 4096
27269 217/300 Read block 3622 len 8192
27270 218/300 Write block 3381 len 24576 data: 7 6 6 5 4 4
27271 219/300 Write block 3955 len 28672 data: 5 6 6 6 7 6 6
27272 220/300 Write block 1805 len 16384 data: 4 5 5 3
27273 221/300 Flush
27274 222/300 Write block 593 len 28672 data: 7 6 7 5 5 4 5
27275 223/300 Write block 2024 len 40960 data: 7 9 9 9 7 6 4 4 4 4
27276 224/300 Write block 4184 len 28672 data: 9 9 5 4 3 5 6
27277 225/300 Flush
27278 226/300 Write block 3105 len 36864 data: 3 3 4 4 3 4 3 4 3
27279 227/300 Write block 2991 len 12288 data: 5 5 3
27280 228/300 Flush
27281 229/300 Read block 1775 len 4096
27282 230/300 Read block 4814 len 12288
27283 231/300 Write block 2254 len 40960 data: 5 5 4 4 4 3 3 4 6 6
27284 232/300 Write block 2633 len 12288 data: 5 3 4
27285 233/300 Read block 3985 len 20480
27286 234/300 Read block 2143 len 32768
27287 235/300 Read block 4818 len 28672
27288 236/300 Write block 1286 len 16384 data: 4 3 5 3
27289 237/300 Read block 4839 len 32768
27290 238/300 Read block 3188 len 20480
27291 239/300 Flush
27292 240/300 Read block 1579 len 20480
27293 241/300 Flush
27294 242/300 Read block 2021 len 20480
27295 243/300 Read block 2771 len 20480
27296 244/300 Read block 1190 len 12288
27297 245/300 Write block 3748 len 36864 data: 4 4 5 6 5 4 5 5 5
27298 246/300 Write block 2923 len 40960 data: 3 3 3 3 3 2 2 3 5 5
27299 247/300 Read block 2682 len 4096
27300 248/300 Read block 2950 len 28672
27301 249/300 Write block 1625 len 32768 data: 4 4 4 5 4 5 7 7
27302 250/300 Read block 1921 len 36864
27303 251/300 Write block 4774 len 4096 data: 6
27304 252/300 Read block 2275 len 8192
27305 253/300 Read block 432 len 20480
27306 254/300 Write block 681 len 20480 data: 7 6 6 5 5
27307 255/300 Read block 684 len 16384
27308 256/300 Read block 4792 len 40960
27309 257/300 Write block 234 len 20480 data: 7 7 8 6 5
27310 258/300 Read block 1675 len 20480
27311 259/300 Read block 2977 len 32768
27312 260/300 Read block 582 len 12288
27313 261/300 Read block 1848 len 20480
27314 262/300 Write block 2502 len 20480 data: 3 3 3 3 3
27315 263/300 Write block 3545 len 24576 data: 4 4 3 5 5 5
27316 264/300 Flush
27317 265/300 Read block 1130 len 8192
27318 266/300 Read block 4849 len 32768
27319 267/300 Write block 4100 len 4096 data: 3
27320 268/300 Read block 2896 len 16384
27321 269/300 Write block 4333 len 40960 data: 6 6 7 5 5 6 8 7 4 3
27322 270/300 Write block 2525 len 40960 data: 7 6 9 10 10 9 10 10 7 6
27323 271/300 Read block 4434 len 4096
27324 272/300 Read block 926 len 16384
27325 273/300 Write block 2255 len 12288 data: 6 5 5
27326 274/300 Read block 2849 len 16384
27327 275/300 Read block 3641 len 8192
27328 276/300 Write block 1987 len 36864 data: 6 5 4 5 5 4 4 3 4
27329 277/300 Write block 1072 len 12288 data: 7 7 4
27330 278/300 Read block 3504 len 28672
27331 279/300 Read block 1476 len 20480
27332 280/300 Read block 1949 len 28672
27333 281/300 Read block 594 len 32768
27334 282/300 Write block 572 len 40960 data: 6 6 6 6 5 6 5 4 3 3
27335 283/300 Read block 2437 len 28672
27336 284/300 Read block 4877 len 4096
27337 285/300 Write block 2196 len 8192 data: 2 2
27338 286/300 Read block 1662 len 40960
27339 287/300 Read block 4253 len 24576
27340 288/300 Write block 3902 len 40960 data: 9 9 8 7 8 6 5 6 4 3
27341 289/300 Write block 3900 len 40960 data: 9 10 10 10 9 8 9 7 6 7
27342 290/300 Read block 2427 len 28672
27343 291/300 Write block 1229 len 4096 data: 5
27344 292/300 Write block 2302 len 24576 data: 6 5 6 5 5 7
27345 293/300 Read block 1900 len 8192
27346 294/300 Write block 741 len 36864 data: 4 4 3 3 3 2 4 4 4
27347 295/300 Flush
27348 296/300 Write block 1166 len 40960 data: 2 3 3 3 3 4 4 4 6 7
27349 297/300 Read block 3546 len 20480
27350 298/300 Write block 4427 len 40960 data: 6 5 4 4 4 4 4 4 6 6
27351 299/300 Flush
27352 300/300 Write block 4702 len 16384 data: 8 9 9 9
27353 301/300 Read block 4218 len 40960
27354 Main task has work to do, go find it
27355 got dsc Start(1)
27356 start 1
27357 Nov 15 02:03:13.307 INFO request completed, latency_us: 214, response_code: 204, uri: /start/cid/1, method: POST, req_id: a6ce31c2-0674-4b32-9432-26e67a4adbee, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
27358 Replay: started 1, returned:Ok(())
27359 ----------------------------------------------------------------
27360 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
27361 States: Active Offline Active
27362 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
27363 7340 Acked Write 3 Done Sent Done false
27364 7341 Acked Read 1 Done Sent Done false
27365 7342 Acked Write 2 Done Sent Done false
27366 7343 Acked Write 8 Done Sent Done false
27367 7344 Acked Write 4 Done Sent Done false
27368 7345 Acked Read 5 Done Sent Done false
27369 7346 Acked Read 5 Done Sent Done false
27370 7347 Acked Read 2 Done Sent Done false
27371 7348 Acked Write 5 Done Sent Done false
27372 7349 Acked Write 5 Done Sent Done false
27373 7350 Acked Flush 0 Done Sent Done false
27374 7351 Acked Write 3 Done Sent Done false
27375 7352 Acked Flush 0 Done Sent Done false
27376 7353 Acked Write 7 Done Sent Done false
27377 7354 Acked Read 2 Done Sent Done false
27378 7355 Acked Write 2 Done Sent Done false
27379 7356 Acked Write 10 Done Sent Done false
27380 7357 Acked Flush 0 Done Sent Done false
27381 7358 Acked Read 7 Done Sent Done false
27382 7359 Acked Write 10 Done Sent Done false
27383 7360 Acked Read 2 Done Sent Done false
27384 7361 Acked Write 10 Done Sent Done false
27385 7362 Acked Write 10 Done Sent Done false
27386 7363 Acked Write 1 Done Sent Done false
27387 7364 Acked Write 4 Done Sent Done false
27388 7365 Acked Write 9 Done Sent Done false
27389 7366 Acked Write 5 Done Sent Done false
27390 7367 Acked Read 8 Done Sent Done false
27391 7368 Acked Read 4 Done Sent Done false
27392 7369 Acked Write 5 Done Sent Done false
27393 7370 Acked Read 1 Done Sent Done false
27394 7371 Acked Write 7 Done Sent Done false
27395 7372 Acked Read 10 Done Sent Done false
27396 7373 Acked Flush 0 Done Sent Done false
27397 7374 Acked Write 5 Done Sent Done false
27398 7375 Acked Read 8 Done Sent Done false
27399 7376 Acked Write 8 Done Sent Done false
27400 7377 Acked Read 6 Done Sent Done false
27401 7378 Acked Read 4 Done Sent Done false
27402 7379 Acked Write 5 Done Sent Done false
27403 7380 Acked Read 10 Done Sent Done false
27404 7381 Acked Write 4 Done Sent Done false
27405 7382 Acked Flush 0 Done Sent Done false
27406 7383 Acked Write 2 Done Sent Done false
27407 7384 Acked Write 7 Done Sent Done false
27408 7385 Acked Write 1 Done Sent Done false
27409 7386 Acked Read 2 Done Sent Done false
27410 7387 Acked Flush 0 Done Sent Done false
27411 7388 Acked Flush 0 Done Sent Done false
27412 7389 Acked Write 7 Done Sent Done false
27413 7390 Acked Read 2 Done Sent Done false
27414 7391 Acked Write 1 Done Sent Done false
27415 7392 Acked Read 9 Done Sent Done false
27416 7393 Acked Read 2 Done Sent Done false
27417 7394 Acked Write 9 Done Sent Done false
27418 7395 Acked Write 1 Done Sent Done false
27419 7396 Acked Read 3 Done Sent Done false
27420 7397 Acked Read 1 Done Sent Done false
27421 7398 Acked Write 6 Done Sent Done false
27422 7399 Acked Read 4 Done Sent Done false
27423 7400 Acked Read 9 Done Sent Done false
27424 7401 Acked Read 9 Done Sent Done false
27425 7402 Acked Read 4 Done Sent Done false
27426 7403 Acked Write 5 Done Sent Done false
27427 7404 Acked Write 4 Done Sent Done false
27428 7405 Acked Read 3 Done Sent Done false
27429 7406 Acked Read 2 Done Sent Done false
27430 7407 Acked Write 2 Done Sent Done false
27431 7408 Acked Read 6 Done Sent Done false
27432 7409 Acked Read 9 Done Sent Done false
27433 7410 Acked Read 1 Done Sent Done false
27434 7411 Acked Write 3 Done Sent Done false
27435 7412 Acked Write 9 Done Sent Done false
27436 7413 Acked Write 4 Done Sent Done false
27437 7414 Acked Read 9 Done Sent Done false
27438 7415 Acked Read 2 Done Sent Done false
27439 7416 Acked Write 7 Done Sent Done false
27440 7417 Acked Read 8 Done Sent Done false
27441 7418 Acked Write 6 Done Sent Done false
27442 7419 Acked Read 10 Done Sent Done false
27443 7420 Acked Read 10 Done Sent Done false
27444 7421 Acked Read 10 Done Sent Done false
27445 7422 Acked Flush 0 Done Sent Done false
27446 7423 Acked Read 5 Done Sent Done false
27447 7424 Acked Read 1 Done Sent Done false
27448 7425 Acked Flush 0 Done Sent Done false
27449 7426 Acked Flush 0 Done Sent Done false
27450 7427 Acked Write 1 Done Sent Done false
27451 7428 Acked Read 5 Done Sent Done false
27452 7429 Acked Write 10 Done Sent Done false
27453 7430 Acked Write 10 Done Sent Done false
27454 7431 Acked Write 6 Done Sent Done false
27455 7432 Acked Write 4 Done Sent Done false
27456 7433 Acked Write 6 Done Sent Done false
27457 7434 Acked Read 6 Done Sent Done false
27458 7435 Acked Flush 0 Done Sent Done false
27459 7436 Acked Write 3 Done Sent Done false
27460 7437 Acked Read 5 Done Sent Done false
27461 7438 Acked Read 3 Done Sent Done false
27462 7439 Acked Read 8 Done Sent Done false
27463 7440 Acked Write 3 Done Sent Done false
27464 7441 Acked Write 2 Done Sent Done false
27465 7442 Acked Flush 0 Done Sent Done false
27466 7443 Acked Write 8 Done Sent Done false
27467 7444 Acked Read 4 Done Sent Done false
27468 7445 Acked Read 10 Done Sent Done false
27469 7446 Acked Read 6 Done Sent Done false
27470 7447 Acked Flush 0 Done Sent Done false
27471 7448 Acked Read 9 Done Sent Done false
27472 7449 Acked Write 5 Done Sent Done false
27473 7450 Acked Write 9 Done Sent Done false
27474 7451 Acked Read 8 Done Sent Done false
27475 7452 Acked Read 4 Done Sent Done false
27476 7453 Acked Write 5 Done Sent Done false
27477 7454 Acked Write 10 Done Sent Done false
27478 7455 Acked Write 6 Done Sent Done false
27479 7456 Acked Write 9 Done Sent Done false
27480 7457 Acked Read 2 Done Sent Done false
27481 7458 Acked Write 10 Done Sent Done false
27482 7459 Acked Read 8 Done Sent Done false
27483 7460 Acked Read 9 Done Sent Done false
27484 7461 Acked Read 3 Done Sent Done false
27485 7462 Acked Read 8 Done Sent Done false
27486 7463 Acked Read 2 Done Sent Done false
27487 7464 Acked Write 1 Done Sent Done false
27488 7465 Acked Read 5 Done Sent Done false
27489 7466 Acked Write 7 Done Sent Done false
27490 7467 Acked Read 3 Done Sent Done false
27491 7468 Acked Write 7 Done Sent Done false
27492 7469 Acked Read 1 Done Sent Done false
27493 7470 Acked Read 4 Done Sent Done false
27494 7471 Acked Write 9 Done Sent Done false
27495 7472 Acked Read 1 Done Sent Done false
27496 7473 Acked Read 5 Done Sent Done false
27497 7474 Acked Read 7 Done Sent Done false
27498 7475 Acked Write 6 Done Sent Done false
27499 7476 Acked Read 9 Done Sent Done false
27500 7477 Acked Read 2 Done Sent Done false
27501 7478 Acked Read 6 Done Sent Done false
27502 7479 Acked Flush 0 Done Sent Done false
27503 7480 Acked Write 7 Done Sent Done false
27504 7481 Acked Read 8 Done Sent Done false
27505 7482 Acked Flush 0 Done Sent Done false
27506 7483 Acked Read 5 Done Sent Done false
27507 7484 Acked Read 5 Done Sent Done false
27508 7485 Acked Write 2 Done Sent Done false
27509 7486 Acked Read 7 Done Sent Done false
27510 7487 Acked Read 7 Done Sent Done false
27511 7488 Acked Write 6 Done Sent Done false
27512 7489 Acked Read 10 Done Sent Done false
27513 7490 Acked Write 4 Done Sent Done false
27514 7491 Acked Read 3 Done Sent Done false
27515 7492 Acked Write 9 Done Sent Done false
27516 7493 Acked Read 1 Done Sent Done false
27517 7494 Acked Flush 0 Done Sent Done false
27518 7495 Acked Write 7 Done Sent Done false
27519 7496 Acked Read 4 Done Sent Done false
27520 7497 Acked Write 6 Done Sent Done false
27521 7498 Acked Write 6 Done Sent Done false
27522 7499 Acked Flush 0 Done Sent Done false
27523 7500 Acked Write 7 Done Sent Done false
27524 7501 Acked Read 5 Done Sent Done false
27525 7502 Acked Read 4 Done Sent Done false
27526 7503 Acked Write 1 Done Sent Done false
27527 7504 Acked Write 7 Done Sent Done false
27528 7505 Acked Read 7 Done Sent Done false
27529 7506 Acked Write 8 Done Sent Done false
27530 7507 Acked Flush 0 Done Sent Done false
27531 7508 Acked Read 6 Done Sent Done false
27532 7509 Acked Read 2 Done Sent Done false
27533 7510 Acked Read 4 Done Sent Done false
27534 7511 Acked Read 3 Done Sent Done false
27535 7512 Acked Write 1 Done Sent Done false
27536 7513 Acked Read 3 Done Sent Done false
27537 7514 Acked Write 4 Done Sent Done false
27538 7515 Acked Flush 0 Done Sent Done false
27539 7516 Acked Read 5 Done Sent Done false
27540 7517 Acked Read 7 Done Sent Done false
27541 7518 Acked Write 2 Done Sent Done false
27542 7519 Acked Flush 0 Done Sent Done false
27543 7520 Acked Read 10 Done Sent Done false
27544 7521 Acked Read 8 Done Sent Done false
27545 7522 Acked Read 4 Done Sent Done false
27546 7523 Acked Read 8 Done Sent Done false
27547 7524 Acked Flush 0 Done Sent Done false
27548 7525 Acked Write 5 Done Sent Done false
27549 7526 Acked Flush 0 Done Sent Done false
27550 7527 Acked Flush 0 Done Sent Done false
27551 7528 Acked Flush 0 Done Sent Done false
27552 7529 Acked Write 5 Done Sent Done false
27553 7530 Acked Flush 0 Done Sent Done false
27554 7531 Acked Write 5 Done Sent Done false
27555 7532 Acked Read 2 Done Sent Done false
27556 7533 Acked Read 9 Done Sent Done false
27557 7534 Acked Read 9 Done Sent Done false
27558 7535 Acked Read 6 Done Sent Done false
27559 7536 Acked Write 3 Done Sent Done false
27560 7537 Acked Write 7 Done Sent Done false
27561 7538 Acked Read 6 Done Sent Done false
27562 7539 Acked Flush 0 Done Sent Done false
27563 7540 Acked Flush 0 Done Sent Done false
27564 7541 Acked Read 7 Done Sent Done false
27565 7542 Acked Write 4 Done Sent Done false
27566 7543 Acked Read 10 Done Sent Done false
27567 7544 Acked Read 10 Done Sent Done false
27568 7545 Acked Write 1 Done Sent Done false
27569 7546 Acked Read 8 Done Sent Done false
27570 7547 Acked Flush 0 Done Sent Done false
27571 7548 Acked Write 9 Done Sent Done false
27572 7549 Acked Write 1 Done Sent Done false
27573 7550 Acked Write 1 Done Sent Done false
27574 7551 Acked Flush 0 Done Sent Done false
27575 7552 Acked Write 4 Done Sent Done false
27576 7553 Acked Read 1 Done Sent Done false
27577 7554 Acked Flush 0 Done Sent Done false
27578 7555 Acked Read 1 Done Sent Done false
27579 7556 Acked Read 2 Done Sent Done false
27580 7557 Acked Write 6 Done Sent Done false
27581 7558 Acked Write 7 Done Sent Done false
27582 7559 Acked Write 4 Done Sent Done false
27583 7560 Acked Flush 0 Done Sent Done false
27584 7561 Acked Write 7 Done Sent Done false
27585 7562 Acked Write 10 Done Sent Done false
27586 7563 Acked Write 7 Done Sent Done false
27587 7564 Acked Flush 0 Done Sent Done false
27588 7565 Acked Write 9 Done Sent Done false
27589 7566 Acked Write 3 Done Sent Done false
27590 7567 Acked Flush 0 Done Sent Done false
27591 7568 Acked Read 1 Done Sent Done false
27592 7569 Acked Read 3 Done Sent Done false
27593 7570 Acked Write 10 Done Sent Done false
27594 7571 Acked Write 3 Done Sent Done false
27595 7572 Acked Read 5 Done Sent Done false
27596 7573 Acked Read 8 Done Sent Done false
27597 7574 Acked Read 7 Done Sent Done false
27598 7575 Acked Write 4 Done Sent Done false
27599 7576 Acked Read 8 Done Sent Done false
27600 7577 Acked Read 5 Done Sent Done false
27601 7578 Acked Flush 0 Done Sent Done false
27602 7579 Acked Read 5 Done Sent Done false
27603 7580 Acked Flush 0 Done Sent Done false
27604 7581 Acked Read 5 Done Sent Done false
27605 7582 Acked Read 5 Done Sent Done false
27606 7583 Acked Read 3 Done Sent Done false
27607 7584 Acked Write 9 Done Sent Done false
27608 7585 Acked Write 10 Done Sent Done false
27609 7586 Acked Read 1 Done Sent Done false
27610 7587 Acked Read 7 Done Sent Done false
27611 7588 Acked Write 8 Done Sent Done false
27612 7589 Acked Read 9 Done Sent Done false
27613 7590 Acked Write 1 Done Sent Done false
27614 7591 Acked Read 2 Done Sent Done false
27615 7592 Acked Read 5 Done Sent Done false
27616 7593 Acked Write 5 Done Sent Done false
27617 7594 Acked Read 4 Done Sent Done false
27618 7595 Acked Read 10 Done Sent Done false
27619 7596 Acked Write 5 Done Sent Done false
27620 7597 Acked Read 5 Done Sent Done false
27621 7598 Acked Read 8 Done Sent Done false
27622 7599 Acked Read 3 Done Sent Done false
27623 7600 Acked Read 5 Done Sent Done false
27624 7601 Acked Write 5 Done Sent Done false
27625 7602 Acked Write 6 Done Sent Done false
27626 7603 Acked Flush 0 Done Sent Done false
27627 7604 Acked Read 2 Done Sent Done false
27628 7605 Acked Read 8 Done Sent Done false
27629 7606 Acked Write 1 Done Sent Done false
27630 7607 Acked Read 4 Done Sent Done false
27631 7608 Acked Write 10 Done Sent Done false
27632 7609 Acked Write 10 Done Sent Done false
27633 7610 Acked Read 1 Done Sent Done false
27634 7611 Acked Read 4 Done Sent Done false
27635 7612 Acked Write 3 Done Sent Done false
27636 7613 Acked Read 4 Done Sent Done false
27637 7614 Acked Read 2 Done Sent Done false
27638 7615 Acked Write 9 Done Sent Done false
27639 7616 Acked Write 3 Done Sent Done false
27640 7617 Acked Read 7 Done Sent Done false
27641 7618 Acked Read 5 Done Sent Done false
27642 7619 Acked Read 7 Done Sent Done false
27643 7620 Acked Read 8 Done Sent Done false
27644 7621 Acked Write 10 Done Sent Done false
27645 7622 Acked Read 7 Done Sent Done false
27646 7623 Acked Read 1 Done Sent Done false
27647 7624 Acked Write 2 Done Sent Done false
27648 7625 Acked Read 10 Done Sent Done false
27649 7626 Acked Read 6 Done Sent Done false
27650 7627 Acked Write 10 Done Sent Done false
27651 7628 Acked Write 10 Done Sent Done false
27652 7629 Acked Read 7 Done Sent Done false
27653 7630 Acked Write 1 Done Sent Done false
27654 7631 Acked Write 6 Done Sent Done false
27655 7632 Acked Read 2 Done Sent Done false
27656 7633 Acked Write 9 Done Sent Done false
27657 7634 Acked Flush 0 Done Sent Done false
27658 7635 Acked Write 10 Done Sent Done false
27659 7636 Acked Read 5 Done Sent Done false
27660 7637 Acked Write 10 Done Sent Done false
27661 7638 Acked Flush 0 Done Sent Done false
27662 7639 Acked Write 4 Done Sent Done false
27663 7640 Acked Read 10 Done Sent Done false
27664 STATES DS:0 DS:1 DS:2 TOTAL
27665 Sent 0 301 0 301
27666 Done 301 0 301 602
27667 Skipped 0 0 0 0
27668 Error 0 0 0 0
27669 Last Flush: 7638 7339 7638
27670 Downstairs last five completed: 7339 7338 7337 7336 7335
27671 Upstairs last five completed: 7640 7639 7638 7637 7636
27672 CLIENT: Up:0 ds:301 act:2
27673 [8820] got start action
27674 [8820] I am going to restart
27675 Starting downstairs at port 8820
27676 Make output file at "/tmp/dsc/downstairs-8820.txt"
27677 [8820][1] reports Starting
27678 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1227)
27679 [8820][1] reports Running
27680 ----------------------------------------------------------------
27681 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
27682 States: Active Offline Active
27683 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
27684 7340 Acked Write 3 Done Sent Done false
27685 7341 Acked Read 1 Done Sent Done false
27686 7342 Acked Write 2 Done Sent Done false
27687 7343 Acked Write 8 Done Sent Done false
27688 7344 Acked Write 4 Done Sent Done false
27689 7345 Acked Read 5 Done Sent Done false
27690 7346 Acked Read 5 Done Sent Done false
27691 7347 Acked Read 2 Done Sent Done false
27692 7348 Acked Write 5 Done Sent Done false
27693 7349 Acked Write 5 Done Sent Done false
27694 7350 Acked Flush 0 Done Sent Done false
27695 7351 Acked Write 3 Done Sent Done false
27696 7352 Acked Flush 0 Done Sent Done false
27697 7353 Acked Write 7 Done Sent Done false
27698 7354 Acked Read 2 Done Sent Done false
27699 7355 Acked Write 2 Done Sent Done false
27700 7356 Acked Write 10 Done Sent Done false
27701 7357 Acked Flush 0 Done Sent Done false
27702 7358 Acked Read 7 Done Sent Done false
27703 7359 Acked Write 10 Done Sent Done false
27704 7360 Acked Read 2 Done Sent Done false
27705 7361 Acked Write 10 Done Sent Done false
27706 7362 Acked Write 10 Done Sent Done false
27707 7363 Acked Write 1 Done Sent Done false
27708 7364 Acked Write 4 Done Sent Done false
27709 7365 Acked Write 9 Done Sent Done false
27710 7366 Acked Write 5 Done Sent Done false
27711 7367 Acked Read 8 Done Sent Done false
27712 7368 Acked Read 4 Done Sent Done false
27713 7369 Acked Write 5 Done Sent Done false
27714 7370 Acked Read 1 Done Sent Done false
27715 7371 Acked Write 7 Done Sent Done false
27716 7372 Acked Read 10 Done Sent Done false
27717 7373 Acked Flush 0 Done Sent Done false
27718 7374 Acked Write 5 Done Sent Done false
27719 7375 Acked Read 8 Done Sent Done false
27720 7376 Acked Write 8 Done Sent Done false
27721 7377 Acked Read 6 Done Sent Done false
27722 7378 Acked Read 4 Done Sent Done false
27723 7379 Acked Write 5 Done Sent Done false
27724 7380 Acked Read 10 Done Sent Done false
27725 7381 Acked Write 4 Done Sent Done false
27726 7382 Acked Flush 0 Done Sent Done false
27727 7383 Acked Write 2 Done Sent Done false
27728 7384 Acked Write 7 Done Sent Done false
27729 7385 Acked Write 1 Done Sent Done false
27730 7386 Acked Read 2 Done Sent Done false
27731 7387 Acked Flush 0 Done Sent Done false
27732 7388 Acked Flush 0 Done Sent Done false
27733 7389 Acked Write 7 Done Sent Done false
27734 7390 Acked Read 2 Done Sent Done false
27735 7391 Acked Write 1 Done Sent Done false
27736 7392 Acked Read 9 Done Sent Done false
27737 7393 Acked Read 2 Done Sent Done false
27738 7394 Acked Write 9 Done Sent Done false
27739 7395 Acked Write 1 Done Sent Done false
27740 7396 Acked Read 3 Done Sent Done false
27741 7397 Acked Read 1 Done Sent Done false
27742 7398 Acked Write 6 Done Sent Done false
27743 7399 Acked Read 4 Done Sent Done false
27744 7400 Acked Read 9 Done Sent Done false
27745 7401 Acked Read 9 Done Sent Done false
27746 7402 Acked Read 4 Done Sent Done false
27747 7403 Acked Write 5 Done Sent Done false
27748 7404 Acked Write 4 Done Sent Done false
27749 7405 Acked Read 3 Done Sent Done false
27750 7406 Acked Read 2 Done Sent Done false
27751 7407 Acked Write 2 Done Sent Done false
27752 7408 Acked Read 6 Done Sent Done false
27753 7409 Acked Read 9 Done Sent Done false
27754 7410 Acked Read 1 Done Sent Done false
27755 7411 Acked Write 3 Done Sent Done false
27756 7412 Acked Write 9 Done Sent Done false
27757 7413 Acked Write 4 Done Sent Done false
27758 7414 Acked Read 9 Done Sent Done false
27759 7415 Acked Read 2 Done Sent Done false
27760 7416 Acked Write 7 Done Sent Done false
27761 7417 Acked Read 8 Done Sent Done false
27762 7418 Acked Write 6 Done Sent Done false
27763 7419 Acked Read 10 Done Sent Done false
27764 7420 Acked Read 10 Done Sent Done false
27765 7421 Acked Read 10 Done Sent Done false
27766 7422 Acked Flush 0 Done Sent Done false
27767 7423 Acked Read 5 Done Sent Done false
27768 7424 Acked Read 1 Done Sent Done false
27769 7425 Acked Flush 0 Done Sent Done false
27770 7426 Acked Flush 0 Done Sent Done false
27771 7427 Acked Write 1 Done Sent Done false
27772 7428 Acked Read 5 Done Sent Done false
27773 7429 Acked Write 10 Done Sent Done false
27774 7430 Acked Write 10 Done Sent Done false
27775 7431 Acked Write 6 Done Sent Done false
27776 7432 Acked Write 4 Done Sent Done false
27777 7433 Acked Write 6 Done Sent Done false
27778 7434 Acked Read 6 Done Sent Done false
27779 7435 Acked Flush 0 Done Sent Done false
27780 7436 Acked Write 3 Done Sent Done false
27781 7437 Acked Read 5 Done Sent Done false
27782 7438 Acked Read 3 Done Sent Done false
27783 7439 Acked Read 8 Done Sent Done false
27784 7440 Acked Write 3 Done Sent Done false
27785 7441 Acked Write 2 Done Sent Done false
27786 7442 Acked Flush 0 Done Sent Done false
27787 7443 Acked Write 8 Done Sent Done false
27788 7444 Acked Read 4 Done Sent Done false
27789 7445 Acked Read 10 Done Sent Done false
27790 7446 Acked Read 6 Done Sent Done false
27791 7447 Acked Flush 0 Done Sent Done false
27792 7448 Acked Read 9 Done Sent Done false
27793 7449 Acked Write 5 Done Sent Done false
27794 7450 Acked Write 9 Done Sent Done false
27795 7451 Acked Read 8 Done Sent Done false
27796 7452 Acked Read 4 Done Sent Done false
27797 7453 Acked Write 5 Done Sent Done false
27798 7454 Acked Write 10 Done Sent Done false
27799 7455 Acked Write 6 Done Sent Done false
27800 7456 Acked Write 9 Done Sent Done false
27801 7457 Acked Read 2 Done Sent Done false
27802 7458 Acked Write 10 Done Sent Done false
27803 7459 Acked Read 8 Done Sent Done false
27804 7460 Acked Read 9 Done Sent Done false
27805 7461 Acked Read 3 Done Sent Done false
27806 7462 Acked Read 8 Done Sent Done false
27807 7463 Acked Read 2 Done Sent Done false
27808 7464 Acked Write 1 Done Sent Done false
27809 7465 Acked Read 5 Done Sent Done false
27810 7466 Acked Write 7 Done Sent Done false
27811 7467 Acked Read 3 Done Sent Done false
27812 7468 Acked Write 7 Done Sent Done false
27813 7469 Acked Read 1 Done Sent Done false
27814 7470 Acked Read 4 Done Sent Done false
27815 7471 Acked Write 9 Done Sent Done false
27816 7472 Acked Read 1 Done Sent Done false
27817 7473 Acked Read 5 Done Sent Done false
27818 7474 Acked Read 7 Done Sent Done false
27819 7475 Acked Write 6 Done Sent Done false
27820 7476 Acked Read 9 Done Sent Done false
27821 7477 Acked Read 2 Done Sent Done false
27822 7478 Acked Read 6 Done Sent Done false
27823 7479 Acked Flush 0 Done Sent Done false
27824 7480 Acked Write 7 Done Sent Done false
27825 7481 Acked Read 8 Done Sent Done false
27826 7482 Acked Flush 0 Done Sent Done false
27827 7483 Acked Read 5 Done Sent Done false
27828 7484 Acked Read 5 Done Sent Done false
27829 7485 Acked Write 2 Done Sent Done false
27830 7486 Acked Read 7 Done Sent Done false
27831 7487 Acked Read 7 Done Sent Done false
27832 7488 Acked Write 6 Done Sent Done false
27833 7489 Acked Read 10 Done Sent Done false
27834 7490 Acked Write 4 Done Sent Done false
27835 7491 Acked Read 3 Done Sent Done false
27836 7492 Acked Write 9 Done Sent Done false
27837 7493 Acked Read 1 Done Sent Done false
27838 7494 Acked Flush 0 Done Sent Done false
27839 7495 Acked Write 7 Done Sent Done false
27840 7496 Acked Read 4 Done Sent Done false
27841 7497 Acked Write 6 Done Sent Done false
27842 7498 Acked Write 6 Done Sent Done false
27843 7499 Acked Flush 0 Done Sent Done false
27844 7500 Acked Write 7 Done Sent Done false
27845 7501 Acked Read 5 Done Sent Done false
27846 7502 Acked Read 4 Done Sent Done false
27847 7503 Acked Write 1 Done Sent Done false
27848 7504 Acked Write 7 Done Sent Done false
27849 7505 Acked Read 7 Done Sent Done false
27850 7506 Acked Write 8 Done Sent Done false
27851 7507 Acked Flush 0 Done Sent Done false
27852 7508 Acked Read 6 Done Sent Done false
27853 7509 Acked Read 2 Done Sent Done false
27854 7510 Acked Read 4 Done Sent Done false
27855 7511 Acked Read 3 Done Sent Done false
27856 7512 Acked Write 1 Done Sent Done false
27857 7513 Acked Read 3 Done Sent Done false
27858 7514 Acked Write 4 Done Sent Done false
27859 7515 Acked Flush 0 Done Sent Done false
27860 7516 Acked Read 5 Done Sent Done false
27861 7517 Acked Read 7 Done Sent Done false
27862 7518 Acked Write 2 Done Sent Done false
27863 7519 Acked Flush 0 Done Sent Done false
27864 7520 Acked Read 10 Done Sent Done false
27865 7521 Acked Read 8 Done Sent Done false
27866 7522 Acked Read 4 Done Sent Done false
27867 7523 Acked Read 8 Done Sent Done false
27868 7524 Acked Flush 0 Done Sent Done false
27869 7525 Acked Write 5 Done Sent Done false
27870 7526 Acked Flush 0 Done Sent Done false
27871 7527 Acked Flush 0 Done Sent Done false
27872 7528 Acked Flush 0 Done Sent Done false
27873 7529 Acked Write 5 Done Sent Done false
27874 7530 Acked Flush 0 Done Sent Done false
27875 7531 Acked Write 5 Done Sent Done false
27876 7532 Acked Read 2 Done Sent Done false
27877 7533 Acked Read 9 Done Sent Done false
27878 7534 Acked Read 9 Done Sent Done false
27879 7535 Acked Read 6 Done Sent Done false
27880 7536 Acked Write 3 Done Sent Done false
27881 7537 Acked Write 7 Done Sent Done false
27882 7538 Acked Read 6 Done Sent Done false
27883 7539 Acked Flush 0 Done Sent Done false
27884 7540 Acked Flush 0 Done Sent Done false
27885 7541 Acked Read 7 Done Sent Done false
27886 7542 Acked Write 4 Done Sent Done false
27887 7543 Acked Read 10 Done Sent Done false
27888 7544 Acked Read 10 Done Sent Done false
27889 7545 Acked Write 1 Done Sent Done false
27890 7546 Acked Read 8 Done Sent Done false
27891 7547 Acked Flush 0 Done Sent Done false
27892 7548 Acked Write 9 Done Sent Done false
27893 7549 Acked Write 1 Done Sent Done false
27894 7550 Acked Write 1 Done Sent Done false
27895 7551 Acked Flush 0 Done Sent Done false
27896 7552 Acked Write 4 Done Sent Done false
27897 7553 Acked Read 1 Done Sent Done false
27898 7554 Acked Flush 0 Done Sent Done false
27899 7555 Acked Read 1 Done Sent Done false
27900 7556 Acked Read 2 Done Sent Done false
27901 7557 Acked Write 6 Done Sent Done false
27902 7558 Acked Write 7 Done Sent Done false
27903 7559 Acked Write 4 Done Sent Done false
27904 7560 Acked Flush 0 Done Sent Done false
27905 7561 Acked Write 7 Done Sent Done false
27906 7562 Acked Write 10 Done Sent Done false
27907 7563 Acked Write 7 Done Sent Done false
27908 7564 Acked Flush 0 Done Sent Done false
27909 7565 Acked Write 9 Done Sent Done false
27910 7566 Acked Write 3 Done Sent Done false
27911 7567 Acked Flush 0 Done Sent Done false
27912 7568 Acked Read 1 Done Sent Done false
27913 7569 Acked Read 3 Done Sent Done false
27914 7570 Acked Write 10 Done Sent Done false
27915 7571 Acked Write 3 Done Sent Done false
27916 7572 Acked Read 5 Done Sent Done false
27917 7573 Acked Read 8 Done Sent Done false
27918 7574 Acked Read 7 Done Sent Done false
27919 7575 Acked Write 4 Done Sent Done false
27920 7576 Acked Read 8 Done Sent Done false
27921 7577 Acked Read 5 Done Sent Done false
27922 7578 Acked Flush 0 Done Sent Done false
27923 7579 Acked Read 5 Done Sent Done false
27924 7580 Acked Flush 0 Done Sent Done false
27925 7581 Acked Read 5 Done Sent Done false
27926 7582 Acked Read 5 Done Sent Done false
27927 7583 Acked Read 3 Done Sent Done false
27928 7584 Acked Write 9 Done Sent Done false
27929 7585 Acked Write 10 Done Sent Done false
27930 7586 Acked Read 1 Done Sent Done false
27931 7587 Acked Read 7 Done Sent Done false
27932 7588 Acked Write 8 Done Sent Done false
27933 7589 Acked Read 9 Done Sent Done false
27934 7590 Acked Write 1 Done Sent Done false
27935 7591 Acked Read 2 Done Sent Done false
27936 7592 Acked Read 5 Done Sent Done false
27937 7593 Acked Write 5 Done Sent Done false
27938 7594 Acked Read 4 Done Sent Done false
27939 7595 Acked Read 10 Done Sent Done false
27940 7596 Acked Write 5 Done Sent Done false
27941 7597 Acked Read 5 Done Sent Done false
27942 7598 Acked Read 8 Done Sent Done false
27943 7599 Acked Read 3 Done Sent Done false
27944 7600 Acked Read 5 Done Sent Done false
27945 7601 Acked Write 5 Done Sent Done false
27946 7602 Acked Write 6 Done Sent Done false
27947 7603 Acked Flush 0 Done Sent Done false
27948 7604 Acked Read 2 Done Sent Done false
27949 7605 Acked Read 8 Done Sent Done false
27950 7606 Acked Write 1 Done Sent Done false
27951 7607 Acked Read 4 Done Sent Done false
27952 7608 Acked Write 10 Done Sent Done false
27953 7609 Acked Write 10 Done Sent Done false
27954 7610 Acked Read 1 Done Sent Done false
27955 7611 Acked Read 4 Done Sent Done false
27956 7612 Acked Write 3 Done Sent Done false
27957 7613 Acked Read 4 Done Sent Done false
27958 7614 Acked Read 2 Done Sent Done false
27959 7615 Acked Write 9 Done Sent Done false
27960 7616 Acked Write 3 Done Sent Done false
27961 7617 Acked Read 7 Done Sent Done false
27962 7618 Acked Read 5 Done Sent Done false
27963 7619 Acked Read 7 Done Sent Done false
27964 7620 Acked Read 8 Done Sent Done false
27965 7621 Acked Write 10 Done Sent Done false
27966 7622 Acked Read 7 Done Sent Done false
27967 7623 Acked Read 1 Done Sent Done false
27968 7624 Acked Write 2 Done Sent Done false
27969 7625 Acked Read 10 Done Sent Done false
27970 7626 Acked Read 6 Done Sent Done false
27971 7627 Acked Write 10 Done Sent Done false
27972 7628 Acked Write 10 Done Sent Done false
27973 7629 Acked Read 7 Done Sent Done false
27974 7630 Acked Write 1 Done Sent Done false
27975 7631 Acked Write 6 Done Sent Done false
27976 7632 Acked Read 2 Done Sent Done false
27977 7633 Acked Write 9 Done Sent Done false
27978 7634 Acked Flush 0 Done Sent Done false
27979 7635 Acked Write 10 Done Sent Done false
27980 7636 Acked Read 5 Done Sent Done false
27981 7637 Acked Write 10 Done Sent Done false
27982 7638 Acked Flush 0 Done Sent Done false
27983 7639 Acked Write 4 Done Sent Done false
27984 7640 Acked Read 10 Done Sent Done false
27985 7641 Acked Flush 0 Done Sent Done false
27986 STATES DS:0 DS:1 DS:2 TOTAL
27987 Sent 0 302 0 302
27988 Done 302 0 302 604
27989 Skipped 0 0 0 0
27990 Error 0 0 0 0
27991 Last Flush: 7641 7339 7641
27992 Downstairs last five completed: 7339 7338 7337 7336 7335
27993 Upstairs last five completed: 7641 7640 7639 7638 7637
27994 CLIENT: Up:0 ds:302 act:2
279952024-11-15T02:03:19.074ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
279962024-11-15T02:03:19.074ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:60311) = io task client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
279972024-11-15T02:03:19.075ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
279982024-11-15T02:03:19.076ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
279992024-11-15T02:03:19.076ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
280002024-11-15T02:03:19.076ZINFOcrucible: send last flush ID to this DS: 7339 = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
280012024-11-15T02:03:19.076ZINFOcrucible: Replied this last flush ID: 7339 = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
280022024-11-15T02:03:19.076ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
280032024-11-15T02:03:19.076ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
280042024-11-15T02:03:19.076ZINFOcrucible: [1] client re-new 302 jobs since flush 7339 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
280052024-11-15T02:03:19.076ZINFOcrucible: [1] Replayed 302 jobs since flush: 7339 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
28006 ----------------------------------------------------------------
28007 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
28008 Downstairs last five completed: 7641 7640 7639 7638 7637
28009 Upstairs last five completed: 7641 7640 7639 7638 7637
28010 CLIENT: Up:0 ds:0 act:3
28011 Replay: All jobs finished
28012 Main task has work to do, go find it
28013 got dsc Stop(0)
28014 stop 0
28015 Nov 15 02:03:21.312[8810] Got stop action so:false kr:false
28016 INFO request completed, latency_us: 345, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 6e193569-4fb7-4b34-8d21-b7badc61bc2d, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
28017 Nov 15 02:03:21.313 INFO request completed, latency_us: 137, response_code: 200, uri: /state/cid/0, method: GET, req_id: 30eeb1a9-8299-4c91-bf5c-cc64387fe44f, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
280182024-11-15T02:03:21.313ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
280192024-11-15T02:03:21.313ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
280202024-11-15T02:03:21.313ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
280212024-11-15T02:03:21.313ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
280222024-11-15T02:03:21.313ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
280232024-11-15T02:03:21.313ZINFOcrucible: client task is exiting = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
28024 [8810] Exited with: ExitStatus(unix_wait_status(9))
28025 [8810][0] reports Exit
28026 Nov 15 02:03:25.315 INFO request completed, latency_us: 213, response_code: 200, uri: /state/cid/0, method: GET, req_id: 784f1321-3cb9-4d91-82f0-27da609f5e09, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
28027 001/300 Read block 3374 len 24576
28028 002/300 Flush
28029 003/300 Read block 1588 len 12288
28030 004/300 Read block 32 len 20480
28031 005/300 Flush
28032 006/300 Read block 4066 len 36864
28033 007/300 Write block 2700 len 24576 data: 5 3 3 3 2 2
28034 008/300 Write block 2381 len 24576 data: 4 4 6 7 7 6
28035 009/300 Read block 2529 len 40960
28036 010/300 Read block 378 len 20480
28037 011/300 Write block 1702 len 8192 data: 6 5
28038 012/300 Flush
28039 013/300 Read block 1398 len 36864
28040 014/300 Read block 3507 len 24576
28041 015/300 Read block 3577 len 24576
28042 016/300 Flush
28043 017/300 Read block 2615 len 4096
28044 018/300 Flush
28045 019/300 Read block 3889 len 4096
28046 020/300 Write block 461 len 16384 data: 9 9 8 7
28047 021/300 Read block 4630 len 40960
28048 022/300 Read block 2811 len 4096
28049 023/300 Write block 4973 len 24576 data: 2 2 2 3 3 3
28050 024/300 Flush
28051 025/300 Write block 3971 len 28672 data: 5 5 5 5 4 5 5
28052 026/300 Flush
28053 027/300 Read block 3962 len 8192
28054 028/300 Read block 2763 len 24576
28055 029/300 Read block 3355 len 36864
28056 030/300 Read block 3414 len 36864
28057 031/300 Write block 4942 len 32768 data: 4 4 4 4 4 4 4 5
28058 032/300 Read block 2790 len 12288
28059 033/300 Read block 1340 len 40960
28060 034/300 Write block 1800 len 16384 data: 4 3 4 4
28061 035/300 Read block 2015 len 8192
28062 036/300 Read block 298 len 4096
28063 037/300 Write block 1931 len 24576 data: 5 7 7 8 5 5
28064 038/300 Read block 3487 len 40960
28065 039/300 Read block 3435 len 20480
28066 040/300 Read block 4767 len 8192
28067 041/300 Read block 2329 len 8192
28068 042/300 Read block 3268 len 20480
28069 043/300 Read block 4747 len 28672
28070 044/300 Write block 1357 len 8192 data: 7 8
28071 045/300 Write block 1560 len 20480 data: 6 7 8 8 7
28072 046/300 Read block 4592 len 28672
28073 047/300 Write block 4434 len 12288 data: 5 7 7
28074 048/300 Read block 4661 len 24576
28075 049/300 Read block 2655 len 32768
28076 050/300 Write block 1295 len 28672 data: 6 6 7 6 7 5 5
28077 051/300 Read block 1509 len 36864
28078 052/300 Write block 4773 len 16384 data: 4 7 6 8
28079 053/300 Read block 4361 len 4096
28080 054/300 Read block 3001 len 20480
28081 055/300 Read block 3786 len 32768
28082 056/300 Read block 3955 len 8192
28083 057/300 Write block 1124 len 12288 data: 5 5 4
28084 058/300 Flush
28085 059/300 Read block 1284 len 36864
28086 060/300 Flush
28087 061/300 Flush
28088 062/300 Write block 3973 len 40960 data: 6 6 5 6 6 6 6 6 5 3
28089 063/300 Read block 1653 len 20480
28090 064/300 Read block 2742 len 32768
28091 065/300 Write block 1600 len 20480 data: 5 4 5 5 4
28092 066/300 Read block 4505 len 40960
28093 067/300 Read block 937 len 4096
28094 068/300 Write block 1073 len 20480 data: 8 5 5 4 4
28095 069/300 Read block 4563 len 36864
28096 070/300 Read block 476 len 24576
28097 071/300 Write block 2096 len 12288 data: 5 6 5
28098 072/300 Read block 4477 len 24576
28099 073/300 Write block 3564 len 20480 data: 3 4 4 4 5
28100 074/300 Read block 428 len 36864
28101 075/300 Read block 3863 len 20480
28102 076/300 Read block 689 len 40960
28103 077/300 Read block 538 len 32768
28104 078/300 Write block 3340 len 12288 data: 6 4 3
28105 079/300 Read block 4553 len 36864
28106 080/300 Write block 1274 len 28672 data: 4 5 6 6 6 5 6
28107 081/300 Write block 3088 len 12288 data: 4 4 4
28108 082/300 Write block 1857 len 16384 data: 4 4 4 4
28109 083/300 Read block 1917 len 8192
28110 084/300 Read block 355 len 32768
28111 085/300 Read block 4226 len 16384
28112 086/300 Read block 4405 len 4096
28113 087/300 Write block 64 len 12288 data: 4 5 6
28114 088/300 Flush
28115 089/300 Read block 3036 len 36864
28116 090/300 Read block 1047 len 28672
28117 091/300 Write block 4336 len 16384 data: 6 6 7 9
28118 092/300 Write block 1275 len 40960 data: 6 7 7 7 6 7 6 7 7 5
28119 093/300 Write block 1301 len 40960 data: 6 3 5 4 3 3 3 4 4 4
28120 094/300 Write block 1266 len 36864 data: 3 5 5 6 4 5 5 6 5
28121 095/300 Read block 130 len 4096
28122 096/300 Read block 2956 len 32768
28123 097/300 Read block 1252 len 28672
28124 098/300 Write block 4557 len 40960 data: 4 4 4 4 4 3 4 4 4 4
28125 099/300 Write block 4933 len 32768 data: 2 2 2 2 2 2 3 4
28126 100/300 Write block 1580 len 40960 data: 2 3 3 4 4 4 4 4 5 4
28127 101/300 Read block 1445 len 40960
28128 102/300 Write block 3108 len 40960 data: 5 4 5 4 5 4 3 3 3 4
28129 103/300 Flush
28130 104/300 Flush
28131 105/300 Write block 1965 len 4096 data: 2
28132 106/300 Read block 1503 len 8192
28133 107/300 Read block 1628 len 24576
28134 108/300 Flush
28135 109/300 Read block 4311 len 16384
28136 110/300 Write block 1390 len 8192 data: 2 3
28137 111/300 Read block 459 len 12288
28138 112/300 Write block 3927 len 32768 data: 6 5 4 4 4 5 4 4
28139 113/300 Read block 4892 len 24576
28140 114/300 Write block 2618 len 28672 data: 6 5 6 5 5 4 3
28141 115/300 Read block 4954 len 24576
28142 116/300 Write block 3385 len 40960 data: 5 5 3 3 2 2 3 6 6 5
28143 117/300 Read block 2589 len 36864
28144 118/300 Read block 2350 len 24576
28145 119/300 Read block 4706 len 8192
28146 120/300 Flush
28147 121/300 Read block 1187 len 40960
28148 122/300 Flush
28149 123/300 Write block 2087 len 16384 data: 4 4 4 3
28150 124/300 Read block 1393 len 20480
28151 125/300 Read block 1416 len 32768
28152 126/300 Flush
28153 127/300 Flush
28154 128/300 Flush
28155 129/300 Write block 2926 len 4096 data: 4
28156 130/300 Read block 3336 len 40960
28157 131/300 Read block 803 len 32768
28158 132/300 Read block 3141 len 20480
28159 133/300 Write block 3962 len 12288 data: 7 6 6
28160 134/300 Read block 3337 len 8192
28161 135/300 Write block 2566 len 24576 data: 9 7 4 5 5 4
28162 136/300 Read block 1285 len 36864
28163 137/300 Read block 2115 len 4096
28164 138/300 Read block 1974 len 8192
28165 139/300 Write block 3632 len 32768 data: 5 3 3 3 3 3 3 5
28166 140/300 Write block 4040 len 4096 data: 10
28167 141/300 Write block 224 len 8192 data: 3 4
28168 142/300 Read block 3463 len 12288
28169 143/300 Read block 188 len 4096
28170 144/300 Write block 2670 len 40960 data: 3 3 2 3 3 3 3 4 5 5
28171 145/300 Write block 3846 len 36864 data: 3 4 3 4 6 6 6 6 6
28172 146/300 Read block 2393 len 32768
28173 147/300 Write block 689 len 40960 data: 3 4 4 4 5 4 3 5 5 5
28174 148/300 Flush
28175 149/300 Read block 474 len 32768
28176 150/300 Read block 3948 len 16384
28177 151/300 Read block 4435 len 24576
28178 152/300 Write block 1090 len 12288 data: 3 3 3
28179 153/300 Write block 326 len 8192 data: 8 9
28180 154/300 Write block 667 len 4096 data: 7
28181 155/300 Read block 2990 len 16384
28182 156/300 Write block 385 len 32768 data: 4 4 4 7 7 4 6 5
28183 157/300 Read block 279 len 24576
28184 158/300 Write block 4698 len 8192 data: 5 5
28185 159/300 Write block 3948 len 36864 data: 4 4 4 3 5 3 4 6 7
28186 160/300 Read block 1889 len 24576
28187 161/300 Read block 2047 len 32768
28188 162/300 Write block 798 len 36864 data: 7 5 4 5 4 4 3 3 3
28189 163/300 Read block 3368 len 40960
28190 164/300 Flush
28191 165/300 Write block 1285 len 36864 data: 5 5 4 6 4 5 5 4 4
28192 166/300 Read block 4941 len 40960
28193 167/300 Read block 3973 len 32768
28194 168/300 Write block 22 len 8192 data: 6 7
28195 169/300 Flush
28196 170/300 Write block 1432 len 4096 data: 3
28197 171/300 Read block 3883 len 8192
28198 172/300 Read block 1188 len 40960
28199 173/300 Flush
28200 174/300 Read block 2740 len 28672
28201 175/300 Read block 91 len 28672
28202 176/300 Read block 4166 len 40960
28203 177/300 Read block 2473 len 8192
28204 178/300 Write block 2098 len 24576 data: 6 5 5 5 5 6
28205 179/300 Write block 4709 len 12288 data: 7 7 6
28206 180/300 Read block 4975 len 8192
28207 181/300 Write block 1398 len 36864 data: 2 2 2 3 3 3 4 7 6
28208 182/300 Read block 4742 len 12288
28209 183/300 Write block 3196 len 32768 data: 5 6 6 5 4 4 4 4
28210 184/300 Write block 273 len 16384 data: 3 3 3 3
28211 185/300 Read block 4288 len 20480
28212 186/300 Write block 153 len 16384 data: 3 2 2 3
28213 187/300 Read block 1334 len 40960
28214 188/300 Write block 4092 len 32768 data: 5 6 6 6 6 5 4 3
28215 189/300 Read block 2252 len 24576
28216 190/300 Read block 714 len 28672
28217 191/300 Write block 946 len 8192 data: 2 3
28218 192/300 Read block 1809 len 32768
28219 193/300 Write block 2530 len 28672 data: 10 11 11 8 7 5 5
28220 194/300 Read block 4545 len 24576
28221 195/300 Read block 4079 len 4096
28222 196/300 Read block 4978 len 12288
28223 197/300 Read block 599 len 8192
28224 198/300 Write block 3938 len 28672 data: 5 5 4 3 3 3 3
28225 199/300 Read block 698 len 12288
28226 200/300 Flush
28227 201/300 Write block 2080 len 36864 data: 6 6 4 3 3 3 4 5 5
28228 202/300 Read block 4329 len 24576
28229 203/300 Read block 473 len 24576
28230 204/300 Read block 2235 len 32768
28231 205/300 Read block 1087 len 24576
28232 206/300 Write block 983 len 24576 data: 2 3 4 5 5 5
28233 207/300 Read block 951 len 16384
28234 208/300 Read block 1381 len 40960
28235 209/300 Flush
28236 210/300 Read block 2424 len 36864
28237 211/300 Read block 3833 len 4096
28238 212/300 Read block 4442 len 8192
28239 213/300 Write block 2477 len 12288 data: 9 7 5
28240 214/300 Read block 1872 len 12288
28241 215/300 Flush
28242 216/300 Write block 3804 len 32768 data: 6 6 7 8 7 6 4 6
28243 217/300 Write block 2528 len 36864 data: 11 11 11 12 12 9 8 6 6
28244 218/300 Write block 3109 len 28672 data: 5 6 5 6 5 4 4
28245 219/300 Read block 2710 len 12288
28246 220/300 Write block 1575 len 28672 data: 3 3 2 2 2 3 4
28247 221/300 Write block 3707 len 36864 data: 5 6 7 6 6 6 5 4 4
28248 222/300 Flush
28249 223/300 Write block 3295 len 4096 data: 3
28250 224/300 Read block 18 len 36864
28251 225/300 Read block 3809 len 28672
28252 226/300 Write block 587 len 32768 data: 4 6 6 6 7 7 8 7
28253 227/300 Read block 3460 len 28672
28254 228/300 Read block 3687 len 36864
28255 229/300 Read block 1740 len 16384
28256 230/300 Write block 2042 len 40960 data: 4 4 4 4 4 4 3 3 2 2
28257 231/300 Write block 1018 len 40960 data: 6 6 5 6 4 4 5 4 4 4
28258 232/300 Write block 800 len 24576 data: 5 6 5 5 4 4
28259 233/300 Read block 201 len 36864
28260 234/300 Write block 1909 len 20480 data: 3 3 4 4 3
28261 235/300 Flush
28262 236/300 Read block 675 len 4096
28263 237/300 Write block 901 len 16384 data: 3 5 5 5
28264 238/300 Read block 1430 len 28672
28265 239/300 Read block 3254 len 12288
28266 240/300 Flush
28267 241/300 Read block 45 len 40960
28268 242/300 Read block 63 len 20480
28269 243/300 Write block 3938 len 12288 data: 6 6 5
28270 244/300 Write block 2447 len 4096 data: 6
28271 245/300 Write block 1927 len 16384 data: 3 5 5 5
28272 246/300 Flush
28273 247/300 Read block 3306 len 32768
28274 248/300 Read block 3831 len 32768
28275 249/300 Write block 2398 len 40960 data: 3 3 4 5 6 8 8 8 6 5
28276 250/300 Read block 948 len 28672
28277 251/300 Write block 1463 len 8192 data: 6 8
28278 252/300 Write block 3934 len 12288 data: 5 5 6
28279 253/300 Flush
28280 254/300 Read block 2183 len 36864
28281 255/300 Read block 1114 len 32768
28282 256/300 Read block 3469 len 32768
28283 257/300 Read block 1548 len 20480
28284 258/300 Write block 1744 len 32768 data: 5 5 5 6 4 3 3 2
28285 259/300 Read block 828 len 4096
28286 260/300 Read block 3050 len 8192
28287 261/300 Write block 1032 len 12288 data: 3 4 4
28288 262/300 Flush
28289 263/300 Read block 526 len 28672
28290 264/300 Flush
28291 265/300 Write block 544 len 36864 data: 5 5 5 6 6 3 4 3 2
28292 266/300 Read block 4883 len 36864
28293 267/300 Write block 2474 len 36864 data: 11 11 9 10 8 6 3 4 6
28294 268/300 Read block 2499 len 16384
28295 269/300 Write block 3774 len 4096 data: 4
28296 270/300 Write block 248 len 16384 data: 5 5 6 4
28297 271/300 Read block 1790 len 4096
28298 272/300 Read block 2370 len 24576
28299 273/300 Flush
28300 274/300 Read block 4691 len 32768
28301 275/300 Read block 4819 len 16384
28302 276/300 Read block 1572 len 32768
28303 277/300 Read block 122 len 20480
28304 278/300 Flush
28305 279/300 Write block 2815 len 28672 data: 7 8 7 8 8 6 6
28306 280/300 Write block 1104 len 24576 data: 3 2 3 3 3 4
28307 281/300 Flush
28308 282/300 Read block 3062 len 24576
28309 283/300 Read block 3239 len 32768
28310 284/300 Read block 4902 len 24576
28311 285/300 Read block 4595 len 12288
28312 286/300 Read block 4444 len 40960
28313 287/300 Write block 2155 len 40960 data: 6 5 5 5 6 5 5 5 6 5
28314 288/300 Write block 1307 len 16384 data: 4 5 5 5
28315 289/300 Read block 4059 len 20480
28316 290/300 Write block 2742 len 24576 data: 5 5 4 5 5 4
28317 291/300 Write block 1089 len 32768 data: 3 4 4 4 3 3 6 7
28318 292/300 Flush
28319 293/300 Read block 3073 len 12288
28320 294/300 Read block 4777 len 28672
28321 295/300 Read block 2483 len 20480
28322 296/300 Read block 3373 len 40960
28323 297/300 Read block 1121 len 20480
28324 298/300 Flush
28325 299/300 Read block 2337 len 32768
28326 300/300 Read block 2092 len 24576
28327 301/300 Write block 373 len 28672 data: 3 3 6 5 3 3 2
28328 Main task has work to do, go find it
28329 got dsc Start(0)
28330 start 0
28331 Nov 15 02:03:25.569 INFO request completed, latency_us: 224, response_code: 204, uri: /start/cid/0, method: POST, req_id: f4e1c7f3-73fc-4ff3-9da5-606b4e9400eb, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
28332 Replay: started 0, returned:Ok(())
28333 ----------------------------------------------------------------
28334 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
28335 States: Offline Active Active
28336 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
28337 7642 Acked Read 6 Sent Done Done false
28338 7643 Acked Flush 0 Sent Done Done false
28339 7644 Acked Read 3 Sent Done Done false
28340 7645 Acked Read 5 Sent Done Done false
28341 7646 Acked Flush 0 Sent Done Done false
28342 7647 Acked Read 9 Sent Done Done false
28343 7648 Acked Write 6 Sent Done Done false
28344 7649 Acked Write 6 Sent Done Done false
28345 7650 Acked Read 10 Sent Done Done false
28346 7651 Acked Read 5 Sent Done Done false
28347 7652 Acked Write 2 Sent Done Done false
28348 7653 Acked Flush 0 Sent Done Done false
28349 7654 Acked Read 9 Sent Done Done false
28350 7655 Acked Read 6 Sent Done Done false
28351 7656 Acked Read 6 Sent Done Done false
28352 7657 Acked Flush 0 Sent Done Done false
28353 7658 Acked Read 1 Sent Done Done false
28354 7659 Acked Flush 0 Sent Done Done false
28355 7660 Acked Read 1 Sent Done Done false
28356 7661 Acked Write 4 Sent Done Done false
28357 7662 Acked Read 10 Sent Done Done false
28358 7663 Acked Read 1 Sent Done Done false
28359 7664 Acked Write 6 Sent Done Done false
28360 7665 Acked Flush 0 Sent Done Done false
28361 7666 Acked Write 7 Sent Done Done false
28362 7667 Acked Flush 0 Sent Done Done false
28363 7668 Acked Read 2 Sent Done Done false
28364 7669 Acked Read 6 Sent Done Done false
28365 7670 Acked Read 9 Sent Done Done false
28366 7671 Acked Read 9 Sent Done Done false
28367 7672 Acked Write 8 Sent Done Done false
28368 7673 Acked Read 3 Sent Done Done false
28369 7674 Acked Read 10 Sent Done Done false
28370 7675 Acked Write 4 Sent Done Done false
28371 7676 Acked Read 2 Sent Done Done false
28372 7677 Acked Read 1 Sent Done Done false
28373 7678 Acked Write 6 Sent Done Done false
28374 7679 Acked Read 10 Sent Done Done false
28375 7680 Acked Read 5 Sent Done Done false
28376 7681 Acked Read 2 Sent Done Done false
28377 7682 Acked Read 2 Sent Done Done false
28378 7683 Acked Read 5 Sent Done Done false
28379 7684 Acked Read 7 Sent Done Done false
28380 7685 Acked Write 2 Sent Done Done false
28381 7686 Acked Write 5 Sent Done Done false
28382 7687 Acked Read 7 Sent Done Done false
28383 7688 Acked Write 3 Sent Done Done false
28384 7689 Acked Read 6 Sent Done Done false
28385 7690 Acked Read 8 Sent Done Done false
28386 7691 Acked Write 7 Sent Done Done false
28387 7692 Acked Read 9 Sent Done Done false
28388 7693 Acked Write 4 Sent Done Done false
28389 7694 Acked Read 1 Sent Done Done false
28390 7695 Acked Read 5 Sent Done Done false
28391 7696 Acked Read 8 Sent Done Done false
28392 7697 Acked Read 2 Sent Done Done false
28393 7698 Acked Write 3 Sent Done Done false
28394 7699 Acked Flush 0 Sent Done Done false
28395 7700 Acked Read 9 Sent Done Done false
28396 7701 Acked Flush 0 Sent Done Done false
28397 7702 Acked Flush 0 Sent Done Done false
28398 7703 Acked Write 10 Sent Done Done false
28399 7704 Acked Read 5 Sent Done Done false
28400 7705 Acked Read 8 Sent Done Done false
28401 7706 Acked Write 5 Sent Done Done false
28402 7707 Acked Read 10 Sent Done Done false
28403 7708 Acked Read 1 Sent Done Done false
28404 7709 Acked Write 5 Sent Done Done false
28405 7710 Acked Read 9 Sent Done Done false
28406 7711 Acked Read 6 Sent Done Done false
28407 7712 Acked Write 3 Sent Done Done false
28408 7713 Acked Read 6 Sent Done Done false
28409 7714 Acked Write 5 Sent Done Done false
28410 7715 Acked Read 9 Sent Done Done false
28411 7716 Acked Read 5 Sent Done Done false
28412 7717 Acked Read 10 Sent Done Done false
28413 7718 Acked Read 8 Sent Done Done false
28414 7719 Acked Write 3 Sent Done Done false
28415 7720 Acked Read 9 Sent Done Done false
28416 7721 Acked Write 7 Sent Done Done false
28417 7722 Acked Write 3 Sent Done Done false
28418 7723 Acked Write 4 Sent Done Done false
28419 7724 Acked Read 2 Sent Done Done false
28420 7725 Acked Read 8 Sent Done Done false
28421 7726 Acked Read 4 Sent Done Done false
28422 7727 Acked Read 1 Sent Done Done false
28423 7728 Acked Write 3 Sent Done Done false
28424 7729 Acked Flush 0 Sent Done Done false
28425 7730 Acked Read 9 Sent Done Done false
28426 7731 Acked Read 7 Sent Done Done false
28427 7732 Acked Write 4 Sent Done Done false
28428 7733 Acked Write 10 Sent Done Done false
28429 7734 Acked Write 10 Sent Done Done false
28430 7735 Acked Write 9 Sent Done Done false
28431 7736 Acked Read 1 Sent Done Done false
28432 7737 Acked Read 8 Sent Done Done false
28433 7738 Acked Read 7 Sent Done Done false
28434 7739 Acked Write 10 Sent Done Done false
28435 7740 Acked Write 8 Sent Done Done false
28436 7741 Acked Write 10 Sent Done Done false
28437 7742 Acked Read 10 Sent Done Done false
28438 7743 Acked Write 10 Sent Done Done false
28439 7744 Acked Flush 0 Sent Done Done false
28440 7745 Acked Flush 0 Sent Done Done false
28441 7746 Acked Write 1 Sent Done Done false
28442 7747 Acked Read 2 Sent Done Done false
28443 7748 Acked Read 6 Sent Done Done false
28444 7749 Acked Flush 0 Sent Done Done false
28445 7750 Acked Read 4 Sent Done Done false
28446 7751 Acked Write 2 Sent Done Done false
28447 7752 Acked Read 3 Sent Done Done false
28448 7753 Acked Write 8 Sent Done Done false
28449 7754 Acked Read 6 Sent Done Done false
28450 7755 Acked Write 7 Sent Done Done false
28451 7756 Acked Read 6 Sent Done Done false
28452 7757 Acked Write 10 Sent Done Done false
28453 7758 Acked Read 9 Sent Done Done false
28454 7759 Acked Read 6 Sent Done Done false
28455 7760 Acked Read 2 Sent Done Done false
28456 7761 Acked Flush 0 Sent Done Done false
28457 7762 Acked Read 10 Sent Done Done false
28458 7763 Acked Flush 0 Sent Done Done false
28459 7764 Acked Write 4 Sent Done Done false
28460 7765 Acked Read 5 Sent Done Done false
28461 7766 Acked Read 8 Sent Done Done false
28462 7767 Acked Flush 0 Sent Done Done false
28463 7768 Acked Flush 0 Sent Done Done false
28464 7769 Acked Flush 0 Sent Done Done false
28465 7770 Acked Write 1 Sent Done Done false
28466 7771 Acked Read 10 Sent Done Done false
28467 7772 Acked Read 8 Sent Done Done false
28468 7773 Acked Read 5 Sent Done Done false
28469 7774 Acked Write 3 Sent Done Done false
28470 7775 Acked Read 2 Sent Done Done false
28471 7776 Acked Write 6 Sent Done Done false
28472 7777 Acked Read 9 Sent Done Done false
28473 7778 Acked Read 1 Sent Done Done false
28474 7779 Acked Read 2 Sent Done Done false
28475 7780 Acked Write 8 Sent Done Done false
28476 7781 Acked Write 1 Sent Done Done false
28477 7782 Acked Write 2 Sent Done Done false
28478 7783 Acked Read 3 Sent Done Done false
28479 7784 Acked Read 1 Sent Done Done false
28480 7785 Acked Write 10 Sent Done Done false
28481 7786 Acked Write 9 Sent Done Done false
28482 7787 Acked Read 8 Sent Done Done false
28483 7788 Acked Write 10 Sent Done Done false
28484 7789 Acked Flush 0 Sent Done Done false
28485 7790 Acked Read 8 Sent Done Done false
28486 7791 Acked Read 4 Sent Done Done false
28487 7792 Acked Read 6 Sent Done Done false
28488 7793 Acked Write 3 Sent Done Done false
28489 7794 Acked Write 2 Sent Done Done false
28490 7795 Acked Write 1 Sent Done Done false
28491 7796 Acked Read 4 Sent Done Done false
28492 7797 Acked Write 8 Sent Done Done false
28493 7798 Acked Read 6 Sent Done Done false
28494 7799 Acked Write 2 Sent Done Done false
28495 7800 Acked Write 9 Sent Done Done false
28496 7801 Acked Read 6 Sent Done Done false
28497 7802 Acked Read 8 Sent Done Done false
28498 7803 Acked Write 9 Sent Done Done false
28499 7804 Acked Read 10 Sent Done Done false
28500 7805 Acked Flush 0 Sent Done Done false
28501 7806 Acked Write 9 Sent Done Done false
28502 7807 Acked Read 10 Sent Done Done false
28503 7808 Acked Read 8 Sent Done Done false
28504 7809 Acked Write 2 Sent Done Done false
28505 7810 Acked Flush 0 Sent Done Done false
28506 7811 Acked Write 1 Sent Done Done false
28507 7812 Acked Read 2 Sent Done Done false
28508 7813 Acked Read 10 Sent Done Done false
28509 7814 Acked Flush 0 Sent Done Done false
28510 7815 Acked Read 7 Sent Done Done false
28511 7816 Acked Read 7 Sent Done Done false
28512 7817 Acked Read 10 Sent Done Done false
28513 7818 Acked Read 2 Sent Done Done false
28514 7819 Acked Write 6 Sent Done Done false
28515 7820 Acked Write 3 Sent Done Done false
28516 7821 Acked Read 2 Sent Done Done false
28517 7822 Acked Write 9 Sent Done Done false
28518 7823 Acked Read 3 Sent Done Done false
28519 7824 Acked Write 8 Sent Done Done false
28520 7825 Acked Write 4 Sent Done Done false
28521 7826 Acked Read 5 Sent Done Done false
28522 7827 Acked Write 4 Sent Done Done false
28523 7828 Acked Read 10 Sent Done Done false
28524 7829 Acked Write 8 Sent Done Done false
28525 7830 Acked Read 6 Sent Done Done false
28526 7831 Acked Read 7 Sent Done Done false
28527 7832 Acked Write 2 Sent Done Done false
28528 7833 Acked Read 8 Sent Done Done false
28529 7834 Acked Write 7 Sent Done Done false
28530 7835 Acked Read 6 Sent Done Done false
28531 7836 Acked Read 1 Sent Done Done false
28532 7837 Acked Read 3 Sent Done Done false
28533 7838 Acked Read 2 Sent Done Done false
28534 7839 Acked Write 7 Sent Done Done false
28535 7840 Acked Read 3 Sent Done Done false
28536 7841 Acked Flush 0 Sent Done Done false
28537 7842 Acked Write 9 Sent Done Done false
28538 7843 Acked Read 6 Sent Done Done false
28539 7844 Acked Read 6 Sent Done Done false
28540 7845 Acked Read 8 Sent Done Done false
28541 7846 Acked Read 6 Sent Done Done false
28542 7847 Acked Write 6 Sent Done Done false
28543 7848 Acked Read 4 Sent Done Done false
28544 7849 Acked Read 10 Sent Done Done false
28545 7850 Acked Flush 0 Sent Done Done false
28546 7851 Acked Read 9 Sent Done Done false
28547 7852 Acked Read 1 Sent Done Done false
28548 7853 Acked Read 2 Sent Done Done false
28549 7854 Acked Write 3 Sent Done Done false
28550 7855 Acked Read 3 Sent Done Done false
28551 7856 Acked Flush 0 Sent Done Done false
28552 7857 Acked Write 8 Sent Done Done false
28553 7858 Acked Write 9 Sent Done Done false
28554 7859 Acked Write 7 Sent Done Done false
28555 7860 Acked Read 3 Sent Done Done false
28556 7861 Acked Write 7 Sent Done Done false
28557 7862 Acked Write 9 Sent Done Done false
28558 7863 Acked Flush 0 Sent Done Done false
28559 7864 Acked Write 1 Sent Done Done false
28560 7865 Acked Read 9 Sent Done Done false
28561 7866 Acked Read 7 Sent Done Done false
28562 7867 Acked Write 8 Sent Done Done false
28563 7868 Acked Read 7 Sent Done Done false
28564 7869 Acked Read 9 Sent Done Done false
28565 7870 Acked Read 4 Sent Done Done false
28566 7871 Acked Write 10 Sent Done Done false
28567 7872 Acked Write 10 Sent Done Done false
28568 7873 Acked Write 6 Sent Done Done false
28569 7874 Acked Read 9 Sent Done Done false
28570 7875 Acked Write 5 Sent Done Done false
28571 7876 Acked Flush 0 Sent Done Done false
28572 7877 Acked Read 1 Sent Done Done false
28573 7878 Acked Write 4 Sent Done Done false
28574 7879 Acked Read 7 Sent Done Done false
28575 7880 Acked Read 3 Sent Done Done false
28576 7881 Acked Flush 0 Sent Done Done false
28577 7882 Acked Read 10 Sent Done Done false
28578 7883 Acked Read 5 Sent Done Done false
28579 7884 Acked Write 3 Sent Done Done false
28580 7885 Acked Write 1 Sent Done Done false
28581 7886 Acked Write 4 Sent Done Done false
28582 7887 Acked Flush 0 Sent Done Done false
28583 7888 Acked Read 8 Sent Done Done false
28584 7889 Acked Read 8 Sent Done Done false
28585 7890 Acked Write 10 Sent Done Done false
28586 7891 Acked Read 7 Sent Done Done false
28587 7892 Acked Write 2 Sent Done Done false
28588 7893 Acked Write 3 Sent Done Done false
28589 7894 Acked Flush 0 Sent Done Done false
28590 7895 Acked Read 9 Sent Done Done false
28591 7896 Acked Read 8 Sent Done Done false
28592 7897 Acked Read 8 Sent Done Done false
28593 7898 Acked Read 5 Sent Done Done false
28594 7899 Acked Write 8 Sent Done Done false
28595 7900 Acked Read 1 Sent Done Done false
28596 7901 Acked Read 2 Sent Done Done false
28597 7902 Acked Write 3 Sent Done Done false
28598 7903 Acked Flush 0 Sent Done Done false
28599 7904 Acked Read 7 Sent Done Done false
28600 7905 Acked Flush 0 Sent Done Done false
28601 7906 Acked Write 9 Sent Done Done false
28602 7907 Acked Read 9 Sent Done Done false
28603 7908 Acked Write 9 Sent Done Done false
28604 7909 Acked Read 4 Sent Done Done false
28605 7910 Acked Write 1 Sent Done Done false
28606 7911 Acked Write 4 Sent Done Done false
28607 7912 Acked Read 1 Sent Done Done false
28608 7913 Acked Read 6 Sent Done Done false
28609 7914 Acked Flush 0 Sent Done Done false
28610 7915 Acked Read 8 Sent Done Done false
28611 7916 Acked Read 4 Sent Done Done false
28612 7917 Acked Read 8 Sent Done Done false
28613 7918 Acked Read 5 Sent Done Done false
28614 7919 Acked Flush 0 Sent Done Done false
28615 7920 Acked Write 7 Sent Done Done false
28616 7921 Acked Write 6 Sent Done Done false
28617 7922 Acked Flush 0 Sent Done Done false
28618 7923 Acked Read 6 Sent Done Done false
28619 7924 Acked Read 8 Sent Done Done false
28620 7925 Acked Read 6 Sent Done Done false
28621 7926 Acked Read 3 Sent Done Done false
28622 7927 Acked Read 10 Sent Done Done false
28623 7928 Acked Write 10 Sent Done Done false
28624 7929 Acked Write 4 Sent Done Done false
28625 7930 Acked Read 5 Sent Done Done false
28626 7931 Acked Write 6 Sent Done Done false
28627 7932 Acked Write 8 Sent Done Done false
28628 7933 Acked Flush 0 Sent Done Done false
28629 7934 Acked Read 3 Sent Done Done false
28630 7935 Acked Read 7 Sent Done Done false
28631 7936 Acked Read 5 Sent Done Done false
28632 7937 Acked Read 10 Sent Done Done false
28633 7938 Acked Read 5 Sent Done Done false
28634 7939 Acked Flush 0 Sent Done Done false
28635 7940 Acked Read 8 Sent Done Done false
28636 7941 Acked Read 6 Sent Done Done false
28637 7942 Acked Write 7 Sent Done Done false
28638 STATES DS:0 DS:1 DS:2 TOTAL
28639 Sent 301 0 0 301
28640 Done 0 301 301 602
28641 Skipped 0 0 0 0
28642 Error 0 0 0 0
28643 Last Flush: 7641 7939 7939
28644 Downstairs last five completed: 7641 7640 7639 7638 7637
28645 Upstairs last five completed: 7942 7941 7940 7939 7938
28646 CLIENT: Up:0 ds:301 act:2
28647 ----------------------------------------------------------------
28648 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
28649 States: Offline Active Active
28650 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
28651 7642 Acked Read 6 Sent Done Done false
28652 7643 Acked Flush 0 Sent Done Done false
28653 7644 Acked Read 3 Sent Done Done false
28654 7645 Acked Read 5 Sent Done Done false
28655 7646 Acked Flush 0 Sent Done Done false
28656 7647 Acked Read 9 Sent Done Done false
28657 7648 Acked Write 6 Sent Done Done false
28658 7649 Acked Write 6 Sent Done Done false
28659 7650 Acked Read 10 Sent Done Done false
28660 7651 Acked Read 5 Sent Done Done false
28661 7652 Acked Write 2 Sent Done Done false
28662 7653 Acked Flush 0 Sent Done Done false
28663 7654 Acked Read 9 Sent Done Done false
28664 7655 Acked Read 6 Sent Done Done false
28665 7656 Acked Read 6 Sent Done Done false
28666 7657 Acked Flush 0 Sent Done Done false
28667 7658 Acked Read 1 Sent Done Done false
28668 7659 Acked Flush 0 Sent Done Done false
28669 7660 Acked Read 1 Sent Done Done false
28670 7661 Acked Write 4 Sent Done Done false
28671 7662 Acked Read 10 Sent Done Done false
28672 7663 Acked Read 1 Sent Done Done false
28673 7664 Acked Write 6 Sent Done Done false
28674 7665 Acked Flush 0 Sent Done Done false
28675 7666 Acked Write 7 Sent Done Done false
28676 7667 Acked Flush 0 Sent Done Done false
28677 7668 Acked Read 2 Sent Done Done false
28678 7669 Acked Read 6 Sent Done Done false
28679 7670 Acked Read 9 Sent Done Done false
28680 7671 Acked Read 9 Sent Done Done false
28681 7672 Acked Write 8 Sent Done Done false
28682 7673 Acked Read 3 Sent Done Done false
28683 7674 Acked Read 10 Sent Done Done false
28684 7675 Acked Write 4 Sent Done Done false
28685 7676 Acked Read 2 Sent Done Done false
28686 7677 Acked Read 1 Sent Done Done false
28687 7678 Acked Write 6 Sent Done Done false
28688 7679 Acked Read 10 Sent Done Done false
28689 7680 Acked Read 5 Sent Done Done false
28690 7681 Acked Read 2 Sent Done Done false
28691 7682 Acked Read 2 Sent Done Done false
28692 7683 Acked Read 5 Sent Done Done false
28693 7684 Acked Read 7 Sent Done Done false
28694 7685 Acked Write 2 Sent Done Done false
28695 7686 Acked Write 5 Sent Done Done false
28696 7687 Acked Read 7 Sent Done Done false
28697 7688 Acked Write 3 Sent Done Done false
28698 7689 Acked Read 6 Sent Done Done false
28699 7690 Acked Read 8 Sent Done Done false
28700 7691 Acked Write 7 Sent Done Done false
28701 7692 Acked Read 9 Sent Done Done false
28702 7693 Acked Write 4 Sent Done Done false
28703 7694 Acked Read 1 Sent Done Done false
28704 7695 Acked Read 5 Sent Done Done false
28705 7696 Acked Read 8 Sent Done Done false
28706 7697 Acked Read 2 Sent Done Done false
28707 7698 Acked Write 3 Sent Done Done false
28708 7699 Acked Flush 0 Sent Done Done false
28709 7700 Acked Read 9 Sent Done Done false
28710 7701 Acked Flush 0 Sent Done Done false
28711 7702 Acked Flush 0 Sent Done Done false
28712 7703 Acked Write 10 Sent Done Done false
28713 7704 Acked Read 5 Sent Done Done false
28714 7705 Acked Read 8 Sent Done Done false
28715 7706 Acked Write 5 Sent Done Done false
28716 7707 Acked Read 10 Sent Done Done false
28717 7708 Acked Read 1 Sent Done Done false
28718 7709 Acked Write 5 Sent Done Done false
28719 7710 Acked Read 9 Sent Done Done false
28720 7711 Acked Read 6 Sent Done Done false
28721 7712 Acked Write 3 Sent Done Done false
28722 7713 Acked Read 6 Sent Done Done false
28723 7714 Acked Write 5 Sent Done Done false
28724 7715 Acked Read 9 Sent Done Done false
28725 7716 Acked Read 5 Sent Done Done false
28726 7717 Acked Read 10 Sent Done Done false
28727 7718 Acked Read 8 Sent Done Done false
28728 7719 Acked Write 3 Sent Done Done false
28729 7720 Acked Read 9 Sent Done Done false
28730 7721 Acked Write 7 Sent Done Done false
28731 7722 Acked Write 3 Sent Done Done false
28732 7723 Acked Write 4 Sent Done Done false
28733 7724 Acked Read 2 Sent Done Done false
28734 7725 Acked Read 8 Sent Done Done false
28735 7726 Acked Read 4 Sent Done Done false
28736 7727 Acked Read 1 Sent Done Done false
28737 7728 Acked Write 3 Sent Done Done false
28738 7729 Acked Flush 0 Sent Done Done false
28739 7730 Acked Read 9 Sent Done Done false
28740 7731 Acked Read 7 Sent Done Done false
28741 7732 Acked Write 4 Sent Done Done false
28742 7733 Acked Write 10 Sent Done Done false
28743 7734 Acked Write 10 Sent Done Done false
28744 7735 Acked Write 9 Sent Done Done false
28745 7736 Acked Read 1 Sent Done Done false
28746 7737 Acked Read 8 Sent Done Done false
28747 7738 Acked Read 7 Sent Done Done false
28748 7739 Acked Write 10 Sent Done Done false
28749 7740 Acked Write 8 Sent Done Done false
28750 7741 Acked Write 10 Sent Done Done false
28751 7742 Acked Read 10 Sent Done Done false
28752 7743 Acked Write 10 Sent Done Done false
28753 7744 Acked Flush 0 Sent Done Done false
28754 7745 Acked Flush 0 Sent Done Done false
28755 7746 Acked Write 1 Sent Done Done false
28756 7747 Acked Read 2 Sent Done Done false
28757 7748 Acked Read 6 Sent Done Done false
28758 7749 Acked Flush 0 Sent Done Done false
28759 7750 Acked Read 4 Sent Done Done false
28760 7751 Acked Write 2 Sent Done Done false
28761 7752 Acked Read 3 Sent Done Done false
28762 7753 Acked Write 8 Sent Done Done false
28763 7754 Acked Read 6 Sent Done Done false
28764 7755 Acked Write 7 Sent Done Done false
28765 7756 Acked Read 6 Sent Done Done false
28766 7757 Acked Write 10 Sent Done Done false
28767 7758 Acked Read 9 Sent Done Done false
28768 7759 Acked Read 6 Sent Done Done false
28769 7760 Acked Read 2 Sent Done Done false
28770 7761 Acked Flush 0 Sent Done Done false
28771 7762 Acked Read 10 Sent Done Done false
28772 7763 Acked Flush 0 Sent Done Done false
28773 7764 Acked Write 4 Sent Done Done false
28774 7765 Acked Read 5 Sent Done Done false
28775 7766 Acked Read 8 Sent Done Done false
28776 7767 Acked Flush 0 Sent Done Done false
28777 7768 Acked Flush 0 Sent Done Done false
28778 7769 Acked Flush 0 Sent Done Done false
28779 7770 Acked Write 1 Sent Done Done false
28780 7771 Acked Read 10 Sent Done Done false
28781 7772 Acked Read 8 Sent Done Done false
28782 7773 Acked Read 5 Sent Done Done false
28783 7774 Acked Write 3 Sent Done Done false
28784 7775 Acked Read 2 Sent Done Done false
28785 7776 Acked Write 6 Sent Done Done false
28786 7777 Acked Read 9 Sent Done Done false
28787 7778 Acked Read 1 Sent Done Done false
28788 7779 Acked Read 2 Sent Done Done false
28789 7780 Acked Write 8 Sent Done Done false
28790 7781 Acked Write 1 Sent Done Done false
28791 7782 Acked Write 2 Sent Done Done false
28792 7783 Acked Read 3 Sent Done Done false
28793 7784 Acked Read 1 Sent Done Done false
28794 7785 Acked Write 10 Sent Done Done false
28795 7786 Acked Write 9 Sent Done Done false
28796 7787 Acked Read 8 Sent Done Done false
28797 7788 Acked Write 10 Sent Done Done false
28798 7789 Acked Flush 0 Sent Done Done false
28799 7790 Acked Read 8 Sent Done Done false
28800 7791 Acked Read 4 Sent Done Done false
28801 7792 Acked Read 6 Sent Done Done false
28802 7793 Acked Write 3 Sent Done Done false
28803 7794 Acked Write 2 Sent Done Done false
28804 7795 Acked Write 1 Sent Done Done false
28805 7796 Acked Read 4 Sent Done Done false
28806 7797 Acked Write 8 Sent Done Done false
28807 7798 Acked Read 6 Sent Done Done false
28808 7799 Acked Write 2 Sent Done Done false
28809 7800 Acked Write 9 Sent Done Done false
28810 7801 Acked Read 6 Sent Done Done false
28811 7802 Acked Read 8 Sent Done Done false
28812 7803 Acked Write 9 Sent Done Done false
28813 7804 Acked Read 10 Sent Done Done false
28814 7805 Acked Flush 0 Sent Done Done false
28815 7806 Acked Write 9 Sent Done Done false
28816 7807 Acked Read 10 Sent Done Done false
28817 7808 Acked Read 8 Sent Done Done false
28818 7809 Acked Write 2 Sent Done Done false
28819 7810 Acked Flush 0 Sent Done Done false
28820 7811 Acked Write 1 Sent Done Done false
28821 7812 Acked Read 2 Sent Done Done false
28822 7813 Acked Read 10 Sent Done Done false
28823 7814 Acked Flush 0 Sent Done Done false
28824 7815 Acked Read 7 Sent Done Done false
28825 7816 Acked Read 7 Sent Done Done false
28826 7817 Acked Read 10 Sent Done Done false
28827 7818 Acked Read 2 Sent Done Done false
28828 7819 Acked Write 6 Sent Done Done false
28829 7820 Acked Write 3 Sent Done Done false
28830 7821 Acked Read 2 Sent Done Done false
28831 7822 Acked Write 9 Sent Done Done false
28832 7823 Acked Read 3 Sent Done Done false
28833 7824 Acked Write 8 Sent Done Done false
28834 7825 Acked Write 4 Sent Done Done false
28835 7826 Acked Read 5 Sent Done Done false
28836 7827 Acked Write 4 Sent Done Done false
28837 7828 Acked Read 10 Sent Done Done false
28838 7829 Acked Write 8 Sent Done Done false
28839 7830 Acked Read 6 Sent Done Done false
28840 7831 Acked Read 7 Sent Done Done false
28841 7832 Acked Write 2 Sent Done Done false
28842 7833 Acked Read 8 Sent Done Done false
28843 7834 Acked Write 7 Sent Done Done false
28844 7835 Acked Read 6 Sent Done Done false
28845 7836 Acked Read 1 Sent Done Done false
28846 7837 Acked Read 3 Sent Done Done false
28847 7838 Acked Read 2 Sent Done Done false
28848 7839 Acked Write 7 Sent Done Done false
28849 7840 Acked Read 3 Sent Done Done false
28850 7841 Acked Flush 0 Sent Done Done false
28851 7842 Acked Write 9 Sent Done Done false
28852 7843 Acked Read 6 Sent Done Done false
28853 7844 Acked Read 6 Sent Done Done false
28854 7845 Acked Read 8 Sent Done Done false
28855 7846 Acked Read 6 Sent Done Done false
28856 7847 Acked Write 6 Sent Done Done false
28857 7848 Acked Read 4 Sent Done Done false
28858 7849 Acked Read 10 Sent Done Done false
28859 7850 Acked Flush 0 Sent Done Done false
28860 7851 Acked Read 9 Sent Done Done false
28861 7852 Acked Read 1 Sent Done Done false
28862 7853 Acked Read 2 Sent Done Done false
28863 7854 Acked Write 3 Sent Done Done false
28864 7855 Acked Read 3 Sent Done Done false
28865 7856 Acked Flush 0 Sent Done Done false
28866 7857 Acked Write 8 Sent Done Done false
28867 7858 Acked Write 9 Sent Done Done false
28868 7859 Acked Write 7 Sent Done Done false
28869 7860 Acked Read 3 Sent Done Done false
28870 7861 Acked Write 7 Sent Done Done false
28871 7862 Acked Write 9 Sent Done Done false
28872 7863 Acked Flush 0 Sent Done Done false
28873 7864 Acked Write 1 Sent Done Done false
28874 7865 Acked Read 9 Sent Done Done false
28875 7866 Acked Read 7 Sent Done Done false
28876 7867 Acked Write 8 Sent Done Done false
28877 7868 Acked Read 7 Sent Done Done false
28878 7869 Acked Read 9 Sent Done Done false
28879 7870 Acked Read 4 Sent Done Done false
28880 7871 Acked Write 10 Sent Done Done false
28881 7872 Acked Write 10 Sent Done Done false
28882 7873 Acked Write 6 Sent Done Done false
28883 7874 Acked Read 9 Sent Done Done false
28884 7875 Acked Write 5 Sent Done Done false
28885 7876 Acked Flush 0 Sent Done Done false
28886 7877 Acked Read 1 Sent Done Done false
28887 7878 Acked Write 4 Sent Done Done false
28888 7879 Acked Read 7 Sent Done Done false
28889 7880 Acked Read 3 Sent Done Done false
28890 7881 Acked Flush 0 Sent Done Done false
28891 7882 Acked Read 10 Sent Done Done false
28892 7883 Acked Read 5 Sent Done Done false
28893 7884 Acked Write 3 Sent Done Done false
28894 7885 Acked Write 1 Sent Done Done false
28895 7886 Acked Write 4 Sent Done Done false
28896 7887 Acked Flush 0 Sent Done Done false
28897 7888 Acked Read 8 Sent Done Done false
28898 7889 Acked Read 8 Sent Done Done false
28899 7890 Acked Write 10 Sent Done Done false
28900 7891 Acked Read 7 Sent Done Done false
28901 7892 Acked Write 2 Sent Done Done false
28902 7893 Acked Write 3 Sent Done Done false
28903 7894 Acked Flush 0 Sent Done Done false
28904 7895 Acked Read 9 Sent Done Done false
28905 7896 Acked Read 8 Sent Done Done false
28906 7897 Acked Read 8 Sent Done Done false
28907 7898 Acked Read 5 Sent Done Done false
28908 7899 Acked Write 8 Sent Done Done false
28909 7900 Acked Read 1 Sent Done Done false
28910 7901 Acked Read 2 Sent Done Done false
28911 7902 Acked Write 3 Sent Done Done false
28912 7903 Acked Flush 0 Sent Done Done false
28913 7904 Acked Read 7 Sent Done Done false
28914 7905 Acked Flush 0 Sent Done Done false
28915 7906 Acked Write 9 Sent Done Done false
28916 7907 Acked Read 9 Sent Done Done false
28917 7908 Acked Write 9 Sent Done Done false
28918 7909 Acked Read 4 Sent Done Done false
28919 7910 Acked Write 1 Sent Done Done false
28920 7911 Acked Write 4 Sent Done Done false
28921 7912 Acked Read 1 Sent Done Done false
28922 7913 Acked Read 6 Sent Done Done false
28923 7914 Acked Flush 0 Sent Done Done false
28924 7915 Acked Read 8 Sent Done Done false
28925 7916 Acked Read 4 Sent Done Done false
28926 7917 Acked Read 8 Sent Done Done false
28927 7918 Acked Read 5 Sent Done Done false
28928 7919 Acked Flush 0 Sent Done Done false
28929 7920 Acked Write 7 Sent Done Done false
28930 7921 Acked Write 6 Sent Done Done false
28931 7922 Acked Flush 0 Sent Done Done false
28932 7923 Acked Read 6 Sent Done Done false
28933 7924 Acked Read 8 Sent Done Done false
28934 7925 Acked Read 6 Sent Done Done false
28935 7926 Acked Read 3 Sent Done Done false
28936 7927 Acked Read 10 Sent Done Done false
28937 7928 Acked Write 10 Sent Done Done false
28938 7929 Acked Write 4 Sent Done Done false
28939 7930 Acked Read 5 Sent Done Done false
28940 7931 Acked Write 6 Sent Done Done false
28941 7932 Acked Write 8 Sent Done Done false
28942 7933 Acked Flush 0 Sent Done Done false
28943 7934 Acked Read 3 Sent Done Done false
28944 7935 Acked Read 7 Sent Done Done false
28945 7936 Acked Read 5 Sent Done Done false
28946 7937 Acked Read 10 Sent Done Done false
28947 7938 Acked Read 5 Sent Done Done false
28948 7939 Acked Flush 0 Sent Done Done false
28949 7940 Acked Read 8 Sent Done Done false
28950 7941 Acked Read 6 Sent Done Done false
28951 7942 Acked Write 7 Sent Done Done false
28952 7943 Acked Flush 0 Sent Done Done false
28953 STATES DS:0 DS:1 DS:2 TOTAL
28954 Sent 302 0 0 302
28955 Done 0 302 302 604
28956 Skipped 0 0 0 0
28957 Error 0 0 0 0
28958 Last Flush: 7641 7943 7943
28959 Downstairs last five completed: 7641 7640 7639 7638 7637
28960 Upstairs last five completed: 7943 7942 7941 7940 7939
28961 CLIENT: Up:0 ds:302 act:2
289622024-11-15T02:03:31.314ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
289632024-11-15T02:03:31.314ZWARNcrucible: ds_connection connect to 127.0.0.1:8810 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
289642024-11-15T02:03:31.314ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
289652024-11-15T02:03:31.314ZWARNcrucible: downstairs task for 0 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
289662024-11-15T02:03:31.314ZINFOcrucible: client task is exiting = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
28967 [8810] got start action
28968 [8810] I am going to restart
28969 Starting downstairs at port 8810
28970 Make output file at "/tmp/dsc/downstairs-8810.txt"
28971 [8810][0] reports Starting
28972 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1228)
28973 [8810][0] reports Running
28974 ----------------------------------------------------------------
28975 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
28976 States: Offline Active Active
28977 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
28978 7642 Acked Read 6 Sent Done Done false
28979 7643 Acked Flush 0 Sent Done Done false
28980 7644 Acked Read 3 Sent Done Done false
28981 7645 Acked Read 5 Sent Done Done false
28982 7646 Acked Flush 0 Sent Done Done false
28983 7647 Acked Read 9 Sent Done Done false
28984 7648 Acked Write 6 Sent Done Done false
28985 7649 Acked Write 6 Sent Done Done false
28986 7650 Acked Read 10 Sent Done Done false
28987 7651 Acked Read 5 Sent Done Done false
28988 7652 Acked Write 2 Sent Done Done false
28989 7653 Acked Flush 0 Sent Done Done false
28990 7654 Acked Read 9 Sent Done Done false
28991 7655 Acked Read 6 Sent Done Done false
28992 7656 Acked Read 6 Sent Done Done false
28993 7657 Acked Flush 0 Sent Done Done false
28994 7658 Acked Read 1 Sent Done Done false
28995 7659 Acked Flush 0 Sent Done Done false
28996 7660 Acked Read 1 Sent Done Done false
28997 7661 Acked Write 4 Sent Done Done false
28998 7662 Acked Read 10 Sent Done Done false
28999 7663 Acked Read 1 Sent Done Done false
29000 7664 Acked Write 6 Sent Done Done false
29001 7665 Acked Flush 0 Sent Done Done false
29002 7666 Acked Write 7 Sent Done Done false
29003 7667 Acked Flush 0 Sent Done Done false
29004 7668 Acked Read 2 Sent Done Done false
29005 7669 Acked Read 6 Sent Done Done false
29006 7670 Acked Read 9 Sent Done Done false
29007 7671 Acked Read 9 Sent Done Done false
29008 7672 Acked Write 8 Sent Done Done false
29009 7673 Acked Read 3 Sent Done Done false
29010 7674 Acked Read 10 Sent Done Done false
29011 7675 Acked Write 4 Sent Done Done false
29012 7676 Acked Read 2 Sent Done Done false
29013 7677 Acked Read 1 Sent Done Done false
29014 7678 Acked Write 6 Sent Done Done false
29015 7679 Acked Read 10 Sent Done Done false
29016 7680 Acked Read 5 Sent Done Done false
29017 7681 Acked Read 2 Sent Done Done false
29018 7682 Acked Read 2 Sent Done Done false
29019 7683 Acked Read 5 Sent Done Done false
29020 7684 Acked Read 7 Sent Done Done false
29021 7685 Acked Write 2 Sent Done Done false
29022 7686 Acked Write 5 Sent Done Done false
29023 7687 Acked Read 7 Sent Done Done false
29024 7688 Acked Write 3 Sent Done Done false
29025 7689 Acked Read 6 Sent Done Done false
29026 7690 Acked Read 8 Sent Done Done false
29027 7691 Acked Write 7 Sent Done Done false
29028 7692 Acked Read 9 Sent Done Done false
29029 7693 Acked Write 4 Sent Done Done false
29030 7694 Acked Read 1 Sent Done Done false
29031 7695 Acked Read 5 Sent Done Done false
29032 7696 Acked Read 8 Sent Done Done false
29033 7697 Acked Read 2 Sent Done Done false
29034 7698 Acked Write 3 Sent Done Done false
29035 7699 Acked Flush 0 Sent Done Done false
29036 7700 Acked Read 9 Sent Done Done false
29037 7701 Acked Flush 0 Sent Done Done false
29038 7702 Acked Flush 0 Sent Done Done false
29039 7703 Acked Write 10 Sent Done Done false
29040 7704 Acked Read 5 Sent Done Done false
29041 7705 Acked Read 8 Sent Done Done false
29042 7706 Acked Write 5 Sent Done Done false
29043 7707 Acked Read 10 Sent Done Done false
29044 7708 Acked Read 1 Sent Done Done false
29045 7709 Acked Write 5 Sent Done Done false
29046 7710 Acked Read 9 Sent Done Done false
29047 7711 Acked Read 6 Sent Done Done false
29048 7712 Acked Write 3 Sent Done Done false
29049 7713 Acked Read 6 Sent Done Done false
29050 7714 Acked Write 5 Sent Done Done false
29051 7715 Acked Read 9 Sent Done Done false
29052 7716 Acked Read 5 Sent Done Done false
29053 7717 Acked Read 10 Sent Done Done false
29054 7718 Acked Read 8 Sent Done Done false
29055 7719 Acked Write 3 Sent Done Done false
29056 7720 Acked Read 9 Sent Done Done false
29057 7721 Acked Write 7 Sent Done Done false
29058 7722 Acked Write 3 Sent Done Done false
29059 7723 Acked Write 4 Sent Done Done false
29060 7724 Acked Read 2 Sent Done Done false
29061 7725 Acked Read 8 Sent Done Done false
29062 7726 Acked Read 4 Sent Done Done false
29063 7727 Acked Read 1 Sent Done Done false
29064 7728 Acked Write 3 Sent Done Done false
29065 7729 Acked Flush 0 Sent Done Done false
29066 7730 Acked Read 9 Sent Done Done false
29067 7731 Acked Read 7 Sent Done Done false
29068 7732 Acked Write 4 Sent Done Done false
29069 7733 Acked Write 10 Sent Done Done false
29070 7734 Acked Write 10 Sent Done Done false
29071 7735 Acked Write 9 Sent Done Done false
29072 7736 Acked Read 1 Sent Done Done false
29073 7737 Acked Read 8 Sent Done Done false
29074 7738 Acked Read 7 Sent Done Done false
29075 7739 Acked Write 10 Sent Done Done false
29076 7740 Acked Write 8 Sent Done Done false
29077 7741 Acked Write 10 Sent Done Done false
29078 7742 Acked Read 10 Sent Done Done false
29079 7743 Acked Write 10 Sent Done Done false
29080 7744 Acked Flush 0 Sent Done Done false
29081 7745 Acked Flush 0 Sent Done Done false
29082 7746 Acked Write 1 Sent Done Done false
29083 7747 Acked Read 2 Sent Done Done false
29084 7748 Acked Read 6 Sent Done Done false
29085 7749 Acked Flush 0 Sent Done Done false
29086 7750 Acked Read 4 Sent Done Done false
29087 7751 Acked Write 2 Sent Done Done false
29088 7752 Acked Read 3 Sent Done Done false
29089 7753 Acked Write 8 Sent Done Done false
29090 7754 Acked Read 6 Sent Done Done false
29091 7755 Acked Write 7 Sent Done Done false
29092 7756 Acked Read 6 Sent Done Done false
29093 7757 Acked Write 10 Sent Done Done false
29094 7758 Acked Read 9 Sent Done Done false
29095 7759 Acked Read 6 Sent Done Done false
29096 7760 Acked Read 2 Sent Done Done false
29097 7761 Acked Flush 0 Sent Done Done false
29098 7762 Acked Read 10 Sent Done Done false
29099 7763 Acked Flush 0 Sent Done Done false
29100 7764 Acked Write 4 Sent Done Done false
29101 7765 Acked Read 5 Sent Done Done false
29102 7766 Acked Read 8 Sent Done Done false
29103 7767 Acked Flush 0 Sent Done Done false
29104 7768 Acked Flush 0 Sent Done Done false
29105 7769 Acked Flush 0 Sent Done Done false
29106 7770 Acked Write 1 Sent Done Done false
29107 7771 Acked Read 10 Sent Done Done false
29108 7772 Acked Read 8 Sent Done Done false
29109 7773 Acked Read 5 Sent Done Done false
29110 7774 Acked Write 3 Sent Done Done false
29111 7775 Acked Read 2 Sent Done Done false
29112 7776 Acked Write 6 Sent Done Done false
29113 7777 Acked Read 9 Sent Done Done false
29114 7778 Acked Read 1 Sent Done Done false
29115 7779 Acked Read 2 Sent Done Done false
29116 7780 Acked Write 8 Sent Done Done false
29117 7781 Acked Write 1 Sent Done Done false
29118 7782 Acked Write 2 Sent Done Done false
29119 7783 Acked Read 3 Sent Done Done false
29120 7784 Acked Read 1 Sent Done Done false
29121 7785 Acked Write 10 Sent Done Done false
29122 7786 Acked Write 9 Sent Done Done false
29123 7787 Acked Read 8 Sent Done Done false
29124 7788 Acked Write 10 Sent Done Done false
29125 7789 Acked Flush 0 Sent Done Done false
29126 7790 Acked Read 8 Sent Done Done false
29127 7791 Acked Read 4 Sent Done Done false
29128 7792 Acked Read 6 Sent Done Done false
29129 7793 Acked Write 3 Sent Done Done false
29130 7794 Acked Write 2 Sent Done Done false
29131 7795 Acked Write 1 Sent Done Done false
29132 7796 Acked Read 4 Sent Done Done false
29133 7797 Acked Write 8 Sent Done Done false
29134 7798 Acked Read 6 Sent Done Done false
29135 7799 Acked Write 2 Sent Done Done false
29136 7800 Acked Write 9 Sent Done Done false
29137 7801 Acked Read 6 Sent Done Done false
29138 7802 Acked Read 8 Sent Done Done false
29139 7803 Acked Write 9 Sent Done Done false
29140 7804 Acked Read 10 Sent Done Done false
29141 7805 Acked Flush 0 Sent Done Done false
29142 7806 Acked Write 9 Sent Done Done false
29143 7807 Acked Read 10 Sent Done Done false
29144 7808 Acked Read 8 Sent Done Done false
29145 7809 Acked Write 2 Sent Done Done false
29146 7810 Acked Flush 0 Sent Done Done false
29147 7811 Acked Write 1 Sent Done Done false
29148 7812 Acked Read 2 Sent Done Done false
29149 7813 Acked Read 10 Sent Done Done false
29150 7814 Acked Flush 0 Sent Done Done false
29151 7815 Acked Read 7 Sent Done Done false
29152 7816 Acked Read 7 Sent Done Done false
29153 7817 Acked Read 10 Sent Done Done false
29154 7818 Acked Read 2 Sent Done Done false
29155 7819 Acked Write 6 Sent Done Done false
29156 7820 Acked Write 3 Sent Done Done false
29157 7821 Acked Read 2 Sent Done Done false
29158 7822 Acked Write 9 Sent Done Done false
29159 7823 Acked Read 3 Sent Done Done false
29160 7824 Acked Write 8 Sent Done Done false
29161 7825 Acked Write 4 Sent Done Done false
29162 7826 Acked Read 5 Sent Done Done false
29163 7827 Acked Write 4 Sent Done Done false
29164 7828 Acked Read 10 Sent Done Done false
29165 7829 Acked Write 8 Sent Done Done false
29166 7830 Acked Read 6 Sent Done Done false
29167 7831 Acked Read 7 Sent Done Done false
29168 7832 Acked Write 2 Sent Done Done false
29169 7833 Acked Read 8 Sent Done Done false
29170 7834 Acked Write 7 Sent Done Done false
29171 7835 Acked Read 6 Sent Done Done false
29172 7836 Acked Read 1 Sent Done Done false
29173 7837 Acked Read 3 Sent Done Done false
29174 7838 Acked Read 2 Sent Done Done false
29175 7839 Acked Write 7 Sent Done Done false
29176 7840 Acked Read 3 Sent Done Done false
29177 7841 Acked Flush 0 Sent Done Done false
29178 7842 Acked Write 9 Sent Done Done false
29179 7843 Acked Read 6 Sent Done Done false
29180 7844 Acked Read 6 Sent Done Done false
29181 7845 Acked Read 8 Sent Done Done false
29182 7846 Acked Read 6 Sent Done Done false
29183 7847 Acked Write 6 Sent Done Done false
29184 7848 Acked Read 4 Sent Done Done false
29185 7849 Acked Read 10 Sent Done Done false
29186 7850 Acked Flush 0 Sent Done Done false
29187 7851 Acked Read 9 Sent Done Done false
29188 7852 Acked Read 1 Sent Done Done false
29189 7853 Acked Read 2 Sent Done Done false
29190 7854 Acked Write 3 Sent Done Done false
29191 7855 Acked Read 3 Sent Done Done false
29192 7856 Acked Flush 0 Sent Done Done false
29193 7857 Acked Write 8 Sent Done Done false
29194 7858 Acked Write 9 Sent Done Done false
29195 7859 Acked Write 7 Sent Done Done false
29196 7860 Acked Read 3 Sent Done Done false
29197 7861 Acked Write 7 Sent Done Done false
29198 7862 Acked Write 9 Sent Done Done false
29199 7863 Acked Flush 0 Sent Done Done false
29200 7864 Acked Write 1 Sent Done Done false
29201 7865 Acked Read 9 Sent Done Done false
29202 7866 Acked Read 7 Sent Done Done false
29203 7867 Acked Write 8 Sent Done Done false
29204 7868 Acked Read 7 Sent Done Done false
29205 7869 Acked Read 9 Sent Done Done false
29206 7870 Acked Read 4 Sent Done Done false
29207 7871 Acked Write 10 Sent Done Done false
29208 7872 Acked Write 10 Sent Done Done false
29209 7873 Acked Write 6 Sent Done Done false
29210 7874 Acked Read 9 Sent Done Done false
29211 7875 Acked Write 5 Sent Done Done false
29212 7876 Acked Flush 0 Sent Done Done false
29213 7877 Acked Read 1 Sent Done Done false
29214 7878 Acked Write 4 Sent Done Done false
29215 7879 Acked Read 7 Sent Done Done false
29216 7880 Acked Read 3 Sent Done Done false
29217 7881 Acked Flush 0 Sent Done Done false
29218 7882 Acked Read 10 Sent Done Done false
29219 7883 Acked Read 5 Sent Done Done false
29220 7884 Acked Write 3 Sent Done Done false
29221 7885 Acked Write 1 Sent Done Done false
29222 7886 Acked Write 4 Sent Done Done false
29223 7887 Acked Flush 0 Sent Done Done false
29224 7888 Acked Read 8 Sent Done Done false
29225 7889 Acked Read 8 Sent Done Done false
29226 7890 Acked Write 10 Sent Done Done false
29227 7891 Acked Read 7 Sent Done Done false
29228 7892 Acked Write 2 Sent Done Done false
29229 7893 Acked Write 3 Sent Done Done false
29230 7894 Acked Flush 0 Sent Done Done false
29231 7895 Acked Read 9 Sent Done Done false
29232 7896 Acked Read 8 Sent Done Done false
29233 7897 Acked Read 8 Sent Done Done false
29234 7898 Acked Read 5 Sent Done Done false
29235 7899 Acked Write 8 Sent Done Done false
29236 7900 Acked Read 1 Sent Done Done false
29237 7901 Acked Read 2 Sent Done Done false
29238 7902 Acked Write 3 Sent Done Done false
29239 7903 Acked Flush 0 Sent Done Done false
29240 7904 Acked Read 7 Sent Done Done false
29241 7905 Acked Flush 0 Sent Done Done false
29242 7906 Acked Write 9 Sent Done Done false
29243 7907 Acked Read 9 Sent Done Done false
29244 7908 Acked Write 9 Sent Done Done false
29245 7909 Acked Read 4 Sent Done Done false
29246 7910 Acked Write 1 Sent Done Done false
29247 7911 Acked Write 4 Sent Done Done false
29248 7912 Acked Read 1 Sent Done Done false
29249 7913 Acked Read 6 Sent Done Done false
29250 7914 Acked Flush 0 Sent Done Done false
29251 7915 Acked Read 8 Sent Done Done false
29252 7916 Acked Read 4 Sent Done Done false
29253 7917 Acked Read 8 Sent Done Done false
29254 7918 Acked Read 5 Sent Done Done false
29255 7919 Acked Flush 0 Sent Done Done false
29256 7920 Acked Write 7 Sent Done Done false
29257 7921 Acked Write 6 Sent Done Done false
29258 7922 Acked Flush 0 Sent Done Done false
29259 7923 Acked Read 6 Sent Done Done false
29260 7924 Acked Read 8 Sent Done Done false
29261 7925 Acked Read 6 Sent Done Done false
29262 7926 Acked Read 3 Sent Done Done false
29263 7927 Acked Read 10 Sent Done Done false
29264 7928 Acked Write 10 Sent Done Done false
29265 7929 Acked Write 4 Sent Done Done false
29266 7930 Acked Read 5 Sent Done Done false
29267 7931 Acked Write 6 Sent Done Done false
29268 7932 Acked Write 8 Sent Done Done false
29269 7933 Acked Flush 0 Sent Done Done false
29270 7934 Acked Read 3 Sent Done Done false
29271 7935 Acked Read 7 Sent Done Done false
29272 7936 Acked Read 5 Sent Done Done false
29273 7937 Acked Read 10 Sent Done Done false
29274 7938 Acked Read 5 Sent Done Done false
29275 7939 Acked Flush 0 Sent Done Done false
29276 7940 Acked Read 8 Sent Done Done false
29277 7941 Acked Read 6 Sent Done Done false
29278 7942 Acked Write 7 Sent Done Done false
29279 7943 Acked Flush 0 Sent Done Done false
29280 STATES DS:0 DS:1 DS:2 TOTAL
29281 Sent 302 0 0 302
29282 Done 0 302 302 604
29283 Skipped 0 0 0 0
29284 Error 0 0 0 0
29285 Last Flush: 7641 7943 7943
29286 Downstairs last five completed: 7641 7640 7639 7638 7637
29287 Upstairs last five completed: 7943 7942 7941 7940 7939
29288 CLIENT: Up:0 ds:302 act:2
29289 ----------------------------------------------------------------
29290 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
29291 States: Offline Active Active
29292 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
29293 7642 Acked Read 6 Sent Done Done false
29294 7643 Acked Flush 0 Sent Done Done false
29295 7644 Acked Read 3 Sent Done Done false
29296 7645 Acked Read 5 Sent Done Done false
29297 7646 Acked Flush 0 Sent Done Done false
29298 7647 Acked Read 9 Sent Done Done false
29299 7648 Acked Write 6 Sent Done Done false
29300 7649 Acked Write 6 Sent Done Done false
29301 7650 Acked Read 10 Sent Done Done false
29302 7651 Acked Read 5 Sent Done Done false
29303 7652 Acked Write 2 Sent Done Done false
29304 7653 Acked Flush 0 Sent Done Done false
29305 7654 Acked Read 9 Sent Done Done false
29306 7655 Acked Read 6 Sent Done Done false
29307 7656 Acked Read 6 Sent Done Done false
29308 7657 Acked Flush 0 Sent Done Done false
29309 7658 Acked Read 1 Sent Done Done false
29310 7659 Acked Flush 0 Sent Done Done false
29311 7660 Acked Read 1 Sent Done Done false
29312 7661 Acked Write 4 Sent Done Done false
29313 7662 Acked Read 10 Sent Done Done false
29314 7663 Acked Read 1 Sent Done Done false
29315 7664 Acked Write 6 Sent Done Done false
29316 7665 Acked Flush 0 Sent Done Done false
29317 7666 Acked Write 7 Sent Done Done false
29318 7667 Acked Flush 0 Sent Done Done false
29319 7668 Acked Read 2 Sent Done Done false
29320 7669 Acked Read 6 Sent Done Done false
29321 7670 Acked Read 9 Sent Done Done false
29322 7671 Acked Read 9 Sent Done Done false
29323 7672 Acked Write 8 Sent Done Done false
29324 7673 Acked Read 3 Sent Done Done false
29325 7674 Acked Read 10 Sent Done Done false
29326 7675 Acked Write 4 Sent Done Done false
29327 7676 Acked Read 2 Sent Done Done false
29328 7677 Acked Read 1 Sent Done Done false
29329 7678 Acked Write 6 Sent Done Done false
29330 7679 Acked Read 10 Sent Done Done false
29331 7680 Acked Read 5 Sent Done Done false
29332 7681 Acked Read 2 Sent Done Done false
29333 7682 Acked Read 2 Sent Done Done false
29334 7683 Acked Read 5 Sent Done Done false
29335 7684 Acked Read 7 Sent Done Done false
29336 7685 Acked Write 2 Sent Done Done false
29337 7686 Acked Write 5 Sent Done Done false
29338 7687 Acked Read 7 Sent Done Done false
29339 7688 Acked Write 3 Sent Done Done false
29340 7689 Acked Read 6 Sent Done Done false
29341 7690 Acked Read 8 Sent Done Done false
29342 7691 Acked Write 7 Sent Done Done false
29343 7692 Acked Read 9 Sent Done Done false
29344 7693 Acked Write 4 Sent Done Done false
29345 7694 Acked Read 1 Sent Done Done false
29346 7695 Acked Read 5 Sent Done Done false
29347 7696 Acked Read 8 Sent Done Done false
29348 7697 Acked Read 2 Sent Done Done false
29349 7698 Acked Write 3 Sent Done Done false
29350 7699 Acked Flush 0 Sent Done Done false
29351 7700 Acked Read 9 Sent Done Done false
29352 7701 Acked Flush 0 Sent Done Done false
29353 7702 Acked Flush 0 Sent Done Done false
29354 7703 Acked Write 10 Sent Done Done false
29355 7704 Acked Read 5 Sent Done Done false
29356 7705 Acked Read 8 Sent Done Done false
29357 7706 Acked Write 5 Sent Done Done false
29358 7707 Acked Read 10 Sent Done Done false
29359 7708 Acked Read 1 Sent Done Done false
29360 7709 Acked Write 5 Sent Done Done false
29361 7710 Acked Read 9 Sent Done Done false
29362 7711 Acked Read 6 Sent Done Done false
29363 7712 Acked Write 3 Sent Done Done false
29364 7713 Acked Read 6 Sent Done Done false
29365 7714 Acked Write 5 Sent Done Done false
29366 7715 Acked Read 9 Sent Done Done false
29367 7716 Acked Read 5 Sent Done Done false
29368 7717 Acked Read 10 Sent Done Done false
29369 7718 Acked Read 8 Sent Done Done false
29370 7719 Acked Write 3 Sent Done Done false
29371 7720 Acked Read 9 Sent Done Done false
29372 7721 Acked Write 7 Sent Done Done false
29373 7722 Acked Write 3 Sent Done Done false
29374 7723 Acked Write 4 Sent Done Done false
29375 7724 Acked Read 2 Sent Done Done false
29376 7725 Acked Read 8 Sent Done Done false
29377 7726 Acked Read 4 Sent Done Done false
29378 7727 Acked Read 1 Sent Done Done false
29379 7728 Acked Write 3 Sent Done Done false
29380 7729 Acked Flush 0 Sent Done Done false
29381 7730 Acked Read 9 Sent Done Done false
29382 7731 Acked Read 7 Sent Done Done false
29383 7732 Acked Write 4 Sent Done Done false
29384 7733 Acked Write 10 Sent Done Done false
29385 7734 Acked Write 10 Sent Done Done false
29386 7735 Acked Write 9 Sent Done Done false
29387 7736 Acked Read 1 Sent Done Done false
29388 7737 Acked Read 8 Sent Done Done false
29389 7738 Acked Read 7 Sent Done Done false
29390 7739 Acked Write 10 Sent Done Done false
29391 7740 Acked Write 8 Sent Done Done false
29392 7741 Acked Write 10 Sent Done Done false
29393 7742 Acked Read 10 Sent Done Done false
29394 7743 Acked Write 10 Sent Done Done false
29395 7744 Acked Flush 0 Sent Done Done false
29396 7745 Acked Flush 0 Sent Done Done false
29397 7746 Acked Write 1 Sent Done Done false
29398 7747 Acked Read 2 Sent Done Done false
29399 7748 Acked Read 6 Sent Done Done false
29400 7749 Acked Flush 0 Sent Done Done false
29401 7750 Acked Read 4 Sent Done Done false
29402 7751 Acked Write 2 Sent Done Done false
29403 7752 Acked Read 3 Sent Done Done false
29404 7753 Acked Write 8 Sent Done Done false
29405 7754 Acked Read 6 Sent Done Done false
29406 7755 Acked Write 7 Sent Done Done false
29407 7756 Acked Read 6 Sent Done Done false
29408 7757 Acked Write 10 Sent Done Done false
29409 7758 Acked Read 9 Sent Done Done false
29410 7759 Acked Read 6 Sent Done Done false
29411 7760 Acked Read 2 Sent Done Done false
29412 7761 Acked Flush 0 Sent Done Done false
29413 7762 Acked Read 10 Sent Done Done false
29414 7763 Acked Flush 0 Sent Done Done false
29415 7764 Acked Write 4 Sent Done Done false
29416 7765 Acked Read 5 Sent Done Done false
29417 7766 Acked Read 8 Sent Done Done false
29418 7767 Acked Flush 0 Sent Done Done false
29419 7768 Acked Flush 0 Sent Done Done false
29420 7769 Acked Flush 0 Sent Done Done false
29421 7770 Acked Write 1 Sent Done Done false
29422 7771 Acked Read 10 Sent Done Done false
29423 7772 Acked Read 8 Sent Done Done false
29424 7773 Acked Read 5 Sent Done Done false
29425 7774 Acked Write 3 Sent Done Done false
29426 7775 Acked Read 2 Sent Done Done false
29427 7776 Acked Write 6 Sent Done Done false
29428 7777 Acked Read 9 Sent Done Done false
29429 7778 Acked Read 1 Sent Done Done false
29430 7779 Acked Read 2 Sent Done Done false
29431 7780 Acked Write 8 Sent Done Done false
29432 7781 Acked Write 1 Sent Done Done false
29433 7782 Acked Write 2 Sent Done Done false
29434 7783 Acked Read 3 Sent Done Done false
29435 7784 Acked Read 1 Sent Done Done false
29436 7785 Acked Write 10 Sent Done Done false
29437 7786 Acked Write 9 Sent Done Done false
29438 7787 Acked Read 8 Sent Done Done false
29439 7788 Acked Write 10 Sent Done Done false
29440 7789 Acked Flush 0 Sent Done Done false
29441 7790 Acked Read 8 Sent Done Done false
29442 7791 Acked Read 4 Sent Done Done false
29443 7792 Acked Read 6 Sent Done Done false
29444 7793 Acked Write 3 Sent Done Done false
29445 7794 Acked Write 2 Sent Done Done false
29446 7795 Acked Write 1 Sent Done Done false
29447 7796 Acked Read 4 Sent Done Done false
29448 7797 Acked Write 8 Sent Done Done false
29449 7798 Acked Read 6 Sent Done Done false
29450 7799 Acked Write 2 Sent Done Done false
29451 7800 Acked Write 9 Sent Done Done false
29452 7801 Acked Read 6 Sent Done Done false
29453 7802 Acked Read 8 Sent Done Done false
29454 7803 Acked Write 9 Sent Done Done false
29455 7804 Acked Read 10 Sent Done Done false
29456 7805 Acked Flush 0 Sent Done Done false
29457 7806 Acked Write 9 Sent Done Done false
29458 7807 Acked Read 10 Sent Done Done false
29459 7808 Acked Read 8 Sent Done Done false
29460 7809 Acked Write 2 Sent Done Done false
29461 7810 Acked Flush 0 Sent Done Done false
29462 7811 Acked Write 1 Sent Done Done false
29463 7812 Acked Read 2 Sent Done Done false
29464 7813 Acked Read 10 Sent Done Done false
29465 7814 Acked Flush 0 Sent Done Done false
29466 7815 Acked Read 7 Sent Done Done false
29467 7816 Acked Read 7 Sent Done Done false
29468 7817 Acked Read 10 Sent Done Done false
29469 7818 Acked Read 2 Sent Done Done false
29470 7819 Acked Write 6 Sent Done Done false
29471 7820 Acked Write 3 Sent Done Done false
29472 7821 Acked Read 2 Sent Done Done false
29473 7822 Acked Write 9 Sent Done Done false
29474 7823 Acked Read 3 Sent Done Done false
29475 7824 Acked Write 8 Sent Done Done false
29476 7825 Acked Write 4 Sent Done Done false
29477 7826 Acked Read 5 Sent Done Done false
29478 7827 Acked Write 4 Sent Done Done false
29479 7828 Acked Read 10 Sent Done Done false
29480 7829 Acked Write 8 Sent Done Done false
29481 7830 Acked Read 6 Sent Done Done false
29482 7831 Acked Read 7 Sent Done Done false
29483 7832 Acked Write 2 Sent Done Done false
29484 7833 Acked Read 8 Sent Done Done false
29485 7834 Acked Write 7 Sent Done Done false
29486 7835 Acked Read 6 Sent Done Done false
29487 7836 Acked Read 1 Sent Done Done false
29488 7837 Acked Read 3 Sent Done Done false
29489 7838 Acked Read 2 Sent Done Done false
29490 7839 Acked Write 7 Sent Done Done false
29491 7840 Acked Read 3 Sent Done Done false
29492 7841 Acked Flush 0 Sent Done Done false
29493 7842 Acked Write 9 Sent Done Done false
29494 7843 Acked Read 6 Sent Done Done false
29495 7844 Acked Read 6 Sent Done Done false
29496 7845 Acked Read 8 Sent Done Done false
29497 7846 Acked Read 6 Sent Done Done false
29498 7847 Acked Write 6 Sent Done Done false
29499 7848 Acked Read 4 Sent Done Done false
29500 7849 Acked Read 10 Sent Done Done false
29501 7850 Acked Flush 0 Sent Done Done false
29502 7851 Acked Read 9 Sent Done Done false
29503 7852 Acked Read 1 Sent Done Done false
29504 7853 Acked Read 2 Sent Done Done false
29505 7854 Acked Write 3 Sent Done Done false
29506 7855 Acked Read 3 Sent Done Done false
29507 7856 Acked Flush 0 Sent Done Done false
29508 7857 Acked Write 8 Sent Done Done false
29509 7858 Acked Write 9 Sent Done Done false
29510 7859 Acked Write 7 Sent Done Done false
29511 7860 Acked Read 3 Sent Done Done false
29512 7861 Acked Write 7 Sent Done Done false
29513 7862 Acked Write 9 Sent Done Done false
29514 7863 Acked Flush 0 Sent Done Done false
29515 7864 Acked Write 1 Sent Done Done false
29516 7865 Acked Read 9 Sent Done Done false
29517 7866 Acked Read 7 Sent Done Done false
29518 7867 Acked Write 8 Sent Done Done false
29519 7868 Acked Read 7 Sent Done Done false
29520 7869 Acked Read 9 Sent Done Done false
29521 7870 Acked Read 4 Sent Done Done false
29522 7871 Acked Write 10 Sent Done Done false
29523 7872 Acked Write 10 Sent Done Done false
29524 7873 Acked Write 6 Sent Done Done false
29525 7874 Acked Read 9 Sent Done Done false
29526 7875 Acked Write 5 Sent Done Done false
29527 7876 Acked Flush 0 Sent Done Done false
29528 7877 Acked Read 1 Sent Done Done false
29529 7878 Acked Write 4 Sent Done Done false
29530 7879 Acked Read 7 Sent Done Done false
29531 7880 Acked Read 3 Sent Done Done false
29532 7881 Acked Flush 0 Sent Done Done false
29533 7882 Acked Read 10 Sent Done Done false
29534 7883 Acked Read 5 Sent Done Done false
29535 7884 Acked Write 3 Sent Done Done false
29536 7885 Acked Write 1 Sent Done Done false
29537 7886 Acked Write 4 Sent Done Done false
29538 7887 Acked Flush 0 Sent Done Done false
29539 7888 Acked Read 8 Sent Done Done false
29540 7889 Acked Read 8 Sent Done Done false
29541 7890 Acked Write 10 Sent Done Done false
29542 7891 Acked Read 7 Sent Done Done false
29543 7892 Acked Write 2 Sent Done Done false
29544 7893 Acked Write 3 Sent Done Done false
29545 7894 Acked Flush 0 Sent Done Done false
29546 7895 Acked Read 9 Sent Done Done false
29547 7896 Acked Read 8 Sent Done Done false
29548 7897 Acked Read 8 Sent Done Done false
29549 7898 Acked Read 5 Sent Done Done false
29550 7899 Acked Write 8 Sent Done Done false
29551 7900 Acked Read 1 Sent Done Done false
29552 7901 Acked Read 2 Sent Done Done false
29553 7902 Acked Write 3 Sent Done Done false
29554 7903 Acked Flush 0 Sent Done Done false
29555 7904 Acked Read 7 Sent Done Done false
29556 7905 Acked Flush 0 Sent Done Done false
29557 7906 Acked Write 9 Sent Done Done false
29558 7907 Acked Read 9 Sent Done Done false
29559 7908 Acked Write 9 Sent Done Done false
29560 7909 Acked Read 4 Sent Done Done false
29561 7910 Acked Write 1 Sent Done Done false
29562 7911 Acked Write 4 Sent Done Done false
29563 7912 Acked Read 1 Sent Done Done false
29564 7913 Acked Read 6 Sent Done Done false
29565 7914 Acked Flush 0 Sent Done Done false
29566 7915 Acked Read 8 Sent Done Done false
29567 7916 Acked Read 4 Sent Done Done false
29568 7917 Acked Read 8 Sent Done Done false
29569 7918 Acked Read 5 Sent Done Done false
29570 7919 Acked Flush 0 Sent Done Done false
29571 7920 Acked Write 7 Sent Done Done false
29572 7921 Acked Write 6 Sent Done Done false
29573 7922 Acked Flush 0 Sent Done Done false
29574 7923 Acked Read 6 Sent Done Done false
29575 7924 Acked Read 8 Sent Done Done false
29576 7925 Acked Read 6 Sent Done Done false
29577 7926 Acked Read 3 Sent Done Done false
29578 7927 Acked Read 10 Sent Done Done false
29579 7928 Acked Write 10 Sent Done Done false
29580 7929 Acked Write 4 Sent Done Done false
29581 7930 Acked Read 5 Sent Done Done false
29582 7931 Acked Write 6 Sent Done Done false
29583 7932 Acked Write 8 Sent Done Done false
29584 7933 Acked Flush 0 Sent Done Done false
29585 7934 Acked Read 3 Sent Done Done false
29586 7935 Acked Read 7 Sent Done Done false
29587 7936 Acked Read 5 Sent Done Done false
29588 7937 Acked Read 10 Sent Done Done false
29589 7938 Acked Read 5 Sent Done Done false
29590 7939 Acked Flush 0 Sent Done Done false
29591 7940 Acked Read 8 Sent Done Done false
29592 7941 Acked Read 6 Sent Done Done false
29593 7942 Acked Write 7 Sent Done Done false
29594 7943 Acked Flush 0 Sent Done Done false
29595 STATES DS:0 DS:1 DS:2 TOTAL
29596 Sent 302 0 0 302
29597 Done 0 302 302 604
29598 Skipped 0 0 0 0
29599 Error 0 0 0 0
29600 Last Flush: 7641 7943 7943
29601 Downstairs last five completed: 7641 7640 7639 7638 7637
29602 Upstairs last five completed: 7943 7942 7941 7940 7939
29603 CLIENT: Up:0 ds:302 act:2
296042024-11-15T02:03:41.316ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
296052024-11-15T02:03:41.316ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:36700) = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
296062024-11-15T02:03:41.317ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
296072024-11-15T02:03:41.318ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
296082024-11-15T02:03:41.318ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
296092024-11-15T02:03:41.318ZINFOcrucible: send last flush ID to this DS: 7641 = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
296102024-11-15T02:03:41.318ZINFOcrucible: Replied this last flush ID: 7641 = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
296112024-11-15T02:03:41.318ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
296122024-11-15T02:03:41.318ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
296132024-11-15T02:03:41.318ZINFOcrucible: [0] client re-new 302 jobs since flush 7641 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
296142024-11-15T02:03:41.318ZINFOcrucible: [0] Replayed 302 jobs since flush: 7641 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
29615 ----------------------------------------------------------------
29616 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
29617 Downstairs last five completed: 7943 7942 7941 7940 7939
29618 Upstairs last five completed: 7943 7942 7941 7940 7939
29619 CLIENT: Up:0 ds:0 act:3
29620 Replay: All jobs finished
29621 Main task has work to do, go find it
29622 got dsc Stop(2)
29623 stop 2
29624 Nov 15 02:03:41.580 INFO request completed, latency_us: 349[8830] Got stop action so:false kr:false
29625 , response_code: 204, uri: /stop/cid/2, method: POST, req_id: eeddcf2a-c7b5-4f47-bebe-b1c8baced04a, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
29626 Nov 15 02:03:41.581 INFO accepted connection, remote_addr: 127.0.0.1:33797, local_addr: 127.0.0.1:9998
296272024-11-15T02:03:41.581ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
296282024-11-15T02:03:41.581ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
296292024-11-15T02:03:41.581ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
296302024-11-15T02:03:41.581ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
296312024-11-15T02:03:41.581ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
296322024-11-15T02:03:41.581ZINFOcrucible: client task is exiting = io task client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
29633 Nov 15 02:03:41.581 INFO request completed, latency_us: 245, response_code: 200, uri: /state/cid/2, method: GET, req_id: d47bebbc-d975-450d-8b6e-c1383f207835, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
29634 [8830] Exited with: ExitStatus(unix_wait_status(9))
29635 [8830][2] reports Exit
29636 Nov 15 02:03:45.584 INFO request completed, latency_us: 208, response_code: 200, uri: /state/cid/2, method: GET, req_id: d355bdf9-d9a7-4f88-bd82-8877b073f719, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
29637 001/300 Write block 2829 len 4096 data: 6
29638 002/300 Read block 2618 len 36864
29639 003/300 Write block 2237 len 20480 data: 6 6 5 6 6
29640 004/300 Write block 1082 len 20480 data: 3 4 4 5 4
29641 005/300 Write block 4050 len 16384 data: 3 3 2 2
29642 006/300 Flush
29643 007/300 Read block 768 len 36864
29644 008/300 Read block 1289 len 8192
29645 009/300 Write block 1978 len 16384 data: 2 3 3 4
29646 010/300 Read block 4156 len 12288
29647 011/300 Write block 3721 len 36864 data: 4 4 4 4 4 3 3 2 3
29648 012/300 Read block 4701 len 40960
29649 013/300 Write block 4802 len 40960 data: 4 3 4 3 3 3 3 3 3 4
29650 014/300 Write block 381 len 12288 data: 4 4 4
29651 015/300 Flush
29652 016/300 Write block 4927 len 32768 data: 5 3 3 2 3 3 3 3
29653 017/300 Read block 104 len 32768
29654 018/300 Write block 286 len 24576 data: 4 4 4 4 5 5
29655 019/300 Read block 4255 len 20480
29656 020/300 Write block 1951 len 36864 data: 4 5 6 7 7 6 7 6 5
29657 021/300 Read block 664 len 8192
29658 022/300 Write block 4865 len 20480 data: 8 7 6 4 4
29659 023/300 Write block 2433 len 40960 data: 5 4 4 3 3 3 3 3 5 4
29660 024/300 Write block 3654 len 40960 data: 7 8 6 6 6 6 7 7 7 6
29661 025/300 Write block 1156 len 20480 data: 4 4 4 4 2
29662 026/300 Read block 19 len 40960
29663 027/300 Write block 2400 len 36864 data: 5 6 7 9 9 9 7 6 5
29664 028/300 Read block 4565 len 28672
29665 029/300 Write block 990 len 28672 data: 5 9 9 9 9 8 8
29666 030/300 Write block 2843 len 24576 data: 4 4 4 4 5 6
29667 031/300 Read block 4455 len 8192
29668 032/300 Write block 682 len 36864 data: 7 7 6 6 4 5 4 4 5
29669 033/300 Write block 2994 len 8192 data: 3 3
29670 034/300 Write block 1331 len 20480 data: 3 3 3 3 3
29671 035/300 Write block 4392 len 28672 data: 4 3 5 6 5 4 4
29672 036/300 Read block 708 len 12288
29673 037/300 Write block 3130 len 24576 data: 3 3 3 3 4 4
29674 038/300 Read block 2837 len 36864
29675 039/300 Write block 3238 len 40960 data: 4 3 3 2 3 5 5 7 7 7
29676 040/300 Flush
29677 041/300 Write block 800 len 20480 data: 6 7 6 6 5
29678 042/300 Read block 585 len 16384
29679 043/300 Read block 2450 len 32768
29680 044/300 Read block 1546 len 16384
29681 045/300 Write block 950 len 36864 data: 3 3 3 3 2 2 2 2 2
29682 046/300 Read block 4475 len 16384
29683 047/300 Read block 4236 len 40960
29684 048/300 Write block 1287 len 40960 data: 5 7 5 6 6 5 5 5 7 7
29685 049/300 Read block 4678 len 28672
29686 050/300 Read block 2457 len 8192
29687 051/300 Read block 3783 len 8192
29688 052/300 Read block 1167 len 32768
29689 053/300 Read block 4828 len 40960
29690 054/300 Write block 2054 len 20480 data: 4 5 4 4 4
29691 055/300 Read block 2541 len 16384
29692 056/300 Read block 2623 len 28672
29693 057/300 Read block 467 len 8192
29694 058/300 Write block 851 len 20480 data: 6 8 7 7 6
29695 059/300 Read block 4261 len 36864
29696 060/300 Read block 529 len 32768
29697 061/300 Write block 2903 len 28672 data: 4 4 5 4 3 3 4
29698 062/300 Read block 3660 len 8192
29699 063/300 Read block 101 len 24576
29700 064/300 Read block 1602 len 4096
29701 065/300 Write block 2473 len 12288 data: 10 12 12
29702 066/300 Read block 56 len 36864
29703 067/300 Read block 2083 len 28672
29704 068/300 Flush
29705 069/300 Write block 2699 len 12288 data: 5 6 4
29706 070/300 Flush
29707 071/300 Read block 2186 len 24576
29708 072/300 Write block 1435 len 12288 data: 4 3 3
29709 073/300 Write block 1806 len 4096 data: 6
29710 074/300 Read block 3210 len 40960
29711 075/300 Read block 4551 len 12288
29712 076/300 Write block 2087 len 36864 data: 6 6 5 4 3 4 4 3 3
29713 077/300 Flush
29714 078/300 Write block 4204 len 8192 data: 9 11
29715 079/300 Write block 4833 len 20480 data: 6 6 7 6 7
29716 080/300 Read block 1735 len 32768
29717 081/300 Write block 4534 len 12288 data: 6 6 7
29718 082/300 Read block 3172 len 16384
29719 083/300 Read block 2922 len 4096
29720 084/300 Write block 4965 len 40960 data: 5 7 4 4 4 5 3 3 3 3
29721 085/300 Write block 1585 len 24576 data: 5 5 5 6 5 4
29722 086/300 Write block 4939 len 24576 data: 4 5 5 5 5 5
29723 087/300 Read block 3135 len 36864
29724 088/300 Read block 3442 len 4096
29725 089/300 Read block 1428 len 36864
29726 090/300 Read block 3805 len 4096
29727 091/300 Flush
29728 092/300 Flush
29729 093/300 Flush
29730 094/300 Read block 1297 len 32768
29731 095/300 Read block 433 len 4096
29732 096/300 Read block 2328 len 8192
29733 097/300 Write block 4916 len 4096 data: 4
29734 098/300 Write block 4234 len 12288 data: 6 6 6
29735 099/300 Read block 1440 len 12288
29736 100/300 Read block 604 len 28672
29737 101/300 Read block 2047 len 4096
29738 102/300 Read block 4988 len 24576
29739 103/300 Read block 2904 len 36864
29740 104/300 Read block 2244 len 24576
29741 105/300 Write block 4011 len 8192 data: 5 6
29742 106/300 Write block 694 len 8192 data: 5 4
29743 107/300 Read block 2011 len 12288
29744 108/300 Write block 4623 len 32768 data: 4 4 2 3 2 2 3 3
29745 109/300 Read block 451 len 40960
29746 110/300 Write block 4862 len 12288 data: 9 10 9
29747 111/300 Read block 75 len 20480
29748 112/300 Write block 115 len 40960 data: 3 4 4 3 3 3 4 3 3 3
29749 113/300 Read block 4801 len 36864
29750 114/300 Write block 916 len 8192 data: 3 3
29751 115/300 Write block 3967 len 32768 data: 4 4 4 5 6 6 7 7
29752 116/300 Write block 480 len 20480 data: 7 5 5 5 5
29753 117/300 Write block 293 len 8192 data: 6 7
29754 118/300 Read block 714 len 36864
29755 119/300 Write block 4256 len 20480 data: 6 6 6 8 7
29756 120/300 Write block 162 len 16384 data: 7 7 7 6
29757 121/300 Read block 3717 len 28672
29758 122/300 Read block 3636 len 32768
29759 123/300 Write block 206 len 24576 data: 4 5 5 5 5 6
29760 124/300 Read block 4472 len 20480
29761 125/300 Write block 3893 len 32768 data: 4 6 6 5 7 10 9 10
29762 126/300 Read block 4789 len 8192
29763 127/300 Write block 4989 len 4096 data: 4
29764 128/300 Write block 4900 len 8192 data: 7 8
29765 129/300 Read block 2573 len 8192
29766 130/300 Read block 1692 len 40960
29767 131/300 Read block 2291 len 28672
29768 132/300 Write block 4201 len 24576 data: 6 8 9 10 12 12
29769 133/300 Read block 27 len 24576
29770 134/300 Write block 751 len 4096 data: 2
29771 135/300 Write block 402 len 12288 data: 4 3 3
29772 136/300 Read block 1050 len 20480
29773 137/300 Read block 3276 len 24576
29774 138/300 Write block 1336 len 40960 data: 3 2 3 5 6 6 5 5 6 5
29775 139/300 Read block 916 len 16384
29776 140/300 Read block 2961 len 32768
29777 141/300 Read block 140 len 4096
29778 142/300 Flush
29779 143/300 Read block 1662 len 28672
29780 144/300 Read block 1843 len 28672
29781 145/300 Read block 1430 len 24576
29782 146/300 Write block 1889 len 20480 data: 5 6 6 7 7
29783 147/300 Read block 1004 len 4096
29784 148/300 Write block 2112 len 32768 data: 3 3 2 3 3 4 4 3
29785 149/300 Write block 3019 len 40960 data: 2 2 2 4 4 4 2 3 4 3
29786 150/300 Write block 2977 len 32768 data: 5 6 7 7 8 8 10 9
29787 151/300 Read block 285 len 8192
29788 152/300 Read block 4556 len 4096
29789 153/300 Read block 4265 len 20480
29790 154/300 Read block 1138 len 40960
29791 155/300 Write block 2484 len 8192 data: 5 4
29792 156/300 Read block 4130 len 24576
29793 157/300 Write block 1506 len 8192 data: 4 3
29794 158/300 Read block 4547 len 12288
29795 159/300 Write block 3428 len 4096 data: 2
29796 160/300 Read block 4568 len 36864
29797 161/300 Read block 1662 len 12288
29798 162/300 Write block 592 len 20480 data: 8 9 8 8 6
29799 163/300 Write block 3895 len 40960 data: 7 6 8 11 10 11 11 11 11 10
29800 164/300 Write block 2636 len 20480 data: 3 3 3 3 3
29801 165/300 Write block 1866 len 36864 data: 4 5 4 5 4 5 4 4 5
29802 166/300 Flush
29803 167/300 Write block 179 len 4096 data: 4
29804 168/300 Write block 4222 len 20480 data: 5 6 6 6 6
29805 169/300 Write block 1840 len 4096 data: 6
29806 170/300 Flush
29807 171/300 Read block 4367 len 28672
29808 172/300 Read block 2443 len 20480
29809 173/300 Flush
29810 174/300 Read block 3235 len 24576
29811 175/300 Flush
29812 176/300 Read block 1831 len 12288
29813 177/300 Read block 187 len 8192
29814 178/300 Read block 2626 len 40960
29815 179/300 Read block 277 len 4096
29816 180/300 Write block 4935 len 8192 data: 3 3
29817 181/300 Write block 2379 len 32768 data: 3 4 5 5 7 8 8 7
29818 182/300 Write block 3919 len 28672 data: 4 6 6 6 8 8 6
29819 183/300 Read block 1729 len 24576
29820 184/300 Read block 3328 len 20480
29821 185/300 Read block 244 len 20480
29822 186/300 Read block 1990 len 20480
29823 187/300 Write block 3774 len 24576 data: 5 6 5 4 4 4
29824 188/300 Write block 4730 len 32768 data: 7 6 4 4 4 4 4 4
29825 189/300 Read block 1845 len 8192
29826 190/300 Flush
29827 191/300 Flush
29828 192/300 Read block 4523 len 24576
29829 193/300 Read block 618 len 4096
29830 194/300 Write block 120 len 28672 data: 4 5 4 4 4 3 3
29831 195/300 Flush
29832 196/300 Write block 3103 len 24576 data: 4 3 4 4 5 6
29833 197/300 Write block 2296 len 12288 data: 3 4 4
29834 198/300 Flush
29835 199/300 Read block 2045 len 4096
29836 200/300 Write block 534 len 24576 data: 8 8 5 5 5 5
29837 201/300 Write block 1515 len 16384 data: 6 5 5 4
29838 202/300 Read block 1021 len 40960
29839 203/300 Write block 3953 len 32768 data: 4 5 7 8 7 7 8 7
29840 204/300 Flush
29841 205/300 Read block 120 len 24576
29842 206/300 Write block 3294 len 4096 data: 3
29843 207/300 Flush
29844 208/300 Write block 4307 len 12288 data: 6 8 8
29845 209/300 Write block 1160 len 28672 data: 3 3 2 2 2 2 3
29846 210/300 Read block 3259 len 36864
29847 211/300 Read block 4694 len 8192
29848 212/300 Write block 3721 len 32768 data: 5 5 5 5 5 4 4 3
29849 213/300 Read block 510 len 20480
29850 214/300 Write block 2747 len 4096 data: 5
29851 215/300 Write block 506 len 16384 data: 6 6 6 5
29852 216/300 Write block 2795 len 28672 data: 3 3 3 3 2 2 2
29853 217/300 Write block 1508 len 4096 data: 4
29854 218/300 Read block 1067 len 16384
29855 219/300 Read block 1057 len 16384
29856 220/300 Read block 769 len 32768
29857 221/300 Read block 4340 len 28672
29858 222/300 Flush
29859 223/300 Write block 3862 len 24576 data: 6 6 6 8 8 9
29860 224/300 Write block 3278 len 8192 data: 5 5
29861 225/300 Flush
29862 226/300 Write block 3472 len 36864 data: 4 4 3 4 4 3 3 3 4
29863 227/300 Read block 4417 len 4096
29864 228/300 Read block 2904 len 20480
29865 229/300 Write block 326 len 8192 data: 9 10
29866 230/300 Read block 3341 len 4096
29867 231/300 Read block 1170 len 8192
29868 232/300 Flush
29869 233/300 Write block 3190 len 16384 data: 7 7 7 5
29870 234/300 Read block 2092 len 12288
29871 235/300 Read block 3786 len 8192
29872 236/300 Read block 1246 len 8192
29873 237/300 Flush
29874 238/300 Read block 3492 len 4096
29875 239/300 Read block 661 len 24576
29876 240/300 Write block 1392 len 8192 data: 3 2
29877 241/300 Write block 1248 len 40960 data: 7 6 6 6 5 4 4 4 4 4
29878 242/300 Read block 4773 len 8192
29879 243/300 Write block 3484 len 28672 data: 3 5 5 5 6 7 8
29880 244/300 Read block 3790 len 8192
29881 245/300 Read block 1394 len 12288
29882 246/300 Write block 1541 len 20480 data: 7 6 7 5 4
29883 247/300 Read block 4503 len 16384
29884 248/300 Read block 2290 len 4096
29885 249/300 Write block 3544 len 40960 data: 3 5 5 4 6 6 6 6 8 7
29886 250/300 Flush
29887 251/300 Read block 2078 len 28672
29888 252/300 Read block 2767 len 20480
29889 253/300 Write block 2918 len 40960 data: 4 4 4 4 4 4 4 4 5 4
29890 254/300 Write block 3399 len 36864 data: 3 5 6 6 5 5 5 5 5
29891 255/300 Read block 709 len 16384
29892 256/300 Write block 4069 len 24576 data: 6 6 5 3 5 5
29893 257/300 Read block 123 len 28672
29894 258/300 Read block 2070 len 40960
29895 259/300 Write block 3471 len 4096 data: 4
29896 260/300 Write block 4530 len 28672 data: 9 7 7 7 7 7 8
29897 261/300 Read block 4921 len 36864
29898 262/300 Flush
29899 263/300 Read block 1006 len 8192
29900 264/300 Write block 3535 len 8192 data: 7 7
29901 265/300 Read block 288 len 40960
29902 266/300 Read block 4748 len 20480
29903 267/300 Read block 197 len 8192
29904 268/300 Write block 1382 len 32768 data: 5 3 4 3 3 3 3 3
29905 269/300 Read block 809 len 24576
29906 270/300 Write block 2826 len 20480 data: 6 6 6 7 6
29907 271/300 Read block 4034 len 12288
29908 272/300 Read block 1251 len 12288
29909 273/300 Write block 1243 len 36864 data: 5 5 6 7 7 8 7 7 7
29910 274/300 Write block 2808 len 20480 data: 3 3 5 5 5
29911 275/300 Read block 1112 len 24576
29912 276/300 Read block 990 len 8192
29913 277/300 Write block 3834 len 4096 data: 4
29914 278/300 Flush
29915 279/300 Read block 1164 len 24576
29916 280/300 Write block 3010 len 20480 data: 6 6 5 5 5
29917 281/300 Write block 4310 len 20480 data: 6 7 9 9 8
29918 282/300 Write block 1189 len 12288 data: 5 3 3
29919 283/300 Read block 3820 len 28672
29920 284/300 Read block 3828 len 36864
29921 285/300 Flush
29922 286/300 Write block 4350 len 12288 data: 3 3 3
29923 287/300 Flush
29924 288/300 Read block 1220 len 4096
29925 289/300 Write block 3780 len 16384 data: 4 6 6 4
29926 290/300 Read block 3384 len 28672
29927 291/300 Write block 4389 len 28672 data: 4 4 4 5 4 6 7
29928 292/300 Read block 4993 len 16384
29929 293/300 Write block 1423 len 32768 data: 3 3 4 4 2 2 4 4
29930 294/300 Read block 1998 len 40960
29931 295/300 Flush
29932 296/300 Read block 435 len 40960
29933 297/300 Read block 1406 len 20480
29934 298/300 Read block 3190 len 32768
29935 299/300 Read block 2333 len 20480
29936 300/300 Read block 3781 len 28672
29937 301/300 Read block 4264 len 32768
29938 Main task has work to do, go find it
29939 got dsc Start(2)
29940 start 2
29941 Nov 15 02:03:45.815 INFO request completed, latency_us: 212, response_code: 204, uri: /start/cid/2, method: POST, req_id: 32c68bb8-0ddd-4f8a-927f-deea0d9a3257, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
29942 Replay: started 2, returned:Ok(())
29943 ----------------------------------------------------------------
29944 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
29945 States: Active Active Offline
29946 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
29947 7944 Acked Write 1 Done Done Sent false
29948 7945 Acked Read 9 Done Done Sent false
29949 7946 Acked Write 5 Done Done Sent false
29950 7947 Acked Write 5 Done Done Sent false
29951 7948 Acked Write 4 Done Done Sent false
29952 7949 Acked Flush 0 Done Done Sent false
29953 7950 Acked Read 9 Done Done Sent false
29954 7951 Acked Read 2 Done Done Sent false
29955 7952 Acked Write 4 Done Done Sent false
29956 7953 Acked Read 3 Done Done Sent false
29957 7954 Acked Write 9 Done Done Sent false
29958 7955 Acked Read 10 Done Done Sent false
29959 7956 Acked Write 10 Done Done Sent false
29960 7957 Acked Write 3 Done Done Sent false
29961 7958 Acked Flush 0 Done Done Sent false
29962 7959 Acked Write 8 Done Done Sent false
29963 7960 Acked Read 8 Done Done Sent false
29964 7961 Acked Write 6 Done Done Sent false
29965 7962 Acked Read 5 Done Done Sent false
29966 7963 Acked Write 9 Done Done Sent false
29967 7964 Acked Read 2 Done Done Sent false
29968 7965 Acked Write 5 Done Done Sent false
29969 7966 Acked Write 10 Done Done Sent false
29970 7967 Acked Write 10 Done Done Sent false
29971 7968 Acked Write 5 Done Done Sent false
29972 7969 Acked Read 10 Done Done Sent false
29973 7970 Acked Write 9 Done Done Sent false
29974 7971 Acked Read 7 Done Done Sent false
29975 7972 Acked Write 7 Done Done Sent false
29976 7973 Acked Write 6 Done Done Sent false
29977 7974 Acked Read 2 Done Done Sent false
29978 7975 Acked Write 9 Done Done Sent false
29979 7976 Acked Write 2 Done Done Sent false
29980 7977 Acked Write 5 Done Done Sent false
29981 7978 Acked Write 7 Done Done Sent false
29982 7979 Acked Read 3 Done Done Sent false
29983 7980 Acked Write 6 Done Done Sent false
29984 7981 Acked Read 9 Done Done Sent false
29985 7982 Acked Write 10 Done Done Sent false
29986 7983 Acked Flush 0 Done Done Sent false
29987 7984 Acked Write 5 Done Done Sent false
29988 7985 Acked Read 4 Done Done Sent false
29989 7986 Acked Read 8 Done Done Sent false
29990 7987 Acked Read 4 Done Done Sent false
29991 7988 Acked Write 9 Done Done Sent false
29992 7989 Acked Read 4 Done Done Sent false
29993 7990 Acked Read 10 Done Done Sent false
29994 7991 Acked Write 10 Done Done Sent false
29995 7992 Acked Read 7 Done Done Sent false
29996 7993 Acked Read 2 Done Done Sent false
29997 7994 Acked Read 2 Done Done Sent false
29998 7995 Acked Read 8 Done Done Sent false
29999 7996 Acked Read 10 Done Done Sent false
30000 7997 Acked Write 5 Done Done Sent false
30001 7998 Acked Read 4 Done Done Sent false
30002 7999 Acked Read 7 Done Done Sent false
30003 8000 Acked Read 2 Done Done Sent false
30004 8001 Acked Write 5 Done Done Sent false
30005 8002 Acked Read 9 Done Done Sent false
30006 8003 Acked Read 8 Done Done Sent false
30007 8004 Acked Write 7 Done Done Sent false
30008 8005 Acked Read 2 Done Done Sent false
30009 8006 Acked Read 6 Done Done Sent false
30010 8007 Acked Read 1 Done Done Sent false
30011 8008 Acked Write 3 Done Done Sent false
30012 8009 Acked Read 9 Done Done Sent false
30013 8010 Acked Read 7 Done Done Sent false
30014 8011 Acked Flush 0 Done Done Sent false
30015 8012 Acked Write 3 Done Done Sent false
30016 8013 Acked Flush 0 Done Done Sent false
30017 8014 Acked Read 6 Done Done Sent false
30018 8015 Acked Write 3 Done Done Sent false
30019 8016 Acked Write 1 Done Done Sent false
30020 8017 Acked Read 10 Done Done Sent false
30021 8018 Acked Read 3 Done Done Sent false
30022 8019 Acked Write 9 Done Done Sent false
30023 8020 Acked Flush 0 Done Done Sent false
30024 8021 Acked Write 2 Done Done Sent false
30025 8022 Acked Write 5 Done Done Sent false
30026 8023 Acked Read 8 Done Done Sent false
30027 8024 Acked Write 3 Done Done Sent false
30028 8025 Acked Read 4 Done Done Sent false
30029 8026 Acked Read 1 Done Done Sent false
30030 8027 Acked Write 10 Done Done Sent false
30031 8028 Acked Write 6 Done Done Sent false
30032 8029 Acked Write 6 Done Done Sent false
30033 8030 Acked Read 9 Done Done Sent false
30034 8031 Acked Read 1 Done Done Sent false
30035 8032 Acked Read 9 Done Done Sent false
30036 8033 Acked Read 1 Done Done Sent false
30037 8034 Acked Flush 0 Done Done Sent false
30038 8035 Acked Flush 0 Done Done Sent false
30039 8036 Acked Flush 0 Done Done Sent false
30040 8037 Acked Read 8 Done Done Sent false
30041 8038 Acked Read 1 Done Done Sent false
30042 8039 Acked Read 2 Done Done Sent false
30043 8040 Acked Write 1 Done Done Sent false
30044 8041 Acked Write 3 Done Done Sent false
30045 8042 Acked Read 3 Done Done Sent false
30046 8043 Acked Read 7 Done Done Sent false
30047 8044 Acked Read 1 Done Done Sent false
30048 8045 Acked Read 6 Done Done Sent false
30049 8046 Acked Read 9 Done Done Sent false
30050 8047 Acked Read 6 Done Done Sent false
30051 8048 Acked Write 2 Done Done Sent false
30052 8049 Acked Write 2 Done Done Sent false
30053 8050 Acked Read 3 Done Done Sent false
30054 8051 Acked Write 8 Done Done Sent false
30055 8052 Acked Read 10 Done Done Sent false
30056 8053 Acked Write 3 Done Done Sent false
30057 8054 Acked Read 5 Done Done Sent false
30058 8055 Acked Write 10 Done Done Sent false
30059 8056 Acked Read 9 Done Done Sent false
30060 8057 Acked Write 2 Done Done Sent false
30061 8058 Acked Write 8 Done Done Sent false
30062 8059 Acked Write 5 Done Done Sent false
30063 8060 Acked Write 2 Done Done Sent false
30064 8061 Acked Read 9 Done Done Sent false
30065 8062 Acked Write 5 Done Done Sent false
30066 8063 Acked Write 4 Done Done Sent false
30067 8064 Acked Read 7 Done Done Sent false
30068 8065 Acked Read 8 Done Done Sent false
30069 8066 Acked Write 6 Done Done Sent false
30070 8067 Acked Read 5 Done Done Sent false
30071 8068 Acked Write 8 Done Done Sent false
30072 8069 Acked Read 2 Done Done Sent false
30073 8070 Acked Write 1 Done Done Sent false
30074 8071 Acked Write 2 Done Done Sent false
30075 8072 Acked Read 2 Done Done Sent false
30076 8073 Acked Read 10 Done Done Sent false
30077 8074 Acked Read 7 Done Done Sent false
30078 8075 Acked Write 6 Done Done Sent false
30079 8076 Acked Read 6 Done Done Sent false
30080 8077 Acked Write 1 Done Done Sent false
30081 8078 Acked Write 3 Done Done Sent false
30082 8079 Acked Read 5 Done Done Sent false
30083 8080 Acked Read 6 Done Done Sent false
30084 8081 Acked Write 10 Done Done Sent false
30085 8082 Acked Read 4 Done Done Sent false
30086 8083 Acked Read 8 Done Done Sent false
30087 8084 Acked Read 1 Done Done Sent false
30088 8085 Acked Flush 0 Done Done Sent false
30089 8086 Acked Read 7 Done Done Sent false
30090 8087 Acked Read 7 Done Done Sent false
30091 8088 Acked Read 6 Done Done Sent false
30092 8089 Acked Write 5 Done Done Sent false
30093 8090 Acked Read 1 Done Done Sent false
30094 8091 Acked Write 8 Done Done Sent false
30095 8092 Acked Write 10 Done Done Sent false
30096 8093 Acked Write 8 Done Done Sent false
30097 8094 Acked Read 2 Done Done Sent false
30098 8095 Acked Read 1 Done Done Sent false
30099 8096 Acked Read 5 Done Done Sent false
30100 8097 Acked Read 10 Done Done Sent false
30101 8098 Acked Write 2 Done Done Sent false
30102 8099 Acked Read 6 Done Done Sent false
30103 8100 Acked Write 2 Done Done Sent false
30104 8101 Acked Read 3 Done Done Sent false
30105 8102 Acked Write 1 Done Done Sent false
30106 8103 Acked Read 9 Done Done Sent false
30107 8104 Acked Read 3 Done Done Sent false
30108 8105 Acked Write 5 Done Done Sent false
30109 8106 Acked Write 10 Done Done Sent false
30110 8107 Acked Write 5 Done Done Sent false
30111 8108 Acked Write 9 Done Done Sent false
30112 8109 Acked Flush 0 Done Done Sent false
30113 8110 Acked Write 1 Done Done Sent false
30114 8111 Acked Write 5 Done Done Sent false
30115 8112 Acked Write 1 Done Done Sent false
30116 8113 Acked Flush 0 Done Done Sent false
30117 8114 Acked Read 7 Done Done Sent false
30118 8115 Acked Read 5 Done Done Sent false
30119 8116 Acked Flush 0 Done Done Sent false
30120 8117 Acked Read 6 Done Done Sent false
30121 8118 Acked Flush 0 Done Done Sent false
30122 8119 Acked Read 3 Done Done Sent false
30123 8120 Acked Read 2 Done Done Sent false
30124 8121 Acked Read 10 Done Done Sent false
30125 8122 Acked Read 1 Done Done Sent false
30126 8123 Acked Write 2 Done Done Sent false
30127 8124 Acked Write 8 Done Done Sent false
30128 8125 Acked Write 7 Done Done Sent false
30129 8126 Acked Read 6 Done Done Sent false
30130 8127 Acked Read 5 Done Done Sent false
30131 8128 Acked Read 5 Done Done Sent false
30132 8129 Acked Read 5 Done Done Sent false
30133 8130 Acked Write 6 Done Done Sent false
30134 8131 Acked Write 8 Done Done Sent false
30135 8132 Acked Read 2 Done Done Sent false
30136 8133 Acked Flush 0 Done Done Sent false
30137 8134 Acked Flush 0 Done Done Sent false
30138 8135 Acked Read 6 Done Done Sent false
30139 8136 Acked Read 1 Done Done Sent false
30140 8137 Acked Write 7 Done Done Sent false
30141 8138 Acked Flush 0 Done Done Sent false
30142 8139 Acked Write 6 Done Done Sent false
30143 8140 Acked Write 3 Done Done Sent false
30144 8141 Acked Flush 0 Done Done Sent false
30145 8142 Acked Read 1 Done Done Sent false
30146 8143 Acked Write 6 Done Done Sent false
30147 8144 Acked Write 4 Done Done Sent false
30148 8145 Acked Read 10 Done Done Sent false
30149 8146 Acked Write 8 Done Done Sent false
30150 8147 Acked Flush 0 Done Done Sent false
30151 8148 Acked Read 6 Done Done Sent false
30152 8149 Acked Write 1 Done Done Sent false
30153 8150 Acked Flush 0 Done Done Sent false
30154 8151 Acked Write 3 Done Done Sent false
30155 8152 Acked Write 7 Done Done Sent false
30156 8153 Acked Read 9 Done Done Sent false
30157 8154 Acked Read 2 Done Done Sent false
30158 8155 Acked Write 8 Done Done Sent false
30159 8156 Acked Read 5 Done Done Sent false
30160 8157 Acked Write 1 Done Done Sent false
30161 8158 Acked Write 4 Done Done Sent false
30162 8159 Acked Write 7 Done Done Sent false
30163 8160 Acked Write 1 Done Done Sent false
30164 8161 Acked Read 4 Done Done Sent false
30165 8162 Acked Read 4 Done Done Sent false
30166 8163 Acked Read 8 Done Done Sent false
30167 8164 Acked Read 7 Done Done Sent false
30168 8165 Acked Flush 0 Done Done Sent false
30169 8166 Acked Write 6 Done Done Sent false
30170 8167 Acked Write 2 Done Done Sent false
30171 8168 Acked Flush 0 Done Done Sent false
30172 8169 Acked Write 9 Done Done Sent false
30173 8170 Acked Read 1 Done Done Sent false
30174 8171 Acked Read 5 Done Done Sent false
30175 8172 Acked Write 2 Done Done Sent false
30176 8173 Acked Read 1 Done Done Sent false
30177 8174 Acked Read 2 Done Done Sent false
30178 8175 Acked Flush 0 Done Done Sent false
30179 8176 Acked Write 4 Done Done Sent false
30180 8177 Acked Read 3 Done Done Sent false
30181 8178 Acked Read 2 Done Done Sent false
30182 8179 Acked Read 2 Done Done Sent false
30183 8180 Acked Flush 0 Done Done Sent false
30184 8181 Acked Read 1 Done Done Sent false
30185 8182 Acked Read 6 Done Done Sent false
30186 8183 Acked Write 2 Done Done Sent false
30187 8184 Acked Write 10 Done Done Sent false
30188 8185 Acked Read 2 Done Done Sent false
30189 8186 Acked Write 7 Done Done Sent false
30190 8187 Acked Read 2 Done Done Sent false
30191 8188 Acked Read 3 Done Done Sent false
30192 8189 Acked Write 5 Done Done Sent false
30193 8190 Acked Read 4 Done Done Sent false
30194 8191 Acked Read 1 Done Done Sent false
30195 8192 Acked Write 10 Done Done Sent false
30196 8193 Acked Flush 0 Done Done Sent false
30197 8194 Acked Read 7 Done Done Sent false
30198 8195 Acked Read 5 Done Done Sent false
30199 8196 Acked Write 10 Done Done Sent false
30200 8197 Acked Write 9 Done Done Sent false
30201 8198 Acked Read 4 Done Done Sent false
30202 8199 Acked Write 6 Done Done Sent false
30203 8200 Acked Read 7 Done Done Sent false
30204 8201 Acked Read 10 Done Done Sent false
30205 8202 Acked Write 1 Done Done Sent false
30206 8203 Acked Write 7 Done Done Sent false
30207 8204 Acked Read 9 Done Done Sent false
30208 8205 Acked Flush 0 Done Done Sent false
30209 8206 Acked Read 2 Done Done Sent false
30210 8207 Acked Write 2 Done Done Sent false
30211 8208 Acked Read 10 Done Done Sent false
30212 8209 Acked Read 5 Done Done Sent false
30213 8210 Acked Read 2 Done Done Sent false
30214 8211 Acked Write 8 Done Done Sent false
30215 8212 Acked Read 6 Done Done Sent false
30216 8213 Acked Write 5 Done Done Sent false
30217 8214 Acked Read 3 Done Done Sent false
30218 8215 Acked Read 3 Done Done Sent false
30219 8216 Acked Write 9 Done Done Sent false
30220 8217 Acked Write 5 Done Done Sent false
30221 8218 Acked Read 6 Done Done Sent false
30222 8219 Acked Read 2 Done Done Sent false
30223 8220 Acked Write 1 Done Done Sent false
30224 8221 Acked Flush 0 Done Done Sent false
30225 8222 Acked Read 6 Done Done Sent false
30226 8223 Acked Write 5 Done Done Sent false
30227 8224 Acked Write 5 Done Done Sent false
30228 8225 Acked Write 3 Done Done Sent false
30229 8226 Acked Read 7 Done Done Sent false
30230 8227 Acked Read 9 Done Done Sent false
30231 8228 Acked Flush 0 Done Done Sent false
30232 8229 Acked Write 3 Done Done Sent false
30233 8230 Acked Flush 0 Done Done Sent false
30234 8231 Acked Read 1 Done Done Sent false
30235 8232 Acked Write 4 Done Done Sent false
30236 8233 Acked Read 7 Done Done Sent false
30237 8234 Acked Write 7 Done Done Sent false
30238 8235 Acked Read 4 Done Done Sent false
30239 8236 Acked Write 8 Done Done Sent false
30240 8237 Acked Read 10 Done Done Sent false
30241 8238 Acked Flush 0 Done Done Sent false
30242 8239 Acked Read 10 Done Done Sent false
30243 8240 Acked Read 5 Done Done Sent false
30244 8241 Acked Read 8 Done Done Sent false
30245 8242 Acked Read 5 Done Done Sent false
30246 8243 Acked Read 7 Done Done Sent false
30247 8244 Acked Read 8 Done Done Sent false
30248 STATES DS:0 DS:1 DS:2 TOTAL
30249 Sent 0 0 301 301
30250 Done 301 301 0 602
30251 Skipped 0 0 0 0
30252 Error 0 0 0 0
30253 Last Flush: 8238 8238 7943
30254 Downstairs last five completed: 7943 7942 7941 7940 7939
30255 Upstairs last five completed: 8244 8243 8242 8241 8240
30256 CLIENT: Up:0 ds:301 act:2
30257 [8830] got start action
30258 [8830] I am going to restart
30259 Starting downstairs at port 8830
30260 Make output file at "/tmp/dsc/downstairs-8830.txt"
30261 [8830][2] reports Starting
30262 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1229)
30263 [8830][2] reports Running
30264 ----------------------------------------------------------------
30265 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
30266 States: Active Active Offline
30267 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
30268 7944 Acked Write 1 Done Done Sent false
30269 7945 Acked Read 9 Done Done Sent false
30270 7946 Acked Write 5 Done Done Sent false
30271 7947 Acked Write 5 Done Done Sent false
30272 7948 Acked Write 4 Done Done Sent false
30273 7949 Acked Flush 0 Done Done Sent false
30274 7950 Acked Read 9 Done Done Sent false
30275 7951 Acked Read 2 Done Done Sent false
30276 7952 Acked Write 4 Done Done Sent false
30277 7953 Acked Read 3 Done Done Sent false
30278 7954 Acked Write 9 Done Done Sent false
30279 7955 Acked Read 10 Done Done Sent false
30280 7956 Acked Write 10 Done Done Sent false
30281 7957 Acked Write 3 Done Done Sent false
30282 7958 Acked Flush 0 Done Done Sent false
30283 7959 Acked Write 8 Done Done Sent false
30284 7960 Acked Read 8 Done Done Sent false
30285 7961 Acked Write 6 Done Done Sent false
30286 7962 Acked Read 5 Done Done Sent false
30287 7963 Acked Write 9 Done Done Sent false
30288 7964 Acked Read 2 Done Done Sent false
30289 7965 Acked Write 5 Done Done Sent false
30290 7966 Acked Write 10 Done Done Sent false
30291 7967 Acked Write 10 Done Done Sent false
30292 7968 Acked Write 5 Done Done Sent false
30293 7969 Acked Read 10 Done Done Sent false
30294 7970 Acked Write 9 Done Done Sent false
30295 7971 Acked Read 7 Done Done Sent false
30296 7972 Acked Write 7 Done Done Sent false
30297 7973 Acked Write 6 Done Done Sent false
30298 7974 Acked Read 2 Done Done Sent false
30299 7975 Acked Write 9 Done Done Sent false
30300 7976 Acked Write 2 Done Done Sent false
30301 7977 Acked Write 5 Done Done Sent false
30302 7978 Acked Write 7 Done Done Sent false
30303 7979 Acked Read 3 Done Done Sent false
30304 7980 Acked Write 6 Done Done Sent false
30305 7981 Acked Read 9 Done Done Sent false
30306 7982 Acked Write 10 Done Done Sent false
30307 7983 Acked Flush 0 Done Done Sent false
30308 7984 Acked Write 5 Done Done Sent false
30309 7985 Acked Read 4 Done Done Sent false
30310 7986 Acked Read 8 Done Done Sent false
30311 7987 Acked Read 4 Done Done Sent false
30312 7988 Acked Write 9 Done Done Sent false
30313 7989 Acked Read 4 Done Done Sent false
30314 7990 Acked Read 10 Done Done Sent false
30315 7991 Acked Write 10 Done Done Sent false
30316 7992 Acked Read 7 Done Done Sent false
30317 7993 Acked Read 2 Done Done Sent false
30318 7994 Acked Read 2 Done Done Sent false
30319 7995 Acked Read 8 Done Done Sent false
30320 7996 Acked Read 10 Done Done Sent false
30321 7997 Acked Write 5 Done Done Sent false
30322 7998 Acked Read 4 Done Done Sent false
30323 7999 Acked Read 7 Done Done Sent false
30324 8000 Acked Read 2 Done Done Sent false
30325 8001 Acked Write 5 Done Done Sent false
30326 8002 Acked Read 9 Done Done Sent false
30327 8003 Acked Read 8 Done Done Sent false
30328 8004 Acked Write 7 Done Done Sent false
30329 8005 Acked Read 2 Done Done Sent false
30330 8006 Acked Read 6 Done Done Sent false
30331 8007 Acked Read 1 Done Done Sent false
30332 8008 Acked Write 3 Done Done Sent false
30333 8009 Acked Read 9 Done Done Sent false
30334 8010 Acked Read 7 Done Done Sent false
30335 8011 Acked Flush 0 Done Done Sent false
30336 8012 Acked Write 3 Done Done Sent false
30337 8013 Acked Flush 0 Done Done Sent false
30338 8014 Acked Read 6 Done Done Sent false
30339 8015 Acked Write 3 Done Done Sent false
30340 8016 Acked Write 1 Done Done Sent false
30341 8017 Acked Read 10 Done Done Sent false
30342 8018 Acked Read 3 Done Done Sent false
30343 8019 Acked Write 9 Done Done Sent false
30344 8020 Acked Flush 0 Done Done Sent false
30345 8021 Acked Write 2 Done Done Sent false
30346 8022 Acked Write 5 Done Done Sent false
30347 8023 Acked Read 8 Done Done Sent false
30348 8024 Acked Write 3 Done Done Sent false
30349 8025 Acked Read 4 Done Done Sent false
30350 8026 Acked Read 1 Done Done Sent false
30351 8027 Acked Write 10 Done Done Sent false
30352 8028 Acked Write 6 Done Done Sent false
30353 8029 Acked Write 6 Done Done Sent false
30354 8030 Acked Read 9 Done Done Sent false
30355 8031 Acked Read 1 Done Done Sent false
30356 8032 Acked Read 9 Done Done Sent false
30357 8033 Acked Read 1 Done Done Sent false
30358 8034 Acked Flush 0 Done Done Sent false
30359 8035 Acked Flush 0 Done Done Sent false
30360 8036 Acked Flush 0 Done Done Sent false
30361 8037 Acked Read 8 Done Done Sent false
30362 8038 Acked Read 1 Done Done Sent false
30363 8039 Acked Read 2 Done Done Sent false
30364 8040 Acked Write 1 Done Done Sent false
30365 8041 Acked Write 3 Done Done Sent false
30366 8042 Acked Read 3 Done Done Sent false
30367 8043 Acked Read 7 Done Done Sent false
30368 8044 Acked Read 1 Done Done Sent false
30369 8045 Acked Read 6 Done Done Sent false
30370 8046 Acked Read 9 Done Done Sent false
30371 8047 Acked Read 6 Done Done Sent false
30372 8048 Acked Write 2 Done Done Sent false
30373 8049 Acked Write 2 Done Done Sent false
30374 8050 Acked Read 3 Done Done Sent false
30375 8051 Acked Write 8 Done Done Sent false
30376 8052 Acked Read 10 Done Done Sent false
30377 8053 Acked Write 3 Done Done Sent false
30378 8054 Acked Read 5 Done Done Sent false
30379 8055 Acked Write 10 Done Done Sent false
30380 8056 Acked Read 9 Done Done Sent false
30381 8057 Acked Write 2 Done Done Sent false
30382 8058 Acked Write 8 Done Done Sent false
30383 8059 Acked Write 5 Done Done Sent false
30384 8060 Acked Write 2 Done Done Sent false
30385 8061 Acked Read 9 Done Done Sent false
30386 8062 Acked Write 5 Done Done Sent false
30387 8063 Acked Write 4 Done Done Sent false
30388 8064 Acked Read 7 Done Done Sent false
30389 8065 Acked Read 8 Done Done Sent false
30390 8066 Acked Write 6 Done Done Sent false
30391 8067 Acked Read 5 Done Done Sent false
30392 8068 Acked Write 8 Done Done Sent false
30393 8069 Acked Read 2 Done Done Sent false
30394 8070 Acked Write 1 Done Done Sent false
30395 8071 Acked Write 2 Done Done Sent false
30396 8072 Acked Read 2 Done Done Sent false
30397 8073 Acked Read 10 Done Done Sent false
30398 8074 Acked Read 7 Done Done Sent false
30399 8075 Acked Write 6 Done Done Sent false
30400 8076 Acked Read 6 Done Done Sent false
30401 8077 Acked Write 1 Done Done Sent false
30402 8078 Acked Write 3 Done Done Sent false
30403 8079 Acked Read 5 Done Done Sent false
30404 8080 Acked Read 6 Done Done Sent false
30405 8081 Acked Write 10 Done Done Sent false
30406 8082 Acked Read 4 Done Done Sent false
30407 8083 Acked Read 8 Done Done Sent false
30408 8084 Acked Read 1 Done Done Sent false
30409 8085 Acked Flush 0 Done Done Sent false
30410 8086 Acked Read 7 Done Done Sent false
30411 8087 Acked Read 7 Done Done Sent false
30412 8088 Acked Read 6 Done Done Sent false
30413 8089 Acked Write 5 Done Done Sent false
30414 8090 Acked Read 1 Done Done Sent false
30415 8091 Acked Write 8 Done Done Sent false
30416 8092 Acked Write 10 Done Done Sent false
30417 8093 Acked Write 8 Done Done Sent false
30418 8094 Acked Read 2 Done Done Sent false
30419 8095 Acked Read 1 Done Done Sent false
30420 8096 Acked Read 5 Done Done Sent false
30421 8097 Acked Read 10 Done Done Sent false
30422 8098 Acked Write 2 Done Done Sent false
30423 8099 Acked Read 6 Done Done Sent false
30424 8100 Acked Write 2 Done Done Sent false
30425 8101 Acked Read 3 Done Done Sent false
30426 8102 Acked Write 1 Done Done Sent false
30427 8103 Acked Read 9 Done Done Sent false
30428 8104 Acked Read 3 Done Done Sent false
30429 8105 Acked Write 5 Done Done Sent false
30430 8106 Acked Write 10 Done Done Sent false
30431 8107 Acked Write 5 Done Done Sent false
30432 8108 Acked Write 9 Done Done Sent false
30433 8109 Acked Flush 0 Done Done Sent false
30434 8110 Acked Write 1 Done Done Sent false
30435 8111 Acked Write 5 Done Done Sent false
30436 8112 Acked Write 1 Done Done Sent false
30437 8113 Acked Flush 0 Done Done Sent false
30438 8114 Acked Read 7 Done Done Sent false
30439 8115 Acked Read 5 Done Done Sent false
30440 8116 Acked Flush 0 Done Done Sent false
30441 8117 Acked Read 6 Done Done Sent false
30442 8118 Acked Flush 0 Done Done Sent false
30443 8119 Acked Read 3 Done Done Sent false
30444 8120 Acked Read 2 Done Done Sent false
30445 8121 Acked Read 10 Done Done Sent false
30446 8122 Acked Read 1 Done Done Sent false
30447 8123 Acked Write 2 Done Done Sent false
30448 8124 Acked Write 8 Done Done Sent false
30449 8125 Acked Write 7 Done Done Sent false
30450 8126 Acked Read 6 Done Done Sent false
30451 8127 Acked Read 5 Done Done Sent false
30452 8128 Acked Read 5 Done Done Sent false
30453 8129 Acked Read 5 Done Done Sent false
30454 8130 Acked Write 6 Done Done Sent false
30455 8131 Acked Write 8 Done Done Sent false
30456 8132 Acked Read 2 Done Done Sent false
30457 8133 Acked Flush 0 Done Done Sent false
30458 8134 Acked Flush 0 Done Done Sent false
30459 8135 Acked Read 6 Done Done Sent false
30460 8136 Acked Read 1 Done Done Sent false
30461 8137 Acked Write 7 Done Done Sent false
30462 8138 Acked Flush 0 Done Done Sent false
30463 8139 Acked Write 6 Done Done Sent false
30464 8140 Acked Write 3 Done Done Sent false
30465 8141 Acked Flush 0 Done Done Sent false
30466 8142 Acked Read 1 Done Done Sent false
30467 8143 Acked Write 6 Done Done Sent false
30468 8144 Acked Write 4 Done Done Sent false
30469 8145 Acked Read 10 Done Done Sent false
30470 8146 Acked Write 8 Done Done Sent false
30471 8147 Acked Flush 0 Done Done Sent false
30472 8148 Acked Read 6 Done Done Sent false
30473 8149 Acked Write 1 Done Done Sent false
30474 8150 Acked Flush 0 Done Done Sent false
30475 8151 Acked Write 3 Done Done Sent false
30476 8152 Acked Write 7 Done Done Sent false
30477 8153 Acked Read 9 Done Done Sent false
30478 8154 Acked Read 2 Done Done Sent false
30479 8155 Acked Write 8 Done Done Sent false
30480 8156 Acked Read 5 Done Done Sent false
30481 8157 Acked Write 1 Done Done Sent false
30482 8158 Acked Write 4 Done Done Sent false
30483 8159 Acked Write 7 Done Done Sent false
30484 8160 Acked Write 1 Done Done Sent false
30485 8161 Acked Read 4 Done Done Sent false
30486 8162 Acked Read 4 Done Done Sent false
30487 8163 Acked Read 8 Done Done Sent false
30488 8164 Acked Read 7 Done Done Sent false
30489 8165 Acked Flush 0 Done Done Sent false
30490 8166 Acked Write 6 Done Done Sent false
30491 8167 Acked Write 2 Done Done Sent false
30492 8168 Acked Flush 0 Done Done Sent false
30493 8169 Acked Write 9 Done Done Sent false
30494 8170 Acked Read 1 Done Done Sent false
30495 8171 Acked Read 5 Done Done Sent false
30496 8172 Acked Write 2 Done Done Sent false
30497 8173 Acked Read 1 Done Done Sent false
30498 8174 Acked Read 2 Done Done Sent false
30499 8175 Acked Flush 0 Done Done Sent false
30500 8176 Acked Write 4 Done Done Sent false
30501 8177 Acked Read 3 Done Done Sent false
30502 8178 Acked Read 2 Done Done Sent false
30503 8179 Acked Read 2 Done Done Sent false
30504 8180 Acked Flush 0 Done Done Sent false
30505 8181 Acked Read 1 Done Done Sent false
30506 8182 Acked Read 6 Done Done Sent false
30507 8183 Acked Write 2 Done Done Sent false
30508 8184 Acked Write 10 Done Done Sent false
30509 8185 Acked Read 2 Done Done Sent false
30510 8186 Acked Write 7 Done Done Sent false
30511 8187 Acked Read 2 Done Done Sent false
30512 8188 Acked Read 3 Done Done Sent false
30513 8189 Acked Write 5 Done Done Sent false
30514 8190 Acked Read 4 Done Done Sent false
30515 8191 Acked Read 1 Done Done Sent false
30516 8192 Acked Write 10 Done Done Sent false
30517 8193 Acked Flush 0 Done Done Sent false
30518 8194 Acked Read 7 Done Done Sent false
30519 8195 Acked Read 5 Done Done Sent false
30520 8196 Acked Write 10 Done Done Sent false
30521 8197 Acked Write 9 Done Done Sent false
30522 8198 Acked Read 4 Done Done Sent false
30523 8199 Acked Write 6 Done Done Sent false
30524 8200 Acked Read 7 Done Done Sent false
30525 8201 Acked Read 10 Done Done Sent false
30526 8202 Acked Write 1 Done Done Sent false
30527 8203 Acked Write 7 Done Done Sent false
30528 8204 Acked Read 9 Done Done Sent false
30529 8205 Acked Flush 0 Done Done Sent false
30530 8206 Acked Read 2 Done Done Sent false
30531 8207 Acked Write 2 Done Done Sent false
30532 8208 Acked Read 10 Done Done Sent false
30533 8209 Acked Read 5 Done Done Sent false
30534 8210 Acked Read 2 Done Done Sent false
30535 8211 Acked Write 8 Done Done Sent false
30536 8212 Acked Read 6 Done Done Sent false
30537 8213 Acked Write 5 Done Done Sent false
30538 8214 Acked Read 3 Done Done Sent false
30539 8215 Acked Read 3 Done Done Sent false
30540 8216 Acked Write 9 Done Done Sent false
30541 8217 Acked Write 5 Done Done Sent false
30542 8218 Acked Read 6 Done Done Sent false
30543 8219 Acked Read 2 Done Done Sent false
30544 8220 Acked Write 1 Done Done Sent false
30545 8221 Acked Flush 0 Done Done Sent false
30546 8222 Acked Read 6 Done Done Sent false
30547 8223 Acked Write 5 Done Done Sent false
30548 8224 Acked Write 5 Done Done Sent false
30549 8225 Acked Write 3 Done Done Sent false
30550 8226 Acked Read 7 Done Done Sent false
30551 8227 Acked Read 9 Done Done Sent false
30552 8228 Acked Flush 0 Done Done Sent false
30553 8229 Acked Write 3 Done Done Sent false
30554 8230 Acked Flush 0 Done Done Sent false
30555 8231 Acked Read 1 Done Done Sent false
30556 8232 Acked Write 4 Done Done Sent false
30557 8233 Acked Read 7 Done Done Sent false
30558 8234 Acked Write 7 Done Done Sent false
30559 8235 Acked Read 4 Done Done Sent false
30560 8236 Acked Write 8 Done Done Sent false
30561 8237 Acked Read 10 Done Done Sent false
30562 8238 Acked Flush 0 Done Done Sent false
30563 8239 Acked Read 10 Done Done Sent false
30564 8240 Acked Read 5 Done Done Sent false
30565 8241 Acked Read 8 Done Done Sent false
30566 8242 Acked Read 5 Done Done Sent false
30567 8243 Acked Read 7 Done Done Sent false
30568 8244 Acked Read 8 Done Done Sent false
30569 8245 Acked Flush 0 Done Done Sent false
30570 STATES DS:0 DS:1 DS:2 TOTAL
30571 Sent 0 0 302 302
30572 Done 302 302 0 604
30573 Skipped 0 0 0 0
30574 Error 0 0 0 0
30575 Last Flush: 8245 8245 7943
30576 Downstairs last five completed: 7943 7942 7941 7940 7939
30577 Upstairs last five completed: 8245 8244 8243 8242 8241
30578 CLIENT: Up:0 ds:302 act:2
305792024-11-15T02:03:51.583ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
305802024-11-15T02:03:51.583ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:51853) = io task client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
305812024-11-15T02:03:51.584ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
305822024-11-15T02:03:51.584ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
305832024-11-15T02:03:51.584ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
305842024-11-15T02:03:51.585ZINFOcrucible: send last flush ID to this DS: 7943 = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
305852024-11-15T02:03:51.585ZINFOcrucible: Replied this last flush ID: 7943 = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
305862024-11-15T02:03:51.585ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
305872024-11-15T02:03:51.585ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
305882024-11-15T02:03:51.585ZINFOcrucible: [2] client re-new 302 jobs since flush 7943 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
305892024-11-15T02:03:51.585ZINFOcrucible: [2] Replayed 302 jobs since flush: 7943 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
30590 ----------------------------------------------------------------
30591 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
30592 Downstairs last five completed: 8245 8244 8243 8242 8241
30593 Upstairs last five completed: 8245 8244 8243 8242 8241
30594 CLIENT: Up:0 ds:0 act:3
30595 Replay: All jobs finished
30596 Main task has work to do, go find it
30597 got dsc Stop(1)
30598 stop 1
30599 Nov 15 02:03:53.822 INFO request completed, latency_us: 359, response_code: 204[8820] Got stop action so:false kr:false
30600 , uri: /stop/cid/1, method: POST, req_id: b45741ed-cbee-42fb-8ea6-59612c5d4e80, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
306012024-11-15T02:03:53.823ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
30602 Nov 15 02:03:53.823 INFO request completed, latency_us: 129, response_code: 200, uri: /state/cid/1, method: GET, req_id: e0a67bb8-13dc-4b8d-b7f6-7882cf429b4d, remote_addr: 127{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2024-11-15T02:03:53.823403332Z","hostname":"w-01JCPRHT5F2MPA44KVJHM3S2PB","pid":1205,"":"io task","client":"1","":"downstairs","session_id":"971dcb2e-6891-4ecf-b367-1e04e52065ee"}
30603 .0.0.1:40008, local_addr: 127.0.0.1:9998
306042024-11-15T02:03:53.823ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
306052024-11-15T02:03:53.823ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
306062024-11-15T02:03:53.823ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
306072024-11-15T02:03:53.823ZINFOcrucible: client task is exiting = io task client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
30608 [8820] Exited with: ExitStatus(unix_wait_status(9))
30609 [8820][1] reports Exit
30610 Nov 15 02:03:57.825 INFO request completed, latency_us: 228, response_code: 200, uri: /state/cid/1, method: GET, req_id: e906c718-6343-4e55-ba3e-40198a86afc1, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
30611 001/300 Read block 2892 len 32768
30612 002/300 Write block 1680 len 28672 data: 7 8 6 6 8 7 7
30613 003/300 Write block 2713 len 36864 data: 3 4 3 4 3 3 3 3 3
30614 004/300 Read block 650 len 36864
30615 005/300 Read block 1574 len 28672
30616 006/300 Read block 2885 len 24576
30617 007/300 Read block 2202 len 12288
30618 008/300 Read block 3040 len 4096
30619 009/300 Read block 4622 len 12288
30620 010/300 Read block 4859 len 20480
30621 011/300 Write block 180 len 16384 data: 4 4 4 2
30622 012/300 Read block 2701 len 8192
30623 013/300 Write block 2283 len 32768 data: 7 8 9 8 7 7 5 3
30624 014/300 Write block 1948 len 4096 data: 2
30625 015/300 Write block 2335 len 16384 data: 4 7 7 5
30626 016/300 Write block 1445 len 8192 data: 4 5
30627 017/300 Flush
30628 018/300 Write block 1336 len 40960 data: 4 3 4 6 7 7 6 6 7 6
30629 019/300 Read block 4693 len 12288
30630 020/300 Flush
30631 021/300 Read block 477 len 20480
30632 022/300 Write block 2592 len 20480 data: 6 5 6 6 7
30633 023/300 Write block 1581 len 16384 data: 5 4 5 5
30634 024/300 Read block 3036 len 36864
30635 025/300 Write block 2718 len 4096 data: 4
30636 026/300 Read block 2318 len 12288
30637 027/300 Read block 3470 len 12288
30638 028/300 Flush
30639 029/300 Read block 2537 len 36864
30640 030/300 Read block 1162 len 36864
30641 031/300 Write block 3030 len 40960 data: 3 3 4 5 6 6 6 6 6 6
30642 032/300 Write block 2665 len 24576 data: 3 4 2 2 3 4
30643 033/300 Flush
30644 034/300 Flush
30645 035/300 Read block 4721 len 16384
30646 036/300 Read block 4455 len 36864
30647 037/300 Flush
30648 038/300 Read block 1766 len 40960
30649 039/300 Write block 4361 len 40960 data: 4 3 3 3 3 5 6 5 4 5
30650 040/300 Read block 4145 len 24576
30651 041/300 Write block 1486 len 32768 data: 3 3 3 3 3 5 4 3
30652 042/300 Write block 3452 len 40960 data: 4 3 3 3 4 4 4 3 3 3
30653 043/300 Read block 3381 len 16384
30654 044/300 Read block 1423 len 16384
30655 045/300 Write block 4345 len 36864 data: 2 2 3 3 3 4 4 4 4
30656 046/300 Flush
30657 047/300 Read block 3467 len 28672
30658 048/300 Write block 2475 len 12288 data: 13 10 11
30659 049/300 Write block 2124 len 36864 data: 4 4 4 4 4 5 7 6 6
30660 050/300 Flush
30661 051/300 Read block 9 len 8192
30662 052/300 Read block 4608 len 16384
30663 053/300 Write block 4500 len 4096 data: 4
30664 054/300 Write block 277 len 28672 data: 3 3 3 2 3 3 3
30665 055/300 Write block 776 len 28672 data: 3 3 3 4 4 6 6
30666 056/300 Read block 4415 len 32768
30667 057/300 Read block 3641 len 20480
30668 058/300 Write block 4927 len 16384 data: 6 4 4 3
30669 059/300 Read block 462 len 28672
30670 060/300 Write block 4255 len 24576 data: 6 7 7 7 9 8
30671 061/300 Flush
30672 062/300 Write block 1286 len 36864 data: 6 6 8 6 7 7 6 6 6
30673 063/300 Write block 641 len 36864 data: 3 3 2 2 2 2 2 4 5
30674 064/300 Read block 564 len 28672
30675 065/300 Write block 2963 len 8192 data: 6 5
30676 066/300 Flush
30677 067/300 Write block 1512 len 32768 data: 5 6 6 7 6 6 5 4
30678 068/300 Write block 1246 len 20480 data: 8 8 9 8 8
30679 069/300 Write block 4443 len 32768 data: 5 6 6 6 5 5 3 3
30680 070/300 Read block 2991 len 16384
30681 071/300 Write block 4468 len 4096 data: 4
30682 072/300 Write block 902 len 12288 data: 6 6 6
30683 073/300 Read block 1144 len 8192
30684 074/300 Read block 2777 len 4096
30685 075/300 Write block 2363 len 32768 data: 5 4 7 6 6 6 3 2
30686 076/300 Write block 4567 len 20480 data: 5 5 6 5 4
30687 077/300 Flush
30688 078/300 Read block 3595 len 12288
30689 079/300 Read block 441 len 28672
30690 080/300 Read block 4774 len 8192
30691 081/300 Read block 2744 len 24576
30692 082/300 Flush
30693 083/300 Read block 916 len 36864
30694 084/300 Write block 1317 len 40960 data: 6 2 2 3 3 4 4 4 5 5
30695 085/300 Read block 4760 len 4096
30696 086/300 Read block 4579 len 40960
30697 087/300 Write block 3533 len 4096 data: 9
30698 088/300 Write block 1757 len 24576 data: 4 6 7 6 6 7
30699 089/300 Write block 3408 len 16384 data: 7 8 5 4
30700 090/300 Read block 3254 len 28672
30701 091/300 Write block 2246 len 16384 data: 5 5 5 5
30702 092/300 Read block 3336 len 36864
30703 093/300 Read block 4231 len 24576
30704 094/300 Read block 888 len 28672
30705 095/300 Read block 304 len 24576
30706 096/300 Read block 3818 len 8192
30707 097/300 Write block 4144 len 24576 data: 2 2 3 3 4 6
30708 098/300 Read block 957 len 28672
30709 099/300 Read block 3829 len 4096
30710 100/300 Read block 657 len 16384
30711 101/300 Write block 4865 len 8192 data: 9 8
30712 102/300 Read block 481 len 16384
30713 103/300 Read block 3193 len 12288
30714 104/300 Read block 1497 len 12288
30715 105/300 Write block 1411 len 28672 data: 5 5 5 4 2 2 4
30716 106/300 Read block 4043 len 36864
30717 107/300 Read block 4165 len 20480
30718 108/300 Flush
30719 109/300 Flush
30720 110/300 Write block 1700 len 40960 data: 6 6 7 6 4 4 4 3 3 2
30721 111/300 Write block 2312 len 8192 data: 7 6
30722 112/300 Write block 926 len 36864 data: 3 3 4 4 6 6 4 4 3
30723 113/300 Write block 1801 len 28672 data: 4 5 5 4 5 7 6
30724 114/300 Read block 166 len 36864
30725 115/300 Read block 1941 len 36864
30726 116/300 Write block 2316 len 4096 data: 6
30727 117/300 Read block 4423 len 28672
30728 118/300 Write block 2282 len 12288 data: 6 8 9
30729 119/300 Write block 2022 len 20480 data: 6 5 8 10 10
30730 120/300 Write block 140 len 8192 data: 4 4
30731 121/300 Write block 4199 len 24576 data: 8 7 7 9 10 11
30732 122/300 Read block 897 len 16384
30733 123/300 Read block 2336 len 4096
30734 124/300 Read block 3041 len 8192
30735 125/300 Write block 3326 len 24576 data: 5 6 6 6 7 5
30736 126/300 Flush
30737 127/300 Write block 289 len 24576 data: 5 6 6 6 7 8
30738 128/300 Write block 966 len 32768 data: 5 6 5 5 6 4 4 2
30739 129/300 Read block 4762 len 20480
30740 130/300 Read block 1938 len 20480
30741 131/300 Read block 2298 len 12288
30742 132/300 Write block 285 len 28672 data: 4 5 5 5 6 7 7
30743 133/300 Write block 1015 len 40960 data: 8 7 7 7 7 6 7 5 5 6
30744 134/300 Read block 805 len 32768
30745 135/300 Write block 3012 len 12288 data: 6 6 6
30746 136/300 Flush
30747 137/300 Read block 3468 len 8192
30748 138/300 Read block 4912 len 8192
30749 139/300 Read block 3528 len 32768
30750 140/300 Read block 545 len 32768
30751 141/300 Read block 4682 len 36864
30752 142/300 Read block 177 len 12288
30753 143/300 Write block 3262 len 4096 data: 6
30754 144/300 Read block 413 len 8192
30755 145/300 Write block 1731 len 4096 data: 3
30756 146/300 Read block 2052 len 28672
30757 147/300 Write block 2821 len 24576 data: 7 7 7 7 6 7
30758 148/300 Read block 1268 len 20480
30759 149/300 Read block 2503 len 16384
30760 150/300 Read block 4691 len 32768
30761 151/300 Write block 941 len 36864 data: 5 3 4 3 3 3 4 3 3
30762 152/300 Read block 866 len 36864
30763 153/300 Read block 4863 len 8192
30764 154/300 Read block 2474 len 32768
30765 155/300 Write block 1929 len 40960 data: 6 6 6 8 8 9 6 6 5 7
30766 156/300 Write block 1391 len 12288 data: 4 4 3
30767 157/300 Write block 4599 len 12288 data: 4 5 6
30768 158/300 Write block 4915 len 40960 data: 3 5 4 6 5 7 8 9 9 7
30769 159/300 Read block 3425 len 24576
30770 160/300 Read block 528 len 8192
30771 161/300 Write block 3031 len 28672 data: 4 5 6 7 7 7 7
30772 162/300 Read block 980 len 40960
30773 163/300 Read block 4015 len 4096
30774 164/300 Read block 969 len 16384
30775 165/300 Read block 3533 len 36864
30776 166/300 Write block 1610 len 4096 data: 2
30777 167/300 Write block 3167 len 8192 data: 5 5
30778 168/300 Read block 1560 len 4096
30779 169/300 Read block 3826 len 8192
30780 170/300 Write block 449 len 12288 data: 3 3 5
30781 171/300 Write block 2510 len 28672 data: 4 3 2 2 4 5 6
30782 172/300 Write block 2683 len 24576 data: 4 3 5 6 6 7
30783 173/300 Write block 4646 len 16384 data: 6 5 4 5
30784 174/300 Read block 24 len 16384
30785 175/300 Read block 2199 len 24576
30786 176/300 Read block 1372 len 20480
30787 177/300 Write block 1204 len 8192 data: 3 2
30788 178/300 Write block 2365 len 4096 data: 8
30789 179/300 Read block 3651 len 4096
30790 180/300 Flush
30791 181/300 Read block 2242 len 20480
30792 182/300 Read block 3829 len 8192
30793 183/300 Flush
30794 184/300 Write block 1326 len 36864 data: 6 5 4 3 3 4 4 4 4
30795 185/300 Write block 133 len 8192 data: 4 4
30796 186/300 Read block 3393 len 40960
30797 187/300 Read block 4252 len 8192
30798 188/300 Read block 3072 len 36864
30799 189/300 Read block 2741 len 4096
30800 190/300 Write block 3070 len 8192 data: 2 2
30801 191/300 Write block 622 len 40960 data: 8 10 9 7 5 5 5 4 5 6
30802 192/300 Read block 4074 len 32768
30803 193/300 Read block 2124 len 36864
30804 194/300 Flush
30805 195/300 Write block 2570 len 8192 data: 6 5
30806 196/300 Write block 1160 len 36864 data: 4 4 3 3 3 3 4 4 4
30807 197/300 Write block 2711 len 28672 data: 3 3 4 5 4 5 4
30808 198/300 Write block 3809 len 12288 data: 7 5 7
30809 199/300 Read block 3822 len 20480
30810 200/300 Read block 3686 len 12288
30811 201/300 Write block 3020 len 4096 data: 3
30812 202/300 Read block 4366 len 24576
30813 203/300 Write block 1716 len 16384 data: 3 4 3 3
30814 204/300 Write block 1400 len 40960 data: 3 4 4 4 5 8 7 6 6 4
30815 205/300 Write block 3765 len 16384 data: 5 5 4 4
30816 206/300 Write block 4530 len 4096 data: 10
30817 207/300 Read block 203 len 12288
30818 208/300 Write block 4063 len 16384 data: 7 7 8 7
30819 209/300 Read block 251 len 16384
30820 210/300 Read block 4619 len 24576
30821 211/300 Read block 4810 len 16384
30822 212/300 Read block 1644 len 4096
30823 213/300 Write block 1147 len 28672 data: 8 7 5 4 3 3 3
30824 214/300 Read block 179 len 8192
30825 215/300 Write block 4678 len 16384 data: 4 5 5 4
30826 216/300 Read block 2530 len 20480
30827 217/300 Read block 967 len 28672
30828 218/300 Read block 1801 len 20480
30829 219/300 Read block 4028 len 16384
30830 220/300 Write block 561 len 36864 data: 5 6 5 3 2 2 3 4 3
30831 221/300 Write block 4067 len 4096 data: 7
30832 222/300 Read block 3107 len 12288
30833 223/300 Read block 272 len 32768
30834 224/300 Read block 4159 len 12288
30835 225/300 Read block 3815 len 16384
30836 226/300 Write block 4291 len 32768 data: 5 4 4 3 3 3 4 7
30837 227/300 Write block 3124 len 32768 data: 3 3 3 3 3 3 4 4
30838 228/300 Read block 1426 len 20480
30839 229/300 Read block 4187 len 28672
30840 230/300 Read block 774 len 20480
30841 231/300 Write block 3363 len 8192 data: 3 4
30842 232/300 Read block 3393 len 28672
30843 233/300 Write block 4939 len 8192 data: 5 6
30844 234/300 Write block 3935 len 36864 data: 6 7 4 7 7 6 4 4 4
30845 235/300 Read block 4394 len 8192
30846 236/300 Write block 1454 len 28672 data: 4 6 5 4 5 5 5
30847 237/300 Write block 2633 len 8192 data: 6 4
30848 238/300 Write block 2803 len 12288 data: 4 4 3
30849 239/300 Write block 1830 len 32768 data: 5 6 5 4 5 6 6 7
30850 240/300 Flush
30851 241/300 Read block 1424 len 4096
30852 242/300 Read block 1029 len 16384
30853 243/300 Write block 197 len 40960 data: 5 5 5 6 6 5 5 4 5 5
30854 244/300 Read block 2439 len 16384
30855 245/300 Write block 4708 len 28672 data: 8 8 8 7 4 5 5
30856 246/300 Write block 3831 len 16384 data: 3 5 4 5
30857 247/300 Read block 4346 len 24576
30858 248/300 Write block 2251 len 40960 data: 4 5 4 6 7 6 6 5 4 4
30859 249/300 Write block 2304 len 32768 data: 7 6 6 8 9 9 7 7
30860 250/300 Write block 4494 len 20480 data: 7 7 5 4 4
30861 251/300 Write block 3227 len 24576 data: 5 5 5 4 4 6
30862 252/300 Flush
30863 253/300 Flush
30864 254/300 Write block 4125 len 4096 data: 8
30865 255/300 Write block 3835 len 24576 data: 5 7 7 7 8 8
30866 256/300 Write block 3651 len 16384 data: 10 9 9 8
30867 257/300 Write block 4716 len 8192 data: 5 4
30868 258/300 Write block 1304 len 36864 data: 5 4 4 5 6 6 6 4 4
30869 259/300 Read block 822 len 40960
30870 260/300 Write block 2190 len 32768 data: 4 3 4 4 3 2 3 3
30871 261/300 Write block 661 len 16384 data: 7 6 7 5
30872 262/300 Read block 2613 len 12288
30873 263/300 Write block 1606 len 16384 data: 7 5 4 3
30874 264/300 Write block 35 len 16384 data: 5 5 5 7
30875 265/300 Write block 4800 len 4096 data: 4
30876 266/300 Write block 4995 len 16384 data: 2 2 2 3
30877 267/300 Read block 4901 len 16384
30878 268/300 Read block 2872 len 8192
30879 269/300 Flush
30880 270/300 Write block 506 len 20480 data: 7 7 7 6 4
30881 271/300 Write block 2829 len 20480 data: 8 7 5 5 6
30882 272/300 Write block 1063 len 40960 data: 4 4 5 6 7 8 7 6 6 8
30883 273/300 Write block 4245 len 32768 data: 4 5 5 5 4 5 6 6
30884 274/300 Write block 4526 len 40960 data: 4 5 8 9 11 8 8 8 8 8
30885 275/300 Read block 1205 len 20480
30886 276/300 Flush
30887 277/300 Flush
30888 278/300 Read block 3467 len 12288
30889 279/300 Read block 2528 len 4096
30890 280/300 Write block 1017 len 32768 data: 8 8 8 7 8 6 6 7
30891 281/300 Read block 2388 len 12288
30892 282/300 Read block 963 len 4096
30893 283/300 Read block 123 len 36864
30894 284/300 Write block 3933 len 40960 data: 5 6 7 8 5 8 8 7 5 5
30895 285/300 Read block 3718 len 28672
30896 286/300 Read block 650 len 20480
30897 287/300 Write block 2744 len 20480 data: 5 6 6 6 4
30898 288/300 Read block 3220 len 24576
30899 289/300 Read block 3031 len 32768
30900 290/300 Write block 1305 len 32768 data: 5 5 6 7 7 7 5 5
30901 291/300 Read block 4000 len 20480
30902 292/300 Read block 3551 len 4096
30903 293/300 Flush
30904 294/300 Write block 3722 len 32768 data: 6 6 6 6 5 5 4 4
30905 295/300 Read block 3287 len 8192
30906 296/300 Write block 4194 len 28672 data: 7 8 9 10 8 9 8
30907 297/300 Write block 962 len 28672 data: 4 5 5 5 6 7 6
30908 298/300 Flush
30909 299/300 Flush
30910 300/300 Read block 3063 len 36864
30911 301/300 Read block 1401 len 8192
30912 Main task has work to do, go find it
30913 got dsc Start(1)
30914 start 1
30915 Nov 15 02:03:58.049 INFO request completed, latency_us: 228, response_code: 204, uri: /start/cid/1, method: POST, req_id: 780f9fc5-e853-4884-84ad-7b7677fb04db, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
30916 Replay: started 1, returned:Ok(())
30917 ----------------------------------------------------------------
30918 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
30919 States: Active Offline Active
30920 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
30921 8246 Acked Read 8 Done Sent Done false
30922 8247 Acked Write 7 Done Sent Done false
30923 8248 Acked Write 9 Done Sent Done false
30924 8249 Acked Read 9 Done Sent Done false
30925 8250 Acked Read 7 Done Sent Done false
30926 8251 Acked Read 6 Done Sent Done false
30927 8252 Acked Read 3 Done Sent Done false
30928 8253 Acked Read 1 Done Sent Done false
30929 8254 Acked Read 3 Done Sent Done false
30930 8255 Acked Read 5 Done Sent Done false
30931 8256 Acked Write 4 Done Sent Done false
30932 8257 Acked Read 2 Done Sent Done false
30933 8258 Acked Write 8 Done Sent Done false
30934 8259 Acked Write 1 Done Sent Done false
30935 8260 Acked Write 4 Done Sent Done false
30936 8261 Acked Write 2 Done Sent Done false
30937 8262 Acked Flush 0 Done Sent Done false
30938 8263 Acked Write 10 Done Sent Done false
30939 8264 Acked Read 3 Done Sent Done false
30940 8265 Acked Flush 0 Done Sent Done false
30941 8266 Acked Read 5 Done Sent Done false
30942 8267 Acked Write 5 Done Sent Done false
30943 8268 Acked Write 4 Done Sent Done false
30944 8269 Acked Read 9 Done Sent Done false
30945 8270 Acked Write 1 Done Sent Done false
30946 8271 Acked Read 3 Done Sent Done false
30947 8272 Acked Read 3 Done Sent Done false
30948 8273 Acked Flush 0 Done Sent Done false
30949 8274 Acked Read 9 Done Sent Done false
30950 8275 Acked Read 9 Done Sent Done false
30951 8276 Acked Write 10 Done Sent Done false
30952 8277 Acked Write 6 Done Sent Done false
30953 8278 Acked Flush 0 Done Sent Done false
30954 8279 Acked Flush 0 Done Sent Done false
30955 8280 Acked Read 4 Done Sent Done false
30956 8281 Acked Read 9 Done Sent Done false
30957 8282 Acked Flush 0 Done Sent Done false
30958 8283 Acked Read 10 Done Sent Done false
30959 8284 Acked Write 10 Done Sent Done false
30960 8285 Acked Read 6 Done Sent Done false
30961 8286 Acked Write 8 Done Sent Done false
30962 8287 Acked Write 10 Done Sent Done false
30963 8288 Acked Read 4 Done Sent Done false
30964 8289 Acked Read 4 Done Sent Done false
30965 8290 Acked Write 9 Done Sent Done false
30966 8291 Acked Flush 0 Done Sent Done false
30967 8292 Acked Read 7 Done Sent Done false
30968 8293 Acked Write 3 Done Sent Done false
30969 8294 Acked Write 9 Done Sent Done false
30970 8295 Acked Flush 0 Done Sent Done false
30971 8296 Acked Read 2 Done Sent Done false
30972 8297 Acked Read 4 Done Sent Done false
30973 8298 Acked Write 1 Done Sent Done false
30974 8299 Acked Write 7 Done Sent Done false
30975 8300 Acked Write 7 Done Sent Done false
30976 8301 Acked Read 8 Done Sent Done false
30977 8302 Acked Read 5 Done Sent Done false
30978 8303 Acked Write 4 Done Sent Done false
30979 8304 Acked Read 7 Done Sent Done false
30980 8305 Acked Write 6 Done Sent Done false
30981 8306 Acked Flush 0 Done Sent Done false
30982 8307 Acked Write 9 Done Sent Done false
30983 8308 Acked Write 9 Done Sent Done false
30984 8309 Acked Read 7 Done Sent Done false
30985 8310 Acked Write 2 Done Sent Done false
30986 8311 Acked Flush 0 Done Sent Done false
30987 8312 Acked Write 8 Done Sent Done false
30988 8313 Acked Write 5 Done Sent Done false
30989 8314 Acked Write 8 Done Sent Done false
30990 8315 Acked Read 4 Done Sent Done false
30991 8316 Acked Write 1 Done Sent Done false
30992 8317 Acked Write 3 Done Sent Done false
30993 8318 Acked Read 2 Done Sent Done false
30994 8319 Acked Read 1 Done Sent Done false
30995 8320 Acked Write 8 Done Sent Done false
30996 8321 Acked Write 5 Done Sent Done false
30997 8322 Acked Flush 0 Done Sent Done false
30998 8323 Acked Read 3 Done Sent Done false
30999 8324 Acked Read 7 Done Sent Done false
31000 8325 Acked Read 2 Done Sent Done false
31001 8326 Acked Read 6 Done Sent Done false
31002 8327 Acked Flush 0 Done Sent Done false
31003 8328 Acked Read 9 Done Sent Done false
31004 8329 Acked Write 10 Done Sent Done false
31005 8330 Acked Read 1 Done Sent Done false
31006 8331 Acked Read 10 Done Sent Done false
31007 8332 Acked Write 1 Done Sent Done false
31008 8333 Acked Write 6 Done Sent Done false
31009 8334 Acked Write 4 Done Sent Done false
31010 8335 Acked Read 7 Done Sent Done false
31011 8336 Acked Write 4 Done Sent Done false
31012 8337 Acked Read 9 Done Sent Done false
31013 8338 Acked Read 6 Done Sent Done false
31014 8339 Acked Read 7 Done Sent Done false
31015 8340 Acked Read 6 Done Sent Done false
31016 8341 Acked Read 2 Done Sent Done false
31017 8342 Acked Write 6 Done Sent Done false
31018 8343 Acked Read 7 Done Sent Done false
31019 8344 Acked Read 1 Done Sent Done false
31020 8345 Acked Read 4 Done Sent Done false
31021 8346 Acked Write 2 Done Sent Done false
31022 8347 Acked Read 4 Done Sent Done false
31023 8348 Acked Read 3 Done Sent Done false
31024 8349 Acked Read 3 Done Sent Done false
31025 8350 Acked Write 7 Done Sent Done false
31026 8351 Acked Read 9 Done Sent Done false
31027 8352 Acked Read 5 Done Sent Done false
31028 8353 Acked Flush 0 Done Sent Done false
31029 8354 Acked Flush 0 Done Sent Done false
31030 8355 Acked Write 10 Done Sent Done false
31031 8356 Acked Write 2 Done Sent Done false
31032 8357 Acked Write 9 Done Sent Done false
31033 8358 Acked Write 7 Done Sent Done false
31034 8359 Acked Read 9 Done Sent Done false
31035 8360 Acked Read 9 Done Sent Done false
31036 8361 Acked Write 1 Done Sent Done false
31037 8362 Acked Read 7 Done Sent Done false
31038 8363 Acked Write 3 Done Sent Done false
31039 8364 Acked Write 5 Done Sent Done false
31040 8365 Acked Write 2 Done Sent Done false
31041 8366 Acked Write 6 Done Sent Done false
31042 8367 Acked Read 4 Done Sent Done false
31043 8368 Acked Read 1 Done Sent Done false
31044 8369 Acked Read 2 Done Sent Done false
31045 8370 Acked Write 6 Done Sent Done false
31046 8371 Acked Flush 0 Done Sent Done false
31047 8372 Acked Write 6 Done Sent Done false
31048 8373 Acked Write 8 Done Sent Done false
31049 8374 Acked Read 5 Done Sent Done false
31050 8375 Acked Read 5 Done Sent Done false
31051 8376 Acked Read 3 Done Sent Done false
31052 8377 Acked Write 7 Done Sent Done false
31053 8378 Acked Write 10 Done Sent Done false
31054 8379 Acked Read 8 Done Sent Done false
31055 8380 Acked Write 3 Done Sent Done false
31056 8381 Acked Flush 0 Done Sent Done false
31057 8382 Acked Read 2 Done Sent Done false
31058 8383 Acked Read 2 Done Sent Done false
31059 8384 Acked Read 8 Done Sent Done false
31060 8385 Acked Read 8 Done Sent Done false
31061 8386 Acked Read 9 Done Sent Done false
31062 8387 Acked Read 3 Done Sent Done false
31063 8388 Acked Write 1 Done Sent Done false
31064 8389 Acked Read 2 Done Sent Done false
31065 8390 Acked Write 1 Done Sent Done false
31066 8391 Acked Read 7 Done Sent Done false
31067 8392 Acked Write 6 Done Sent Done false
31068 8393 Acked Read 5 Done Sent Done false
31069 8394 Acked Read 4 Done Sent Done false
31070 8395 Acked Read 8 Done Sent Done false
31071 8396 Acked Write 9 Done Sent Done false
31072 8397 Acked Read 9 Done Sent Done false
31073 8398 Acked Read 2 Done Sent Done false
31074 8399 Acked Read 8 Done Sent Done false
31075 8400 Acked Write 10 Done Sent Done false
31076 8401 Acked Write 3 Done Sent Done false
31077 8402 Acked Write 3 Done Sent Done false
31078 8403 Acked Write 10 Done Sent Done false
31079 8404 Acked Read 6 Done Sent Done false
31080 8405 Acked Read 2 Done Sent Done false
31081 8406 Acked Write 7 Done Sent Done false
31082 8407 Acked Read 10 Done Sent Done false
31083 8408 Acked Read 1 Done Sent Done false
31084 8409 Acked Read 4 Done Sent Done false
31085 8410 Acked Read 9 Done Sent Done false
31086 8411 Acked Write 1 Done Sent Done false
31087 8412 Acked Write 2 Done Sent Done false
31088 8413 Acked Read 1 Done Sent Done false
31089 8414 Acked Read 2 Done Sent Done false
31090 8415 Acked Write 3 Done Sent Done false
31091 8416 Acked Write 7 Done Sent Done false
31092 8417 Acked Write 6 Done Sent Done false
31093 8418 Acked Write 4 Done Sent Done false
31094 8419 Acked Read 4 Done Sent Done false
31095 8420 Acked Read 6 Done Sent Done false
31096 8421 Acked Read 5 Done Sent Done false
31097 8422 Acked Write 2 Done Sent Done false
31098 8423 Acked Write 1 Done Sent Done false
31099 8424 Acked Read 1 Done Sent Done false
31100 8425 Acked Flush 0 Done Sent Done false
31101 8426 Acked Read 5 Done Sent Done false
31102 8427 Acked Read 2 Done Sent Done false
31103 8428 Acked Flush 0 Done Sent Done false
31104 8429 Acked Write 9 Done Sent Done false
31105 8430 Acked Write 2 Done Sent Done false
31106 8431 Acked Read 10 Done Sent Done false
31107 8432 Acked Read 2 Done Sent Done false
31108 8433 Acked Read 9 Done Sent Done false
31109 8434 Acked Read 1 Done Sent Done false
31110 8435 Acked Write 2 Done Sent Done false
31111 8436 Acked Write 10 Done Sent Done false
31112 8437 Acked Read 8 Done Sent Done false
31113 8438 Acked Read 9 Done Sent Done false
31114 8439 Acked Flush 0 Done Sent Done false
31115 8440 Acked Write 2 Done Sent Done false
31116 8441 Acked Write 9 Done Sent Done false
31117 8442 Acked Write 7 Done Sent Done false
31118 8443 Acked Write 3 Done Sent Done false
31119 8444 Acked Read 5 Done Sent Done false
31120 8445 Acked Read 3 Done Sent Done false
31121 8446 Acked Write 1 Done Sent Done false
31122 8447 Acked Read 6 Done Sent Done false
31123 8448 Acked Write 4 Done Sent Done false
31124 8449 Acked Write 10 Done Sent Done false
31125 8450 Acked Write 4 Done Sent Done false
31126 8451 Acked Write 1 Done Sent Done false
31127 8452 Acked Read 3 Done Sent Done false
31128 8453 Acked Write 4 Done Sent Done false
31129 8454 Acked Read 4 Done Sent Done false
31130 8455 Acked Read 6 Done Sent Done false
31131 8456 Acked Read 4 Done Sent Done false
31132 8457 Acked Read 1 Done Sent Done false
31133 8458 Acked Write 7 Done Sent Done false
31134 8459 Acked Read 2 Done Sent Done false
31135 8460 Acked Write 4 Done Sent Done false
31136 8461 Acked Read 5 Done Sent Done false
31137 8462 Acked Read 7 Done Sent Done false
31138 8463 Acked Read 5 Done Sent Done false
31139 8464 Acked Read 4 Done Sent Done false
31140 8465 Acked Write 9 Done Sent Done false
31141 8466 Acked Write 1 Done Sent Done false
31142 8467 Acked Read 3 Done Sent Done false
31143 8468 Acked Read 8 Done Sent Done false
31144 8469 Acked Read 3 Done Sent Done false
31145 8470 Acked Read 4 Done Sent Done false
31146 8471 Acked Write 8 Done Sent Done false
31147 8472 Acked Write 8 Done Sent Done false
31148 8473 Acked Read 5 Done Sent Done false
31149 8474 Acked Read 7 Done Sent Done false
31150 8475 Acked Read 5 Done Sent Done false
31151 8476 Acked Write 2 Done Sent Done false
31152 8477 Acked Read 7 Done Sent Done false
31153 8478 Acked Write 2 Done Sent Done false
31154 8479 Acked Write 9 Done Sent Done false
31155 8480 Acked Read 2 Done Sent Done false
31156 8481 Acked Write 7 Done Sent Done false
31157 8482 Acked Write 2 Done Sent Done false
31158 8483 Acked Write 3 Done Sent Done false
31159 8484 Acked Write 8 Done Sent Done false
31160 8485 Acked Flush 0 Done Sent Done false
31161 8486 Acked Read 1 Done Sent Done false
31162 8487 Acked Read 4 Done Sent Done false
31163 8488 Acked Write 10 Done Sent Done false
31164 8489 Acked Read 4 Done Sent Done false
31165 8490 Acked Write 7 Done Sent Done false
31166 8491 Acked Write 4 Done Sent Done false
31167 8492 Acked Read 6 Done Sent Done false
31168 8493 Acked Write 10 Done Sent Done false
31169 8494 Acked Write 8 Done Sent Done false
31170 8495 Acked Write 5 Done Sent Done false
31171 8496 Acked Write 6 Done Sent Done false
31172 8497 Acked Flush 0 Done Sent Done false
31173 8498 Acked Flush 0 Done Sent Done false
31174 8499 Acked Write 1 Done Sent Done false
31175 8500 Acked Write 6 Done Sent Done false
31176 8501 Acked Write 4 Done Sent Done false
31177 8502 Acked Write 2 Done Sent Done false
31178 8503 Acked Write 9 Done Sent Done false
31179 8504 Acked Read 10 Done Sent Done false
31180 8505 Acked Write 8 Done Sent Done false
31181 8506 Acked Write 4 Done Sent Done false
31182 8507 Acked Read 3 Done Sent Done false
31183 8508 Acked Write 4 Done Sent Done false
31184 8509 Acked Write 4 Done Sent Done false
31185 8510 Acked Write 1 Done Sent Done false
31186 8511 Acked Write 4 Done Sent Done false
31187 8512 Acked Read 4 Done Sent Done false
31188 8513 Acked Read 2 Done Sent Done false
31189 8514 Acked Flush 0 Done Sent Done false
31190 8515 Acked Write 5 Done Sent Done false
31191 8516 Acked Write 5 Done Sent Done false
31192 8517 Acked Write 10 Done Sent Done false
31193 8518 Acked Write 8 Done Sent Done false
31194 8519 Acked Write 10 Done Sent Done false
31195 8520 Acked Read 5 Done Sent Done false
31196 8521 Acked Flush 0 Done Sent Done false
31197 8522 Acked Flush 0 Done Sent Done false
31198 8523 Acked Read 3 Done Sent Done false
31199 8524 Acked Read 1 Done Sent Done false
31200 8525 Acked Write 8 Done Sent Done false
31201 8526 Acked Read 3 Done Sent Done false
31202 8527 Acked Read 1 Done Sent Done false
31203 8528 Acked Read 9 Done Sent Done false
31204 8529 Acked Write 10 Done Sent Done false
31205 8530 Acked Read 7 Done Sent Done false
31206 8531 Acked Read 5 Done Sent Done false
31207 8532 Acked Write 5 Done Sent Done false
31208 8533 Acked Read 6 Done Sent Done false
31209 8534 Acked Read 8 Done Sent Done false
31210 8535 Acked Write 8 Done Sent Done false
31211 8536 Acked Read 5 Done Sent Done false
31212 8537 Acked Read 1 Done Sent Done false
31213 8538 Acked Flush 0 Done Sent Done false
31214 8539 Acked Write 8 Done Sent Done false
31215 8540 Acked Read 2 Done Sent Done false
31216 8541 Acked Write 7 Done Sent Done false
31217 8542 Acked Write 7 Done Sent Done false
31218 8543 Acked Flush 0 Done Sent Done false
31219 8544 Acked Flush 0 Done Sent Done false
31220 8545 Acked Read 9 Done Sent Done false
31221 8546 Acked Read 2 Done Sent Done false
31222 STATES DS:0 DS:1 DS:2 TOTAL
31223 Sent 0 301 0 301
31224 Done 301 0 301 602
31225 Skipped 0 0 0 0
31226 Error 0 0 0 0
31227 Last Flush: 8544 8245 8544
31228 Downstairs last five completed: 8245 8244 8243 8242 8241
31229 Upstairs last five completed: 8546 8545 8544 8543 8542
31230 CLIENT: Up:0 ds:301 act:2
31231 [8820] got start action
31232 [8820] I am going to restart
31233 Starting downstairs at port 8820
31234 Make output file at "/tmp/dsc/downstairs-8820.txt"
31235 [8820][1] reports Starting
31236 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1230)
31237 [8820][1] reports Running
31238 ----------------------------------------------------------------
31239 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
31240 States: Active Offline Active
31241 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
31242 8246 Acked Read 8 Done Sent Done false
31243 8247 Acked Write 7 Done Sent Done false
31244 8248 Acked Write 9 Done Sent Done false
31245 8249 Acked Read 9 Done Sent Done false
31246 8250 Acked Read 7 Done Sent Done false
31247 8251 Acked Read 6 Done Sent Done false
31248 8252 Acked Read 3 Done Sent Done false
31249 8253 Acked Read 1 Done Sent Done false
31250 8254 Acked Read 3 Done Sent Done false
31251 8255 Acked Read 5 Done Sent Done false
31252 8256 Acked Write 4 Done Sent Done false
31253 8257 Acked Read 2 Done Sent Done false
31254 8258 Acked Write 8 Done Sent Done false
31255 8259 Acked Write 1 Done Sent Done false
31256 8260 Acked Write 4 Done Sent Done false
31257 8261 Acked Write 2 Done Sent Done false
31258 8262 Acked Flush 0 Done Sent Done false
31259 8263 Acked Write 10 Done Sent Done false
31260 8264 Acked Read 3 Done Sent Done false
31261 8265 Acked Flush 0 Done Sent Done false
31262 8266 Acked Read 5 Done Sent Done false
31263 8267 Acked Write 5 Done Sent Done false
31264 8268 Acked Write 4 Done Sent Done false
31265 8269 Acked Read 9 Done Sent Done false
31266 8270 Acked Write 1 Done Sent Done false
31267 8271 Acked Read 3 Done Sent Done false
31268 8272 Acked Read 3 Done Sent Done false
31269 8273 Acked Flush 0 Done Sent Done false
31270 8274 Acked Read 9 Done Sent Done false
31271 8275 Acked Read 9 Done Sent Done false
31272 8276 Acked Write 10 Done Sent Done false
31273 8277 Acked Write 6 Done Sent Done false
31274 8278 Acked Flush 0 Done Sent Done false
31275 8279 Acked Flush 0 Done Sent Done false
31276 8280 Acked Read 4 Done Sent Done false
31277 8281 Acked Read 9 Done Sent Done false
31278 8282 Acked Flush 0 Done Sent Done false
31279 8283 Acked Read 10 Done Sent Done false
31280 8284 Acked Write 10 Done Sent Done false
31281 8285 Acked Read 6 Done Sent Done false
31282 8286 Acked Write 8 Done Sent Done false
31283 8287 Acked Write 10 Done Sent Done false
31284 8288 Acked Read 4 Done Sent Done false
31285 8289 Acked Read 4 Done Sent Done false
31286 8290 Acked Write 9 Done Sent Done false
31287 8291 Acked Flush 0 Done Sent Done false
31288 8292 Acked Read 7 Done Sent Done false
31289 8293 Acked Write 3 Done Sent Done false
31290 8294 Acked Write 9 Done Sent Done false
31291 8295 Acked Flush 0 Done Sent Done false
31292 8296 Acked Read 2 Done Sent Done false
31293 8297 Acked Read 4 Done Sent Done false
31294 8298 Acked Write 1 Done Sent Done false
31295 8299 Acked Write 7 Done Sent Done false
31296 8300 Acked Write 7 Done Sent Done false
31297 8301 Acked Read 8 Done Sent Done false
31298 8302 Acked Read 5 Done Sent Done false
31299 8303 Acked Write 4 Done Sent Done false
31300 8304 Acked Read 7 Done Sent Done false
31301 8305 Acked Write 6 Done Sent Done false
31302 8306 Acked Flush 0 Done Sent Done false
31303 8307 Acked Write 9 Done Sent Done false
31304 8308 Acked Write 9 Done Sent Done false
31305 8309 Acked Read 7 Done Sent Done false
31306 8310 Acked Write 2 Done Sent Done false
31307 8311 Acked Flush 0 Done Sent Done false
31308 8312 Acked Write 8 Done Sent Done false
31309 8313 Acked Write 5 Done Sent Done false
31310 8314 Acked Write 8 Done Sent Done false
31311 8315 Acked Read 4 Done Sent Done false
31312 8316 Acked Write 1 Done Sent Done false
31313 8317 Acked Write 3 Done Sent Done false
31314 8318 Acked Read 2 Done Sent Done false
31315 8319 Acked Read 1 Done Sent Done false
31316 8320 Acked Write 8 Done Sent Done false
31317 8321 Acked Write 5 Done Sent Done false
31318 8322 Acked Flush 0 Done Sent Done false
31319 8323 Acked Read 3 Done Sent Done false
31320 8324 Acked Read 7 Done Sent Done false
31321 8325 Acked Read 2 Done Sent Done false
31322 8326 Acked Read 6 Done Sent Done false
31323 8327 Acked Flush 0 Done Sent Done false
31324 8328 Acked Read 9 Done Sent Done false
31325 8329 Acked Write 10 Done Sent Done false
31326 8330 Acked Read 1 Done Sent Done false
31327 8331 Acked Read 10 Done Sent Done false
31328 8332 Acked Write 1 Done Sent Done false
31329 8333 Acked Write 6 Done Sent Done false
31330 8334 Acked Write 4 Done Sent Done false
31331 8335 Acked Read 7 Done Sent Done false
31332 8336 Acked Write 4 Done Sent Done false
31333 8337 Acked Read 9 Done Sent Done false
31334 8338 Acked Read 6 Done Sent Done false
31335 8339 Acked Read 7 Done Sent Done false
31336 8340 Acked Read 6 Done Sent Done false
31337 8341 Acked Read 2 Done Sent Done false
31338 8342 Acked Write 6 Done Sent Done false
31339 8343 Acked Read 7 Done Sent Done false
31340 8344 Acked Read 1 Done Sent Done false
31341 8345 Acked Read 4 Done Sent Done false
31342 8346 Acked Write 2 Done Sent Done false
31343 8347 Acked Read 4 Done Sent Done false
31344 8348 Acked Read 3 Done Sent Done false
31345 8349 Acked Read 3 Done Sent Done false
31346 8350 Acked Write 7 Done Sent Done false
31347 8351 Acked Read 9 Done Sent Done false
31348 8352 Acked Read 5 Done Sent Done false
31349 8353 Acked Flush 0 Done Sent Done false
31350 8354 Acked Flush 0 Done Sent Done false
31351 8355 Acked Write 10 Done Sent Done false
31352 8356 Acked Write 2 Done Sent Done false
31353 8357 Acked Write 9 Done Sent Done false
31354 8358 Acked Write 7 Done Sent Done false
31355 8359 Acked Read 9 Done Sent Done false
31356 8360 Acked Read 9 Done Sent Done false
31357 8361 Acked Write 1 Done Sent Done false
31358 8362 Acked Read 7 Done Sent Done false
31359 8363 Acked Write 3 Done Sent Done false
31360 8364 Acked Write 5 Done Sent Done false
31361 8365 Acked Write 2 Done Sent Done false
31362 8366 Acked Write 6 Done Sent Done false
31363 8367 Acked Read 4 Done Sent Done false
31364 8368 Acked Read 1 Done Sent Done false
31365 8369 Acked Read 2 Done Sent Done false
31366 8370 Acked Write 6 Done Sent Done false
31367 8371 Acked Flush 0 Done Sent Done false
31368 8372 Acked Write 6 Done Sent Done false
31369 8373 Acked Write 8 Done Sent Done false
31370 8374 Acked Read 5 Done Sent Done false
31371 8375 Acked Read 5 Done Sent Done false
31372 8376 Acked Read 3 Done Sent Done false
31373 8377 Acked Write 7 Done Sent Done false
31374 8378 Acked Write 10 Done Sent Done false
31375 8379 Acked Read 8 Done Sent Done false
31376 8380 Acked Write 3 Done Sent Done false
31377 8381 Acked Flush 0 Done Sent Done false
31378 8382 Acked Read 2 Done Sent Done false
31379 8383 Acked Read 2 Done Sent Done false
31380 8384 Acked Read 8 Done Sent Done false
31381 8385 Acked Read 8 Done Sent Done false
31382 8386 Acked Read 9 Done Sent Done false
31383 8387 Acked Read 3 Done Sent Done false
31384 8388 Acked Write 1 Done Sent Done false
31385 8389 Acked Read 2 Done Sent Done false
31386 8390 Acked Write 1 Done Sent Done false
31387 8391 Acked Read 7 Done Sent Done false
31388 8392 Acked Write 6 Done Sent Done false
31389 8393 Acked Read 5 Done Sent Done false
31390 8394 Acked Read 4 Done Sent Done false
31391 8395 Acked Read 8 Done Sent Done false
31392 8396 Acked Write 9 Done Sent Done false
31393 8397 Acked Read 9 Done Sent Done false
31394 8398 Acked Read 2 Done Sent Done false
31395 8399 Acked Read 8 Done Sent Done false
31396 8400 Acked Write 10 Done Sent Done false
31397 8401 Acked Write 3 Done Sent Done false
31398 8402 Acked Write 3 Done Sent Done false
31399 8403 Acked Write 10 Done Sent Done false
31400 8404 Acked Read 6 Done Sent Done false
31401 8405 Acked Read 2 Done Sent Done false
31402 8406 Acked Write 7 Done Sent Done false
31403 8407 Acked Read 10 Done Sent Done false
31404 8408 Acked Read 1 Done Sent Done false
31405 8409 Acked Read 4 Done Sent Done false
31406 8410 Acked Read 9 Done Sent Done false
31407 8411 Acked Write 1 Done Sent Done false
31408 8412 Acked Write 2 Done Sent Done false
31409 8413 Acked Read 1 Done Sent Done false
31410 8414 Acked Read 2 Done Sent Done false
31411 8415 Acked Write 3 Done Sent Done false
31412 8416 Acked Write 7 Done Sent Done false
31413 8417 Acked Write 6 Done Sent Done false
31414 8418 Acked Write 4 Done Sent Done false
31415 8419 Acked Read 4 Done Sent Done false
31416 8420 Acked Read 6 Done Sent Done false
31417 8421 Acked Read 5 Done Sent Done false
31418 8422 Acked Write 2 Done Sent Done false
31419 8423 Acked Write 1 Done Sent Done false
31420 8424 Acked Read 1 Done Sent Done false
31421 8425 Acked Flush 0 Done Sent Done false
31422 8426 Acked Read 5 Done Sent Done false
31423 8427 Acked Read 2 Done Sent Done false
31424 8428 Acked Flush 0 Done Sent Done false
31425 8429 Acked Write 9 Done Sent Done false
31426 8430 Acked Write 2 Done Sent Done false
31427 8431 Acked Read 10 Done Sent Done false
31428 8432 Acked Read 2 Done Sent Done false
31429 8433 Acked Read 9 Done Sent Done false
31430 8434 Acked Read 1 Done Sent Done false
31431 8435 Acked Write 2 Done Sent Done false
31432 8436 Acked Write 10 Done Sent Done false
31433 8437 Acked Read 8 Done Sent Done false
31434 8438 Acked Read 9 Done Sent Done false
31435 8439 Acked Flush 0 Done Sent Done false
31436 8440 Acked Write 2 Done Sent Done false
31437 8441 Acked Write 9 Done Sent Done false
31438 8442 Acked Write 7 Done Sent Done false
31439 8443 Acked Write 3 Done Sent Done false
31440 8444 Acked Read 5 Done Sent Done false
31441 8445 Acked Read 3 Done Sent Done false
31442 8446 Acked Write 1 Done Sent Done false
31443 8447 Acked Read 6 Done Sent Done false
31444 8448 Acked Write 4 Done Sent Done false
31445 8449 Acked Write 10 Done Sent Done false
31446 8450 Acked Write 4 Done Sent Done false
31447 8451 Acked Write 1 Done Sent Done false
31448 8452 Acked Read 3 Done Sent Done false
31449 8453 Acked Write 4 Done Sent Done false
31450 8454 Acked Read 4 Done Sent Done false
31451 8455 Acked Read 6 Done Sent Done false
31452 8456 Acked Read 4 Done Sent Done false
31453 8457 Acked Read 1 Done Sent Done false
31454 8458 Acked Write 7 Done Sent Done false
31455 8459 Acked Read 2 Done Sent Done false
31456 8460 Acked Write 4 Done Sent Done false
31457 8461 Acked Read 5 Done Sent Done false
31458 8462 Acked Read 7 Done Sent Done false
31459 8463 Acked Read 5 Done Sent Done false
31460 8464 Acked Read 4 Done Sent Done false
31461 8465 Acked Write 9 Done Sent Done false
31462 8466 Acked Write 1 Done Sent Done false
31463 8467 Acked Read 3 Done Sent Done false
31464 8468 Acked Read 8 Done Sent Done false
31465 8469 Acked Read 3 Done Sent Done false
31466 8470 Acked Read 4 Done Sent Done false
31467 8471 Acked Write 8 Done Sent Done false
31468 8472 Acked Write 8 Done Sent Done false
31469 8473 Acked Read 5 Done Sent Done false
31470 8474 Acked Read 7 Done Sent Done false
31471 8475 Acked Read 5 Done Sent Done false
31472 8476 Acked Write 2 Done Sent Done false
31473 8477 Acked Read 7 Done Sent Done false
31474 8478 Acked Write 2 Done Sent Done false
31475 8479 Acked Write 9 Done Sent Done false
31476 8480 Acked Read 2 Done Sent Done false
31477 8481 Acked Write 7 Done Sent Done false
31478 8482 Acked Write 2 Done Sent Done false
31479 8483 Acked Write 3 Done Sent Done false
31480 8484 Acked Write 8 Done Sent Done false
31481 8485 Acked Flush 0 Done Sent Done false
31482 8486 Acked Read 1 Done Sent Done false
31483 8487 Acked Read 4 Done Sent Done false
31484 8488 Acked Write 10 Done Sent Done false
31485 8489 Acked Read 4 Done Sent Done false
31486 8490 Acked Write 7 Done Sent Done false
31487 8491 Acked Write 4 Done Sent Done false
31488 8492 Acked Read 6 Done Sent Done false
31489 8493 Acked Write 10 Done Sent Done false
31490 8494 Acked Write 8 Done Sent Done false
31491 8495 Acked Write 5 Done Sent Done false
31492 8496 Acked Write 6 Done Sent Done false
31493 8497 Acked Flush 0 Done Sent Done false
31494 8498 Acked Flush 0 Done Sent Done false
31495 8499 Acked Write 1 Done Sent Done false
31496 8500 Acked Write 6 Done Sent Done false
31497 8501 Acked Write 4 Done Sent Done false
31498 8502 Acked Write 2 Done Sent Done false
31499 8503 Acked Write 9 Done Sent Done false
31500 8504 Acked Read 10 Done Sent Done false
31501 8505 Acked Write 8 Done Sent Done false
31502 8506 Acked Write 4 Done Sent Done false
31503 8507 Acked Read 3 Done Sent Done false
31504 8508 Acked Write 4 Done Sent Done false
31505 8509 Acked Write 4 Done Sent Done false
31506 8510 Acked Write 1 Done Sent Done false
31507 8511 Acked Write 4 Done Sent Done false
31508 8512 Acked Read 4 Done Sent Done false
31509 8513 Acked Read 2 Done Sent Done false
31510 8514 Acked Flush 0 Done Sent Done false
31511 8515 Acked Write 5 Done Sent Done false
31512 8516 Acked Write 5 Done Sent Done false
31513 8517 Acked Write 10 Done Sent Done false
31514 8518 Acked Write 8 Done Sent Done false
31515 8519 Acked Write 10 Done Sent Done false
31516 8520 Acked Read 5 Done Sent Done false
31517 8521 Acked Flush 0 Done Sent Done false
31518 8522 Acked Flush 0 Done Sent Done false
31519 8523 Acked Read 3 Done Sent Done false
31520 8524 Acked Read 1 Done Sent Done false
31521 8525 Acked Write 8 Done Sent Done false
31522 8526 Acked Read 3 Done Sent Done false
31523 8527 Acked Read 1 Done Sent Done false
31524 8528 Acked Read 9 Done Sent Done false
31525 8529 Acked Write 10 Done Sent Done false
31526 8530 Acked Read 7 Done Sent Done false
31527 8531 Acked Read 5 Done Sent Done false
31528 8532 Acked Write 5 Done Sent Done false
31529 8533 Acked Read 6 Done Sent Done false
31530 8534 Acked Read 8 Done Sent Done false
31531 8535 Acked Write 8 Done Sent Done false
31532 8536 Acked Read 5 Done Sent Done false
31533 8537 Acked Read 1 Done Sent Done false
31534 8538 Acked Flush 0 Done Sent Done false
31535 8539 Acked Write 8 Done Sent Done false
31536 8540 Acked Read 2 Done Sent Done false
31537 8541 Acked Write 7 Done Sent Done false
31538 8542 Acked Write 7 Done Sent Done false
31539 8543 Acked Flush 0 Done Sent Done false
31540 8544 Acked Flush 0 Done Sent Done false
31541 8545 Acked Read 9 Done Sent Done false
31542 8546 Acked Read 2 Done Sent Done false
31543 8547 Acked Flush 0 Done Sent Done false
31544 STATES DS:0 DS:1 DS:2 TOTAL
31545 Sent 0 302 0 302
31546 Done 302 0 302 604
31547 Skipped 0 0 0 0
31548 Error 0 0 0 0
31549 Last Flush: 8547 8245 8547
31550 Downstairs last five completed: 8245 8244 8243 8242 8241
31551 Upstairs last five completed: 8547 8546 8545 8544 8543
31552 CLIENT: Up:0 ds:302 act:2
315532024-11-15T02:04:03.825ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
315542024-11-15T02:04:03.825ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:38945) = io task client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
315552024-11-15T02:04:03.826ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
315562024-11-15T02:04:03.827ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
315572024-11-15T02:04:03.827ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
315582024-11-15T02:04:03.827ZINFOcrucible: send last flush ID to this DS: 8245 = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
315592024-11-15T02:04:03.827ZINFOcrucible: Replied this last flush ID: 8245 = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
315602024-11-15T02:04:03.827ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
315612024-11-15T02:04:03.827ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
315622024-11-15T02:04:03.827ZINFOcrucible: [1] client re-new 302 jobs since flush 8245 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
315632024-11-15T02:04:03.827ZINFOcrucible: [1] Replayed 302 jobs since flush: 8245 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
31564 ----------------------------------------------------------------
31565 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
31566 Downstairs last five completed: 8547 8546 8545 8544 8543
31567 Upstairs last five completed: 8547 8546 8545 8544 8543
31568 CLIENT: Up:0 ds:0 act:3
31569 Replay: All jobs finished
31570 Main task has work to do, go find it
31571 got dsc Stop(0)
31572 stop 0
31573 [8810] Got stop action so:false kr:false
31574 Nov 15 02:04:06.055 INFO request completed, latency_us: 362, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 16ee0aee-110c-40c2-a9ca-197c03d891aa, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
315752024-11-15T02:04:06.055ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
315762024-11-15T02:04:06.055ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
31577 Nov 15 02:04:06.055 INFO request completed, latency_us{"msg":"downstairs task for 0 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2024-11-15T02:04:06.055869865Z","hostname":"w-01JCPRHT5F2MPA44KVJHM3S2PB","pid":1205,"session_id":"971dcb2e-6891-4ecf-b367-1e04e52065ee"}
31578 : 136, response_code: 200, uri: /state/cid/0, method: GET, {"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-15T02:04:06.055887974Z","hostname":"w-01JCPRHT5F2MPA44KVJHM3S2PB","pid":1205,"client":"0","":"downstairs","session_id":"971dcb2e-6891-4ecf-b367-1e04e52065ee"}
31579 req_id: {"msg":"[0] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-15T02:04:06.055908013Z","hostname":"w-01JCPRHT5F2MPA44KVJHM3S2PB","pid":1205,"client":"0","":"downstairs","session_id":"971dcb2e-6891-4ecf-b367-1e04e52065ee"}
31580 602be4f9-d791-4ccf-b22f-c05c0440573a, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2024-11-15T02:04:06.05595507Z","hostname":"w-01JCPRHT5F2MPA44KVJHM3S2PB","pid":1205,"":"io task","client":"0","":"downstairs","session_id":"971dcb2e-6891-4ecf-b367-1e04e52065ee"}
31581 9998
31582 [8810] Exited with: ExitStatus(unix_wait_status(9))
31583 [8810][0] reports Exit
31584 Nov 15 02:04:10.058 INFO request completed, latency_us: 215, response_code: 200, uri: /state/cid/0, method: GET, req_id: 06ed4c49-ac13-4934-bef2-9f62de23ae96, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
31585 001/300 Read block 2346 len 24576
31586 002/300 Read block 597 len 36864
31587 003/300 Read block 1445 len 4096
31588 004/300 Read block 3531 len 36864
31589 005/300 Read block 4684 len 40960
31590 006/300 Read block 3593 len 4096
31591 007/300 Flush
31592 008/300 Flush
31593 009/300 Write block 201 len 20480 data: 7 6 6 5 6
31594 010/300 Write block 3970 len 4096 data: 6
31595 011/300 Write block 1853 len 36864 data: 6 6 6 5 5 5 5 5 4
31596 012/300 Write block 4210 len 24576 data: 6 5 3 4 5 5
31597 013/300 Write block 4151 len 4096 data: 6
31598 014/300 Read block 4087 len 8192
31599 015/300 Write block 4695 len 8192 data: 5 6
31600 016/300 Read block 1011 len 16384
31601 017/300 Read block 3416 len 28672
31602 018/300 Read block 90 len 16384
31603 019/300 Read block 3420 len 40960
31604 020/300 Write block 296 len 12288 data: 5 5 4
31605 021/300 Read block 4146 len 16384
31606 022/300 Read block 3795 len 12288
31607 023/300 Flush
31608 024/300 Write block 1314 len 28672 data: 7 7 5 7 3 3 4
31609 025/300 Write block 2398 len 4096 data: 4
31610 026/300 Read block 852 len 20480
31611 027/300 Write block 1747 len 8192 data: 7 5
31612 028/300 Write block 4642 len 4096 data: 3
31613 029/300 Write block 698 len 8192 data: 6 6
31614 030/300 Read block 3604 len 4096
31615 031/300 Flush
31616 032/300 Write block 667 len 12288 data: 8 6 5
31617 033/300 Write block 2439 len 36864 data: 4 4 6 5 3 4 5 5 7
31618 034/300 Write block 4441 len 4096 data: 6
31619 035/300 Write block 897 len 40960 data: 4 4 4 3 4 7 7 7 6 7
31620 036/300 Write block 2748 len 20480 data: 5 3 3 4 2
31621 037/300 Read block 4699 len 32768
31622 038/300 Read block 2540 len 28672
31623 039/300 Write block 2933 len 28672 data: 6 7 5 5 5 6 6
31624 040/300 Flush
31625 041/300 Read block 1210 len 40960
31626 042/300 Write block 2410 len 28672 data: 3 4 4 4 5 5 4
31627 043/300 Read block 4088 len 16384
31628 044/300 Read block 3076 len 8192
31629 045/300 Flush
31630 046/300 Write block 739 len 28672 data: 5 4 5 5 4 4 4
31631 047/300 Write block 4239 len 20480 data: 5 4 5 6 5
31632 048/300 Read block 1499 len 40960
31633 049/300 Read block 3396 len 4096
31634 050/300 Read block 2812 len 8192
31635 051/300 Read block 2741 len 20480
31636 052/300 Read block 2445 len 16384
31637 053/300 Write block 1316 len 20480 data: 6 8 4 4 5
31638 054/300 Write block 1202 len 20480 data: 4 4 4 3 3
31639 055/300 Write block 3340 len 28672 data: 7 5 4 5 4 4 4
31640 056/300 Read block 685 len 12288
31641 057/300 Write block 1615 len 24576 data: 2 2 4 4 4 5
31642 058/300 Read block 1699 len 28672
31643 059/300 Read block 3597 len 12288
31644 060/300 Read block 2992 len 20480
31645 061/300 Write block 1164 len 4096 data: 4
31646 062/300 Read block 164 len 12288
31647 063/300 Write block 4180 len 8192 data: 10 9
31648 064/300 Read block 308 len 28672
31649 065/300 Flush
31650 066/300 Write block 1643 len 36864 data: 7 7 6 5 6 5 4 4 5
31651 067/300 Read block 2555 len 12288
31652 068/300 Read block 2229 len 40960
31653 069/300 Read block 3304 len 16384
31654 070/300 Write block 616 len 12288 data: 2 4 5
31655 071/300 Write block 4699 len 36864 data: 6 5 8 9 10 10 10 10 8
31656 072/300 Read block 2168 len 40960
31657 073/300 Read block 2919 len 12288
31658 074/300 Read block 4013 len 12288
31659 075/300 Read block 1278 len 36864
31660 076/300 Write block 409 len 28672 data: 3 4 3 4 5 5 6
31661 077/300 Read block 2907 len 28672
31662 078/300 Read block 4355 len 12288
31663 079/300 Write block 674 len 28672 data: 5 5 5 6 6 7 7
31664 080/300 Read block 2324 len 24576
31665 081/300 Flush
31666 082/300 Flush
31667 083/300 Read block 3226 len 32768
31668 084/300 Read block 808 len 24576
31669 085/300 Write block 4296 len 40960 data: 4 5 8 8 5 6 6 4 4 5
31670 086/300 Read block 2822 len 8192
31671 087/300 Flush
31672 088/300 Write block 3881 len 20480 data: 6 6 6 6 7
31673 089/300 Write block 3912 len 12288 data: 4 4 4
31674 090/300 Write block 3220 len 40960 data: 7 6 6 5 6 7 7 6 6 6
31675 091/300 Read block 1249 len 24576
31676 092/300 Read block 44 len 36864
31677 093/300 Read block 1753 len 20480
31678 094/300 Write block 1944 len 12288 data: 3 3 2
31679 095/300 Write block 1846 len 12288 data: 5 5 4
31680 096/300 Write block 4028 len 12288 data: 4 3 3
31681 097/300 Write block 4320 len 32768 data: 8 9 7 6 6 5 6 5
31682 098/300 Write block 1575 len 16384 data: 4 4 3 3
31683 099/300 Write block 2117 len 4096 data: 5
31684 100/300 Read block 3094 len 32768
31685 101/300 Write block 554 len 32768 data: 3 3 5 7 5 5 5 6
31686 102/300 Read block 3478 len 16384
31687 103/300 Read block 3446 len 8192
31688 104/300 Read block 3313 len 36864
31689 105/300 Read block 2672 len 20480
31690 106/300 Write block 2965 len 12288 data: 4 3 5
31691 107/300 Read block 3241 len 12288
31692 108/300 Read block 4933 len 40960
31693 109/300 Write block 2369 len 32768 data: 4 3 3 4 5 5 5 4
31694 110/300 Read block 3586 len 20480
31695 111/300 Flush
31696 112/300 Flush
31697 113/300 Write block 3178 len 32768 data: 3 5 5 5 5 6 4 5
31698 114/300 Write block 1557 len 28672 data: 4 6 6 7 8 9 9
31699 115/300 Read block 1684 len 24576
31700 116/300 Write block 1052 len 20480 data: 5 5 6 5 5
31701 117/300 Write block 134 len 20480 data: 5 5 5 4 4
31702 118/300 Write block 3809 len 24576 data: 8 6 8 5 4 4
31703 119/300 Read block 1725 len 28672
31704 120/300 Flush
31705 121/300 Write block 991 len 8192 data: 10 10
31706 122/300 Write block 2828 len 36864 data: 7 9 8 6 6 7 6 5 5
31707 123/300 Read block 1421 len 28672
31708 124/300 Read block 2088 len 20480
31709 125/300 Read block 979 len 12288
31710 126/300 Read block 2777 len 36864
31711 127/300 Write block 4788 len 12288 data: 4 4 4
31712 128/300 Write block 69 len 12288 data: 8 8 9
31713 129/300 Read block 2885 len 36864
31714 130/300 Write block 1129 len 12288 data: 4 5 5
31715 131/300 Write block 1727 len 20480 data: 3 3 3 3 4
31716 132/300 Read block 1479 len 16384
31717 133/300 Write block 4935 len 16384 data: 4 4 3 3
31718 134/300 Read block 1457 len 24576
31719 135/300 Flush
31720 136/300 Write block 870 len 28672 data: 2 2 3 6 4 5 5
31721 137/300 Read block 907 len 24576
31722 138/300 Write block 2231 len 40960 data: 7 8 9 7 4 4 7 7 6 7
31723 139/300 Read block 4306 len 24576
31724 140/300 Read block 1538 len 40960
31725 141/300 Read block 940 len 32768
31726 142/300 Write block 4002 len 36864 data: 4 5 7 6 5 4 5 5 5
31727 143/300 Read block 4424 len 16384
31728 144/300 Write block 3267 len 28672 data: 4 5 4 5 6 6 7
31729 145/300 Flush
31730 146/300 Write block 3206 len 8192 data: 3 3
31731 147/300 Read block 2277 len 40960
31732 148/300 Read block 4325 len 40960
31733 149/300 Read block 1643 len 16384
31734 150/300 Read block 3888 len 20480
31735 151/300 Read block 4711 len 12288
31736 152/300 Flush
31737 153/300 Read block 3928 len 8192
31738 154/300 Write block 1860 len 24576 data: 6 5 4 3 3 4
31739 155/300 Write block 4301 len 4096 data: 7
31740 156/300 Write block 207 len 32768 data: 6 6 6 6 7 5 5 6
31741 157/300 Read block 3742 len 28672
31742 158/300 Flush
31743 159/300 Read block 4867 len 32768
31744 160/300 Write block 4212 len 40960 data: 4 5 6 6 5 5 4 5 7 6
31745 161/300 Write block 3139 len 40960 data: 3 5 5 5 6 6 6 6 6 6
31746 162/300 Read block 1522 len 4096
31747 163/300 Read block 3216 len 12288
31748 164/300 Read block 3175 len 16384
31749 165/300 Write block 1166 len 12288 data: 5 5 5
31750 166/300 Write block 2643 len 4096 data: 3
31751 167/300 Write block 4050 len 40960 data: 4 4 3 3 2 2 2 2 4 4
31752 168/300 Read block 3035 len 12288
31753 169/300 Write block 3904 len 8192 data: 11 9
31754 170/300 Write block 2265 len 8192 data: 7 7
31755 171/300 Read block 652 len 12288
31756 172/300 Read block 4907 len 32768
31757 173/300 Write block 2022 len 24576 data: 7 6 9 11 11 10
31758 174/300 Write block 4848 len 20480 data: 2 2 2 2 3
31759 175/300 Read block 4178 len 20480
31760 176/300 Write block 963 len 28672 data: 6 6 6 7 8 7 6
31761 177/300 Read block 2040 len 32768
31762 178/300 Write block 4895 len 16384 data: 8 8 8 6
31763 179/300 Write block 1651 len 4096 data: 6
31764 180/300 Read block 3658 len 40960
31765 181/300 Write block 1429 len 32768 data: 5 5 3 4 3 4 5 4
31766 182/300 Read block 2601 len 28672
31767 183/300 Flush
31768 184/300 Write block 2758 len 8192 data: 6 7
31769 185/300 Read block 3334 len 8192
31770 186/300 Write block 566 len 12288 data: 3 4 5
31771 187/300 Read block 4177 len 24576
31772 188/300 Read block 994 len 8192
31773 189/300 Write block 4326 len 4096 data: 7
31774 190/300 Read block 744 len 8192
31775 191/300 Flush
31776 192/300 Read block 1597 len 28672
31777 193/300 Read block 1453 len 20480
31778 194/300 Write block 2277 len 40960 data: 4 5 4 4 6 7 9 10 10 9
31779 195/300 Read block 3648 len 28672
31780 196/300 Write block 4569 len 12288 data: 7 6 5
31781 197/300 Read block 198 len 20480
31782 198/300 Write block 516 len 36864 data: 3 3 3 3 4 4 4 4 4
31783 199/300 Write block 4605 len 24576 data: 6 7 7 7 6 4
31784 200/300 Read block 4114 len 20480
31785 201/300 Write block 3514 len 36864 data: 3 3 3 4 3 3 3 5 5
31786 202/300 Write block 341 len 20480 data: 3 3 4 4 4
31787 203/300 Read block 1450 len 4096
31788 204/300 Read block 542 len 12288
31789 205/300 Read block 1421 len 20480
31790 206/300 Read block 45 len 4096
31791 207/300 Read block 3350 len 36864
31792 208/300 Read block 3965 len 4096
31793 209/300 Write block 4730 len 36864 data: 8 7 5 5 5 5 5 5 4
31794 210/300 Read block 2623 len 32768
31795 211/300 Write block 1923 len 32768 data: 4 3 3 4 4 6 7 7
31796 212/300 Read block 4776 len 32768
31797 213/300 Write block 3966 len 40960 data: 4 5 5 5 7 7 7 8 8 6
31798 214/300 Flush
31799 215/300 Write block 1330 len 28672 data: 4 5 5 5 5 4 5
31800 216/300 Read block 2106 len 24576
31801 217/300 Read block 2294 len 24576
31802 218/300 Read block 3115 len 4096
31803 219/300 Read block 869 len 36864
31804 220/300 Write block 107 len 20480 data: 5 7 7 6 5
31805 221/300 Write block 538 len 20480 data: 6 6 5 5 4
31806 222/300 Read block 3785 len 28672
31807 223/300 Read block 3170 len 28672
31808 224/300 Read block 1458 len 12288
31809 225/300 Flush
31810 226/300 Read block 1154 len 24576
31811 227/300 Read block 88 len 8192
31812 228/300 Flush
31813 229/300 Write block 4294 len 8192 data: 4 4
31814 230/300 Read block 291 len 36864
31815 231/300 Write block 4937 len 32768 data: 4 4 6 7 6 6 6 6
31816 232/300 Write block 2882 len 16384 data: 5 4 3 4
31817 233/300 Write block 3509 len 12288 data: 3 3 3
31818 234/300 Read block 3670 len 12288
31819 235/300 Write block 390 len 12288 data: 5 7 6
31820 236/300 Write block 4511 len 12288 data: 3 4 6
31821 237/300 Write block 2294 len 4096 data: 4
31822 238/300 Write block 3568 len 40960 data: 6 7 6 6 5 6 6 7 6 5
31823 239/300 Write block 1873 len 24576 data: 5 6 6 6 5 5
31824 240/300 Write block 1344 len 36864 data: 8 7 5 5 3 5 6 5 4
31825 241/300 Write block 105 len 32768 data: 4 3 6 8 8 7 6 4
31826 242/300 Read block 1452 len 16384
31827 243/300 Write block 1858 len 32768 data: 6 6 7 6 5 4 4 5
31828 244/300 Write block 725 len 4096 data: 3
31829 245/300 Read block 2507 len 4096
31830 246/300 Read block 803 len 20480
31831 247/300 Read block 1468 len 8192
31832 248/300 Write block 4122 len 36864 data: 6 5 7 9 8 7 9 8 8
31833 249/300 Write block 2986 len 40960 data: 7 6 5 4 5 6 6 4 4 4
31834 250/300 Read block 1934 len 16384
31835 251/300 Read block 861 len 4096
31836 252/300 Write block 653 len 24576 data: 8 8 6 5 4 3
31837 253/300 Write block 4217 len 36864 data: 6 5 6 8 7 6 7 7 7
31838 254/300 Read block 2381 len 12288
31839 255/300 Write block 4119 len 32768 data: 3 4 4 7 6 8 10 9
31840 256/300 Read block 640 len 16384
31841 257/300 Read block 4324 len 16384
31842 258/300 Write block 529 len 24576 data: 4 6 6 6 7 9
31843 259/300 Write block 234 len 40960 data: 8 8 9 7 6 5 5 4 3 3
31844 260/300 Read block 2158 len 32768
31845 261/300 Write block 3771 len 36864 data: 5 4 4 6 7 6 5 5 5
31846 262/300 Read block 4255 len 4096
31847 263/300 Read block 3603 len 32768
31848 264/300 Write block 4436 len 4096 data: 8
31849 265/300 Write block 4483 len 20480 data: 4 4 4 4 4
31850 266/300 Write block 3375 len 4096 data: 6
31851 267/300 Read block 3532 len 36864
31852 268/300 Read block 3513 len 24576
31853 269/300 Write block 2073 len 24576 data: 4 5 4 5 6 7
31854 270/300 Write block 915 len 24576 data: 3 4 4 4 4 5
31855 271/300 Read block 1877 len 28672
31856 272/300 Read block 605 len 20480
31857 273/300 Read block 2653 len 8192
31858 274/300 Flush
31859 275/300 Write block 4286 len 24576 data: 6 7 7 6 5 6
31860 276/300 Write block 1442 len 24576 data: 3 3 4 5 6 4
31861 277/300 Read block 536 len 12288
31862 278/300 Flush
31863 279/300 Read block 2097 len 28672
31864 280/300 Flush
31865 281/300 Write block 2413 len 24576 data: 5 6 6 5 4 4
31866 282/300 Write block 1550 len 16384 data: 7 6 5 5
31867 283/300 Write block 1750 len 36864 data: 4 3 2 2 2 3 3 5 7
31868 284/300 Write block 2058 len 28672 data: 5 6 7 8 9 8 6
31869 285/300 Write block 3303 len 16384 data: 4 4 4 4
31870 286/300 Write block 2546 len 24576 data: 6 5 5 5 5 4
31871 287/300 Write block 4645 len 24576 data: 4 7 6 5 6 6
31872 288/300 Write block 487 len 36864 data: 3 3 3 5 6 6 6 6 6
31873 289/300 Write block 3702 len 8192 data: 6 5
31874 290/300 Write block 2319 len 12288 data: 9 7 6
31875 291/300 Read block 3055 len 40960
31876 292/300 Read block 1716 len 16384
31877 293/300 Flush
31878 294/300 Write block 1720 len 8192 data: 3 3
31879 295/300 Write block 665 len 4096 data: 6
31880 296/300 Write block 4124 len 24576 data: 9 11 10 8 10 9
31881 297/300 Write block 4676 len 24576 data: 3 3 5 6 6 5
31882 298/300 Read block 2287 len 20480
31883 299/300 Read block 4710 len 40960
31884 300/300 Read block 4913 len 24576
31885 301/300 Read block 2972 len 4096
31886 Main task has work to do, go find it
31887 got dsc Start(0)
31888 start 0
31889 Nov 15 02:04:10.290 INFO request completed, latency_us: 219, response_code: 204, uri: /start/cid/0, method: POST, req_id: 678a7958-7745-41eb-926e-18c11fdd8307, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
31890 Replay: started 0, returned:Ok(())
31891 ----------------------------------------------------------------
31892 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
31893 States: Offline Active Active
31894 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
31895 8548 Acked Read 6 Sent Done Done false
31896 8549 Acked Read 9 Sent Done Done false
31897 8550 Acked Read 1 Sent Done Done false
31898 8551 Acked Read 9 Sent Done Done false
31899 8552 Acked Read 10 Sent Done Done false
31900 8553 Acked Read 1 Sent Done Done false
31901 8554 Acked Flush 0 Sent Done Done false
31902 8555 Acked Flush 0 Sent Done Done false
31903 8556 Acked Write 5 Sent Done Done false
31904 8557 Acked Write 1 Sent Done Done false
31905 8558 Acked Write 9 Sent Done Done false
31906 8559 Acked Write 6 Sent Done Done false
31907 8560 Acked Write 1 Sent Done Done false
31908 8561 Acked Read 2 Sent Done Done false
31909 8562 Acked Write 2 Sent Done Done false
31910 8563 Acked Read 4 Sent Done Done false
31911 8564 Acked Read 7 Sent Done Done false
31912 8565 Acked Read 4 Sent Done Done false
31913 8566 Acked Read 10 Sent Done Done false
31914 8567 Acked Write 3 Sent Done Done false
31915 8568 Acked Read 4 Sent Done Done false
31916 8569 Acked Read 3 Sent Done Done false
31917 8570 Acked Flush 0 Sent Done Done false
31918 8571 Acked Write 7 Sent Done Done false
31919 8572 Acked Write 1 Sent Done Done false
31920 8573 Acked Read 5 Sent Done Done false
31921 8574 Acked Write 2 Sent Done Done false
31922 8575 Acked Write 1 Sent Done Done false
31923 8576 Acked Write 2 Sent Done Done false
31924 8577 Acked Read 1 Sent Done Done false
31925 8578 Acked Flush 0 Sent Done Done false
31926 8579 Acked Write 3 Sent Done Done false
31927 8580 Acked Write 9 Sent Done Done false
31928 8581 Acked Write 1 Sent Done Done false
31929 8582 Acked Write 10 Sent Done Done false
31930 8583 Acked Write 5 Sent Done Done false
31931 8584 Acked Read 8 Sent Done Done false
31932 8585 Acked Read 7 Sent Done Done false
31933 8586 Acked Write 7 Sent Done Done false
31934 8587 Acked Flush 0 Sent Done Done false
31935 8588 Acked Read 10 Sent Done Done false
31936 8589 Acked Write 7 Sent Done Done false
31937 8590 Acked Read 4 Sent Done Done false
31938 8591 Acked Read 2 Sent Done Done false
31939 8592 Acked Flush 0 Sent Done Done false
31940 8593 Acked Write 7 Sent Done Done false
31941 8594 Acked Write 5 Sent Done Done false
31942 8595 Acked Read 10 Sent Done Done false
31943 8596 Acked Read 1 Sent Done Done false
31944 8597 Acked Read 2 Sent Done Done false
31945 8598 Acked Read 5 Sent Done Done false
31946 8599 Acked Read 4 Sent Done Done false
31947 8600 Acked Write 5 Sent Done Done false
31948 8601 Acked Write 5 Sent Done Done false
31949 8602 Acked Write 7 Sent Done Done false
31950 8603 Acked Read 3 Sent Done Done false
31951 8604 Acked Write 6 Sent Done Done false
31952 8605 Acked Read 7 Sent Done Done false
31953 8606 Acked Read 3 Sent Done Done false
31954 8607 Acked Read 5 Sent Done Done false
31955 8608 Acked Write 1 Sent Done Done false
31956 8609 Acked Read 3 Sent Done Done false
31957 8610 Acked Write 2 Sent Done Done false
31958 8611 Acked Read 7 Sent Done Done false
31959 8612 Acked Flush 0 Sent Done Done false
31960 8613 Acked Write 9 Sent Done Done false
31961 8614 Acked Read 3 Sent Done Done false
31962 8615 Acked Read 10 Sent Done Done false
31963 8616 Acked Read 4 Sent Done Done false
31964 8617 Acked Write 3 Sent Done Done false
31965 8618 Acked Write 9 Sent Done Done false
31966 8619 Acked Read 10 Sent Done Done false
31967 8620 Acked Read 3 Sent Done Done false
31968 8621 Acked Read 3 Sent Done Done false
31969 8622 Acked Read 9 Sent Done Done false
31970 8623 Acked Write 7 Sent Done Done false
31971 8624 Acked Read 7 Sent Done Done false
31972 8625 Acked Read 3 Sent Done Done false
31973 8626 Acked Write 7 Sent Done Done false
31974 8627 Acked Read 6 Sent Done Done false
31975 8628 Acked Flush 0 Sent Done Done false
31976 8629 Acked Flush 0 Sent Done Done false
31977 8630 Acked Read 8 Sent Done Done false
31978 8631 Acked Read 6 Sent Done Done false
31979 8632 Acked Write 10 Sent Done Done false
31980 8633 Acked Read 2 Sent Done Done false
31981 8634 Acked Flush 0 Sent Done Done false
31982 8635 Acked Write 5 Sent Done Done false
31983 8636 Acked Write 3 Sent Done Done false
31984 8637 Acked Write 10 Sent Done Done false
31985 8638 Acked Read 6 Sent Done Done false
31986 8639 Acked Read 9 Sent Done Done false
31987 8640 Acked Read 5 Sent Done Done false
31988 8641 Acked Write 3 Sent Done Done false
31989 8642 Acked Write 3 Sent Done Done false
31990 8643 Acked Write 3 Sent Done Done false
31991 8644 Acked Write 8 Sent Done Done false
31992 8645 Acked Write 4 Sent Done Done false
31993 8646 Acked Write 1 Sent Done Done false
31994 8647 Acked Read 8 Sent Done Done false
31995 8648 Acked Write 8 Sent Done Done false
31996 8649 Acked Read 4 Sent Done Done false
31997 8650 Acked Read 2 Sent Done Done false
31998 8651 Acked Read 9 Sent Done Done false
31999 8652 Acked Read 5 Sent Done Done false
32000 8653 Acked Write 3 Sent Done Done false
32001 8654 Acked Read 3 Sent Done Done false
32002 8655 Acked Read 10 Sent Done Done false
32003 8656 Acked Write 8 Sent Done Done false
32004 8657 Acked Read 5 Sent Done Done false
32005 8658 Acked Flush 0 Sent Done Done false
32006 8659 Acked Flush 0 Sent Done Done false
32007 8660 Acked Write 8 Sent Done Done false
32008 8661 Acked Write 7 Sent Done Done false
32009 8662 Acked Read 6 Sent Done Done false
32010 8663 Acked Write 5 Sent Done Done false
32011 8664 Acked Write 5 Sent Done Done false
32012 8665 Acked Write 6 Sent Done Done false
32013 8666 Acked Read 7 Sent Done Done false
32014 8667 Acked Flush 0 Sent Done Done false
32015 8668 Acked Write 2 Sent Done Done false
32016 8669 Acked Write 9 Sent Done Done false
32017 8670 Acked Read 7 Sent Done Done false
32018 8671 Acked Read 5 Sent Done Done false
32019 8672 Acked Read 3 Sent Done Done false
32020 8673 Acked Read 9 Sent Done Done false
32021 8674 Acked Write 3 Sent Done Done false
32022 8675 Acked Write 3 Sent Done Done false
32023 8676 Acked Read 9 Sent Done Done false
32024 8677 Acked Write 3 Sent Done Done false
32025 8678 Acked Write 5 Sent Done Done false
32026 8679 Acked Read 4 Sent Done Done false
32027 8680 Acked Write 4 Sent Done Done false
32028 8681 Acked Read 6 Sent Done Done false
32029 8682 Acked Flush 0 Sent Done Done false
32030 8683 Acked Write 7 Sent Done Done false
32031 8684 Acked Read 6 Sent Done Done false
32032 8685 Acked Write 10 Sent Done Done false
32033 8686 Acked Read 6 Sent Done Done false
32034 8687 Acked Read 10 Sent Done Done false
32035 8688 Acked Read 8 Sent Done Done false
32036 8689 Acked Write 9 Sent Done Done false
32037 8690 Acked Read 4 Sent Done Done false
32038 8691 Acked Write 7 Sent Done Done false
32039 8692 Acked Flush 0 Sent Done Done false
32040 8693 Acked Write 2 Sent Done Done false
32041 8694 Acked Read 10 Sent Done Done false
32042 8695 Acked Read 10 Sent Done Done false
32043 8696 Acked Read 4 Sent Done Done false
32044 8697 Acked Read 5 Sent Done Done false
32045 8698 Acked Read 3 Sent Done Done false
32046 8699 Acked Flush 0 Sent Done Done false
32047 8700 Acked Read 2 Sent Done Done false
32048 8701 Acked Write 6 Sent Done Done false
32049 8702 Acked Write 1 Sent Done Done false
32050 8703 Acked Write 8 Sent Done Done false
32051 8704 Acked Read 7 Sent Done Done false
32052 8705 Acked Flush 0 Sent Done Done false
32053 8706 Acked Read 8 Sent Done Done false
32054 8707 Acked Write 10 Sent Done Done false
32055 8708 Acked Write 10 Sent Done Done false
32056 8709 Acked Read 1 Sent Done Done false
32057 8710 Acked Read 3 Sent Done Done false
32058 8711 Acked Read 4 Sent Done Done false
32059 8712 Acked Write 3 Sent Done Done false
32060 8713 Acked Write 1 Sent Done Done false
32061 8714 Acked Write 10 Sent Done Done false
32062 8715 Acked Read 3 Sent Done Done false
32063 8716 Acked Write 2 Sent Done Done false
32064 8717 Acked Write 2 Sent Done Done false
32065 8718 Acked Read 3 Sent Done Done false
32066 8719 Acked Read 8 Sent Done Done false
32067 8720 Acked Write 6 Sent Done Done false
32068 8721 Acked Write 5 Sent Done Done false
32069 8722 Acked Read 5 Sent Done Done false
32070 8723 Acked Write 7 Sent Done Done false
32071 8724 Acked Read 8 Sent Done Done false
32072 8725 Acked Write 4 Sent Done Done false
32073 8726 Acked Write 1 Sent Done Done false
32074 8727 Acked Read 10 Sent Done Done false
32075 8728 Acked Write 8 Sent Done Done false
32076 8729 Acked Read 7 Sent Done Done false
32077 8730 Acked Flush 0 Sent Done Done false
32078 8731 Acked Write 2 Sent Done Done false
32079 8732 Acked Read 2 Sent Done Done false
32080 8733 Acked Write 3 Sent Done Done false
32081 8734 Acked Read 6 Sent Done Done false
32082 8735 Acked Read 2 Sent Done Done false
32083 8736 Acked Write 1 Sent Done Done false
32084 8737 Acked Read 2 Sent Done Done false
32085 8738 Acked Flush 0 Sent Done Done false
32086 8739 Acked Read 7 Sent Done Done false
32087 8740 Acked Read 5 Sent Done Done false
32088 8741 Acked Write 10 Sent Done Done false
32089 8742 Acked Read 7 Sent Done Done false
32090 8743 Acked Write 3 Sent Done Done false
32091 8744 Acked Read 5 Sent Done Done false
32092 8745 Acked Write 9 Sent Done Done false
32093 8746 Acked Write 6 Sent Done Done false
32094 8747 Acked Read 5 Sent Done Done false
32095 8748 Acked Write 9 Sent Done Done false
32096 8749 Acked Write 5 Sent Done Done false
32097 8750 Acked Read 1 Sent Done Done false
32098 8751 Acked Read 3 Sent Done Done false
32099 8752 Acked Read 5 Sent Done Done false
32100 8753 Acked Read 1 Sent Done Done false
32101 8754 Acked Read 9 Sent Done Done false
32102 8755 Acked Read 1 Sent Done Done false
32103 8756 Acked Write 9 Sent Done Done false
32104 8757 Acked Read 8 Sent Done Done false
32105 8758 Acked Write 8 Sent Done Done false
32106 8759 Acked Read 8 Sent Done Done false
32107 8760 Acked Write 10 Sent Done Done false
32108 8761 Acked Flush 0 Sent Done Done false
32109 8762 Acked Write 7 Sent Done Done false
32110 8763 Acked Read 6 Sent Done Done false
32111 8764 Acked Read 6 Sent Done Done false
32112 8765 Acked Read 1 Sent Done Done false
32113 8766 Acked Read 9 Sent Done Done false
32114 8767 Acked Write 5 Sent Done Done false
32115 8768 Acked Write 5 Sent Done Done false
32116 8769 Acked Read 7 Sent Done Done false
32117 8770 Acked Read 7 Sent Done Done false
32118 8771 Acked Read 3 Sent Done Done false
32119 8772 Acked Flush 0 Sent Done Done false
32120 8773 Acked Read 6 Sent Done Done false
32121 8774 Acked Read 2 Sent Done Done false
32122 8775 Acked Flush 0 Sent Done Done false
32123 8776 Acked Write 2 Sent Done Done false
32124 8777 Acked Read 9 Sent Done Done false
32125 8778 Acked Write 8 Sent Done Done false
32126 8779 Acked Write 4 Sent Done Done false
32127 8780 Acked Write 3 Sent Done Done false
32128 8781 Acked Read 3 Sent Done Done false
32129 8782 Acked Write 3 Sent Done Done false
32130 8783 Acked Write 3 Sent Done Done false
32131 8784 Acked Write 1 Sent Done Done false
32132 8785 Acked Write 10 Sent Done Done false
32133 8786 Acked Write 6 Sent Done Done false
32134 8787 Acked Write 9 Sent Done Done false
32135 8788 Acked Write 8 Sent Done Done false
32136 8789 Acked Read 4 Sent Done Done false
32137 8790 Acked Write 8 Sent Done Done false
32138 8791 Acked Write 1 Sent Done Done false
32139 8792 Acked Read 1 Sent Done Done false
32140 8793 Acked Read 5 Sent Done Done false
32141 8794 Acked Read 2 Sent Done Done false
32142 8795 Acked Write 9 Sent Done Done false
32143 8796 Acked Write 10 Sent Done Done false
32144 8797 Acked Read 4 Sent Done Done false
32145 8798 Acked Read 1 Sent Done Done false
32146 8799 Acked Write 6 Sent Done Done false
32147 8800 Acked Write 9 Sent Done Done false
32148 8801 Acked Read 3 Sent Done Done false
32149 8802 Acked Write 8 Sent Done Done false
32150 8803 Acked Read 4 Sent Done Done false
32151 8804 Acked Read 4 Sent Done Done false
32152 8805 Acked Write 6 Sent Done Done false
32153 8806 Acked Write 10 Sent Done Done false
32154 8807 Acked Read 8 Sent Done Done false
32155 8808 Acked Write 9 Sent Done Done false
32156 8809 Acked Read 1 Sent Done Done false
32157 8810 Acked Read 8 Sent Done Done false
32158 8811 Acked Write 1 Sent Done Done false
32159 8812 Acked Write 5 Sent Done Done false
32160 8813 Acked Write 1 Sent Done Done false
32161 8814 Acked Read 9 Sent Done Done false
32162 8815 Acked Read 6 Sent Done Done false
32163 8816 Acked Write 6 Sent Done Done false
32164 8817 Acked Write 6 Sent Done Done false
32165 8818 Acked Read 7 Sent Done Done false
32166 8819 Acked Read 5 Sent Done Done false
32167 8820 Acked Read 2 Sent Done Done false
32168 8821 Acked Flush 0 Sent Done Done false
32169 8822 Acked Write 6 Sent Done Done false
32170 8823 Acked Write 6 Sent Done Done false
32171 8824 Acked Read 3 Sent Done Done false
32172 8825 Acked Flush 0 Sent Done Done false
32173 8826 Acked Read 7 Sent Done Done false
32174 8827 Acked Flush 0 Sent Done Done false
32175 8828 Acked Write 6 Sent Done Done false
32176 8829 Acked Write 4 Sent Done Done false
32177 8830 Acked Write 9 Sent Done Done false
32178 8831 Acked Write 7 Sent Done Done false
32179 8832 Acked Write 4 Sent Done Done false
32180 8833 Acked Write 6 Sent Done Done false
32181 8834 Acked Write 6 Sent Done Done false
32182 8835 Acked Write 9 Sent Done Done false
32183 8836 Acked Write 2 Sent Done Done false
32184 8837 Acked Write 3 Sent Done Done false
32185 8838 Acked Read 10 Sent Done Done false
32186 8839 Acked Read 4 Sent Done Done false
32187 8840 Acked Flush 0 Sent Done Done false
32188 8841 Acked Write 2 Sent Done Done false
32189 8842 Acked Write 1 Sent Done Done false
32190 8843 Acked Write 6 Sent Done Done false
32191 8844 Acked Write 6 Sent Done Done false
32192 8845 Acked Read 5 Sent Done Done false
32193 8846 Acked Read 10 Sent Done Done false
32194 8847 Acked Read 6 Sent Done Done false
32195 8848 Acked Read 1 Sent Done Done false
32196 STATES DS:0 DS:1 DS:2 TOTAL
32197 Sent 301 0 0 301
32198 Done 0 301 301 602
32199 Skipped 0 0 0 0
32200 Error 0 0 0 0
32201 Last Flush: 8547 8840 8840
32202 Downstairs last five completed: 8547 8546 8545 8544 8543
32203 Upstairs last five completed: 8848 8847 8846 8845 8844
32204 CLIENT: Up:0 ds:301 act:2
32205 ----------------------------------------------------------------
32206 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
32207 States: Offline Active Active
32208 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
32209 8548 Acked Read 6 Sent Done Done false
32210 8549 Acked Read 9 Sent Done Done false
32211 8550 Acked Read 1 Sent Done Done false
32212 8551 Acked Read 9 Sent Done Done false
32213 8552 Acked Read 10 Sent Done Done false
32214 8553 Acked Read 1 Sent Done Done false
32215 8554 Acked Flush 0 Sent Done Done false
32216 8555 Acked Flush 0 Sent Done Done false
32217 8556 Acked Write 5 Sent Done Done false
32218 8557 Acked Write 1 Sent Done Done false
32219 8558 Acked Write 9 Sent Done Done false
32220 8559 Acked Write 6 Sent Done Done false
32221 8560 Acked Write 1 Sent Done Done false
32222 8561 Acked Read 2 Sent Done Done false
32223 8562 Acked Write 2 Sent Done Done false
32224 8563 Acked Read 4 Sent Done Done false
32225 8564 Acked Read 7 Sent Done Done false
32226 8565 Acked Read 4 Sent Done Done false
32227 8566 Acked Read 10 Sent Done Done false
32228 8567 Acked Write 3 Sent Done Done false
32229 8568 Acked Read 4 Sent Done Done false
32230 8569 Acked Read 3 Sent Done Done false
32231 8570 Acked Flush 0 Sent Done Done false
32232 8571 Acked Write 7 Sent Done Done false
32233 8572 Acked Write 1 Sent Done Done false
32234 8573 Acked Read 5 Sent Done Done false
32235 8574 Acked Write 2 Sent Done Done false
32236 8575 Acked Write 1 Sent Done Done false
32237 8576 Acked Write 2 Sent Done Done false
32238 8577 Acked Read 1 Sent Done Done false
32239 8578 Acked Flush 0 Sent Done Done false
32240 8579 Acked Write 3 Sent Done Done false
32241 8580 Acked Write 9 Sent Done Done false
32242 8581 Acked Write 1 Sent Done Done false
32243 8582 Acked Write 10 Sent Done Done false
32244 8583 Acked Write 5 Sent Done Done false
32245 8584 Acked Read 8 Sent Done Done false
32246 8585 Acked Read 7 Sent Done Done false
32247 8586 Acked Write 7 Sent Done Done false
32248 8587 Acked Flush 0 Sent Done Done false
32249 8588 Acked Read 10 Sent Done Done false
32250 8589 Acked Write 7 Sent Done Done false
32251 8590 Acked Read 4 Sent Done Done false
32252 8591 Acked Read 2 Sent Done Done false
32253 8592 Acked Flush 0 Sent Done Done false
32254 8593 Acked Write 7 Sent Done Done false
32255 8594 Acked Write 5 Sent Done Done false
32256 8595 Acked Read 10 Sent Done Done false
32257 8596 Acked Read 1 Sent Done Done false
32258 8597 Acked Read 2 Sent Done Done false
32259 8598 Acked Read 5 Sent Done Done false
32260 8599 Acked Read 4 Sent Done Done false
32261 8600 Acked Write 5 Sent Done Done false
32262 8601 Acked Write 5 Sent Done Done false
32263 8602 Acked Write 7 Sent Done Done false
32264 8603 Acked Read 3 Sent Done Done false
32265 8604 Acked Write 6 Sent Done Done false
32266 8605 Acked Read 7 Sent Done Done false
32267 8606 Acked Read 3 Sent Done Done false
32268 8607 Acked Read 5 Sent Done Done false
32269 8608 Acked Write 1 Sent Done Done false
32270 8609 Acked Read 3 Sent Done Done false
32271 8610 Acked Write 2 Sent Done Done false
32272 8611 Acked Read 7 Sent Done Done false
32273 8612 Acked Flush 0 Sent Done Done false
32274 8613 Acked Write 9 Sent Done Done false
32275 8614 Acked Read 3 Sent Done Done false
32276 8615 Acked Read 10 Sent Done Done false
32277 8616 Acked Read 4 Sent Done Done false
32278 8617 Acked Write 3 Sent Done Done false
32279 8618 Acked Write 9 Sent Done Done false
32280 8619 Acked Read 10 Sent Done Done false
32281 8620 Acked Read 3 Sent Done Done false
32282 8621 Acked Read 3 Sent Done Done false
32283 8622 Acked Read 9 Sent Done Done false
32284 8623 Acked Write 7 Sent Done Done false
32285 8624 Acked Read 7 Sent Done Done false
32286 8625 Acked Read 3 Sent Done Done false
32287 8626 Acked Write 7 Sent Done Done false
32288 8627 Acked Read 6 Sent Done Done false
32289 8628 Acked Flush 0 Sent Done Done false
32290 8629 Acked Flush 0 Sent Done Done false
32291 8630 Acked Read 8 Sent Done Done false
32292 8631 Acked Read 6 Sent Done Done false
32293 8632 Acked Write 10 Sent Done Done false
32294 8633 Acked Read 2 Sent Done Done false
32295 8634 Acked Flush 0 Sent Done Done false
32296 8635 Acked Write 5 Sent Done Done false
32297 8636 Acked Write 3 Sent Done Done false
32298 8637 Acked Write 10 Sent Done Done false
32299 8638 Acked Read 6 Sent Done Done false
32300 8639 Acked Read 9 Sent Done Done false
32301 8640 Acked Read 5 Sent Done Done false
32302 8641 Acked Write 3 Sent Done Done false
32303 8642 Acked Write 3 Sent Done Done false
32304 8643 Acked Write 3 Sent Done Done false
32305 8644 Acked Write 8 Sent Done Done false
32306 8645 Acked Write 4 Sent Done Done false
32307 8646 Acked Write 1 Sent Done Done false
32308 8647 Acked Read 8 Sent Done Done false
32309 8648 Acked Write 8 Sent Done Done false
32310 8649 Acked Read 4 Sent Done Done false
32311 8650 Acked Read 2 Sent Done Done false
32312 8651 Acked Read 9 Sent Done Done false
32313 8652 Acked Read 5 Sent Done Done false
32314 8653 Acked Write 3 Sent Done Done false
32315 8654 Acked Read 3 Sent Done Done false
32316 8655 Acked Read 10 Sent Done Done false
32317 8656 Acked Write 8 Sent Done Done false
32318 8657 Acked Read 5 Sent Done Done false
32319 8658 Acked Flush 0 Sent Done Done false
32320 8659 Acked Flush 0 Sent Done Done false
32321 8660 Acked Write 8 Sent Done Done false
32322 8661 Acked Write 7 Sent Done Done false
32323 8662 Acked Read 6 Sent Done Done false
32324 8663 Acked Write 5 Sent Done Done false
32325 8664 Acked Write 5 Sent Done Done false
32326 8665 Acked Write 6 Sent Done Done false
32327 8666 Acked Read 7 Sent Done Done false
32328 8667 Acked Flush 0 Sent Done Done false
32329 8668 Acked Write 2 Sent Done Done false
32330 8669 Acked Write 9 Sent Done Done false
32331 8670 Acked Read 7 Sent Done Done false
32332 8671 Acked Read 5 Sent Done Done false
32333 8672 Acked Read 3 Sent Done Done false
32334 8673 Acked Read 9 Sent Done Done false
32335 8674 Acked Write 3 Sent Done Done false
32336 8675 Acked Write 3 Sent Done Done false
32337 8676 Acked Read 9 Sent Done Done false
32338 8677 Acked Write 3 Sent Done Done false
32339 8678 Acked Write 5 Sent Done Done false
32340 8679 Acked Read 4 Sent Done Done false
32341 8680 Acked Write 4 Sent Done Done false
32342 8681 Acked Read 6 Sent Done Done false
32343 8682 Acked Flush 0 Sent Done Done false
32344 8683 Acked Write 7 Sent Done Done false
32345 8684 Acked Read 6 Sent Done Done false
32346 8685 Acked Write 10 Sent Done Done false
32347 8686 Acked Read 6 Sent Done Done false
32348 8687 Acked Read 10 Sent Done Done false
32349 8688 Acked Read 8 Sent Done Done false
32350 8689 Acked Write 9 Sent Done Done false
32351 8690 Acked Read 4 Sent Done Done false
32352 8691 Acked Write 7 Sent Done Done false
32353 8692 Acked Flush 0 Sent Done Done false
32354 8693 Acked Write 2 Sent Done Done false
32355 8694 Acked Read 10 Sent Done Done false
32356 8695 Acked Read 10 Sent Done Done false
32357 8696 Acked Read 4 Sent Done Done false
32358 8697 Acked Read 5 Sent Done Done false
32359 8698 Acked Read 3 Sent Done Done false
32360 8699 Acked Flush 0 Sent Done Done false
32361 8700 Acked Read 2 Sent Done Done false
32362 8701 Acked Write 6 Sent Done Done false
32363 8702 Acked Write 1 Sent Done Done false
32364 8703 Acked Write 8 Sent Done Done false
32365 8704 Acked Read 7 Sent Done Done false
32366 8705 Acked Flush 0 Sent Done Done false
32367 8706 Acked Read 8 Sent Done Done false
32368 8707 Acked Write 10 Sent Done Done false
32369 8708 Acked Write 10 Sent Done Done false
32370 8709 Acked Read 1 Sent Done Done false
32371 8710 Acked Read 3 Sent Done Done false
32372 8711 Acked Read 4 Sent Done Done false
32373 8712 Acked Write 3 Sent Done Done false
32374 8713 Acked Write 1 Sent Done Done false
32375 8714 Acked Write 10 Sent Done Done false
32376 8715 Acked Read 3 Sent Done Done false
32377 8716 Acked Write 2 Sent Done Done false
32378 8717 Acked Write 2 Sent Done Done false
32379 8718 Acked Read 3 Sent Done Done false
32380 8719 Acked Read 8 Sent Done Done false
32381 8720 Acked Write 6 Sent Done Done false
32382 8721 Acked Write 5 Sent Done Done false
32383 8722 Acked Read 5 Sent Done Done false
32384 8723 Acked Write 7 Sent Done Done false
32385 8724 Acked Read 8 Sent Done Done false
32386 8725 Acked Write 4 Sent Done Done false
32387 8726 Acked Write 1 Sent Done Done false
32388 8727 Acked Read 10 Sent Done Done false
32389 8728 Acked Write 8 Sent Done Done false
32390 8729 Acked Read 7 Sent Done Done false
32391 8730 Acked Flush 0 Sent Done Done false
32392 8731 Acked Write 2 Sent Done Done false
32393 8732 Acked Read 2 Sent Done Done false
32394 8733 Acked Write 3 Sent Done Done false
32395 8734 Acked Read 6 Sent Done Done false
32396 8735 Acked Read 2 Sent Done Done false
32397 8736 Acked Write 1 Sent Done Done false
32398 8737 Acked Read 2 Sent Done Done false
32399 8738 Acked Flush 0 Sent Done Done false
32400 8739 Acked Read 7 Sent Done Done false
32401 8740 Acked Read 5 Sent Done Done false
32402 8741 Acked Write 10 Sent Done Done false
32403 8742 Acked Read 7 Sent Done Done false
32404 8743 Acked Write 3 Sent Done Done false
32405 8744 Acked Read 5 Sent Done Done false
32406 8745 Acked Write 9 Sent Done Done false
32407 8746 Acked Write 6 Sent Done Done false
32408 8747 Acked Read 5 Sent Done Done false
32409 8748 Acked Write 9 Sent Done Done false
32410 8749 Acked Write 5 Sent Done Done false
32411 8750 Acked Read 1 Sent Done Done false
32412 8751 Acked Read 3 Sent Done Done false
32413 8752 Acked Read 5 Sent Done Done false
32414 8753 Acked Read 1 Sent Done Done false
32415 8754 Acked Read 9 Sent Done Done false
32416 8755 Acked Read 1 Sent Done Done false
32417 8756 Acked Write 9 Sent Done Done false
32418 8757 Acked Read 8 Sent Done Done false
32419 8758 Acked Write 8 Sent Done Done false
32420 8759 Acked Read 8 Sent Done Done false
32421 8760 Acked Write 10 Sent Done Done false
32422 8761 Acked Flush 0 Sent Done Done false
32423 8762 Acked Write 7 Sent Done Done false
32424 8763 Acked Read 6 Sent Done Done false
32425 8764 Acked Read 6 Sent Done Done false
32426 8765 Acked Read 1 Sent Done Done false
32427 8766 Acked Read 9 Sent Done Done false
32428 8767 Acked Write 5 Sent Done Done false
32429 8768 Acked Write 5 Sent Done Done false
32430 8769 Acked Read 7 Sent Done Done false
32431 8770 Acked Read 7 Sent Done Done false
32432 8771 Acked Read 3 Sent Done Done false
32433 8772 Acked Flush 0 Sent Done Done false
32434 8773 Acked Read 6 Sent Done Done false
32435 8774 Acked Read 2 Sent Done Done false
32436 8775 Acked Flush 0 Sent Done Done false
32437 8776 Acked Write 2 Sent Done Done false
32438 8777 Acked Read 9 Sent Done Done false
32439 8778 Acked Write 8 Sent Done Done false
32440 8779 Acked Write 4 Sent Done Done false
32441 8780 Acked Write 3 Sent Done Done false
32442 8781 Acked Read 3 Sent Done Done false
32443 8782 Acked Write 3 Sent Done Done false
32444 8783 Acked Write 3 Sent Done Done false
32445 8784 Acked Write 1 Sent Done Done false
32446 8785 Acked Write 10 Sent Done Done false
32447 8786 Acked Write 6 Sent Done Done false
32448 8787 Acked Write 9 Sent Done Done false
32449 8788 Acked Write 8 Sent Done Done false
32450 8789 Acked Read 4 Sent Done Done false
32451 8790 Acked Write 8 Sent Done Done false
32452 8791 Acked Write 1 Sent Done Done false
32453 8792 Acked Read 1 Sent Done Done false
32454 8793 Acked Read 5 Sent Done Done false
32455 8794 Acked Read 2 Sent Done Done false
32456 8795 Acked Write 9 Sent Done Done false
32457 8796 Acked Write 10 Sent Done Done false
32458 8797 Acked Read 4 Sent Done Done false
32459 8798 Acked Read 1 Sent Done Done false
32460 8799 Acked Write 6 Sent Done Done false
32461 8800 Acked Write 9 Sent Done Done false
32462 8801 Acked Read 3 Sent Done Done false
32463 8802 Acked Write 8 Sent Done Done false
32464 8803 Acked Read 4 Sent Done Done false
32465 8804 Acked Read 4 Sent Done Done false
32466 8805 Acked Write 6 Sent Done Done false
32467 8806 Acked Write 10 Sent Done Done false
32468 8807 Acked Read 8 Sent Done Done false
32469 8808 Acked Write 9 Sent Done Done false
32470 8809 Acked Read 1 Sent Done Done false
32471 8810 Acked Read 8 Sent Done Done false
32472 8811 Acked Write 1 Sent Done Done false
32473 8812 Acked Write 5 Sent Done Done false
32474 8813 Acked Write 1 Sent Done Done false
32475 8814 Acked Read 9 Sent Done Done false
32476 8815 Acked Read 6 Sent Done Done false
32477 8816 Acked Write 6 Sent Done Done false
32478 8817 Acked Write 6 Sent Done Done false
32479 8818 Acked Read 7 Sent Done Done false
32480 8819 Acked Read 5 Sent Done Done false
32481 8820 Acked Read 2 Sent Done Done false
32482 8821 Acked Flush 0 Sent Done Done false
32483 8822 Acked Write 6 Sent Done Done false
32484 8823 Acked Write 6 Sent Done Done false
32485 8824 Acked Read 3 Sent Done Done false
32486 8825 Acked Flush 0 Sent Done Done false
32487 8826 Acked Read 7 Sent Done Done false
32488 8827 Acked Flush 0 Sent Done Done false
32489 8828 Acked Write 6 Sent Done Done false
32490 8829 Acked Write 4 Sent Done Done false
32491 8830 Acked Write 9 Sent Done Done false
32492 8831 Acked Write 7 Sent Done Done false
32493 8832 Acked Write 4 Sent Done Done false
32494 8833 Acked Write 6 Sent Done Done false
32495 8834 Acked Write 6 Sent Done Done false
32496 8835 Acked Write 9 Sent Done Done false
32497 8836 Acked Write 2 Sent Done Done false
32498 8837 Acked Write 3 Sent Done Done false
32499 8838 Acked Read 10 Sent Done Done false
32500 8839 Acked Read 4 Sent Done Done false
32501 8840 Acked Flush 0 Sent Done Done false
32502 8841 Acked Write 2 Sent Done Done false
32503 8842 Acked Write 1 Sent Done Done false
32504 8843 Acked Write 6 Sent Done Done false
32505 8844 Acked Write 6 Sent Done Done false
32506 8845 Acked Read 5 Sent Done Done false
32507 8846 Acked Read 10 Sent Done Done false
32508 8847 Acked Read 6 Sent Done Done false
32509 8848 Acked Read 1 Sent Done Done false
32510 8849 Acked Flush 0 Sent Done Done false
32511 STATES DS:0 DS:1 DS:2 TOTAL
32512 Sent 302 0 0 302
32513 Done 0 302 302 604
32514 Skipped 0 0 0 0
32515 Error 0 0 0 0
32516 Last Flush: 8547 8849 8849
32517 Downstairs last five completed: 8547 8546 8545 8544 8543
32518 Upstairs last five completed: 8849 8848 8847 8846 8845
32519 CLIENT: Up:0 ds:302 act:2
325202024-11-15T02:04:16.057ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
325212024-11-15T02:04:16.057ZWARNcrucible: ds_connection connect to 127.0.0.1:8810 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
325222024-11-15T02:04:16.057ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
325232024-11-15T02:04:16.057ZWARNcrucible: downstairs task for 0 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
325242024-11-15T02:04:16.057ZINFOcrucible: client task is exiting = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
32525 [8810] got start action
32526 [8810] I am going to restart
32527 Starting downstairs at port 8810
32528 Make output file at "/tmp/dsc/downstairs-8810.txt"
32529 [8810][0] reports Starting
32530 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1231)
32531 [8810][0] reports Running
32532 ----------------------------------------------------------------
32533 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
32534 States: Offline Active Active
32535 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
32536 8548 Acked Read 6 Sent Done Done false
32537 8549 Acked Read 9 Sent Done Done false
32538 8550 Acked Read 1 Sent Done Done false
32539 8551 Acked Read 9 Sent Done Done false
32540 8552 Acked Read 10 Sent Done Done false
32541 8553 Acked Read 1 Sent Done Done false
32542 8554 Acked Flush 0 Sent Done Done false
32543 8555 Acked Flush 0 Sent Done Done false
32544 8556 Acked Write 5 Sent Done Done false
32545 8557 Acked Write 1 Sent Done Done false
32546 8558 Acked Write 9 Sent Done Done false
32547 8559 Acked Write 6 Sent Done Done false
32548 8560 Acked Write 1 Sent Done Done false
32549 8561 Acked Read 2 Sent Done Done false
32550 8562 Acked Write 2 Sent Done Done false
32551 8563 Acked Read 4 Sent Done Done false
32552 8564 Acked Read 7 Sent Done Done false
32553 8565 Acked Read 4 Sent Done Done false
32554 8566 Acked Read 10 Sent Done Done false
32555 8567 Acked Write 3 Sent Done Done false
32556 8568 Acked Read 4 Sent Done Done false
32557 8569 Acked Read 3 Sent Done Done false
32558 8570 Acked Flush 0 Sent Done Done false
32559 8571 Acked Write 7 Sent Done Done false
32560 8572 Acked Write 1 Sent Done Done false
32561 8573 Acked Read 5 Sent Done Done false
32562 8574 Acked Write 2 Sent Done Done false
32563 8575 Acked Write 1 Sent Done Done false
32564 8576 Acked Write 2 Sent Done Done false
32565 8577 Acked Read 1 Sent Done Done false
32566 8578 Acked Flush 0 Sent Done Done false
32567 8579 Acked Write 3 Sent Done Done false
32568 8580 Acked Write 9 Sent Done Done false
32569 8581 Acked Write 1 Sent Done Done false
32570 8582 Acked Write 10 Sent Done Done false
32571 8583 Acked Write 5 Sent Done Done false
32572 8584 Acked Read 8 Sent Done Done false
32573 8585 Acked Read 7 Sent Done Done false
32574 8586 Acked Write 7 Sent Done Done false
32575 8587 Acked Flush 0 Sent Done Done false
32576 8588 Acked Read 10 Sent Done Done false
32577 8589 Acked Write 7 Sent Done Done false
32578 8590 Acked Read 4 Sent Done Done false
32579 8591 Acked Read 2 Sent Done Done false
32580 8592 Acked Flush 0 Sent Done Done false
32581 8593 Acked Write 7 Sent Done Done false
32582 8594 Acked Write 5 Sent Done Done false
32583 8595 Acked Read 10 Sent Done Done false
32584 8596 Acked Read 1 Sent Done Done false
32585 8597 Acked Read 2 Sent Done Done false
32586 8598 Acked Read 5 Sent Done Done false
32587 8599 Acked Read 4 Sent Done Done false
32588 8600 Acked Write 5 Sent Done Done false
32589 8601 Acked Write 5 Sent Done Done false
32590 8602 Acked Write 7 Sent Done Done false
32591 8603 Acked Read 3 Sent Done Done false
32592 8604 Acked Write 6 Sent Done Done false
32593 8605 Acked Read 7 Sent Done Done false
32594 8606 Acked Read 3 Sent Done Done false
32595 8607 Acked Read 5 Sent Done Done false
32596 8608 Acked Write 1 Sent Done Done false
32597 8609 Acked Read 3 Sent Done Done false
32598 8610 Acked Write 2 Sent Done Done false
32599 8611 Acked Read 7 Sent Done Done false
32600 8612 Acked Flush 0 Sent Done Done false
32601 8613 Acked Write 9 Sent Done Done false
32602 8614 Acked Read 3 Sent Done Done false
32603 8615 Acked Read 10 Sent Done Done false
32604 8616 Acked Read 4 Sent Done Done false
32605 8617 Acked Write 3 Sent Done Done false
32606 8618 Acked Write 9 Sent Done Done false
32607 8619 Acked Read 10 Sent Done Done false
32608 8620 Acked Read 3 Sent Done Done false
32609 8621 Acked Read 3 Sent Done Done false
32610 8622 Acked Read 9 Sent Done Done false
32611 8623 Acked Write 7 Sent Done Done false
32612 8624 Acked Read 7 Sent Done Done false
32613 8625 Acked Read 3 Sent Done Done false
32614 8626 Acked Write 7 Sent Done Done false
32615 8627 Acked Read 6 Sent Done Done false
32616 8628 Acked Flush 0 Sent Done Done false
32617 8629 Acked Flush 0 Sent Done Done false
32618 8630 Acked Read 8 Sent Done Done false
32619 8631 Acked Read 6 Sent Done Done false
32620 8632 Acked Write 10 Sent Done Done false
32621 8633 Acked Read 2 Sent Done Done false
32622 8634 Acked Flush 0 Sent Done Done false
32623 8635 Acked Write 5 Sent Done Done false
32624 8636 Acked Write 3 Sent Done Done false
32625 8637 Acked Write 10 Sent Done Done false
32626 8638 Acked Read 6 Sent Done Done false
32627 8639 Acked Read 9 Sent Done Done false
32628 8640 Acked Read 5 Sent Done Done false
32629 8641 Acked Write 3 Sent Done Done false
32630 8642 Acked Write 3 Sent Done Done false
32631 8643 Acked Write 3 Sent Done Done false
32632 8644 Acked Write 8 Sent Done Done false
32633 8645 Acked Write 4 Sent Done Done false
32634 8646 Acked Write 1 Sent Done Done false
32635 8647 Acked Read 8 Sent Done Done false
32636 8648 Acked Write 8 Sent Done Done false
32637 8649 Acked Read 4 Sent Done Done false
32638 8650 Acked Read 2 Sent Done Done false
32639 8651 Acked Read 9 Sent Done Done false
32640 8652 Acked Read 5 Sent Done Done false
32641 8653 Acked Write 3 Sent Done Done false
32642 8654 Acked Read 3 Sent Done Done false
32643 8655 Acked Read 10 Sent Done Done false
32644 8656 Acked Write 8 Sent Done Done false
32645 8657 Acked Read 5 Sent Done Done false
32646 8658 Acked Flush 0 Sent Done Done false
32647 8659 Acked Flush 0 Sent Done Done false
32648 8660 Acked Write 8 Sent Done Done false
32649 8661 Acked Write 7 Sent Done Done false
32650 8662 Acked Read 6 Sent Done Done false
32651 8663 Acked Write 5 Sent Done Done false
32652 8664 Acked Write 5 Sent Done Done false
32653 8665 Acked Write 6 Sent Done Done false
32654 8666 Acked Read 7 Sent Done Done false
32655 8667 Acked Flush 0 Sent Done Done false
32656 8668 Acked Write 2 Sent Done Done false
32657 8669 Acked Write 9 Sent Done Done false
32658 8670 Acked Read 7 Sent Done Done false
32659 8671 Acked Read 5 Sent Done Done false
32660 8672 Acked Read 3 Sent Done Done false
32661 8673 Acked Read 9 Sent Done Done false
32662 8674 Acked Write 3 Sent Done Done false
32663 8675 Acked Write 3 Sent Done Done false
32664 8676 Acked Read 9 Sent Done Done false
32665 8677 Acked Write 3 Sent Done Done false
32666 8678 Acked Write 5 Sent Done Done false
32667 8679 Acked Read 4 Sent Done Done false
32668 8680 Acked Write 4 Sent Done Done false
32669 8681 Acked Read 6 Sent Done Done false
32670 8682 Acked Flush 0 Sent Done Done false
32671 8683 Acked Write 7 Sent Done Done false
32672 8684 Acked Read 6 Sent Done Done false
32673 8685 Acked Write 10 Sent Done Done false
32674 8686 Acked Read 6 Sent Done Done false
32675 8687 Acked Read 10 Sent Done Done false
32676 8688 Acked Read 8 Sent Done Done false
32677 8689 Acked Write 9 Sent Done Done false
32678 8690 Acked Read 4 Sent Done Done false
32679 8691 Acked Write 7 Sent Done Done false
32680 8692 Acked Flush 0 Sent Done Done false
32681 8693 Acked Write 2 Sent Done Done false
32682 8694 Acked Read 10 Sent Done Done false
32683 8695 Acked Read 10 Sent Done Done false
32684 8696 Acked Read 4 Sent Done Done false
32685 8697 Acked Read 5 Sent Done Done false
32686 8698 Acked Read 3 Sent Done Done false
32687 8699 Acked Flush 0 Sent Done Done false
32688 8700 Acked Read 2 Sent Done Done false
32689 8701 Acked Write 6 Sent Done Done false
32690 8702 Acked Write 1 Sent Done Done false
32691 8703 Acked Write 8 Sent Done Done false
32692 8704 Acked Read 7 Sent Done Done false
32693 8705 Acked Flush 0 Sent Done Done false
32694 8706 Acked Read 8 Sent Done Done false
32695 8707 Acked Write 10 Sent Done Done false
32696 8708 Acked Write 10 Sent Done Done false
32697 8709 Acked Read 1 Sent Done Done false
32698 8710 Acked Read 3 Sent Done Done false
32699 8711 Acked Read 4 Sent Done Done false
32700 8712 Acked Write 3 Sent Done Done false
32701 8713 Acked Write 1 Sent Done Done false
32702 8714 Acked Write 10 Sent Done Done false
32703 8715 Acked Read 3 Sent Done Done false
32704 8716 Acked Write 2 Sent Done Done false
32705 8717 Acked Write 2 Sent Done Done false
32706 8718 Acked Read 3 Sent Done Done false
32707 8719 Acked Read 8 Sent Done Done false
32708 8720 Acked Write 6 Sent Done Done false
32709 8721 Acked Write 5 Sent Done Done false
32710 8722 Acked Read 5 Sent Done Done false
32711 8723 Acked Write 7 Sent Done Done false
32712 8724 Acked Read 8 Sent Done Done false
32713 8725 Acked Write 4 Sent Done Done false
32714 8726 Acked Write 1 Sent Done Done false
32715 8727 Acked Read 10 Sent Done Done false
32716 8728 Acked Write 8 Sent Done Done false
32717 8729 Acked Read 7 Sent Done Done false
32718 8730 Acked Flush 0 Sent Done Done false
32719 8731 Acked Write 2 Sent Done Done false
32720 8732 Acked Read 2 Sent Done Done false
32721 8733 Acked Write 3 Sent Done Done false
32722 8734 Acked Read 6 Sent Done Done false
32723 8735 Acked Read 2 Sent Done Done false
32724 8736 Acked Write 1 Sent Done Done false
32725 8737 Acked Read 2 Sent Done Done false
32726 8738 Acked Flush 0 Sent Done Done false
32727 8739 Acked Read 7 Sent Done Done false
32728 8740 Acked Read 5 Sent Done Done false
32729 8741 Acked Write 10 Sent Done Done false
32730 8742 Acked Read 7 Sent Done Done false
32731 8743 Acked Write 3 Sent Done Done false
32732 8744 Acked Read 5 Sent Done Done false
32733 8745 Acked Write 9 Sent Done Done false
32734 8746 Acked Write 6 Sent Done Done false
32735 8747 Acked Read 5 Sent Done Done false
32736 8748 Acked Write 9 Sent Done Done false
32737 8749 Acked Write 5 Sent Done Done false
32738 8750 Acked Read 1 Sent Done Done false
32739 8751 Acked Read 3 Sent Done Done false
32740 8752 Acked Read 5 Sent Done Done false
32741 8753 Acked Read 1 Sent Done Done false
32742 8754 Acked Read 9 Sent Done Done false
32743 8755 Acked Read 1 Sent Done Done false
32744 8756 Acked Write 9 Sent Done Done false
32745 8757 Acked Read 8 Sent Done Done false
32746 8758 Acked Write 8 Sent Done Done false
32747 8759 Acked Read 8 Sent Done Done false
32748 8760 Acked Write 10 Sent Done Done false
32749 8761 Acked Flush 0 Sent Done Done false
32750 8762 Acked Write 7 Sent Done Done false
32751 8763 Acked Read 6 Sent Done Done false
32752 8764 Acked Read 6 Sent Done Done false
32753 8765 Acked Read 1 Sent Done Done false
32754 8766 Acked Read 9 Sent Done Done false
32755 8767 Acked Write 5 Sent Done Done false
32756 8768 Acked Write 5 Sent Done Done false
32757 8769 Acked Read 7 Sent Done Done false
32758 8770 Acked Read 7 Sent Done Done false
32759 8771 Acked Read 3 Sent Done Done false
32760 8772 Acked Flush 0 Sent Done Done false
32761 8773 Acked Read 6 Sent Done Done false
32762 8774 Acked Read 2 Sent Done Done false
32763 8775 Acked Flush 0 Sent Done Done false
32764 8776 Acked Write 2 Sent Done Done false
32765 8777 Acked Read 9 Sent Done Done false
32766 8778 Acked Write 8 Sent Done Done false
32767 8779 Acked Write 4 Sent Done Done false
32768 8780 Acked Write 3 Sent Done Done false
32769 8781 Acked Read 3 Sent Done Done false
32770 8782 Acked Write 3 Sent Done Done false
32771 8783 Acked Write 3 Sent Done Done false
32772 8784 Acked Write 1 Sent Done Done false
32773 8785 Acked Write 10 Sent Done Done false
32774 8786 Acked Write 6 Sent Done Done false
32775 8787 Acked Write 9 Sent Done Done false
32776 8788 Acked Write 8 Sent Done Done false
32777 8789 Acked Read 4 Sent Done Done false
32778 8790 Acked Write 8 Sent Done Done false
32779 8791 Acked Write 1 Sent Done Done false
32780 8792 Acked Read 1 Sent Done Done false
32781 8793 Acked Read 5 Sent Done Done false
32782 8794 Acked Read 2 Sent Done Done false
32783 8795 Acked Write 9 Sent Done Done false
32784 8796 Acked Write 10 Sent Done Done false
32785 8797 Acked Read 4 Sent Done Done false
32786 8798 Acked Read 1 Sent Done Done false
32787 8799 Acked Write 6 Sent Done Done false
32788 8800 Acked Write 9 Sent Done Done false
32789 8801 Acked Read 3 Sent Done Done false
32790 8802 Acked Write 8 Sent Done Done false
32791 8803 Acked Read 4 Sent Done Done false
32792 8804 Acked Read 4 Sent Done Done false
32793 8805 Acked Write 6 Sent Done Done false
32794 8806 Acked Write 10 Sent Done Done false
32795 8807 Acked Read 8 Sent Done Done false
32796 8808 Acked Write 9 Sent Done Done false
32797 8809 Acked Read 1 Sent Done Done false
32798 8810 Acked Read 8 Sent Done Done false
32799 8811 Acked Write 1 Sent Done Done false
32800 8812 Acked Write 5 Sent Done Done false
32801 8813 Acked Write 1 Sent Done Done false
32802 8814 Acked Read 9 Sent Done Done false
32803 8815 Acked Read 6 Sent Done Done false
32804 8816 Acked Write 6 Sent Done Done false
32805 8817 Acked Write 6 Sent Done Done false
32806 8818 Acked Read 7 Sent Done Done false
32807 8819 Acked Read 5 Sent Done Done false
32808 8820 Acked Read 2 Sent Done Done false
32809 8821 Acked Flush 0 Sent Done Done false
32810 8822 Acked Write 6 Sent Done Done false
32811 8823 Acked Write 6 Sent Done Done false
32812 8824 Acked Read 3 Sent Done Done false
32813 8825 Acked Flush 0 Sent Done Done false
32814 8826 Acked Read 7 Sent Done Done false
32815 8827 Acked Flush 0 Sent Done Done false
32816 8828 Acked Write 6 Sent Done Done false
32817 8829 Acked Write 4 Sent Done Done false
32818 8830 Acked Write 9 Sent Done Done false
32819 8831 Acked Write 7 Sent Done Done false
32820 8832 Acked Write 4 Sent Done Done false
32821 8833 Acked Write 6 Sent Done Done false
32822 8834 Acked Write 6 Sent Done Done false
32823 8835 Acked Write 9 Sent Done Done false
32824 8836 Acked Write 2 Sent Done Done false
32825 8837 Acked Write 3 Sent Done Done false
32826 8838 Acked Read 10 Sent Done Done false
32827 8839 Acked Read 4 Sent Done Done false
32828 8840 Acked Flush 0 Sent Done Done false
32829 8841 Acked Write 2 Sent Done Done false
32830 8842 Acked Write 1 Sent Done Done false
32831 8843 Acked Write 6 Sent Done Done false
32832 8844 Acked Write 6 Sent Done Done false
32833 8845 Acked Read 5 Sent Done Done false
32834 8846 Acked Read 10 Sent Done Done false
32835 8847 Acked Read 6 Sent Done Done false
32836 8848 Acked Read 1 Sent Done Done false
32837 8849 Acked Flush 0 Sent Done Done false
32838 STATES DS:0 DS:1 DS:2 TOTAL
32839 Sent 302 0 0 302
32840 Done 0 302 302 604
32841 Skipped 0 0 0 0
32842 Error 0 0 0 0
32843 Last Flush: 8547 8849 8849
32844 Downstairs last five completed: 8547 8546 8545 8544 8543
32845 Upstairs last five completed: 8849 8848 8847 8846 8845
32846 CLIENT: Up:0 ds:302 act:2
32847 ----------------------------------------------------------------
32848 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
32849 States: Offline Active Active
32850 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
32851 8548 Acked Read 6 Sent Done Done false
32852 8549 Acked Read 9 Sent Done Done false
32853 8550 Acked Read 1 Sent Done Done false
32854 8551 Acked Read 9 Sent Done Done false
32855 8552 Acked Read 10 Sent Done Done false
32856 8553 Acked Read 1 Sent Done Done false
32857 8554 Acked Flush 0 Sent Done Done false
32858 8555 Acked Flush 0 Sent Done Done false
32859 8556 Acked Write 5 Sent Done Done false
32860 8557 Acked Write 1 Sent Done Done false
32861 8558 Acked Write 9 Sent Done Done false
32862 8559 Acked Write 6 Sent Done Done false
32863 8560 Acked Write 1 Sent Done Done false
32864 8561 Acked Read 2 Sent Done Done false
32865 8562 Acked Write 2 Sent Done Done false
32866 8563 Acked Read 4 Sent Done Done false
32867 8564 Acked Read 7 Sent Done Done false
32868 8565 Acked Read 4 Sent Done Done false
32869 8566 Acked Read 10 Sent Done Done false
32870 8567 Acked Write 3 Sent Done Done false
32871 8568 Acked Read 4 Sent Done Done false
32872 8569 Acked Read 3 Sent Done Done false
32873 8570 Acked Flush 0 Sent Done Done false
32874 8571 Acked Write 7 Sent Done Done false
32875 8572 Acked Write 1 Sent Done Done false
32876 8573 Acked Read 5 Sent Done Done false
32877 8574 Acked Write 2 Sent Done Done false
32878 8575 Acked Write 1 Sent Done Done false
32879 8576 Acked Write 2 Sent Done Done false
32880 8577 Acked Read 1 Sent Done Done false
32881 8578 Acked Flush 0 Sent Done Done false
32882 8579 Acked Write 3 Sent Done Done false
32883 8580 Acked Write 9 Sent Done Done false
32884 8581 Acked Write 1 Sent Done Done false
32885 8582 Acked Write 10 Sent Done Done false
32886 8583 Acked Write 5 Sent Done Done false
32887 8584 Acked Read 8 Sent Done Done false
32888 8585 Acked Read 7 Sent Done Done false
32889 8586 Acked Write 7 Sent Done Done false
32890 8587 Acked Flush 0 Sent Done Done false
32891 8588 Acked Read 10 Sent Done Done false
32892 8589 Acked Write 7 Sent Done Done false
32893 8590 Acked Read 4 Sent Done Done false
32894 8591 Acked Read 2 Sent Done Done false
32895 8592 Acked Flush 0 Sent Done Done false
32896 8593 Acked Write 7 Sent Done Done false
32897 8594 Acked Write 5 Sent Done Done false
32898 8595 Acked Read 10 Sent Done Done false
32899 8596 Acked Read 1 Sent Done Done false
32900 8597 Acked Read 2 Sent Done Done false
32901 8598 Acked Read 5 Sent Done Done false
32902 8599 Acked Read 4 Sent Done Done false
32903 8600 Acked Write 5 Sent Done Done false
32904 8601 Acked Write 5 Sent Done Done false
32905 8602 Acked Write 7 Sent Done Done false
32906 8603 Acked Read 3 Sent Done Done false
32907 8604 Acked Write 6 Sent Done Done false
32908 8605 Acked Read 7 Sent Done Done false
32909 8606 Acked Read 3 Sent Done Done false
32910 8607 Acked Read 5 Sent Done Done false
32911 8608 Acked Write 1 Sent Done Done false
32912 8609 Acked Read 3 Sent Done Done false
32913 8610 Acked Write 2 Sent Done Done false
32914 8611 Acked Read 7 Sent Done Done false
32915 8612 Acked Flush 0 Sent Done Done false
32916 8613 Acked Write 9 Sent Done Done false
32917 8614 Acked Read 3 Sent Done Done false
32918 8615 Acked Read 10 Sent Done Done false
32919 8616 Acked Read 4 Sent Done Done false
32920 8617 Acked Write 3 Sent Done Done false
32921 8618 Acked Write 9 Sent Done Done false
32922 8619 Acked Read 10 Sent Done Done false
32923 8620 Acked Read 3 Sent Done Done false
32924 8621 Acked Read 3 Sent Done Done false
32925 8622 Acked Read 9 Sent Done Done false
32926 8623 Acked Write 7 Sent Done Done false
32927 8624 Acked Read 7 Sent Done Done false
32928 8625 Acked Read 3 Sent Done Done false
32929 8626 Acked Write 7 Sent Done Done false
32930 8627 Acked Read 6 Sent Done Done false
32931 8628 Acked Flush 0 Sent Done Done false
32932 8629 Acked Flush 0 Sent Done Done false
32933 8630 Acked Read 8 Sent Done Done false
32934 8631 Acked Read 6 Sent Done Done false
32935 8632 Acked Write 10 Sent Done Done false
32936 8633 Acked Read 2 Sent Done Done false
32937 8634 Acked Flush 0 Sent Done Done false
32938 8635 Acked Write 5 Sent Done Done false
32939 8636 Acked Write 3 Sent Done Done false
32940 8637 Acked Write 10 Sent Done Done false
32941 8638 Acked Read 6 Sent Done Done false
32942 8639 Acked Read 9 Sent Done Done false
32943 8640 Acked Read 5 Sent Done Done false
32944 8641 Acked Write 3 Sent Done Done false
32945 8642 Acked Write 3 Sent Done Done false
32946 8643 Acked Write 3 Sent Done Done false
32947 8644 Acked Write 8 Sent Done Done false
32948 8645 Acked Write 4 Sent Done Done false
32949 8646 Acked Write 1 Sent Done Done false
32950 8647 Acked Read 8 Sent Done Done false
32951 8648 Acked Write 8 Sent Done Done false
32952 8649 Acked Read 4 Sent Done Done false
32953 8650 Acked Read 2 Sent Done Done false
32954 8651 Acked Read 9 Sent Done Done false
32955 8652 Acked Read 5 Sent Done Done false
32956 8653 Acked Write 3 Sent Done Done false
32957 8654 Acked Read 3 Sent Done Done false
32958 8655 Acked Read 10 Sent Done Done false
32959 8656 Acked Write 8 Sent Done Done false
32960 8657 Acked Read 5 Sent Done Done false
32961 8658 Acked Flush 0 Sent Done Done false
32962 8659 Acked Flush 0 Sent Done Done false
32963 8660 Acked Write 8 Sent Done Done false
32964 8661 Acked Write 7 Sent Done Done false
32965 8662 Acked Read 6 Sent Done Done false
32966 8663 Acked Write 5 Sent Done Done false
32967 8664 Acked Write 5 Sent Done Done false
32968 8665 Acked Write 6 Sent Done Done false
32969 8666 Acked Read 7 Sent Done Done false
32970 8667 Acked Flush 0 Sent Done Done false
32971 8668 Acked Write 2 Sent Done Done false
32972 8669 Acked Write 9 Sent Done Done false
32973 8670 Acked Read 7 Sent Done Done false
32974 8671 Acked Read 5 Sent Done Done false
32975 8672 Acked Read 3 Sent Done Done false
32976 8673 Acked Read 9 Sent Done Done false
32977 8674 Acked Write 3 Sent Done Done false
32978 8675 Acked Write 3 Sent Done Done false
32979 8676 Acked Read 9 Sent Done Done false
32980 8677 Acked Write 3 Sent Done Done false
32981 8678 Acked Write 5 Sent Done Done false
32982 8679 Acked Read 4 Sent Done Done false
32983 8680 Acked Write 4 Sent Done Done false
32984 8681 Acked Read 6 Sent Done Done false
32985 8682 Acked Flush 0 Sent Done Done false
32986 8683 Acked Write 7 Sent Done Done false
32987 8684 Acked Read 6 Sent Done Done false
32988 8685 Acked Write 10 Sent Done Done false
32989 8686 Acked Read 6 Sent Done Done false
32990 8687 Acked Read 10 Sent Done Done false
32991 8688 Acked Read 8 Sent Done Done false
32992 8689 Acked Write 9 Sent Done Done false
32993 8690 Acked Read 4 Sent Done Done false
32994 8691 Acked Write 7 Sent Done Done false
32995 8692 Acked Flush 0 Sent Done Done false
32996 8693 Acked Write 2 Sent Done Done false
32997 8694 Acked Read 10 Sent Done Done false
32998 8695 Acked Read 10 Sent Done Done false
32999 8696 Acked Read 4 Sent Done Done false
33000 8697 Acked Read 5 Sent Done Done false
33001 8698 Acked Read 3 Sent Done Done false
33002 8699 Acked Flush 0 Sent Done Done false
33003 8700 Acked Read 2 Sent Done Done false
33004 8701 Acked Write 6 Sent Done Done false
33005 8702 Acked Write 1 Sent Done Done false
33006 8703 Acked Write 8 Sent Done Done false
33007 8704 Acked Read 7 Sent Done Done false
33008 8705 Acked Flush 0 Sent Done Done false
33009 8706 Acked Read 8 Sent Done Done false
33010 8707 Acked Write 10 Sent Done Done false
33011 8708 Acked Write 10 Sent Done Done false
33012 8709 Acked Read 1 Sent Done Done false
33013 8710 Acked Read 3 Sent Done Done false
33014 8711 Acked Read 4 Sent Done Done false
33015 8712 Acked Write 3 Sent Done Done false
33016 8713 Acked Write 1 Sent Done Done false
33017 8714 Acked Write 10 Sent Done Done false
33018 8715 Acked Read 3 Sent Done Done false
33019 8716 Acked Write 2 Sent Done Done false
33020 8717 Acked Write 2 Sent Done Done false
33021 8718 Acked Read 3 Sent Done Done false
33022 8719 Acked Read 8 Sent Done Done false
33023 8720 Acked Write 6 Sent Done Done false
33024 8721 Acked Write 5 Sent Done Done false
33025 8722 Acked Read 5 Sent Done Done false
33026 8723 Acked Write 7 Sent Done Done false
33027 8724 Acked Read 8 Sent Done Done false
33028 8725 Acked Write 4 Sent Done Done false
33029 8726 Acked Write 1 Sent Done Done false
33030 8727 Acked Read 10 Sent Done Done false
33031 8728 Acked Write 8 Sent Done Done false
33032 8729 Acked Read 7 Sent Done Done false
33033 8730 Acked Flush 0 Sent Done Done false
33034 8731 Acked Write 2 Sent Done Done false
33035 8732 Acked Read 2 Sent Done Done false
33036 8733 Acked Write 3 Sent Done Done false
33037 8734 Acked Read 6 Sent Done Done false
33038 8735 Acked Read 2 Sent Done Done false
33039 8736 Acked Write 1 Sent Done Done false
33040 8737 Acked Read 2 Sent Done Done false
33041 8738 Acked Flush 0 Sent Done Done false
33042 8739 Acked Read 7 Sent Done Done false
33043 8740 Acked Read 5 Sent Done Done false
33044 8741 Acked Write 10 Sent Done Done false
33045 8742 Acked Read 7 Sent Done Done false
33046 8743 Acked Write 3 Sent Done Done false
33047 8744 Acked Read 5 Sent Done Done false
33048 8745 Acked Write 9 Sent Done Done false
33049 8746 Acked Write 6 Sent Done Done false
33050 8747 Acked Read 5 Sent Done Done false
33051 8748 Acked Write 9 Sent Done Done false
33052 8749 Acked Write 5 Sent Done Done false
33053 8750 Acked Read 1 Sent Done Done false
33054 8751 Acked Read 3 Sent Done Done false
33055 8752 Acked Read 5 Sent Done Done false
33056 8753 Acked Read 1 Sent Done Done false
33057 8754 Acked Read 9 Sent Done Done false
33058 8755 Acked Read 1 Sent Done Done false
33059 8756 Acked Write 9 Sent Done Done false
33060 8757 Acked Read 8 Sent Done Done false
33061 8758 Acked Write 8 Sent Done Done false
33062 8759 Acked Read 8 Sent Done Done false
33063 8760 Acked Write 10 Sent Done Done false
33064 8761 Acked Flush 0 Sent Done Done false
33065 8762 Acked Write 7 Sent Done Done false
33066 8763 Acked Read 6 Sent Done Done false
33067 8764 Acked Read 6 Sent Done Done false
33068 8765 Acked Read 1 Sent Done Done false
33069 8766 Acked Read 9 Sent Done Done false
33070 8767 Acked Write 5 Sent Done Done false
33071 8768 Acked Write 5 Sent Done Done false
33072 8769 Acked Read 7 Sent Done Done false
33073 8770 Acked Read 7 Sent Done Done false
33074 8771 Acked Read 3 Sent Done Done false
33075 8772 Acked Flush 0 Sent Done Done false
33076 8773 Acked Read 6 Sent Done Done false
33077 8774 Acked Read 2 Sent Done Done false
33078 8775 Acked Flush 0 Sent Done Done false
33079 8776 Acked Write 2 Sent Done Done false
33080 8777 Acked Read 9 Sent Done Done false
33081 8778 Acked Write 8 Sent Done Done false
33082 8779 Acked Write 4 Sent Done Done false
33083 8780 Acked Write 3 Sent Done Done false
33084 8781 Acked Read 3 Sent Done Done false
33085 8782 Acked Write 3 Sent Done Done false
33086 8783 Acked Write 3 Sent Done Done false
33087 8784 Acked Write 1 Sent Done Done false
33088 8785 Acked Write 10 Sent Done Done false
33089 8786 Acked Write 6 Sent Done Done false
33090 8787 Acked Write 9 Sent Done Done false
33091 8788 Acked Write 8 Sent Done Done false
33092 8789 Acked Read 4 Sent Done Done false
33093 8790 Acked Write 8 Sent Done Done false
33094 8791 Acked Write 1 Sent Done Done false
33095 8792 Acked Read 1 Sent Done Done false
33096 8793 Acked Read 5 Sent Done Done false
33097 8794 Acked Read 2 Sent Done Done false
33098 8795 Acked Write 9 Sent Done Done false
33099 8796 Acked Write 10 Sent Done Done false
33100 8797 Acked Read 4 Sent Done Done false
33101 8798 Acked Read 1 Sent Done Done false
33102 8799 Acked Write 6 Sent Done Done false
33103 8800 Acked Write 9 Sent Done Done false
33104 8801 Acked Read 3 Sent Done Done false
33105 8802 Acked Write 8 Sent Done Done false
33106 8803 Acked Read 4 Sent Done Done false
33107 8804 Acked Read 4 Sent Done Done false
33108 8805 Acked Write 6 Sent Done Done false
33109 8806 Acked Write 10 Sent Done Done false
33110 8807 Acked Read 8 Sent Done Done false
33111 8808 Acked Write 9 Sent Done Done false
33112 8809 Acked Read 1 Sent Done Done false
33113 8810 Acked Read 8 Sent Done Done false
33114 8811 Acked Write 1 Sent Done Done false
33115 8812 Acked Write 5 Sent Done Done false
33116 8813 Acked Write 1 Sent Done Done false
33117 8814 Acked Read 9 Sent Done Done false
33118 8815 Acked Read 6 Sent Done Done false
33119 8816 Acked Write 6 Sent Done Done false
33120 8817 Acked Write 6 Sent Done Done false
33121 8818 Acked Read 7 Sent Done Done false
33122 8819 Acked Read 5 Sent Done Done false
33123 8820 Acked Read 2 Sent Done Done false
33124 8821 Acked Flush 0 Sent Done Done false
33125 8822 Acked Write 6 Sent Done Done false
33126 8823 Acked Write 6 Sent Done Done false
33127 8824 Acked Read 3 Sent Done Done false
33128 8825 Acked Flush 0 Sent Done Done false
33129 8826 Acked Read 7 Sent Done Done false
33130 8827 Acked Flush 0 Sent Done Done false
33131 8828 Acked Write 6 Sent Done Done false
33132 8829 Acked Write 4 Sent Done Done false
33133 8830 Acked Write 9 Sent Done Done false
33134 8831 Acked Write 7 Sent Done Done false
33135 8832 Acked Write 4 Sent Done Done false
33136 8833 Acked Write 6 Sent Done Done false
33137 8834 Acked Write 6 Sent Done Done false
33138 8835 Acked Write 9 Sent Done Done false
33139 8836 Acked Write 2 Sent Done Done false
33140 8837 Acked Write 3 Sent Done Done false
33141 8838 Acked Read 10 Sent Done Done false
33142 8839 Acked Read 4 Sent Done Done false
33143 8840 Acked Flush 0 Sent Done Done false
33144 8841 Acked Write 2 Sent Done Done false
33145 8842 Acked Write 1 Sent Done Done false
33146 8843 Acked Write 6 Sent Done Done false
33147 8844 Acked Write 6 Sent Done Done false
33148 8845 Acked Read 5 Sent Done Done false
33149 8846 Acked Read 10 Sent Done Done false
33150 8847 Acked Read 6 Sent Done Done false
33151 8848 Acked Read 1 Sent Done Done false
33152 8849 Acked Flush 0 Sent Done Done false
33153 STATES DS:0 DS:1 DS:2 TOTAL
33154 Sent 302 0 0 302
33155 Done 0 302 302 604
33156 Skipped 0 0 0 0
33157 Error 0 0 0 0
33158 Last Flush: 8547 8849 8849
33159 Downstairs last five completed: 8547 8546 8545 8544 8543
33160 Upstairs last five completed: 8849 8848 8847 8846 8845
33161 CLIENT: Up:0 ds:302 act:2
331622024-11-15T02:04:26.058ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
331632024-11-15T02:04:26.058ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:59003) = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
331642024-11-15T02:04:26.059ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
331652024-11-15T02:04:26.060ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
331662024-11-15T02:04:26.060ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
331672024-11-15T02:04:26.060ZINFOcrucible: send last flush ID to this DS: 8547 = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
331682024-11-15T02:04:26.060ZINFOcrucible: Replied this last flush ID: 8547 = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
331692024-11-15T02:04:26.060ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
331702024-11-15T02:04:26.060ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
331712024-11-15T02:04:26.060ZINFOcrucible: [0] client re-new 302 jobs since flush 8547 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
331722024-11-15T02:04:26.060ZINFOcrucible: [0] Replayed 302 jobs since flush: 8547 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
33173 ----------------------------------------------------------------
33174 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
33175 Downstairs last five completed: 8849 8848 8847 8846 8845
33176 Upstairs last five completed: 8849 8848 8847 8846 8845
33177 CLIENT: Up:0 ds:0 act:3
33178 Replay: All jobs finished
33179 Main task has work to do, go find it
33180 got dsc Stop(2)
33181 stop 2
33182 [8830] Got stop action so:false kr:false
33183 Nov 15 02:04:26.300 INFO request completed, latency_us: 330, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 4361e3c1-1739-411b-bf66-56e64150a1ac, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
331842024-11-15T02:04:26.301ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
331852024-11-15T02:04:26.301ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
331862024-11-15T02:04:26.301ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
331872024-11-15T02:04:26.301ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
331882024-11-15T02:04:26.301ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
33189 Nov 15 02:04:26.301{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2024-11-15T02:04:26.301543058Z","hostname":"w-01JCPRHT5F2MPA44KVJHM3S2PB","pid":1205,"":"io task","client":"2","":"downstairs","session_id":"971dcb2e-6891-4ecf-b367-1e04e52065ee"}
33190 INFO request completed, latency_us: 158, response_code: 200, uri: /state/cid/2, method: GET, req_id: bdb508c8-add6-46d5-9134-4ffedacce8bc, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
33191 [8830] Exited with: ExitStatus(unix_wait_status(9))
33192 [8830][2] reports Exit
33193 Nov 15 02:04:30.303 INFO request completed, latency_us: 235, response_code: 200, uri: /state/cid/2, method: GET, req_id: d56d9466-282d-46bb-b556-0dbf878bea49, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
33194 001/300 Read block 2253 len 28672
33195 002/300 Read block 1946 len 32768
33196 003/300 Read block 3242 len 12288
33197 004/300 Write block 158 len 40960 data: 4 6 7 8 8 8 8 7 5 5
33198 005/300 Write block 1581 len 40960 data: 6 5 6 6 6 6 6 7 6 5
33199 006/300 Flush
33200 007/300 Read block 4774 len 12288
33201 008/300 Flush
33202 009/300 Read block 2684 len 4096
33203 010/300 Write block 3003 len 8192 data: 4 5
33204 011/300 Read block 2200 len 28672
33205 012/300 Write block 4578 len 28672 data: 4 3 4 5 4 4 5
33206 013/300 Flush
33207 014/300 Write block 2898 len 24576 data: 5 5 4 3 4 5
33208 015/300 Read block 3034 len 28672
33209 016/300 Write block 1635 len 8192 data: 7 7
33210 017/300 Write block 2402 len 8192 data: 8 10
33211 018/300 Read block 4525 len 28672
33212 019/300 Read block 2401 len 12288
33213 020/300 Write block 4447 len 20480 data: 6 6 4 4 4
33214 021/300 Read block 2058 len 36864
33215 022/300 Read block 1058 len 36864
33216 023/300 Read block 3462 len 8192
33217 024/300 Write block 4351 len 8192 data: 5 5
33218 025/300 Write block 4822 len 28672 data: 4 4 4 5 6 6 5
33219 026/300 Read block 2287 len 12288
33220 027/300 Read block 295 len 20480
33221 028/300 Read block 254 len 24576
33222 029/300 Write block 4849 len 16384 data: 3 3 3 4
33223 030/300 Read block 1070 len 24576
33224 031/300 Read block 1295 len 20480
33225 032/300 Write block 2610 len 32768 data: 4 5 6 6 6 6 6 6
33226 033/300 Read block 2843 len 40960
33227 034/300 Flush
33228 035/300 Read block 4071 len 40960
33229 036/300 Write block 4094 len 12288 data: 7 7 7
33230 037/300 Read block 4223 len 8192
33231 038/300 Read block 935 len 4096
33232 039/300 Write block 2142 len 4096 data: 5
33233 040/300 Read block 1723 len 40960
33234 041/300 Read block 1132 len 24576
33235 042/300 Read block 4660 len 32768
33236 043/300 Write block 4114 len 36864 data: 4 2 3 2 2 4 5 5 8
33237 044/300 Write block 2545 len 24576 data: 6 7 6 6 6 6
33238 045/300 Read block 1113 len 12288
33239 046/300 Write block 260 len 20480 data: 6 6 5 5 5
33240 047/300 Read block 1805 len 4096
33241 048/300 Read block 3761 len 16384
33242 049/300 Write block 2334 len 12288 data: 4 5 8
33243 050/300 Write block 4160 len 4096 data: 6
33244 051/300 Read block 4249 len 40960
33245 052/300 Write block 2595 len 4096 data: 7
33246 053/300 Write block 4904 len 40960 data: 8 6 5 2 2 3 3 3 3 4
33247 054/300 Read block 211 len 16384
33248 055/300 Read block 2720 len 12288
33249 056/300 Read block 3977 len 12288
33250 057/300 Read block 1964 len 28672
33251 058/300 Read block 3876 len 24576
33252 059/300 Read block 2489 len 40960
33253 060/300 Read block 129 len 32768
33254 061/300 Read block 384 len 28672
33255 062/300 Write block 2686 len 4096 data: 7
33256 063/300 Read block 116 len 12288
33257 064/300 Write block 3746 len 32768 data: 6 6 5 5 6 7 6 5
33258 065/300 Read block 19 len 8192
33259 066/300 Read block 1460 len 8192
33260 067/300 Read block 4443 len 20480
33261 068/300 Read block 4153 len 32768
33262 069/300 Write block 185 len 36864 data: 5 4 4 4 4 4 5 5 4
33263 070/300 Read block 1413 len 32768
33264 071/300 Flush
33265 072/300 Write block 4180 len 20480 data: 11 10 8 8 10
33266 073/300 Read block 2044 len 16384
33267 074/300 Write block 2174 len 4096 data: 6
33268 075/300 Read block 3038 len 32768
33269 076/300 Write block 1282 len 36864 data: 8 8 6 6 7 7 9 7 8
33270 077/300 Write block 1227 len 12288 data: 5 5 6
33271 078/300 Write block 3748 len 28672 data: 6 6 7 8 7 6 6
33272 079/300 Write block 1737 len 20480 data: 5 5 5 3 4
33273 080/300 Read block 2687 len 40960
33274 081/300 Read block 1181 len 20480
33275 082/300 Write block 513 len 12288 data: 6 5 4
33276 083/300 Read block 4944 len 12288
33277 084/300 Write block 4962 len 16384 data: 4 4 4 6
33278 085/300 Write block 3247 len 36864 data: 8 8 9 9 10 10 11 10 9
33279 086/300 Write block 987 len 36864 data: 6 6 6 6 11 11 10 10 9
33280 087/300 Read block 4383 len 28672
33281 088/300 Write block 4131 len 20480 data: 8 5 5 5 4
33282 089/300 Read block 4346 len 32768
33283 090/300 Read block 3327 len 28672
33284 091/300 Write block 2113 len 36864 data: 4 3 4 4 6 5 4 3 3
33285 092/300 Write block 4987 len 8192 data: 3 3
33286 093/300 Write block 448 len 20480 data: 5 4 4 6 5
33287 094/300 Read block 4244 len 8192
33288 095/300 Write block 1042 len 12288 data: 3 3 3
33289 096/300 Read block 965 len 8192
33290 097/300 Read block 2634 len 8192
33291 098/300 Write block 4270 len 24576 data: 6 6 7 7 6 5
33292 099/300 Write block 1626 len 24576 data: 5 5 6 5 6 8
33293 100/300 Read block 1561 len 40960
33294 101/300 Write block 4919 len 24576 data: 6 8 9 10 10 8
33295 102/300 Read block 3277 len 4096
33296 103/300 Read block 4017 len 36864
33297 104/300 Read block 2701 len 32768
33298 105/300 Read block 4739 len 4096
33299 106/300 Read block 2360 len 4096
33300 107/300 Write block 3951 len 4096 data: 4
33301 108/300 Flush
33302 109/300 Flush
33303 110/300 Write block 2792 len 32768 data: 3 3 3 4 4 4 4 3
33304 111/300 Flush
33305 112/300 Write block 3138 len 16384 data: 4 4 6 6
33306 113/300 Read block 3707 len 4096
33307 114/300 Read block 4237 len 12288
33308 115/300 Read block 3576 len 40960
33309 116/300 Flush
33310 117/300 Read block 2955 len 40960
33311 118/300 Read block 2713 len 8192
33312 119/300 Write block 3278 len 28672 data: 6 6 6 5 5 5 6
33313 120/300 Read block 4109 len 8192
33314 121/300 Write block 3459 len 12288 data: 4 4 4
33315 122/300 Write block 2399 len 4096 data: 4
33316 123/300 Write block 2780 len 16384 data: 6 6 6 6
33317 124/300 Write block 4091 len 8192 data: 5 6
33318 125/300 Read block 1121 len 16384
33319 126/300 Write block 1488 len 8192 data: 4 4
33320 127/300 Write block 3705 len 40960 data: 4 4 6 7 8 7 7 7 6 5
33321 128/300 Read block 3713 len 20480
33322 129/300 Read block 66 len 12288
33323 130/300 Read block 4669 len 12288
33324 131/300 Read block 1207 len 24576
33325 132/300 Read block 3654 len 4096
33326 133/300 Write block 3154 len 40960 data: 5 4 5 5 7 6 7 6 6 6
33327 134/300 Read block 3159 len 4096
33328 135/300 Write block 4997 len 12288 data: 3 4 3
33329 136/300 Read block 1046 len 4096
33330 137/300 Read block 4499 len 36864
33331 138/300 Read block 4688 len 36864
33332 139/300 Write block 1741 len 24576 data: 5 4 5 6 6 6
33333 140/300 Write block 4595 len 24576 data: 5 5 4 3 5 6
33334 141/300 Read block 4976 len 32768
33335 142/300 Write block 712 len 20480 data: 6 4 4 4 4
33336 143/300 Read block 773 len 4096
33337 144/300 Read block 973 len 4096
33338 145/300 Read block 2380 len 36864
33339 146/300 Read block 195 len 28672
33340 147/300 Read block 4462 len 4096
33341 148/300 Read block 4044 len 20480
33342 149/300 Read block 4314 len 20480
33343 150/300 Read block 4926 len 20480
33344 151/300 Write block 2150 len 28672 data: 7 6 7 6 6 7 6
33345 152/300 Read block 2838 len 20480
33346 153/300 Read block 3298 len 20480
33347 154/300 Write block 4002 len 12288 data: 5 6 8
33348 155/300 Write block 2477 len 16384 data: 12 9 7 4
33349 156/300 Write block 2647 len 20480 data: 5 5 4 5 5
33350 157/300 Write block 3677 len 36864 data: 7 7 7 7 5 6 6 5 5
33351 158/300 Flush
33352 159/300 Read block 427 len 28672
33353 160/300 Write block 4462 len 40960 data: 5 4 5 5 4 5 5 4 5 3
33354 161/300 Read block 3040 len 28672
33355 162/300 Read block 1289 len 8192
33356 163/300 Flush
33357 164/300 Write block 3780 len 4096 data: 5
33358 165/300 Read block 2868 len 36864
33359 166/300 Read block 373 len 4096
33360 167/300 Write block 4082 len 32768 data: 4 4 4 4 3 3 3 3
33361 168/300 Read block 1661 len 24576
33362 169/300 Write block 1379 len 20480 data: 7 7 7 6 4
33363 170/300 Flush
33364 171/300 Read block 456 len 4096
33365 172/300 Write block 3363 len 8192 data: 4 5
33366 173/300 Write block 287 len 36864 data: 6 6 7 8 8 7 8 9 6
33367 174/300 Read block 386 len 20480
33368 175/300 Read block 578 len 12288
33369 176/300 Read block 2180 len 24576
33370 177/300 Read block 699 len 32768
33371 178/300 Write block 1014 len 8192 data: 7 9
33372 179/300 Read block 3802 len 28672
33373 180/300 Read block 3773 len 16384
33374 181/300 Write block 2541 len 8192 data: 4 6
33375 182/300 Read block 1930 len 12288
33376 183/300 Read block 333 len 40960
33377 184/300 Write block 180 len 20480 data: 5 5 5 3 4
33378 185/300 Flush
33379 186/300 Write block 1937 len 24576 data: 6 8 7 7 7 5
33380 187/300 Read block 2159 len 4096
33381 188/300 Read block 3332 len 8192
33382 189/300 Read block 78 len 16384
33383 190/300 Read block 938 len 36864
33384 191/300 Flush
33385 192/300 Write block 273 len 28672 data: 4 4 4 4 4 4 4
33386 193/300 Write block 3823 len 32768 data: 3 4 4 4 3 3 3 3
33387 194/300 Flush
33388 195/300 Read block 3522 len 24576
33389 196/300 Flush
33390 197/300 Write block 536 len 24576 data: 6 6 7 7 6 6
33391 198/300 Read block 4293 len 20480
33392 199/300 Read block 2997 len 20480
33393 200/300 Write block 4502 len 32768 data: 3 2 2 2 3 3 3 3
33394 201/300 Read block 2822 len 40960
33395 202/300 Read block 3788 len 24576
33396 203/300 Read block 4295 len 36864
33397 204/300 Read block 2973 len 40960
33398 205/300 Read block 3589 len 28672
33399 206/300 Write block 1234 len 24576 data: 4 5 4 3 3 3
33400 207/300 Read block 1556 len 40960
33401 208/300 Write block 367 len 20480 data: 6 3 3 3 3
33402 209/300 Write block 2004 len 28672 data: 5 4 5 4 3 3 3
33403 210/300 Read block 4118 len 4096
33404 211/300 Read block 2243 len 32768
33405 212/300 Write block 147 len 40960 data: 2 2 3 3 3 3 4 3 3 4
33406 213/300 Write block 435 len 8192 data: 2 3
33407 214/300 Read block 3420 len 36864
33408 215/300 Write block 2010 len 32768 data: 4 4 4 3 3 4 4 4
33409 216/300 Write block 1488 len 20480 data: 5 5 4 6 5
33410 217/300 Read block 4279 len 24576
33411 218/300 Read block 4001 len 20480
33412 219/300 Read block 4593 len 32768
33413 220/300 Read block 3485 len 24576
33414 221/300 Write block 3284 len 32768 data: 7 5 5 3 3 4 4 3
33415 222/300 Read block 4839 len 40960
33416 223/300 Read block 1449 len 24576
33417 224/300 Read block 1847 len 8192
33418 225/300 Read block 1986 len 16384
33419 226/300 Read block 2803 len 40960
33420 227/300 Read block 4733 len 36864
33421 228/300 Read block 2282 len 12288
33422 229/300 Write block 2523 len 36864 data: 4 4 8 7 10 12 12 12 13
33423 230/300 Flush
33424 231/300 Write block 3986 len 8192 data: 3 5
33425 232/300 Write block 4040 len 4096 data: 11
33426 233/300 Write block 4450 len 40960 data: 5 5 4 4 4 4 6 6 6 6
33427 234/300 Read block 662 len 24576
33428 235/300 Write block 338 len 4096 data: 4
33429 236/300 Read block 489 len 12288
33430 237/300 Write block 576 len 8192 data: 6 7
33431 238/300 Write block 933 len 16384 data: 5 4 4 4
33432 239/300 Write block 1672 len 20480 data: 4 5 4 5 5
33433 240/300 Write block 3278 len 8192 data: 7 7
33434 241/300 Read block 1202 len 12288
33435 242/300 Write block 2060 len 32768 data: 8 9 10 9 7 4 3 3
33436 243/300 Read block 2967 len 20480
33437 244/300 Read block 347 len 16384
33438 245/300 Read block 2642 len 32768
33439 246/300 Read block 2430 len 24576
33440 247/300 Read block 559 len 20480
33441 248/300 Flush
33442 249/300 Write block 1545 len 28672 data: 5 5 6 6 5 8 7
33443 250/300 Write block 2648 len 24576 data: 6 5 6 6 3 3
33444 251/300 Read block 2641 len 40960
33445 252/300 Write block 1175 len 36864 data: 8 7 5 4 2 3 3 4 4
33446 253/300 Read block 3340 len 36864
33447 254/300 Read block 2175 len 28672
33448 255/300 Read block 2320 len 16384
33449 256/300 Read block 3396 len 40960
33450 257/300 Write block 4429 len 16384 data: 5 5 5 5
33451 258/300 Flush
33452 259/300 Read block 2737 len 40960
33453 260/300 Write block 1023 len 16384 data: 7 8 5 5
33454 261/300 Flush
33455 262/300 Write block 3943 len 36864 data: 5 4 3 3 3 5 5 5 5
33456 263/300 Read block 2006 len 40960
33457 264/300 Write block 2926 len 28672 data: 6 5 3 3 4 6 6
33458 265/300 Read block 1648 len 12288
33459 266/300 Read block 3750 len 32768
33460 267/300 Read block 2297 len 28672
33461 268/300 Read block 3866 len 32768
33462 269/300 Read block 3320 len 28672
33463 270/300 Write block 3128 len 40960 data: 4 4 5 5 4 4 5 5 5 3
33464 271/300 Read block 4543 len 8192
33465 272/300 Flush
33466 273/300 Read block 2244 len 36864
33467 274/300 Write block 2907 len 32768 data: 4 4 5 4 3 3 3 3
33468 275/300 Read block 3116 len 36864
33469 276/300 Read block 3056 len 28672
33470 277/300 Write block 2642 len 8192 data: 3 4
33471 278/300 Read block 4688 len 24576
33472 279/300 Write block 2850 len 40960 data: 4 4 5 5 5 4 6 4 4 6
33473 280/300 Write block 3051 len 36864 data: 6 4 4 4 4 5 6 5 5
33474 281/300 Write block 3806 len 4096 data: 8
33475 282/300 Flush
33476 283/300 Flush
33477 284/300 Write block 1746 len 32768 data: 7 8 6 4 5 4 3 3
33478 285/300 Read block 4539 len 40960
33479 286/300 Read block 3762 len 16384
33480 287/300 Write block 2386 len 40960 data: 8 8 6 8 7 6 6 6 6 4
33481 288/300 Read block 3827 len 4096
33482 289/300 Read block 953 len 32768
33483 290/300 Write block 984 len 24576 data: 4 5 6 7 7 7
33484 291/300 Read block 4253 len 4096
33485 292/300 Flush
33486 293/300 Write block 116 len 40960 data: 5 5 4 4 5 6 5 5 5 4
33487 294/300 Read block 2792 len 12288
33488 295/300 Write block 3416 len 20480 data: 7 5 4 4 3
33489 296/300 Read block 558 len 36864
33490 297/300 Read block 465 len 16384
33491 298/300 Read block 2564 len 4096
33492 299/300 Flush
33493 300/300 Read block 2517 len 32768
33494 301/300 Read block 1014 len 32768
33495 Main task has work to do, go find it
33496 got dsc Start(2)
33497 start 2
33498 Nov 15 02:04:30.552 INFO request completed, latency_us: 220, response_code: 204, uri: /start/cid/2, method: POST, req_id: dfa35a5e-db3a-4f37-a8a6-a79fdfbae0ff, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
33499 Replay: started 2, returned:Ok(())
33500 ----------------------------------------------------------------
33501 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
33502 States: Active Active Offline
33503 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
33504 8850 Acked Read 7 Done Done Sent false
33505 8851 Acked Read 8 Done Done Sent false
33506 8852 Acked Read 3 Done Done Sent false
33507 8853 Acked Write 10 Done Done Sent false
33508 8854 Acked Write 10 Done Done Sent false
33509 8855 Acked Flush 0 Done Done Sent false
33510 8856 Acked Read 3 Done Done Sent false
33511 8857 Acked Flush 0 Done Done Sent false
33512 8858 Acked Read 1 Done Done Sent false
33513 8859 Acked Write 2 Done Done Sent false
33514 8860 Acked Read 7 Done Done Sent false
33515 8861 Acked Write 7 Done Done Sent false
33516 8862 Acked Flush 0 Done Done Sent false
33517 8863 Acked Write 6 Done Done Sent false
33518 8864 Acked Read 7 Done Done Sent false
33519 8865 Acked Write 2 Done Done Sent false
33520 8866 Acked Write 2 Done Done Sent false
33521 8867 Acked Read 7 Done Done Sent false
33522 8868 Acked Read 3 Done Done Sent false
33523 8869 Acked Write 5 Done Done Sent false
33524 8870 Acked Read 9 Done Done Sent false
33525 8871 Acked Read 9 Done Done Sent false
33526 8872 Acked Read 2 Done Done Sent false
33527 8873 Acked Write 2 Done Done Sent false
33528 8874 Acked Write 7 Done Done Sent false
33529 8875 Acked Read 3 Done Done Sent false
33530 8876 Acked Read 5 Done Done Sent false
33531 8877 Acked Read 6 Done Done Sent false
33532 8878 Acked Write 4 Done Done Sent false
33533 8879 Acked Read 6 Done Done Sent false
33534 8880 Acked Read 5 Done Done Sent false
33535 8881 Acked Write 8 Done Done Sent false
33536 8882 Acked Read 10 Done Done Sent false
33537 8883 Acked Flush 0 Done Done Sent false
33538 8884 Acked Read 10 Done Done Sent false
33539 8885 Acked Write 3 Done Done Sent false
33540 8886 Acked Read 2 Done Done Sent false
33541 8887 Acked Read 1 Done Done Sent false
33542 8888 Acked Write 1 Done Done Sent false
33543 8889 Acked Read 10 Done Done Sent false
33544 8890 Acked Read 6 Done Done Sent false
33545 8891 Acked Read 8 Done Done Sent false
33546 8892 Acked Write 9 Done Done Sent false
33547 8893 Acked Write 6 Done Done Sent false
33548 8894 Acked Read 3 Done Done Sent false
33549 8895 Acked Write 5 Done Done Sent false
33550 8896 Acked Read 1 Done Done Sent false
33551 8897 Acked Read 4 Done Done Sent false
33552 8898 Acked Write 3 Done Done Sent false
33553 8899 Acked Write 1 Done Done Sent false
33554 8900 Acked Read 10 Done Done Sent false
33555 8901 Acked Write 1 Done Done Sent false
33556 8902 Acked Write 10 Done Done Sent false
33557 8903 Acked Read 4 Done Done Sent false
33558 8904 Acked Read 3 Done Done Sent false
33559 8905 Acked Read 3 Done Done Sent false
33560 8906 Acked Read 7 Done Done Sent false
33561 8907 Acked Read 6 Done Done Sent false
33562 8908 Acked Read 10 Done Done Sent false
33563 8909 Acked Read 8 Done Done Sent false
33564 8910 Acked Read 7 Done Done Sent false
33565 8911 Acked Write 1 Done Done Sent false
33566 8912 Acked Read 3 Done Done Sent false
33567 8913 Acked Write 8 Done Done Sent false
33568 8914 Acked Read 2 Done Done Sent false
33569 8915 Acked Read 2 Done Done Sent false
33570 8916 Acked Read 5 Done Done Sent false
33571 8917 Acked Read 8 Done Done Sent false
33572 8918 Acked Write 9 Done Done Sent false
33573 8919 Acked Read 8 Done Done Sent false
33574 8920 Acked Flush 0 Done Done Sent false
33575 8921 Acked Write 5 Done Done Sent false
33576 8922 Acked Read 4 Done Done Sent false
33577 8923 Acked Write 1 Done Done Sent false
33578 8924 Acked Read 8 Done Done Sent false
33579 8925 Acked Write 9 Done Done Sent false
33580 8926 Acked Write 3 Done Done Sent false
33581 8927 Acked Write 7 Done Done Sent false
33582 8928 Acked Write 5 Done Done Sent false
33583 8929 Acked Read 10 Done Done Sent false
33584 8930 Acked Read 5 Done Done Sent false
33585 8931 Acked Write 3 Done Done Sent false
33586 8932 Acked Read 3 Done Done Sent false
33587 8933 Acked Write 4 Done Done Sent false
33588 8934 Acked Write 9 Done Done Sent false
33589 8935 Acked Write 9 Done Done Sent false
33590 8936 Acked Read 7 Done Done Sent false
33591 8937 Acked Write 5 Done Done Sent false
33592 8938 Acked Read 8 Done Done Sent false
33593 8939 Acked Read 7 Done Done Sent false
33594 8940 Acked Write 9 Done Done Sent false
33595 8941 Acked Write 2 Done Done Sent false
33596 8942 Acked Write 5 Done Done Sent false
33597 8943 Acked Read 2 Done Done Sent false
33598 8944 Acked Write 3 Done Done Sent false
33599 8945 Acked Read 2 Done Done Sent false
33600 8946 Acked Read 2 Done Done Sent false
33601 8947 Acked Write 6 Done Done Sent false
33602 8948 Acked Write 6 Done Done Sent false
33603 8949 Acked Read 10 Done Done Sent false
33604 8950 Acked Write 6 Done Done Sent false
33605 8951 Acked Read 1 Done Done Sent false
33606 8952 Acked Read 9 Done Done Sent false
33607 8953 Acked Read 8 Done Done Sent false
33608 8954 Acked Read 1 Done Done Sent false
33609 8955 Acked Read 1 Done Done Sent false
33610 8956 Acked Write 1 Done Done Sent false
33611 8957 Acked Flush 0 Done Done Sent false
33612 8958 Acked Flush 0 Done Done Sent false
33613 8959 Acked Write 8 Done Done Sent false
33614 8960 Acked Flush 0 Done Done Sent false
33615 8961 Acked Write 4 Done Done Sent false
33616 8962 Acked Read 1 Done Done Sent false
33617 8963 Acked Read 3 Done Done Sent false
33618 8964 Acked Read 10 Done Done Sent false
33619 8965 Acked Flush 0 Done Done Sent false
33620 8966 Acked Read 10 Done Done Sent false
33621 8967 Acked Read 2 Done Done Sent false
33622 8968 Acked Write 7 Done Done Sent false
33623 8969 Acked Read 2 Done Done Sent false
33624 8970 Acked Write 3 Done Done Sent false
33625 8971 Acked Write 1 Done Done Sent false
33626 8972 Acked Write 4 Done Done Sent false
33627 8973 Acked Write 2 Done Done Sent false
33628 8974 Acked Read 4 Done Done Sent false
33629 8975 Acked Write 2 Done Done Sent false
33630 8976 Acked Write 10 Done Done Sent false
33631 8977 Acked Read 5 Done Done Sent false
33632 8978 Acked Read 3 Done Done Sent false
33633 8979 Acked Read 3 Done Done Sent false
33634 8980 Acked Read 6 Done Done Sent false
33635 8981 Acked Read 1 Done Done Sent false
33636 8982 Acked Write 10 Done Done Sent false
33637 8983 Acked Read 1 Done Done Sent false
33638 8984 Acked Write 3 Done Done Sent false
33639 8985 Acked Read 1 Done Done Sent false
33640 8986 Acked Read 9 Done Done Sent false
33641 8987 Acked Read 9 Done Done Sent false
33642 8988 Acked Write 6 Done Done Sent false
33643 8989 Acked Write 6 Done Done Sent false
33644 8990 Acked Read 8 Done Done Sent false
33645 8991 Acked Write 5 Done Done Sent false
33646 8992 Acked Read 1 Done Done Sent false
33647 8993 Acked Read 1 Done Done Sent false
33648 8994 Acked Read 9 Done Done Sent false
33649 8995 Acked Read 7 Done Done Sent false
33650 8996 Acked Read 1 Done Done Sent false
33651 8997 Acked Read 5 Done Done Sent false
33652 8998 Acked Read 5 Done Done Sent false
33653 8999 Acked Read 5 Done Done Sent false
33654 9000 Acked Write 7 Done Done Sent false
33655 9001 Acked Read 5 Done Done Sent false
33656 9002 Acked Read 5 Done Done Sent false
33657 9003 Acked Write 3 Done Done Sent false
33658 9004 Acked Write 4 Done Done Sent false
33659 9005 Acked Write 5 Done Done Sent false
33660 9006 Acked Write 9 Done Done Sent false
33661 9007 Acked Flush 0 Done Done Sent false
33662 9008 Acked Read 7 Done Done Sent false
33663 9009 Acked Write 10 Done Done Sent false
33664 9010 Acked Read 7 Done Done Sent false
33665 9011 Acked Read 2 Done Done Sent false
33666 9012 Acked Flush 0 Done Done Sent false
33667 9013 Acked Write 1 Done Done Sent false
33668 9014 Acked Read 9 Done Done Sent false
33669 9015 Acked Read 1 Done Done Sent false
33670 9016 Acked Write 8 Done Done Sent false
33671 9017 Acked Read 6 Done Done Sent false
33672 9018 Acked Write 5 Done Done Sent false
33673 9019 Acked Flush 0 Done Done Sent false
33674 9020 Acked Read 1 Done Done Sent false
33675 9021 Acked Write 2 Done Done Sent false
33676 9022 Acked Write 9 Done Done Sent false
33677 9023 Acked Read 5 Done Done Sent false
33678 9024 Acked Read 3 Done Done Sent false
33679 9025 Acked Read 6 Done Done Sent false
33680 9026 Acked Read 8 Done Done Sent false
33681 9027 Acked Write 2 Done Done Sent false
33682 9028 Acked Read 7 Done Done Sent false
33683 9029 Acked Read 4 Done Done Sent false
33684 9030 Acked Write 2 Done Done Sent false
33685 9031 Acked Read 3 Done Done Sent false
33686 9032 Acked Read 10 Done Done Sent false
33687 9033 Acked Write 5 Done Done Sent false
33688 9034 Acked Flush 0 Done Done Sent false
33689 9035 Acked Write 6 Done Done Sent false
33690 9036 Acked Read 1 Done Done Sent false
33691 9037 Acked Read 2 Done Done Sent false
33692 9038 Acked Read 4 Done Done Sent false
33693 9039 Acked Read 9 Done Done Sent false
33694 9040 Acked Flush 0 Done Done Sent false
33695 9041 Acked Write 7 Done Done Sent false
33696 9042 Acked Write 8 Done Done Sent false
33697 9043 Acked Flush 0 Done Done Sent false
33698 9044 Acked Read 6 Done Done Sent false
33699 9045 Acked Flush 0 Done Done Sent false
33700 9046 Acked Write 6 Done Done Sent false
33701 9047 Acked Read 5 Done Done Sent false
33702 9048 Acked Read 5 Done Done Sent false
33703 9049 Acked Write 8 Done Done Sent false
33704 9050 Acked Read 10 Done Done Sent false
33705 9051 Acked Read 6 Done Done Sent false
33706 9052 Acked Read 9 Done Done Sent false
33707 9053 Acked Read 10 Done Done Sent false
33708 9054 Acked Read 7 Done Done Sent false
33709 9055 Acked Write 6 Done Done Sent false
33710 9056 Acked Read 10 Done Done Sent false
33711 9057 Acked Write 5 Done Done Sent false
33712 9058 Acked Write 7 Done Done Sent false
33713 9059 Acked Read 1 Done Done Sent false
33714 9060 Acked Read 8 Done Done Sent false
33715 9061 Acked Write 10 Done Done Sent false
33716 9062 Acked Write 2 Done Done Sent false
33717 9063 Acked Read 9 Done Done Sent false
33718 9064 Acked Write 8 Done Done Sent false
33719 9065 Acked Write 5 Done Done Sent false
33720 9066 Acked Read 6 Done Done Sent false
33721 9067 Acked Read 5 Done Done Sent false
33722 9068 Acked Read 8 Done Done Sent false
33723 9069 Acked Read 6 Done Done Sent false
33724 9070 Acked Write 8 Done Done Sent false
33725 9071 Acked Read 10 Done Done Sent false
33726 9072 Acked Read 6 Done Done Sent false
33727 9073 Acked Read 2 Done Done Sent false
33728 9074 Acked Read 4 Done Done Sent false
33729 9075 Acked Read 10 Done Done Sent false
33730 9076 Acked Read 9 Done Done Sent false
33731 9077 Acked Read 3 Done Done Sent false
33732 9078 Acked Write 9 Done Done Sent false
33733 9079 Acked Flush 0 Done Done Sent false
33734 9080 Acked Write 2 Done Done Sent false
33735 9081 Acked Write 1 Done Done Sent false
33736 9082 Acked Write 10 Done Done Sent false
33737 9083 Acked Read 6 Done Done Sent false
33738 9084 Acked Write 1 Done Done Sent false
33739 9085 Acked Read 3 Done Done Sent false
33740 9086 Acked Write 2 Done Done Sent false
33741 9087 Acked Write 4 Done Done Sent false
33742 9088 Acked Write 5 Done Done Sent false
33743 9089 Acked Write 2 Done Done Sent false
33744 9090 Acked Read 3 Done Done Sent false
33745 9091 Acked Write 8 Done Done Sent false
33746 9092 Acked Read 5 Done Done Sent false
33747 9093 Acked Read 4 Done Done Sent false
33748 9094 Acked Read 8 Done Done Sent false
33749 9095 Acked Read 6 Done Done Sent false
33750 9096 Acked Read 5 Done Done Sent false
33751 9097 Acked Flush 0 Done Done Sent false
33752 9098 Acked Write 7 Done Done Sent false
33753 9099 Acked Write 6 Done Done Sent false
33754 9100 Acked Read 10 Done Done Sent false
33755 9101 Acked Write 9 Done Done Sent false
33756 9102 Acked Read 9 Done Done Sent false
33757 9103 Acked Read 7 Done Done Sent false
33758 9104 Acked Read 4 Done Done Sent false
33759 9105 Acked Read 10 Done Done Sent false
33760 9106 Acked Write 4 Done Done Sent false
33761 9107 Acked Flush 0 Done Done Sent false
33762 9108 Acked Read 10 Done Done Sent false
33763 9109 Acked Write 4 Done Done Sent false
33764 9110 Acked Flush 0 Done Done Sent false
33765 9111 Acked Write 9 Done Done Sent false
33766 9112 Acked Read 10 Done Done Sent false
33767 9113 Acked Write 7 Done Done Sent false
33768 9114 Acked Read 3 Done Done Sent false
33769 9115 Acked Read 8 Done Done Sent false
33770 9116 Acked Read 7 Done Done Sent false
33771 9117 Acked Read 8 Done Done Sent false
33772 9118 Acked Read 7 Done Done Sent false
33773 9119 Acked Write 10 Done Done Sent false
33774 9120 Acked Read 2 Done Done Sent false
33775 9121 Acked Flush 0 Done Done Sent false
33776 9122 Acked Read 9 Done Done Sent false
33777 9123 Acked Write 8 Done Done Sent false
33778 9124 Acked Read 9 Done Done Sent false
33779 9125 Acked Read 7 Done Done Sent false
33780 9126 Acked Write 2 Done Done Sent false
33781 9127 Acked Read 6 Done Done Sent false
33782 9128 Acked Write 10 Done Done Sent false
33783 9129 Acked Write 9 Done Done Sent false
33784 9130 Acked Write 1 Done Done Sent false
33785 9131 Acked Flush 0 Done Done Sent false
33786 9132 Acked Flush 0 Done Done Sent false
33787 9133 Acked Write 8 Done Done Sent false
33788 9134 Acked Read 10 Done Done Sent false
33789 9135 Acked Read 4 Done Done Sent false
33790 9136 Acked Write 10 Done Done Sent false
33791 9137 Acked Read 1 Done Done Sent false
33792 9138 Acked Read 8 Done Done Sent false
33793 9139 Acked Write 6 Done Done Sent false
33794 9140 Acked Read 1 Done Done Sent false
33795 9141 Acked Flush 0 Done Done Sent false
33796 9142 Acked Write 10 Done Done Sent false
33797 9143 Acked Read 3 Done Done Sent false
33798 9144 Acked Write 5 Done Done Sent false
33799 9145 Acked Read 9 Done Done Sent false
33800 9146 Acked Read 4 Done Done Sent false
33801 9147 Acked Read 1 Done Done Sent false
33802 9148 Acked Flush 0 Done Done Sent false
33803 9149 Acked Read 8 Done Done Sent false
33804 9150 Acked Read 8 Done Done Sent false
33805 STATES DS:0 DS:1 DS:2 TOTAL
33806 Sent 0 0 301 301
33807 Done 301 301 0 602
33808 Skipped 0 0 0 0
33809 Error 0 0 0 0
33810 Last Flush: 9148 9148 8849
33811 Downstairs last five completed: 8849 8848 8847 8846 8845
33812 Upstairs last five completed: 9150 9149 9148 9147 9146
33813 CLIENT: Up:0 ds:301 act:2
33814 ----------------------------------------------------------------
33815 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
33816 States: Active Active Offline
33817 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
33818 8850 Acked Read 7 Done Done Sent false
33819 8851 Acked Read 8 Done Done Sent false
33820 8852 Acked Read 3 Done Done Sent false
33821 8853 Acked Write 10 Done Done Sent false
33822 8854 Acked Write 10 Done Done Sent false
33823 8855 Acked Flush 0 Done Done Sent false
33824 8856 Acked Read 3 Done Done Sent false
33825 8857 Acked Flush 0 Done Done Sent false
33826 8858 Acked Read 1 Done Done Sent false
33827 8859 Acked Write 2 Done Done Sent false
33828 8860 Acked Read 7 Done Done Sent false
33829 8861 Acked Write 7 Done Done Sent false
33830 8862 Acked Flush 0 Done Done Sent false
33831 8863 Acked Write 6 Done Done Sent false
33832 8864 Acked Read 7 Done Done Sent false
33833 8865 Acked Write 2 Done Done Sent false
33834 8866 Acked Write 2 Done Done Sent false
33835 8867 Acked Read 7 Done Done Sent false
33836 8868 Acked Read 3 Done Done Sent false
33837 8869 Acked Write 5 Done Done Sent false
33838 8870 Acked Read 9 Done Done Sent false
33839 8871 Acked Read 9 Done Done Sent false
33840 8872 Acked Read 2 Done Done Sent false
33841 8873 Acked Write 2 Done Done Sent false
33842 8874 Acked Write 7 Done Done Sent false
33843 8875 Acked Read 3 Done Done Sent false
33844 8876 Acked Read 5 Done Done Sent false
33845 8877 Acked Read 6 Done Done Sent false
33846 8878 Acked Write 4 Done Done Sent false
33847 8879 Acked Read 6 Done Done Sent false
33848 8880 Acked Read 5 Done Done Sent false
33849 8881 Acked Write 8 Done Done Sent false
33850 8882 Acked Read 10 Done Done Sent false
33851 8883 Acked Flush 0 Done Done Sent false
33852 8884 Acked Read 10 Done Done Sent false
33853 8885 Acked Write 3 Done Done Sent false
33854 8886 Acked Read 2 Done Done Sent false
33855 8887 Acked Read 1 Done Done Sent false
33856 8888 Acked Write 1 Done Done Sent false
33857 8889 Acked Read 10 Done Done Sent false
33858 8890 Acked Read 6 Done Done Sent false
33859 8891 Acked Read 8 Done Done Sent false
33860 8892 Acked Write 9 Done Done Sent false
33861 8893 Acked Write 6 Done Done Sent false
33862 8894 Acked Read 3 Done Done Sent false
33863 8895 Acked Write 5 Done Done Sent false
33864 8896 Acked Read 1 Done Done Sent false
33865 8897 Acked Read 4 Done Done Sent false
33866 8898 Acked Write 3 Done Done Sent false
33867 8899 Acked Write 1 Done Done Sent false
33868 8900 Acked Read 10 Done Done Sent false
33869 8901 Acked Write 1 Done Done Sent false
33870 8902 Acked Write 10 Done Done Sent false
33871 8903 Acked Read 4 Done Done Sent false
33872 8904 Acked Read 3 Done Done Sent false
33873 8905 Acked Read 3 Done Done Sent false
33874 8906 Acked Read 7 Done Done Sent false
33875 8907 Acked Read 6 Done Done Sent false
33876 8908 Acked Read 10 Done Done Sent false
33877 8909 Acked Read 8 Done Done Sent false
33878 8910 Acked Read 7 Done Done Sent false
33879 8911 Acked Write 1 Done Done Sent false
33880 8912 Acked Read 3 Done Done Sent false
33881 8913 Acked Write 8 Done Done Sent false
33882 8914 Acked Read 2 Done Done Sent false
33883 8915 Acked Read 2 Done Done Sent false
33884 8916 Acked Read 5 Done Done Sent false
33885 8917 Acked Read 8 Done Done Sent false
33886 8918 Acked Write 9 Done Done Sent false
33887 8919 Acked Read 8 Done Done Sent false
33888 8920 Acked Flush 0 Done Done Sent false
33889 8921 Acked Write 5 Done Done Sent false
33890 8922 Acked Read 4 Done Done Sent false
33891 8923 Acked Write 1 Done Done Sent false
33892 8924 Acked Read 8 Done Done Sent false
33893 8925 Acked Write 9 Done Done Sent false
33894 8926 Acked Write 3 Done Done Sent false
33895 8927 Acked Write 7 Done Done Sent false
33896 8928 Acked Write 5 Done Done Sent false
33897 8929 Acked Read 10 Done Done Sent false
33898 8930 Acked Read 5 Done Done Sent false
33899 8931 Acked Write 3 Done Done Sent false
33900 8932 Acked Read 3 Done Done Sent false
33901 8933 Acked Write 4 Done Done Sent false
33902 8934 Acked Write 9 Done Done Sent false
33903 8935 Acked Write 9 Done Done Sent false
33904 8936 Acked Read 7 Done Done Sent false
33905 8937 Acked Write 5 Done Done Sent false
33906 8938 Acked Read 8 Done Done Sent false
33907 8939 Acked Read 7 Done Done Sent false
33908 8940 Acked Write 9 Done Done Sent false
33909 8941 Acked Write 2 Done Done Sent false
33910 8942 Acked Write 5 Done Done Sent false
33911 8943 Acked Read 2 Done Done Sent false
33912 8944 Acked Write 3 Done Done Sent false
33913 8945 Acked Read 2 Done Done Sent false
33914 8946 Acked Read 2 Done Done Sent false
33915 8947 Acked Write 6 Done Done Sent false
33916 8948 Acked Write 6 Done Done Sent false
33917 8949 Acked Read 10 Done Done Sent false
33918 8950 Acked Write 6 Done Done Sent false
33919 8951 Acked Read 1 Done Done Sent false
33920 8952 Acked Read 9 Done Done Sent false
33921 8953 Acked Read 8 Done Done Sent false
33922 8954 Acked Read 1 Done Done Sent false
33923 8955 Acked Read 1 Done Done Sent false
33924 8956 Acked Write 1 Done Done Sent false
33925 8957 Acked Flush 0 Done Done Sent false
33926 8958 Acked Flush 0 Done Done Sent false
33927 8959 Acked Write 8 Done Done Sent false
33928 8960 Acked Flush 0 Done Done Sent false
33929 8961 Acked Write 4 Done Done Sent false
33930 8962 Acked Read 1 Done Done Sent false
33931 8963 Acked Read 3 Done Done Sent false
33932 8964 Acked Read 10 Done Done Sent false
33933 8965 Acked Flush 0 Done Done Sent false
33934 8966 Acked Read 10 Done Done Sent false
33935 8967 Acked Read 2 Done Done Sent false
33936 8968 Acked Write 7 Done Done Sent false
33937 8969 Acked Read 2 Done Done Sent false
33938 8970 Acked Write 3 Done Done Sent false
33939 8971 Acked Write 1 Done Done Sent false
33940 8972 Acked Write 4 Done Done Sent false
33941 8973 Acked Write 2 Done Done Sent false
33942 8974 Acked Read 4 Done Done Sent false
33943 8975 Acked Write 2 Done Done Sent false
33944 8976 Acked Write 10 Done Done Sent false
33945 8977 Acked Read 5 Done Done Sent false
33946 8978 Acked Read 3 Done Done Sent false
33947 8979 Acked Read 3 Done Done Sent false
33948 8980 Acked Read 6 Done Done Sent false
33949 8981 Acked Read 1 Done Done Sent false
33950 8982 Acked Write 10 Done Done Sent false
33951 8983 Acked Read 1 Done Done Sent false
33952 8984 Acked Write 3 Done Done Sent false
33953 8985 Acked Read 1 Done Done Sent false
33954 8986 Acked Read 9 Done Done Sent false
33955 8987 Acked Read 9 Done Done Sent false
33956 8988 Acked Write 6 Done Done Sent false
33957 8989 Acked Write 6 Done Done Sent false
33958 8990 Acked Read 8 Done Done Sent false
33959 8991 Acked Write 5 Done Done Sent false
33960 8992 Acked Read 1 Done Done Sent false
33961 8993 Acked Read 1 Done Done Sent false
33962 8994 Acked Read 9 Done Done Sent false
33963 8995 Acked Read 7 Done Done Sent false
33964 8996 Acked Read 1 Done Done Sent false
33965 8997 Acked Read 5 Done Done Sent false
33966 8998 Acked Read 5 Done Done Sent false
33967 8999 Acked Read 5 Done Done Sent false
33968 9000 Acked Write 7 Done Done Sent false
33969 9001 Acked Read 5 Done Done Sent false
33970 9002 Acked Read 5 Done Done Sent false
33971 9003 Acked Write 3 Done Done Sent false
33972 9004 Acked Write 4 Done Done Sent false
33973 9005 Acked Write 5 Done Done Sent false
33974 9006 Acked Write 9 Done Done Sent false
33975 9007 Acked Flush 0 Done Done Sent false
33976 9008 Acked Read 7 Done Done Sent false
33977 9009 Acked Write 10 Done Done Sent false
33978 9010 Acked Read 7 Done Done Sent false
33979 9011 Acked Read 2 Done Done Sent false
33980 9012 Acked Flush 0 Done Done Sent false
33981 9013 Acked Write 1 Done Done Sent false
33982 9014 Acked Read 9 Done Done Sent false
33983 9015 Acked Read 1 Done Done Sent false
33984 9016 Acked Write 8 Done Done Sent false
33985 9017 Acked Read 6 Done Done Sent false
33986 9018 Acked Write 5 Done Done Sent false
33987 9019 Acked Flush 0 Done Done Sent false
33988 9020 Acked Read 1 Done Done Sent false
33989 9021 Acked Write 2 Done Done Sent false
33990 9022 Acked Write 9 Done Done Sent false
33991 9023 Acked Read 5 Done Done Sent false
33992 9024 Acked Read 3 Done Done Sent false
33993 9025 Acked Read 6 Done Done Sent false
33994 9026 Acked Read 8 Done Done Sent false
33995 9027 Acked Write 2 Done Done Sent false
33996 9028 Acked Read 7 Done Done Sent false
33997 9029 Acked Read 4 Done Done Sent false
33998 9030 Acked Write 2 Done Done Sent false
33999 9031 Acked Read 3 Done Done Sent false
34000 9032 Acked Read 10 Done Done Sent false
34001 9033 Acked Write 5 Done Done Sent false
34002 9034 Acked Flush 0 Done Done Sent false
34003 9035 Acked Write 6 Done Done Sent false
34004 9036 Acked Read 1 Done Done Sent false
34005 9037 Acked Read 2 Done Done Sent false
34006 9038 Acked Read 4 Done Done Sent false
34007 9039 Acked Read 9 Done Done Sent false
34008 9040 Acked Flush 0 Done Done Sent false
34009 9041 Acked Write 7 Done Done Sent false
34010 9042 Acked Write 8 Done Done Sent false
34011 9043 Acked Flush 0 Done Done Sent false
34012 9044 Acked Read 6 Done Done Sent false
34013 9045 Acked Flush 0 Done Done Sent false
34014 9046 Acked Write 6 Done Done Sent false
34015 9047 Acked Read 5 Done Done Sent false
34016 9048 Acked Read 5 Done Done Sent false
34017 9049 Acked Write 8 Done Done Sent false
34018 9050 Acked Read 10 Done Done Sent false
34019 9051 Acked Read 6 Done Done Sent false
34020 9052 Acked Read 9 Done Done Sent false
34021 9053 Acked Read 10 Done Done Sent false
34022 9054 Acked Read 7 Done Done Sent false
34023 9055 Acked Write 6 Done Done Sent false
34024 9056 Acked Read 10 Done Done Sent false
34025 9057 Acked Write 5 Done Done Sent false
34026 9058 Acked Write 7 Done Done Sent false
34027 9059 Acked Read 1 Done Done Sent false
34028 9060 Acked Read 8 Done Done Sent false
34029 9061 Acked Write 10 Done Done Sent false
34030 9062 Acked Write 2 Done Done Sent false
34031 9063 Acked Read 9 Done Done Sent false
34032 9064 Acked Write 8 Done Done Sent false
34033 9065 Acked Write 5 Done Done Sent false
34034 9066 Acked Read 6 Done Done Sent false
34035 9067 Acked Read 5 Done Done Sent false
34036 9068 Acked Read 8 Done Done Sent false
34037 9069 Acked Read 6 Done Done Sent false
34038 9070 Acked Write 8 Done Done Sent false
34039 9071 Acked Read 10 Done Done Sent false
34040 9072 Acked Read 6 Done Done Sent false
34041 9073 Acked Read 2 Done Done Sent false
34042 9074 Acked Read 4 Done Done Sent false
34043 9075 Acked Read 10 Done Done Sent false
34044 9076 Acked Read 9 Done Done Sent false
34045 9077 Acked Read 3 Done Done Sent false
34046 9078 Acked Write 9 Done Done Sent false
34047 9079 Acked Flush 0 Done Done Sent false
34048 9080 Acked Write 2 Done Done Sent false
34049 9081 Acked Write 1 Done Done Sent false
34050 9082 Acked Write 10 Done Done Sent false
34051 9083 Acked Read 6 Done Done Sent false
34052 9084 Acked Write 1 Done Done Sent false
34053 9085 Acked Read 3 Done Done Sent false
34054 9086 Acked Write 2 Done Done Sent false
34055 9087 Acked Write 4 Done Done Sent false
34056 9088 Acked Write 5 Done Done Sent false
34057 9089 Acked Write 2 Done Done Sent false
34058 9090 Acked Read 3 Done Done Sent false
34059 9091 Acked Write 8 Done Done Sent false
34060 9092 Acked Read 5 Done Done Sent false
34061 9093 Acked Read 4 Done Done Sent false
34062 9094 Acked Read 8 Done Done Sent false
34063 9095 Acked Read 6 Done Done Sent false
34064 9096 Acked Read 5 Done Done Sent false
34065 9097 Acked Flush 0 Done Done Sent false
34066 9098 Acked Write 7 Done Done Sent false
34067 9099 Acked Write 6 Done Done Sent false
34068 9100 Acked Read 10 Done Done Sent false
34069 9101 Acked Write 9 Done Done Sent false
34070 9102 Acked Read 9 Done Done Sent false
34071 9103 Acked Read 7 Done Done Sent false
34072 9104 Acked Read 4 Done Done Sent false
34073 9105 Acked Read 10 Done Done Sent false
34074 9106 Acked Write 4 Done Done Sent false
34075 9107 Acked Flush 0 Done Done Sent false
34076 9108 Acked Read 10 Done Done Sent false
34077 9109 Acked Write 4 Done Done Sent false
34078 9110 Acked Flush 0 Done Done Sent false
34079 9111 Acked Write 9 Done Done Sent false
34080 9112 Acked Read 10 Done Done Sent false
34081 9113 Acked Write 7 Done Done Sent false
34082 9114 Acked Read 3 Done Done Sent false
34083 9115 Acked Read 8 Done Done Sent false
34084 9116 Acked Read 7 Done Done Sent false
34085 9117 Acked Read 8 Done Done Sent false
34086 9118 Acked Read 7 Done Done Sent false
34087 9119 Acked Write 10 Done Done Sent false
34088 9120 Acked Read 2 Done Done Sent false
34089 9121 Acked Flush 0 Done Done Sent false
34090 9122 Acked Read 9 Done Done Sent false
34091 9123 Acked Write 8 Done Done Sent false
34092 9124 Acked Read 9 Done Done Sent false
34093 9125 Acked Read 7 Done Done Sent false
34094 9126 Acked Write 2 Done Done Sent false
34095 9127 Acked Read 6 Done Done Sent false
34096 9128 Acked Write 10 Done Done Sent false
34097 9129 Acked Write 9 Done Done Sent false
34098 9130 Acked Write 1 Done Done Sent false
34099 9131 Acked Flush 0 Done Done Sent false
34100 9132 Acked Flush 0 Done Done Sent false
34101 9133 Acked Write 8 Done Done Sent false
34102 9134 Acked Read 10 Done Done Sent false
34103 9135 Acked Read 4 Done Done Sent false
34104 9136 Acked Write 10 Done Done Sent false
34105 9137 Acked Read 1 Done Done Sent false
34106 9138 Acked Read 8 Done Done Sent false
34107 9139 Acked Write 6 Done Done Sent false
34108 9140 Acked Read 1 Done Done Sent false
34109 9141 Acked Flush 0 Done Done Sent false
34110 9142 Acked Write 10 Done Done Sent false
34111 9143 Acked Read 3 Done Done Sent false
34112 9144 Acked Write 5 Done Done Sent false
34113 9145 Acked Read 9 Done Done Sent false
34114 9146 Acked Read 4 Done Done Sent false
34115 9147 Acked Read 1 Done Done Sent false
34116 9148 Acked Flush 0 Done Done Sent false
34117 9149 Acked Read 8 Done Done Sent false
34118 9150 Acked Read 8 Done Done Sent false
34119 9151 Acked Flush 0 Done Done Sent false
34120 STATES DS:0 DS:1 DS:2 TOTAL
34121 Sent 0 0 302 302
34122 Done 302 302 0 604
34123 Skipped 0 0 0 0
34124 Error 0 0 0 0
34125 Last Flush: 9151 9151 8849
34126 Downstairs last five completed: 8849 8848 8847 8846 8845
34127 Upstairs last five completed: 9151 9150 9149 9148 9147
34128 CLIENT: Up:0 ds:302 act:2
341292024-11-15T02:04:36.302ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
341302024-11-15T02:04:36.302ZWARNcrucible: ds_connection connect to 127.0.0.1:8830 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
341312024-11-15T02:04:36.302ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
341322024-11-15T02:04:36.302ZWARNcrucible: downstairs task for 2 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
341332024-11-15T02:04:36.302ZINFOcrucible: client task is exiting = io task client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
34134 ----------------------------------------------------------------
34135 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
34136 States: Active Active Offline
34137 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
34138 8850 Acked Read 7 Done Done Sent false
34139 8851 Acked Read 8 Done Done Sent false
34140 8852 Acked Read 3 Done Done Sent false
34141 8853 Acked Write 10 Done Done Sent false
34142 8854 Acked Write 10 Done Done Sent false
34143 8855 Acked Flush 0 Done Done Sent false
34144 8856 Acked Read 3 Done Done Sent false
34145 8857 Acked Flush 0 Done Done Sent false
34146 8858 Acked Read 1 Done Done Sent false
34147 8859 Acked Write 2 Done Done Sent false
34148 8860 Acked Read 7 Done Done Sent false
34149 8861 Acked Write 7 Done Done Sent false
34150 8862 Acked Flush 0 Done Done Sent false
34151 8863 Acked Write 6 Done Done Sent false
34152 8864 Acked Read 7 Done Done Sent false
34153 8865 Acked Write 2 Done Done Sent false
34154 8866 Acked Write 2 Done Done Sent false
34155 8867 Acked Read 7 Done Done Sent false
34156 8868 Acked Read 3 Done Done Sent false
34157 8869 Acked Write 5 Done Done Sent false
34158 8870 Acked Read 9 Done Done Sent false
34159 8871 Acked Read 9 Done Done Sent false
34160 8872 Acked Read 2 Done Done Sent false
34161 8873 Acked Write 2 Done Done Sent false
34162 8874 Acked Write 7 Done Done Sent false
34163 8875 Acked Read 3 Done Done Sent false
34164 8876 Acked Read 5 Done Done Sent false
34165 8877 Acked Read 6 Done Done Sent false
34166 8878 Acked Write 4 Done Done Sent false
34167 8879 Acked Read 6 Done Done Sent false
34168 8880 Acked Read 5 Done Done Sent false
34169 8881 Acked Write 8 Done Done Sent false
34170 8882 Acked Read 10 Done Done Sent false
34171 8883 Acked Flush 0 Done Done Sent false
34172 8884 Acked Read 10 Done Done Sent false
34173 8885 Acked Write 3 Done Done Sent false
34174 8886 Acked Read 2 Done Done Sent false
34175 8887 Acked Read 1 Done Done Sent false
34176 8888 Acked Write 1 Done Done Sent false
34177 8889 Acked Read 10 Done Done Sent false
34178 8890 Acked Read 6 Done Done Sent false
34179 8891 Acked Read 8 Done Done Sent false
34180 8892 Acked Write 9 Done Done Sent false
34181 8893 Acked Write 6 Done Done Sent false
34182 8894 Acked Read 3 Done Done Sent false
34183 8895 Acked Write 5 Done Done Sent false
34184 8896 Acked Read 1 Done Done Sent false
34185 8897 Acked Read 4 Done Done Sent false
34186 8898 Acked Write 3 Done Done Sent false
34187 8899 Acked Write 1 Done Done Sent false
34188 8900 Acked Read 10 Done Done Sent false
34189 8901 Acked Write 1 Done Done Sent false
34190 8902 Acked Write 10 Done Done Sent false
34191 8903 Acked Read 4 Done Done Sent false
34192 8904 Acked Read 3 Done Done Sent false
34193 8905 Acked Read 3 Done Done Sent false
34194 8906 Acked Read 7 Done Done Sent false
34195 8907 Acked Read 6 Done Done Sent false
34196 8908 Acked Read 10 Done Done Sent false
34197 8909 Acked Read 8 Done Done Sent false
34198 8910 Acked Read 7 Done Done Sent false
34199 8911 Acked Write 1 Done Done Sent false
34200 8912 Acked Read 3 Done Done Sent false
34201 8913 Acked Write 8 Done Done Sent false
34202 8914 Acked Read 2 Done Done Sent false
34203 8915 Acked Read 2 Done Done Sent false
34204 8916 Acked Read 5 Done Done Sent false
34205 8917 Acked Read 8 Done Done Sent false
34206 8918 Acked Write 9 Done Done Sent false
34207 8919 Acked Read 8 Done Done Sent false
34208 8920 Acked Flush 0 Done Done Sent false
34209 8921 Acked Write 5 Done Done Sent false
34210 8922 Acked Read 4 Done Done Sent false
34211 8923 Acked Write 1 Done Done Sent false
34212 8924 Acked Read 8 Done Done Sent false
34213 8925 Acked Write 9 Done Done Sent false
34214 8926 Acked Write 3 Done Done Sent false
34215 8927 Acked Write 7 Done Done Sent false
34216 8928 Acked Write 5 Done Done Sent false
34217 8929 Acked Read 10 Done Done Sent false
34218 8930 Acked Read 5 Done Done Sent false
34219 8931 Acked Write 3 Done Done Sent false
34220 8932 Acked Read 3 Done Done Sent false
34221 8933 Acked Write 4 Done Done Sent false
34222 8934 Acked Write 9 Done Done Sent false
34223 8935 Acked Write 9 Done Done Sent false
34224 8936 Acked Read 7 Done Done Sent false
34225 8937 Acked Write 5 Done Done Sent false
34226 8938 Acked Read 8 Done Done Sent false
34227 8939 Acked Read 7 Done Done Sent false
34228 8940 Acked Write 9 Done Done Sent false
34229 8941 Acked Write 2 Done Done Sent false
34230 8942 Acked Write 5 Done Done Sent false
34231 8943 Acked Read 2 Done Done Sent false
34232 8944 Acked Write 3 Done Done Sent false
34233 8945 Acked Read 2 Done Done Sent false
34234 8946 Acked Read 2 Done Done Sent false
34235 8947 Acked Write 6 Done Done Sent false
34236 8948 Acked Write 6 Done Done Sent false
34237 8949 Acked Read 10 Done Done Sent false
34238 8950 Acked Write 6 Done Done Sent false
34239 8951 Acked Read 1 Done Done Sent false
34240 8952 Acked Read 9 Done Done Sent false
34241 8953 Acked Read 8 Done Done Sent false
34242 8954 Acked Read 1 Done Done Sent false
34243 8955 Acked Read 1 Done Done Sent false
34244 8956 Acked Write 1 Done Done Sent false
34245 8957 Acked Flush 0 Done Done Sent false
34246 8958 Acked Flush 0 Done Done Sent false
34247 8959 Acked Write 8 Done Done Sent false
34248 8960 Acked Flush 0 Done Done Sent false
34249 8961 Acked Write 4 Done Done Sent false
34250 8962 Acked Read 1 Done Done Sent false
34251 8963 Acked Read 3 Done Done Sent false
34252 8964 Acked Read 10 Done Done Sent false
34253 8965 Acked Flush 0 Done Done Sent false
34254 8966 Acked Read 10 Done Done Sent false
34255 8967 Acked Read 2 Done Done Sent false
34256 8968 Acked Write 7 Done Done Sent false
34257 8969 Acked Read 2 Done Done Sent false
34258 8970 Acked Write 3 Done Done Sent false
34259 8971 Acked Write 1 Done Done Sent false
34260 8972 Acked Write 4 Done Done Sent false
34261 8973 Acked Write 2 Done Done Sent false
34262 8974 Acked Read 4 Done Done Sent false
34263 8975 Acked Write 2 Done Done Sent false
34264 8976 Acked Write 10 Done Done Sent false
34265 8977 Acked Read 5 Done Done Sent false
34266 8978 Acked Read 3 Done Done Sent false
34267 8979 Acked Read 3 Done Done Sent false
34268 8980 Acked Read 6 Done Done Sent false
34269 8981 Acked Read 1 Done Done Sent false
34270 8982 Acked Write 10 Done Done Sent false
34271 8983 Acked Read 1 Done Done Sent false
34272 8984 Acked Write 3 Done Done Sent false
34273 8985 Acked Read 1 Done Done Sent false
34274 8986 Acked Read 9 Done Done Sent false
34275 8987 Acked Read 9 Done Done Sent false
34276 8988 Acked Write 6 Done Done Sent false
34277 8989 Acked Write 6 Done Done Sent false
34278 8990 Acked Read 8 Done Done Sent false
34279 8991 Acked Write 5 Done Done Sent false
34280 8992 Acked Read 1 Done Done Sent false
34281 8993 Acked Read 1 Done Done Sent false
34282 8994 Acked Read 9 Done Done Sent false
34283 8995 Acked Read 7 Done Done Sent false
34284 8996 Acked Read 1 Done Done Sent false
34285 8997 Acked Read 5 Done Done Sent false
34286 8998 Acked Read 5 Done Done Sent false
34287 8999 Acked Read 5 Done Done Sent false
34288 9000 Acked Write 7 Done Done Sent false
34289 9001 Acked Read 5 Done Done Sent false
34290 9002 Acked Read 5 Done Done Sent false
34291 9003 Acked Write 3 Done Done Sent false
34292 9004 Acked Write 4 Done Done Sent false
34293 9005 Acked Write 5 Done Done Sent false
34294 9006 Acked Write 9 Done Done Sent false
34295 9007 Acked Flush 0 Done Done Sent false
34296 9008 Acked Read 7 Done Done Sent false
34297 9009 Acked Write 10 Done Done Sent false
34298 9010 Acked Read 7 Done Done Sent false
34299 9011 Acked Read 2 Done Done Sent false
34300 9012 Acked Flush 0 Done Done Sent false
34301 9013 Acked Write 1 Done Done Sent false
34302 9014 Acked Read 9 Done Done Sent false
34303 9015 Acked Read 1 Done Done Sent false
34304 9016 Acked Write 8 Done Done Sent false
34305 9017 Acked Read 6 Done Done Sent false
34306 9018 Acked Write 5 Done Done Sent false
34307 9019 Acked Flush 0 Done Done Sent false
34308 9020 Acked Read 1 Done Done Sent false
34309 9021 Acked Write 2 Done Done Sent false
34310 9022 Acked Write 9 Done Done Sent false
34311 9023 Acked Read 5 Done Done Sent false
34312 9024 Acked Read 3 Done Done Sent false
34313 9025 Acked Read 6 Done Done Sent false
34314 9026 Acked Read 8 Done Done Sent false
34315 9027 Acked Write 2 Done Done Sent false
34316 9028 Acked Read 7 Done Done Sent false
34317 9029 Acked Read 4 Done Done Sent false
34318 9030 Acked Write 2 Done Done Sent false
34319 9031 Acked Read 3 Done Done Sent false
34320 9032 Acked Read 10 Done Done Sent false
34321 9033 Acked Write 5 Done Done Sent false
34322 9034 Acked Flush 0 Done Done Sent false
34323 9035 Acked Write 6 Done Done Sent false
34324 9036 Acked Read 1 Done Done Sent false
34325 9037 Acked Read 2 Done Done Sent false
34326 9038 Acked Read 4 Done Done Sent false
34327 9039 Acked Read 9 Done Done Sent false
34328 9040 Acked Flush 0 Done Done Sent false
34329 9041 Acked Write 7 Done Done Sent false
34330 9042 Acked Write 8 Done Done Sent false
34331 9043 Acked Flush 0 Done Done Sent false
34332 9044 Acked Read 6 Done Done Sent false
34333 9045 Acked Flush 0 Done Done Sent false
34334 9046 Acked Write 6 Done Done Sent false
34335 9047 Acked Read 5 Done Done Sent false
34336 9048 Acked Read 5 Done Done Sent false
34337 9049 Acked Write 8 Done Done Sent false
34338 9050 Acked Read 10 Done Done Sent false
34339 9051 Acked Read 6 Done Done Sent false
34340 9052 Acked Read 9 Done Done Sent false
34341 9053 Acked Read 10 Done Done Sent false
34342 9054 Acked Read 7 Done Done Sent false
34343 9055 Acked Write 6 Done Done Sent false
34344 9056 Acked Read 10 Done Done Sent false
34345 9057 Acked Write 5 Done Done Sent false
34346 9058 Acked Write 7 Done Done Sent false
34347 9059 Acked Read 1 Done Done Sent false
34348 9060 Acked Read 8 Done Done Sent false
34349 9061 Acked Write 10 Done Done Sent false
34350 9062 Acked Write 2 Done Done Sent false
34351 9063 Acked Read 9 Done Done Sent false
34352 9064 Acked Write 8 Done Done Sent false
34353 9065 Acked Write 5 Done Done Sent false
34354 9066 Acked Read 6 Done Done Sent false
34355 9067 Acked Read 5 Done Done Sent false
34356 9068 Acked Read 8 Done Done Sent false
34357 9069 Acked Read 6 Done Done Sent false
34358 9070 Acked Write 8 Done Done Sent false
34359 9071 Acked Read 10 Done Done Sent false
34360 9072 Acked Read 6 Done Done Sent false
34361 9073 Acked Read 2 Done Done Sent false
34362 9074 Acked Read 4 Done Done Sent false
34363 9075 Acked Read 10 Done Done Sent false
34364 9076 Acked Read 9 Done Done Sent false
34365 9077 Acked Read 3 Done Done Sent false
34366 9078 Acked Write 9 Done Done Sent false
34367 9079 Acked Flush 0 Done Done Sent false
34368 9080 Acked Write 2 Done Done Sent false
34369 9081 Acked Write 1 Done Done Sent false
34370 9082 Acked Write 10 Done Done Sent false
34371 9083 Acked Read 6 Done Done Sent false
34372 9084 Acked Write 1 Done Done Sent false
34373 9085 Acked Read 3 Done Done Sent false
34374 9086 Acked Write 2 Done Done Sent false
34375 9087 Acked Write 4 Done Done Sent false
34376 9088 Acked Write 5 Done Done Sent false
34377 9089 Acked Write 2 Done Done Sent false
34378 9090 Acked Read 3 Done Done Sent false
34379 9091 Acked Write 8 Done Done Sent false
34380 9092 Acked Read 5 Done Done Sent false
34381 9093 Acked Read 4 Done Done Sent false
34382 9094 Acked Read 8 Done Done Sent false
34383 9095 Acked Read 6 Done Done Sent false
34384 9096 Acked Read 5 Done Done Sent false
34385 9097 Acked Flush 0 Done Done Sent false
34386 9098 Acked Write 7 Done Done Sent false
34387 9099 Acked Write 6 Done Done Sent false
34388 9100 Acked Read 10 Done Done Sent false
34389 9101 Acked Write 9 Done Done Sent false
34390 9102 Acked Read 9 Done Done Sent false
34391 9103 Acked Read 7 Done Done Sent false
34392 9104 Acked Read 4 Done Done Sent false
34393 9105 Acked Read 10 Done Done Sent false
34394 9106 Acked Write 4 Done Done Sent false
34395 9107 Acked Flush 0 Done Done Sent false
34396 9108 Acked Read 10 Done Done Sent false
34397 9109 Acked Write 4 Done Done Sent false
34398 9110 Acked Flush 0 Done Done Sent false
34399 9111 Acked Write 9 Done Done Sent false
34400 9112 Acked Read 10 Done Done Sent false
34401 9113 Acked Write 7 Done Done Sent false
34402 9114 Acked Read 3 Done Done Sent false
34403 9115 Acked Read 8 Done Done Sent false
34404 9116 Acked Read 7 Done Done Sent false
34405 9117 Acked Read 8 Done Done Sent false
34406 9118 Acked Read 7 Done Done Sent false
34407 9119 Acked Write 10 Done Done Sent false
34408 9120 Acked Read 2 Done Done Sent false
34409 9121 Acked Flush 0 Done Done Sent false
34410 9122 Acked Read 9 Done Done Sent false
34411 9123 Acked Write 8 Done Done Sent false
34412 9124 Acked Read 9 Done Done Sent false
34413 9125 Acked Read 7 Done Done Sent false
34414 9126 Acked Write 2 Done Done Sent false
34415 9127 Acked Read 6 Done Done Sent false
34416 9128 Acked Write 10 Done Done Sent false
34417 9129 Acked Write 9 Done Done Sent false
34418 9130 Acked Write 1 Done Done Sent false
34419 9131 Acked Flush 0 Done Done Sent false
34420 9132 Acked Flush 0 Done Done Sent false
34421 9133 Acked Write 8 Done Done Sent false
34422 9134 Acked Read 10 Done Done Sent false
34423 9135 Acked Read 4 Done Done Sent false
34424 9136 Acked Write 10 Done Done Sent false
34425 9137 Acked Read 1 Done Done Sent false
34426 9138 Acked Read 8 Done Done Sent false
34427 9139 Acked Write 6 Done Done Sent false
34428 9140 Acked Read 1 Done Done Sent false
34429 9141 Acked Flush 0 Done Done Sent false
34430 9142 Acked Write 10 Done Done Sent false
34431 9143 Acked Read 3 Done Done Sent false
34432 9144 Acked Write 5 Done Done Sent false
34433 9145 Acked Read 9 Done Done Sent false
34434 9146 Acked Read 4 Done Done Sent false
34435 9147 Acked Read 1 Done Done Sent false
34436 9148 Acked Flush 0 Done Done Sent false
34437 9149 Acked Read 8 Done Done Sent false
34438 9150 Acked Read 8 Done Done Sent false
34439 9151 Acked Flush 0 Done Done Sent false
34440 STATES DS:0 DS:1 DS:2 TOTAL
34441 Sent 0 0 302 302
34442 Done 302 302 0 604
34443 Skipped 0 0 0 0
34444 Error 0 0 0 0
34445 Last Flush: 9151 9151 8849
34446 Downstairs last five completed: 8849 8848 8847 8846 8845
34447 Upstairs last five completed: 9151 9150 9149 9148 9147
34448 CLIENT: Up:0 ds:302 act:2
34449 ----------------------------------------------------------------
34450 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
34451 States: Active Active Offline
34452 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
34453 8850 Acked Read 7 Done Done Sent false
34454 8851 Acked Read 8 Done Done Sent false
34455 8852 Acked Read 3 Done Done Sent false
34456 8853 Acked Write 10 Done Done Sent false
34457 8854 Acked Write 10 Done Done Sent false
34458 8855 Acked Flush 0 Done Done Sent false
34459 8856 Acked Read 3 Done Done Sent false
34460 8857 Acked Flush 0 Done Done Sent false
34461 8858 Acked Read 1 Done Done Sent false
34462 8859 Acked Write 2 Done Done Sent false
34463 8860 Acked Read 7 Done Done Sent false
34464 8861 Acked Write 7 Done Done Sent false
34465 8862 Acked Flush 0 Done Done Sent false
34466 8863 Acked Write 6 Done Done Sent false
34467 8864 Acked Read 7 Done Done Sent false
34468 8865 Acked Write 2 Done Done Sent false
34469 8866 Acked Write 2 Done Done Sent false
34470 8867 Acked Read 7 Done Done Sent false
34471 8868 Acked Read 3 Done Done Sent false
34472 8869 Acked Write 5 Done Done Sent false
34473 8870 Acked Read 9 Done Done Sent false
34474 8871 Acked Read 9 Done Done Sent false
34475 8872 Acked Read 2 Done Done Sent false
34476 8873 Acked Write 2 Done Done Sent false
34477 8874 Acked Write 7 Done Done Sent false
34478 8875 Acked Read 3 Done Done Sent false
34479 8876 Acked Read 5 Done Done Sent false
34480 8877 Acked Read 6 Done Done Sent false
34481 8878 Acked Write 4 Done Done Sent false
34482 8879 Acked Read 6 Done Done Sent false
34483 8880 Acked Read 5 Done Done Sent false
34484 8881 Acked Write 8 Done Done Sent false
34485 8882 Acked Read 10 Done Done Sent false
34486 8883 Acked Flush 0 Done Done Sent false
34487 8884 Acked Read 10 Done Done Sent false
34488 8885 Acked Write 3 Done Done Sent false
34489 8886 Acked Read 2 Done Done Sent false
34490 8887 Acked Read 1 Done Done Sent false
34491 8888 Acked Write 1 Done Done Sent false
34492 8889 Acked Read 10 Done Done Sent false
34493 8890 Acked Read 6 Done Done Sent false
34494 8891 Acked Read 8 Done Done Sent false
34495 8892 Acked Write 9 Done Done Sent false
34496 8893 Acked Write 6 Done Done Sent false
34497 8894 Acked Read 3 Done Done Sent false
34498 8895 Acked Write 5 Done Done Sent false
34499 8896 Acked Read 1 Done Done Sent false
34500 8897 Acked Read 4 Done Done Sent false
34501 8898 Acked Write 3 Done Done Sent false
34502 8899 Acked Write 1 Done Done Sent false
34503 8900 Acked Read 10 Done Done Sent false
34504 8901 Acked Write 1 Done Done Sent false
34505 8902 Acked Write 10 Done Done Sent false
34506 8903 Acked Read 4 Done Done Sent false
34507 8904 Acked Read 3 Done Done Sent false
34508 8905 Acked Read 3 Done Done Sent false
34509 8906 Acked Read 7 Done Done Sent false
34510 8907 Acked Read 6 Done Done Sent false
34511 8908 Acked Read 10 Done Done Sent false
34512 8909 Acked Read 8 Done Done Sent false
34513 8910 Acked Read 7 Done Done Sent false
34514 8911 Acked Write 1 Done Done Sent false
34515 8912 Acked Read 3 Done Done Sent false
34516 8913 Acked Write 8 Done Done Sent false
34517 8914 Acked Read 2 Done Done Sent false
34518 8915 Acked Read 2 Done Done Sent false
34519 8916 Acked Read 5 Done Done Sent false
34520 8917 Acked Read 8 Done Done Sent false
34521 8918 Acked Write 9 Done Done Sent false
34522 8919 Acked Read 8 Done Done Sent false
34523 8920 Acked Flush 0 Done Done Sent false
34524 8921 Acked Write 5 Done Done Sent false
34525 8922 Acked Read 4 Done Done Sent false
34526 8923 Acked Write 1 Done Done Sent false
34527 8924 Acked Read 8 Done Done Sent false
34528 8925 Acked Write 9 Done Done Sent false
34529 8926 Acked Write 3 Done Done Sent false
34530 8927 Acked Write 7 Done Done Sent false
34531 8928 Acked Write 5 Done Done Sent false
34532 8929 Acked Read 10 Done Done Sent false
34533 8930 Acked Read 5 Done Done Sent false
34534 8931 Acked Write 3 Done Done Sent false
34535 8932 Acked Read 3 Done Done Sent false
34536 8933 Acked Write 4 Done Done Sent false
34537 8934 Acked Write 9 Done Done Sent false
34538 8935 Acked Write 9 Done Done Sent false
34539 8936 Acked Read 7 Done Done Sent false
34540 8937 Acked Write 5 Done Done Sent false
34541 8938 Acked Read 8 Done Done Sent false
34542 8939 Acked Read 7 Done Done Sent false
34543 8940 Acked Write 9 Done Done Sent false
34544 8941 Acked Write 2 Done Done Sent false
34545 8942 Acked Write 5 Done Done Sent false
34546 8943 Acked Read 2 Done Done Sent false
34547 8944 Acked Write 3 Done Done Sent false
34548 8945 Acked Read 2 Done Done Sent false
34549 8946 Acked Read 2 Done Done Sent false
34550 8947 Acked Write 6 Done Done Sent false
34551 8948 Acked Write 6 Done Done Sent false
34552 8949 Acked Read 10 Done Done Sent false
34553 8950 Acked Write 6 Done Done Sent false
34554 8951 Acked Read 1 Done Done Sent false
34555 8952 Acked Read 9 Done Done Sent false
34556 8953 Acked Read 8 Done Done Sent false
34557 8954 Acked Read 1 Done Done Sent false
34558 8955 Acked Read 1 Done Done Sent false
34559 8956 Acked Write 1 Done Done Sent false
34560 8957 Acked Flush 0 Done Done Sent false
34561 8958 Acked Flush 0 Done Done Sent false
34562 8959 Acked Write 8 Done Done Sent false
34563 8960 Acked Flush 0 Done Done Sent false
34564 8961 Acked Write 4 Done Done Sent false
34565 8962 Acked Read 1 Done Done Sent false
34566 8963 Acked Read 3 Done Done Sent false
34567 8964 Acked Read 10 Done Done Sent false
34568 8965 Acked Flush 0 Done Done Sent false
34569 8966 Acked Read 10 Done Done Sent false
34570 8967 Acked Read 2 Done Done Sent false
34571 8968 Acked Write 7 Done Done Sent false
34572 8969 Acked Read 2 Done Done Sent false
34573 8970 Acked Write 3 Done Done Sent false
34574 8971 Acked Write 1 Done Done Sent false
34575 8972 Acked Write 4 Done Done Sent false
34576 8973 Acked Write 2 Done Done Sent false
34577 8974 Acked Read 4 Done Done Sent false
34578 8975 Acked Write 2 Done Done Sent false
34579 8976 Acked Write 10 Done Done Sent false
34580 8977 Acked Read 5 Done Done Sent false
34581 8978 Acked Read 3 Done Done Sent false
34582 8979 Acked Read 3 Done Done Sent false
34583 8980 Acked Read 6 Done Done Sent false
34584 8981 Acked Read 1 Done Done Sent false
34585 8982 Acked Write 10 Done Done Sent false
34586 8983 Acked Read 1 Done Done Sent false
34587 8984 Acked Write 3 Done Done Sent false
34588 8985 Acked Read 1 Done Done Sent false
34589 8986 Acked Read 9 Done Done Sent false
34590 8987 Acked Read 9 Done Done Sent false
34591 8988 Acked Write 6 Done Done Sent false
34592 8989 Acked Write 6 Done Done Sent false
34593 8990 Acked Read 8 Done Done Sent false
34594 8991 Acked Write 5 Done Done Sent false
34595 8992 Acked Read 1 Done Done Sent false
34596 8993 Acked Read 1 Done Done Sent false
34597 8994 Acked Read 9 Done Done Sent false
34598 8995 Acked Read 7 Done Done Sent false
34599 8996 Acked Read 1 Done Done Sent false
34600 8997 Acked Read 5 Done Done Sent false
34601 8998 Acked Read 5 Done Done Sent false
34602 8999 Acked Read 5 Done Done Sent false
34603 9000 Acked Write 7 Done Done Sent false
34604 9001 Acked Read 5 Done Done Sent false
34605 9002 Acked Read 5 Done Done Sent false
34606 9003 Acked Write 3 Done Done Sent false
34607 9004 Acked Write 4 Done Done Sent false
34608 9005 Acked Write 5 Done Done Sent false
34609 9006 Acked Write 9 Done Done Sent false
34610 9007 Acked Flush 0 Done Done Sent false
34611 9008 Acked Read 7 Done Done Sent false
34612 9009 Acked Write 10 Done Done Sent false
34613 9010 Acked Read 7 Done Done Sent false
34614 9011 Acked Read 2 Done Done Sent false
34615 9012 Acked Flush 0 Done Done Sent false
34616 9013 Acked Write 1 Done Done Sent false
34617 9014 Acked Read 9 Done Done Sent false
34618 9015 Acked Read 1 Done Done Sent false
34619 9016 Acked Write 8 Done Done Sent false
34620 9017 Acked Read 6 Done Done Sent false
34621 9018 Acked Write 5 Done Done Sent false
34622 9019 Acked Flush 0 Done Done Sent false
34623 9020 Acked Read 1 Done Done Sent false
34624 9021 Acked Write 2 Done Done Sent false
34625 9022 Acked Write 9 Done Done Sent false
34626 9023 Acked Read 5 Done Done Sent false
34627 9024 Acked Read 3 Done Done Sent false
34628 9025 Acked Read 6 Done Done Sent false
34629 9026 Acked Read 8 Done Done Sent false
34630 9027 Acked Write 2 Done Done Sent false
34631 9028 Acked Read 7 Done Done Sent false
34632 9029 Acked Read 4 Done Done Sent false
34633 9030 Acked Write 2 Done Done Sent false
34634 9031 Acked Read 3 Done Done Sent false
34635 9032 Acked Read 10 Done Done Sent false
34636 9033 Acked Write 5 Done Done Sent false
34637 9034 Acked Flush 0 Done Done Sent false
34638 9035 Acked Write 6 Done Done Sent false
34639 9036 Acked Read 1 Done Done Sent false
34640 9037 Acked Read 2 Done Done Sent false
34641 9038 Acked Read 4 Done Done Sent false
34642 9039 Acked Read 9 Done Done Sent false
34643 9040 Acked Flush 0 Done Done Sent false
34644 9041 Acked Write 7 Done Done Sent false
34645 9042 Acked Write 8 Done Done Sent false
34646 9043 Acked Flush 0 Done Done Sent false
34647 9044 Acked Read 6 Done Done Sent false
34648 9045 Acked Flush 0 Done Done Sent false
34649 9046 Acked Write 6 Done Done Sent false
34650 9047 Acked Read 5 Done Done Sent false
34651 9048 Acked Read 5 Done Done Sent false
34652 9049 Acked Write 8 Done Done Sent false
34653 9050 Acked Read 10 Done Done Sent false
34654 9051 Acked Read 6 Done Done Sent false
34655 9052 Acked Read 9 Done Done Sent false
34656 9053 Acked Read 10 Done Done Sent false
34657 9054 Acked Read 7 Done Done Sent false
34658 9055 Acked Write 6 Done Done Sent false
34659 9056 Acked Read 10 Done Done Sent false
34660 9057 Acked Write 5 Done Done Sent false
34661 9058 Acked Write 7 Done Done Sent false
34662 9059 Acked Read 1 Done Done Sent false
34663 9060 Acked Read 8 Done Done Sent false
34664 9061 Acked Write 10 Done Done Sent false
34665 9062 Acked Write 2 Done Done Sent false
34666 9063 Acked Read 9 Done Done Sent false
34667 9064 Acked Write 8 Done Done Sent false
34668 9065 Acked Write 5 Done Done Sent false
34669 9066 Acked Read 6 Done Done Sent false
34670 9067 Acked Read 5 Done Done Sent false
34671 9068 Acked Read 8 Done Done Sent false
34672 9069 Acked Read 6 Done Done Sent false
34673 9070 Acked Write 8 Done Done Sent false
34674 9071 Acked Read 10 Done Done Sent false
34675 9072 Acked Read 6 Done Done Sent false
34676 9073 Acked Read 2 Done Done Sent false
34677 9074 Acked Read 4 Done Done Sent false
34678 9075 Acked Read 10 Done Done Sent false
34679 9076 Acked Read 9 Done Done Sent false
34680 9077 Acked Read 3 Done Done Sent false
34681 9078 Acked Write 9 Done Done Sent false
34682 9079 Acked Flush 0 Done Done Sent false
34683 9080 Acked Write 2 Done Done Sent false
34684 9081 Acked Write 1 Done Done Sent false
34685 9082 Acked Write 10 Done Done Sent false
34686 9083 Acked Read 6 Done Done Sent false
34687 9084 Acked Write 1 Done Done Sent false
34688 9085 Acked Read 3 Done Done Sent false
34689 9086 Acked Write 2 Done Done Sent false
34690 9087 Acked Write 4 Done Done Sent false
34691 9088 Acked Write 5 Done Done Sent false
34692 9089 Acked Write 2 Done Done Sent false
34693 9090 Acked Read 3 Done Done Sent false
34694 9091 Acked Write 8 Done Done Sent false
34695 9092 Acked Read 5 Done Done Sent false
34696 9093 Acked Read 4 Done Done Sent false
34697 9094 Acked Read 8 Done Done Sent false
34698 9095 Acked Read 6 Done Done Sent false
34699 9096 Acked Read 5 Done Done Sent false
34700 9097 Acked Flush 0 Done Done Sent false
34701 9098 Acked Write 7 Done Done Sent false
34702 9099 Acked Write 6 Done Done Sent false
34703 9100 Acked Read 10 Done Done Sent false
34704 9101 Acked Write 9 Done Done Sent false
34705 9102 Acked Read 9 Done Done Sent false
34706 9103 Acked Read 7 Done Done Sent false
34707 9104 Acked Read 4 Done Done Sent false
34708 9105 Acked Read 10 Done Done Sent false
34709 9106 Acked Write 4 Done Done Sent false
34710 9107 Acked Flush 0 Done Done Sent false
34711 9108 Acked Read 10 Done Done Sent false
34712 9109 Acked Write 4 Done Done Sent false
34713 9110 Acked Flush 0 Done Done Sent false
34714 9111 Acked Write 9 Done Done Sent false
34715 9112 Acked Read 10 Done Done Sent false
34716 9113 Acked Write 7 Done Done Sent false
34717 9114 Acked Read 3 Done Done Sent false
34718 9115 Acked Read 8 Done Done Sent false
34719 9116 Acked Read 7 Done Done Sent false
34720 9117 Acked Read 8 Done Done Sent false
34721 9118 Acked Read 7 Done Done Sent false
34722 9119 Acked Write 10 Done Done Sent false
34723 9120 Acked Read 2 Done Done Sent false
34724 9121 Acked Flush 0 Done Done Sent false
34725 9122 Acked Read 9 Done Done Sent false
34726 9123 Acked Write 8 Done Done Sent false
34727 9124 Acked Read 9 Done Done Sent false
34728 9125 Acked Read 7 Done Done Sent false
34729 9126 Acked Write 2 Done Done Sent false
34730 9127 Acked Read 6 Done Done Sent false
34731 9128 Acked Write 10 Done Done Sent false
34732 9129 Acked Write 9 Done Done Sent false
34733 9130 Acked Write 1 Done Done Sent false
34734 9131 Acked Flush 0 Done Done Sent false
34735 9132 Acked Flush 0 Done Done Sent false
34736 9133 Acked Write 8 Done Done Sent false
34737 9134 Acked Read 10 Done Done Sent false
34738 9135 Acked Read 4 Done Done Sent false
34739 9136 Acked Write 10 Done Done Sent false
34740 9137 Acked Read 1 Done Done Sent false
34741 9138 Acked Read 8 Done Done Sent false
34742 9139 Acked Write 6 Done Done Sent false
34743 9140 Acked Read 1 Done Done Sent false
34744 9141 Acked Flush 0 Done Done Sent false
34745 9142 Acked Write 10 Done Done Sent false
34746 9143 Acked Read 3 Done Done Sent false
34747 9144 Acked Write 5 Done Done Sent false
34748 9145 Acked Read 9 Done Done Sent false
34749 9146 Acked Read 4 Done Done Sent false
34750 9147 Acked Read 1 Done Done Sent false
34751 9148 Acked Flush 0 Done Done Sent false
34752 9149 Acked Read 8 Done Done Sent false
34753 9150 Acked Read 8 Done Done Sent false
34754 9151 Acked Flush 0 Done Done Sent false
34755 STATES DS:0 DS:1 DS:2 TOTAL
34756 Sent 0 0 302 302
34757 Done 302 302 0 604
34758 Skipped 0 0 0 0
34759 Error 0 0 0 0
34760 Last Flush: 9151 9151 8849
34761 Downstairs last five completed: 8849 8848 8847 8846 8845
34762 Upstairs last five completed: 9151 9150 9149 9148 9147
34763 CLIENT: Up:0 ds:302 act:2
34764 [8830] got start action
34765 [8830] I am going to restart
34766 Starting downstairs at port 8830
34767 Make output file at "/tmp/dsc/downstairs-8830.txt"
34768 [8830][2] reports Starting
34769 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1232)
34770 [8830][2] reports Running
347712024-11-15T02:04:46.303ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
347722024-11-15T02:04:46.303ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:62664) = io task client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
347732024-11-15T02:04:46.304ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
347742024-11-15T02:04:46.305ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
347752024-11-15T02:04:46.305ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
347762024-11-15T02:04:46.305ZINFOcrucible: send last flush ID to this DS: 8849 = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
347772024-11-15T02:04:46.305ZINFOcrucible: Replied this last flush ID: 8849 = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
347782024-11-15T02:04:46.305ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
347792024-11-15T02:04:46.305ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
347802024-11-15T02:04:46.305ZINFOcrucible: [2] client re-new 302 jobs since flush 8849 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
347812024-11-15T02:04:46.305ZINFOcrucible: [2] Replayed 302 jobs since flush: 8849 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
34782 ----------------------------------------------------------------
34783 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
34784 Downstairs last five completed: 9151 9150 9149 9148 9147
34785 Upstairs last five completed: 9151 9150 9149 9148 9147
34786 CLIENT: Up:0 ds:0 act:3
34787 Replay: All jobs finished
34788 Main task has work to do, go find it
34789 got dsc Stop(0)
34790 stop 0
34791 [8810] Got stop action so:false kr:false
34792 Nov 15 02:04:46.563 INFO request completed, latency_us: 368, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 13f5a8cc-8c4d-45fb-9698-82b3322ebe90, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
347932024-11-15T02:04:46.564ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
347942024-11-15T02:04:46.564ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
347952024-11-15T02:04:46.564ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
347962024-11-15T02:04:46.564ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
347972024-11-15T02:04:46.564ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
347982024-11-15T02:04:46.564ZINFOcrucible: client task is exiting = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
34799 Nov 15 02:04:46.564 INFO request completed, latency_us: 140, response_code: 200, uri: /state/cid/0, method: GET, req_id: 93fbfcb2-34bc-42b8-8028-19859e056426, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
34800 [8810] Exited with: ExitStatus(unix_wait_status(9))
34801 [8810][0] reports Exit
34802 Nov 15 02:04:50.566 INFO request completed, latency_us: 326, response_code: 200, uri: /state/cid/0, method: GET, req_id: cffbb90e-0111-41ba-a5e5-9141bdab20d7, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
34803 001/300 Read block 4497 len 16384
34804 002/300 Flush
34805 003/300 Write block 2856 len 20480 data: 7 5 5 7 5
34806 004/300 Write block 4547 len 8192 data: 2 3
34807 005/300 Write block 3656 len 8192 data: 7 7
34808 006/300 Read block 4451 len 32768
34809 007/300 Write block 4351 len 32768 data: 6 6 5 4 5 5 4 4
34810 008/300 Read block 125 len 20480
34811 009/300 Flush
34812 010/300 Read block 4333 len 8192
34813 011/300 Read block 4720 len 28672
34814 012/300 Read block 4268 len 4096
34815 013/300 Write block 3548 len 32768 data: 7 7 7 7 9 8 5 5
34816 014/300 Read block 3222 len 12288
34817 015/300 Read block 4257 len 20480
34818 016/300 Write block 4150 len 28672 data: 6 7 5 5 4 5 5
34819 017/300 Read block 1347 len 28672
34820 018/300 Read block 3386 len 28672
34821 019/300 Write block 131 len 4096 data: 3
34822 020/300 Write block 4111 len 12288 data: 4 4 4
34823 021/300 Read block 2764 len 36864
34824 022/300 Read block 2952 len 4096
34825 023/300 Flush
34826 024/300 Write block 1257 len 32768 data: 5 5 4 4 4 4 3 3
34827 025/300 Read block 65 len 8192
34828 026/300 Write block 79 len 40960 data: 9 9 10 9 8 8 8 7 8 8
34829 027/300 Read block 3716 len 12288
34830 028/300 Flush
34831 029/300 Read block 4839 len 20480
34832 030/300 Read block 169 len 16384
34833 031/300 Write block 169 len 16384 data: 5 6 5 7
34834 032/300 Read block 4710 len 36864
34835 033/300 Read block 961 len 20480
34836 034/300 Write block 3460 len 36864 data: 5 5 2 4 5 6 5 6 7
34837 035/300 Write block 4991 len 16384 data: 3 3 3 3
34838 036/300 Write block 3922 len 28672 data: 7 9 9 7 6 7 6
34839 037/300 Read block 2872 len 28672
34840 038/300 Write block 4812 len 4096 data: 4
34841 039/300 Read block 2687 len 8192
34842 040/300 Read block 3038 len 32768
34843 041/300 Read block 2636 len 8192
34844 042/300 Write block 2065 len 24576 data: 5 4 4 4 5 4
34845 043/300 Read block 341 len 4096
34846 044/300 Flush
34847 045/300 Write block 4376 len 12288 data: 3 2 3
34848 046/300 Write block 1646 len 16384 data: 6 7 6 5
34849 047/300 Flush
34850 048/300 Read block 1707 len 16384
34851 049/300 Read block 2005 len 20480
34852 050/300 Read block 274 len 40960
34853 051/300 Read block 2817 len 28672
34854 052/300 Read block 1872 len 24576
34855 053/300 Read block 3033 len 8192
34856 054/300 Read block 576 len 12288
34857 055/300 Read block 3364 len 8192
34858 056/300 Write block 4383 len 4096 data: 4
34859 057/300 Flush
34860 058/300 Write block 4907 len 28672 data: 3 3 4 4 4 4 5
34861 059/300 Read block 1737 len 24576
34862 060/300 Write block 3136 len 16384 data: 6 4 5 5
34863 061/300 Flush
34864 062/300 Read block 290 len 32768
34865 063/300 Read block 559 len 40960
34866 064/300 Write block 2291 len 28672 data: 3 4 3 5 4 4 5
34867 065/300 Read block 4621 len 32768
34868 066/300 Write block 3606 len 28672 data: 4 3 3 3 2 2 2
34869 067/300 Flush
34870 068/300 Read block 2016 len 16384
34871 069/300 Read block 2560 len 28672
34872 070/300 Read block 3876 len 24576
34873 071/300 Write block 3044 len 32768 data: 4 3 5 4 4 6 6 7
34874 072/300 Write block 1660 len 28672 data: 8 9 9 11 11 10 10
34875 073/300 Read block 2035 len 4096
34876 074/300 Read block 910 len 12288
34877 075/300 Read block 315 len 36864
34878 076/300 Read block 1267 len 36864
34879 077/300 Read block 2830 len 36864
34880 078/300 Flush
34881 079/300 Read block 2155 len 4096
34882 080/300 Read block 3929 len 28672
34883 081/300 Read block 3969 len 32768
34884 082/300 Write block 1806 len 32768 data: 8 7 4 4 3 4 4 3
34885 083/300 Read block 3549 len 28672
34886 084/300 Read block 4124 len 4096
34887 085/300 Write block 4779 len 20480 data: 7 7 7 8 7
34888 086/300 Read block 2494 len 28672
34889 087/300 Read block 4538 len 20480
34890 088/300 Write block 4158 len 8192 data: 6 5
34891 089/300 Read block 2210 len 36864
34892 090/300 Read block 682 len 16384
34893 091/300 Read block 1821 len 28672
34894 092/300 Read block 1468 len 28672
34895 093/300 Read block 4276 len 4096
34896 094/300 Read block 3062 len 8192
34897 095/300 Flush
34898 096/300 Write block 534 len 12288 data: 10 9 7
34899 097/300 Write block 2067 len 28672 data: 5 5 6 5 4 4 5
34900 098/300 Write block 2485 len 36864 data: 5 5 4 5 4 6 8 8 6
34901 099/300 Read block 2312 len 32768
34902 100/300 Write block 790 len 40960 data: 9 8 6 6 7 8 7 7 8 6
34903 101/300 Read block 1370 len 16384
34904 102/300 Write block 1241 len 24576 data: 5 4 6 6 7 9
34905 103/300 Read block 4495 len 36864
34906 104/300 Write block 4437 len 4096 data: 5
34907 105/300 Read block 1703 len 32768
34908 106/300 Write block 3207 len 8192 data: 4 3
34909 107/300 Write block 2219 len 4096 data: 5
34910 108/300 Write block 759 len 40960 data: 7 5 5 5 3 4 3 3 5 5
34911 109/300 Read block 1029 len 28672
34912 110/300 Read block 4297 len 4096
34913 111/300 Read block 716 len 8192
34914 112/300 Read block 3204 len 40960
34915 113/300 Read block 1744 len 28672
34916 114/300 Read block 3450 len 40960
34917 115/300 Write block 1819 len 28672 data: 5 6 7 7 6 6 6
34918 116/300 Write block 3475 len 28672 data: 5 5 4 4 4 5 3
34919 117/300 Write block 3140 len 28672 data: 7 7 6 7 7 7 7
34920 118/300 Write block 2168 len 16384 data: 5 5 6 7
34921 119/300 Read block 782 len 40960
34922 120/300 Read block 4305 len 8192
34923 121/300 Write block 770 len 40960 data: 3 4 5 5 5 4 4 4 4 5
34924 122/300 Read block 1552 len 16384
34925 123/300 Read block 3316 len 16384
34926 124/300 Write block 3326 len 24576 data: 6 7 7 7 8 6
34927 125/300 Read block 1009 len 24576
34928 126/300 Write block 481 len 12288 data: 6 6 6
34929 127/300 Flush
34930 128/300 Write block 697 len 12288 data: 6 7 7
34931 129/300 Flush
34932 130/300 Read block 230 len 12288
34933 131/300 Read block 1645 len 8192
34934 132/300 Write block 4405 len 36864 data: 3 3 5 5 5 6 3 3 3
34935 133/300 Read block 2709 len 40960
34936 134/300 Write block 3774 len 40960 data: 7 8 7 6 6 6 6 7 7 5
34937 135/300 Read block 1043 len 20480
34938 136/300 Write block 146 len 36864 data: 2 3 3 4 4 4 4 5 4
34939 137/300 Read block 1321 len 16384
34940 138/300 Write block 4166 len 20480 data: 3 3 3 3 3
34941 139/300 Flush
34942 140/300 Read block 3288 len 4096
34943 141/300 Read block 3488 len 36864
34944 142/300 Write block 2255 len 36864 data: 8 7 7 6 5 5 5 7 7
34945 143/300 Read block 3146 len 32768
34946 144/300 Write block 3179 len 28672 data: 6 6 6 6 7 5 6
34947 145/300 Write block 1357 len 12288 data: 8 9 6
34948 146/300 Write block 3005 len 28672 data: 4 4 4 5 5 7 7
34949 147/300 Read block 2780 len 32768
34950 148/300 Write block 3835 len 20480 data: 6 8 8 8 9
34951 149/300 Flush
34952 150/300 Read block 605 len 36864
34953 151/300 Write block 2607 len 12288 data: 4 4 4
34954 152/300 Write block 2623 len 24576 data: 5 4 2 2 2 3
34955 153/300 Flush
34956 154/300 Read block 637 len 20480
34957 155/300 Read block 625 len 8192
34958 156/300 Write block 2399 len 40960 data: 5 6 7 9 11 10 10 8 7 6
34959 157/300 Flush
34960 158/300 Read block 2025 len 8192
34961 159/300 Flush
34962 160/300 Write block 443 len 8192 data: 5 5
34963 161/300 Write block 2661 len 8192 data: 5 4
34964 162/300 Write block 1714 len 4096 data: 3
34965 163/300 Read block 2925 len 8192
34966 164/300 Read block 68 len 40960
34967 165/300 Write block 2516 len 36864 data: 7 3 3 2 2 2 4 5 5
34968 166/300 Flush
34969 167/300 Read block 3083 len 12288
34970 168/300 Read block 340 len 36864
34971 169/300 Write block 661 len 32768 data: 8 7 8 6 7 5 9 7
34972 170/300 Write block 4917 len 20480 data: 5 7 7 9 10
34973 171/300 Read block 3029 len 8192
34974 172/300 Read block 3255 len 4096
34975 173/300 Flush
34976 174/300 Flush
34977 175/300 Flush
34978 176/300 Write block 418 len 24576 data: 7 7 6 6 6 5
34979 177/300 Read block 751 len 16384
34980 178/300 Read block 4081 len 36864
34981 179/300 Write block 1793 len 36864 data: 5 5 4 4 2 2 2 5 5
34982 180/300 Read block 1200 len 16384
34983 181/300 Write block 1969 len 20480 data: 4 3 3 3 5
34984 182/300 Write block 2820 len 24576 data: 7 8 8 8 8 7
34985 183/300 Write block 859 len 32768 data: 5 3 4 6 5 6 6 4
34986 184/300 Read block 944 len 4096
34987 185/300 Read block 3428 len 36864
34988 186/300 Write block 1556 len 28672 data: 4 5 7 7 8 9 10
34989 187/300 Write block 1127 len 36864 data: 4 4 5 6 6 4 3 3 3
34990 188/300 Read block 2836 len 40960
34991 189/300 Write block 121 len 32768 data: 7 6 6 6 5 4 3 3
34992 190/300 Write block 3358 len 8192 data: 4 4
34993 191/300 Read block 4850 len 20480
34994 192/300 Read block 1732 len 36864
34995 193/300 Flush
34996 194/300 Read block 2277 len 8192
34997 195/300 Read block 504 len 12288
34998 196/300 Write block 3263 len 8192 data: 6 6
34999 197/300 Flush
35000 198/300 Read block 1637 len 24576
35001 199/300 Write block 2883 len 20480 data: 5 4 5 7 8
35002 200/300 Read block 815 len 32768
35003 201/300 Flush
35004 202/300 Flush
35005 203/300 Write block 2012 len 8192 data: 5 4
35006 204/300 Read block 2034 len 16384
35007 205/300 Flush
35008 206/300 Write block 4454 len 16384 data: 5 5 7 7
35009 207/300 Write block 3502 len 4096 data: 5
35010 208/300 Read block 828 len 32768
35011 209/300 Read block 3961 len 12288
35012 210/300 Read block 3654 len 28672
35013 211/300 Write block 3939 len 20480 data: 9 8 6 6 6
35014 212/300 Write block 2013 len 32768 data: 5 4 5 5 5 5 4 6
35015 213/300 Write block 1492 len 12288 data: 6 4 3
35016 214/300 Write block 3980 len 32768 data: 7 6 4 3 3 3 4 6
35017 215/300 Write block 1994 len 24576 data: 4 5 4 4 5 3
35018 216/300 Read block 4174 len 8192
35019 217/300 Read block 3348 len 8192
35020 218/300 Read block 4534 len 36864
35021 219/300 Flush
35022 220/300 Flush
35023 221/300 Read block 4732 len 12288
35024 222/300 Read block 1827 len 40960
35025 223/300 Read block 3901 len 16384
35026 224/300 Read block 6 len 28672
35027 225/300 Flush
35028 226/300 Read block 2685 len 40960
35029 227/300 Read block 3214 len 32768
35030 228/300 Read block 4699 len 32768
35031 229/300 Read block 2290 len 20480
35032 230/300 Write block 4909 len 24576 data: 5 5 5 5 6 3
35033 231/300 Write block 4545 len 40960 data: 3 3 3 4 4 4 5 4 4 4
35034 232/300 Write block 3480 len 36864 data: 6 4 3 4 4 6 6 6 7
35035 233/300 Read block 4303 len 36864
35036 234/300 Read block 4926 len 24576
35037 235/300 Write block 3538 len 40960 data: 6 5 4 3 3 3 4 6 6 5
35038 236/300 Read block 1384 len 24576
35039 237/300 Flush
35040 238/300 Write block 3267 len 40960 data: 5 6 5 6 7 7 8 6 6 6
35041 239/300 Write block 654 len 32768 data: 9 7 6 5 4 5 5 9
35042 240/300 Read block 2241 len 16384
35043 241/300 Write block 166 len 12288 data: 6 6 5
35044 242/300 Write block 1746 len 8192 data: 8 9
35045 243/300 Write block 3902 len 4096 data: 12
35046 244/300 Read block 500 len 28672
35047 245/300 Read block 973 len 16384
35048 246/300 Write block 1689 len 8192 data: 3 4
35049 247/300 Flush
35050 248/300 Read block 4114 len 40960
35051 249/300 Read block 1249 len 28672
35052 250/300 Write block 3020 len 4096 data: 4
35053 251/300 Write block 4818 len 36864 data: 6 5 5 4 5 5 5 6 7
35054 252/300 Write block 2050 len 8192 data: 3 3
35055 253/300 Write block 2522 len 28672 data: 5 6 6 9 8 11 13
35056 254/300 Write block 2637 len 28672 data: 4 4 4 4 2 4 5
35057 255/300 Read block 2582 len 36864
35058 256/300 Read block 46 len 12288
35059 257/300 Write block 2038 len 4096 data: 3
35060 258/300 Read block 2624 len 32768
35061 259/300 Read block 3039 len 24576
35062 260/300 Write block 3485 len 28672 data: 7 7 7 8 8 9 7
35063 261/300 Write block 4874 len 24576 data: 6 9 9 8 6 4
35064 262/300 Flush
35065 263/300 Flush
35066 264/300 Read block 2013 len 8192
35067 265/300 Read block 284 len 20480
35068 266/300 Write block 4447 len 8192 data: 7 7
35069 267/300 Write block 1288 len 12288 data: 10 8 9
35070 268/300 Write block 4010 len 4096 data: 6
35071 269/300 Read block 1330 len 36864
35072 270/300 Write block 2867 len 4096 data: 4
35073 271/300 Write block 331 len 16384 data: 6 6 8 6
35074 272/300 Write block 592 len 4096 data: 9
35075 273/300 Read block 866 len 40960
35076 274/300 Read block 582 len 36864
35077 275/300 Flush
35078 276/300 Write block 1361 len 28672 data: 5 6 7 6 8 7 6
35079 277/300 Read block 2927 len 4096
35080 278/300 Flush
35081 279/300 Read block 49 len 28672
35082 280/300 Write block 3253 len 8192 data: 12 11
35083 281/300 Write block 270 len 36864 data: 3 3 3 5 5 5 5 5 5
35084 282/300 Write block 4580 len 12288 data: 5 6 5
35085 283/300 Read block 3661 len 24576
35086 284/300 Read block 2373 len 20480
35087 285/300 Write block 689 len 16384 data: 5 6 5 5
35088 286/300 Read block 3550 len 8192
35089 287/300 Read block 455 len 12288
35090 288/300 Read block 3156 len 32768
35091 289/300 Read block 2815 len 36864
35092 290/300 Read block 1783 len 20480
35093 291/300 Read block 3568 len 36864
35094 292/300 Write block 4593 len 16384 data: 7 6 6 6
35095 293/300 Flush
35096 294/300 Read block 3783 len 24576
35097 295/300 Read block 3135 len 20480
35098 296/300 Read block 4387 len 24576
35099 297/300 Read block 781 len 24576
35100 298/300 Read block 2059 len 12288
35101 299/300 Write block 1 len 32768 data: 2 2 3 5 4 7 7 6
35102 300/300 Read block 2662 len 32768
35103 301/300 Read block 4709 len 28672
35104 Main task has work to do, go find it
35105 got dsc Start(0)
35106 start 0
35107 Nov 15 02:04:50.803 INFO request completed, latency_us: 212, response_code: 204, uri: /start/cid/0, method: POST, req_id: 3623e2ae-95a4-4125-95af-ed215e2b0612, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
35108 Replay: started 0, returned:Ok(())
35109 ----------------------------------------------------------------
35110 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
35111 States: Offline Active Active
35112 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
35113 9152 Acked Read 4 Sent Done Done false
35114 9153 Acked Flush 0 Sent Done Done false
35115 9154 Acked Write 5 Sent Done Done false
35116 9155 Acked Write 2 Sent Done Done false
35117 9156 Acked Write 2 Sent Done Done false
35118 9157 Acked Read 8 Sent Done Done false
35119 9158 Acked Write 8 Sent Done Done false
35120 9159 Acked Read 5 Sent Done Done false
35121 9160 Acked Flush 0 Sent Done Done false
35122 9161 Acked Read 2 Sent Done Done false
35123 9162 Acked Read 7 Sent Done Done false
35124 9163 Acked Read 1 Sent Done Done false
35125 9164 Acked Write 8 Sent Done Done false
35126 9165 Acked Read 3 Sent Done Done false
35127 9166 Acked Read 5 Sent Done Done false
35128 9167 Acked Write 7 Sent Done Done false
35129 9168 Acked Read 7 Sent Done Done false
35130 9169 Acked Read 7 Sent Done Done false
35131 9170 Acked Write 1 Sent Done Done false
35132 9171 Acked Write 3 Sent Done Done false
35133 9172 Acked Read 9 Sent Done Done false
35134 9173 Acked Read 1 Sent Done Done false
35135 9174 Acked Flush 0 Sent Done Done false
35136 9175 Acked Write 8 Sent Done Done false
35137 9176 Acked Read 2 Sent Done Done false
35138 9177 Acked Write 10 Sent Done Done false
35139 9178 Acked Read 3 Sent Done Done false
35140 9179 Acked Flush 0 Sent Done Done false
35141 9180 Acked Read 5 Sent Done Done false
35142 9181 Acked Read 4 Sent Done Done false
35143 9182 Acked Write 4 Sent Done Done false
35144 9183 Acked Read 9 Sent Done Done false
35145 9184 Acked Read 5 Sent Done Done false
35146 9185 Acked Write 9 Sent Done Done false
35147 9186 Acked Write 4 Sent Done Done false
35148 9187 Acked Write 7 Sent Done Done false
35149 9188 Acked Read 7 Sent Done Done false
35150 9189 Acked Write 1 Sent Done Done false
35151 9190 Acked Read 2 Sent Done Done false
35152 9191 Acked Read 8 Sent Done Done false
35153 9192 Acked Read 2 Sent Done Done false
35154 9193 Acked Write 6 Sent Done Done false
35155 9194 Acked Read 1 Sent Done Done false
35156 9195 Acked Flush 0 Sent Done Done false
35157 9196 Acked Write 3 Sent Done Done false
35158 9197 Acked Write 4 Sent Done Done false
35159 9198 Acked Flush 0 Sent Done Done false
35160 9199 Acked Read 4 Sent Done Done false
35161 9200 Acked Read 5 Sent Done Done false
35162 9201 Acked Read 10 Sent Done Done false
35163 9202 Acked Read 7 Sent Done Done false
35164 9203 Acked Read 6 Sent Done Done false
35165 9204 Acked Read 2 Sent Done Done false
35166 9205 Acked Read 3 Sent Done Done false
35167 9206 Acked Read 2 Sent Done Done false
35168 9207 Acked Write 1 Sent Done Done false
35169 9208 Acked Flush 0 Sent Done Done false
35170 9209 Acked Write 7 Sent Done Done false
35171 9210 Acked Read 6 Sent Done Done false
35172 9211 Acked Write 4 Sent Done Done false
35173 9212 Acked Flush 0 Sent Done Done false
35174 9213 Acked Read 8 Sent Done Done false
35175 9214 Acked Read 10 Sent Done Done false
35176 9215 Acked Write 7 Sent Done Done false
35177 9216 Acked Read 8 Sent Done Done false
35178 9217 Acked Write 7 Sent Done Done false
35179 9218 Acked Flush 0 Sent Done Done false
35180 9219 Acked Read 4 Sent Done Done false
35181 9220 Acked Read 7 Sent Done Done false
35182 9221 Acked Read 6 Sent Done Done false
35183 9222 Acked Write 8 Sent Done Done false
35184 9223 Acked Write 7 Sent Done Done false
35185 9224 Acked Read 1 Sent Done Done false
35186 9225 Acked Read 3 Sent Done Done false
35187 9226 Acked Read 9 Sent Done Done false
35188 9227 Acked Read 9 Sent Done Done false
35189 9228 Acked Read 9 Sent Done Done false
35190 9229 Acked Flush 0 Sent Done Done false
35191 9230 Acked Read 1 Sent Done Done false
35192 9231 Acked Read 7 Sent Done Done false
35193 9232 Acked Read 8 Sent Done Done false
35194 9233 Acked Write 8 Sent Done Done false
35195 9234 Acked Read 7 Sent Done Done false
35196 9235 Acked Read 1 Sent Done Done false
35197 9236 Acked Write 5 Sent Done Done false
35198 9237 Acked Read 7 Sent Done Done false
35199 9238 Acked Read 5 Sent Done Done false
35200 9239 Acked Write 2 Sent Done Done false
35201 9240 Acked Read 9 Sent Done Done false
35202 9241 Acked Read 4 Sent Done Done false
35203 9242 Acked Read 7 Sent Done Done false
35204 9243 Acked Read 7 Sent Done Done false
35205 9244 Acked Read 1 Sent Done Done false
35206 9245 Acked Read 2 Sent Done Done false
35207 9246 Acked Flush 0 Sent Done Done false
35208 9247 Acked Write 3 Sent Done Done false
35209 9248 Acked Write 7 Sent Done Done false
35210 9249 Acked Write 9 Sent Done Done false
35211 9250 Acked Read 8 Sent Done Done false
35212 9251 Acked Write 10 Sent Done Done false
35213 9252 Acked Read 4 Sent Done Done false
35214 9253 Acked Write 6 Sent Done Done false
35215 9254 Acked Read 9 Sent Done Done false
35216 9255 Acked Write 1 Sent Done Done false
35217 9256 Acked Read 8 Sent Done Done false
35218 9257 Acked Write 2 Sent Done Done false
35219 9258 Acked Write 1 Sent Done Done false
35220 9259 Acked Write 10 Sent Done Done false
35221 9260 Acked Read 7 Sent Done Done false
35222 9261 Acked Read 1 Sent Done Done false
35223 9262 Acked Read 2 Sent Done Done false
35224 9263 Acked Read 10 Sent Done Done false
35225 9264 Acked Read 7 Sent Done Done false
35226 9265 Acked Read 10 Sent Done Done false
35227 9266 Acked Write 7 Sent Done Done false
35228 9267 Acked Write 7 Sent Done Done false
35229 9268 Acked Write 7 Sent Done Done false
35230 9269 Acked Write 4 Sent Done Done false
35231 9270 Acked Read 10 Sent Done Done false
35232 9271 Acked Read 2 Sent Done Done false
35233 9272 Acked Write 10 Sent Done Done false
35234 9273 Acked Read 4 Sent Done Done false
35235 9274 Acked Read 4 Sent Done Done false
35236 9275 Acked Write 6 Sent Done Done false
35237 9276 Acked Read 6 Sent Done Done false
35238 9277 Acked Write 3 Sent Done Done false
35239 9278 Acked Flush 0 Sent Done Done false
35240 9279 Acked Write 3 Sent Done Done false
35241 9280 Acked Flush 0 Sent Done Done false
35242 9281 Acked Read 3 Sent Done Done false
35243 9282 Acked Read 2 Sent Done Done false
35244 9283 Acked Write 9 Sent Done Done false
35245 9284 Acked Read 10 Sent Done Done false
35246 9285 Acked Write 10 Sent Done Done false
35247 9286 Acked Read 5 Sent Done Done false
35248 9287 Acked Write 9 Sent Done Done false
35249 9288 Acked Read 4 Sent Done Done false
35250 9289 Acked Write 5 Sent Done Done false
35251 9290 Acked Flush 0 Sent Done Done false
35252 9291 Acked Read 1 Sent Done Done false
35253 9292 Acked Read 9 Sent Done Done false
35254 9293 Acked Write 9 Sent Done Done false
35255 9294 Acked Read 8 Sent Done Done false
35256 9295 Acked Write 7 Sent Done Done false
35257 9296 Acked Write 3 Sent Done Done false
35258 9297 Acked Write 7 Sent Done Done false
35259 9298 Acked Read 8 Sent Done Done false
35260 9299 Acked Write 5 Sent Done Done false
35261 9300 Acked Flush 0 Sent Done Done false
35262 9301 Acked Read 9 Sent Done Done false
35263 9302 Acked Write 3 Sent Done Done false
35264 9303 Acked Write 6 Sent Done Done false
35265 9304 Acked Flush 0 Sent Done Done false
35266 9305 Acked Read 5 Sent Done Done false
35267 9306 Acked Read 2 Sent Done Done false
35268 9307 Acked Write 10 Sent Done Done false
35269 9308 Acked Flush 0 Sent Done Done false
35270 9309 Acked Read 2 Sent Done Done false
35271 9310 Acked Flush 0 Sent Done Done false
35272 9311 Acked Write 2 Sent Done Done false
35273 9312 Acked Write 2 Sent Done Done false
35274 9313 Acked Write 1 Sent Done Done false
35275 9314 Acked Read 2 Sent Done Done false
35276 9315 Acked Read 10 Sent Done Done false
35277 9316 Acked Write 9 Sent Done Done false
35278 9317 Acked Flush 0 Sent Done Done false
35279 9318 Acked Read 3 Sent Done Done false
35280 9319 Acked Read 9 Sent Done Done false
35281 9320 Acked Write 8 Sent Done Done false
35282 9321 Acked Write 5 Sent Done Done false
35283 9322 Acked Read 2 Sent Done Done false
35284 9323 Acked Read 1 Sent Done Done false
35285 9324 Acked Flush 0 Sent Done Done false
35286 9325 Acked Flush 0 Sent Done Done false
35287 9326 Acked Flush 0 Sent Done Done false
35288 9327 Acked Write 6 Sent Done Done false
35289 9328 Acked Read 4 Sent Done Done false
35290 9329 Acked Read 9 Sent Done Done false
35291 9330 Acked Write 9 Sent Done Done false
35292 9331 Acked Read 4 Sent Done Done false
35293 9332 Acked Write 5 Sent Done Done false
35294 9333 Acked Write 6 Sent Done Done false
35295 9334 Acked Write 8 Sent Done Done false
35296 9335 Acked Read 1 Sent Done Done false
35297 9336 Acked Read 9 Sent Done Done false
35298 9337 Acked Write 7 Sent Done Done false
35299 9338 Acked Write 9 Sent Done Done false
35300 9339 Acked Read 10 Sent Done Done false
35301 9340 Acked Write 8 Sent Done Done false
35302 9341 Acked Write 2 Sent Done Done false
35303 9342 Acked Read 5 Sent Done Done false
35304 9343 Acked Read 9 Sent Done Done false
35305 9344 Acked Flush 0 Sent Done Done false
35306 9345 Acked Read 2 Sent Done Done false
35307 9346 Acked Read 3 Sent Done Done false
35308 9347 Acked Write 2 Sent Done Done false
35309 9348 Acked Flush 0 Sent Done Done false
35310 9349 Acked Read 6 Sent Done Done false
35311 9350 Acked Write 5 Sent Done Done false
35312 9351 Acked Read 8 Sent Done Done false
35313 9352 Acked Flush 0 Sent Done Done false
35314 9353 Acked Flush 0 Sent Done Done false
35315 9354 Acked Write 2 Sent Done Done false
35316 9355 Acked Read 4 Sent Done Done false
35317 9356 Acked Flush 0 Sent Done Done false
35318 9357 Acked Write 4 Sent Done Done false
35319 9358 Acked Write 1 Sent Done Done false
35320 9359 Acked Read 8 Sent Done Done false
35321 9360 Acked Read 3 Sent Done Done false
35322 9361 Acked Read 7 Sent Done Done false
35323 9362 Acked Write 5 Sent Done Done false
35324 9363 Acked Write 8 Sent Done Done false
35325 9364 Acked Write 3 Sent Done Done false
35326 9365 Acked Write 8 Sent Done Done false
35327 9366 Acked Write 6 Sent Done Done false
35328 9367 Acked Read 2 Sent Done Done false
35329 9368 Acked Read 2 Sent Done Done false
35330 9369 Acked Read 9 Sent Done Done false
35331 9370 Acked Flush 0 Sent Done Done false
35332 9371 Acked Flush 0 Sent Done Done false
35333 9372 Acked Read 3 Sent Done Done false
35334 9373 Acked Read 10 Sent Done Done false
35335 9374 Acked Read 4 Sent Done Done false
35336 9375 Acked Read 7 Sent Done Done false
35337 9376 Acked Flush 0 Sent Done Done false
35338 9377 Acked Read 10 Sent Done Done false
35339 9378 Acked Read 8 Sent Done Done false
35340 9379 Acked Read 8 Sent Done Done false
35341 9380 Acked Read 5 Sent Done Done false
35342 9381 Acked Write 6 Sent Done Done false
35343 9382 Acked Write 10 Sent Done Done false
35344 9383 Acked Write 9 Sent Done Done false
35345 9384 Acked Read 9 Sent Done Done false
35346 9385 Acked Read 6 Sent Done Done false
35347 9386 Acked Write 10 Sent Done Done false
35348 9387 Acked Read 6 Sent Done Done false
35349 9388 Acked Flush 0 Sent Done Done false
35350 9389 Acked Write 10 Sent Done Done false
35351 9390 Acked Write 8 Sent Done Done false
35352 9391 Acked Read 4 Sent Done Done false
35353 9392 Acked Write 3 Sent Done Done false
35354 9393 Acked Write 2 Sent Done Done false
35355 9394 Acked Write 1 Sent Done Done false
35356 9395 Acked Read 7 Sent Done Done false
35357 9396 Acked Read 4 Sent Done Done false
35358 9397 Acked Write 2 Sent Done Done false
35359 9398 Acked Flush 0 Sent Done Done false
35360 9399 Acked Read 10 Sent Done Done false
35361 9400 Acked Read 7 Sent Done Done false
35362 9401 Acked Write 1 Sent Done Done false
35363 9402 Acked Write 9 Sent Done Done false
35364 9403 Acked Write 2 Sent Done Done false
35365 9404 Acked Write 7 Sent Done Done false
35366 9405 Acked Write 7 Sent Done Done false
35367 9406 Acked Read 9 Sent Done Done false
35368 9407 Acked Read 3 Sent Done Done false
35369 9408 Acked Write 1 Sent Done Done false
35370 9409 Acked Read 8 Sent Done Done false
35371 9410 Acked Read 6 Sent Done Done false
35372 9411 Acked Write 7 Sent Done Done false
35373 9412 Acked Write 6 Sent Done Done false
35374 9413 Acked Flush 0 Sent Done Done false
35375 9414 Acked Flush 0 Sent Done Done false
35376 9415 Acked Read 2 Sent Done Done false
35377 9416 Acked Read 5 Sent Done Done false
35378 9417 Acked Write 2 Sent Done Done false
35379 9418 Acked Write 3 Sent Done Done false
35380 9419 Acked Write 1 Sent Done Done false
35381 9420 Acked Read 9 Sent Done Done false
35382 9421 Acked Write 1 Sent Done Done false
35383 9422 Acked Write 4 Sent Done Done false
35384 9423 Acked Write 1 Sent Done Done false
35385 9424 Acked Read 10 Sent Done Done false
35386 9425 Acked Read 9 Sent Done Done false
35387 9426 Acked Flush 0 Sent Done Done false
35388 9427 Acked Write 7 Sent Done Done false
35389 9428 Acked Read 1 Sent Done Done false
35390 9429 Acked Flush 0 Sent Done Done false
35391 9430 Acked Read 7 Sent Done Done false
35392 9431 Acked Write 2 Sent Done Done false
35393 9432 Acked Write 9 Sent Done Done false
35394 9433 Acked Write 3 Sent Done Done false
35395 9434 Acked Read 6 Sent Done Done false
35396 9435 Acked Read 5 Sent Done Done false
35397 9436 Acked Write 4 Sent Done Done false
35398 9437 Acked Read 2 Sent Done Done false
35399 9438 Acked Read 3 Sent Done Done false
35400 9439 Acked Read 8 Sent Done Done false
35401 9440 Acked Read 9 Sent Done Done false
35402 9441 Acked Read 5 Sent Done Done false
35403 9442 Acked Read 9 Sent Done Done false
35404 9443 Acked Write 4 Sent Done Done false
35405 9444 Acked Flush 0 Sent Done Done false
35406 9445 Acked Read 6 Sent Done Done false
35407 9446 Acked Read 5 Sent Done Done false
35408 9447 Acked Read 6 Sent Done Done false
35409 9448 Acked Read 6 Sent Done Done false
35410 9449 Acked Read 3 Sent Done Done false
35411 9450 Acked Write 8 Sent Done Done false
35412 9451 Acked Read 8 Sent Done Done false
35413 9452 Acked Read 7 Sent Done Done false
35414 STATES DS:0 DS:1 DS:2 TOTAL
35415 Sent 301 0 0 301
35416 Done 0 301 301 602
35417 Skipped 0 0 0 0
35418 Error 0 0 0 0
35419 Last Flush: 9151 9444 9444
35420 Downstairs last five completed: 9151 9150 9149 9148 9147
35421 Upstairs last five completed: 9452 9451 9450 9449 9448
35422 CLIENT: Up:0 ds:301 act:2
35423 [8810] got start action
35424 [8810] I am going to restart
35425 Starting downstairs at port 8810
35426 Make output file at "/tmp/dsc/downstairs-8810.txt"
35427 [8810][0] reports Starting
35428 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1233)
35429 [8810][0] reports Running
35430 ----------------------------------------------------------------
35431 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
35432 States: Offline Active Active
35433 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
35434 9152 Acked Read 4 Sent Done Done false
35435 9153 Acked Flush 0 Sent Done Done false
35436 9154 Acked Write 5 Sent Done Done false
35437 9155 Acked Write 2 Sent Done Done false
35438 9156 Acked Write 2 Sent Done Done false
35439 9157 Acked Read 8 Sent Done Done false
35440 9158 Acked Write 8 Sent Done Done false
35441 9159 Acked Read 5 Sent Done Done false
35442 9160 Acked Flush 0 Sent Done Done false
35443 9161 Acked Read 2 Sent Done Done false
35444 9162 Acked Read 7 Sent Done Done false
35445 9163 Acked Read 1 Sent Done Done false
35446 9164 Acked Write 8 Sent Done Done false
35447 9165 Acked Read 3 Sent Done Done false
35448 9166 Acked Read 5 Sent Done Done false
35449 9167 Acked Write 7 Sent Done Done false
35450 9168 Acked Read 7 Sent Done Done false
35451 9169 Acked Read 7 Sent Done Done false
35452 9170 Acked Write 1 Sent Done Done false
35453 9171 Acked Write 3 Sent Done Done false
35454 9172 Acked Read 9 Sent Done Done false
35455 9173 Acked Read 1 Sent Done Done false
35456 9174 Acked Flush 0 Sent Done Done false
35457 9175 Acked Write 8 Sent Done Done false
35458 9176 Acked Read 2 Sent Done Done false
35459 9177 Acked Write 10 Sent Done Done false
35460 9178 Acked Read 3 Sent Done Done false
35461 9179 Acked Flush 0 Sent Done Done false
35462 9180 Acked Read 5 Sent Done Done false
35463 9181 Acked Read 4 Sent Done Done false
35464 9182 Acked Write 4 Sent Done Done false
35465 9183 Acked Read 9 Sent Done Done false
35466 9184 Acked Read 5 Sent Done Done false
35467 9185 Acked Write 9 Sent Done Done false
35468 9186 Acked Write 4 Sent Done Done false
35469 9187 Acked Write 7 Sent Done Done false
35470 9188 Acked Read 7 Sent Done Done false
35471 9189 Acked Write 1 Sent Done Done false
35472 9190 Acked Read 2 Sent Done Done false
35473 9191 Acked Read 8 Sent Done Done false
35474 9192 Acked Read 2 Sent Done Done false
35475 9193 Acked Write 6 Sent Done Done false
35476 9194 Acked Read 1 Sent Done Done false
35477 9195 Acked Flush 0 Sent Done Done false
35478 9196 Acked Write 3 Sent Done Done false
35479 9197 Acked Write 4 Sent Done Done false
35480 9198 Acked Flush 0 Sent Done Done false
35481 9199 Acked Read 4 Sent Done Done false
35482 9200 Acked Read 5 Sent Done Done false
35483 9201 Acked Read 10 Sent Done Done false
35484 9202 Acked Read 7 Sent Done Done false
35485 9203 Acked Read 6 Sent Done Done false
35486 9204 Acked Read 2 Sent Done Done false
35487 9205 Acked Read 3 Sent Done Done false
35488 9206 Acked Read 2 Sent Done Done false
35489 9207 Acked Write 1 Sent Done Done false
35490 9208 Acked Flush 0 Sent Done Done false
35491 9209 Acked Write 7 Sent Done Done false
35492 9210 Acked Read 6 Sent Done Done false
35493 9211 Acked Write 4 Sent Done Done false
35494 9212 Acked Flush 0 Sent Done Done false
35495 9213 Acked Read 8 Sent Done Done false
35496 9214 Acked Read 10 Sent Done Done false
35497 9215 Acked Write 7 Sent Done Done false
35498 9216 Acked Read 8 Sent Done Done false
35499 9217 Acked Write 7 Sent Done Done false
35500 9218 Acked Flush 0 Sent Done Done false
35501 9219 Acked Read 4 Sent Done Done false
35502 9220 Acked Read 7 Sent Done Done false
35503 9221 Acked Read 6 Sent Done Done false
35504 9222 Acked Write 8 Sent Done Done false
35505 9223 Acked Write 7 Sent Done Done false
35506 9224 Acked Read 1 Sent Done Done false
35507 9225 Acked Read 3 Sent Done Done false
35508 9226 Acked Read 9 Sent Done Done false
35509 9227 Acked Read 9 Sent Done Done false
35510 9228 Acked Read 9 Sent Done Done false
35511 9229 Acked Flush 0 Sent Done Done false
35512 9230 Acked Read 1 Sent Done Done false
35513 9231 Acked Read 7 Sent Done Done false
35514 9232 Acked Read 8 Sent Done Done false
35515 9233 Acked Write 8 Sent Done Done false
35516 9234 Acked Read 7 Sent Done Done false
35517 9235 Acked Read 1 Sent Done Done false
35518 9236 Acked Write 5 Sent Done Done false
35519 9237 Acked Read 7 Sent Done Done false
35520 9238 Acked Read 5 Sent Done Done false
35521 9239 Acked Write 2 Sent Done Done false
35522 9240 Acked Read 9 Sent Done Done false
35523 9241 Acked Read 4 Sent Done Done false
35524 9242 Acked Read 7 Sent Done Done false
35525 9243 Acked Read 7 Sent Done Done false
35526 9244 Acked Read 1 Sent Done Done false
35527 9245 Acked Read 2 Sent Done Done false
35528 9246 Acked Flush 0 Sent Done Done false
35529 9247 Acked Write 3 Sent Done Done false
35530 9248 Acked Write 7 Sent Done Done false
35531 9249 Acked Write 9 Sent Done Done false
35532 9250 Acked Read 8 Sent Done Done false
35533 9251 Acked Write 10 Sent Done Done false
35534 9252 Acked Read 4 Sent Done Done false
35535 9253 Acked Write 6 Sent Done Done false
35536 9254 Acked Read 9 Sent Done Done false
35537 9255 Acked Write 1 Sent Done Done false
35538 9256 Acked Read 8 Sent Done Done false
35539 9257 Acked Write 2 Sent Done Done false
35540 9258 Acked Write 1 Sent Done Done false
35541 9259 Acked Write 10 Sent Done Done false
35542 9260 Acked Read 7 Sent Done Done false
35543 9261 Acked Read 1 Sent Done Done false
35544 9262 Acked Read 2 Sent Done Done false
35545 9263 Acked Read 10 Sent Done Done false
35546 9264 Acked Read 7 Sent Done Done false
35547 9265 Acked Read 10 Sent Done Done false
35548 9266 Acked Write 7 Sent Done Done false
35549 9267 Acked Write 7 Sent Done Done false
35550 9268 Acked Write 7 Sent Done Done false
35551 9269 Acked Write 4 Sent Done Done false
35552 9270 Acked Read 10 Sent Done Done false
35553 9271 Acked Read 2 Sent Done Done false
35554 9272 Acked Write 10 Sent Done Done false
35555 9273 Acked Read 4 Sent Done Done false
35556 9274 Acked Read 4 Sent Done Done false
35557 9275 Acked Write 6 Sent Done Done false
35558 9276 Acked Read 6 Sent Done Done false
35559 9277 Acked Write 3 Sent Done Done false
35560 9278 Acked Flush 0 Sent Done Done false
35561 9279 Acked Write 3 Sent Done Done false
35562 9280 Acked Flush 0 Sent Done Done false
35563 9281 Acked Read 3 Sent Done Done false
35564 9282 Acked Read 2 Sent Done Done false
35565 9283 Acked Write 9 Sent Done Done false
35566 9284 Acked Read 10 Sent Done Done false
35567 9285 Acked Write 10 Sent Done Done false
35568 9286 Acked Read 5 Sent Done Done false
35569 9287 Acked Write 9 Sent Done Done false
35570 9288 Acked Read 4 Sent Done Done false
35571 9289 Acked Write 5 Sent Done Done false
35572 9290 Acked Flush 0 Sent Done Done false
35573 9291 Acked Read 1 Sent Done Done false
35574 9292 Acked Read 9 Sent Done Done false
35575 9293 Acked Write 9 Sent Done Done false
35576 9294 Acked Read 8 Sent Done Done false
35577 9295 Acked Write 7 Sent Done Done false
35578 9296 Acked Write 3 Sent Done Done false
35579 9297 Acked Write 7 Sent Done Done false
35580 9298 Acked Read 8 Sent Done Done false
35581 9299 Acked Write 5 Sent Done Done false
35582 9300 Acked Flush 0 Sent Done Done false
35583 9301 Acked Read 9 Sent Done Done false
35584 9302 Acked Write 3 Sent Done Done false
35585 9303 Acked Write 6 Sent Done Done false
35586 9304 Acked Flush 0 Sent Done Done false
35587 9305 Acked Read 5 Sent Done Done false
35588 9306 Acked Read 2 Sent Done Done false
35589 9307 Acked Write 10 Sent Done Done false
35590 9308 Acked Flush 0 Sent Done Done false
35591 9309 Acked Read 2 Sent Done Done false
35592 9310 Acked Flush 0 Sent Done Done false
35593 9311 Acked Write 2 Sent Done Done false
35594 9312 Acked Write 2 Sent Done Done false
35595 9313 Acked Write 1 Sent Done Done false
35596 9314 Acked Read 2 Sent Done Done false
35597 9315 Acked Read 10 Sent Done Done false
35598 9316 Acked Write 9 Sent Done Done false
35599 9317 Acked Flush 0 Sent Done Done false
35600 9318 Acked Read 3 Sent Done Done false
35601 9319 Acked Read 9 Sent Done Done false
35602 9320 Acked Write 8 Sent Done Done false
35603 9321 Acked Write 5 Sent Done Done false
35604 9322 Acked Read 2 Sent Done Done false
35605 9323 Acked Read 1 Sent Done Done false
35606 9324 Acked Flush 0 Sent Done Done false
35607 9325 Acked Flush 0 Sent Done Done false
35608 9326 Acked Flush 0 Sent Done Done false
35609 9327 Acked Write 6 Sent Done Done false
35610 9328 Acked Read 4 Sent Done Done false
35611 9329 Acked Read 9 Sent Done Done false
35612 9330 Acked Write 9 Sent Done Done false
35613 9331 Acked Read 4 Sent Done Done false
35614 9332 Acked Write 5 Sent Done Done false
35615 9333 Acked Write 6 Sent Done Done false
35616 9334 Acked Write 8 Sent Done Done false
35617 9335 Acked Read 1 Sent Done Done false
35618 9336 Acked Read 9 Sent Done Done false
35619 9337 Acked Write 7 Sent Done Done false
35620 9338 Acked Write 9 Sent Done Done false
35621 9339 Acked Read 10 Sent Done Done false
35622 9340 Acked Write 8 Sent Done Done false
35623 9341 Acked Write 2 Sent Done Done false
35624 9342 Acked Read 5 Sent Done Done false
35625 9343 Acked Read 9 Sent Done Done false
35626 9344 Acked Flush 0 Sent Done Done false
35627 9345 Acked Read 2 Sent Done Done false
35628 9346 Acked Read 3 Sent Done Done false
35629 9347 Acked Write 2 Sent Done Done false
35630 9348 Acked Flush 0 Sent Done Done false
35631 9349 Acked Read 6 Sent Done Done false
35632 9350 Acked Write 5 Sent Done Done false
35633 9351 Acked Read 8 Sent Done Done false
35634 9352 Acked Flush 0 Sent Done Done false
35635 9353 Acked Flush 0 Sent Done Done false
35636 9354 Acked Write 2 Sent Done Done false
35637 9355 Acked Read 4 Sent Done Done false
35638 9356 Acked Flush 0 Sent Done Done false
35639 9357 Acked Write 4 Sent Done Done false
35640 9358 Acked Write 1 Sent Done Done false
35641 9359 Acked Read 8 Sent Done Done false
35642 9360 Acked Read 3 Sent Done Done false
35643 9361 Acked Read 7 Sent Done Done false
35644 9362 Acked Write 5 Sent Done Done false
35645 9363 Acked Write 8 Sent Done Done false
35646 9364 Acked Write 3 Sent Done Done false
35647 9365 Acked Write 8 Sent Done Done false
35648 9366 Acked Write 6 Sent Done Done false
35649 9367 Acked Read 2 Sent Done Done false
35650 9368 Acked Read 2 Sent Done Done false
35651 9369 Acked Read 9 Sent Done Done false
35652 9370 Acked Flush 0 Sent Done Done false
35653 9371 Acked Flush 0 Sent Done Done false
35654 9372 Acked Read 3 Sent Done Done false
35655 9373 Acked Read 10 Sent Done Done false
35656 9374 Acked Read 4 Sent Done Done false
35657 9375 Acked Read 7 Sent Done Done false
35658 9376 Acked Flush 0 Sent Done Done false
35659 9377 Acked Read 10 Sent Done Done false
35660 9378 Acked Read 8 Sent Done Done false
35661 9379 Acked Read 8 Sent Done Done false
35662 9380 Acked Read 5 Sent Done Done false
35663 9381 Acked Write 6 Sent Done Done false
35664 9382 Acked Write 10 Sent Done Done false
35665 9383 Acked Write 9 Sent Done Done false
35666 9384 Acked Read 9 Sent Done Done false
35667 9385 Acked Read 6 Sent Done Done false
35668 9386 Acked Write 10 Sent Done Done false
35669 9387 Acked Read 6 Sent Done Done false
35670 9388 Acked Flush 0 Sent Done Done false
35671 9389 Acked Write 10 Sent Done Done false
35672 9390 Acked Write 8 Sent Done Done false
35673 9391 Acked Read 4 Sent Done Done false
35674 9392 Acked Write 3 Sent Done Done false
35675 9393 Acked Write 2 Sent Done Done false
35676 9394 Acked Write 1 Sent Done Done false
35677 9395 Acked Read 7 Sent Done Done false
35678 9396 Acked Read 4 Sent Done Done false
35679 9397 Acked Write 2 Sent Done Done false
35680 9398 Acked Flush 0 Sent Done Done false
35681 9399 Acked Read 10 Sent Done Done false
35682 9400 Acked Read 7 Sent Done Done false
35683 9401 Acked Write 1 Sent Done Done false
35684 9402 Acked Write 9 Sent Done Done false
35685 9403 Acked Write 2 Sent Done Done false
35686 9404 Acked Write 7 Sent Done Done false
35687 9405 Acked Write 7 Sent Done Done false
35688 9406 Acked Read 9 Sent Done Done false
35689 9407 Acked Read 3 Sent Done Done false
35690 9408 Acked Write 1 Sent Done Done false
35691 9409 Acked Read 8 Sent Done Done false
35692 9410 Acked Read 6 Sent Done Done false
35693 9411 Acked Write 7 Sent Done Done false
35694 9412 Acked Write 6 Sent Done Done false
35695 9413 Acked Flush 0 Sent Done Done false
35696 9414 Acked Flush 0 Sent Done Done false
35697 9415 Acked Read 2 Sent Done Done false
35698 9416 Acked Read 5 Sent Done Done false
35699 9417 Acked Write 2 Sent Done Done false
35700 9418 Acked Write 3 Sent Done Done false
35701 9419 Acked Write 1 Sent Done Done false
35702 9420 Acked Read 9 Sent Done Done false
35703 9421 Acked Write 1 Sent Done Done false
35704 9422 Acked Write 4 Sent Done Done false
35705 9423 Acked Write 1 Sent Done Done false
35706 9424 Acked Read 10 Sent Done Done false
35707 9425 Acked Read 9 Sent Done Done false
35708 9426 Acked Flush 0 Sent Done Done false
35709 9427 Acked Write 7 Sent Done Done false
35710 9428 Acked Read 1 Sent Done Done false
35711 9429 Acked Flush 0 Sent Done Done false
35712 9430 Acked Read 7 Sent Done Done false
35713 9431 Acked Write 2 Sent Done Done false
35714 9432 Acked Write 9 Sent Done Done false
35715 9433 Acked Write 3 Sent Done Done false
35716 9434 Acked Read 6 Sent Done Done false
35717 9435 Acked Read 5 Sent Done Done false
35718 9436 Acked Write 4 Sent Done Done false
35719 9437 Acked Read 2 Sent Done Done false
35720 9438 Acked Read 3 Sent Done Done false
35721 9439 Acked Read 8 Sent Done Done false
35722 9440 Acked Read 9 Sent Done Done false
35723 9441 Acked Read 5 Sent Done Done false
35724 9442 Acked Read 9 Sent Done Done false
35725 9443 Acked Write 4 Sent Done Done false
35726 9444 Acked Flush 0 Sent Done Done false
35727 9445 Acked Read 6 Sent Done Done false
35728 9446 Acked Read 5 Sent Done Done false
35729 9447 Acked Read 6 Sent Done Done false
35730 9448 Acked Read 6 Sent Done Done false
35731 9449 Acked Read 3 Sent Done Done false
35732 9450 Acked Write 8 Sent Done Done false
35733 9451 Acked Read 8 Sent Done Done false
35734 9452 Acked Read 7 Sent Done Done false
35735 9453 Acked Flush 0 Sent Done Done false
35736 STATES DS:0 DS:1 DS:2 TOTAL
35737 Sent 302 0 0 302
35738 Done 0 302 302 604
35739 Skipped 0 0 0 0
35740 Error 0 0 0 0
35741 Last Flush: 9151 9453 9453
35742 Downstairs last five completed: 9151 9150 9149 9148 9147
35743 Upstairs last five completed: 9453 9452 9451 9450 9449
35744 CLIENT: Up:0 ds:302 act:2
357452024-11-15T02:04:56.565ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
357462024-11-15T02:04:56.565ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:41414) = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
357472024-11-15T02:04:56.566ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
357482024-11-15T02:04:56.567ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
357492024-11-15T02:04:56.567ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
357502024-11-15T02:04:56.567ZINFOcrucible: send last flush ID to this DS: 9151 = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
357512024-11-15T02:04:56.567ZINFOcrucible: Replied this last flush ID: 9151 = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
357522024-11-15T02:04:56.567ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
357532024-11-15T02:04:56.567ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
357542024-11-15T02:04:56.567ZINFOcrucible: [0] client re-new 302 jobs since flush 9151 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
357552024-11-15T02:04:56.567ZINFOcrucible: [0] Replayed 302 jobs since flush: 9151 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
35756 ----------------------------------------------------------------
35757 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
35758 Downstairs last five completed: 9453 9452 9451 9450 9449
35759 Upstairs last five completed: 9453 9452 9451 9450 9449
35760 CLIENT: Up:0 ds:0 act:3
35761 Replay: All jobs finished
35762 Main task has work to do, go find it
35763 got dsc Stop(0)
35764 stop 0
35765 Nov 15 02:04:58.808 INFO request completed, latency_us: 394, response_code: 204, uri: /stop/cid/0, method: POST, req_id: fd70b5d0-42ff-4829-9e9e-1ace0f27ab66, remote_addr:[8810] Got stop action so:false kr:false
35766 127.0.0.1:40008, local_addr: 127.0.0.1:9998
35767 Nov 15 02:04:58.809 INFO request completed, latency_us: 123, response_code: 200, uri: /state/cid/0, method: GET, req_id: 6defedab-38f8-4b90-b280-f251db61651d, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
357682024-11-15T02:04:58.809ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
357692024-11-15T02:04:58.809ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
357702024-11-15T02:04:58.809ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
357712024-11-15T02:04:58.809ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
357722024-11-15T02:04:58.809ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
357732024-11-15T02:04:58.809ZINFOcrucible: client task is exiting = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
35774 [8810] Exited with: ExitStatus(unix_wait_status(9))
35775 [8810][0] reports Exit
35776 Nov 15 02:05:02.811 INFO request completed, latency_us: 208, response_code: 200, uri: /state/cid/0, method: GET, req_id: a90b7fe6-280b-4a77-a3bb-dd9a905224b1, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
35777 001/300 Read block 755 len 4096
35778 002/300 Write block 2336 len 24576 data: 9 8 6 7 8 8
35779 003/300 Write block 492 len 16384 data: 7 7 7 7
35780 004/300 Read block 1536 len 40960
35781 005/300 Write block 1108 len 4096 data: 4
35782 006/300 Read block 3553 len 12288
35783 007/300 Read block 1196 len 16384
35784 008/300 Flush
35785 009/300 Read block 4610 len 32768
35786 010/300 Read block 1319 len 28672
35787 011/300 Read block 4807 len 4096
35788 012/300 Write block 4578 len 8192 data: 5 4
35789 013/300 Write block 3292 len 40960 data: 3 3 4 4 2 2 2 2 2 2
35790 014/300 Read block 664 len 32768
35791 015/300 Write block 503 len 12288 data: 6 6 7
35792 016/300 Write block 1796 len 4096 data: 5
35793 017/300 Write block 2425 len 20480 data: 2 4 5 5 5
35794 018/300 Write block 4934 len 40960 data: 4 5 5 5 5 7 8 7 7 7
35795 019/300 Write block 4919 len 36864 data: 8 10 11 11 11 9 6 6 7
35796 020/300 Read block 3980 len 4096
35797 021/300 Write block 4857 len 40960 data: 4 3 5 7 8 10 11 10 10 9
35798 022/300 Read block 3057 len 4096
35799 023/300 Write block 2858 len 32768 data: 6 8 6 4 3 3 3 3
35800 024/300 Read block 1430 len 32768
35801 025/300 Read block 4748 len 40960
35802 026/300 Read block 3655 len 4096
35803 027/300 Write block 3828 len 20480 data: 4 4 4 4 6
35804 028/300 Write block 1470 len 16384 data: 5 7 6 5
35805 029/300 Flush
35806 030/300 Read block 1507 len 16384
35807 031/300 Read block 3222 len 24576
35808 032/300 Read block 671 len 8192
35809 033/300 Write block 1832 len 8192 data: 6 5
35810 034/300 Read block 2695 len 32768
35811 035/300 Read block 3529 len 28672
35812 036/300 Read block 2231 len 28672
35813 037/300 Read block 1644 len 8192
35814 038/300 Write block 4181 len 12288 data: 11 9 9
35815 039/300 Write block 4389 len 36864 data: 5 5 5 6 5 7 8 6 5
35816 040/300 Write block 1630 len 32768 data: 7 9 8 7 8 8 8 7
35817 041/300 Read block 1916 len 36864
35818 042/300 Read block 3919 len 36864
35819 043/300 Write block 4545 len 40960 data: 4 4 4 5 5 5 6 5 5 5
35820 044/300 Write block 4640 len 16384 data: 3 3 4 5
35821 045/300 Write block 4891 len 12288 data: 8 9 9
35822 046/300 Write block 4377 len 40960 data: 3 4 3 3 3 3 5 3 4 3
35823 047/300 Read block 3907 len 36864
35824 048/300 Read block 333 len 20480
35825 049/300 Read block 1463 len 40960
35826 050/300 Write block 1459 len 16384 data: 6 6 5 5
35827 051/300 Read block 3924 len 24576
35828 052/300 Read block 1107 len 40960
35829 053/300 Write block 982 len 16384 data: 2 3 5 6
35830 054/300 Read block 2479 len 24576
35831 055/300 Read block 1471 len 40960
35832 056/300 Write block 719 len 16384 data: 4 4 4 4
35833 057/300 Read block 1107 len 12288
35834 058/300 Read block 3315 len 20480
35835 059/300 Read block 1748 len 32768
35836 060/300 Read block 2440 len 16384
35837 061/300 Flush
35838 062/300 Write block 2175 len 16384 data: 5 5 5 4
35839 063/300 Write block 4133 len 12288 data: 6 6 5
35840 064/300 Write block 1358 len 16384 data: 10 7 5 6
35841 065/300 Read block 2695 len 8192
35842 066/300 Read block 578 len 28672
35843 067/300 Read block 3116 len 28672
35844 068/300 Write block 2589 len 24576 data: 5 6 6 7 6 7
35845 069/300 Flush
35846 070/300 Write block 993 len 4096 data: 11
35847 071/300 Write block 2456 len 32768 data: 9 10 8 8 9 9 8 9
35848 072/300 Read block 398 len 24576
35849 073/300 Write block 3690 len 16384 data: 9 9 7 5
35850 074/300 Write block 391 len 36864 data: 8 7 5 5 5 4 4 4 5
35851 075/300 Read block 1865 len 36864
35852 076/300 Read block 4268 len 12288
35853 077/300 Read block 4681 len 20480
35854 078/300 Write block 1481 len 20480 data: 3 3 3 3 3
35855 079/300 Read block 1547 len 28672
35856 080/300 Read block 2218 len 24576
35857 081/300 Read block 3137 len 8192
35858 082/300 Write block 4035 len 4096 data: 5
35859 083/300 Write block 4585 len 16384 data: 6 7 6 5
35860 084/300 Write block 4986 len 4096 data: 2
35861 085/300 Write block 4261 len 36864 data: 6 6 6 7 8 9 7 7 6
35862 086/300 Read block 4183 len 16384
35863 087/300 Write block 841 len 28672 data: 7 6 7 7 6 5 5
35864 088/300 Write block 3317 len 36864 data: 4 4 4 4 4 3 4 6 5
35865 089/300 Write block 4942 len 4096 data: 8
35866 090/300 Flush
35867 091/300 Write block 2023 len 40960 data: 7 10 12 12 11 8 7 5 5 5
35868 092/300 Write block 2029 len 40960 data: 8 6 6 6 5 4 4 4 3 4
35869 093/300 Read block 29 len 24576
35870 094/300 Write block 1769 len 12288 data: 6 4 4
35871 095/300 Read block 2764 len 16384
35872 096/300 Read block 4784 len 36864
35873 097/300 Write block 2297 len 4096 data: 6
35874 098/300 Write block 3811 len 32768 data: 9 6 5 5 5 4 2 2
35875 099/300 Flush
35876 100/300 Read block 4772 len 24576
35877 101/300 Read block 1641 len 24576
35878 102/300 Read block 949 len 28672
35879 103/300 Write block 3708 len 16384 data: 8 9 8 8
35880 104/300 Read block 4063 len 40960
35881 105/300 Write block 1643 len 32768 data: 8 8 7 7 8 7 6 5
35882 106/300 Read block 3071 len 24576
35883 107/300 Flush
35884 108/300 Read block 738 len 32768
35885 109/300 Write block 2297 len 28672 data: 7 5 4 4 6 7 6
35886 110/300 Write block 1441 len 28672 data: 4 4 4 5 6 7 5
35887 111/300 Read block 2246 len 24576
35888 112/300 Write block 4742 len 32768 data: 3 4 4 4 4 4 4 5
35889 113/300 Read block 4330 len 40960
35890 114/300 Write block 4839 len 4096 data: 5
35891 115/300 Write block 2078 len 16384 data: 8 7 7 7
35892 116/300 Write block 1312 len 16384 data: 6 4 8 8
35893 117/300 Read block 1251 len 8192
35894 118/300 Read block 4472 len 36864
35895 119/300 Write block 1375 len 40960 data: 7 6 6 6 8 8 8 7 5 5
35896 120/300 Read block 1273 len 8192
35897 121/300 Read block 1962 len 4096
35898 122/300 Read block 4730 len 20480
35899 123/300 Write block 2462 len 4096 data: 9
35900 124/300 Write block 3807 len 4096 data: 9
35901 125/300 Read block 1767 len 36864
35902 126/300 Write block 3918 len 36864 data: 2 5 7 7 8 10 10 8 7
35903 127/300 Write block 4413 len 20480 data: 4 3 3 3 3
35904 128/300 Write block 1913 len 36864 data: 4 5 6 6 5 5 5 5 4
35905 129/300 Read block 3447 len 8192
35906 130/300 Read block 3042 len 12288
35907 131/300 Write block 1529 len 20480 data: 4 5 5 5 5
35908 132/300 Flush
35909 133/300 Read block 1148 len 4096
35910 134/300 Write block 1627 len 16384 data: 6 7 6 8
35911 135/300 Flush
35912 136/300 Read block 1542 len 20480
35913 137/300 Write block 4763 len 8192 data: 4 3
35914 138/300 Write block 4673 len 36864 data: 5 2 3 4 4 6 7 7 6
35915 139/300 Read block 1464 len 36864
35916 140/300 Read block 4917 len 4096
35917 141/300 Write block 4455 len 32768 data: 6 8 8 7 7 6 6 6
35918 142/300 Write block 1663 len 12288 data: 12 12 11
35919 143/300 Flush
35920 144/300 Read block 880 len 24576
35921 145/300 Write block 2548 len 12288 data: 7 7 7
35922 146/300 Write block 1028 len 8192 data: 3 4
35923 147/300 Read block 4041 len 8192
35924 148/300 Read block 3148 len 20480
35925 149/300 Read block 3239 len 4096
35926 150/300 Flush
35927 151/300 Read block 2496 len 28672
35928 152/300 Read block 1667 len 40960
35929 153/300 Read block 2701 len 16384
35930 154/300 Read block 2103 len 16384
35931 155/300 Read block 2233 len 28672
35932 156/300 Read block 2165 len 16384
35933 157/300 Read block 2538 len 16384
35934 158/300 Read block 4456 len 32768
35935 159/300 Read block 3636 len 28672
35936 160/300 Read block 4122 len 12288
35937 161/300 Read block 4054 len 4096
35938 162/300 Read block 521 len 24576
35939 163/300 Flush
35940 164/300 Write block 4861 len 40960 data: 9 11 12 11 11 10 7 5 5 3
35941 165/300 Read block 2146 len 28672
35942 166/300 Write block 2314 len 24576 data: 7 6 7 5 7 10
35943 167/300 Read block 1139 len 12288
35944 168/300 Write block 2444 len 36864 data: 5 6 6 8 8 8 7 7 6
35945 169/300 Write block 4008 len 16384 data: 6 6 7 6
35946 170/300 Write block 582 len 24576 data: 4 4 4 3 3 5
35947 171/300 Read block 1336 len 12288
35948 172/300 Write block 4934 len 20480 data: 5 6 6 6 6
35949 173/300 Read block 3084 len 28672
35950 174/300 Read block 4983 len 28672
35951 175/300 Write block 1357 len 36864 data: 9 11 8 6 7 7 8 7 9
35952 176/300 Read block 4176 len 36864
35953 177/300 Read block 4861 len 4096
35954 178/300 Write block 4678 len 4096 data: 7
35955 179/300 Flush
35956 180/300 Read block 2084 len 12288
35957 181/300 Write block 3662 len 12288 data: 8 7 6
35958 182/300 Write block 1793 len 12288 data: 6 6 5
35959 183/300 Read block 194 len 16384
35960 184/300 Read block 1811 len 8192
35961 185/300 Read block 4924 len 4096
35962 186/300 Read block 3897 len 4096
35963 187/300 Write block 3026 len 12288 data: 4 5 4
35964 188/300 Flush
35965 189/300 Read block 1737 len 40960
35966 190/300 Write block 1736 len 20480 data: 5 6 6 6 4
35967 191/300 Read block 2032 len 12288
35968 192/300 Read block 3907 len 16384
35969 193/300 Write block 3282 len 28672 data: 6 6 8 6 6 4 4
35970 194/300 Read block 2733 len 32768
35971 195/300 Read block 1379 len 8192
35972 196/300 Read block 380 len 4096
35973 197/300 Flush
35974 198/300 Read block 1877 len 12288
35975 199/300 Flush
35976 200/300 Read block 3925 len 36864
35977 201/300 Read block 1016 len 4096
35978 202/300 Read block 258 len 36864
35979 203/300 Write block 2077 len 16384 data: 7 9 8 8
35980 204/300 Read block 872 len 36864
35981 205/300 Read block 4228 len 40960
35982 206/300 Read block 2985 len 16384
35983 207/300 Write block 494 len 40960 data: 8 8 5 5 6 6 5 7 7 7
35984 208/300 Read block 525 len 16384
35985 209/300 Write block 1600 len 40960 data: 6 5 6 6 5 6 8 6 5 4
35986 210/300 Read block 28 len 12288
35987 211/300 Write block 1689 len 8192 data: 4 5
35988 212/300 Write block 2164 len 12288 data: 6 4 5
35989 213/300 Read block 324 len 16384
35990 214/300 Write block 4610 len 28672 data: 5 3 6 5 6 6 7
35991 215/300 Write block 2030 len 24576 data: 7 7 7 6 5 5
35992 216/300 Write block 2875 len 8192 data: 5 4
35993 217/300 Flush
35994 218/300 Flush
35995 219/300 Write block 3169 len 20480 data: 4 5 5 4 4
35996 220/300 Flush
35997 221/300 Flush
35998 222/300 Read block 3604 len 4096
35999 223/300 Flush
36000 224/300 Read block 1353 len 8192
36001 225/300 Read block 2162 len 40960
36002 226/300 Read block 3965 len 4096
36003 227/300 Flush
36004 228/300 Write block 4434 len 8192 data: 6 8
36005 229/300 Read block 338 len 28672
36006 230/300 Read block 2365 len 36864
36007 231/300 Read block 4981 len 28672
36008 232/300 Read block 1697 len 28672
36009 233/300 Write block 1916 len 16384 data: 7 6 6 6
36010 234/300 Write block 4479 len 36864 data: 4 4 4 4 5 5 5 5 5
36011 235/300 Read block 1036 len 12288
36012 236/300 Write block 3079 len 8192 data: 9 10
36013 237/300 Read block 1418 len 16384
36014 238/300 Read block 3980 len 20480
36015 239/300 Write block 1742 len 4096 data: 5
36016 240/300 Write block 4186 len 24576 data: 6 5 4 6 7 6
36017 241/300 Read block 2415 len 36864
36018 242/300 Flush
36019 243/300 Write block 1234 len 40960 data: 5 6 5 4 4 4 3 6 5 7
36020 244/300 Write block 123 len 16384 data: 7 7 6 5
36021 245/300 Read block 1179 len 32768
36022 246/300 Write block 4891 len 8192 data: 9 10
36023 247/300 Write block 1781 len 8192 data: 7 8
36024 248/300 Write block 1954 len 28672 data: 8 8 7 8 7 6 4
36025 249/300 Write block 3996 len 28672 data: 3 3 3 2 4 3 6
36026 250/300 Write block 1780 len 40960 data: 7 8 9 8 6 5 4 5 6 5
36027 251/300 Read block 1763 len 16384
36028 252/300 Read block 4154 len 32768
36029 253/300 Write block 4188 len 4096 data: 5
36030 254/300 Write block 4516 len 16384 data: 7 5 5 4
36031 255/300 Read block 990 len 36864
36032 256/300 Read block 3666 len 16384
36033 257/300 Read block 149 len 16384
36034 258/300 Write block 1346 len 32768 data: 6 6 4 6 7 6 5 4
36035 259/300 Flush
36036 260/300 Write block 56 len 20480 data: 5 7 8 7 7
36037 261/300 Read block 1401 len 12288
36038 262/300 Read block 833 len 20480
36039 263/300 Read block 4924 len 8192
36040 264/300 Read block 4352 len 36864
36041 265/300 Write block 4914 len 12288 data: 4 4 6
36042 266/300 Flush
36043 267/300 Read block 3143 len 20480
36044 268/300 Write block 2497 len 8192 data: 6 7
36045 269/300 Read block 3553 len 4096
36046 270/300 Write block 1892 len 32768 data: 8 8 7 6 6 7 5 6
36047 271/300 Flush
36048 272/300 Write block 2641 len 16384 data: 3 5 6 2
36049 273/300 Write block 1650 len 24576 data: 6 7 6 5 7 7
36050 274/300 Read block 3438 len 8192
36051 275/300 Write block 2914 len 20480 data: 4 3 3 3 5
36052 276/300 Read block 1446 len 28672
36053 277/300 Read block 4714 len 12288
36054 278/300 Write block 3614 len 20480 data: 3 3 3 3 4
36055 279/300 Write block 1879 len 28672 data: 5 6 7 5 5 4 5
36056 280/300 Read block 2849 len 32768
36057 281/300 Write block 1687 len 4096 data: 6
36058 282/300 Read block 4664 len 8192
36059 283/300 Read block 4198 len 12288
36060 284/300 Write block 4022 len 36864 data: 6 5 5 5 5 5 5 4 4
36061 285/300 Flush
36062 286/300 Write block 226 len 4096 data: 3
36063 287/300 Write block 821 len 4096 data: 3
36064 288/300 Read block 551 len 36864
36065 289/300 Flush
36066 290/300 Read block 681 len 40960
36067 291/300 Flush
36068 292/300 Read block 4346 len 36864
36069 293/300 Write block 333 len 20480 data: 9 7 6 5 6
36070 294/300 Write block 3872 len 24576 data: 7 6 8 8 5 4
36071 295/300 Write block 4357 len 20480 data: 5 5 4 4 5
36072 296/300 Read block 3618 len 28672
36073 297/300 Write block 3251 len 12288 data: 11 11 13
36074 298/300 Read block 2585 len 28672
36075 299/300 Write block 2863 len 32768 data: 4 4 4 3 5 4 6 6
36076 300/300 Read block 1864 len 16384
36077 301/300 Write block 4615 len 8192 data: 7 8
36078 Main task has work to do, go find it
36079 got dsc Start(0)
36080 start 0
36081 Nov 15 02:05:03.041 INFO request completed, latency_us: 224, response_code: 204, uri: /start/cid/0, method: POST, req_id: 86596a74-c434-4964-b4b3-7dddc0dfe52c, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
36082 Replay: started 0, returned:Ok(())
36083 ----------------------------------------------------------------
36084 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
36085 States: Offline Active Active
36086 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
36087 9454 Acked Read 1 Sent Done Done false
36088 9455 Acked Write 6 Sent Done Done false
36089 9456 Acked Write 4 Sent Done Done false
36090 9457 Acked Read 10 Sent Done Done false
36091 9458 Acked Write 1 Sent Done Done false
36092 9459 Acked Read 3 Sent Done Done false
36093 9460 Acked Read 4 Sent Done Done false
36094 9461 Acked Flush 0 Sent Done Done false
36095 9462 Acked Read 8 Sent Done Done false
36096 9463 Acked Read 7 Sent Done Done false
36097 9464 Acked Read 1 Sent Done Done false
36098 9465 Acked Write 2 Sent Done Done false
36099 9466 Acked Write 10 Sent Done Done false
36100 9467 Acked Read 8 Sent Done Done false
36101 9468 Acked Write 3 Sent Done Done false
36102 9469 Acked Write 1 Sent Done Done false
36103 9470 Acked Write 5 Sent Done Done false
36104 9471 Acked Write 10 Sent Done Done false
36105 9472 Acked Write 9 Sent Done Done false
36106 9473 Acked Read 1 Sent Done Done false
36107 9474 Acked Write 10 Sent Done Done false
36108 9475 Acked Read 1 Sent Done Done false
36109 9476 Acked Write 8 Sent Done Done false
36110 9477 Acked Read 8 Sent Done Done false
36111 9478 Acked Read 10 Sent Done Done false
36112 9479 Acked Read 1 Sent Done Done false
36113 9480 Acked Write 5 Sent Done Done false
36114 9481 Acked Write 4 Sent Done Done false
36115 9482 Acked Flush 0 Sent Done Done false
36116 9483 Acked Read 4 Sent Done Done false
36117 9484 Acked Read 6 Sent Done Done false
36118 9485 Acked Read 2 Sent Done Done false
36119 9486 Acked Write 2 Sent Done Done false
36120 9487 Acked Read 8 Sent Done Done false
36121 9488 Acked Read 7 Sent Done Done false
36122 9489 Acked Read 7 Sent Done Done false
36123 9490 Acked Read 2 Sent Done Done false
36124 9491 Acked Write 3 Sent Done Done false
36125 9492 Acked Write 9 Sent Done Done false
36126 9493 Acked Write 8 Sent Done Done false
36127 9494 Acked Read 9 Sent Done Done false
36128 9495 Acked Read 9 Sent Done Done false
36129 9496 Acked Write 10 Sent Done Done false
36130 9497 Acked Write 4 Sent Done Done false
36131 9498 Acked Write 3 Sent Done Done false
36132 9499 Acked Write 10 Sent Done Done false
36133 9500 Acked Read 9 Sent Done Done false
36134 9501 Acked Read 5 Sent Done Done false
36135 9502 Acked Read 10 Sent Done Done false
36136 9503 Acked Write 4 Sent Done Done false
36137 9504 Acked Read 6 Sent Done Done false
36138 9505 Acked Read 10 Sent Done Done false
36139 9506 Acked Write 4 Sent Done Done false
36140 9507 Acked Read 6 Sent Done Done false
36141 9508 Acked Read 10 Sent Done Done false
36142 9509 Acked Write 4 Sent Done Done false
36143 9510 Acked Read 3 Sent Done Done false
36144 9511 Acked Read 5 Sent Done Done false
36145 9512 Acked Read 8 Sent Done Done false
36146 9513 Acked Read 4 Sent Done Done false
36147 9514 Acked Flush 0 Sent Done Done false
36148 9515 Acked Write 4 Sent Done Done false
36149 9516 Acked Write 3 Sent Done Done false
36150 9517 Acked Write 4 Sent Done Done false
36151 9518 Acked Read 2 Sent Done Done false
36152 9519 Acked Read 7 Sent Done Done false
36153 9520 Acked Read 7 Sent Done Done false
36154 9521 Acked Write 6 Sent Done Done false
36155 9522 Acked Flush 0 Sent Done Done false
36156 9523 Acked Write 1 Sent Done Done false
36157 9524 Acked Write 8 Sent Done Done false
36158 9525 Acked Read 6 Sent Done Done false
36159 9526 Acked Write 4 Sent Done Done false
36160 9527 Acked Write 9 Sent Done Done false
36161 9528 Acked Read 9 Sent Done Done false
36162 9529 Acked Read 3 Sent Done Done false
36163 9530 Acked Read 5 Sent Done Done false
36164 9531 Acked Write 5 Sent Done Done false
36165 9532 Acked Read 7 Sent Done Done false
36166 9533 Acked Read 6 Sent Done Done false
36167 9534 Acked Read 2 Sent Done Done false
36168 9535 Acked Write 1 Sent Done Done false
36169 9536 Acked Write 4 Sent Done Done false
36170 9537 Acked Write 1 Sent Done Done false
36171 9538 Acked Write 9 Sent Done Done false
36172 9539 Acked Read 4 Sent Done Done false
36173 9540 Acked Write 7 Sent Done Done false
36174 9541 Acked Write 9 Sent Done Done false
36175 9542 Acked Write 1 Sent Done Done false
36176 9543 Acked Flush 0 Sent Done Done false
36177 9544 Acked Write 10 Sent Done Done false
36178 9545 Acked Write 10 Sent Done Done false
36179 9546 Acked Read 6 Sent Done Done false
36180 9547 Acked Write 3 Sent Done Done false
36181 9548 Acked Read 4 Sent Done Done false
36182 9549 Acked Read 9 Sent Done Done false
36183 9550 Acked Write 1 Sent Done Done false
36184 9551 Acked Write 8 Sent Done Done false
36185 9552 Acked Flush 0 Sent Done Done false
36186 9553 Acked Read 6 Sent Done Done false
36187 9554 Acked Read 6 Sent Done Done false
36188 9555 Acked Read 7 Sent Done Done false
36189 9556 Acked Write 4 Sent Done Done false
36190 9557 Acked Read 10 Sent Done Done false
36191 9558 Acked Write 8 Sent Done Done false
36192 9559 Acked Read 6 Sent Done Done false
36193 9560 Acked Flush 0 Sent Done Done false
36194 9561 Acked Read 8 Sent Done Done false
36195 9562 Acked Write 7 Sent Done Done false
36196 9563 Acked Write 7 Sent Done Done false
36197 9564 Acked Read 6 Sent Done Done false
36198 9565 Acked Write 8 Sent Done Done false
36199 9566 Acked Read 10 Sent Done Done false
36200 9567 Acked Write 1 Sent Done Done false
36201 9568 Acked Write 4 Sent Done Done false
36202 9569 Acked Write 4 Sent Done Done false
36203 9570 Acked Read 2 Sent Done Done false
36204 9571 Acked Read 9 Sent Done Done false
36205 9572 Acked Write 10 Sent Done Done false
36206 9573 Acked Read 2 Sent Done Done false
36207 9574 Acked Read 1 Sent Done Done false
36208 9575 Acked Read 5 Sent Done Done false
36209 9576 Acked Write 1 Sent Done Done false
36210 9577 Acked Write 1 Sent Done Done false
36211 9578 Acked Read 9 Sent Done Done false
36212 9579 Acked Write 9 Sent Done Done false
36213 9580 Acked Write 5 Sent Done Done false
36214 9581 Acked Write 9 Sent Done Done false
36215 9582 Acked Read 2 Sent Done Done false
36216 9583 Acked Read 3 Sent Done Done false
36217 9584 Acked Write 5 Sent Done Done false
36218 9585 Acked Flush 0 Sent Done Done false
36219 9586 Acked Read 1 Sent Done Done false
36220 9587 Acked Write 4 Sent Done Done false
36221 9588 Acked Flush 0 Sent Done Done false
36222 9589 Acked Read 5 Sent Done Done false
36223 9590 Acked Write 2 Sent Done Done false
36224 9591 Acked Write 9 Sent Done Done false
36225 9592 Acked Read 9 Sent Done Done false
36226 9593 Acked Read 1 Sent Done Done false
36227 9594 Acked Write 8 Sent Done Done false
36228 9595 Acked Write 3 Sent Done Done false
36229 9596 Acked Flush 0 Sent Done Done false
36230 9597 Acked Read 6 Sent Done Done false
36231 9598 Acked Write 3 Sent Done Done false
36232 9599 Acked Write 2 Sent Done Done false
36233 9600 Acked Read 2 Sent Done Done false
36234 9601 Acked Read 5 Sent Done Done false
36235 9602 Acked Read 1 Sent Done Done false
36236 9603 Acked Flush 0 Sent Done Done false
36237 9604 Acked Read 7 Sent Done Done false
36238 9605 Acked Read 10 Sent Done Done false
36239 9606 Acked Read 4 Sent Done Done false
36240 9607 Acked Read 4 Sent Done Done false
36241 9608 Acked Read 7 Sent Done Done false
36242 9609 Acked Read 4 Sent Done Done false
36243 9610 Acked Read 4 Sent Done Done false
36244 9611 Acked Read 8 Sent Done Done false
36245 9612 Acked Read 7 Sent Done Done false
36246 9613 Acked Read 3 Sent Done Done false
36247 9614 Acked Read 1 Sent Done Done false
36248 9615 Acked Read 6 Sent Done Done false
36249 9616 Acked Flush 0 Sent Done Done false
36250 9617 Acked Write 10 Sent Done Done false
36251 9618 Acked Read 7 Sent Done Done false
36252 9619 Acked Write 6 Sent Done Done false
36253 9620 Acked Read 3 Sent Done Done false
36254 9621 Acked Write 9 Sent Done Done false
36255 9622 Acked Write 4 Sent Done Done false
36256 9623 Acked Write 6 Sent Done Done false
36257 9624 Acked Read 3 Sent Done Done false
36258 9625 Acked Write 5 Sent Done Done false
36259 9626 Acked Read 7 Sent Done Done false
36260 9627 Acked Read 7 Sent Done Done false
36261 9628 Acked Write 9 Sent Done Done false
36262 9629 Acked Read 9 Sent Done Done false
36263 9630 Acked Read 1 Sent Done Done false
36264 9631 Acked Write 1 Sent Done Done false
36265 9632 Acked Flush 0 Sent Done Done false
36266 9633 Acked Read 3 Sent Done Done false
36267 9634 Acked Write 3 Sent Done Done false
36268 9635 Acked Write 3 Sent Done Done false
36269 9636 Acked Read 4 Sent Done Done false
36270 9637 Acked Read 2 Sent Done Done false
36271 9638 Acked Read 1 Sent Done Done false
36272 9639 Acked Read 1 Sent Done Done false
36273 9640 Acked Write 3 Sent Done Done false
36274 9641 Acked Flush 0 Sent Done Done false
36275 9642 Acked Read 10 Sent Done Done false
36276 9643 Acked Write 5 Sent Done Done false
36277 9644 Acked Read 3 Sent Done Done false
36278 9645 Acked Read 4 Sent Done Done false
36279 9646 Acked Write 7 Sent Done Done false
36280 9647 Acked Read 8 Sent Done Done false
36281 9648 Acked Read 2 Sent Done Done false
36282 9649 Acked Read 1 Sent Done Done false
36283 9650 Acked Flush 0 Sent Done Done false
36284 9651 Acked Read 3 Sent Done Done false
36285 9652 Acked Flush 0 Sent Done Done false
36286 9653 Acked Read 9 Sent Done Done false
36287 9654 Acked Read 1 Sent Done Done false
36288 9655 Acked Read 9 Sent Done Done false
36289 9656 Acked Write 4 Sent Done Done false
36290 9657 Acked Read 9 Sent Done Done false
36291 9658 Acked Read 10 Sent Done Done false
36292 9659 Acked Read 4 Sent Done Done false
36293 9660 Acked Write 10 Sent Done Done false
36294 9661 Acked Read 4 Sent Done Done false
36295 9662 Acked Write 10 Sent Done Done false
36296 9663 Acked Read 3 Sent Done Done false
36297 9664 Acked Write 2 Sent Done Done false
36298 9665 Acked Write 3 Sent Done Done false
36299 9666 Acked Read 4 Sent Done Done false
36300 9667 Acked Write 7 Sent Done Done false
36301 9668 Acked Write 6 Sent Done Done false
36302 9669 Acked Write 2 Sent Done Done false
36303 9670 Acked Flush 0 Sent Done Done false
36304 9671 Acked Flush 0 Sent Done Done false
36305 9672 Acked Write 5 Sent Done Done false
36306 9673 Acked Flush 0 Sent Done Done false
36307 9674 Acked Flush 0 Sent Done Done false
36308 9675 Acked Read 1 Sent Done Done false
36309 9676 Acked Flush 0 Sent Done Done false
36310 9677 Acked Read 2 Sent Done Done false
36311 9678 Acked Read 10 Sent Done Done false
36312 9679 Acked Read 1 Sent Done Done false
36313 9680 Acked Flush 0 Sent Done Done false
36314 9681 Acked Write 2 Sent Done Done false
36315 9682 Acked Read 7 Sent Done Done false
36316 9683 Acked Read 9 Sent Done Done false
36317 9684 Acked Read 7 Sent Done Done false
36318 9685 Acked Read 7 Sent Done Done false
36319 9686 Acked Write 4 Sent Done Done false
36320 9687 Acked Write 9 Sent Done Done false
36321 9688 Acked Read 3 Sent Done Done false
36322 9689 Acked Write 2 Sent Done Done false
36323 9690 Acked Read 4 Sent Done Done false
36324 9691 Acked Read 5 Sent Done Done false
36325 9692 Acked Write 1 Sent Done Done false
36326 9693 Acked Write 6 Sent Done Done false
36327 9694 Acked Read 9 Sent Done Done false
36328 9695 Acked Flush 0 Sent Done Done false
36329 9696 Acked Write 10 Sent Done Done false
36330 9697 Acked Write 4 Sent Done Done false
36331 9698 Acked Read 8 Sent Done Done false
36332 9699 Acked Write 2 Sent Done Done false
36333 9700 Acked Write 2 Sent Done Done false
36334 9701 Acked Write 7 Sent Done Done false
36335 9702 Acked Write 7 Sent Done Done false
36336 9703 Acked Write 10 Sent Done Done false
36337 9704 Acked Read 4 Sent Done Done false
36338 9705 Acked Read 8 Sent Done Done false
36339 9706 Acked Write 1 Sent Done Done false
36340 9707 Acked Write 4 Sent Done Done false
36341 9708 Acked Read 9 Sent Done Done false
36342 9709 Acked Read 4 Sent Done Done false
36343 9710 Acked Read 4 Sent Done Done false
36344 9711 Acked Write 8 Sent Done Done false
36345 9712 Acked Flush 0 Sent Done Done false
36346 9713 Acked Write 5 Sent Done Done false
36347 9714 Acked Read 3 Sent Done Done false
36348 9715 Acked Read 5 Sent Done Done false
36349 9716 Acked Read 2 Sent Done Done false
36350 9717 Acked Read 9 Sent Done Done false
36351 9718 Acked Write 3 Sent Done Done false
36352 9719 Acked Flush 0 Sent Done Done false
36353 9720 Acked Read 5 Sent Done Done false
36354 9721 Acked Write 2 Sent Done Done false
36355 9722 Acked Read 1 Sent Done Done false
36356 9723 Acked Write 8 Sent Done Done false
36357 9724 Acked Flush 0 Sent Done Done false
36358 9725 Acked Write 4 Sent Done Done false
36359 9726 Acked Write 6 Sent Done Done false
36360 9727 Acked Read 2 Sent Done Done false
36361 9728 Acked Write 5 Sent Done Done false
36362 9729 Acked Read 7 Sent Done Done false
36363 9730 Acked Read 3 Sent Done Done false
36364 9731 Acked Write 5 Sent Done Done false
36365 9732 Acked Write 7 Sent Done Done false
36366 9733 Acked Read 8 Sent Done Done false
36367 9734 Acked Write 1 Sent Done Done false
36368 9735 Acked Read 2 Sent Done Done false
36369 9736 Acked Read 3 Sent Done Done false
36370 9737 Acked Write 9 Sent Done Done false
36371 9738 Acked Flush 0 Sent Done Done false
36372 9739 Acked Write 1 Sent Done Done false
36373 9740 Acked Write 1 Sent Done Done false
36374 9741 Acked Read 9 Sent Done Done false
36375 9742 Acked Flush 0 Sent Done Done false
36376 9743 Acked Read 10 Sent Done Done false
36377 9744 Acked Flush 0 Sent Done Done false
36378 9745 Acked Read 9 Sent Done Done false
36379 9746 Acked Write 5 Sent Done Done false
36380 9747 Acked Write 6 Sent Done Done false
36381 9748 Acked Write 5 Sent Done Done false
36382 9749 Acked Read 7 Sent Done Done false
36383 9750 Acked Write 3 Sent Done Done false
36384 9751 Acked Read 7 Sent Done Done false
36385 9752 Acked Write 8 Sent Done Done false
36386 9753 Acked Read 4 Sent Done Done false
36387 9754 Acked Write 2 Sent Done Done false
36388 STATES DS:0 DS:1 DS:2 TOTAL
36389 Sent 301 0 0 301
36390 Done 0 301 301 602
36391 Skipped 0 0 0 0
36392 Error 0 0 0 0
36393 Last Flush: 9453 9744 9744
36394 Downstairs last five completed: 9453 9452 9451 9450 9449
36395 Upstairs last five completed: 9754 9753 9752 9751 9750
36396 CLIENT: Up:0 ds:301 act:2
36397 [8810] got start action
36398 [8810] I am going to restart
36399 Starting downstairs at port 8810
36400 Make output file at "/tmp/dsc/downstairs-8810.txt"
36401 [8810][0] reports Starting
36402 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1234)
36403 [8810][0] reports Running
36404 ----------------------------------------------------------------
36405 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
36406 States: Offline Active Active
36407 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
36408 9454 Acked Read 1 Sent Done Done false
36409 9455 Acked Write 6 Sent Done Done false
36410 9456 Acked Write 4 Sent Done Done false
36411 9457 Acked Read 10 Sent Done Done false
36412 9458 Acked Write 1 Sent Done Done false
36413 9459 Acked Read 3 Sent Done Done false
36414 9460 Acked Read 4 Sent Done Done false
36415 9461 Acked Flush 0 Sent Done Done false
36416 9462 Acked Read 8 Sent Done Done false
36417 9463 Acked Read 7 Sent Done Done false
36418 9464 Acked Read 1 Sent Done Done false
36419 9465 Acked Write 2 Sent Done Done false
36420 9466 Acked Write 10 Sent Done Done false
36421 9467 Acked Read 8 Sent Done Done false
36422 9468 Acked Write 3 Sent Done Done false
36423 9469 Acked Write 1 Sent Done Done false
36424 9470 Acked Write 5 Sent Done Done false
36425 9471 Acked Write 10 Sent Done Done false
36426 9472 Acked Write 9 Sent Done Done false
36427 9473 Acked Read 1 Sent Done Done false
36428 9474 Acked Write 10 Sent Done Done false
36429 9475 Acked Read 1 Sent Done Done false
36430 9476 Acked Write 8 Sent Done Done false
36431 9477 Acked Read 8 Sent Done Done false
36432 9478 Acked Read 10 Sent Done Done false
36433 9479 Acked Read 1 Sent Done Done false
36434 9480 Acked Write 5 Sent Done Done false
36435 9481 Acked Write 4 Sent Done Done false
36436 9482 Acked Flush 0 Sent Done Done false
36437 9483 Acked Read 4 Sent Done Done false
36438 9484 Acked Read 6 Sent Done Done false
36439 9485 Acked Read 2 Sent Done Done false
36440 9486 Acked Write 2 Sent Done Done false
36441 9487 Acked Read 8 Sent Done Done false
36442 9488 Acked Read 7 Sent Done Done false
36443 9489 Acked Read 7 Sent Done Done false
36444 9490 Acked Read 2 Sent Done Done false
36445 9491 Acked Write 3 Sent Done Done false
36446 9492 Acked Write 9 Sent Done Done false
36447 9493 Acked Write 8 Sent Done Done false
36448 9494 Acked Read 9 Sent Done Done false
36449 9495 Acked Read 9 Sent Done Done false
36450 9496 Acked Write 10 Sent Done Done false
36451 9497 Acked Write 4 Sent Done Done false
36452 9498 Acked Write 3 Sent Done Done false
36453 9499 Acked Write 10 Sent Done Done false
36454 9500 Acked Read 9 Sent Done Done false
36455 9501 Acked Read 5 Sent Done Done false
36456 9502 Acked Read 10 Sent Done Done false
36457 9503 Acked Write 4 Sent Done Done false
36458 9504 Acked Read 6 Sent Done Done false
36459 9505 Acked Read 10 Sent Done Done false
36460 9506 Acked Write 4 Sent Done Done false
36461 9507 Acked Read 6 Sent Done Done false
36462 9508 Acked Read 10 Sent Done Done false
36463 9509 Acked Write 4 Sent Done Done false
36464 9510 Acked Read 3 Sent Done Done false
36465 9511 Acked Read 5 Sent Done Done false
36466 9512 Acked Read 8 Sent Done Done false
36467 9513 Acked Read 4 Sent Done Done false
36468 9514 Acked Flush 0 Sent Done Done false
36469 9515 Acked Write 4 Sent Done Done false
36470 9516 Acked Write 3 Sent Done Done false
36471 9517 Acked Write 4 Sent Done Done false
36472 9518 Acked Read 2 Sent Done Done false
36473 9519 Acked Read 7 Sent Done Done false
36474 9520 Acked Read 7 Sent Done Done false
36475 9521 Acked Write 6 Sent Done Done false
36476 9522 Acked Flush 0 Sent Done Done false
36477 9523 Acked Write 1 Sent Done Done false
36478 9524 Acked Write 8 Sent Done Done false
36479 9525 Acked Read 6 Sent Done Done false
36480 9526 Acked Write 4 Sent Done Done false
36481 9527 Acked Write 9 Sent Done Done false
36482 9528 Acked Read 9 Sent Done Done false
36483 9529 Acked Read 3 Sent Done Done false
36484 9530 Acked Read 5 Sent Done Done false
36485 9531 Acked Write 5 Sent Done Done false
36486 9532 Acked Read 7 Sent Done Done false
36487 9533 Acked Read 6 Sent Done Done false
36488 9534 Acked Read 2 Sent Done Done false
36489 9535 Acked Write 1 Sent Done Done false
36490 9536 Acked Write 4 Sent Done Done false
36491 9537 Acked Write 1 Sent Done Done false
36492 9538 Acked Write 9 Sent Done Done false
36493 9539 Acked Read 4 Sent Done Done false
36494 9540 Acked Write 7 Sent Done Done false
36495 9541 Acked Write 9 Sent Done Done false
36496 9542 Acked Write 1 Sent Done Done false
36497 9543 Acked Flush 0 Sent Done Done false
36498 9544 Acked Write 10 Sent Done Done false
36499 9545 Acked Write 10 Sent Done Done false
36500 9546 Acked Read 6 Sent Done Done false
36501 9547 Acked Write 3 Sent Done Done false
36502 9548 Acked Read 4 Sent Done Done false
36503 9549 Acked Read 9 Sent Done Done false
36504 9550 Acked Write 1 Sent Done Done false
36505 9551 Acked Write 8 Sent Done Done false
36506 9552 Acked Flush 0 Sent Done Done false
36507 9553 Acked Read 6 Sent Done Done false
36508 9554 Acked Read 6 Sent Done Done false
36509 9555 Acked Read 7 Sent Done Done false
36510 9556 Acked Write 4 Sent Done Done false
36511 9557 Acked Read 10 Sent Done Done false
36512 9558 Acked Write 8 Sent Done Done false
36513 9559 Acked Read 6 Sent Done Done false
36514 9560 Acked Flush 0 Sent Done Done false
36515 9561 Acked Read 8 Sent Done Done false
36516 9562 Acked Write 7 Sent Done Done false
36517 9563 Acked Write 7 Sent Done Done false
36518 9564 Acked Read 6 Sent Done Done false
36519 9565 Acked Write 8 Sent Done Done false
36520 9566 Acked Read 10 Sent Done Done false
36521 9567 Acked Write 1 Sent Done Done false
36522 9568 Acked Write 4 Sent Done Done false
36523 9569 Acked Write 4 Sent Done Done false
36524 9570 Acked Read 2 Sent Done Done false
36525 9571 Acked Read 9 Sent Done Done false
36526 9572 Acked Write 10 Sent Done Done false
36527 9573 Acked Read 2 Sent Done Done false
36528 9574 Acked Read 1 Sent Done Done false
36529 9575 Acked Read 5 Sent Done Done false
36530 9576 Acked Write 1 Sent Done Done false
36531 9577 Acked Write 1 Sent Done Done false
36532 9578 Acked Read 9 Sent Done Done false
36533 9579 Acked Write 9 Sent Done Done false
36534 9580 Acked Write 5 Sent Done Done false
36535 9581 Acked Write 9 Sent Done Done false
36536 9582 Acked Read 2 Sent Done Done false
36537 9583 Acked Read 3 Sent Done Done false
36538 9584 Acked Write 5 Sent Done Done false
36539 9585 Acked Flush 0 Sent Done Done false
36540 9586 Acked Read 1 Sent Done Done false
36541 9587 Acked Write 4 Sent Done Done false
36542 9588 Acked Flush 0 Sent Done Done false
36543 9589 Acked Read 5 Sent Done Done false
36544 9590 Acked Write 2 Sent Done Done false
36545 9591 Acked Write 9 Sent Done Done false
36546 9592 Acked Read 9 Sent Done Done false
36547 9593 Acked Read 1 Sent Done Done false
36548 9594 Acked Write 8 Sent Done Done false
36549 9595 Acked Write 3 Sent Done Done false
36550 9596 Acked Flush 0 Sent Done Done false
36551 9597 Acked Read 6 Sent Done Done false
36552 9598 Acked Write 3 Sent Done Done false
36553 9599 Acked Write 2 Sent Done Done false
36554 9600 Acked Read 2 Sent Done Done false
36555 9601 Acked Read 5 Sent Done Done false
36556 9602 Acked Read 1 Sent Done Done false
36557 9603 Acked Flush 0 Sent Done Done false
36558 9604 Acked Read 7 Sent Done Done false
36559 9605 Acked Read 10 Sent Done Done false
36560 9606 Acked Read 4 Sent Done Done false
36561 9607 Acked Read 4 Sent Done Done false
36562 9608 Acked Read 7 Sent Done Done false
36563 9609 Acked Read 4 Sent Done Done false
36564 9610 Acked Read 4 Sent Done Done false
36565 9611 Acked Read 8 Sent Done Done false
36566 9612 Acked Read 7 Sent Done Done false
36567 9613 Acked Read 3 Sent Done Done false
36568 9614 Acked Read 1 Sent Done Done false
36569 9615 Acked Read 6 Sent Done Done false
36570 9616 Acked Flush 0 Sent Done Done false
36571 9617 Acked Write 10 Sent Done Done false
36572 9618 Acked Read 7 Sent Done Done false
36573 9619 Acked Write 6 Sent Done Done false
36574 9620 Acked Read 3 Sent Done Done false
36575 9621 Acked Write 9 Sent Done Done false
36576 9622 Acked Write 4 Sent Done Done false
36577 9623 Acked Write 6 Sent Done Done false
36578 9624 Acked Read 3 Sent Done Done false
36579 9625 Acked Write 5 Sent Done Done false
36580 9626 Acked Read 7 Sent Done Done false
36581 9627 Acked Read 7 Sent Done Done false
36582 9628 Acked Write 9 Sent Done Done false
36583 9629 Acked Read 9 Sent Done Done false
36584 9630 Acked Read 1 Sent Done Done false
36585 9631 Acked Write 1 Sent Done Done false
36586 9632 Acked Flush 0 Sent Done Done false
36587 9633 Acked Read 3 Sent Done Done false
36588 9634 Acked Write 3 Sent Done Done false
36589 9635 Acked Write 3 Sent Done Done false
36590 9636 Acked Read 4 Sent Done Done false
36591 9637 Acked Read 2 Sent Done Done false
36592 9638 Acked Read 1 Sent Done Done false
36593 9639 Acked Read 1 Sent Done Done false
36594 9640 Acked Write 3 Sent Done Done false
36595 9641 Acked Flush 0 Sent Done Done false
36596 9642 Acked Read 10 Sent Done Done false
36597 9643 Acked Write 5 Sent Done Done false
36598 9644 Acked Read 3 Sent Done Done false
36599 9645 Acked Read 4 Sent Done Done false
36600 9646 Acked Write 7 Sent Done Done false
36601 9647 Acked Read 8 Sent Done Done false
36602 9648 Acked Read 2 Sent Done Done false
36603 9649 Acked Read 1 Sent Done Done false
36604 9650 Acked Flush 0 Sent Done Done false
36605 9651 Acked Read 3 Sent Done Done false
36606 9652 Acked Flush 0 Sent Done Done false
36607 9653 Acked Read 9 Sent Done Done false
36608 9654 Acked Read 1 Sent Done Done false
36609 9655 Acked Read 9 Sent Done Done false
36610 9656 Acked Write 4 Sent Done Done false
36611 9657 Acked Read 9 Sent Done Done false
36612 9658 Acked Read 10 Sent Done Done false
36613 9659 Acked Read 4 Sent Done Done false
36614 9660 Acked Write 10 Sent Done Done false
36615 9661 Acked Read 4 Sent Done Done false
36616 9662 Acked Write 10 Sent Done Done false
36617 9663 Acked Read 3 Sent Done Done false
36618 9664 Acked Write 2 Sent Done Done false
36619 9665 Acked Write 3 Sent Done Done false
36620 9666 Acked Read 4 Sent Done Done false
36621 9667 Acked Write 7 Sent Done Done false
36622 9668 Acked Write 6 Sent Done Done false
36623 9669 Acked Write 2 Sent Done Done false
36624 9670 Acked Flush 0 Sent Done Done false
36625 9671 Acked Flush 0 Sent Done Done false
36626 9672 Acked Write 5 Sent Done Done false
36627 9673 Acked Flush 0 Sent Done Done false
36628 9674 Acked Flush 0 Sent Done Done false
36629 9675 Acked Read 1 Sent Done Done false
36630 9676 Acked Flush 0 Sent Done Done false
36631 9677 Acked Read 2 Sent Done Done false
36632 9678 Acked Read 10 Sent Done Done false
36633 9679 Acked Read 1 Sent Done Done false
36634 9680 Acked Flush 0 Sent Done Done false
36635 9681 Acked Write 2 Sent Done Done false
36636 9682 Acked Read 7 Sent Done Done false
36637 9683 Acked Read 9 Sent Done Done false
36638 9684 Acked Read 7 Sent Done Done false
36639 9685 Acked Read 7 Sent Done Done false
36640 9686 Acked Write 4 Sent Done Done false
36641 9687 Acked Write 9 Sent Done Done false
36642 9688 Acked Read 3 Sent Done Done false
36643 9689 Acked Write 2 Sent Done Done false
36644 9690 Acked Read 4 Sent Done Done false
36645 9691 Acked Read 5 Sent Done Done false
36646 9692 Acked Write 1 Sent Done Done false
36647 9693 Acked Write 6 Sent Done Done false
36648 9694 Acked Read 9 Sent Done Done false
36649 9695 Acked Flush 0 Sent Done Done false
36650 9696 Acked Write 10 Sent Done Done false
36651 9697 Acked Write 4 Sent Done Done false
36652 9698 Acked Read 8 Sent Done Done false
36653 9699 Acked Write 2 Sent Done Done false
36654 9700 Acked Write 2 Sent Done Done false
36655 9701 Acked Write 7 Sent Done Done false
36656 9702 Acked Write 7 Sent Done Done false
36657 9703 Acked Write 10 Sent Done Done false
36658 9704 Acked Read 4 Sent Done Done false
36659 9705 Acked Read 8 Sent Done Done false
36660 9706 Acked Write 1 Sent Done Done false
36661 9707 Acked Write 4 Sent Done Done false
36662 9708 Acked Read 9 Sent Done Done false
36663 9709 Acked Read 4 Sent Done Done false
36664 9710 Acked Read 4 Sent Done Done false
36665 9711 Acked Write 8 Sent Done Done false
36666 9712 Acked Flush 0 Sent Done Done false
36667 9713 Acked Write 5 Sent Done Done false
36668 9714 Acked Read 3 Sent Done Done false
36669 9715 Acked Read 5 Sent Done Done false
36670 9716 Acked Read 2 Sent Done Done false
36671 9717 Acked Read 9 Sent Done Done false
36672 9718 Acked Write 3 Sent Done Done false
36673 9719 Acked Flush 0 Sent Done Done false
36674 9720 Acked Read 5 Sent Done Done false
36675 9721 Acked Write 2 Sent Done Done false
36676 9722 Acked Read 1 Sent Done Done false
36677 9723 Acked Write 8 Sent Done Done false
36678 9724 Acked Flush 0 Sent Done Done false
36679 9725 Acked Write 4 Sent Done Done false
36680 9726 Acked Write 6 Sent Done Done false
36681 9727 Acked Read 2 Sent Done Done false
36682 9728 Acked Write 5 Sent Done Done false
36683 9729 Acked Read 7 Sent Done Done false
36684 9730 Acked Read 3 Sent Done Done false
36685 9731 Acked Write 5 Sent Done Done false
36686 9732 Acked Write 7 Sent Done Done false
36687 9733 Acked Read 8 Sent Done Done false
36688 9734 Acked Write 1 Sent Done Done false
36689 9735 Acked Read 2 Sent Done Done false
36690 9736 Acked Read 3 Sent Done Done false
36691 9737 Acked Write 9 Sent Done Done false
36692 9738 Acked Flush 0 Sent Done Done false
36693 9739 Acked Write 1 Sent Done Done false
36694 9740 Acked Write 1 Sent Done Done false
36695 9741 Acked Read 9 Sent Done Done false
36696 9742 Acked Flush 0 Sent Done Done false
36697 9743 Acked Read 10 Sent Done Done false
36698 9744 Acked Flush 0 Sent Done Done false
36699 9745 Acked Read 9 Sent Done Done false
36700 9746 Acked Write 5 Sent Done Done false
36701 9747 Acked Write 6 Sent Done Done false
36702 9748 Acked Write 5 Sent Done Done false
36703 9749 Acked Read 7 Sent Done Done false
36704 9750 Acked Write 3 Sent Done Done false
36705 9751 Acked Read 7 Sent Done Done false
36706 9752 Acked Write 8 Sent Done Done false
36707 9753 Acked Read 4 Sent Done Done false
36708 9754 Acked Write 2 Sent Done Done false
36709 9755 Acked Flush 0 Sent Done Done false
36710 STATES DS:0 DS:1 DS:2 TOTAL
36711 Sent 302 0 0 302
36712 Done 0 302 302 604
36713 Skipped 0 0 0 0
36714 Error 0 0 0 0
36715 Last Flush: 9453 9755 9755
36716 Downstairs last five completed: 9453 9452 9451 9450 9449
36717 Upstairs last five completed: 9755 9754 9753 9752 9751
36718 CLIENT: Up:0 ds:302 act:2
367192024-11-15T02:05:08.811ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
367202024-11-15T02:05:08.811ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:52694) = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
367212024-11-15T02:05:08.812ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
367222024-11-15T02:05:08.813ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
367232024-11-15T02:05:08.813ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
367242024-11-15T02:05:08.813ZINFOcrucible: send last flush ID to this DS: 9453 = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
367252024-11-15T02:05:08.813ZINFOcrucible: Replied this last flush ID: 9453 = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
367262024-11-15T02:05:08.813ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
367272024-11-15T02:05:08.813ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
367282024-11-15T02:05:08.813ZINFOcrucible: [0] client re-new 302 jobs since flush 9453 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
367292024-11-15T02:05:08.813ZINFOcrucible: [0] Replayed 302 jobs since flush: 9453 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
36730 ----------------------------------------------------------------
36731 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
36732 Downstairs last five completed: 9755 9754 9753 9752 9751
36733 Upstairs last five completed: 9755 9754 9753 9752 9751
36734 CLIENT: Up:0 ds:0 act:3
36735 Replay: All jobs finished
36736 Main task has work to do, go find it
36737 got dsc Stop(0)
36738 stop 0
36739 Nov 15 02:05:11.046 INFO request completed, latency_us: 383, response_code: 204[8810] Got stop action so:false kr:false
36740 , uri: /stop/cid/0, method: POST, req_id: 0cc8a252-642f-4099-adeb-1bf2f6f876e3, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
367412024-11-15T02:05:11.047ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
36742 Nov 15 02:05:11.047 INFO request completed, latency_us: 114, response_code: {"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2024-11-15T02:05:11.047401304Z","hostname":"w-01JCPRHT5F2MPA44KVJHM3S2PB","pid":1205,"":"io task","client":"0","":"downstairs","session_id":"971dcb2e-6891-4ecf-b367-1e04e52065ee"}
36743 200, uri: /state/cid/0, method: GET, req_id: 28e0c44b-ac92-4348-919f-6e31f9b5d5b6, remote_addr: 127.0{"msg":"downstairs task for 0 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2024-11-15T02:05:11.047432092Z","hostname":"w-01JCPRHT5F2MPA44KVJHM3S2PB","pid":1205,"session_id":"971dcb2e-6891-4ecf-b367-1e04e52065ee"}
36744 .0.1:40008, local_addr: 127.0.{"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-15T02:05:11.047456881Z","hostname":"w-01JCPRHT5F2MPA44KVJHM3S2PB","pid":1205,"client":"0","":"downstairs","session_id":"971dcb2e-6891-4ecf-b367-1e04e52065ee"}
36745 0.1:9998
367462024-11-15T02:05:11.047ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
367472024-11-15T02:05:11.047ZINFOcrucible: client task is exiting = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
36748 [8810] Exited with: ExitStatus(unix_wait_status(9))
36749 [8810][0] reports Exit
36750 Nov 15 02:05:15.049 INFO request completed, latency_us: 217, response_code: 200, uri: /state/cid/0, method: GET, req_id: 56c8d3fb-46a7-4db6-81f3-17fc3ef59bc6, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
36751 001/300 Write block 2640 len 12288 data: 5 4 6
36752 002/300 Read block 1966 len 40960
36753 003/300 Read block 4988 len 16384
36754 004/300 Read block 2303 len 20480
36755 005/300 Write block 1484 len 8192 data: 4 4
36756 006/300 Read block 1005 len 40960
36757 007/300 Write block 856 len 24576 data: 6 5 5 6 4 5
36758 008/300 Write block 2073 len 8192 data: 6 6
36759 009/300 Read block 1682 len 12288
36760 010/300 Read block 4016 len 4096
36761 011/300 Read block 3708 len 28672
36762 012/300 Flush
36763 013/300 Flush
36764 014/300 Write block 3538 len 24576 data: 7 6 5 4 4 4
36765 015/300 Read block 1925 len 20480
36766 016/300 Read block 3320 len 24576
36767 017/300 Read block 1998 len 4096
36768 018/300 Read block 307 len 40960
36769 019/300 Read block 414 len 12288
36770 020/300 Read block 1416 len 8192
36771 021/300 Read block 4594 len 16384
36772 022/300 Write block 3385 len 40960 data: 6 6 4 4 3 3 4 7 7 6
36773 023/300 Read block 4760 len 16384
36774 024/300 Write block 2460 len 16384 data: 10 10 10 10
36775 025/300 Flush
36776 026/300 Write block 4180 len 16384 data: 12 12 10 10
36777 027/300 Read block 2406 len 4096
36778 028/300 Read block 1820 len 40960
36779 029/300 Write block 4707 len 16384 data: 9 9 9 9
36780 030/300 Read block 906 len 40960
36781 031/300 Write block 2784 len 36864 data: 5 5 4 4 3 4 5 5 4
36782 032/300 Write block 2583 len 4096 data: 2
36783 033/300 Write block 4977 len 32768 data: 4 4 3 3 3 3 3 3
36784 034/300 Write block 3718 len 12288 data: 3 4 4
36785 035/300 Read block 823 len 24576
36786 036/300 Write block 1802 len 4096 data: 6
36787 037/300 Read block 380 len 4096
36788 038/300 Read block 4121 len 20480
36789 039/300 Read block 3905 len 28672
36790 040/300 Read block 4393 len 36864
36791 041/300 Write block 4850 len 20480 data: 4 4 5 3 3
36792 042/300 Read block 1771 len 12288
36793 043/300 Read block 2739 len 40960
36794 044/300 Read block 2435 len 36864
36795 045/300 Read block 175 len 28672
36796 046/300 Flush
36797 047/300 Flush
36798 048/300 Write block 786 len 36864 data: 10 10 9 9 10 9 7 7 8
36799 049/300 Read block 3187 len 40960
36800 050/300 Write block 1495 len 4096 data: 3
36801 051/300 Read block 3177 len 28672
36802 052/300 Read block 4259 len 28672
36803 053/300 Read block 1242 len 12288
36804 054/300 Read block 2531 len 24576
36805 055/300 Read block 4305 len 12288
36806 056/300 Write block 793 len 12288 data: 8 9 9
36807 057/300 Read block 618 len 36864
36808 058/300 Read block 1905 len 4096
36809 059/300 Read block 2829 len 12288
36810 060/300 Write block 256 len 20480 data: 4 4 5 5 7
36811 061/300 Write block 1617 len 32768 data: 5 5 5 6 5 5 4 5
36812 062/300 Write block 3167 len 36864 data: 6 6 5 6 6 5 5 4 3
36813 063/300 Read block 3856 len 20480
36814 064/300 Read block 4890 len 28672
36815 065/300 Write block 2456 len 16384 data: 10 11 9 9
36816 066/300 Read block 3342 len 28672
36817 067/300 Read block 2629 len 32768
36818 068/300 Write block 1812 len 20480 data: 5 4 4 4 5
36819 069/300 Read block 286 len 24576
36820 070/300 Read block 3211 len 36864
36821 071/300 Read block 3667 len 28672
36822 072/300 Read block 3564 len 32768
36823 073/300 Write block 174 len 20480 data: 5 4 5 5 4
36824 074/300 Read block 4764 len 20480
36825 075/300 Read block 1101 len 16384
36826 076/300 Write block 1560 len 16384 data: 9 10 11 10
36827 077/300 Read block 782 len 24576
36828 078/300 Read block 1479 len 32768
36829 079/300 Write block 3063 len 36864 data: 3 3 3 2 2 2 2 3 3
36830 080/300 Read block 2987 len 40960
36831 081/300 Read block 4893 len 40960
36832 082/300 Read block 4408 len 4096
36833 083/300 Read block 658 len 28672
36834 084/300 Read block 4445 len 40960
36835 085/300 Flush
36836 086/300 Read block 4913 len 40960
36837 087/300 Write block 3543 len 28672 data: 5 5 7 7 6 8 8
36838 088/300 Write block 3913 len 24576 data: 5 5 3 3 2 3
36839 089/300 Write block 1716 len 28672 data: 4 5 4 4 4 4 4
36840 090/300 Write block 2682 len 36864 data: 6 5 4 6 8 7 8 6 5
36841 091/300 Read block 2467 len 24576
36842 092/300 Read block 2913 len 4096
36843 093/300 Read block 2506 len 40960
36844 094/300 Read block 2388 len 40960
36845 095/300 Write block 2060 len 32768 data: 9 10 11 10 8 6 5 6
36846 096/300 Read block 4636 len 28672
36847 097/300 Write block 3087 len 8192 data: 3 5
36848 098/300 Read block 3373 len 32768
36849 099/300 Read block 4991 len 12288
36850 100/300 Flush
36851 101/300 Write block 409 len 20480 data: 4 5 4 5 6
36852 102/300 Write block 4149 len 24576 data: 7 7 8 6 6 5
36853 103/300 Read block 1818 len 32768
36854 104/300 Write block 4548 len 36864 data: 6 6 6 7 6 6 6 5 5
36855 105/300 Write block 4079 len 8192 data: 6 6
36856 106/300 Write block 1370 len 40960 data: 6 6 6 7 6 8 7 7 7 9
36857 107/300 Write block 642 len 20480 data: 4 3 3 3 3
36858 108/300 Read block 4481 len 20480
36859 109/300 Write block 2496 len 28672 data: 7 7 8 7 7 5 4
36860 110/300 Read block 3277 len 32768
36861 111/300 Read block 501 len 24576
36862 112/300 Write block 4377 len 20480 data: 4 5 4 4 4
36863 113/300 Read block 1073 len 32768
36864 114/300 Read block 2593 len 32768
36865 115/300 Flush
36866 116/300 Flush
36867 117/300 Read block 1549 len 28672
36868 118/300 Read block 2842 len 28672
36869 119/300 Write block 2870 len 4096 data: 7
36870 120/300 Write block 421 len 8192 data: 7 7
36871 121/300 Flush
36872 122/300 Write block 3313 len 36864 data: 4 3 2 3 5 5 5 5 5
36873 123/300 Read block 3299 len 12288
36874 124/300 Read block 3667 len 24576
36875 125/300 Read block 3644 len 8192
36876 126/300 Read block 1045 len 40960
36877 127/300 Read block 1056 len 16384
36878 128/300 Write block 2846 len 12288 data: 5 6 7
36879 129/300 Write block 26 len 8192 data: 7 6
36880 130/300 Read block 521 len 20480
36881 131/300 Read block 294 len 32768
36882 132/300 Write block 2382 len 32768 data: 6 8 9 9 9 9 7 9
36883 133/300 Write block 1135 len 24576 data: 4 3 3 4 4 6
36884 134/300 Write block 2330 len 36864 data: 2 3 5 5 5 6 10 9 7
36885 135/300 Read block 4635 len 32768
36886 136/300 Write block 234 len 36864 data: 9 9 10 8 7 6 6 5 4
36887 137/300 Read block 4051 len 32768
36888 138/300 Write block 3983 len 28672 data: 4 4 4 5 7 5 6
36889 139/300 Write block 417 len 16384 data: 6 8 8 7
36890 140/300 Write block 1383 len 8192 data: 6 6
36891 141/300 Write block 2974 len 12288 data: 5 5 5
36892 142/300 Write block 90 len 8192 data: 5 4
36893 143/300 Write block 924 len 28672 data: 3 4 4 4 5 5 7
36894 144/300 Read block 2195 len 12288
36895 145/300 Read block 4039 len 24576
36896 146/300 Read block 250 len 8192
36897 147/300 Read block 2065 len 40960
36898 148/300 Write block 1336 len 16384 data: 6 4 5 7
36899 149/300 Read block 2077 len 32768
36900 150/300 Read block 3251 len 16384
36901 151/300 Read block 4512 len 24576
36902 152/300 Write block 4392 len 4096 data: 7
36903 153/300 Write block 137 len 36864 data: 5 5 4 5 5 3 2 3 2
36904 154/300 Write block 1236 len 16384 data: 6 5 5 5
36905 155/300 Flush
36906 156/300 Read block 4287 len 40960
36907 157/300 Write block 404 len 24576 data: 4 4 5 4 4 5
36908 158/300 Read block 628 len 24576
36909 159/300 Read block 4164 len 12288
36910 160/300 Write block 4656 len 16384 data: 7 7 8 7
36911 161/300 Read block 970 len 32768
36912 162/300 Write block 20 len 20480 data: 7 6 7 8 6
36913 163/300 Flush
36914 164/300 Read block 3400 len 12288
36915 165/300 Write block 2995 len 16384 data: 5 3 2 3
36916 166/300 Write block 570 len 8192 data: 6 5
36917 167/300 Read block 2838 len 4096
36918 168/300 Read block 4621 len 32768
36919 169/300 Read block 4755 len 24576
36920 170/300 Read block 2406 len 40960
36921 171/300 Flush
36922 172/300 Write block 3284 len 28672 data: 9 7 7 5 5 5 5
36923 173/300 Write block 2147 len 20480 data: 5 7 7 8 7
36924 174/300 Write block 103 len 16384 data: 5 5 5 4
36925 175/300 Write block 1940 len 16384 data: 8 8 6 4
36926 176/300 Flush
36927 177/300 Read block 842 len 8192
36928 178/300 Read block 603 len 24576
36929 179/300 Read block 1501 len 8192
36930 180/300 Read block 4719 len 12288
36931 181/300 Read block 3419 len 24576
36932 182/300 Write block 68 len 24576 data: 6 9 9 10 9 7
36933 183/300 Read block 245 len 28672
36934 184/300 Write block 4967 len 20480 data: 5 5 5 6 4
36935 185/300 Read block 2676 len 36864
36936 186/300 Read block 4361 len 40960
36937 187/300 Write block 4469 len 16384 data: 5 6 4 4
36938 188/300 Flush
36939 189/300 Read block 1272 len 4096
36940 190/300 Read block 4108 len 24576
36941 191/300 Write block 3609 len 24576 data: 4 3 3 3 2 4
36942 192/300 Read block 1579 len 40960
36943 193/300 Read block 3852 len 4096
36944 194/300 Read block 3149 len 16384
36945 195/300 Read block 3752 len 24576
36946 196/300 Write block 4087 len 16384 data: 4 4 4 3
36947 197/300 Read block 4402 len 8192
36948 198/300 Write block 3194 len 8192 data: 4 3
36949 199/300 Write block 2437 len 28672 data: 4 4 5 5 7 6 4
36950 200/300 Read block 1526 len 12288
36951 201/300 Write block 2823 len 16384 data: 9 9 8 8
36952 202/300 Write block 1521 len 40960 data: 4 3 3 5 5 5 4 5 5 6
36953 203/300 Read block 1357 len 8192
36954 204/300 Read block 3775 len 20480
36955 205/300 Write block 1491 len 20480 data: 7 7 5 4 4
36956 206/300 Read block 2576 len 20480
36957 207/300 Read block 1359 len 4096
36958 208/300 Write block 2262 len 28672 data: 8 8 7 8 8 6 4
36959 209/300 Read block 1719 len 4096
36960 210/300 Write block 3011 len 28672 data: 8 7 7 7 5 3 2
36961 211/300 Write block 2537 len 8192 data: 4 5
36962 212/300 Write block 600 len 40960 data: 5 4 4 4 4 5 5 3 4 4
36963 213/300 Write block 3095 len 24576 data: 4 4 4 5 5 4
36964 214/300 Read block 2302 len 8192
36965 215/300 Read block 3738 len 12288
36966 216/300 Read block 4540 len 24576
36967 217/300 Flush
36968 218/300 Write block 4100 len 12288 data: 4 4 5
36969 219/300 Write block 3183 len 4096 data: 8
36970 220/300 Read block 2716 len 4096
36971 221/300 Write block 4592 len 36864 data: 7 8 7 7 7 5 4 6 7
36972 222/300 Write block 4569 len 20480 data: 8 7 6 5 4
36973 223/300 Read block 2049 len 24576
36974 224/300 Flush
36975 225/300 Write block 2306 len 20480 data: 7 9 10 10 8
36976 226/300 Read block 1534 len 20480
36977 227/300 Write block 2294 len 4096 data: 6
36978 228/300 Read block 879 len 20480
36979 229/300 Write block 1381 len 28672 data: 9 8 7 7 4 4 4
36980 230/300 Read block 3779 len 24576
36981 231/300 Read block 748 len 20480
36982 232/300 Write block 207 len 36864 data: 7 7 7 7 8 6 6 7 6
36983 233/300 Write block 561 len 40960 data: 7 7 6 4 3 4 5 6 4 7
36984 234/300 Read block 2239 len 36864
36985 235/300 Write block 3897 len 16384 data: 9 12 11 12
36986 236/300 Write block 3259 len 4096 data: 6
36987 237/300 Write block 4094 len 16384 data: 8 8 8 6
36988 238/300 Flush
36989 239/300 Read block 2044 len 24576
36990 240/300 Flush
36991 241/300 Read block 615 len 4096
36992 242/300 Read block 2682 len 28672
36993 243/300 Read block 2917 len 16384
36994 244/300 Write block 41 len 16384 data: 6 6 6 5
36995 245/300 Read block 3578 len 8192
36996 246/300 Write block 1449 len 16384 data: 4 5 6 5
36997 247/300 Write block 3744 len 32768 data: 6 6 7 7 7 7 8 9
36998 248/300 Write block 3958 len 24576 data: 8 9 8 7 8 7
36999 249/300 Write block 1164 len 8192 data: 5 4
37000 250/300 Write block 2828 len 16384 data: 8 10 9 7
37001 251/300 Flush
37002 252/300 Read block 953 len 4096
37003 253/300 Write block 1214 len 24576 data: 4 4 4 3 4 4
37004 254/300 Write block 4861 len 36864 data: 10 12 13 12 12 11 8 6 6
37005 255/300 Write block 3084 len 16384 data: 5 5 4 4
37006 256/300 Read block 2680 len 28672
37007 257/300 Read block 3706 len 20480
37008 258/300 Read block 2440 len 4096
37009 259/300 Write block 1125 len 24576 data: 6 5 5 5 6 7
37010 260/300 Write block 2622 len 24576 data: 6 6 5 3 3 3
37011 261/300 Write block 2507 len 8192 data: 3 4
37012 262/300 Write block 2046 len 8192 data: 5 5
37013 263/300 Flush
37014 264/300 Read block 1186 len 24576
37015 265/300 Write block 331 len 28672 data: 7 7 10 8 7 6 7
37016 266/300 Write block 138 len 12288 data: 6 5 6
37017 267/300 Write block 1813 len 20480 data: 5 5 5 6 5
37018 268/300 Write block 2738 len 28672 data: 4 5 5 5 6 6 6
37019 269/300 Read block 705 len 36864
37020 270/300 Write block 4465 len 12288 data: 6 5 6
37021 271/300 Flush
37022 272/300 Flush
37023 273/300 Write block 285 len 4096 data: 5
37024 274/300 Read block 716 len 12288
37025 275/300 Read block 625 len 16384
37026 276/300 Read block 1691 len 36864
37027 277/300 Read block 2637 len 36864
37028 278/300 Read block 4599 len 40960
37029 279/300 Read block 4750 len 24576
37030 280/300 Write block 1906 len 16384 data: 5 5 5 4
37031 281/300 Read block 2805 len 40960
37032 282/300 Write block 2651 len 8192 data: 7 4
37033 283/300 Write block 2240 len 40960 data: 8 7 6 5 6 6 6 6 6 6
37034 284/300 Read block 180 len 12288
37035 285/300 Read block 1015 len 28672
37036 286/300 Write block 2346 len 40960 data: 7 6 4 4 4 3 3 2 2 2
37037 287/300 Read block 955 len 8192
37038 288/300 Write block 1337 len 16384 data: 5 6 8 8
37039 289/300 Read block 4691 len 16384
37040 290/300 Read block 2522 len 8192
37041 291/300 Read block 4773 len 24576
37042 292/300 Read block 4615 len 8192
37043 293/300 Read block 152 len 8192
37044 294/300 Write block 3634 len 20480 data: 4 4 4 4 4
37045 295/300 Read block 1876 len 20480
37046 296/300 Write block 604 len 40960 data: 5 6 6 4 5 5 5 4 4 4
37047 297/300 Read block 2233 len 32768
37048 298/300 Read block 1488 len 32768
37049 299/300 Write block 1291 len 36864 data: 8 7 7 7 8 8 8 7 8
37050 300/300 Write block 3547 len 32768 data: 7 9 9 8 8 10 9 6
37051 301/300 Write block 4094 len 20480 data: 9 9 9 7 5
37052 Main task has work to do, go find it
37053 got dsc Start(0)
37054 start 0
37055 Nov 15 02:05:15.289 INFO request completed, latency_us: 235, response_code: 204, uri: /start/cid/0, method: POST, req_id: f446626e-f94c-47d7-89a8-23840578cdca, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
37056 Replay: started 0, returned:Ok(())
37057 ----------------------------------------------------------------
37058 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
37059 States: Offline Active Active
37060 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
37061 9756 Acked Write 3 Sent Done Done false
37062 9757 Acked Read 10 Sent Done Done false
37063 9758 Acked Read 4 Sent Done Done false
37064 9759 Acked Read 5 Sent Done Done false
37065 9760 Acked Write 2 Sent Done Done false
37066 9761 Acked Read 10 Sent Done Done false
37067 9762 Acked Write 6 Sent Done Done false
37068 9763 Acked Write 2 Sent Done Done false
37069 9764 Acked Read 3 Sent Done Done false
37070 9765 Acked Read 1 Sent Done Done false
37071 9766 Acked Read 7 Sent Done Done false
37072 9767 Acked Flush 0 Sent Done Done false
37073 9768 Acked Flush 0 Sent Done Done false
37074 9769 Acked Write 6 Sent Done Done false
37075 9770 Acked Read 5 Sent Done Done false
37076 9771 Acked Read 6 Sent Done Done false
37077 9772 Acked Read 1 Sent Done Done false
37078 9773 Acked Read 10 Sent Done Done false
37079 9774 Acked Read 3 Sent Done Done false
37080 9775 Acked Read 2 Sent Done Done false
37081 9776 Acked Read 4 Sent Done Done false
37082 9777 Acked Write 10 Sent Done Done false
37083 9778 Acked Read 4 Sent Done Done false
37084 9779 Acked Write 4 Sent Done Done false
37085 9780 Acked Flush 0 Sent Done Done false
37086 9781 Acked Write 4 Sent Done Done false
37087 9782 Acked Read 1 Sent Done Done false
37088 9783 Acked Read 10 Sent Done Done false
37089 9784 Acked Write 4 Sent Done Done false
37090 9785 Acked Read 10 Sent Done Done false
37091 9786 Acked Write 9 Sent Done Done false
37092 9787 Acked Write 1 Sent Done Done false
37093 9788 Acked Write 8 Sent Done Done false
37094 9789 Acked Write 3 Sent Done Done false
37095 9790 Acked Read 6 Sent Done Done false
37096 9791 Acked Write 1 Sent Done Done false
37097 9792 Acked Read 1 Sent Done Done false
37098 9793 Acked Read 5 Sent Done Done false
37099 9794 Acked Read 7 Sent Done Done false
37100 9795 Acked Read 9 Sent Done Done false
37101 9796 Acked Write 5 Sent Done Done false
37102 9797 Acked Read 3 Sent Done Done false
37103 9798 Acked Read 10 Sent Done Done false
37104 9799 Acked Read 9 Sent Done Done false
37105 9800 Acked Read 7 Sent Done Done false
37106 9801 Acked Flush 0 Sent Done Done false
37107 9802 Acked Flush 0 Sent Done Done false
37108 9803 Acked Write 9 Sent Done Done false
37109 9804 Acked Read 10 Sent Done Done false
37110 9805 Acked Write 1 Sent Done Done false
37111 9806 Acked Read 7 Sent Done Done false
37112 9807 Acked Read 7 Sent Done Done false
37113 9808 Acked Read 3 Sent Done Done false
37114 9809 Acked Read 6 Sent Done Done false
37115 9810 Acked Read 3 Sent Done Done false
37116 9811 Acked Write 3 Sent Done Done false
37117 9812 Acked Read 9 Sent Done Done false
37118 9813 Acked Read 1 Sent Done Done false
37119 9814 Acked Read 3 Sent Done Done false
37120 9815 Acked Write 5 Sent Done Done false
37121 9816 Acked Write 8 Sent Done Done false
37122 9817 Acked Write 9 Sent Done Done false
37123 9818 Acked Read 5 Sent Done Done false
37124 9819 Acked Read 7 Sent Done Done false
37125 9820 Acked Write 4 Sent Done Done false
37126 9821 Acked Read 7 Sent Done Done false
37127 9822 Acked Read 8 Sent Done Done false
37128 9823 Acked Write 5 Sent Done Done false
37129 9824 Acked Read 6 Sent Done Done false
37130 9825 Acked Read 9 Sent Done Done false
37131 9826 Acked Read 7 Sent Done Done false
37132 9827 Acked Read 8 Sent Done Done false
37133 9828 Acked Write 5 Sent Done Done false
37134 9829 Acked Read 5 Sent Done Done false
37135 9830 Acked Read 4 Sent Done Done false
37136 9831 Acked Write 4 Sent Done Done false
37137 9832 Acked Read 6 Sent Done Done false
37138 9833 Acked Read 8 Sent Done Done false
37139 9834 Acked Write 9 Sent Done Done false
37140 9835 Acked Read 10 Sent Done Done false
37141 9836 Acked Read 10 Sent Done Done false
37142 9837 Acked Read 1 Sent Done Done false
37143 9838 Acked Read 7 Sent Done Done false
37144 9839 Acked Read 10 Sent Done Done false
37145 9840 Acked Flush 0 Sent Done Done false
37146 9841 Acked Read 10 Sent Done Done false
37147 9842 Acked Write 7 Sent Done Done false
37148 9843 Acked Write 6 Sent Done Done false
37149 9844 Acked Write 7 Sent Done Done false
37150 9845 Acked Write 9 Sent Done Done false
37151 9846 Acked Read 6 Sent Done Done false
37152 9847 Acked Read 1 Sent Done Done false
37153 9848 Acked Read 10 Sent Done Done false
37154 9849 Acked Read 10 Sent Done Done false
37155 9850 Acked Write 8 Sent Done Done false
37156 9851 Acked Read 7 Sent Done Done false
37157 9852 Acked Write 2 Sent Done Done false
37158 9853 Acked Read 8 Sent Done Done false
37159 9854 Acked Read 3 Sent Done Done false
37160 9855 Acked Flush 0 Sent Done Done false
37161 9856 Acked Write 5 Sent Done Done false
37162 9857 Acked Write 6 Sent Done Done false
37163 9858 Acked Read 8 Sent Done Done false
37164 9859 Acked Write 9 Sent Done Done false
37165 9860 Acked Write 2 Sent Done Done false
37166 9861 Acked Write 10 Sent Done Done false
37167 9862 Acked Write 5 Sent Done Done false
37168 9863 Acked Read 5 Sent Done Done false
37169 9864 Acked Write 7 Sent Done Done false
37170 9865 Acked Read 8 Sent Done Done false
37171 9866 Acked Read 6 Sent Done Done false
37172 9867 Acked Write 5 Sent Done Done false
37173 9868 Acked Read 8 Sent Done Done false
37174 9869 Acked Read 8 Sent Done Done false
37175 9870 Acked Flush 0 Sent Done Done false
37176 9871 Acked Flush 0 Sent Done Done false
37177 9872 Acked Read 7 Sent Done Done false
37178 9873 Acked Read 7 Sent Done Done false
37179 9874 Acked Write 1 Sent Done Done false
37180 9875 Acked Write 2 Sent Done Done false
37181 9876 Acked Flush 0 Sent Done Done false
37182 9877 Acked Write 9 Sent Done Done false
37183 9878 Acked Read 3 Sent Done Done false
37184 9879 Acked Read 6 Sent Done Done false
37185 9880 Acked Read 2 Sent Done Done false
37186 9881 Acked Read 10 Sent Done Done false
37187 9882 Acked Read 4 Sent Done Done false
37188 9883 Acked Write 3 Sent Done Done false
37189 9884 Acked Write 2 Sent Done Done false
37190 9885 Acked Read 5 Sent Done Done false
37191 9886 Acked Read 8 Sent Done Done false
37192 9887 Acked Write 8 Sent Done Done false
37193 9888 Acked Write 6 Sent Done Done false
37194 9889 Acked Write 9 Sent Done Done false
37195 9890 Acked Read 8 Sent Done Done false
37196 9891 Acked Write 9 Sent Done Done false
37197 9892 Acked Read 8 Sent Done Done false
37198 9893 Acked Write 7 Sent Done Done false
37199 9894 Acked Write 4 Sent Done Done false
37200 9895 Acked Write 2 Sent Done Done false
37201 9896 Acked Write 3 Sent Done Done false
37202 9897 Acked Write 2 Sent Done Done false
37203 9898 Acked Write 7 Sent Done Done false
37204 9899 Acked Read 3 Sent Done Done false
37205 9900 Acked Read 6 Sent Done Done false
37206 9901 Acked Read 2 Sent Done Done false
37207 9902 Acked Read 10 Sent Done Done false
37208 9903 Acked Write 4 Sent Done Done false
37209 9904 Acked Read 8 Sent Done Done false
37210 9905 Acked Read 4 Sent Done Done false
37211 9906 Acked Read 6 Sent Done Done false
37212 9907 Acked Write 1 Sent Done Done false
37213 9908 Acked Write 9 Sent Done Done false
37214 9909 Acked Write 4 Sent Done Done false
37215 9910 Acked Flush 0 Sent Done Done false
37216 9911 Acked Read 10 Sent Done Done false
37217 9912 Acked Write 6 Sent Done Done false
37218 9913 Acked Read 6 Sent Done Done false
37219 9914 Acked Read 3 Sent Done Done false
37220 9915 Acked Write 4 Sent Done Done false
37221 9916 Acked Read 8 Sent Done Done false
37222 9917 Acked Write 5 Sent Done Done false
37223 9918 Acked Flush 0 Sent Done Done false
37224 9919 Acked Read 3 Sent Done Done false
37225 9920 Acked Write 4 Sent Done Done false
37226 9921 Acked Write 2 Sent Done Done false
37227 9922 Acked Read 1 Sent Done Done false
37228 9923 Acked Read 8 Sent Done Done false
37229 9924 Acked Read 6 Sent Done Done false
37230 9925 Acked Read 10 Sent Done Done false
37231 9926 Acked Flush 0 Sent Done Done false
37232 9927 Acked Write 7 Sent Done Done false
37233 9928 Acked Write 5 Sent Done Done false
37234 9929 Acked Write 4 Sent Done Done false
37235 9930 Acked Write 4 Sent Done Done false
37236 9931 Acked Flush 0 Sent Done Done false
37237 9932 Acked Read 2 Sent Done Done false
37238 9933 Acked Read 6 Sent Done Done false
37239 9934 Acked Read 2 Sent Done Done false
37240 9935 Acked Read 3 Sent Done Done false
37241 9936 Acked Read 6 Sent Done Done false
37242 9937 Acked Write 6 Sent Done Done false
37243 9938 Acked Read 7 Sent Done Done false
37244 9939 Acked Write 5 Sent Done Done false
37245 9940 Acked Read 9 Sent Done Done false
37246 9941 Acked Read 10 Sent Done Done false
37247 9942 Acked Write 4 Sent Done Done false
37248 9943 Acked Flush 0 Sent Done Done false
37249 9944 Acked Read 1 Sent Done Done false
37250 9945 Acked Read 6 Sent Done Done false
37251 9946 Acked Write 6 Sent Done Done false
37252 9947 Acked Read 10 Sent Done Done false
37253 9948 Acked Read 1 Sent Done Done false
37254 9949 Acked Read 4 Sent Done Done false
37255 9950 Acked Read 6 Sent Done Done false
37256 9951 Acked Write 4 Sent Done Done false
37257 9952 Acked Read 2 Sent Done Done false
37258 9953 Acked Write 2 Sent Done Done false
37259 9954 Acked Write 7 Sent Done Done false
37260 9955 Acked Read 3 Sent Done Done false
37261 9956 Acked Write 4 Sent Done Done false
37262 9957 Acked Write 10 Sent Done Done false
37263 9958 Acked Read 2 Sent Done Done false
37264 9959 Acked Read 5 Sent Done Done false
37265 9960 Acked Write 5 Sent Done Done false
37266 9961 Acked Read 5 Sent Done Done false
37267 9962 Acked Read 1 Sent Done Done false
37268 9963 Acked Write 7 Sent Done Done false
37269 9964 Acked Read 1 Sent Done Done false
37270 9965 Acked Write 7 Sent Done Done false
37271 9966 Acked Write 2 Sent Done Done false
37272 9967 Acked Write 10 Sent Done Done false
37273 9968 Acked Write 6 Sent Done Done false
37274 9969 Acked Read 2 Sent Done Done false
37275 9970 Acked Read 3 Sent Done Done false
37276 9971 Acked Read 6 Sent Done Done false
37277 9972 Acked Flush 0 Sent Done Done false
37278 9973 Acked Write 3 Sent Done Done false
37279 9974 Acked Write 1 Sent Done Done false
37280 9975 Acked Read 1 Sent Done Done false
37281 9976 Acked Write 9 Sent Done Done false
37282 9977 Acked Write 5 Sent Done Done false
37283 9978 Acked Read 6 Sent Done Done false
37284 9979 Acked Flush 0 Sent Done Done false
37285 9980 Acked Write 5 Sent Done Done false
37286 9981 Acked Read 5 Sent Done Done false
37287 9982 Acked Write 1 Sent Done Done false
37288 9983 Acked Read 5 Sent Done Done false
37289 9984 Acked Write 7 Sent Done Done false
37290 9985 Acked Read 6 Sent Done Done false
37291 9986 Acked Read 5 Sent Done Done false
37292 9987 Acked Write 9 Sent Done Done false
37293 9988 Acked Write 10 Sent Done Done false
37294 9989 Acked Read 9 Sent Done Done false
37295 9990 Acked Write 4 Sent Done Done false
37296 9991 Acked Write 1 Sent Done Done false
37297 9992 Acked Write 4 Sent Done Done false
37298 9993 Acked Flush 0 Sent Done Done false
37299 9994 Acked Read 6 Sent Done Done false
37300 9995 Acked Flush 0 Sent Done Done false
37301 9996 Acked Read 1 Sent Done Done false
37302 9997 Acked Read 7 Sent Done Done false
37303 9998 Acked Read 4 Sent Done Done false
37304 9999 Acked Write 4 Sent Done Done false
37305 10000 Acked Read 2 Sent Done Done false
37306 10001 Acked Write 4 Sent Done Done false
37307 10002 Acked Write 8 Sent Done Done false
37308 10003 Acked Write 6 Sent Done Done false
37309 10004 Acked Write 2 Sent Done Done false
37310 10005 Acked Write 4 Sent Done Done false
37311 10006 Acked Flush 0 Sent Done Done false
37312 10007 Acked Read 1 Sent Done Done false
37313 10008 Acked Write 6 Sent Done Done false
37314 10009 Acked Write 9 Sent Done Done false
37315 10010 Acked Write 4 Sent Done Done false
37316 10011 Acked Read 7 Sent Done Done false
37317 10012 Acked Read 5 Sent Done Done false
37318 10013 Acked Read 1 Sent Done Done false
37319 10014 Acked Write 6 Sent Done Done false
37320 10015 Acked Write 6 Sent Done Done false
37321 10016 Acked Write 2 Sent Done Done false
37322 10017 Acked Write 2 Sent Done Done false
37323 10018 Acked Flush 0 Sent Done Done false
37324 10019 Acked Read 6 Sent Done Done false
37325 10020 Acked Write 7 Sent Done Done false
37326 10021 Acked Write 3 Sent Done Done false
37327 10022 Acked Write 5 Sent Done Done false
37328 10023 Acked Write 7 Sent Done Done false
37329 10024 Acked Read 9 Sent Done Done false
37330 10025 Acked Write 3 Sent Done Done false
37331 10026 Acked Flush 0 Sent Done Done false
37332 10027 Acked Flush 0 Sent Done Done false
37333 10028 Acked Write 1 Sent Done Done false
37334 10029 Acked Read 3 Sent Done Done false
37335 10030 Acked Read 4 Sent Done Done false
37336 10031 Acked Read 9 Sent Done Done false
37337 10032 Acked Read 9 Sent Done Done false
37338 10033 Acked Read 10 Sent Done Done false
37339 10034 Acked Read 6 Sent Done Done false
37340 10035 Acked Write 4 Sent Done Done false
37341 10036 Acked Read 10 Sent Done Done false
37342 10037 Acked Write 2 Sent Done Done false
37343 10038 Acked Write 10 Sent Done Done false
37344 10039 Acked Read 3 Sent Done Done false
37345 10040 Acked Read 7 Sent Done Done false
37346 10041 Acked Write 10 Sent Done Done false
37347 10042 Acked Read 2 Sent Done Done false
37348 10043 Acked Write 4 Sent Done Done false
37349 10044 Acked Read 4 Sent Done Done false
37350 10045 Acked Read 2 Sent Done Done false
37351 10046 Acked Read 6 Sent Done Done false
37352 10047 Acked Read 2 Sent Done Done false
37353 10048 Acked Read 2 Sent Done Done false
37354 10049 Acked Write 5 Sent Done Done false
37355 10050 Acked Read 5 Sent Done Done false
37356 10051 Acked Write 10 Sent Done Done false
37357 10052 Acked Read 8 Sent Done Done false
37358 10053 Acked Read 8 Sent Done Done false
37359 10054 Acked Write 9 Sent Done Done false
37360 10055 Acked Write 8 Sent Done Done false
37361 10056 Acked Write 5 Sent Done Done false
37362 STATES DS:0 DS:1 DS:2 TOTAL
37363 Sent 301 0 0 301
37364 Done 0 301 301 602
37365 Skipped 0 0 0 0
37366 Error 0 0 0 0
37367 Last Flush: 9755 10027 10027
37368 Downstairs last five completed: 9755 9754 9753 9752 9751
37369 Upstairs last five completed: 10056 10055 10054 10053 10052
37370 CLIENT: Up:0 ds:301 act:2
37371 [8810] got start action
37372 [8810] I am going to restart
37373 Starting downstairs at port 8810
37374 Make output file at "/tmp/dsc/downstairs-8810.txt"
37375 [8810][0] reports Starting
37376 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1235)
37377 [8810][0] reports Running
37378 ----------------------------------------------------------------
37379 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
37380 States: Offline Active Active
37381 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
37382 9756 Acked Write 3 Sent Done Done false
37383 9757 Acked Read 10 Sent Done Done false
37384 9758 Acked Read 4 Sent Done Done false
37385 9759 Acked Read 5 Sent Done Done false
37386 9760 Acked Write 2 Sent Done Done false
37387 9761 Acked Read 10 Sent Done Done false
37388 9762 Acked Write 6 Sent Done Done false
37389 9763 Acked Write 2 Sent Done Done false
37390 9764 Acked Read 3 Sent Done Done false
37391 9765 Acked Read 1 Sent Done Done false
37392 9766 Acked Read 7 Sent Done Done false
37393 9767 Acked Flush 0 Sent Done Done false
37394 9768 Acked Flush 0 Sent Done Done false
37395 9769 Acked Write 6 Sent Done Done false
37396 9770 Acked Read 5 Sent Done Done false
37397 9771 Acked Read 6 Sent Done Done false
37398 9772 Acked Read 1 Sent Done Done false
37399 9773 Acked Read 10 Sent Done Done false
37400 9774 Acked Read 3 Sent Done Done false
37401 9775 Acked Read 2 Sent Done Done false
37402 9776 Acked Read 4 Sent Done Done false
37403 9777 Acked Write 10 Sent Done Done false
37404 9778 Acked Read 4 Sent Done Done false
37405 9779 Acked Write 4 Sent Done Done false
37406 9780 Acked Flush 0 Sent Done Done false
37407 9781 Acked Write 4 Sent Done Done false
37408 9782 Acked Read 1 Sent Done Done false
37409 9783 Acked Read 10 Sent Done Done false
37410 9784 Acked Write 4 Sent Done Done false
37411 9785 Acked Read 10 Sent Done Done false
37412 9786 Acked Write 9 Sent Done Done false
37413 9787 Acked Write 1 Sent Done Done false
37414 9788 Acked Write 8 Sent Done Done false
37415 9789 Acked Write 3 Sent Done Done false
37416 9790 Acked Read 6 Sent Done Done false
37417 9791 Acked Write 1 Sent Done Done false
37418 9792 Acked Read 1 Sent Done Done false
37419 9793 Acked Read 5 Sent Done Done false
37420 9794 Acked Read 7 Sent Done Done false
37421 9795 Acked Read 9 Sent Done Done false
37422 9796 Acked Write 5 Sent Done Done false
37423 9797 Acked Read 3 Sent Done Done false
37424 9798 Acked Read 10 Sent Done Done false
37425 9799 Acked Read 9 Sent Done Done false
37426 9800 Acked Read 7 Sent Done Done false
37427 9801 Acked Flush 0 Sent Done Done false
37428 9802 Acked Flush 0 Sent Done Done false
37429 9803 Acked Write 9 Sent Done Done false
37430 9804 Acked Read 10 Sent Done Done false
37431 9805 Acked Write 1 Sent Done Done false
37432 9806 Acked Read 7 Sent Done Done false
37433 9807 Acked Read 7 Sent Done Done false
37434 9808 Acked Read 3 Sent Done Done false
37435 9809 Acked Read 6 Sent Done Done false
37436 9810 Acked Read 3 Sent Done Done false
37437 9811 Acked Write 3 Sent Done Done false
37438 9812 Acked Read 9 Sent Done Done false
37439 9813 Acked Read 1 Sent Done Done false
37440 9814 Acked Read 3 Sent Done Done false
37441 9815 Acked Write 5 Sent Done Done false
37442 9816 Acked Write 8 Sent Done Done false
37443 9817 Acked Write 9 Sent Done Done false
37444 9818 Acked Read 5 Sent Done Done false
37445 9819 Acked Read 7 Sent Done Done false
37446 9820 Acked Write 4 Sent Done Done false
37447 9821 Acked Read 7 Sent Done Done false
37448 9822 Acked Read 8 Sent Done Done false
37449 9823 Acked Write 5 Sent Done Done false
37450 9824 Acked Read 6 Sent Done Done false
37451 9825 Acked Read 9 Sent Done Done false
37452 9826 Acked Read 7 Sent Done Done false
37453 9827 Acked Read 8 Sent Done Done false
37454 9828 Acked Write 5 Sent Done Done false
37455 9829 Acked Read 5 Sent Done Done false
37456 9830 Acked Read 4 Sent Done Done false
37457 9831 Acked Write 4 Sent Done Done false
37458 9832 Acked Read 6 Sent Done Done false
37459 9833 Acked Read 8 Sent Done Done false
37460 9834 Acked Write 9 Sent Done Done false
37461 9835 Acked Read 10 Sent Done Done false
37462 9836 Acked Read 10 Sent Done Done false
37463 9837 Acked Read 1 Sent Done Done false
37464 9838 Acked Read 7 Sent Done Done false
37465 9839 Acked Read 10 Sent Done Done false
37466 9840 Acked Flush 0 Sent Done Done false
37467 9841 Acked Read 10 Sent Done Done false
37468 9842 Acked Write 7 Sent Done Done false
37469 9843 Acked Write 6 Sent Done Done false
37470 9844 Acked Write 7 Sent Done Done false
37471 9845 Acked Write 9 Sent Done Done false
37472 9846 Acked Read 6 Sent Done Done false
37473 9847 Acked Read 1 Sent Done Done false
37474 9848 Acked Read 10 Sent Done Done false
37475 9849 Acked Read 10 Sent Done Done false
37476 9850 Acked Write 8 Sent Done Done false
37477 9851 Acked Read 7 Sent Done Done false
37478 9852 Acked Write 2 Sent Done Done false
37479 9853 Acked Read 8 Sent Done Done false
37480 9854 Acked Read 3 Sent Done Done false
37481 9855 Acked Flush 0 Sent Done Done false
37482 9856 Acked Write 5 Sent Done Done false
37483 9857 Acked Write 6 Sent Done Done false
37484 9858 Acked Read 8 Sent Done Done false
37485 9859 Acked Write 9 Sent Done Done false
37486 9860 Acked Write 2 Sent Done Done false
37487 9861 Acked Write 10 Sent Done Done false
37488 9862 Acked Write 5 Sent Done Done false
37489 9863 Acked Read 5 Sent Done Done false
37490 9864 Acked Write 7 Sent Done Done false
37491 9865 Acked Read 8 Sent Done Done false
37492 9866 Acked Read 6 Sent Done Done false
37493 9867 Acked Write 5 Sent Done Done false
37494 9868 Acked Read 8 Sent Done Done false
37495 9869 Acked Read 8 Sent Done Done false
37496 9870 Acked Flush 0 Sent Done Done false
37497 9871 Acked Flush 0 Sent Done Done false
37498 9872 Acked Read 7 Sent Done Done false
37499 9873 Acked Read 7 Sent Done Done false
37500 9874 Acked Write 1 Sent Done Done false
37501 9875 Acked Write 2 Sent Done Done false
37502 9876 Acked Flush 0 Sent Done Done false
37503 9877 Acked Write 9 Sent Done Done false
37504 9878 Acked Read 3 Sent Done Done false
37505 9879 Acked Read 6 Sent Done Done false
37506 9880 Acked Read 2 Sent Done Done false
37507 9881 Acked Read 10 Sent Done Done false
37508 9882 Acked Read 4 Sent Done Done false
37509 9883 Acked Write 3 Sent Done Done false
37510 9884 Acked Write 2 Sent Done Done false
37511 9885 Acked Read 5 Sent Done Done false
37512 9886 Acked Read 8 Sent Done Done false
37513 9887 Acked Write 8 Sent Done Done false
37514 9888 Acked Write 6 Sent Done Done false
37515 9889 Acked Write 9 Sent Done Done false
37516 9890 Acked Read 8 Sent Done Done false
37517 9891 Acked Write 9 Sent Done Done false
37518 9892 Acked Read 8 Sent Done Done false
37519 9893 Acked Write 7 Sent Done Done false
37520 9894 Acked Write 4 Sent Done Done false
37521 9895 Acked Write 2 Sent Done Done false
37522 9896 Acked Write 3 Sent Done Done false
37523 9897 Acked Write 2 Sent Done Done false
37524 9898 Acked Write 7 Sent Done Done false
37525 9899 Acked Read 3 Sent Done Done false
37526 9900 Acked Read 6 Sent Done Done false
37527 9901 Acked Read 2 Sent Done Done false
37528 9902 Acked Read 10 Sent Done Done false
37529 9903 Acked Write 4 Sent Done Done false
37530 9904 Acked Read 8 Sent Done Done false
37531 9905 Acked Read 4 Sent Done Done false
37532 9906 Acked Read 6 Sent Done Done false
37533 9907 Acked Write 1 Sent Done Done false
37534 9908 Acked Write 9 Sent Done Done false
37535 9909 Acked Write 4 Sent Done Done false
37536 9910 Acked Flush 0 Sent Done Done false
37537 9911 Acked Read 10 Sent Done Done false
37538 9912 Acked Write 6 Sent Done Done false
37539 9913 Acked Read 6 Sent Done Done false
37540 9914 Acked Read 3 Sent Done Done false
37541 9915 Acked Write 4 Sent Done Done false
37542 9916 Acked Read 8 Sent Done Done false
37543 9917 Acked Write 5 Sent Done Done false
37544 9918 Acked Flush 0 Sent Done Done false
37545 9919 Acked Read 3 Sent Done Done false
37546 9920 Acked Write 4 Sent Done Done false
37547 9921 Acked Write 2 Sent Done Done false
37548 9922 Acked Read 1 Sent Done Done false
37549 9923 Acked Read 8 Sent Done Done false
37550 9924 Acked Read 6 Sent Done Done false
37551 9925 Acked Read 10 Sent Done Done false
37552 9926 Acked Flush 0 Sent Done Done false
37553 9927 Acked Write 7 Sent Done Done false
37554 9928 Acked Write 5 Sent Done Done false
37555 9929 Acked Write 4 Sent Done Done false
37556 9930 Acked Write 4 Sent Done Done false
37557 9931 Acked Flush 0 Sent Done Done false
37558 9932 Acked Read 2 Sent Done Done false
37559 9933 Acked Read 6 Sent Done Done false
37560 9934 Acked Read 2 Sent Done Done false
37561 9935 Acked Read 3 Sent Done Done false
37562 9936 Acked Read 6 Sent Done Done false
37563 9937 Acked Write 6 Sent Done Done false
37564 9938 Acked Read 7 Sent Done Done false
37565 9939 Acked Write 5 Sent Done Done false
37566 9940 Acked Read 9 Sent Done Done false
37567 9941 Acked Read 10 Sent Done Done false
37568 9942 Acked Write 4 Sent Done Done false
37569 9943 Acked Flush 0 Sent Done Done false
37570 9944 Acked Read 1 Sent Done Done false
37571 9945 Acked Read 6 Sent Done Done false
37572 9946 Acked Write 6 Sent Done Done false
37573 9947 Acked Read 10 Sent Done Done false
37574 9948 Acked Read 1 Sent Done Done false
37575 9949 Acked Read 4 Sent Done Done false
37576 9950 Acked Read 6 Sent Done Done false
37577 9951 Acked Write 4 Sent Done Done false
37578 9952 Acked Read 2 Sent Done Done false
37579 9953 Acked Write 2 Sent Done Done false
37580 9954 Acked Write 7 Sent Done Done false
37581 9955 Acked Read 3 Sent Done Done false
37582 9956 Acked Write 4 Sent Done Done false
37583 9957 Acked Write 10 Sent Done Done false
37584 9958 Acked Read 2 Sent Done Done false
37585 9959 Acked Read 5 Sent Done Done false
37586 9960 Acked Write 5 Sent Done Done false
37587 9961 Acked Read 5 Sent Done Done false
37588 9962 Acked Read 1 Sent Done Done false
37589 9963 Acked Write 7 Sent Done Done false
37590 9964 Acked Read 1 Sent Done Done false
37591 9965 Acked Write 7 Sent Done Done false
37592 9966 Acked Write 2 Sent Done Done false
37593 9967 Acked Write 10 Sent Done Done false
37594 9968 Acked Write 6 Sent Done Done false
37595 9969 Acked Read 2 Sent Done Done false
37596 9970 Acked Read 3 Sent Done Done false
37597 9971 Acked Read 6 Sent Done Done false
37598 9972 Acked Flush 0 Sent Done Done false
37599 9973 Acked Write 3 Sent Done Done false
37600 9974 Acked Write 1 Sent Done Done false
37601 9975 Acked Read 1 Sent Done Done false
37602 9976 Acked Write 9 Sent Done Done false
37603 9977 Acked Write 5 Sent Done Done false
37604 9978 Acked Read 6 Sent Done Done false
37605 9979 Acked Flush 0 Sent Done Done false
37606 9980 Acked Write 5 Sent Done Done false
37607 9981 Acked Read 5 Sent Done Done false
37608 9982 Acked Write 1 Sent Done Done false
37609 9983 Acked Read 5 Sent Done Done false
37610 9984 Acked Write 7 Sent Done Done false
37611 9985 Acked Read 6 Sent Done Done false
37612 9986 Acked Read 5 Sent Done Done false
37613 9987 Acked Write 9 Sent Done Done false
37614 9988 Acked Write 10 Sent Done Done false
37615 9989 Acked Read 9 Sent Done Done false
37616 9990 Acked Write 4 Sent Done Done false
37617 9991 Acked Write 1 Sent Done Done false
37618 9992 Acked Write 4 Sent Done Done false
37619 9993 Acked Flush 0 Sent Done Done false
37620 9994 Acked Read 6 Sent Done Done false
37621 9995 Acked Flush 0 Sent Done Done false
37622 9996 Acked Read 1 Sent Done Done false
37623 9997 Acked Read 7 Sent Done Done false
37624 9998 Acked Read 4 Sent Done Done false
37625 9999 Acked Write 4 Sent Done Done false
37626 10000 Acked Read 2 Sent Done Done false
37627 10001 Acked Write 4 Sent Done Done false
37628 10002 Acked Write 8 Sent Done Done false
37629 10003 Acked Write 6 Sent Done Done false
37630 10004 Acked Write 2 Sent Done Done false
37631 10005 Acked Write 4 Sent Done Done false
37632 10006 Acked Flush 0 Sent Done Done false
37633 10007 Acked Read 1 Sent Done Done false
37634 10008 Acked Write 6 Sent Done Done false
37635 10009 Acked Write 9 Sent Done Done false
37636 10010 Acked Write 4 Sent Done Done false
37637 10011 Acked Read 7 Sent Done Done false
37638 10012 Acked Read 5 Sent Done Done false
37639 10013 Acked Read 1 Sent Done Done false
37640 10014 Acked Write 6 Sent Done Done false
37641 10015 Acked Write 6 Sent Done Done false
37642 10016 Acked Write 2 Sent Done Done false
37643 10017 Acked Write 2 Sent Done Done false
37644 10018 Acked Flush 0 Sent Done Done false
37645 10019 Acked Read 6 Sent Done Done false
37646 10020 Acked Write 7 Sent Done Done false
37647 10021 Acked Write 3 Sent Done Done false
37648 10022 Acked Write 5 Sent Done Done false
37649 10023 Acked Write 7 Sent Done Done false
37650 10024 Acked Read 9 Sent Done Done false
37651 10025 Acked Write 3 Sent Done Done false
37652 10026 Acked Flush 0 Sent Done Done false
37653 10027 Acked Flush 0 Sent Done Done false
37654 10028 Acked Write 1 Sent Done Done false
37655 10029 Acked Read 3 Sent Done Done false
37656 10030 Acked Read 4 Sent Done Done false
37657 10031 Acked Read 9 Sent Done Done false
37658 10032 Acked Read 9 Sent Done Done false
37659 10033 Acked Read 10 Sent Done Done false
37660 10034 Acked Read 6 Sent Done Done false
37661 10035 Acked Write 4 Sent Done Done false
37662 10036 Acked Read 10 Sent Done Done false
37663 10037 Acked Write 2 Sent Done Done false
37664 10038 Acked Write 10 Sent Done Done false
37665 10039 Acked Read 3 Sent Done Done false
37666 10040 Acked Read 7 Sent Done Done false
37667 10041 Acked Write 10 Sent Done Done false
37668 10042 Acked Read 2 Sent Done Done false
37669 10043 Acked Write 4 Sent Done Done false
37670 10044 Acked Read 4 Sent Done Done false
37671 10045 Acked Read 2 Sent Done Done false
37672 10046 Acked Read 6 Sent Done Done false
37673 10047 Acked Read 2 Sent Done Done false
37674 10048 Acked Read 2 Sent Done Done false
37675 10049 Acked Write 5 Sent Done Done false
37676 10050 Acked Read 5 Sent Done Done false
37677 10051 Acked Write 10 Sent Done Done false
37678 10052 Acked Read 8 Sent Done Done false
37679 10053 Acked Read 8 Sent Done Done false
37680 10054 Acked Write 9 Sent Done Done false
37681 10055 Acked Write 8 Sent Done Done false
37682 10056 Acked Write 5 Sent Done Done false
37683 10057 Acked Flush 0 Sent Done Done false
37684 STATES DS:0 DS:1 DS:2 TOTAL
37685 Sent 302 0 0 302
37686 Done 0 302 302 604
37687 Skipped 0 0 0 0
37688 Error 0 0 0 0
37689 Last Flush: 9755 10057 10057
37690 Downstairs last five completed: 9755 9754 9753 9752 9751
37691 Upstairs last five completed: 10057 10056 10055 10054 10053
37692 CLIENT: Up:0 ds:302 act:2
376932024-11-15T02:05:21.048ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
376942024-11-15T02:05:21.048ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:49964) = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
376952024-11-15T02:05:21.049ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
376962024-11-15T02:05:21.050ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
376972024-11-15T02:05:21.050ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
376982024-11-15T02:05:21.050ZINFOcrucible: send last flush ID to this DS: 9755 = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
376992024-11-15T02:05:21.050ZINFOcrucible: Replied this last flush ID: 9755 = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
377002024-11-15T02:05:21.050ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
377012024-11-15T02:05:21.050ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
377022024-11-15T02:05:21.050ZINFOcrucible: [0] client re-new 302 jobs since flush 9755 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
377032024-11-15T02:05:21.051ZINFOcrucible: [0] Replayed 302 jobs since flush: 9755 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
37704 ----------------------------------------------------------------
37705 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
37706 Downstairs last five completed: 10057 10056 10055 10054 10053
37707 Upstairs last five completed: 10057 10056 10055 10054 10053
37708 CLIENT: Up:0 ds:0 act:3
37709 Replay: All jobs finished
37710 Main task has work to do, go find it
37711 got dsc Stop(0)
37712 stop 0
37713 [8810] Got stop action so:false kr:false
37714 Nov 15 02:05:23.294 INFO request completed, latency_us: 397, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 77f76e6a-842b-40e6-b978-d67ec955a505, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
37715 Nov 15 02:05:23.294 INFO request completed, latency_us: 111, response_code: 200, uri{"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2024-11-15T02:05:23.294847865Z","hostname":"w-01JCPRHT5F2MPA44KVJHM3S2PB","pid":1205,"":"io task","client":"0","":"downstairs","session_id":"971dcb2e-6891-4ecf-b367-1e04e52065ee"}
37716 : /state/cid/0, method: GET, req_id: a45ef7a5-8894-4b3a-b8e2-ac41984a25cd, {"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2024-11-15T02:05:23.294910071Z","hostname":"w-01JCPRHT5F2MPA44KVJHM3S2PB","pid":1205,"":"io task","client":"0","":"downstairs","session_id":"971dcb2e-6891-4ecf-b367-1e04e52065ee"}
37717 remote_addr: 127{"msg":"downstairs task for 0 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2024-11-15T02:05:23.294939779Z","hostname":"w-01JCPRHT5F2MPA44KVJHM3S2PB","pid":1205,"session_id":"971dcb2e-6891-4ecf-b367-1e04e52065ee"}
37718 .0.0.1:40008, local_addr: 127.0.0{"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-15T02:05:23.294969727Z","hostname":"w-01JCPRHT5F2MPA44KVJHM3S2PB","pid":1205,"client":"0","":"downstairs","session_id":"971dcb2e-6891-4ecf-b367-1e04e52065ee"}
37719 .{"msg":"[0] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-15T02:05:23.294998596Z","hostname":"w-01JCPRHT5F2MPA44KVJHM3S2PB","pid":1205,"client":"0","":"downstairs","session_id":"971dcb2e-6891-4ecf-b367-1e04e52065ee"}
37720 1:9998
377212024-11-15T02:05:23.295ZINFOcrucible: client task is exiting = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
37722 [8810] Exited with: ExitStatus(unix_wait_status(9))
37723 [8810][0] reports Exit
37724 Nov 15 02:05:27.296 INFO request completed, latency_us: 208, response_code: 200, uri: /state/cid/0, method: GET, req_id: 3362f94f-02ce-453a-ad83-1151676a039d, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
37725 001/300 Write block 241 len 24576 data: 6 5 4 3 3 4
37726 002/300 Read block 672 len 12288
37727 003/300 Read block 589 len 4096
37728 004/300 Write block 1279 len 20480 data: 7 8 7 9 9
37729 005/300 Write block 3856 len 4096 data: 3
37730 006/300 Write block 4187 len 8192 data: 6 6
37731 007/300 Write block 1104 len 28672 data: 4 3 4 4 5 5 3
37732 008/300 Flush
37733 009/300 Read block 874 len 24576
37734 010/300 Write block 3846 len 36864 data: 4 5 4 5 7 7 7 7 7
37735 011/300 Write block 4363 len 4096 data: 4
37736 012/300 Write block 302 len 20480 data: 4 4 4 4 7
37737 013/300 Read block 1709 len 16384
37738 014/300 Flush
37739 015/300 Flush
37740 016/300 Write block 4278 len 36864 data: 5 4 5 5 6 4 5 6 7
37741 017/300 Read block 841 len 36864
37742 018/300 Flush
37743 019/300 Read block 1876 len 12288
37744 020/300 Read block 4919 len 8192
37745 021/300 Read block 2315 len 28672
37746 022/300 Read block 201 len 24576
37747 023/300 Read block 2746 len 8192
37748 024/300 Read block 900 len 16384
37749 025/300 Read block 3261 len 36864
37750 026/300 Write block 11 len 40960 data: 9 7 6 6 4 4 3 3 5 8
37751 027/300 Read block 291 len 40960
37752 028/300 Read block 1844 len 28672
37753 029/300 Write block 611 len 24576 data: 5 5 5 4 3 3
37754 030/300 Read block 1710 len 4096
37755 031/300 Write block 2762 len 24576 data: 6 5 5 5 6 5
37756 032/300 Write block 3661 len 32768 data: 8 9 8 7 5 5 6 4
37757 033/300 Read block 3808 len 40960
37758 034/300 Read block 427 len 32768
37759 035/300 Read block 1652 len 4096
37760 036/300 Read block 2630 len 28672
37761 037/300 Write block 4057 len 24576 data: 3 5 5 4 4 4
37762 038/300 Write block 614 len 16384 data: 5 4 4 5
37763 039/300 Write block 3750 len 36864 data: 9 10 8 7 7 6 6 6 6
37764 040/300 Read block 1158 len 4096
37765 041/300 Write block 2057 len 40960 data: 5 6 7 10 11 12 11 9 7 6
37766 042/300 Write block 3638 len 24576 data: 5 6 3 3 3 3
37767 043/300 Flush
37768 044/300 Write block 3355 len 32768 data: 9 6 4 5 5 3 5 4
37769 045/300 Read block 2511 len 16384
37770 046/300 Read block 1140 len 4096
37771 047/300 Read block 1559 len 24576
37772 048/300 Read block 3108 len 24576
37773 049/300 Read block 3202 len 4096
37774 050/300 Flush
37775 051/300 Read block 2757 len 12288
37776 052/300 Read block 3520 len 8192
37777 053/300 Read block 325 len 12288
37778 054/300 Flush
37779 055/300 Read block 3455 len 4096
37780 056/300 Write block 373 len 24576 data: 4 4 7 6 4 4
37781 057/300 Flush
37782 058/300 Write block 4211 len 12288 data: 6 5 6
37783 059/300 Read block 3808 len 36864
37784 060/300 Write block 1163 len 12288 data: 4 6 5
37785 061/300 Write block 2101 len 40960 data: 6 6 7 6 5 4 3 3 3 3
37786 062/300 Write block 3804 len 36864 data: 7 7 9 10 8 9 7 10 7
37787 063/300 Write block 1136 len 40960 data: 4 4 5 5 7 6 6 7 7 7
37788 064/300 Flush
37789 065/300 Flush
37790 066/300 Write block 3341 len 12288 data: 6 5 6
37791 067/300 Read block 3483 len 40960
37792 068/300 Write block 4142 len 32768 data: 6 6 3 3 4 4 5 8
37793 069/300 Write block 1594 len 16384 data: 6 5 4 5
37794 070/300 Read block 516 len 24576
37795 071/300 Flush
37796 072/300 Read block 4890 len 32768
37797 073/300 Read block 4748 len 28672
37798 074/300 Write block 2878 len 40960 data: 5 4 4 5 6 6 5 6 8 9
37799 075/300 Write block 160 len 16384 data: 8 9 9 9
37800 076/300 Write block 916 len 40960 data: 5 5 5 5 6 4 4 4 4 5
37801 077/300 Read block 3133 len 16384
37802 078/300 Read block 1393 len 36864
37803 079/300 Read block 3354 len 36864
37804 080/300 Write block 1818 len 40960 data: 5 6 7 8 8 7 7 7 5 4
37805 081/300 Write block 1178 len 28672 data: 5 3 4 4 5 5 3
37806 082/300 Write block 600 len 20480 data: 6 5 5 5 6
37807 083/300 Read block 2044 len 16384
37808 084/300 Write block 3943 len 12288 data: 7 5 4
37809 085/300 Write block 3966 len 32768 data: 5 6 6 6 8 8 8 9
37810 086/300 Flush
37811 087/300 Write block 1522 len 36864 data: 4 4 6 6 6 5 6 6 7
37812 088/300 Read block 4745 len 28672
37813 089/300 Write block 1106 len 4096 data: 5
37814 090/300 Read block 1766 len 36864
37815 091/300 Write block 980 len 8192 data: 3 2
37816 092/300 Read block 415 len 12288
37817 093/300 Write block 2929 len 8192 data: 4 5
37818 094/300 Read block 164 len 40960
37819 095/300 Flush
37820 096/300 Write block 3209 len 8192 data: 3 4
37821 097/300 Write block 3905 len 36864 data: 10 10 8 7 8 5 4 5 6
37822 098/300 Write block 2062 len 24576 data: 13 12 10 8 7 7
37823 099/300 Read block 3668 len 28672
37824 100/300 Write block 4601 len 24576 data: 7 6 5 6 7 8
37825 101/300 Flush
37826 102/300 Write block 2349 len 16384 data: 5 5 4 4
37827 103/300 Read block 3547 len 16384
37828 104/300 Write block 2898 len 12288 data: 6 6 5
37829 105/300 Read block 4997 len 8192
37830 106/300 Read block 3765 len 20480
37831 107/300 Write block 1687 len 4096 data: 7
37832 108/300 Read block 1951 len 12288
37833 109/300 Write block 3633 len 4096 data: 4
37834 110/300 Read block 4773 len 4096
37835 111/300 Read block 823 len 20480
37836 112/300 Read block 4119 len 20480
37837 113/300 Flush
37838 114/300 Flush
37839 115/300 Write block 4254 len 40960 data: 8 7 8 8 8 10 9 7 7 7
37840 116/300 Write block 3518 len 20480 data: 4 4 4 6 6
37841 117/300 Read block 1069 len 12288
37842 118/300 Write block 1556 len 12288 data: 5 6 8
37843 119/300 Write block 143 len 20480 data: 3 4 3 3 4
37844 120/300 Read block 4368 len 28672
37845 121/300 Flush
37846 122/300 Write block 4227 len 16384 data: 6 6 7 6
37847 123/300 Read block 3642 len 40960
37848 124/300 Read block 2348 len 16384
37849 125/300 Write block 1701 len 16384 data: 7 8 7 5
37850 126/300 Read block 2069 len 32768
37851 127/300 Read block 1114 len 4096
37852 128/300 Flush
37853 129/300 Write block 2845 len 40960 data: 5 6 7 8 6 5 5 6 6 6
37854 130/300 Read block 2934 len 24576
37855 131/300 Write block 2329 len 32768 data: 3 3 4 6 6 6 7 11
37856 132/300 Write block 2649 len 36864 data: 6 7 8 5 4 3 4 3 5
37857 133/300 Read block 4811 len 32768
37858 134/300 Flush
37859 135/300 Write block 4717 len 4096 data: 5
37860 136/300 Write block 531 len 16384 data: 7 7 8 11
37861 137/300 Read block 2961 len 24576
37862 138/300 Flush
37863 139/300 Read block 1415 len 24576
37864 140/300 Read block 1177 len 8192
37865 141/300 Read block 3304 len 16384
37866 142/300 Write block 1942 len 40960 data: 7 5 4 4 3 2 3 2 3 5
37867 143/300 Flush
37868 144/300 Flush
37869 145/300 Write block 2099 len 16384 data: 6 6 7 7
37870 146/300 Read block 2818 len 8192
37871 147/300 Read block 3546 len 8192
37872 148/300 Read block 3777 len 20480
37873 149/300 Read block 1646 len 36864
37874 150/300 Write block 4572 len 16384 data: 6 5 4 4
37875 151/300 Write block 4214 len 24576 data: 7 7 6 7 6 7
37876 152/300 Read block 2401 len 24576
37877 153/300 Read block 1430 len 16384
37878 154/300 Read block 3235 len 4096
37879 155/300 Write block 82 len 4096 data: 10
37880 156/300 Read block 878 len 24576
37881 157/300 Write block 4263 len 28672 data: 8 8 9 10 8 8 7
37882 158/300 Read block 4378 len 32768
37883 159/300 Write block 3461 len 8192 data: 6 3
37884 160/300 Read block 2070 len 4096
37885 161/300 Write block 4340 len 12288 data: 8 5 4
37886 162/300 Read block 3048 len 20480
37887 163/300 Read block 4388 len 24576
37888 164/300 Write block 179 len 12288 data: 5 6 6
37889 165/300 Write block 3534 len 20480 data: 6 8 8 6 8
37890 166/300 Write block 137 len 28672 data: 6 7 6 7 6 4 4
37891 167/300 Flush
37892 168/300 Write block 2050 len 20480 data: 4 4 3 3 5
37893 169/300 Read block 2390 len 32768
37894 170/300 Read block 1162 len 16384
37895 171/300 Read block 1480 len 4096
37896 172/300 Write block 3381 len 28672 data: 8 7 7 6 7 7 5
37897 173/300 Read block 961 len 8192
37898 174/300 Read block 1200 len 32768
37899 175/300 Read block 1371 len 16384
37900 176/300 Write block 2042 len 12288 data: 5 5 5
37901 177/300 Write block 2241 len 8192 data: 8 7
37902 178/300 Read block 858 len 4096
37903 179/300 Read block 4326 len 16384
37904 180/300 Write block 2966 len 28672 data: 4 6 4 3 3 5 5
37905 181/300 Write block 3204 len 32768 data: 3 3 4 5 4 4 5 4
37906 182/300 Read block 2159 len 40960
37907 183/300 Read block 3715 len 28672
37908 184/300 Write block 2254 len 24576 data: 7 9 8 8 7 6
37909 185/300 Write block 3508 len 16384 data: 4 4 4 4
37910 186/300 Write block 2939 len 16384 data: 7 6 5 6
37911 187/300 Read block 3563 len 24576
37912 188/300 Write block 1953 len 16384 data: 7 9 9 8
37913 189/300 Read block 4603 len 32768
37914 190/300 Read block 3864 len 32768
37915 191/300 Write block 2504 len 32768 data: 4 4 4 4 5 4 5 4
37916 192/300 Read block 4008 len 32768
37917 193/300 Read block 2536 len 12288
37918 194/300 Write block 485 len 36864 data: 6 3 4 4 4 6 7 8 8
37919 195/300 Write block 3787 len 28672 data: 5 5 5 4 3 3 3
37920 196/300 Write block 588 len 4096 data: 7
37921 197/300 Write block 4586 len 32768 data: 8 7 6 7 8 8 8 9
37922 198/300 Read block 782 len 16384
37923 199/300 Read block 1516 len 12288
37924 200/300 Read block 3622 len 8192
37925 201/300 Write block 3927 len 4096 data: 8
37926 202/300 Write block 1780 len 36864 data: 8 9 10 9 7 6 5 6 7
37927 203/300 Write block 3871 len 32768 data: 9 8 7 9 9 6 5 5
37928 204/300 Write block 368 len 28672 data: 4 4 4 4 3 5 5
37929 205/300 Write block 4796 len 32768 data: 6 6 7 7 5 5 5 4
37930 206/300 Write block 523 len 32768 data: 5 5 5 6 6 5 5 7
37931 207/300 Read block 2517 len 32768
37932 208/300 Read block 4450 len 24576
37933 209/300 Read block 27 len 8192
37934 210/300 Read block 1020 len 8192
37935 211/300 Read block 2587 len 36864
37936 212/300 Read block 3620 len 40960
37937 213/300 Write block 1702 len 36864 data: 9 8 6 5 5 4 4 3 2
37938 214/300 Read block 626 len 36864
37939 215/300 Write block 207 len 40960 data: 8 8 8 8 9 7 7 8 7 6
37940 216/300 Read block 3222 len 4096
37941 217/300 Read block 3576 len 4096
37942 218/300 Flush
37943 219/300 Write block 1137 len 20480 data: 5 6 6 8 7
37944 220/300 Write block 1055 len 8192 data: 6 6
37945 221/300 Write block 259 len 40960 data: 6 8 7 6 6 6 5 4 4 4
37946 222/300 Read block 4833 len 4096
37947 223/300 Write block 2710 len 28672 data: 3 4 4 5 6 5 6
37948 224/300 Write block 642 len 40960 data: 5 4 4 4 4 3 5 6 5 7
37949 225/300 Read block 1797 len 4096
37950 226/300 Read block 2599 len 28672
37951 227/300 Flush
37952 228/300 Write block 3492 len 32768 data: 6 5 5 6 6 6 7 8
37953 229/300 Read block 1311 len 12288
37954 230/300 Read block 3062 len 40960
37955 231/300 Read block 3459 len 16384
37956 232/300 Read block 849 len 12288
37957 233/300 Flush
37958 234/300 Read block 3926 len 40960
37959 235/300 Read block 4460 len 4096
37960 236/300 Read block 4990 len 12288
37961 237/300 Write block 59 len 40960 data: 8 8 6 5 5 5 6 7 6 7
37962 238/300 Write block 2771 len 16384 data: 5 3 4 4
37963 239/300 Read block 2161 len 32768
37964 240/300 Write block 620 len 36864 data: 6 8 9 11 10 8 6 6 6
37965 241/300 Read block 915 len 8192
37966 242/300 Read block 3644 len 20480
37967 243/300 Read block 2694 len 20480
37968 244/300 Write block 1357 len 28672 data: 10 12 9 7 8 8 9
37969 245/300 Write block 1000 len 20480 data: 10 10 9 9 7
37970 246/300 Read block 1604 len 40960
37971 247/300 Read block 1150 len 8192
37972 248/300 Write block 1835 len 20480 data: 7 7 8 8 7
37973 249/300 Write block 903 len 16384 data: 8 8 7 8
37974 250/300 Read block 123 len 32768
37975 251/300 Read block 3430 len 20480
37976 252/300 Write block 2213 len 24576 data: 4 4 5 6 5 6
37977 253/300 Read block 1631 len 36864
37978 254/300 Read block 2568 len 28672
37979 255/300 Write block 1455 len 28672 data: 7 6 5 6 7 7 6
37980 256/300 Write block 3701 len 12288 data: 6 7 6
37981 257/300 Write block 2810 len 20480 data: 6 6 6 5 6
37982 258/300 Read block 2394 len 24576
37983 259/300 Read block 806 len 36864
37984 260/300 Write block 3688 len 20480 data: 6 6 10 10 8
37985 261/300 Flush
37986 262/300 Read block 1799 len 12288
37987 263/300 Read block 3953 len 16384
37988 264/300 Read block 2481 len 8192
37989 265/300 Read block 1472 len 8192
37990 266/300 Write block 3236 len 28672 data: 3 3 5 4 4 3 4
37991 267/300 Write block 1595 len 32768 data: 6 5 6 5 5 7 6 7
37992 268/300 Flush
37993 269/300 Read block 3249 len 28672
37994 270/300 Read block 253 len 40960
37995 271/300 Read block 4465 len 40960
37996 272/300 Read block 1931 len 36864
37997 273/300 Read block 4490 len 20480
37998 274/300 Read block 4496 len 12288
37999 275/300 Read block 2066 len 16384
38000 276/300 Write block 86 len 36864 data: 8 9 9 6 6 5 5 4 6
38001 277/300 Write block 2254 len 20480 data: 8 10 9 9 8
38002 278/300 Read block 3480 len 8192
38003 279/300 Write block 4261 len 32768 data: 8 8 9 9 10 11 9 9
38004 280/300 Write block 3837 len 8192 data: 9 9
38005 281/300 Read block 724 len 16384
38006 282/300 Read block 3909 len 16384
38007 283/300 Read block 4244 len 4096
38008 284/300 Read block 871 len 28672
38009 285/300 Read block 1396 len 20480
38010 286/300 Write block 4289 len 28672 data: 7 6 7 5 5 5 5
38011 287/300 Write block 3541 len 8192 data: 5 5
38012 288/300 Write block 4477 len 20480 data: 4 5 5 5 5
38013 289/300 Write block 625 len 24576 data: 9 7 7 7 5 6
38014 290/300 Read block 4682 len 16384
38015 291/300 Write block 2127 len 12288 data: 5 5 6
38016 292/300 Read block 1545 len 4096
38017 293/300 Write block 2751 len 20480 data: 5 3 5 5 5
38018 294/300 Read block 1241 len 20480
38019 295/300 Read block 545 len 24576
38020 296/300 Read block 632 len 32768
38021 297/300 Read block 170 len 24576
38022 298/300 Read block 1062 len 20480
38023 299/300 Read block 1427 len 32768
38024 300/300 Write block 1815 len 4096 data: 6
38025 301/300 Read block 479 len 20480
38026 Main task has work to do, go find it
38027 got dsc Start(0)
38028 start 0
38029 Nov 15 02:05:27.527 INFO request completed, latency_us: 227, response_code: 204, uri: /start/cid/0, method: POST, req_id: e2b1f5b0-093b-462e-919a-92f0cac4413c, remote_addr: 127.0.0.1:40008, local_addr: 127.0.0.1:9998
38030 Replay: started 0, returned:Ok(())
38031 ----------------------------------------------------------------
38032 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
38033 States: Offline Active Active
38034 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
38035 10058 Acked Write 6 Sent Done Done false
38036 10059 Acked Read 3 Sent Done Done false
38037 10060 Acked Read 1 Sent Done Done false
38038 10061 Acked Write 5 Sent Done Done false
38039 10062 Acked Write 1 Sent Done Done false
38040 10063 Acked Write 2 Sent Done Done false
38041 10064 Acked Write 7 Sent Done Done false
38042 10065 Acked Flush 0 Sent Done Done false
38043 10066 Acked Read 6 Sent Done Done false
38044 10067 Acked Write 9 Sent Done Done false
38045 10068 Acked Write 1 Sent Done Done false
38046 10069 Acked Write 5 Sent Done Done false
38047 10070 Acked Read 4 Sent Done Done false
38048 10071 Acked Flush 0 Sent Done Done false
38049 10072 Acked Flush 0 Sent Done Done false
38050 10073 Acked Write 9 Sent Done Done false
38051 10074 Acked Read 9 Sent Done Done false
38052 10075 Acked Flush 0 Sent Done Done false
38053 10076 Acked Read 3 Sent Done Done false
38054 10077 Acked Read 2 Sent Done Done false
38055 10078 Acked Read 7 Sent Done Done false
38056 10079 Acked Read 6 Sent Done Done false
38057 10080 Acked Read 2 Sent Done Done false
38058 10081 Acked Read 4 Sent Done Done false
38059 10082 Acked Read 9 Sent Done Done false
38060 10083 Acked Write 10 Sent Done Done false
38061 10084 Acked Read 10 Sent Done Done false
38062 10085 Acked Read 7 Sent Done Done false
38063 10086 Acked Write 6 Sent Done Done false
38064 10087 Acked Read 1 Sent Done Done false
38065 10088 Acked Write 6 Sent Done Done false
38066 10089 Acked Write 8 Sent Done Done false
38067 10090 Acked Read 10 Sent Done Done false
38068 10091 Acked Read 8 Sent Done Done false
38069 10092 Acked Read 1 Sent Done Done false
38070 10093 Acked Read 7 Sent Done Done false
38071 10094 Acked Write 6 Sent Done Done false
38072 10095 Acked Write 4 Sent Done Done false
38073 10096 Acked Write 9 Sent Done Done false
38074 10097 Acked Read 1 Sent Done Done false
38075 10098 Acked Write 10 Sent Done Done false
38076 10099 Acked Write 6 Sent Done Done false
38077 10100 Acked Flush 0 Sent Done Done false
38078 10101 Acked Write 8 Sent Done Done false
38079 10102 Acked Read 4 Sent Done Done false
38080 10103 Acked Read 1 Sent Done Done false
38081 10104 Acked Read 6 Sent Done Done false
38082 10105 Acked Read 6 Sent Done Done false
38083 10106 Acked Read 1 Sent Done Done false
38084 10107 Acked Flush 0 Sent Done Done false
38085 10108 Acked Read 3 Sent Done Done false
38086 10109 Acked Read 2 Sent Done Done false
38087 10110 Acked Read 3 Sent Done Done false
38088 10111 Acked Flush 0 Sent Done Done false
38089 10112 Acked Read 1 Sent Done Done false
38090 10113 Acked Write 6 Sent Done Done false
38091 10114 Acked Flush 0 Sent Done Done false
38092 10115 Acked Write 3 Sent Done Done false
38093 10116 Acked Read 9 Sent Done Done false
38094 10117 Acked Write 3 Sent Done Done false
38095 10118 Acked Write 10 Sent Done Done false
38096 10119 Acked Write 9 Sent Done Done false
38097 10120 Acked Write 10 Sent Done Done false
38098 10121 Acked Flush 0 Sent Done Done false
38099 10122 Acked Flush 0 Sent Done Done false
38100 10123 Acked Write 3 Sent Done Done false
38101 10124 Acked Read 10 Sent Done Done false
38102 10125 Acked Write 8 Sent Done Done false
38103 10126 Acked Write 4 Sent Done Done false
38104 10127 Acked Read 6 Sent Done Done false
38105 10128 Acked Flush 0 Sent Done Done false
38106 10129 Acked Read 8 Sent Done Done false
38107 10130 Acked Read 7 Sent Done Done false
38108 10131 Acked Write 10 Sent Done Done false
38109 10132 Acked Write 4 Sent Done Done false
38110 10133 Acked Write 10 Sent Done Done false
38111 10134 Acked Read 4 Sent Done Done false
38112 10135 Acked Read 9 Sent Done Done false
38113 10136 Acked Read 9 Sent Done Done false
38114 10137 Acked Write 10 Sent Done Done false
38115 10138 Acked Write 7 Sent Done Done false
38116 10139 Acked Write 5 Sent Done Done false
38117 10140 Acked Read 4 Sent Done Done false
38118 10141 Acked Write 3 Sent Done Done false
38119 10142 Acked Write 8 Sent Done Done false
38120 10143 Acked Flush 0 Sent Done Done false
38121 10144 Acked Write 9 Sent Done Done false
38122 10145 Acked Read 7 Sent Done Done false
38123 10146 Acked Write 1 Sent Done Done false
38124 10147 Acked Read 9 Sent Done Done false
38125 10148 Acked Write 2 Sent Done Done false
38126 10149 Acked Read 3 Sent Done Done false
38127 10150 Acked Write 2 Sent Done Done false
38128 10151 Acked Read 10 Sent Done Done false
38129 10152 Acked Flush 0 Sent Done Done false
38130 10153 Acked Write 2 Sent Done Done false
38131 10154 Acked Write 9 Sent Done Done false
38132 10155 Acked Write 6 Sent Done Done false
38133 10156 Acked Read 7 Sent Done Done false
38134 10157 Acked Write 6 Sent Done Done false
38135 10158 Acked Flush 0 Sent Done Done false
38136 10159 Acked Write 4 Sent Done Done false
38137 10160 Acked Read 4 Sent Done Done false
38138 10161 Acked Write 3 Sent Done Done false
38139 10162 Acked Read 2 Sent Done Done false
38140 10163 Acked Read 5 Sent Done Done false
38141 10164 Acked Write 1 Sent Done Done false
38142 10165 Acked Read 3 Sent Done Done false
38143 10166 Acked Write 1 Sent Done Done false
38144 10167 Acked Read 1 Sent Done Done false
38145 10168 Acked Read 5 Sent Done Done false
38146 10169 Acked Read 5 Sent Done Done false
38147 10170 Acked Flush 0 Sent Done Done false
38148 10171 Acked Flush 0 Sent Done Done false
38149 10172 Acked Write 10 Sent Done Done false
38150 10173 Acked Write 5 Sent Done Done false
38151 10174 Acked Read 3 Sent Done Done false
38152 10175 Acked Write 3 Sent Done Done false
38153 10176 Acked Write 5 Sent Done Done false
38154 10177 Acked Read 7 Sent Done Done false
38155 10178 Acked Flush 0 Sent Done Done false
38156 10179 Acked Write 4 Sent Done Done false
38157 10180 Acked Read 10 Sent Done Done false
38158 10181 Acked Read 4 Sent Done Done false
38159 10182 Acked Write 4 Sent Done Done false
38160 10183 Acked Read 8 Sent Done Done false
38161 10184 Acked Read 1 Sent Done Done false
38162 10185 Acked Flush 0 Sent Done Done false
38163 10186 Acked Write 10 Sent Done Done false
38164 10187 Acked Read 6 Sent Done Done false
38165 10188 Acked Write 8 Sent Done Done false
38166 10189 Acked Write 9 Sent Done Done false
38167 10190 Acked Read 8 Sent Done Done false
38168 10191 Acked Flush 0 Sent Done Done false
38169 10192 Acked Write 1 Sent Done Done false
38170 10193 Acked Write 4 Sent Done Done false
38171 10194 Acked Read 6 Sent Done Done false
38172 10195 Acked Flush 0 Sent Done Done false
38173 10196 Acked Read 6 Sent Done Done false
38174 10197 Acked Read 2 Sent Done Done false
38175 10198 Acked Read 4 Sent Done Done false
38176 10199 Acked Write 10 Sent Done Done false
38177 10200 Acked Flush 0 Sent Done Done false
38178 10201 Acked Flush 0 Sent Done Done false
38179 10202 Acked Write 4 Sent Done Done false
38180 10203 Acked Read 2 Sent Done Done false
38181 10204 Acked Read 2 Sent Done Done false
38182 10205 Acked Read 5 Sent Done Done false
38183 10206 Acked Read 9 Sent Done Done false
38184 10207 Acked Write 4 Sent Done Done false
38185 10208 Acked Write 6 Sent Done Done false
38186 10209 Acked Read 6 Sent Done Done false
38187 10210 Acked Read 4 Sent Done Done false
38188 10211 Acked Read 1 Sent Done Done false
38189 10212 Acked Write 1 Sent Done Done false
38190 10213 Acked Read 6 Sent Done Done false
38191 10214 Acked Write 7 Sent Done Done false
38192 10215 Acked Read 8 Sent Done Done false
38193 10216 Acked Write 2 Sent Done Done false
38194 10217 Acked Read 1 Sent Done Done false
38195 10218 Acked Write 3 Sent Done Done false
38196 10219 Acked Read 5 Sent Done Done false
38197 10220 Acked Read 6 Sent Done Done false
38198 10221 Acked Write 3 Sent Done Done false
38199 10222 Acked Write 5 Sent Done Done false
38200 10223 Acked Write 7 Sent Done Done false
38201 10224 Acked Flush 0 Sent Done Done false
38202 10225 Acked Write 5 Sent Done Done false
38203 10226 Acked Read 8 Sent Done Done false
38204 10227 Acked Read 4 Sent Done Done false
38205 10228 Acked Read 1 Sent Done Done false
38206 10229 Acked Write 7 Sent Done Done false
38207 10230 Acked Read 2 Sent Done Done false
38208 10231 Acked Read 8 Sent Done Done false
38209 10232 Acked Read 4 Sent Done Done false
38210 10233 Acked Write 3 Sent Done Done false
38211 10234 Acked Write 2 Sent Done Done false
38212 10235 Acked Read 1 Sent Done Done false
38213 10236 Acked Read 4 Sent Done Done false
38214 10237 Acked Write 7 Sent Done Done false
38215 10238 Acked Write 8 Sent Done Done false
38216 10239 Acked Read 10 Sent Done Done false
38217 10240 Acked Read 7 Sent Done Done false
38218 10241 Acked Write 6 Sent Done Done false
38219 10242 Acked Write 4 Sent Done Done false
38220 10243 Acked Write 4 Sent Done Done false
38221 10244 Acked Read 6 Sent Done Done false
38222 10245 Acked Write 4 Sent Done Done false
38223 10246 Acked Read 8 Sent Done Done false
38224 10247 Acked Read 8 Sent Done Done false
38225 10248 Acked Write 8 Sent Done Done false
38226 10249 Acked Read 8 Sent Done Done false
38227 10250 Acked Read 3 Sent Done Done false
38228 10251 Acked Write 9 Sent Done Done false
38229 10252 Acked Write 7 Sent Done Done false
38230 10253 Acked Write 1 Sent Done Done false
38231 10254 Acked Write 8 Sent Done Done false
38232 10255 Acked Read 4 Sent Done Done false
38233 10256 Acked Read 3 Sent Done Done false
38234 10257 Acked Read 2 Sent Done Done false
38235 10258 Acked Write 1 Sent Done Done false
38236 10259 Acked Write 9 Sent Done Done false
38237 10260 Acked Write 8 Sent Done Done false
38238 10261 Acked Write 7 Sent Done Done false
38239 10262 Acked Write 8 Sent Done Done false
38240 10263 Acked Write 8 Sent Done Done false
38241 10264 Acked Read 8 Sent Done Done false
38242 10265 Acked Read 6 Sent Done Done false
38243 10266 Acked Read 2 Sent Done Done false
38244 10267 Acked Read 2 Sent Done Done false
38245 10268 Acked Read 9 Sent Done Done false
38246 10269 Acked Read 10 Sent Done Done false
38247 10270 Acked Write 9 Sent Done Done false
38248 10271 Acked Read 9 Sent Done Done false
38249 10272 Acked Write 10 Sent Done Done false
38250 10273 Acked Read 1 Sent Done Done false
38251 10274 Acked Read 1 Sent Done Done false
38252 10275 Acked Flush 0 Sent Done Done false
38253 10276 Acked Write 5 Sent Done Done false
38254 10277 Acked Write 2 Sent Done Done false
38255 10278 Acked Write 10 Sent Done Done false
38256 10279 Acked Read 1 Sent Done Done false
38257 10280 Acked Write 7 Sent Done Done false
38258 10281 Acked Write 10 Sent Done Done false
38259 10282 Acked Read 1 Sent Done Done false
38260 10283 Acked Read 7 Sent Done Done false
38261 10284 Acked Flush 0 Sent Done Done false
38262 10285 Acked Write 8 Sent Done Done false
38263 10286 Acked Read 3 Sent Done Done false
38264 10287 Acked Read 10 Sent Done Done false
38265 10288 Acked Read 4 Sent Done Done false
38266 10289 Acked Read 3 Sent Done Done false
38267 10290 Acked Flush 0 Sent Done Done false
38268 10291 Acked Read 10 Sent Done Done false
38269 10292 Acked Read 1 Sent Done Done false
38270 10293 Acked Read 3 Sent Done Done false
38271 10294 Acked Write 10 Sent Done Done false
38272 10295 Acked Write 4 Sent Done Done false
38273 10296 Acked Read 8 Sent Done Done false
38274 10297 Acked Write 9 Sent Done Done false
38275 10298 Acked Read 2 Sent Done Done false
38276 10299 Acked Read 5 Sent Done Done false
38277 10300 Acked Read 5 Sent Done Done false
38278 10301 Acked Write 7 Sent Done Done false
38279 10302 Acked Write 5 Sent Done Done false
38280 10303 Acked Read 10 Sent Done Done false
38281 10304 Acked Read 2 Sent Done Done false
38282 10305 Acked Write 5 Sent Done Done false
38283 10306 Acked Write 4 Sent Done Done false
38284 10307 Acked Read 8 Sent Done Done false
38285 10308 Acked Read 5 Sent Done Done false
38286 10309 Acked Write 6 Sent Done Done false
38287 10310 Acked Read 9 Sent Done Done false
38288 10311 Acked Read 7 Sent Done Done false
38289 10312 Acked Write 7 Sent Done Done false
38290 10313 Acked Write 3 Sent Done Done false
38291 10314 Acked Write 5 Sent Done Done false
38292 10315 Acked Read 6 Sent Done Done false
38293 10316 Acked Read 9 Sent Done Done false
38294 10317 Acked Write 5 Sent Done Done false
38295 10318 Acked Flush 0 Sent Done Done false
38296 10319 Acked Read 3 Sent Done Done false
38297 10320 Acked Read 4 Sent Done Done false
38298 10321 Acked Read 2 Sent Done Done false
38299 10322 Acked Read 2 Sent Done Done false
38300 10323 Acked Write 7 Sent Done Done false
38301 10324 Acked Write 8 Sent Done Done false
38302 10325 Acked Flush 0 Sent Done Done false
38303 10326 Acked Read 7 Sent Done Done false
38304 10327 Acked Read 10 Sent Done Done false
38305 10328 Acked Read 10 Sent Done Done false
38306 10329 Acked Read 9 Sent Done Done false
38307 10330 Acked Read 5 Sent Done Done false
38308 10331 Acked Read 3 Sent Done Done false
38309 10332 Acked Read 4 Sent Done Done false
38310 10333 Acked Write 9 Sent Done Done false
38311 10334 Acked Write 5 Sent Done Done false
38312 10335 Acked Read 2 Sent Done Done false
38313 10336 Acked Write 8 Sent Done Done false
38314 10337 Acked Write 2 Sent Done Done false
38315 10338 Acked Read 4 Sent Done Done false
38316 10339 Acked Read 4 Sent Done Done false
38317 10340 Acked Read 1 Sent Done Done false
38318 10341 Acked Read 7 Sent Done Done false
38319 10342 Acked Read 5 Sent Done Done false
38320 10343 Acked Write 7 Sent Done Done false
38321 10344 Acked Write 2 Sent Done Done false
38322 10345 Acked Write 5 Sent Done Done false
38323 10346 Acked Write 6 Sent Done Done false
38324 10347 Acked Read 4 Sent Done Done false
38325 10348 Acked Write 3 Sent Done Done false
38326 10349 Acked Read 1 Sent Done Done false
38327 10350 Acked Write 5 Sent Done Done false
38328 10351 Acked Read 5 Sent Done Done false
38329 10352 Acked Read 6 Sent Done Done false
38330 10353 Acked Read 8 Sent Done Done false
38331 10354 Acked Read 6 Sent Done Done false
38332 10355 Acked Read 5 Sent Done Done false
38333 10356 Acked Read 8 Sent Done Done false
38334 10357 Acked Write 1 Sent Done Done false
38335 10358 Acked Read 5 Sent Done Done false
38336 STATES DS:0 DS:1 DS:2 TOTAL
38337 Sent 301 0 0 301
38338 Done 0 301 301 602
38339 Skipped 0 0 0 0
38340 Error 0 0 0 0
38341 Last Flush: 10057 10325 10325
38342 Downstairs last five completed: 10057 10056 10055 10054 10053
38343 Upstairs last five completed: 10358 10357 10356 10355 10354
38344 CLIENT: Up:0 ds:301 act:2
38345 [8810] got start action
38346 [8810] I am going to restart
38347 Starting downstairs at port 8810
38348 Make output file at "/tmp/dsc/downstairs-8810.txt"
38349 [8810][0] reports Starting
38350 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1236)
38351 [8810][0] reports Running
38352 ----------------------------------------------------------------
38353 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
38354 States: Offline Active Active
38355 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
38356 10058 Acked Write 6 Sent Done Done false
38357 10059 Acked Read 3 Sent Done Done false
38358 10060 Acked Read 1 Sent Done Done false
38359 10061 Acked Write 5 Sent Done Done false
38360 10062 Acked Write 1 Sent Done Done false
38361 10063 Acked Write 2 Sent Done Done false
38362 10064 Acked Write 7 Sent Done Done false
38363 10065 Acked Flush 0 Sent Done Done false
38364 10066 Acked Read 6 Sent Done Done false
38365 10067 Acked Write 9 Sent Done Done false
38366 10068 Acked Write 1 Sent Done Done false
38367 10069 Acked Write 5 Sent Done Done false
38368 10070 Acked Read 4 Sent Done Done false
38369 10071 Acked Flush 0 Sent Done Done false
38370 10072 Acked Flush 0 Sent Done Done false
38371 10073 Acked Write 9 Sent Done Done false
38372 10074 Acked Read 9 Sent Done Done false
38373 10075 Acked Flush 0 Sent Done Done false
38374 10076 Acked Read 3 Sent Done Done false
38375 10077 Acked Read 2 Sent Done Done false
38376 10078 Acked Read 7 Sent Done Done false
38377 10079 Acked Read 6 Sent Done Done false
38378 10080 Acked Read 2 Sent Done Done false
38379 10081 Acked Read 4 Sent Done Done false
38380 10082 Acked Read 9 Sent Done Done false
38381 10083 Acked Write 10 Sent Done Done false
38382 10084 Acked Read 10 Sent Done Done false
38383 10085 Acked Read 7 Sent Done Done false
38384 10086 Acked Write 6 Sent Done Done false
38385 10087 Acked Read 1 Sent Done Done false
38386 10088 Acked Write 6 Sent Done Done false
38387 10089 Acked Write 8 Sent Done Done false
38388 10090 Acked Read 10 Sent Done Done false
38389 10091 Acked Read 8 Sent Done Done false
38390 10092 Acked Read 1 Sent Done Done false
38391 10093 Acked Read 7 Sent Done Done false
38392 10094 Acked Write 6 Sent Done Done false
38393 10095 Acked Write 4 Sent Done Done false
38394 10096 Acked Write 9 Sent Done Done false
38395 10097 Acked Read 1 Sent Done Done false
38396 10098 Acked Write 10 Sent Done Done false
38397 10099 Acked Write 6 Sent Done Done false
38398 10100 Acked Flush 0 Sent Done Done false
38399 10101 Acked Write 8 Sent Done Done false
38400 10102 Acked Read 4 Sent Done Done false
38401 10103 Acked Read 1 Sent Done Done false
38402 10104 Acked Read 6 Sent Done Done false
38403 10105 Acked Read 6 Sent Done Done false
38404 10106 Acked Read 1 Sent Done Done false
38405 10107 Acked Flush 0 Sent Done Done false
38406 10108 Acked Read 3 Sent Done Done false
38407 10109 Acked Read 2 Sent Done Done false
38408 10110 Acked Read 3 Sent Done Done false
38409 10111 Acked Flush 0 Sent Done Done false
38410 10112 Acked Read 1 Sent Done Done false
38411 10113 Acked Write 6 Sent Done Done false
38412 10114 Acked Flush 0 Sent Done Done false
38413 10115 Acked Write 3 Sent Done Done false
38414 10116 Acked Read 9 Sent Done Done false
38415 10117 Acked Write 3 Sent Done Done false
38416 10118 Acked Write 10 Sent Done Done false
38417 10119 Acked Write 9 Sent Done Done false
38418 10120 Acked Write 10 Sent Done Done false
38419 10121 Acked Flush 0 Sent Done Done false
38420 10122 Acked Flush 0 Sent Done Done false
38421 10123 Acked Write 3 Sent Done Done false
38422 10124 Acked Read 10 Sent Done Done false
38423 10125 Acked Write 8 Sent Done Done false
38424 10126 Acked Write 4 Sent Done Done false
38425 10127 Acked Read 6 Sent Done Done false
38426 10128 Acked Flush 0 Sent Done Done false
38427 10129 Acked Read 8 Sent Done Done false
38428 10130 Acked Read 7 Sent Done Done false
38429 10131 Acked Write 10 Sent Done Done false
38430 10132 Acked Write 4 Sent Done Done false
38431 10133 Acked Write 10 Sent Done Done false
38432 10134 Acked Read 4 Sent Done Done false
38433 10135 Acked Read 9 Sent Done Done false
38434 10136 Acked Read 9 Sent Done Done false
38435 10137 Acked Write 10 Sent Done Done false
38436 10138 Acked Write 7 Sent Done Done false
38437 10139 Acked Write 5 Sent Done Done false
38438 10140 Acked Read 4 Sent Done Done false
38439 10141 Acked Write 3 Sent Done Done false
38440 10142 Acked Write 8 Sent Done Done false
38441 10143 Acked Flush 0 Sent Done Done false
38442 10144 Acked Write 9 Sent Done Done false
38443 10145 Acked Read 7 Sent Done Done false
38444 10146 Acked Write 1 Sent Done Done false
38445 10147 Acked Read 9 Sent Done Done false
38446 10148 Acked Write 2 Sent Done Done false
38447 10149 Acked Read 3 Sent Done Done false
38448 10150 Acked Write 2 Sent Done Done false
38449 10151 Acked Read 10 Sent Done Done false
38450 10152 Acked Flush 0 Sent Done Done false
38451 10153 Acked Write 2 Sent Done Done false
38452 10154 Acked Write 9 Sent Done Done false
38453 10155 Acked Write 6 Sent Done Done false
38454 10156 Acked Read 7 Sent Done Done false
38455 10157 Acked Write 6 Sent Done Done false
38456 10158 Acked Flush 0 Sent Done Done false
38457 10159 Acked Write 4 Sent Done Done false
38458 10160 Acked Read 4 Sent Done Done false
38459 10161 Acked Write 3 Sent Done Done false
38460 10162 Acked Read 2 Sent Done Done false
38461 10163 Acked Read 5 Sent Done Done false
38462 10164 Acked Write 1 Sent Done Done false
38463 10165 Acked Read 3 Sent Done Done false
38464 10166 Acked Write 1 Sent Done Done false
38465 10167 Acked Read 1 Sent Done Done false
38466 10168 Acked Read 5 Sent Done Done false
38467 10169 Acked Read 5 Sent Done Done false
38468 10170 Acked Flush 0 Sent Done Done false
38469 10171 Acked Flush 0 Sent Done Done false
38470 10172 Acked Write 10 Sent Done Done false
38471 10173 Acked Write 5 Sent Done Done false
38472 10174 Acked Read 3 Sent Done Done false
38473 10175 Acked Write 3 Sent Done Done false
38474 10176 Acked Write 5 Sent Done Done false
38475 10177 Acked Read 7 Sent Done Done false
38476 10178 Acked Flush 0 Sent Done Done false
38477 10179 Acked Write 4 Sent Done Done false
38478 10180 Acked Read 10 Sent Done Done false
38479 10181 Acked Read 4 Sent Done Done false
38480 10182 Acked Write 4 Sent Done Done false
38481 10183 Acked Read 8 Sent Done Done false
38482 10184 Acked Read 1 Sent Done Done false
38483 10185 Acked Flush 0 Sent Done Done false
38484 10186 Acked Write 10 Sent Done Done false
38485 10187 Acked Read 6 Sent Done Done false
38486 10188 Acked Write 8 Sent Done Done false
38487 10189 Acked Write 9 Sent Done Done false
38488 10190 Acked Read 8 Sent Done Done false
38489 10191 Acked Flush 0 Sent Done Done false
38490 10192 Acked Write 1 Sent Done Done false
38491 10193 Acked Write 4 Sent Done Done false
38492 10194 Acked Read 6 Sent Done Done false
38493 10195 Acked Flush 0 Sent Done Done false
38494 10196 Acked Read 6 Sent Done Done false
38495 10197 Acked Read 2 Sent Done Done false
38496 10198 Acked Read 4 Sent Done Done false
38497 10199 Acked Write 10 Sent Done Done false
38498 10200 Acked Flush 0 Sent Done Done false
38499 10201 Acked Flush 0 Sent Done Done false
38500 10202 Acked Write 4 Sent Done Done false
38501 10203 Acked Read 2 Sent Done Done false
38502 10204 Acked Read 2 Sent Done Done false
38503 10205 Acked Read 5 Sent Done Done false
38504 10206 Acked Read 9 Sent Done Done false
38505 10207 Acked Write 4 Sent Done Done false
38506 10208 Acked Write 6 Sent Done Done false
38507 10209 Acked Read 6 Sent Done Done false
38508 10210 Acked Read 4 Sent Done Done false
38509 10211 Acked Read 1 Sent Done Done false
38510 10212 Acked Write 1 Sent Done Done false
38511 10213 Acked Read 6 Sent Done Done false
38512 10214 Acked Write 7 Sent Done Done false
38513 10215 Acked Read 8 Sent Done Done false
38514 10216 Acked Write 2 Sent Done Done false
38515 10217 Acked Read 1 Sent Done Done false
38516 10218 Acked Write 3 Sent Done Done false
38517 10219 Acked Read 5 Sent Done Done false
38518 10220 Acked Read 6 Sent Done Done false
38519 10221 Acked Write 3 Sent Done Done false
38520 10222 Acked Write 5 Sent Done Done false
38521 10223 Acked Write 7 Sent Done Done false
38522 10224 Acked Flush 0 Sent Done Done false
38523 10225 Acked Write 5 Sent Done Done false
38524 10226 Acked Read 8 Sent Done Done false
38525 10227 Acked Read 4 Sent Done Done false
38526 10228 Acked Read 1 Sent Done Done false
38527 10229 Acked Write 7 Sent Done Done false
38528 10230 Acked Read 2 Sent Done Done false
38529 10231 Acked Read 8 Sent Done Done false
38530 10232 Acked Read 4 Sent Done Done false
38531 10233 Acked Write 3 Sent Done Done false
38532 10234 Acked Write 2 Sent Done Done false
38533 10235 Acked Read 1 Sent Done Done false
38534 10236 Acked Read 4 Sent Done Done false
38535 10237 Acked Write 7 Sent Done Done false
38536 10238 Acked Write 8 Sent Done Done false
38537 10239 Acked Read 10 Sent Done Done false
38538 10240 Acked Read 7 Sent Done Done false
38539 10241 Acked Write 6 Sent Done Done false
38540 10242 Acked Write 4 Sent Done Done false
38541 10243 Acked Write 4 Sent Done Done false
38542 10244 Acked Read 6 Sent Done Done false
38543 10245 Acked Write 4 Sent Done Done false
38544 10246 Acked Read 8 Sent Done Done false
38545 10247 Acked Read 8 Sent Done Done false
38546 10248 Acked Write 8 Sent Done Done false
38547 10249 Acked Read 8 Sent Done Done false
38548 10250 Acked Read 3 Sent Done Done false
38549 10251 Acked Write 9 Sent Done Done false
38550 10252 Acked Write 7 Sent Done Done false
38551 10253 Acked Write 1 Sent Done Done false
38552 10254 Acked Write 8 Sent Done Done false
38553 10255 Acked Read 4 Sent Done Done false
38554 10256 Acked Read 3 Sent Done Done false
38555 10257 Acked Read 2 Sent Done Done false
38556 10258 Acked Write 1 Sent Done Done false
38557 10259 Acked Write 9 Sent Done Done false
38558 10260 Acked Write 8 Sent Done Done false
38559 10261 Acked Write 7 Sent Done Done false
38560 10262 Acked Write 8 Sent Done Done false
38561 10263 Acked Write 8 Sent Done Done false
38562 10264 Acked Read 8 Sent Done Done false
38563 10265 Acked Read 6 Sent Done Done false
38564 10266 Acked Read 2 Sent Done Done false
38565 10267 Acked Read 2 Sent Done Done false
38566 10268 Acked Read 9 Sent Done Done false
38567 10269 Acked Read 10 Sent Done Done false
38568 10270 Acked Write 9 Sent Done Done false
38569 10271 Acked Read 9 Sent Done Done false
38570 10272 Acked Write 10 Sent Done Done false
38571 10273 Acked Read 1 Sent Done Done false
38572 10274 Acked Read 1 Sent Done Done false
38573 10275 Acked Flush 0 Sent Done Done false
38574 10276 Acked Write 5 Sent Done Done false
38575 10277 Acked Write 2 Sent Done Done false
38576 10278 Acked Write 10 Sent Done Done false
38577 10279 Acked Read 1 Sent Done Done false
38578 10280 Acked Write 7 Sent Done Done false
38579 10281 Acked Write 10 Sent Done Done false
38580 10282 Acked Read 1 Sent Done Done false
38581 10283 Acked Read 7 Sent Done Done false
38582 10284 Acked Flush 0 Sent Done Done false
38583 10285 Acked Write 8 Sent Done Done false
38584 10286 Acked Read 3 Sent Done Done false
38585 10287 Acked Read 10 Sent Done Done false
38586 10288 Acked Read 4 Sent Done Done false
38587 10289 Acked Read 3 Sent Done Done false
38588 10290 Acked Flush 0 Sent Done Done false
38589 10291 Acked Read 10 Sent Done Done false
38590 10292 Acked Read 1 Sent Done Done false
38591 10293 Acked Read 3 Sent Done Done false
38592 10294 Acked Write 10 Sent Done Done false
38593 10295 Acked Write 4 Sent Done Done false
38594 10296 Acked Read 8 Sent Done Done false
38595 10297 Acked Write 9 Sent Done Done false
38596 10298 Acked Read 2 Sent Done Done false
38597 10299 Acked Read 5 Sent Done Done false
38598 10300 Acked Read 5 Sent Done Done false
38599 10301 Acked Write 7 Sent Done Done false
38600 10302 Acked Write 5 Sent Done Done false
38601 10303 Acked Read 10 Sent Done Done false
38602 10304 Acked Read 2 Sent Done Done false
38603 10305 Acked Write 5 Sent Done Done false
38604 10306 Acked Write 4 Sent Done Done false
38605 10307 Acked Read 8 Sent Done Done false
38606 10308 Acked Read 5 Sent Done Done false
38607 10309 Acked Write 6 Sent Done Done false
38608 10310 Acked Read 9 Sent Done Done false
38609 10311 Acked Read 7 Sent Done Done false
38610 10312 Acked Write 7 Sent Done Done false
38611 10313 Acked Write 3 Sent Done Done false
38612 10314 Acked Write 5 Sent Done Done false
38613 10315 Acked Read 6 Sent Done Done false
38614 10316 Acked Read 9 Sent Done Done false
38615 10317 Acked Write 5 Sent Done Done false
38616 10318 Acked Flush 0 Sent Done Done false
38617 10319 Acked Read 3 Sent Done Done false
38618 10320 Acked Read 4 Sent Done Done false
38619 10321 Acked Read 2 Sent Done Done false
38620 10322 Acked Read 2 Sent Done Done false
38621 10323 Acked Write 7 Sent Done Done false
38622 10324 Acked Write 8 Sent Done Done false
38623 10325 Acked Flush 0 Sent Done Done false
38624 10326 Acked Read 7 Sent Done Done false
38625 10327 Acked Read 10 Sent Done Done false
38626 10328 Acked Read 10 Sent Done Done false
38627 10329 Acked Read 9 Sent Done Done false
38628 10330 Acked Read 5 Sent Done Done false
38629 10331 Acked Read 3 Sent Done Done false
38630 10332 Acked Read 4 Sent Done Done false
38631 10333 Acked Write 9 Sent Done Done false
38632 10334 Acked Write 5 Sent Done Done false
38633 10335 Acked Read 2 Sent Done Done false
38634 10336 Acked Write 8 Sent Done Done false
38635 10337 Acked Write 2 Sent Done Done false
38636 10338 Acked Read 4 Sent Done Done false
38637 10339 Acked Read 4 Sent Done Done false
38638 10340 Acked Read 1 Sent Done Done false
38639 10341 Acked Read 7 Sent Done Done false
38640 10342 Acked Read 5 Sent Done Done false
38641 10343 Acked Write 7 Sent Done Done false
38642 10344 Acked Write 2 Sent Done Done false
38643 10345 Acked Write 5 Sent Done Done false
38644 10346 Acked Write 6 Sent Done Done false
38645 10347 Acked Read 4 Sent Done Done false
38646 10348 Acked Write 3 Sent Done Done false
38647 10349 Acked Read 1 Sent Done Done false
38648 10350 Acked Write 5 Sent Done Done false
38649 10351 Acked Read 5 Sent Done Done false
38650 10352 Acked Read 6 Sent Done Done false
38651 10353 Acked Read 8 Sent Done Done false
38652 10354 Acked Read 6 Sent Done Done false
38653 10355 Acked Read 5 Sent Done Done false
38654 10356 Acked Read 8 Sent Done Done false
38655 10357 Acked Write 1 Sent Done Done false
38656 10358 Acked Read 5 Sent Done Done false
38657 10359 Acked Flush 0 Sent Done Done false
38658 STATES DS:0 DS:1 DS:2 TOTAL
38659 Sent 302 0 0 302
38660 Done 0 302 302 604
38661 Skipped 0 0 0 0
38662 Error 0 0 0 0
38663 Last Flush: 10057 10359 10359
38664 Downstairs last five completed: 10057 10056 10055 10054 10053
38665 Upstairs last five completed: 10359 10358 10357 10356 10355
38666 CLIENT: Up:0 ds:302 act:2
386672024-11-15T02:05:33.295ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
386682024-11-15T02:05:33.295ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:58262) = io task client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
386692024-11-15T02:05:33.296ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
386702024-11-15T02:05:33.297ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
386712024-11-15T02:05:33.297ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
386722024-11-15T02:05:33.297ZINFOcrucible: send last flush ID to this DS: 10057 = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
386732024-11-15T02:05:33.297ZINFOcrucible: Replied this last flush ID: 10057 = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
386742024-11-15T02:05:33.297ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
386752024-11-15T02:05:33.297ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
386762024-11-15T02:05:33.297ZINFOcrucible: [0] client re-new 302 jobs since flush 10057 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
386772024-11-15T02:05:33.297ZINFOcrucible: [0] Replayed 302 jobs since flush: 10057 = downstairs session_id = 971dcb2e-6891-4ecf-b367-1e04e52065ee
38678 ----------------------------------------------------------------
38679 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
38680 Downstairs last five completed: 10359 10358 10357 10356 10355
38681 Upstairs last five completed: 10359 10358 10357 10356 10355
38682 CLIENT: Up:0 ds:0 act:3
38683 Replay: All jobs finished
38684 Test replay has completed
38685 Wrote out file "/tmp/test_replay_verify.log"
38686 CLIENT: Tests done. All submitted work has been ACK'd
38687 ----------------------------------------------------------------
38688 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
38689 Downstairs last five completed: 10359 10358 10357 10356 10355
38690 Upstairs last five completed: 10359 10358 10357 10356 10355
38691 CLIENT: Up:0 ds:0 act:3
38692 CLIENT: All jobs finished, all DS active.
386932024-11-15T02:05:35.535ZWARNcrucible: Guest handle has been dropped
38694 Replays:30 time: 8:04
38695 Do final verify
38696 Nov 15 02:05:35.609 INFO accepted connection, remote_addr: 127.0.0.1:64391, local_addr: 127.0.0.1:9998
38697 Nov 15 02:05:35.610 INFO request completed, latency_us: 202, response_code: 200, uri: /regioninfo, method: GET, req_id: 9d68b2c4-012f-4a1c-a290-b100e419d7a0, remote_addr: 127.0.0.1:64391, local_addr: 127.0.0.1:9998
386982024-11-15T02:05:35.610ZINFOcrucible: Use this region info from dsc: RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 } task = crutest
38699 Nov 15 02:05:35.611 INFO request completed, latency_us: 112, response_code: 200, uri: /regioncount, method: GET, req_id: f47d0bd6-9884-48aa-b4cb-b0fc0982804e, remote_addr: 127.0.0.1:64391, local_addr: 127.0.0.1:9998
387002024-11-15T02:05:35.611ZINFOcrucible: dsc has 3 regions. This means 1 sub_volumes task = crutest
38701 Nov 15 02:05:35.611 INFO request completed, latency_us: 111, response_code: 200, uri: /port/cid/0, method: GET, req_id: c399d9d6-a7a2-48c7-8ab5-81b92f9836ba, remote_addr: 127.0.0.1:64391, local_addr: 127.0.0.1:9998
38702 Nov 15 02:05:35.612 INFO request completed, latency_us: 94, response_code: 200, uri: /port/cid/1, method: GET, req_id: e7d344f0-afac-4e4d-9c8e-810d4a417b68, remote_addr: 127.0.0.1:64391, local_addr: 127.0.0.1:9998
38703 Nov 15 02:05:35.613 INFO request completed, latency_us: 93, response_code: 200, uri: /port/cid/2, method: GET, req_id: 4deae152-7c1d-4eb3-bac8-79db9e834f34, remote_addr: 127.0.0.1:64391, local_addr: 127.0.0.1:9998
387042024-11-15T02:05:35.613ZINFOcrucible: SV 0 has targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830] task = crutest
387052024-11-15T02:05:35.613ZINFOcrucible: Upstairs starts
387062024-11-15T02:05:35.613ZINFOcrucible: Crucible Version: BuildInfo { version: "0.0.1", git_sha: "5698e843f5bfd164775ff7ff0d12625b59494d67", git_commit_timestamp: "2024-11-15T01:38:50.000000000Z", git_branch: "alan/global-state-from-dtrace", rustc_semver: "1.82.0", rustc_channel: "stable", rustc_host_triple: "x86_64-unknown-illumos", rustc_commit_sha: "f6e511eec7342f59a25f7c0534f1dbea00d01b14", cargo_triple: "x86_64-unknown-illumos", debug: true, opt_level: 0, }
387072024-11-15T02:05:35.613ZINFOcrucible: Upstairs <-> Downstairs Message Version: 12
387082024-11-15T02:05:35.613ZINFOcrucible: Using region definition RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 00000000-0000-0000-0000-000000000000, encrypted: false, database_read_version: 1, database_write_version: 1 }
387092024-11-15T02:05:35.614ZINFOcrucible: Crucible 16fe4894-c7df-4434-b37b-9fbbd5c72f4f has session id: 21da52e5-eac9-4881-9a24-ca141dc547e1 session_id = 21da52e5-eac9-4881-9a24-ca141dc547e1
387102024-11-15T02:05:35.614ZINFOcrucible: Upstairs opts: Upstairs UUID: 16fe4894-c7df-4434-b37b-9fbbd5c72f4f, Targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830], lossy: false, flush_timeout: None, key populated: false, cert_pem populated: false, key_pem populated: false, root_cert_pem populated: false, Control: None, read_only: false session_id = 21da52e5-eac9-4881-9a24-ca141dc547e1
387112024-11-15T02:05:35.614ZINFOcrucible: Crucible stats registered with UUID: 16fe4894-c7df-4434-b37b-9fbbd5c72f4f session_id = 21da52e5-eac9-4881-9a24-ca141dc547e1
387122024-11-15T02:05:35.647ZINFOcrucible: Downstairs in volume = 3 task = crutest
387132024-11-15T02:05:35.647ZINFOcrucible: The guest has requested activation with gen:3
387142024-11-15T02:05:35.647ZINFOcrucible: 16fe4894-c7df-4434-b37b-9fbbd5c72f4f active request set session_id = 21da52e5-eac9-4881-9a24-ca141dc547e1
387152024-11-15T02:05:35.647ZINFOcrucible: sending connect oneshot to client = downstairs client = 0 session_id = 21da52e5-eac9-4881-9a24-ca141dc547e1
387162024-11-15T02:05:35.647ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 0 session_id = 21da52e5-eac9-4881-9a24-ca141dc547e1
387172024-11-15T02:05:35.647ZINFOcrucible: sending connect oneshot to client = downstairs client = 1 session_id = 21da52e5-eac9-4881-9a24-ca141dc547e1
387182024-11-15T02:05:35.647ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 1 session_id = 21da52e5-eac9-4881-9a24-ca141dc547e1
387192024-11-15T02:05:35.647ZINFOcrucible: sending connect oneshot to client = downstairs client = 2 session_id = 21da52e5-eac9-4881-9a24-ca141dc547e1
387202024-11-15T02:05:35.647ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 2 session_id = 21da52e5-eac9-4881-9a24-ca141dc547e1
387212024-11-15T02:05:35.647ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 21da52e5-eac9-4881-9a24-ca141dc547e1
387222024-11-15T02:05:35.647ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 21da52e5-eac9-4881-9a24-ca141dc547e1
387232024-11-15T02:05:35.647ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 21da52e5-eac9-4881-9a24-ca141dc547e1
387242024-11-15T02:05:35.647ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:43802) = io task client = 0 session_id = 21da52e5-eac9-4881-9a24-ca141dc547e1
387252024-11-15T02:05:35.647ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:54242) = io task client = 2 session_id = 21da52e5-eac9-4881-9a24-ca141dc547e1
387262024-11-15T02:05:35.647ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:59596) = io task client = 1 session_id = 21da52e5-eac9-4881-9a24-ca141dc547e1
387272024-11-15T02:05:35.648ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 0 session_id = 21da52e5-eac9-4881-9a24-ca141dc547e1
387282024-11-15T02:05:35.648ZINFOcrucible: [0] Transition from New to WaitActive = downstairs client = 0 session_id = 21da52e5-eac9-4881-9a24-ca141dc547e1
387292024-11-15T02:05:35.648ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 2 session_id = 21da52e5-eac9-4881-9a24-ca141dc547e1
387302024-11-15T02:05:35.648ZINFOcrucible: [2] Transition from New to WaitActive = downstairs client = 2 session_id = 21da52e5-eac9-4881-9a24-ca141dc547e1
387312024-11-15T02:05:35.648ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 1 session_id = 21da52e5-eac9-4881-9a24-ca141dc547e1
387322024-11-15T02:05:35.648ZINFOcrucible: [1] Transition from New to WaitActive = downstairs client = 1 session_id = 21da52e5-eac9-4881-9a24-ca141dc547e1
387332024-11-15T02:05:35.648ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 21da52e5-eac9-4881-9a24-ca141dc547e1
387342024-11-15T02:05:35.648ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 21da52e5-eac9-4881-9a24-ca141dc547e1
387352024-11-15T02:05:35.648ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 21da52e5-eac9-4881-9a24-ca141dc547e1
387362024-11-15T02:05:35.649ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 0 session_id = 21da52e5-eac9-4881-9a24-ca141dc547e1
387372024-11-15T02:05:35.649ZINFOcrucible: [0] Transition from WaitActive to WaitQuorum = downstairs client = 0 session_id = 21da52e5-eac9-4881-9a24-ca141dc547e1
387382024-11-15T02:05:35.649ZINFOcrucible: Waiting for more clients to be ready session_id = 21da52e5-eac9-4881-9a24-ca141dc547e1
387392024-11-15T02:05:35.649ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 1 session_id = 21da52e5-eac9-4881-9a24-ca141dc547e1
387402024-11-15T02:05:35.649ZINFOcrucible: [1] Transition from WaitActive to WaitQuorum = downstairs client = 1 session_id = 21da52e5-eac9-4881-9a24-ca141dc547e1
387412024-11-15T02:05:35.649ZINFOcrucible: Waiting for more clients to be ready session_id = 21da52e5-eac9-4881-9a24-ca141dc547e1
387422024-11-15T02:05:35.649ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 2 session_id = 21da52e5-eac9-4881-9a24-ca141dc547e1
387432024-11-15T02:05:35.649ZINFOcrucible: [2] Transition from WaitActive to WaitQuorum = downstairs client = 2 session_id = 21da52e5-eac9-4881-9a24-ca141dc547e1
387442024-11-15T02:05:35.649ZINFOcrucible: [0]R flush_numbers[0..12]: [945, 939, 941, 940, 940, 940, 945, 898, 893, 943, 943, 941] = downstairs session_id = 21da52e5-eac9-4881-9a24-ca141dc547e1
387452024-11-15T02:05:35.649ZINFOcrucible: [0]R generation[0..12]: [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2] = downstairs session_id = 21da52e5-eac9-4881-9a24-ca141dc547e1
387462024-11-15T02:05:35.649ZINFOcrucible: [0]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 21da52e5-eac9-4881-9a24-ca141dc547e1
387472024-11-15T02:05:35.649ZINFOcrucible: [1]R flush_numbers[0..12]: [945, 939, 941, 940, 940, 940, 945, 898, 893, 943, 943, 941] = downstairs session_id = 21da52e5-eac9-4881-9a24-ca141dc547e1
387482024-11-15T02:05:35.649ZINFOcrucible: [1]R generation[0..12]: [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2] = downstairs session_id = 21da52e5-eac9-4881-9a24-ca141dc547e1
387492024-11-15T02:05:35.649ZINFOcrucible: [1]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 21da52e5-eac9-4881-9a24-ca141dc547e1
387502024-11-15T02:05:35.649ZINFOcrucible: [2]R flush_numbers[0..12]: [945, 939, 941, 940, 940, 940, 945, 898, 893, 943, 943, 941] = downstairs session_id = 21da52e5-eac9-4881-9a24-ca141dc547e1
387512024-11-15T02:05:35.649ZINFOcrucible: [2]R generation[0..12]: [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2] = downstairs session_id = 21da52e5-eac9-4881-9a24-ca141dc547e1
387522024-11-15T02:05:35.649ZINFOcrucible: [2]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 21da52e5-eac9-4881-9a24-ca141dc547e1
387532024-11-15T02:05:35.649ZINFOcrucible: Max found gen is 3 = downstairs session_id = 21da52e5-eac9-4881-9a24-ca141dc547e1
387542024-11-15T02:05:35.649ZINFOcrucible: Generation requested: 3 >= found:3 = downstairs session_id = 21da52e5-eac9-4881-9a24-ca141dc547e1
387552024-11-15T02:05:35.649ZINFOcrucible: Next flush: 946 = downstairs session_id = 21da52e5-eac9-4881-9a24-ca141dc547e1
387562024-11-15T02:05:35.649ZINFOcrucible: All extents match = downstairs session_id = 21da52e5-eac9-4881-9a24-ca141dc547e1
38757 {"msg":"No downstairs reconciliation required","v":0,"name":"crucible","level":30,"Wait for a query_work_queue command to finish before sending IO
38758 time":"2024-11-15T02:05:35.649966165Z","hostname":"w-01JCPRHT5F2MPA44KVJHM3S2PB","pid":1241,"session_id":"21da52e5-eac9-4881-9a24-ca141dc547e1"}
387592024-11-15T02:05:35.649ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 0 session_id = 21da52e5-eac9-4881-9a24-ca141dc547e1
387602024-11-15T02:05:35.650ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 1 session_id = 21da52e5-eac9-4881-9a24-ca141dc547e1
387612024-11-15T02:05:35.650ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 2 session_id = 21da52e5-eac9-4881-9a24-ca141dc547e1
387622024-11-15T02:05:35.650ZINFOcrucible: All required reconciliation work is completed session_id = 21da52e5-eac9-4881-9a24-ca141dc547e1
387632024-11-15T02:05:35.650ZINFOcrucible: Set Downstairs and Upstairs active after reconciliation session_id = 21da52e5-eac9-4881-9a24-ca141dc547e1
387642024-11-15T02:05:35.650ZINFOcrucible: 16fe4894-c7df-4434-b37b-9fbbd5c72f4f is now active with session: 21da52e5-eac9-4881-9a24-ca141dc547e1 session_id = 21da52e5-eac9-4881-9a24-ca141dc547e1
387652024-11-15T02:05:35.650ZINFOcrucible: Set Active after no reconciliation session_id = 21da52e5-eac9-4881-9a24-ca141dc547e1
387662024-11-15T02:05:35.650ZINFOcrucible: The guest has finished waiting for activation with:3
38767 Region: sv:1 bs:4096 ts:20480000 tb:5000 max_io:256 or 1048576
38768 Loading write count information from file "/tmp/test_replay_verify.log"
38769 Read and Verify all blocks (0..5000 range:false)
38770 Wrote out file "/tmp/test_replay_verify.log"
38771 Verify test completed
38772 Wrote out file "/tmp/test_replay_verify.log"
38773 CLIENT: Tests done. All submitted work has been ACK'd
38774 ----------------------------------------------------------------
38775 Crucible gen:3 GIO:true work queues: Upstairs:0 downstairs:20
38776 States: Active Active Active
38777 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
38778 1000 Acked Read 256 Done Done Done false
38779 1001 Acked Read 256 Done Done Done false
38780 1002 Acked Read 256 Done Done Done false
38781 1003 Acked Read 256 Done Done Done false
38782 1004 Acked Read 256 Done Done Done false
38783 1005 Acked Read 256 Done Done Done false
38784 1006 Acked Read 256 Done Done Done false
38785 1007 Acked Read 256 Done Done Done false
38786 1008 Acked Read 256 Done Done Done false
38787 1009 Acked Read 256 Done Done Done false
38788 1010 Acked Read 256 Done Done Done false
38789 1011 Acked Read 256 Done Done Done false
38790 1012 Acked Read 256 Done Done Done false
38791 1013 Acked Read 256 Done Done Done false
38792 1014 Acked Read 256 Done Done Done false
38793 1015 Acked Read 256 Done Done Done false
38794 1016 Acked Read 256 Done Done Done false
38795 1017 Acked Read 256 Done Done Done false
38796 1018 Acked Read 136 Done Done Done false
38797 1019 Acked Read 256 Done Done Done false
38798 STATES DS:0 DS:1 DS:2 TOTAL
38799 Sent 0 0 0 0
38800 Done 20 20 20 60
38801 Skipped 0 0 0 0
38802 Error 0 0 0 0
38803 Last Flush: 0 0 0
38804 Downstairs last five completed:
38805 Upstairs last five completed: 1019 1018 1017 1016 1015
38806 CLIENT: Up:0 ds:20 act:3
38807 ----------------------------------------------------------------
38808 Crucible gen:3 GIO:true work queues: Upstairs:0 downstairs:0
38809 Downstairs last five completed: 1020 1019 1018 1017 1016
38810 Upstairs last five completed: 1020 1019 1018 1017 1016
38811 CLIENT: Up:0 ds:0 act:3
38812 CLIENT: All crucible jobs finished, exiting program
388132024-11-15T02:05:40.061ZWARNcrucible: Guest handle has been dropped
38814 Nov 15 02:05:40.126 INFO accepted connection, remote_addr: 127.0.0.1:35712, local_addr: 127.0.0.1:9998
38815 Main task has work to do, go find it
38816 got dsc Shutdown
38817 Shutdown
38818 Nov 15 02:05:40.127 INFO request completed, latency_us: 165, response_code: 204, uri:[8810] Disable keep_running
38819 /shutdown, methodShut it down
38820 : POST, req_id: 7bbbba13-746d-47a0-aa55-ace3d7345c73, remote_addr: 127.0.0.1:35712, local_addr: 127.0.0.1:9998
38821 [8820] Disable keep_running
38822 [8820] Got stop action so:false kr:false
38823 [8830] Disable keep_running
38824 [8830] Got stop action so:false kr:false
38825 [8810] Got stop action so:false kr:false
38826 [8820] Exited with: ExitStatus(unix_wait_status(9))
38827 [8820][1] reports Exit
38828 [8830] Exited with: ExitStatus(unix_wait_status(9))
38829 [8810] Exited with: ExitStatus(unix_wait_status(9))
38830 [8830][2] reports Exit
38831 [8810][0] reports Exit
38832 November 15, 2024 at 02:05:45 AM UTC Test ends with 0